A Method and System for Defect Localization in Low-Voltage Electrical Appliances Based on Dynamic Weighted Feature Fusion

Abstract

This application relates to the field of industrial vision inspection technology, and discloses a method and system for localizing defects in low-voltage electrical appliances based on dynamic weighted feature fusion. The method includes: acquiring an image to be inspected and extracting multi-scale feature maps; constructing a state vector based on the feature maps and an uncertainty map generated by a reconnaissance network; inputting the state vector into a recurrent policy agent to generate a composite action containing dynamic fusion weights and processing path selection; dynamically weighting and fusing the multi-scale feature maps according to the composite action, and calling the detection head corresponding to the selected path for decoding to obtain the defect localization result. The system includes: a feature extraction module, a state construction module, a recurrent policy agent, and a policy-driven execution module. This invention, by introducing a recurrent policy agent to adaptively control the feature fusion and decoding process through reinforcement learning, can improve localization accuracy and robustness while considering decision-making costs, thus enhancing the level of intelligence for diverse defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial visual inspection, in particular to a low-voltage electrical appliance defect positioning method and system based on dynamic weighted feature fusion. BACKGROUND

[0002] Low-voltage electrical appliances are basic components in power systems, and their quality directly affects the safe and stable operation of the entire system. Therefore, efficient and accurate detection of various defects produced during the production process of low-voltage electrical appliances is a key link to ensure product quality. Traditional defect detection mainly relies on manual visual inspection, which not only has high labor intensity and low efficiency, but also is easily affected by subjective factors, making it difficult to meet the needs of modern large-scale production.

[0003] With the development of computer vision technology, automated defect detection methods based on deep learning have become the mainstream technology in this field. These methods usually use convolutional neural networks (CNN) to extract multi-scale features from electrical appliance images, and combine detailed information and semantic information at different levels through feature fusion techniques to improve the recognition ability of defects of different sizes and shapes.

[0004] However, existing deep learning detection methods still have certain limitations in practical applications. Most of these methods use static, pre-designed feature fusion structures, such as fixed element-wise addition or channel concatenation. This "one-size-fits-all" fusion method cannot adaptively adjust the contribution of different scale features according to the specific content of the input image and the unique characteristics of the defects. For a variety of low-voltage electrical appliance defects, the accurate positioning of some defects may rely more on high-resolution detailed features, while others may need more high-level semantic information to assist in judgment. The fixed fusion strategy limits the flexibility of feature expression, which may lead to poor detection performance for certain types of defects.

[0005] In addition, these methods usually use a single, fixed decoding path to process the fused features and generate the final positioning results. This fixed processing flow lacks strategy and cannot call the optimal decoding strategy for different defect patterns, further limiting the detection accuracy and robustness of the model when facing complex and varied defect types.

[0006] Therefore, the present application proposes a low-voltage electrical appliance defect positioning method and system based on dynamic weighted feature fusion to solve the deficiencies of the prior art. SUMMARY

[0007] To address the shortcomings of existing technologies, this invention provides a method and system for locating defects in low-voltage electrical appliances based on dynamic weighted feature fusion. This solves the problem that industrial defect location methods typically employ fixed network structures and processing flows, which are insufficiently adaptable when faced with defects of diverse shapes and sizes, and struggle to achieve an effective balance between location accuracy and computational resource consumption.

[0008] To address the aforementioned technical problems, this invention provides a defect localization technology that can adaptively adjust the feature fusion strategy and processing path based on the content of the input image.

[0009] The first aspect of this invention provides a method for locating defects in low-voltage electrical appliances based on dynamic weighted feature fusion, the method comprising:

[0010] Acquire an image of the low-voltage electrical appliance to be detected, and extract a multi-scale feature map set from the low-voltage electrical appliance image;

[0011] A state vector is constructed based on the extracted multi-scale feature map set;

[0012] The state vector is input into the recurrent policy agent to generate a composite action that includes dynamically fused weights and the selected processing path;

[0013] Based on the dynamic fusion weights in the composite action, the multi-scale feature map set is weighted and fused to generate a fused feature map;

[0014] Based on the selected processing path in the composite action, the generated fused feature map is processed to finally obtain the defect location result of the low-voltage electrical appliance.

[0015] Preferably, the step of constructing a state vector based on the extracted multi-scale feature map set specifically includes:

[0016] The high-order semantic features in the multi-scale feature map set are input into the reconnaissance network to generate a preliminary defect probability prediction map.

[0017] Information entropy is calculated based on the defect probability prediction map described in the preliminary step to generate an uncertainty map that quantifies the degree of uncertainty in the model's perception of each image region;

[0018] The uncertainty map and the multi-scale feature map set are concatenated along the channel dimension, and the information is compressed through an encoder network to generate a state vector.

[0019] Preferably, the recurrent strategy agent integrates a recurrent neural network unit and outputs the composite action based on the state vector and the hidden state of the previous time step.

[0020] Preferably, the process of inputting the state vector into the recurrent policy agent to generate a dynamic fusion weight includes:

[0021] Input the hidden state of the current time step into the fully connected layer;

[0022] The Softmax function is applied to the output of the fully connected layer to generate the dynamic fusion weights.

[0023] Preferably, the process of inputting the state vector into the recurrent policy agent to generate the composite action of the selected processing path includes:

[0024] The hidden state at the current time step is input into another fully connected layer to output a score vector;

[0025] The selected processing path is determined based on the probability distribution of the fractional vector transformation.

[0026] Preferably, the step of processing the generated fused feature map according to the selected processing path in the composite action specifically includes:

[0027] The system has a pre-defined processing path library containing multiple parallel detection heads optimized for different defect modes;

[0028] The corresponding detection head is indexed from the processing path library according to the selected processing path;

[0029] The fused feature map is input into the indexed detection head to generate defect location results.

[0030] Preferably, the multiple detection heads in the processing path library include:

[0031] A structural relationship decoder, the network structure of which is configured to enhance global context awareness of features to decode structural defects characterizing spatial relationships between low-voltage electrical components;

[0032] A surface anomaly decoder, the network structure of which is configured to maintain sensitivity to high-resolution detail features to decode surface defects characterizing physical damage to the surface of low-voltage electrical appliances.

[0033] Preferably, the step of weighted fusion of the multi-scale feature map set specifically includes:

[0034] Each feature map in the multi-scale feature map set is adjusted to a uniform spatial resolution through an upsampling operation;

[0035] Each resolution-aligned feature map is multiplied pixel-by-pixel with its corresponding dynamic fusion weight, and then summed pixel-by-pixel to generate the fused feature map.

[0036] Preferably, the method further includes a step of training a recurrent policy agent, wherein the training optimizes the agent's parameters by maximizing a final reward signal, wherein the final reward signal is obtained by subtracting the decision cost incurred by the agent's decision from the accuracy reward of the localization result, and the decision cost specifically includes:

[0037] The fusion weight decision cost associated with the dynamic fusion weights, which is determined by calculating the information entropy of the dynamic fusion weight vector to penalize uncertain weight allocations; and the preset computational overhead associated with the selected processing path.

[0038] A second aspect of the present invention provides a low-voltage electrical appliance defect location system based on dynamic weighted feature fusion, the system comprising:

[0039] The feature extraction module is used to acquire images of low-voltage electrical appliances to be detected and extract a set of multi-scale feature maps from the images of low-voltage electrical appliances.

[0040] The state construction module is used to construct a state vector based on the multi-scale feature map set extracted by the feature extraction module;

[0041] A recurrent policy agent is used to receive the state vector and output a composite action that includes dynamically fused weights and the selected processing path.

[0042] The strategy-driven execution module is used to perform weighted fusion of the multi-scale feature map set to generate a fused feature map based on the composite action output by the recurrent strategy agent, and to process the generated fused feature map to finally obtain the defect location result of the low-voltage electrical appliance.

[0043] This invention provides a method and system for locating defects in low-voltage electrical appliances based on dynamic weighted feature fusion. It has the following beneficial effects:

[0044] 1. This invention introduces a cyclic strategy intelligent agent, which can generate dynamic fusion weights in real time based on the input low-voltage electrical appliance image features. This method overcomes the limitations of traditional fixed-weight fusion, enabling the system to adaptively adjust the emphasis on different levels of features in the multi-scale feature map set for different defect morphologies, thereby improving the detection accuracy and robustness for diverse and atypical defects.

[0045] 2. In constructing the state vector, this invention introduces an uncertainty graph that represents the degree of cognitive uncertainty in the model. This uncertainty graph provides the recurrent policy agent with important prior information about where in the image is difficult to determine, enabling it to focus more on blurred and difficult areas in the image when formulating fusion weights and selecting processing paths. This achieves intelligent and efficient decision-making, ultimately improving the reliability of the overall localization results.

[0046] 3. This invention establishes a processing path library containing multiple parallel detection heads, each optimized for a specific defect pattern. The cyclic strategy agent can select the most suitable processing path from the library for subsequent decoding based on its understanding of the image content. This targeted processing approach, compared to a single general decoding structure, enables more refined analysis of specific complex defect patterns, thereby improving the accuracy of defect localization.

[0047] 4. The training process of this invention adopts a method of maximizing the final reward signal. This signal not only includes the accuracy reward for the defect localization result, but also introduces the decision cost related to the selected processing path and fusion weights. By balancing accuracy and cost in the optimization objective, this method can drive the recurrent policy agent to learn a decision strategy that balances high accuracy and low resource consumption, thereby effectively controlling computational overhead while ensuring detection performance and improving the practicality of the method. Attached Figure Description

[0048] Figure 1 This is a flowchart of the low-voltage electrical appliance defect localization method based on dynamic weighted feature fusion of the present invention;

[0049] Figure 2 This is a block diagram of the low-voltage electrical appliance defect location system based on dynamic weighted feature fusion according to the present invention.

[0050] Figure 3 This is a schematic diagram of the state vector construction process of the present invention;

[0051] Figure 4 This is a schematic diagram of the training mechanism of the loop strategy agent of the present invention.

[0052] Among them, 10 is the feature extraction module; 20 is the state construction module; 30 is the recurrent policy agent; and 40 is the policy-driven execution module. Detailed Implementation

[0053] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Reference Figure 1 , Figure 1 This is a flowchart of a low-voltage electrical appliance defect localization method based on dynamic weighted feature fusion according to an embodiment of the present invention. The present invention provides a low-voltage electrical appliance defect localization method based on dynamic weighted feature fusion, which may include the following steps:

[0055] S1. Obtain an image of the low-voltage electrical appliance to be detected, and extract a set of multi-scale feature maps from the low-voltage electrical appliance image.

[0056] S2. Construct a state vector based on the extracted multi-scale feature map set.

[0057] S3. Input the state vector into the recurrent policy agent to generate a composite action that includes dynamic fusion weights and the selected processing path.

[0058] S4. Based on the dynamic fusion weights in the composite action, the multi-scale feature map set is weighted and fused to generate a fused feature map.

[0059] S5. Based on the processing path selected in the composite action, the generated fused feature map is processed to finally obtain the defect location result of the low-voltage electrical appliance.

[0060] In a specific embodiment of the present invention, the above process is further described.

[0061] In step S1, the system inputs an image of a low-voltage electrical appliance to be detected into a convolutional neural network and outputs feature maps from multiple different depth levels of the network to form a multi-scale feature map set. These feature maps have different characteristics: shallow feature maps retain high spatial resolution details, while deep feature maps contain high-level semantic information.

[0062] In step S2, in order to construct a state vector that can comprehensively reflect the current image detection state, the system first uses high-order semantic features in the multi-scale feature map set to generate an uncertainty map. This process obtains a preliminary defect probability prediction through a reconnaissance network and calculates the information entropy based on the defect probability prediction.

[0063] In step S3, the state vector is input to the recurrent policy agent; the recurrent policy agent integrates a recurrent neural network unit to process temporal information and outputs a composite action based on the current state vector and the hidden state of the previous time step. The composite action explicitly includes a set of dynamically fused weights for guiding subsequent steps and a selected processing path.

[0064] In step S4, the system performs weighted fusion of the multi-scale feature map set according to the dynamic fusion weights in the composite action. To achieve this operation, all feature maps are first adjusted to a uniform spatial resolution through upsampling. Then, these resolution-aligned feature maps are weighted and summed to generate a fused feature map that incorporates multi-scale information. .

[0065] In step S5, the system processes the fused feature map generated in the previous step according to the selected processing path in the composite action. Decoding is performed; the system has a pre-set processing path library, which contains multiple detection heads optimized for different defect patterns; the system indexes the corresponding detection head according to the selected processing path and inputs the fused feature map into it to generate the final defect localization result.

[0066] Furthermore, the method of the present invention also includes a step of training a recurrent policy agent, which optimizes the agent's parameters by maximizing the final reward signal; the final reward signal takes into account both the accuracy reward of the localization result and the decision cost generated by the agent's decision.

[0067] Reference Figure 2 , Figure 2 This is a structural block diagram of a low-voltage electrical appliance defect location system based on dynamic weighted feature fusion according to an embodiment of the present invention. The system may include:

[0068] The feature extraction module 10 is used to perform step S1, acquire the image of the low-voltage electrical appliance to be detected, and extract a multi-scale feature map set from the low-voltage electrical appliance image.

[0069] The state construction module 20, whose input is connected to the output of the feature extraction module 10, is used to execute step S2, which constructs a state vector based on the multi-scale feature map set extracted by the feature extraction module 10.

[0070] The recurrent policy agent 30, whose input is connected to the output of the state construction module 20, is used to execute step S3. It is configured to receive the state vector generated by the state construction module 20 and output a composite action containing dynamic fusion weights and the selected processing path.

[0071] The strategy-driven execution module 40 has its input terminals connected to the output terminals of the feature extraction module 10 and the cyclic strategy agent 30, respectively. It is used to execute steps S4 and S5. Based on the composite action output by the cyclic strategy agent 30, it performs weighted fusion on the multi-scale feature map set to generate a fused feature map. The generated fused feature map is then processed to finally obtain the defect location result of the low-voltage electrical appliance.

[0072] The technical solutions in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0073] Reference Figure 1 and Figure 2 Step S1 is performed by the feature extraction module 10, which extracts a set of multi-scale feature maps containing rich information from the input low-voltage electrical appliance image. In a specific embodiment, the feature extraction module 10 is implemented as a convolutional neural network (CNN) backbone network.

[0074] The convolutional neural network backbone can be a network structure that has proven its effectiveness in image recognition tasks, such as, but not limited to, the ResNet series (e.g., ResNet-50, ResNet-101) or the EfficientNet series. In one implementation, the backbone is initialized with weights pre-trained on a large public dataset (e.g., ImageNet), thereby giving it an initial ability to extract general visual features. Then, through fine-tuning specific to a low-voltage electrical appliance defect dataset, it is made more focused on the feature representation in this field.

[0075] When an image of a low-voltage electrical appliance to be detected is input into the feature extraction module 10, the image of the low-voltage electrical appliance passes through multiple layers of a convolutional neural network in sequence. The structural characteristics of the convolutional neural network determine that it can learn feature representations at different depths. Therefore, the present invention selects multiple output nodes from different depths of the backbone network to obtain a set of feature maps, which constitute a multi-scale feature map set.

[0076] Specifically, feature maps extracted from shallower layers of the backbone network (such as the output of the res2 layer in a ResNet network) undergo fewer downsampling steps and therefore have higher spatial resolution. These feature maps preserve rich spatial details, edge contours, and texture information in the image, which is crucial for locating defects with small size or clear boundaries.

[0077] Meanwhile, feature maps extracted from deeper layers of the backbone network (such as the output of res4 or res5 layers in ResNet) undergo multiple downsampling operations, resulting in a significant reduction in spatial resolution. However, through multi-layer nonlinear transformations, these feature maps have a larger receptive field and contain higher-level, more abstract semantic information, such as expressions of concepts about object parts and material properties. This is crucial for understanding large-area damage or defects in complex scenes.

[0078] Finally, these feature maps extracted from different depth levels (e.g., denoted as...) Together, they form a multi-scale feature map set; this multi-scale feature map set, as a whole, contains both low-level detailed information and high-level semantic information of the low-voltage electrical appliance image, providing comprehensive and necessary input data for subsequent state construction, dynamic fusion and defect localization steps.

[0079] Reference Figure 1 , Figure 2 and Figure 3 Step S2 is executed by the state construction module 20. The function of the state construction module 20 is to construct a fixed-length state vector that can comprehensively and accurately describe the current state of the image to be detected based on the multi-scale feature map set output by the feature extraction module 10. This state vector will serve as the direct basis for the subsequent decision-making of the recurrent policy agent 30.

[0080] In one specific embodiment, the construction process of the state vector includes two main stages: the generation of the uncertainty graph and the integration and encoding of multi-source information.

[0081] First, an uncertainty map is generated; the purpose of this stage is to quantify the degree of cognitive uncertainty of the model regarding the presence of defects in various regions of the image; this process selects the feature map with the highest level of semantic information from the multi-scale feature map set (e.g., denoted as...). This high-order semantic feature map is used as input; The input is fed into a separate reconnaissance network; the reconnaissance network is a lightweight convolutional neural network whose structure is designed for fast processing of high-order features to output a preliminary defect probability prediction map, denoted as . The dimension of the defect probability prediction map is the same as that of the input feature map. Corresponding to the spatial dimension, the value of each pixel in the image A value between 0 and 1 indicates the initial probability that a defect exists at that location.

[0082] Obtain preliminary defect probability prediction map Then, the system calculates the information entropy based on this graph to generate the final uncertainty graph, denoted as . Specifically, for The information entropy of each pixel in the image. Calculated using the following formula:

[0083] ;

[0084] In the formula, This is the pixel in the preliminary defect probability prediction map. The probability value in; It is the calculated information entropy value of that pixel, that is, its value in the uncertainty map. The value in; according to the characteristics of information entropy, when the probability When it approaches 0.5, the information entropy The highest value indicates that the model's judgment on this position is most uncertain; when the probability... Information entropy is close to 0 or 1. A value close to 0 indicates that the model has high certainty in its judgment of this position.

[0085] Next, the multi-source information is integrated and encoded; this stage involves generating the uncertainty graph from the previous steps. With the original multi-scale feature map set The features are combined to form a comprehensive representation that includes global visual information and uncertain prior information. In one implementation, all feature maps and uncertainty maps in the multi-scale feature map set are first adjusted to a preset uniform spatial resolution through operations such as bilinear interpolation or adaptive pooling.

[0086] After spatial resolution alignment, these adjusted feature maps and uncertainty maps are concatenated along the channel dimension to form a thicker fused feature block. Finally, this fused feature block is input into an encoder network, which can consist of several convolutional layers and a global average pooling layer. The encoder network performs dimensionality reduction and information compression on the fused feature block, ultimately outputting a one-dimensional, fixed-length vector, which is the state vector at the current time step. The state vector It comprehensively encodes all the information needed for subsequent decision-making.

[0087] Reference Figure 1 , Figure 2 and Figure 4 Step S3 is executed by the recurrent policy agent 30; the core function of the recurrent policy agent 30 is to receive the state vector generated by the state construction module 20. Based on this state, it autonomously generates composite actions. This composite action will be used to guide the subsequent feature fusion and defect localization process.

[0088] In one specific embodiment, the internal structure of the recurrent policy agent 30 is based on recurrent neural network (RNN) units, such as Long Short-Term Memory (LSTM) units or gated recurrent units (GRUs). This structure enables the agent to maintain an internal hidden state. This hidden state is passed between different time steps, thereby encoding information about the historical observation sequence.

[0089] at any time step The recurrent policy agent 30 receives two inputs: the current state vector provided by the state construction module 20. And from the previous time step Hidden state transmitted The RNN unit processes these two inputs, updates its internal state, and outputs the hidden state at the current time step. This update process makes It not only includes the current image state The understanding also incorporates the context of historical information.

[0090] Based on the updated current hidden state The recurrent strategy agent 30 generates compound actions. This compound action It consists of two parts: a set of dynamic fusion weights and a selected processing path .

[0091] Dynamic fusion weights The generation process is as follows: First, the current hidden state vector is... The input is fed into a fully connected layer, which performs a linear transformation to... Mapping to one dimension and the number of multi-scale feature maps Equal vectors are then normalized using the Softmax function to generate the final dynamic fusion weight vector. This process can be represented by the following formula:

[0092] ;

[0093] In the formula, The elements of the generated dynamic fusion weight vector For the corresponding number The weights of each feature map, and the sum of all elements is 1; This is the hidden state vector at the current time step; This is the weight matrix of the fully connected layer used to generate the weights; This is the bias vector used to generate the weights of the fully connected layer; This is the Softmax function, used to convert an input vector into a probability distribution.

[0094] Selected processing path The generation process and the weight generation process are performed in parallel; the current hidden state vector is... The input is fed into another independent fully connected layer, which outputs a score vector, where each dimension of the score vector corresponds to an optional detection head in the processing path library. Similarly, this score vector is converted into a probability distribution using the Softmax function, and the recurrent policy agent 30 samples according to this probability distribution to determine the final selected processing path. .

[0095] Ultimately, the dynamic fusion weights and the selected processing path The combined action The output is from the loop strategy agent 30 and passed to the strategy-driven execution module 40 for precise control of subsequent fusion and positioning operations.

[0096] Reference Figure 1 and Figure 2 Step S4 is implemented by a portion of the functionality of the policy-driven execution module 40. The goal of this step is to execute the compound action output by the recurrent policy agent 30. Dynamic fusion weights The multi-scale feature map set provided by feature extraction module 10 Weighted fusion is performed to generate a single, information-rich fused feature map.

[0097] The fusion process first requires spatial resolution alignment of the feature maps in the multi-scale feature map set. Since these feature maps are extracted from different depths of convolutional neural networks, their spatial dimensions (i.e., height and width) differ. To perform effective pixel-wise weighted summation, they must be unified to the same spatial resolution. In one specific implementation, the system selects a target resolution, which can be set to the highest resolution in the original multi-scale feature map set. Subsequently, an upsampling operation is performed to enlarge the size of all feature maps with spatial resolutions lower than the target resolution to that target resolution. The upsampling operation can be implemented using bilinear interpolation, thus preserving the spatial continuity of the features during the enlargement process. After this operation, a set of spatially resolution-aligned feature maps is obtained. .

[0098] In obtaining a set of feature maps aligned to spatial resolution and the dynamic fusion weight vector generated by the recurrent policy agent 30 Next, the system performs a weighted summation operation. This weighted summation operation combines each aligned feature map... Its corresponding dynamic fusion weight Perform pixel-by-pixel multiplication, then sum all the weighted feature maps pixel-by-pixel to finally generate a fused feature map. The dynamic weighted fusion process can be represented by the following formula:

[0099] ;

[0100] In the formula, This is the final generated fused feature map; This represents the total number of feature maps in the multi-scale feature map set. For the index of the feature map; For the recurrent policy agent 30, applied to the first Dynamic fusion weights for each feature map; For the first Original feature map The feature map is obtained after upsampling, where its spatial resolution is aligned with the target resolution.

[0101] The generated fusion feature map is obtained through this dynamic weighted fusion process. Based on the characteristics of the current image, effective information from feature maps of different scales is adaptively integrated and passed to the subsequent processing unit of the policy-driven execution module 40 for final defect localization.

[0102] Reference Figure 1 and Figure 2 Step S5 is implemented by a portion of the functionality of the policy-driven execution module 40; this step utilizes the composite actions generated by the recurrent policy agent 30. The selected processing path The fusion feature map generated in step S4 Perform the final decoding to obtain the defect location results.

[0103] In one specific embodiment, a processing path library is pre-configured within the system. This processing path library contains multiple parallel, structurally independent detection heads. Each detection head is a decoder network whose network structure, layers, and parameters are designed and optimized to specifically handle a particular category of defect patterns. For example, one detection head may be optimized to identify large-area, diffusely shaped defects, while another detection head may be optimized to accurately locate minute, sharp-edged scratch-like defects. These detection heads together constitute a set of selectable, specialized processing paths.

[0104] When the policy-driven execution module 40 receives the selected processing path output by the recurrent policy agent 30 At that time, this processing path As a well-defined identifier, it is used to index and invoke the exact corresponding detection header from the processing path library; for example, if the processing path library contains Each detection head is denoted as... The selected processing path for The system will then select and enable the first one. Each detection head .

[0105] After selecting the detection head, the system will use the fused feature map generated in the previous step. As input, it is passed to the selected detection head. The detection head performs a series of decoding operations on its input, which typically include several upsampling and convolution operations, aiming to progressively recover the spatial resolution of the feature map and decode it into pixel-level predictions.

[0106] Finally, the selected detection head outputs a defect location map corresponding to the spatial size of the original input image; the value of each pixel in the defect location map represents the probability or category of the presence of a defect at the corresponding location in the original image, thus completing the final location of the low-voltage electrical appliance defect; through this strategy-driven path selection mechanism, the present invention can call the decoder most suitable for its features for processing different input images, thereby achieving the adaptation and refinement of the localization process.

[0107] Reference Figure 4 , Figure 4 This is a schematic diagram of a training mechanism for a recurrent policy agent according to an embodiment of the present invention. This section describes in detail the training method for parameter optimization of the recurrent policy agent 30. This training process is fundamental to ensuring that the agent can learn and execute effective decision-making strategies.

[0108] In one specific embodiment, the training process is based on a reinforcement learning framework; in this reinforcement learning framework, the recurrent policy agent 30 acts as an agent at each time step. Observe the state provided by the state construction module 20 Perform a compound action And after this compound action leads to the final defect localization result, a scalar final reward signal is obtained from the environment. The goal of training is to maximize the expected cumulative reward obtained by the recurrent policy agent 30 across the entire training dataset by adjusting the internal network parameters of its RNN units and fully connected layers (e.g., the weights and biases of its RNN units and fully connected layers).

[0109] Final reward signal The design of this signal is crucial, as it directly guides the agent's learning direction; the final reward signal consists of two parts: an accuracy reward used to measure the accuracy of the localization result. And decision costs used to quantify the expense of the decision itself. .

[0110] Accuracy Bonus The calculation is based on the final defect location result output by the system. Compared to manually pre-labeled ground-truth tags. The degree of matching between them; in one implementation, this degree of matching can be quantified by calculating the Dice-Coefficient between the two. The calculation formula is as follows:

[0111] ;

[0112] In the formula, As an accuracy bonus, its value range is [0,1], and the higher the value, the more accurate the positioning result; The defect location result image generated by the system is usually a binary image; The corresponding baseline label image is also a binary image; express and The number of pixels in the intersection of the two is the number of pixels that are correctly detected as defects. express The total number of defective pixels in the image; express The total number of defective pixels.

[0113] Decision costs It consists of a weighted sum of two costs: the cost of the selected processing path and the cost of the chosen processing path. The associated computational overhead, and the dynamic fusion weights The associated decision uncertainty costs. First, there are the costs associated with weighted decision-making. By calculating the weight vector The information entropy is used to determine the weight allocation, which is used to penalize uncertain, tending-to-uniform distributions; its calculation formula is as follows:

[0114] ;

[0115] In the formula, To integrate weighted decision-making costs; The number of multi-scale feature maps; For application to the Dynamic fusion weights for each feature map.

[0116] Subsequently, this fusion weighted decision cost Preset computational overhead of the selected processing path Combining these, we obtain the total decision cost. .

[0117] Finally, the final reward signal By rewarding accuracy Subtract total decision-making cost This calculation method clearly establishes an optimization orientation that requires controlling decision-making costs while pursuing high accuracy. The calculation formula is as follows:

[0118] ;

[0119] In the formula, This is the final reward signal used to train the recurrent policy agent 30; A reward is given based on the accuracy calculated from the location results; This is the total decision cost calculated based on the compound actions.

[0120] In each iteration of training, the system uses algorithms such as policy gradient to calculate the final reward signal. This updates the network parameters of the recurrent policy agent 30, making it more inclined to produce compound actions that yield higher final rewards.

[0121] To further illustrate the technical solution and implementation of the present invention, a specific embodiment will be described in detail below.

[0122] The technical solution uses a low-voltage electrical appliance (voltage sensor) that plays a key role in power and automation control systems as the specific application object and explanatory carrier. As the core monitoring element, the main function of this device is to collect voltage signals in the circuit in real time, providing key data for the stable operation and safety protection of the system. Therefore, the assembly accuracy and integrity of the line voltage sensor directly determine its measurement accuracy and long-term reliability. Any structural or surface defects may lead to signal distortion, equipment failure, or even safety accidents. Therefore, it is essential to conduct rigorous and efficient automated quality testing on it.

[0123] In the automated production of online pressure sensor components, various factors such as process, materials, and environment can lead to various types of defects. These defects can be broadly categorized into two types. The first type is structural defects, which encompass all functional problems caused by incorrect component assembly. Specifically, the characteristics of this type of defect include, but are not limited to: mis-installed components, where an electronic component is installed but its position, orientation, or model does not conform to design requirements; and missing components, where necessary components are omitted during the production process, resulting in their complete absence in their proper assembly positions. The second type is surface defects, which encompass physical damage to the device body or its surface caused by external forces, contamination, etc. Specifically, the characteristics of this type of defect include, but are not limited to: device breakage, manifested as cracks, chipped corners, or fractures in the device casing or internal components; in addition, it also includes common imperfections such as surface scratches and oil stains.

[0124] Accurate and robust automated localization of the aforementioned diverse defects presents new technical challenges. These challenges mainly manifest in the following aspects: the high diversity of defect morphologies, such as the visually distinct differences between linear cracks, blocky missing areas, and irregularly shaped stains; the vast differences in defect scale, ranging from minute surface scratches to large-area component missingness, requiring detection algorithms to possess multi-scale perception capabilities, which necessitates the extraction and utilization of multi-scale feature map sets; and the low contrast between defects and normal backgrounds, where the visual features of some scratches or micro-cracks may be very weak and easily lost in complex background textures. The proposed technical solution addresses these challenges by dynamically weighting and fusing multi-scale features and adaptively selecting the optimal processing path to achieve accurate localization of low-voltage electrical defects in complex industrial scenarios.

[0125] This solution employs a specific industrial vision hardware system to acquire high-quality images of linear voltage sensor devices. The core of this system is an industrial CMOS camera, specifically a Basler-acA2500-14gm model, whose built-in Onsemi-MT9P031 sensor provides high-resolution images of 2592x1942 pixels. The camera is equipped with a Computar-M2514-MP2 lens, featuring a 25mm fixed focal length and a large F1.4 aperture, ensuring image clarity and light intake. In the acquisition setup, the camera is vertically fixed directly above the linear voltage sensor device to be inspected. To achieve uniform and shadow-free illumination, a ring light source is installed around the camera lens, emitting light that illuminates the device surface perpendicularly, maximizing the highlighting of surface texture details and contour features, and avoiding interference from shadows caused by side lighting in defect assessment. The entire acquisition process is controlled by an external computer; the captured images are directly acquired and stored for subsequent processing.

[0126] Based on the aforementioned image acquisition system, a defect dataset for model training and validation was constructed. This dataset contains 6000 acquired images of line voltage sensor devices. Before training, all images were preprocessed, and their size was uniformly adjusted to 1024x1024 pixels to meet the input requirements of the neural network model. To achieve supervised training of the model, each image containing defects in the dataset underwent meticulous manual annotation. This annotation process employed pixel-level segmentation, using professional image annotation software, where technicians precisely delineated the complete outline of each defect (such as misassembly, omission, or damage areas), generating a binary mask image with the same size as the original image. In this type of mask image, the pixel value of the defect area is 1, and the pixel value of the background area is 0. This dataset and its corresponding pixel-level labels together provide accurate input data and supervision signals for subsequent model training.

[0127] In this embodiment, the abstract functional modules involved in the aforementioned methods and systems are all configured with specific models and parameters; the feature extraction module 10 is specifically implemented as a convolutional neural network backbone network based on the ResNet-50 architecture; this network is pre-trained on the large public dataset ImageNet, and on this basis, it is fine-tuned using the voltage sensor device defect dataset constructed in this embodiment to make its extracted features more targeted; the processing path library is a key component of the policy-driven execution module 40, and its functionality is specifically implemented in this embodiment by configuring two parallel, expert detection heads (decoder networks); the first detection head is designed as a structural relationship decoder, whose network structure focuses on understanding and reconstructing the spatial relationships and global context of features, and is specifically used to handle structural defects such as component misassembly and component omission; the second detection head is designed as a surface anomaly decoder, whose network structure is more sensitive to detailed features such as local texture, edges and color abrupt changes, and is optimized for accurately locating surface defects such as device damage and surface scratches.

[0128] When training the recurrent policy agent 30 and the entire system, the Adam optimizer was used for parameter updates; the initial learning rate was set to 1e-4, and the learning rate was dynamically adjusted using a cosine-annealing strategy; the batch size during model training was set to 8 based on the hardware memory limit; the entire training process was iterated on the dataset for 100 epochs to ensure that the model parameters were fully converged.

[0129] To objectively evaluate the performance of the technical solution of this invention, a comparative experiment was conducted in a unified hardware and software environment. The hardware platform used in the experiment included an Intel Core i9-12900K processor, 64GB of memory, and an NVIDIA RTX 4090 graphics processing unit (GPU) for accelerating neural network calculations. The software environment was based on the Ubuntu 20.04 operating system and used PyTorch 1.12 as the deep learning framework.

[0130] To quantify the accuracy of defect localization results, this embodiment uses the Mean Intersection-over-Union (mIoU), a widely recognized metric in image segmentation, as the core evaluation indicator. This metric measures the degree of overlap between the predicted defect region and the actual defect region (i.e., the manually labeled baseline label) by calculating the ratio of their intersection to their union. The closer the mIoU value is to 100%, the more accurate the localization result. The calculation formula is as follows:

[0131] ;

[0132] In the formula, The final calculated average crossover-union ratio is a metric for measuring the overall performance of the model. The total number of defect categories; Indicates category True positives, meaning those correctly predicted as class A The number of pixels; Indicates category False positives, i.e., those incorrectly predicted as category A The number of pixels; Indicates category False negatives, meaning those that actually belong to the category However, the number of pixels that could not be successfully predicted.

[0133] In the experiment, the performance of the method of this invention was compared with two well-known advanced benchmark methods in the field of image segmentation, namely U-Net network and DeepLabv3+ network. In addition, to verify the effectiveness of the core content of this invention, an ablation version of the method of this invention was also set up for comparison, which is referred to as "the method of this invention (fixed strategy)". In this ablation version, the dynamic decision-making function executed by the recurrent policy agent 30 is disabled, and instead a set of fixed, preset fusion weights are used. The system always calls a general detection head for decoding, that is, it does not perform processing path selection. By comparing with this ablation version, the real performance gain brought about by the combination mechanism of dynamic weighted fusion and adaptive path selection can be clearly isolated and verified.

[0134] The test results of each method on the voltage sensor defect dataset are clearly presented in the table below:

[0135]

[0136] Conclusion: In terms of the core accuracy metric, the method of this invention (full version) achieved a mIoU score of 94.5%, which is not only significantly better than the two advanced general methods U-Net (88.1%) and DeepLabv3+ (90.3%), but also significantly higher than their fixed-strategy ablation versions (91.2%). This fully demonstrates that the overall technical framework proposed in this invention has significant advantages in the task of locating defects in low-voltage electrical appliances.

[0137] In terms of operational efficiency, this invention demonstrates its unique advantage in dynamic decision-making. Although the inference speed (17 FPS) of the method of this invention (fixed strategy) is slightly lower than that of DeepLabv3+ due to the need to process multi-scale features after fusion, the complete solution of this invention achieves an average inference speed of approximately 25 FPS by introducing dynamic path selection. This is because the recurrent strategy agent can autonomously select the computation path according to the complexity of the image content. When relatively simple surface defects are detected, the system calls the surface anomaly decoder with lower computational cost, thereby significantly improving the processing speed. Only when facing structural defects that require complex global understanding is the computationally intensive structural relationship decoder called. This intelligent processing method enables this invention to achieve the highest accuracy while maintaining a high average processing efficiency, meeting the real-time requirements of industrial automated production lines.

[0138] From the perspective of model complexity, the method of this invention has the highest total number of parameters (68.2M) among all the comparative methods due to the integration of a recurrent policy agent and multiple parallel detection heads. However, the experimental results strongly demonstrate that this increase in the number of parameters is an efficient and necessary investment. Compared with the method of this invention (fixed policy), only about 5.7M more parameters (mainly used for additional detection heads and decision networks) are added, but this results in a 3.3 percentage point improvement in mIoU and a nearly 47% increase in average speed. This shows that the present invention does not simply stack the number of network layers, but efficiently schedules and utilizes these increased parameters through a decision mechanism, achieving a "non-linear" increase in accuracy and efficiency, proving the advanced nature and rationality of its design.

[0139] In summary, this embodiment, through multi-dimensional and quantifiable comparative experiments, fully demonstrates that the technical solution described in this invention achieves superior overall results compared to existing technologies in specific industrial application scenarios. This invention not only reaches a new level in positioning accuracy but also achieves an excellent balance between model accuracy, operational efficiency, and intelligence through its core dynamic decision-making mechanism, showcasing technological progress and practical application value.

[0140] Reference Figures 1 to 4 This invention provides a method and system for locating defects in low-voltage electrical appliances based on dynamic weighted feature fusion.

[0141] The method and system obtain a set of multi-scale feature maps of the image to be detected through the feature extraction module 10;

[0142] Subsequently, the state construction module 20 constructs a comprehensive state vector based on the feature map set and combined with the uncertainty information generated by the reconnaissance network.

[0143] The recurrent policy agent 30 receives the state vector and generates a composite action that includes dynamic fusion weights and processing path selection;

[0144] Based on this composite action, the strategy-driven execution module 40 adaptively performs weighted fusion of multi-scale feature maps and calls the detection head most suitable for the current scene for decoding, thereby completing defect localization.

[0145] This mechanism is optimized through reinforcement learning-based training, enabling the system to strike a balance between detection accuracy and computational resource consumption.

[0146] The systems and methods of this invention can be implemented in hardware or software. When implemented in software, they can be stored as program modules in a computer-readable storage medium, such as random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or optical disk. When these program modules are executed by a processor, they can perform the functions defined in the aforementioned methods.

[0147] In a specific hardware implementation environment, the system of the present invention can be deployed on a computer device. This computer device includes, but is not limited to, one or more processors (CPUs), a graphics processing unit (GPU), memory, and a communication interface for communicating with other devices. The functions of the feature extraction module 10, the state construction module 20, the recurrent policy agent 30, and the policy-driven execution module 40 can all be implemented by the processor executing computer program instructions stored in the memory. Tasks involving massive parallel computation, such as convolutional neural network inference, can be accelerated by the graphics processing unit (GPU) to improve the overall operating efficiency of the system.

[0148] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A low-voltage electrical appliance defect positioning method based on dynamic weighted feature fusion, characterized in that, The method comprises: acquiring a low-voltage electrical appliance image to be detected, and extracting a multi-scale feature map set from the low-voltage electrical appliance image; constructing a state vector based on the extracted multi-scale feature map set; wherein: high-order semantic features in the multi-scale feature map set are input into a reconnaissance network to generate a preliminary defect probability prediction map; information entropy is calculated based on the preliminary defect probability prediction map to generate an uncertainty map quantifying the degree of cognitive uncertainty of the model for each image region; the uncertainty map and the multi-scale feature map set are spliced along the channel dimension, and information compression is performed through an encoder network to generate a state vector; inputting the state vector into a recurrent policy agent to generate a composite action containing a dynamic fusion weight and a selected processing path; performing weighted fusion on the multi-scale feature map set according to the dynamic fusion weight in the composite action to generate a fused feature map; processing the generated fused feature map according to the selected processing path in the composite action to ultimately obtain a defect positioning result of the low-voltage electrical appliance; wherein: a processing path library containing multiple parallel detection heads optimized for different defect modes is pre-set in the system; the corresponding detection head is indexed from the processing path library according to the selected processing path; and the fused feature map is input into the indexed detection head to generate a defect positioning result.

2. The low-voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 1, characterized in that, The recurrent policy agent is internally integrated with a recurrent neural network unit, and outputs the composite action based on the state vector and the hidden state of the previous time step.

3. The low voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 2, characterized in that, The process of inputting the state vector into the recurrent policy agent to generate a composite action containing a dynamic fusion weight comprises: inputting the hidden state of the current time step into a fully connected layer; applying a Softmax function to the output of the fully connected layer to generate the dynamic fusion weight.

4. The low voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 2, characterized in that, The process of inputting the state vector into the recurrent policy agent to generate a composite action containing a selected processing path comprises: inputting the hidden state of the current time step into another fully connected layer to output a score vector; determining the selected processing path according to a probability distribution converted based on the score vector.

5. The low voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 1, characterized in that, The multiple detection heads in the processing path library comprise: a structural relationship decoder, the network structure of which is configured to enhance the global context perception ability of features to decode structural defects representing the spatial relationship between low-voltage electrical appliance components; a surface anomaly decoder, the network structure of which is configured to maintain sensitivity to high-resolution detail features to decode surface defects representing physical damage on the surface of the low-voltage electrical appliance.

6. The low voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 1, characterized in that, The step of performing weighted fusion on the multi-scale feature map set comprises: adjusting each feature map in the multi-scale feature map set to a unified spatial resolution through an upsampling operation; pixel-by-pixel multiplying each feature map with the corresponding dynamic fusion weight after aligning the resolution, and then performing pixel-by-pixel addition to generate the fused feature map.

7. The low voltage electrical appliance defect positioning method based on dynamic weighting feature fusion according to claim 1, characterized in that, The method further comprises a step of training the recurrent policy agent by optimizing parameters of the agent by maximizing a final reward signal, wherein the final reward signal is obtained by subtracting a decision cost resulting from decisions of the agent from an accuracy reward of the positioning result, and the decision cost specifically comprises: a fusion weight decision cost related to the dynamic fusion weight, the fusion weight decision cost being determined by calculating an information entropy of the dynamic fusion weight vector to punish uncertain weight distribution; and a pre-design computation overhead related to the selected processing path.

8. A low voltage apparatus defect locating system based on dynamic weighted feature fusion, applied to the method of any one of claims 1-7, characterized in that, The system comprises: a feature extraction module configured to acquire a low-voltage electrical appliance image to be detected and extract a multi-scale feature map set from the low-voltage electrical appliance image; a state construction module configured to construct a state vector based on the multi-scale feature map set extracted by the feature extraction module; a recurrent policy agent configured to receive the state vector and output a composite action comprising a dynamic fusion weight and a selected processing path; a policy-driven execution module configured to perform weighted fusion on the multi-scale feature map set according to the composite action output by the recurrent policy agent to generate a fused feature map, and process the generated fused feature map to finally obtain a defect positioning result of the low-voltage electrical appliance.

Images