Lightweight method for image recognition and target detection based on deep learning

Abstract

This invention relates to the field of deep learning technology and discloses a lightweight method for image recognition and object detection based on deep learning, comprising the following steps: Step S1, acquiring the image to be inspected; Step S2, generating a specular feature map; Step S3, calculating enhanced backbone features; Step S4, calculating multi-scale features; Step S5, calculating a corrected score; Step S6, jointly optimizing the object detection model with detection loss and sparse loss and extracting category thresholds; Step S7, outputting the object category, object location, and confidence score. This invention addresses the need for detecting reflective micro-components in electronic product repair scenarios. By using specular features to transform specular interference into usable structural cues, it guides the model to focus on micro-structural details obscured by specular light. Through channel sparsity using feature constraints, it retains core detection channels while compressing the model size, achieving a balance between lightweight model and detection performance, and is adaptable to offline deployment requirements of low-computing-power terminals.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically, to a lightweight method for image recognition and object detection based on deep learning. Background Technology

[0002] With the rapid development of the consumer electronics industry, the demand for automated and intelligent testing tools in the electronics repair industry continues to increase. Electronic products contain numerous tiny components at the millimeter or even sub-millimeter level, requiring rapid location and defect identification during repair. Deep learning-based target detection technology is the core technological path to achieving this. Due to equipment limitations in repair scenarios, testing models need to be deployed on low-computing-power devices such as mobile phones, tablets, or embedded terminals, making lightweight testing models the primary application form in this scenario.

[0003] Existing lightweight object detection models are mostly designed for general scenarios, exhibiting significant adaptability deficiencies in maintenance scenarios. In maintenance scenarios, tiny metal and glass components produce specular highlights under maintenance lighting. These highlights form areas of high brightness, obscuring key discriminative structures such as grooves, gaps, and cracks on the component itself. To reduce computational load, existing lightweight models reduce parameter size through multiple downsampling and channel compression. Strong gradient features in the highlight regions are preferentially preserved during feature extraction, while the microstructural details obscured by the highlights are gradually lost.

[0004] Existing technologies for handling specular interference primarily involve image enhancement or specular region removal, failing to distinguish between interfering specular highlights and those carrying structural information, easily leading to secondary loss of microstructural information. The pruning and compression processes of general lightweight models prioritize parameter and computational efficiency, indiscriminately reducing network channels. This can easily remove critical channels relevant to the detection of minute components, ultimately resulting in missed detections, false detections, and location misalignments, failing to meet the actual testing needs of electronic product repair scenarios. Summary of the Invention

[0005] This invention provides a lightweight method for image recognition and object detection based on deep learning, which solves the technical problems mentioned in the background.

[0006] This invention provides a lightweight method for image recognition and object detection based on deep learning, comprising the following steps: Step S1: Acquire the image to be inspected and extract the normalized brightness, gradient magnitude and gradient direction of the image to be inspected; Step S2: Calculate the highlight weight and highlight centroid based on the normalized brightness. Determine the radial and tangential directions from the highlight centroid. Combine the highlight weight, gradient magnitude, and gradient direction to obtain the radial and tangential energy along the radial and tangential directions. Generate the highlight feature map from the radial and tangential energy. Step S3: Input the image to be inspected and the highlight feature map into the backbone network to extract backbone features. Generate a spatial gating map from the highlight feature map. Modulate the backbone features with the spatial gating map to obtain enhanced backbone features. Step S4: The enhanced backbone features and the specular feature map are input into the fusion network and multi-scale features are obtained by spatial gating map modulation. Step S5: Input the multi-scale feature into the detection network to generate candidate boxes, class probabilities and initial scores, and use the value of the highlight feature map in the candidate box to correct the initial score to obtain the corrected score; Step S6: Set sample weights based on the values ​​of the highlight feature maps within the labeled boxes, construct a detection loss by combining candidate boxes and class probabilities, generate a sparse loss based on the mapping relationship between channel activation and highlight feature map values, optimize the target detection model by combining the detection loss and the sparse loss, and extract the class threshold. Step S7: Perform decay calculation on the corrected scores of overlapping candidate boxes of the same type to obtain the final score, extract candidate boxes whose final scores are not less than the category threshold, and output the target category, target location and confidence level.

[0007] The beneficial effects of this invention are as follows: This invention addresses the need for detecting reflective micro-components in electronic product repair scenarios by constructing a complete lightweight detection process. By defining specular features, this invention transforms specular interference into usable structural cues, guiding the model to focus on microstructural details obscured by specular highlights. This invention integrates this feature throughout the entire process of feature extraction, multi-scale fusion, candidate box generation, and model training, enabling the lightweight model to retain key discriminative information about microstructures even under low computing power conditions. Furthermore, this invention achieves a balance between lightweight model design and detection performance by using channel sparsity through feature constraints, compressing the model size while preserving core detection channels, thus adapting to the offline deployment requirements of low-computing-power terminals. Attached Figure Description

[0008] Figure 1 This is a flowchart of the computation of the lightweight image recognition and object detection method based on deep learning of the present invention. Detailed Implementation

[0009] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0010] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0011] like Figure 1 As shown, a lightweight method for image recognition and object detection based on deep learning includes the following steps: Step S1: Acquire the image to be inspected and extract the normalized brightness, gradient magnitude and gradient direction of the image to be inspected; Step S2: Calculate the highlight weight and highlight centroid based on the normalized brightness. Determine the radial and tangential directions from the highlight centroid. Combine the highlight weight, gradient magnitude, and gradient direction to obtain the radial and tangential energy along the radial and tangential directions. Generate the highlight feature map from the radial and tangential energy. Step S3: Input the image to be inspected and the highlight feature map into the backbone network to extract backbone features. Generate a spatial gating map from the highlight feature map. Modulate the backbone features with the spatial gating map to obtain enhanced backbone features. Step S4: The enhanced backbone features and the specular feature map are input into the fusion network and multi-scale features are obtained by spatial gating map modulation. Step S5: Input the multi-scale feature into the detection network to generate candidate boxes, class probabilities and initial scores, and use the value of the highlight feature map in the candidate box to correct the initial score to obtain the corrected score; Step S6: Set sample weights based on the values ​​of the highlight feature maps within the labeled boxes, construct a detection loss by combining candidate boxes and class probabilities, generate a sparse loss based on the mapping relationship between channel activation and highlight feature map values, optimize the target detection model by combining the detection loss and the sparse loss, and extract the class threshold. Step S7: Perform decay calculation on the corrected scores of overlapping candidate boxes of the same type to obtain the final score, extract candidate boxes whose final scores are not less than the category threshold, and output the target category, target location and confidence level.

[0012] In one embodiment of the present invention, the calculation process of step S1 specifically includes: The formula for calculating the brightness value is as follows, based on the pixel values ​​of the red channel, green channel, and blue channel, and the brightness conversion factor: in, This is the brightness value. These are the pixel values ​​for the red channel. These are the pixel values ​​for the green channel. These are the pixel values ​​for the blue channel; 0.299, 0.587, and 0.114 are brightness conversion factors. Based on the brightness value, the mean brightness value of the image to be inspected, and the smallest positive number, the formula for calculating the normalized brightness is as follows: in, For normalized brightness, For logarithmic function operations, This is the brightness value. The average brightness of the image to be tested. It is a very small positive number; Based on the horizontal and vertical luminance differences of the normalized luminance, the formula for calculating the luminance gradient vector is as follows: ; in, The brightness gradient vector, For horizontal brightness difference, For vertical brightness difference; The formula for calculating the gradient magnitude based on the brightness gradient vector is as follows: in, For gradient magnitude, The brightness gradient vector, For L2 norm operations; Based on the brightness gradient vector, gradient magnitude, and minimum positive number, the formula for calculating the gradient direction is as follows: ; in, For the gradient direction, The brightness gradient vector, For gradient magnitude, It is a very small positive number.

[0013] It should be noted that the image under inspection is a single-frame red, green, and blue three-channel digital image captured in an electronic product repair bench setting. It can be acquired using an industrial area scan camera, a specialized repair microscope lens, or a mobile phone's main camera paired with an image acquisition module. During acquisition, it is crucial to ensure the lens optical axis is perpendicular to the repair bench plane to avoid perspective distortion. The height of the image under inspection is the total number of pixels in its vertical dimension. The width of the image under inspection is the total number of pixels in its horizontal dimension. The pixel values ​​of the red, green, and blue channels are the sampled and quantized values ​​of each pixel in the image under inspection in the visible light red, visible light green, and visible light blue bands, respectively. The brightness conversion coefficients for the red, green, and blue channels are the brightness conversion weights corresponding to the pixel values ​​of the red, green, and blue channels, respectively. In other words, the three-channel brightness conversion coefficients conform to the international standard for digital video brightness conversion, matching the human eye's sensitivity to wavelength light intensity. It should be noted that the brightness value is a single-channel value representing the brightness of each pixel in the image, obtained by multiplying the three-channel pixel values ​​by their corresponding brightness conversion factors and then summing them. The mean brightness value of the image under test is the average of the brightness values ​​of all pixels in the entire image, used to represent the overall brightness level of the image. The minimum positive number is a fixed value used to avoid the denominator being zero in division operations, preferably 10 to the power of -6. Normalized brightness is single-channel brightness data after eliminating the overall exposure differences of the image under different acquisition conditions and compressing the numerical range of strong highlight areas, obtained by performing a logarithmic operation on the brightness value, the mean brightness value of the image under test, and the minimum positive number. The horizontal brightness difference is the brightness difference between adjacent pixels in the horizontal direction of the normalized brightness map, used to represent the intensity and direction of brightness changes in the horizontal direction of the image. The vertical brightness difference is the brightness difference between adjacent pixels in the vertical direction of the normalized brightness map, used to represent the intensity and direction of brightness changes in the vertical direction of the image. The luminance gradient vector is a two-dimensional vector formed by combining horizontal and vertical luminance differences, used to simultaneously characterize the intensity and direction of luminance changes at each pixel location in the image. The gradient magnitude is a single-channel value obtained by performing a L2 norm operation on the luminance gradient vector, used to characterize the overall strength of luminance changes at each pixel location in the image. The gradient direction is a two-dimensional unit vector obtained by normalizing the luminance gradient vector, used to retain only the directional information of the luminance change, weakening the interference caused by differences in gradient magnitude.

[0014] It should be noted that the normalization method for the red, green, and blue three-channel images to be inspected is linear normalization. Specifically, the pixel values ​​of each channel in the acquired original image are first linearly mapped from the integer range of 0 to 255 output by the acquisition device to the floating-point range of 0 to 1. The mapping rule is that the normalized pixel value equals the original pixel value divided by 255. The horizontal and vertical brightness differences are calculated using the center difference method. This method better preserves the edge information of microstructures and is suitable for the inspection needs of small components in maintenance scenarios. For pixels at the image edges, a mirror filling method is used to supplement the pixel values ​​outside the boundary before performing center difference calculation, avoiding the loss of gradient information in the edge areas.

[0015] In one embodiment of the present invention, the calculation process of step S2 specifically includes: The formula for calculating highlight weight is as follows, based on normalized brightness, mean local window brightness, standard deviation of local window brightness, and minimum positive number: in, For highlight weight, For activation function computation, For normalized brightness, This represents the average brightness of a local window. The standard deviation of local window brightness. For local windows, It is a very small positive number. Center pixel These are pixel coordinates; The formula for calculating the specular centroid, based on specular weight, pixel coordinates, and a minimum positive number, is as follows: in, For the center of gravity of the highlight, For summation operations within a local window, For highlight weight, For pixel coordinates, It is a very small positive number; The formula for calculating the radial direction, based on pixel coordinates, specular centroid, and a minimum positive number, is as follows: in, Radial direction For pixel coordinates, For the center of gravity of the highlight, The L2 norm of the positional difference. It is a very small positive number; The formula for calculating the tangential direction is as follows, based on the transverse and longitudinal components of the radial direction: in, Tangential direction For the vertical component, For horizontal components; Based on the highlight weight, gradient magnitude, gradient direction, and radial direction, the formula for calculating radial energy is as follows: in, Radial energy, To perform a summation operation within a local window, For highlight weight, For gradient magnitude, For the gradient direction, Radial direction This is for absolute value operations; Based on the highlight weight, gradient magnitude, gradient direction, and tangential direction, the formula for calculating the tangential energy is as follows: in, For tangential energy, To perform a summation operation within a local window, For highlight weight, For gradient magnitude, For the gradient direction, Tangential direction This is for absolute value operations; The formula for calculating the specular feature map is as follows, based on radial energy, tangential energy, minimum positive number, specular weight of the center pixel, L2 norm of the difference between the center pixel and the specular centroid, and feature step size: in, This is a highlight feature map. For linear rectified function operations, Radial energy, For tangential energy, The highlight weight of the center pixel, Center pixel For the center of gravity of the highlight, The L2 norm of the difference between the center pixel and the specular centroid. For characteristic step size, It is a very small positive number.

[0016] It should be noted that the detection scale number is an identifier used to distinguish different feature extraction scales. The feature stride is the scaling factor of the feature map relative to the original image at the corresponding detection scale, used to match the downsampling factor of the lightweight detection network, ensuring that the size of feature maps at different scales is aligned with the output features of the backbone network. The center pixel is the core location for performing specular feature calculation in the feature map, and each center pixel corresponds to an independent local calculation window and feature value. Pixel coordinates are the position identifiers of each pixel within the local window in the image coordinate system. The infinity norm operation is a standard operation used to define the range of the local window, ensuring that the maximum distance from each pixel within the window to the center pixel is consistent, adapting to the feature calculation requirements of circular specular regions. The local calculation window is a square calculation area centered on the center pixel, used to limit the pixel range of a single set of feature calculations, and the window size matches the feature stride of the corresponding scale.

[0017] It should be noted that the mean brightness within a local window is used to characterize the overall brightness level within that window. The standard deviation of brightness within a local window is used to characterize the dispersion of brightness values ​​within the local window, distinguishing between highlight areas and ordinary texture areas. The highlight weight is a continuous value between 0 and 1 representing the probability that a corresponding pixel belongs to a highlight area, obtained by applying a Sigmoid activation function based on the normalized brightness, mean brightness, standard deviation of brightness, and the smallest positive number within the local window. The local highlight centroid is the coordinate value representing the center position of the highlight distribution within the local window, calculated by weighted averaging based on the highlight weight and pixel coordinates of each pixel within the local window. The radial direction is a two-dimensional unit vector obtained by normalizing the vector pointing from the local highlight centroid to the corresponding pixel coordinates, used to characterize the radial diffusion direction of the highlight area. The vertical component of the radial direction is the projection value of the radial direction unit vector onto the vertical coordinate axis of the image. The horizontal component of the radial direction is the projection value of the radial direction unit vector onto the horizontal coordinate axis of the image. The tangential direction is a two-dimensional unit vector that is completely perpendicular to the radial unit vector, corresponding to the extension direction of microstructures such as screw slots, pad gaps, and cracks.

[0018] It should be noted that radial energy is a numerical value representing the sum of gradient intensities along the radial direction of the highlight region within a local window. It is obtained by summing the products of the highlight weight, gradient magnitude, and the absolute value of the dot product of the gradient direction and the radial direction for each pixel within the local window. Tangential energy is a numerical value representing the sum of gradient intensities along the tangential direction of the microstructure within a local window. It is obtained by summing the products of the non-highlight weight, gradient magnitude, and the absolute value of the dot product of the gradient direction and the tangential direction for each pixel within the local window. The linear rectified function operation is a nonlinear operation used to filter invalid feature responses; that is, it filters out invalid regions where the radial energy is less than or equal to the tangential energy, retaining only the valid feature responses of the highlight-occluded microstructure. The highlight feature map is a single-channel feature map representing the probability of highlight-occluded microstructures at each pixel location. It is calculated based on radial energy, tangential energy, highlight weight, and highlight eccentricity distance. A higher value indicates that a real microstructure obscured by highlight is more likely to exist at that location.

[0019] It should be noted that the number of detection scales is set to three, corresponding to scale numbers 1, 2, and 3. The specific feature step size for each scale is 4 for scale 1, 8 for scale 2, and 16 for scale 3. This value perfectly matches the feature map size of the fourth, third, and second subsampling outputs of the lightweight detection network, ensuring complete spatial alignment between the specular feature map and the backbone network output features, thus enabling subsequent gating modulation and feature fusion. In practical implementation, if it is necessary to detect even smaller submicron-level micro-components, scale 0 can be added, with a corresponding feature step size of 2, to accommodate higher resolution microscopic image input.

[0020] It should be noted that the specific size setting rule for the local computation window is as follows: the window side length is equal to twice the feature step size plus 1 at the corresponding scale, ensuring that the window is a square with an odd-numbered side length, and the distance between the center pixel and the window edge is equal to the feature step size at the corresponding scale. The local window side lengths for scales 1, 2, and 3 are 9, 17, and 33, respectively, and the corresponding window sizes are 9×9, 17×17, and 33×33. This setting rule ensures that the size of the local window perfectly matches the specular region range at the corresponding scale, which can completely cover the complete distribution of the specular region without introducing too much interference from irrelevant background areas, thus ensuring the accuracy of specular centroid and gradient energy calculation.

[0021] It should be noted that the size matching rule for multi-scale feature maps is as follows: the size of the specular feature map at each scale must be exactly the same as the size of the corresponding scale's backbone network output feature. Specifically, for the feature map at scale 1, the calculation result at the original image resolution is used directly; for the feature map at scale 2, the calculation result at the original resolution is downsampled by a factor of 2; and for the feature map at scale 3, the calculation result at the original resolution is downsampled by a factor of 4. The downsampling process uses a window averaging method to ensure that the downsampled feature values ​​accurately correspond to the specular occlusion characteristics of the corresponding region, achieving precise matching with the backbone network features.

[0022] In one embodiment of the present invention, the calculation process of step S3 specifically includes: Based on the image to be inspected and the first-scale specular feature map, the formula for calculating the initial features of the network is as follows: ; in, These are the initial characteristics of the network. For spatial convolution operations, For channel splicing, The image to be inspected. This is the first-scale specular feature map; Based on the input features from the previous layer, the formula for calculating the backbone features is as follows: in, Main characteristics For the first point convolution operation, For depthwise convolution operations, For the second point convolution operation, These are the input features for the previous layer; Based on the specular feature map, feature step size, and backbone features, the formula for calculating the spatial gating map is as follows: ; in, For spatial gating diagrams, For activation function computation, For gated convolution operations, For channel splicing, For average downsampling processing, This is a highlight feature map. For characteristic step size, Main characteristics; Based on the backbone features, scaling factor, spatial gating map, and input features from the previous layer, the formula for calculating the enhanced backbone features is as follows: in, To enhance the characteristics of the main trunk, Main characteristics This is an element-wise multiplication operation. This is the scaling factor. For spatial gating diagrams, For size matching projection calculation, This is the input feature of the previous layer.

[0023] It should be noted that the 3×3 spatial convolution operation is a two-dimensional convolution operation used to extract initial spatial features. It is well-suited to the low computational requirements of lightweight networks, effectively extracting the initial spatial features of the input image while maintaining the feature map size consistent with the input image. Channel concatenation is a standard operation used to merge multiple feature maps. By concatenating and merging multiple feature maps along the channel dimension, it can fuse the visual features of the original image with the structural features of the specular feature map without changing the spatial size of the feature map. The first-scale specular feature map is the feature calculation result corresponding to scale 1, used to construct the initial input features of the backbone network, guiding the network to focus on the microstructure regions of specular occlusion from the initial stage. The initial input features of the backbone network are the input feature data of the first layer of the backbone network, obtained by concatenating the image to be examined with the first-scale specular feature map through channel concatenation and then performing a 3×3 spatial convolution operation. The input features of the next layer of the backbone network are the enhanced backbone features output from the previous layer of the backbone network, serving as the input data for the current layer's convolution operation, realizing the layer-by-layer transfer and extraction of features.

[0024] It should be noted that the first 1×1 point convolution operation is a two-dimensional convolution operation used for channel expansion. This can improve the channel representation capability of features with low computational cost, adapting to the feature extraction requirements of subsequent depthwise convolutions. The 3×3 depthwise convolution operation is a lightweight spatial feature extraction operation; each convolution kernel corresponds to only one input channel, extracting spatial information of features with extremely low computational cost. The second 1×1 point convolution operation is a two-dimensional convolution operation used for channel compression. It can compress and mix the channel information of the features output by the depthwise convolution, restoring the expanded channel number to the input channel number, achieving structural closure of the inverse residual unit, and reducing the overall computational cost. The ungated backbone features are the unmodulated raw features extracted from the current layer of the backbone network, obtained by sequentially processing the input features of the previous layer through the first 1×1 point convolution operation, the 3×3 depthwise convolution operation, and the second 1×1 point convolution operation. The 1×1 point convolution operation used to generate the gated graph is a two-dimensional convolution operation used to generate spatial weights. It can map the concatenated multi-channel features into a single-channel gated weight map, thereby generating weights in the spatial dimension.

[0025] It's important to note that the average downsampling operation corresponding to the feature step size is a pooling operation used to adjust the feature map size. It downsamples high-resolution feature maps to the same spatial size as the backbone features, achieving precise feature concatenation. The spatial gating map is a single-channel weight map representing the importance of each spatial location in the feature map. It is obtained by concatenating the downsampled feature map with the backbone features, followed by gated point convolution and a sigmoid activation function. Higher weights indicate that the network needs to focus more on that location. Element-wise multiplication is a standard operation for feature weighted modulation. Multiplying the corresponding pixel values ​​of two feature maps of the same size allows for spatial weighted modulation of the backbone features by the gating map, enhancing the feature response of important regions and suppressing feature interference from irrelevant regions. The learnable gating scaling factor is a learnable parameter used to control the gating modulation intensity. It controls the modulation intensity of the backbone features by the gating, ensuring network stability in the early stages of training and preventing excessive gating modulation from damaging the original feature representation. Size-matched projection is an operation used for size alignment of residual connections. It adjusts the number of channels and size of the input features to perfectly match the output features, achieving size alignment of residual connections and ensuring stable feature transfer. Enhanced backbone features are the final output features after spatial gating modulation and residual connections. They retain the original visual features while enhancing the feature responses of microstructure regions with specular occlusion.

[0026] It should be noted that the key hyperparameters for the 3×3 spatial convolution operation are: 16 kernels, a stride of 1, and identical padding with a padding value of 1, ensuring the spatial dimensions of the output feature map are completely identical to the input image. The key hyperparameters for the first 1×1 point convolution operation are: twice the number of input channels, a stride of 1, effective padding with a padding value of 0, achieving channel expansion. The key hyperparameters for the 3×3 depthwise convolution operation are: the number of kernels is exactly the same as the number of input channels, a stride of 1, identical padding with a padding value of 1, ensuring each input channel corresponds to an independent kernel. The key hyperparameters for the second 1×1 point convolution operation are: half the number of input channels, a stride of 1, effective padding with a padding value of 0, achieving channel compression and restoration. The key hyperparameters for the 1×1 point convolution operation used to generate the gated graph are: kernel number set to 1, stride set to 1, padding mode set to effective padding, padding value set to 0, and outputting a single-channel gated weight map.

[0027] It should be noted that the backbone network has a total of 16 layers, divided into 4 stages. Each stage contains 4 inverse residual units. The first stage corresponds to scale 1, with a feature stride of 4, and the output feature map size is 1 / 4 of the input image. The second stage corresponds to scale 2, with a feature stride of 8, and the output feature map size is 1 / 8 of the input image. The third and fourth stages correspond to scale 3, with a feature stride of 16, and the output feature map size is 1 / 16 of the input image. The last layer of each stage outputs the enhanced backbone features and spatial gating map of the corresponding scale, which are used for subsequent feature fusion and gating modulation. In practical implementation, if the computing power of the deployed equipment is sufficient, the number of inverse residual units in each stage can be increased to 6 to improve feature representation capability. If the computing power of the deployed equipment is limited, the number of inverse residual units in each stage can be reduced to 2 to further reduce the computational load.

[0028] It should be noted that the size-matching projection operation is implemented using a 1×1 point convolution. When the number of channels in the input and output features is inconsistent, the number of channels in the input features is adjusted through a 1×1 point convolution to make it completely consistent with the number of channels in the output features. When the spatial dimensions of the input and output features are inconsistent, an average pooling operation with a stride of 2 is added before the 1×1 point convolution to downsample the spatial dimensions of the input features to be consistent with the output features. When the number of channels and spatial dimensions of the input and output features are completely consistent, the size-matching projection operation is an identity mapping, directly passing the input features to the output of the residual connection. This implementation can achieve size alignment of the residual connection with extremely low computational cost, ensuring the stability of feature propagation and avoiding the gradient vanishing problem.

[0029] It should be noted that the learnable gating scaling factor is initialized as a constant, with an initial value uniformly set to 0.1. The value is constrained to be greater than or equal to 0 and less than or equal to 1. During model training, it is optimized and updated end-to-end using the backpropagation algorithm. The rationale for this initial value is that a smaller scaling factor in the early stages of training ensures lower modulation intensity of the gating, preventing excessive modification of the original backbone features by the gating weights and ensuring network training stability. As training progresses, the scaling factor will automatically optimize to its optimal value based on the features of the dataset, balancing the intensity of the original feature representation and the feature guidance. In practical implementation, for datasets with more severe specular interference, the initial value can be set to 0.2 to increase the initial intensity of the gating modulation.

[0030] It should be noted that the specific kernel size and step size for the average downsampling operation corresponding to the feature step size are set as follows: the kernel size is equal to the feature step size at the corresponding scale, and the step size is also equal to the feature step size at the corresponding scale. The padding method is set to effective padding with no additional padding values. For example, the feature step size corresponding to scale 1 is 4, the kernel size for the average downsampling operation is 4×4, and the step size is 4; the feature step size corresponding to scale 2 is 8, the kernel size for the average downsampling operation is 8×8, and the step size is 8; the feature step size corresponding to scale 3 is 16, the kernel size for the average downsampling operation is 16×16, and the step size is 16. This setting ensures that the spatial size of the downsampled feature map is completely consistent with the spatial size of the corresponding layer's backbone features, achieving accurate feature stitching and fusion.

[0031] In one embodiment of the present invention, the calculation process of step S4 specifically includes: Based on the deepest enhanced backbone features, the formula for calculating deep semantic features is as follows: in, For deep semantic features, For point convolution operations, This enhances the core features at the deepest level. Based on the enhanced backbone features, deeper multi-scale features, specular feature maps, and spatial gating maps, the calculation formula for multi-scale features is as follows: in, For multi-scale features, For point convolution operations, For channel splicing, To enhance the characteristics of the main trunk, For upsampling processing, For deeper, multi-scale features, For average downsampling processing, For characteristic step size, This is a highlight feature map. This is an element-wise multiplication operation. This is a spatial gating diagram.

[0032] It should be noted that the 1×1 point convolution operation is a two-dimensional convolution operation used for channel compression and information fusion. It can perform channel compression and information fusion on the concatenated multi-channel features with low computational cost, unifying the number of channels for features at different scales and adapting to the input requirements of subsequent detection heads. The deepest enhanced backbone feature is the output feature corresponding to the third scale of the backbone network, containing the highest-level semantic information, and serves as the starting input for multi-scale feature fusion. The deep semantic feature is the feature map obtained after the deepest enhanced backbone feature undergoes a 1×1 point convolution operation, serving as the initial fusion feature of the deepest layer and initiating the top-down multi-scale fusion process. The upsampling operation is used to enlarge the feature map size, scaling the spatial size of the deep feature map to match that of the shallow feature map. Figure 1 To achieve this, the smoothness of the features is maintained, adapting to the requirement of preserving microstructural details. The deeper multi-scale fusion feature is a fusion output feature from a deeper layer than the current layer. After upsampling, it is concatenated and fused with the shallow features of the current layer. The current-scale multi-scale fusion feature is a fusion feature map that simultaneously contains shallow spatial detail information and deep semantic information, while preserving the microstructural features of the specular occlusion region. It is obtained by concatenating the current layer's enhanced backbone features, the upsampled deep fusion feature, and the feature map at the corresponding scale, followed by 1×1 point convolution and spatial gating modulation.

[0033] It should be noted that the upsampling operation is implemented using bilinear interpolation upsampling, with a fixed upsampling factor of 2. The specific calculation rule is as follows: for each pixel position in the target feature map, the target pixel value is calculated by weighted averaging the values ​​of its four neighboring pixels in the original feature map. The weighting is inversely proportional to the distance between pixels. Compared to nearest neighbor interpolation upsampling, this implementation can generate a smoother feature map, avoid jagged edge artifacts, better preserve the edge details of microstructures, and is suitable for the detection needs of small components in maintenance scenarios.

[0034] It should be noted that the key hyperparameters for the 1×1 point convolution operation used for deep semantic feature computation are: 96 kernels, a stride of 1, effective padding with a padding value of 0. The same applies to the 1×1 point convolution operation used for multi-scale fusion feature computation. This kernel number setting ensures sufficient channel representation capability for the fused features without introducing excessive computation, thus adapting to the deployment requirements of lightweight networks. In practical implementation, if the deployment device has sufficient computing power, the kernel number can be increased to 128 to enhance feature representation capability; if the deployment device has limited computing power, the kernel number can be reduced to 64 to further reduce computation.

[0035] It should be noted that the total number of feature fusion layers is set to 3, corresponding to 3 detection scales. The fusion order is from top to bottom, starting with the deepest scale 3, and then proceeding to scale 2 and scale 1. The fused features at scale 3 are deep semantic features, directly obtained from the deepest enhanced backbone features through a 1×1 point convolution. The fused features at scale 2 are calculated by concatenating the enhanced backbone features of scale 2, the upsampled fused features of scale 3, and the specular feature map of scale 2. The fused features at scale 1 are calculated by concatenating the enhanced backbone features of scale 1, the upsampled fused features of scale 2, and the specular feature map of scale 1. The fused features from the three scales together constitute a multi-scale feature set, which is input into the subsequent detection head for candidate box and score calculation.

[0036] It should be noted that the size matching rule for multi-scale features is as follows: the spatial size of the fused feature at each scale is completely consistent with the spatial size of the enhanced backbone feature at the corresponding scale. In specific implementation, the fused features of the deeper layer are upsampled by 2x bilinear interpolation to enlarge their spatial size to the size of the enhanced backbone feature of the current layer, while the specular feature map at the corresponding scale is downsampled by averaging the corresponding feature step size to reduce its spatial size to the size of the enhanced backbone feature of the current layer. This ensures that the spatial sizes of the three input features are completely consistent, achieving accurate channel stitching and fusion.

[0037] In one embodiment of the present invention, the calculation process of step S5 specifically includes: Based on multi-scale features, the formula for calculating the prediction vector is as follows: in, For the prediction vector, The horizontal offset parameter of the center point. The vertical offset parameter for the center point. The logarithmic parameter of the target width. The logarithmic parameter of the target height. This is the initial score. For category logical parameters; The formula for calculating the candidate bounding box is as follows, based on the feature step size, horizontal grid position, vertical grid position, center point horizontal offset parameter, center point vertical offset parameter, target width logarithmic parameter, and target height logarithmic parameter:

[0038] in, For candidate boxes, For characteristic step size, For horizontal grid positions, For vertical grid positions, For activation function computation, For exponential function operations, The horizontal offset parameter of the center point. The vertical offset parameter for the center point. The logarithmic parameter of the target width. The logarithmic parameter of the target height; Based on the category logical parameters, the formulas for calculating the predicted category and category probability are as follows: in, For predicting categories, To extract the category corresponding to the maximum value, For class probabilities, For category logical parameters; The formula for calculating the corrected score is as follows, based on the initial score, the arithmetic mean of the highlight feature map values ​​within the candidate boxes, and the class probability: in, To correct the score, For activation function computation, This is the initial score. This is the arithmetic mean of the highlight feature map values ​​within the candidate bounding box. For candidate boxes, These are the values ​​of the highlight feature map. This represents the category probability.

[0039] It should be noted that the detection point index is an identifier used to uniquely identify each detection location on the feature map, composed of three parameters: scale number, horizontal grid position, and vertical grid position. The horizontal grid position is the position identifier of the detection point on the horizontal coordinate axis of the feature map, ranging from 0 to the width of the feature map at the corresponding scale minus 1, used to identify the position of the detection point on the horizontal coordinate axis of the feature map. The vertical grid position is the position identifier of the detection point on the vertical coordinate axis of the feature map. The prediction vector output by the detection head is a multi-dimensional vector output by the detection head at the corresponding detection location, containing target center point offset, size, score, and category information. The horizontal offset parameter of the center point is the horizontal offset of the corresponding target center point in the prediction vector relative to the top-left corner of the current grid. The vertical offset parameter of the center point is the vertical offset of the corresponding target center point in the prediction vector relative to the top-left corner of the current grid. The logarithmic parameter of the target width is the logarithmic predicted value of the target width in the prediction vector. The logarithmic parameter of the target height is the logarithmic predicted value of the target height in the prediction vector.

[0040] It should be noted that the initial score is the original logical value in the prediction vector indicating whether a target exists at the current detection location. The total number of target categories is the number of target categories covered by the detection task. The category logical parameters are the logical values ​​corresponding to each category in the prediction vector, and their number is consistent with the total number of target categories. The predicted candidate box is a rectangular box used to identify the position and size of the target in the image, calculated based on the feature stride, grid position, center point offset parameter, and size logarithmic parameter. The normalized exponential function operation is a non-linear operation used to generate category probabilities, which can map any range of category logical values ​​to category probabilities in the interval of 0 to 1, and the sum of the probabilities of all categories is 1. The corrected score is the final confidence score of the candidate box after specular feature modulation. It is obtained by adding the initial score to the average response of the features within the candidate box, performing a sigmoid activation function operation, and then multiplying by the category probability of the corresponding category. It is used to characterize the confidence level of the candidate box.

[0041] It should be noted that the total number of target categories is set to 8, and the specific definitions of each category are as follows: Category 1 is screws, which includes all types of fixing screws in electronic product repair scenarios; Category 2 is solder pads, which includes all round and square solder pads on the PCB board; Category 3 is solder points, which includes solder points and bridging areas on the solder pads; Category 4 is shielding cover pressure points, which includes the fixing pressure points of metal shielding covers; Category 5 is interface metal edges, which includes the metal shell edges of various charging interfaces and data interfaces; Category 6 is camera module frames, which includes the metal fixing frames of front and rear camera modules; Category 7 is glass fragments, which includes screen glass and lens glass fragments that fall off during repair; and Category 8 is motherboard defects, which includes abnormal defects such as burning, cracks, and gaps on the PCB board.

[0042] It should be noted that the feature map grid is divided according to the following rules: each scale of feature map corresponds to an independent grid system. The number of rows and columns of the grid is exactly the same as the width and height of the feature map at the corresponding scale. Each grid corresponds to a pixel position on the feature map and a square region on the original image. The side length of the region is equal to the feature step size at the corresponding scale. For example, the feature step size for scale 1 is 4, and each grid on the feature map corresponds to a 4×4 square region on the original image; the feature step size for scale 2 is 8, and each grid corresponds to an 8×8 square region on the original image; the feature step size for scale 3 is 16, and each grid corresponds to a 16×16 square region on the original image. The coordinates of the top left corner of each grid on the original image are: the horizontal coordinate equals the horizontal grid position multiplied by the feature step size, and the vertical coordinate equals the vertical grid position multiplied by the feature step size, serving as the reference position for calculating the center point offset.

[0043] It should be noted that the initial screening rule for candidate boxes in post-processing is as follows: first, low-confidence candidate boxes with a correction score below 0.01 are filtered out to reduce the computational load of subsequent overlap attenuation processing. Then, for each category of candidate boxes, they are sorted from highest to lowest correction score, and the top 100 candidate boxes in each category are retained for subsequent overlap attenuation and threshold filtering. This screening rule can significantly reduce the computational load of subsequent processing and improve inference speed without losing valid candidate boxes, thus adapting to the deployment requirements of low-computing-power maintenance terminals.

[0044] In one embodiment of the present invention, the calculation process of step S6 specifically includes: The formula for calculating the sample weight is as follows, based on the average response of the highlight feature map values ​​within the labeled box: ; in, For sample weights, This represents the average response result of the highlight feature map values ​​within the labeled area. For annotation boxes, These are the values ​​for the highlight feature map; The calculation formula for the detection loss is as follows, based on sample weights, perfect intersection-over-union ratio, cross-entropy, and binary cross-entropy loss: in, To detect the loss, To perform a summation over all targets, For sample weights, This represents the intersection-union ratio of the bounding boxes and the candidate boxes. For annotation boxes, For candidate boxes, The cross-entropy between the true class and the predicted class. For the real category, For predicting categories, To sum the results of consecutive additions, The binary cross-entropy loss is used to determine the true target and the output target. For the sake of true objective, To output the target; The formula for calculating the mapping relationship is as follows, based on the sum of the absolute values ​​of channel activations and the mean of the absolute values ​​of the specular feature maps: in, This is a mapping relationship. The result is the sum of the absolute values ​​of the channel activations. The mean of the absolute values ​​of the highlight feature map. It is a very small positive number; Based on the sparsity intensity coefficient, the absolute value of the channel gate, and the mapping relationship, the formula for calculating the sparsity loss is as follows: in, For sparsity loss, The sparsity intensity coefficient, To sum across all channel levels, The absolute value of the passage door. This is a mapping relationship; Based on the detection loss and sparsity loss, the calculation formula for optimizing the target detection model is as follows: in, For the updated network connection parameters, To minimize the solution algorithm, The network connection parameters before the update. To detect the loss, This is a sparse loss.

[0045] It should be noted that the index of the labeled target is an identifier used to uniquely identify each labeled real target in the dataset, with a value ranging from 0 to the total number of labeled targets in the dataset minus 1. The ground truth bounding box of the target is a positional supervision signal used for model training. It can be obtained manually using image annotation tools, and the annotation content is the target's true position and size in the image, in the format of the top-left x-coordinate, top-left y-coordinate, width, and height. The ground truth class of the target is a classification supervision signal used for model training. It can also be obtained manually using image annotation tools, and the annotation content is the target's corresponding class index, completely consistent with the preset target class definition. The sample weight is a value obtained by adding 1 to the average response result of the highlight feature map values ​​within the labeled bounding box. It is used to weight training samples of different difficulties, improving the loss contribution of hard-to-detect samples.

[0046] It should be noted that the candidate boxes predicted by the model are the predicted candidate boxes output by the detection head, and are the objects of calculation for the bounding box regression loss. The class predicted by the model is the predicted class output by the detection head. The cross-entropy loss calculation rule is used to quantify the difference between the predicted class and the true class. The true target label is a supervision signal used for training the target loss, taking a value of 0 or 1. It takes a value of 1 when the detection location corresponds to the center region of the true target, and a value of 0 otherwise. The model output targetness is the initial score output by the detection head. The detection loss function value is a multi-task loss value composed of the sample-weighted bounding box regression loss, classification loss, and targetness binary cross-entropy loss. The neural network channel index is an identifier used to uniquely identify each convolutional channel in the neural network, with a value ranging from 0 to the total number of channels in the network minus 1. The channel feature activation is the value of all pixels in the feature map output by the corresponding channel of the neural network. The region mean of the absolute value of the specular feature map is the arithmetic mean of the absolute values ​​of the feature map within the corresponding feature map region. The correlation between channel activation and feature mapping is a value calculated based on the channel feature activation and the mean of the specular feature map. It characterizes the contribution of the channel to the detection of reflective micro-components; a higher value indicates a greater contribution. The channel gate coefficient is a learnable coefficient applied to the corresponding channel, ranging from 0 to 1. After training, the closer the coefficient is to 0, the smaller the contribution of the channel, and it can be pruned or deleted.

[0047] It should be noted that the sparsity intensity coefficient is a hyperparameter used to control the weights of the sparse loss, balancing the weights of detection loss and sparse loss. This achieves channel sparsity without sacrificing detection accuracy, meeting the needs of lightweight models. The channel sparse loss function value is a sparse loss value calculated based on the channel gate coefficients and the correlation between channel features, used to guide the network to achieve feature-constrained channel sparsity. The network connection parameters before the update include all learnable weights and biases in the neural network. The minimization algorithm is an iterative algorithm used to optimize network parameters, with fast convergence speed and high training stability, suitable for the end-to-end training requirements of lightweight detection networks. The optimal network connection parameters after training and optimization are the network parameters obtained after minimizing the total loss function, including all learnable parameters such as weights, biases, channel gate coefficients, and scaling coefficients of all convolutional layers. The class output confidence threshold is a class-specific confidence threshold determined based on the detection accuracy metric of the validation set.

[0048] It should be noted that the specific formula for calculating the Complete Intersection over Union (CUI) loss is: CUI loss equals 1 minus the CUI value, where the CUI value equals the CUI value minus the center point distance penalty term minus the aspect ratio penalty term. The CUI value is the area of ​​the intersection of the predicted and ground truth boxes divided by the area of ​​their union. The center point distance penalty term is the square of the Euclidean distance between the center points of the predicted and ground truth boxes, divided by the square of the diagonal length of the smallest bounding rectangle of the two boxes. The aspect ratio penalty term is the weight coefficient multiplied by the aspect ratio consistency parameter, with the weight coefficient preferably set to 0.5. The specific formula for calculating the cross-entropy loss is: Cross-entropy loss equals the logarithm of the predicted probability corresponding to the negative ground truth class, used for loss calculation in multi-class classification tasks. The specific formula for calculating the binary cross-entropy loss is: Binary cross-entropy loss equals the logarithm of the predicted probability multiplied by the negative ground truth label, minus the negative 1 minus the logarithm of the predicted probability multiplied by 1, used for loss calculation in binary classification tasks.

[0049] It should be noted that the sparsity intensity coefficient ranges from 10 to the power of -5 to 10 to the power of -3, with a preferred value of 10 to the power of -4. During training, the sparsity intensity coefficient can be adjusted using cosine decay. A larger coefficient is used in the early stages of training to quickly achieve channel sparsity, while a smaller coefficient is used in the later stages of training to ensure that the detection accuracy does not decrease. This will not be elaborated further here.

[0050] It should be noted that the specific type of minimization algorithm is the adaptive moment estimation optimization algorithm. The key hyperparameters of this algorithm are: an initial learning rate of 10^-3, a weight decay coefficient of 10^-4, an exponential decay rate of 0.9 for the first-order moment estimation, an exponential decay rate of 0.999 for the second-order moment estimation, and a numerical stability constant of 10^-8. The training batch size is set to 16, and the total number of training epochs is set to 30. The first 5 epochs are warm-up training, where the learning rate linearly increases from 10^-5 to 10^-3. The following 25 epochs are joint optimization training, where the learning rate decreases from 10^-3 to 10^-5 using cosine decay. In practice, if the dataset is large, the training batch size can be increased to 32 to improve training stability; if the dataset is small, the training batch size can be reduced to 8 to avoid overfitting.

[0051] It should be noted that the channel gate coefficients are initialized using constant initialization, with all channels initially set to 1, and the value range constrained to be greater than or equal to 0 and less than or equal to 1. During model training, the backpropagation algorithm and the adaptive moment estimation optimizer are used for end-to-end joint optimization and updates. This initialization method ensures that all channels are in the open state at the beginning of training, allowing the network to fully learn all feature information. As training progresses, the gate coefficients of channels with low correlation to features are gradually compressed to near 0, while the gate coefficients of channels with high correlation to features remain close to 1, achieving adaptive sparsity of channels.

[0052] It should be noted that the method for determining the category output confidence threshold is as follows: based on the detection results of the validation set, the threshold for each category is determined using the criterion of highest average precision. In practice, after the model training is completed, inference is performed using the validation set data. For each category, different thresholds in the range of 0.01 to 0.99 are tested, and the average precision index corresponding to each threshold is calculated. The threshold with the highest average precision is selected as the final output confidence threshold for that category. The specific preferred values ​​for each category are as follows: screw category threshold is 0.5, solder pad category threshold is 0.5, solder point category threshold is 0.4, shielding cover pressure point category threshold is 0.45, interface metal edge category threshold is 0.5, camera module frame category threshold is 0.55, glass fragment category threshold is 0.4, and motherboard defect category threshold is 0.35. In practice, the threshold for each category can be flexibly adjusted according to the accuracy and recall requirements of the detection scenario.

[0053] It should be noted that the dataset partitioning rule for model training is as follows: the collected maintenance scene image dataset is randomly divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The training set is used for model parameter optimization, the validation set is used for threshold determination and hyperparameter tuning, and the test set is used for final model performance evaluation. Data augmentation methods used during training include random horizontal flipping, random vertical flipping, random rotation, random brightness adjustment, random contrast adjustment, random Gaussian blur, and random mosaic enhancement. The execution probability of all augmentation methods is set to 0.5. Data augmentation improves the model's generalization ability and adapts to detection requirements under different light sources, angles, and exposure conditions.

[0054] In one embodiment of the present invention, the calculation process of step S7 specifically includes: The formula for calculating the final score, based on the corrected score and the intersection-union ratio, is as follows: in, For the final score, This is the corrected score for the current candidate box. For consecutive multiplication, This is a filtering condition, representing the filtering of overlapping candidate boxes that have the same predicted category and whose corrected score is greater than the current candidate box. and For predicting categories, The current candidate box, For overlapping candidate boxes, For exponential function operations, The intersection-union ratio is the numerical value; The formula for calculating the final detection result set based on the final score and category threshold is as follows: in, This is the final set of test results. For the target category, The x-coordinate of the center point The ordinate of the center point is... For width dimensions, For height dimensions, , , and Forming the target location, For confidence level, This is the category threshold.

[0055] It should be noted that the candidate box intersection-union ratio (OCR) is used to quantify the degree of overlap between two candidate boxes, and is preferably taken as the intersection area divided by the union area. The final score after overlap attenuation is the corrected score of the current candidate box, which is multiplied by the suppression attenuation coefficients that meet the screening criteria to obtain the final confidence score, used to characterize the final credibility of the candidate box. The confidence threshold for the corresponding category is used to filter candidate boxes with low confidence to ensure the accuracy of the final detection results. The final detection result set is a set of all candidate boxes whose final scores are greater than or equal to the confidence threshold of the corresponding category, including the category, location, size, and confidence information of each target. The x-coordinate of the target center point is the coordinate value of the candidate box center point on the horizontal axis of the image. The y-coordinate of the target center point is the coordinate value of the candidate box center point on the vertical axis of the image. The target width is the length of the candidate box on the horizontal axis, in pixels. The target height is the length of the candidate box on the vertical axis, in pixels.

[0056] It should be noted that the specific formula for calculating the Intersection over Union (IoU) ratio is as follows: the IoU value of two candidate boxes is equal to the area of ​​their intersection region divided by the area of ​​their union region. The intersection region is the rectangular area where the two candidate boxes overlap, and its area is the width multiplied by its height. If the two candidate boxes do not overlap, the intersection area is 0, and the IoU value is 0. If the two candidate boxes completely overlap, the intersection area equals the union area, and the IoU value is 1. This value quantifies the degree of spatial overlap between the two candidate boxes; a higher value indicates a more severe overlap.

[0057] It should be noted that the coordinate system of the final detection result is defined as the image pixel coordinate system. The origin of the coordinate system is the top left corner of the image, the horizontal axis is positive to the right, and the vertical axis is positive downwards. Each unit corresponds to one pixel in the image. The horizontal and vertical coordinates of the target center point are the pixel coordinates of the center point of the candidate bounding box in this coordinate system, and the target width and height are the pixel lengths of the candidate bounding box on the horizontal and vertical coordinate axes, respectively.

[0058] It should be noted that the filtering rule for the category threshold is as follows: for each candidate box, a specific confidence threshold corresponding to its predicted category is used for filtering. Only candidate boxes with a final score greater than or equal to the corresponding category threshold are retained, while candidate boxes with a final score less than the corresponding category threshold are filtered out. The execution order of the filtering operation is as follows: first, overlap attenuation calculation is performed on all candidate boxes to obtain the final score of each candidate box; then, the candidate boxes are grouped according to the predicted category, and the candidate boxes in each group are filtered using the specific threshold of the corresponding category; finally, all candidate boxes that pass the filtering form the final detection result set. This execution order ensures that the overlap attenuation calculation uses complete candidate box information, avoiding inaccurate attenuation calculations caused by premature filtering.

[0059] It should be noted that traditional non-maximum suppression uses hard deletion, directly removing all candidate boxes whose overlap with the highest-scoring candidate box exceeds a threshold. Continuous exponential decay, on the other hand, uses soft suppression, reducing the score of overlapping candidate boxes by continuously multiplying the decay coefficient. It only filters low-scoring candidate boxes through a final threshold, without directly deleting any candidate boxes. Furthermore, the specific execution steps of continuous exponential decay include: Step S101: Sort the candidate boxes for each category in descending order according to their corrected scores; Step S102: Initialize the final score of all candidate boxes to equal their corrected scores; Step S103: Starting from the candidate box with the highest score, traverse each candidate box in turn. For the candidate box currently being traversed, calculate the intersection-union ratio of all candidate boxes of the same type ranked before it and generate the corresponding exponential decay coefficient. Multiply the final score of the current candidate box by all decay coefficients continuously to update its final score. Step S104: After the traversal is complete, use the category threshold to filter all candidate boxes and generate the final detection result set.

[0060] This method can avoid the missed detection of adjacent targets caused by traditional nonmaximum suppression, and effectively reduce the repeated detection of the same target, which will not be elaborated here.

[0061] Specifically, the actual deployment of this invention can be divided into three stages: data acquisition, model training, and model inference. In the data acquisition stage, images can be acquired using three types of devices: a mobile phone main camera, an industrial area scan camera, and a specialized repair microscope lens. During acquisition, the vertical distance between the lens and the repair table plane must be between 10 cm and 50 cm, and the resolution of the acquired images must be no less than 1920×1080. The acquisition process covers three common light sources: a ring repair light, a mobile phone flash, and a microscope stage light source, as well as common repair objects such as motherboards, screens, and camera modules of different brands of mobile phones, laptops, and tablets. After acquisition, the target locations and categories are labeled using annotation tools to construct a complete training dataset. In the model training stage, training can be completed on a desktop computer or cloud server. A mainstream graphics processor is sufficient for end-to-end training. After training, a lightweight model is exported, with the model file size controlled to within 5MB. This model can be deployed to Android and iOS mobile phones or tablets, as well as embedded repair terminals, supporting offline inference without relying on a network connection. During the model inference phase, the deployed equipment acquires images of the maintenance station in real time via a camera. After inputting the detection process of this invention, it can complete the full-process calculation of a single frame image within 30 milliseconds and output the final detection result.

[0062] The final output of this invention is a standardized quantitative detection set. Each detection result contains four core pieces of information: target category, target location, target size, and confidence level. For example, for mobile phone motherboard repair images, the final output may include: target category: screw; target location: x-coordinate 1240, y-coordinate 860; target width: 36; height: 34; confidence level: 0.91; target category: solder pad anomaly; target location: x-coordinate 820, y-coordinate 1410; target width: 18; height: 12; confidence level: 0.78; target category: glass fragment; target location: x-coordinate 2100, y-coordinate 960; target width: 52; height: 48; confidence level: 0.85.

[0063] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0064] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A lightweight method for image recognition and object detection based on deep learning, characterized in that, Includes the following steps: Step S1: Acquire the image to be inspected and extract the normalized brightness, gradient magnitude and gradient direction of the image to be inspected; Step S2: Calculate the highlight weight and highlight centroid based on the normalized brightness. Determine the radial and tangential directions from the highlight centroid. Combine the highlight weight, gradient magnitude, and gradient direction to obtain the radial and tangential energy along the radial and tangential directions. Generate the highlight feature map from the radial and tangential energy. Step S3: Input the image to be inspected and the highlight feature map into the backbone network to extract backbone features. Generate a spatial gating map from the highlight feature map. Modulate the backbone features with the spatial gating map to obtain enhanced backbone features. Step S4: The enhanced backbone features and the specular feature map are input into the fusion network and multi-scale features are obtained by spatial gating map modulation. Step S5: Input the multi-scale feature into the detection network to generate candidate boxes, class probabilities and initial scores, and use the value of the highlight feature map in the candidate box to correct the initial score to obtain the corrected score; Step S6: Set sample weights based on the values ​​of the highlight feature maps within the labeled boxes, construct a detection loss by combining candidate boxes and class probabilities, generate a sparse loss based on the mapping relationship between channel activation and highlight feature map values, optimize the target detection model by combining the detection loss and the sparse loss, and extract the class threshold. Step S7: Perform decay calculation on the corrected scores of overlapping candidate boxes of the same type to obtain the final score, extract candidate boxes whose final scores are not less than the category threshold, and output the target category, target location and confidence level.

2. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, Extract the pixel values ​​of the red, green, and blue channels of the image to be inspected. Then multiply the pixel values ​​of the red, green, and blue channels by the brightness conversion factor and sum them up to obtain the brightness value. Calculate the average brightness of the image to be inspected, then divide the brightness value by the sum of the average brightness of the image to be inspected and the smallest positive number, add one to the result of the division operation, and finally perform a logarithmic function operation to obtain the normalized brightness. Calculate the horizontal and vertical luminance differences of the normalized luminance, and then combine the horizontal and vertical luminance differences to obtain the luminance gradient vector. Calculate the L2 norm of the brightness gradient vector to obtain the gradient magnitude; The gradient direction is obtained by dividing the brightness gradient vector by the sum of the gradient magnitude and the smallest positive number.

3. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, Calculate the difference between the normalized brightness and the mean brightness of the local window, then divide the difference by the sum of the standard deviation of the local window brightness and the smallest positive number, and perform the activation function operation to obtain the highlight weight. Multiply the highlight weight by the pixel coordinates and perform a summation operation, then divide by the sum of the highlight weight within the local window and the sum of the smallest positive number to obtain the highlight centroid; Subtract the specular centroid from the pixel coordinates to obtain the position difference. Then divide the position difference by the sum of the L2 norm of the position difference and the smallest positive number to obtain the radial direction. Extract the transverse and longitudinal components of the radial direction, then take the opposite value of the longitudinal component as the transverse coordinate parameter of the tangential direction, and take the transverse component as the ordinate parameter of the tangential direction. Finally, combine the transverse and ordinate parameters to obtain the tangential direction.

4. The lightweight method for image recognition and target detection based on deep learning according to claim 3, characterized in that, Calculate the absolute value of the dot product of the gradient direction and the radial direction, then multiply the highlight weight, gradient magnitude and absolute value consecutively and sum them within a local window to obtain the radial energy; The weight difference is obtained by subtracting the highlight weight from one, and the absolute value of the dot product of the gradient direction and the tangential direction is calculated. The weight difference, gradient magnitude and absolute value are then multiplied continuously and summed within a local window to obtain the tangential energy. Calculate the energy difference between radial energy and tangential energy, and calculate the sum of radial energy, tangential energy, and the smallest positive number. Then divide the energy difference by the sum of energy and perform a linear rectification function operation to obtain the rectified value. Calculate the L2 norm of the difference between the center pixel and the specular centroid, then divide it by the sum of the feature step size and the smallest positive number to obtain the distance ratio value. Finally, multiply the rectified value, the specular weight of the center pixel, and the distance ratio value consecutively to obtain the specular feature map.

5. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, The image to be inspected is concatenated with the first-scale spectrophotometric feature map through channels, and then spatial convolution is performed to obtain the initial features of the network. The first point convolution operation, the depthwise convolution operation, and the second point convolution operation are performed sequentially on the input features of the previous layer to obtain the backbone features; The highlight feature map is downsampled according to the feature step size to obtain the downsampled feature map. The downsampled feature map is then concatenated with the backbone feature map to obtain the concatenated result. Gated point convolution operations are performed on the stitched results, and spatial gated graphs are generated using activation functions. Multiply the spatial gating map by the scaling factor and add one to the result to obtain the gating modulation parameters. Then, perform element-wise multiplication of the backbone features with the gating modulation parameters to obtain the modulation features. Size matching projection is performed on the input features of the previous layer to obtain projection parameters. Then, the modulation features are added to the projection parameters to obtain the enhanced backbone features.

6. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, Point convolution is performed on the deepest enhanced backbone features to obtain deep semantic features, and upsampling is performed on the multi-scale features of the deepest layer to obtain upsampled fused features. The specular feature map is downsampled according to the feature step size to obtain the downsampled specular features. The enhanced backbone features, upsampled fusion features and downsampled specular features are concatenated into channels, and then point convolution is performed to obtain the concatenated result. Add one to the spatial gating map, and then perform element-wise multiplication with the stitched result to obtain multi-scale features.

7. The lightweight method for image recognition and object detection based on deep learning according to claim 1, characterized in that, The prediction vector is composed of the horizontal offset parameter of the center point, the vertical offset parameter of the center point, the logarithmic parameter of the target width, the logarithmic parameter of the target height, the initial score and the category logical parameter. Activation functions are applied to the horizontal and vertical offset parameters of the center point. The results are then added to the horizontal and vertical grid positions, respectively, and multiplied by the feature step size to obtain the horizontal and vertical coordinates of the candidate box center point. Perform an exponential function operation on the logarithmic parameters of the target width and the target height, and then multiply by the feature step size to obtain the candidate box width and candidate box height. By concatenating the x-coordinate of the candidate box center point, the y-coordinate of the candidate box center point, the candidate box width, and the candidate box height, the candidate box is obtained. The maximum value of the category logical parameter is extracted to obtain the predicted category. Then, the normalized exponential function is performed on the category logical parameter to obtain the category probability. The arithmetic mean of the highlight feature map values ​​within the candidate box is calculated. Then, the initial score is added to the arithmetic mean and the activation function is applied. Finally, the score is multiplied by the class probability to obtain the corrected score.

8. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, Extract the average response of the highlight feature map values ​​within the labeled box, and then add one to the average response to obtain the sample weight; Calculate the intersection-union ratio (IUU) between the labeled bounding box and the candidate bounding box, and then subtract the IUU from the IUU to obtain the localization regression decision value. Calculate the cross-entropy between the true class and the predicted class to obtain the classification misclassification value; The localization regression judgment value and the classification misclassification judgment value are added together and multiplied by the sample weight. Then, the summation is performed on all targets, and the result of the summation of the binary cross-entropy loss of the true target identity and the output target identity is added to obtain the detection loss.

9. The lightweight method for image recognition and target detection based on deep learning according to claim 8, characterized in that, Calculate the product of the sum of the absolute values ​​of channel activations and the mean of the absolute values ​​of the specular feature map, and then divide the product by the sum of the sum of the absolute values ​​of channel activations and the smallest positive number to obtain the mapping relationship; Divide the mapping relationship by the sum of the mapping relationship plus one, and then subtract the quotient from the division by one to obtain the retention factor; Multiply the absolute value of the channel gate by the retention factor and sum them over all channel levels, then multiply by the sparsity strength coefficient to obtain the sparsity loss; The detection loss and sparsity loss are added together, and then the minimization algorithm is called to process the network connection parameters before the update to obtain the updated network connection parameters, thus completing the optimization of the target detection model and extracting the category threshold.

10. The lightweight method for image recognition and target detection based on deep learning according to claim 1, characterized in that, Based on the filtering criteria, filter overlapping candidate boxes that have the same predicted category and whose corrected scores are greater than the current candidate box, and calculate the intersection-union ratio (IUU) between the current candidate box and the overlapping candidate boxes. Calculate the square of the intersection-union ratio, take the opposite of the square, and then perform an exponential function operation to generate the suppression attenuation coefficient; The final score is obtained by multiplying the corrected score of the current candidate box with the suppression decay coefficient. Candidate boxes with final scores greater than or equal to the category threshold are selected to form the final detection result set; Extract the predicted category of the candidate box as the target category, then merge the x-coordinate, y-coordinate, width and height of the candidate box's center point as the target location, and use the final score as the confidence score. Output the target category, target location and confidence score.

Images