A machine object image detection method and system for industrial warehousing
By generating feature maps through convolution and downsampling in industrial warehousing scenarios, combining attention and self-attention mechanisms for feature fusion, and using adaptive weight allocation and branch detection, the problems of difficult feature extraction for small-sized targets and easy confusion of highly similar parts are solved, thereby improving detection accuracy and precision.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG UNIV OF SCI & TECH
- Filing Date
- 2026-06-12
- Publication Date
- 2026-07-14
AI Technical Summary
In industrial warehousing scenarios, there are problems such as difficulty in extracting features from small-sized targets, easy confusion of highly similar parts, and mutual occlusion of densely stacked targets leading to missed detections and false detections.
Shallow feature maps are generated by convolution and downsampling. Horizontal and vertical attention weights are combined to generate coordinate attention-enhanced features, which are then fused in parallel with multi-head self-attention features. Multi-scale weighted fusion is performed by adaptively optimizing weight allocation. Classification and regression detection are performed by combining one-to-one and one-to-many branches. A residual module is embedded to improve detection accuracy.
It significantly improves the feature discrimination of densely stacked targets, enhances the feature extraction capability of small-sized, highly similar and slender machine objects, reduces missed detections and false detections, and improves detection accuracy.
Smart Images
Figure CN122391764A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image data processing technology, specifically relating to a method and system for detecting machine objects in industrial warehousing. Background Technology
[0002] With the accelerated advancement of industrial intelligent transformation, warehouse management, as a core link in the manufacturing supply chain, directly impacts a company's production efficiency and cost control through its level of automation and precision. Machine parts (such as bolts, nuts, washers, and clamps) are fundamental materials for mechanical assembly and maintenance, characterized by their diverse types, similar shapes, and large size range. Currently, most industrial warehouses still rely on manual visual identification and counting for the entry, inventory, and issuance of machine parts. This model has significant drawbacks: firstly, prolonged repetitive labor easily leads to visual fatigue, causing miscounting and omissions. Secondly, differences in experience among personnel result in discrepancies between inventory data and the actual situation. According to industry survey data, the inventory accuracy rate under traditional warehouse management models is typically only around 85%, while production line losses due to missing materials or incorrect material issuance can account for 2% to 5% of a factory's total output value annually. This extensive management model, reliant on manual labor, can no longer meet the stringent requirements of intelligent manufacturing for accurate material traceability and efficient flow.
[0003] In industrial applications, barcode scanning or RFID technology is commonly used for material management to address the low efficiency and error-prone nature of manual identification of machine parts. While these methods can improve the automation level of data collection and reduce manual entry errors to some extent, they still have many limitations in actual warehousing scenarios and are difficult to adapt to the management needs of multi-category, high-frequency, and small-sized machine parts. Specifically, barcode labels are easily worn and detached in the warehousing environment, and require manual scanning one by one, making it difficult to handle high-frequency, multi-category inbound and outbound operations. In addition, RFID tags are easily affected by shielding and reflection interference in environments with metal machine parts, resulting in poor recognition stability. Coupled with high deployment costs, they are difficult to cover unlabeled, multi-category, and small-sized machine parts. Especially when dealing with metal bolts, nuts, washers, and other machine parts, the difficulty of image acquisition and recognition increases significantly. Machine parts in warehousing scenarios not only have a large size range and highly similar shapes, but also often exhibit characteristics such as dense stacking, mutual obstruction, and arbitrary postures in the warehousing environment. Furthermore, the reflective effect of metal surfaces exacerbates the non-uniformity of image illumination, making it difficult for traditional visual algorithms to reliably extract effective features and limiting the accuracy of subsequent detection and counting.
[0004] In summary, the common problems in the inspection of machine parts in industrial warehousing scenarios include difficulty in extracting features of small-sized targets, easy confusion of highly similar parts, and missed detections and false detections caused by mutual occlusion of densely stacked targets. Summary of the Invention
[0005] The purpose of this invention is to provide a method and system for detecting machine objects in industrial warehousing, in order to solve the problems of difficulty in extracting features of small-sized targets, easy confusion of highly similar parts, and missed detection and false detection caused by mutual occlusion of densely stacked targets in industrial warehousing scenarios.
[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A method for detecting machine parts in industrial warehousing includes the following steps: The machine object image is convolutionally and downsampled to obtain a shallow feature map. The shallow feature map is then averaged along the horizontal and vertical directions, and then convolutionally, normalized, and activated to obtain horizontal and vertical attention weights. These weights are then multiplied element-wise with the shallow feature map to obtain coordinate attention enhancement features. Query, key, and value are generated from the shallow feature map, and after being evenly divided along the feature dimensions, multi-head self-attention features are obtained. These features are then fused in parallel with the coordinate attention enhancement features to obtain shallow hybrid attention features. The shallow feature map is further convolved and downsampled to obtain the high-level feature map; the high-level feature map is then processed using the same steps as the shallow feature map to obtain the high-level hybrid attention feature. Shallow and high-level hybrid attention features are divided into subsets along the channel dimension and assigned learnable weights. The weight allocation is adaptively optimized based on the backpropagation algorithm. After weighted fusion of the features of each subset, multi-scale weighted fusion features are obtained. The multi-scale weighted fusion features are input into the detection head, and then enter into one-to-one and one-to-many branches for classification and regression detection. The residual module is embedded in the one-to-one branch, and the detection results of the machine object are output.
[0007] Preferred, adaptive optimization of weight allocation: Learnable weights are assigned to each sub-feature map and normalized to obtain learnable weight scalars; the current detection error is calculated based on the loss function to obtain the gradient information of the loss with respect to each weight scalar; based on the gradient information, the gradient of each learnable weight is further calculated, and then each learnable weight is iteratively updated according to the gradient descent strategy.
[0008] Preferred types of branches: one-to-one and one-to-many branches Multi-scale weighted fusion features are used as input feature vectors and processed in parallel through one-to-many and one-to-one branches. The one-to-many branch is processed by regular convolution, batch normalization and activation. The one-to-one branch is processed by depthwise separable convolution, feature enhancement and residual module in sequence to output accurate classification results.
[0009] Preferably, residual modules are embedded in one-to-one branches: The residual module directly passes the input features to the output, and is cascaded with depthwise separable convolutions with one-to-one branches and 1×1 convolutions to further enhance the features.
[0010] Preferred features of multi-head self-attention include: By uniformly dividing the query, key, and value into several parts along the feature dimension, and performing dot product and normalization on each part based on the corresponding query, key, and value, the corresponding single-head self-attention features are obtained. After concatenating multiple single-head self-attention features, a linear transformation is performed to obtain multi-head self-attention features.
[0011] Preferably, parallel fusion includes: The multi-head self-attention features and coordinate attention enhancement features are dimensionally aligned, and then element-wise weighted fusion, convolution, and normalization are performed sequentially.
[0012] Preferably, the horizontal attention weights and vertical attention weights include: The horizontal and vertical attention weights are expanded in dimension to match the spatial size of the shallow feature map. The expanded horizontal and vertical attention weights are then added element by element and fused together, and finally weighted with the shallow feature map to obtain the coordinate attention-enhanced feature.
[0013] Preferred, detection head: A non-nonmaximum suppression detection head is used, and the detection results are output in parallel through one-to-one branches and one-to-many branches.
[0014] An image-based inspection system for machine parts in industrial warehousing includes: The shallow hybrid attention module performs convolution and downsampling on the object image to obtain a shallow feature map. The shallow feature map is then subjected to average pooling along both the horizontal and vertical directions, followed by convolution, normalization, and activation to obtain horizontal and vertical attention weights. These weights are then multiplied element-wise with the shallow feature map to obtain coordinate attention enhancement features. A query, key, and value are generated from the shallow feature map, and after being evenly divided along the feature dimensions, multi-head self-attention features are obtained. These features are then fused in parallel with the coordinate attention enhancement features to obtain the shallow hybrid attention features. The high-level hybrid attention module further convolves and downsamples the shallow feature map to obtain a high-level feature map; the high-level feature map is then processed using the same steps as the shallow feature map in the shallow hybrid attention module to obtain the high-level hybrid attention features. The weighted fusion module divides the shallow and high-level mixed attention features into subsets along the channel dimension and assigns learnable weights. It adaptively optimizes the weight allocation based on the backpropagation algorithm and obtains multi-scale weighted fusion features by weighted fusion of the features of each subset. The detection output module inputs multi-scale weighted fusion features into the detection head, and simultaneously performs classification and regression detection in one-to-one and one-to-many branches. The residual module is embedded in the one-to-one branch, and the detection results of the machine object are output.
[0015] Preferably, the weighted fusion module includes: The adaptive optimization weight allocation module assigns learnable weights to each sub-feature map and normalizes them to obtain learnable weight scalars; it calculates the current detection error based on the loss function to obtain the gradient information of the loss with respect to each weight scalar; based on the gradient information, it further calculates the gradient of each learnable weight, and then iteratively updates each learnable weight according to the gradient descent strategy.
[0016] Compared with the prior art, the technical solution provided by this invention has the following advantages: By assigning learnable weight parameters to different channels, the network can adaptively adjust the weight distribution of multi-scale features according to the characteristics of the machine objects, thereby enhancing the feature discrimination of densely stacked targets.
[0017] By fusing coordinate attention with multi-head self-attention, the interference of redundant background information is effectively suppressed, enhancing the feature extraction capability for small-sized, highly similar, and slender machine objects in industrial warehousing scenarios, thereby significantly improving the detection accuracy of highly similar parts.
[0018] By setting up one-to-one and one-to-many branches for classification and regression detection, the ability to classify and distinguish machine objects is further improved, while reducing the phenomenon of false prediction boxes, improving the detection accuracy of highly similar machine objects, and significantly improving the problem of missed detection in densely stacked scenarios. Attached Figure Description
[0019] Figure 1 The detection results are based on the benchmark algorithm. Figure 2 The detection results are those of the improved baseline algorithm in this scheme; Figure 3 This is to compare the detection results of this method with those of other methods. Detailed Implementation
[0020] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments.
[0021] A method for detecting machine objects in industrial warehousing includes: S1. Perform convolution and downsampling on the machine object image to obtain a shallow feature map; perform average pooling on the shallow feature map along the horizontal and vertical directions respectively, and then perform convolution, normalization and activation processing in sequence to obtain horizontal attention weights and vertical attention weights. Multiply them element-wise with the shallow feature map to obtain coordinate attention enhancement features; generate query, key and value from the shallow feature map, divide them evenly along the feature dimension to obtain multi-head self-attention features, and fuse them in parallel with the coordinate attention enhancement features to obtain shallow hybrid attention features.
[0022] Warehouse object image detection is the process of locating and classifying machine objects in an input image. During the detection process, features at different spatial locations contain different information, such as edge details of small parts and subtle differences between highly similar parts. This differentiated information is crucial for accurate target detection.
[0023] To achieve accurate detection results, a coordinate attention method is used to adaptively enhance sensitivity to the spatial location of the target. This results in higher weights for spatial regions that contribute significantly to localization, while background regions receive lower weights. The coordinate attention method automatically learns feature weights in both the horizontal and vertical directions based on the feature information of each spatial location, and adaptively adjusts the information at different locations according to these weights. This method can accurately capture the edge contour details of small, slender objects, effectively enhancing the model's ability to locate such targets and avoiding detection biases caused by small target size or similar shapes.
[0024] Specifically, convolution and downsampling are performed on the machine object image to obtain a shallow feature map. First, vertical pooling is performed to obtain the vertical pooling feature. Horizontal pooling features are obtained by pooling along the horizontal direction. : , , in, Represents the spatial height of the shallow feature map. This represents the spatial width of the shallow feature map. This represents the channel dimension of the shallow feature map. This indicates that all elements are real numbers.
[0025] pass and Obtain the horizontal attention weights and vertical attention weights.
[0026] Preferably, the horizontal attention weights and vertical attention weights include: The horizontal and vertical attention weights are expanded in dimension to match the spatial size of the shallow feature map. The expanded horizontal and vertical attention weights are then added element by element and fused together, and finally weighted with the shallow feature map to obtain the coordinate attention-enhanced feature.
[0027] Will and Horizontal attention weights are generated after convolutional transformation and batch normalization, followed by sigmoid activation. With vertical attention weights : , in, It is the sigmoid activation function. This indicates a batch normalization operation. This indicates a convolution operation.
[0028] The horizontal and vertical attention weights are extended to the shallow feature maps, respectively. Using the same spatial dimensions, element-wise summation is performed to obtain coordinate attention features. Specifically, shallow feature maps... The shape is Horizontal attention weights The shape is In order to enable it to be with To align, you need to copy it along its width. The portion, expanded to The tensor of vertical attention weights. The shape is Copy it along the height direction The portion, expanded to The attention tensor is then expanded. Subsequently, the corresponding positions in the expanded attention tensor are summed element-wise to obtain the coordinate attention features. .
[0029] Integrating coordinate attention features with shallow feature maps Element-wise multiplication followed by coordinate attention injection yields coordinate attention-enhanced features. : , By injecting coordinate attention, spatial location features in the horizontal and vertical directions are fused, thereby enhancing the coordinate attention features. It has accurate orientation information to improve the ability to distinguish highly similar, slender machine objects in subsequent steps.
[0030] In warehouse object inspection, due to the random and disordered placement of targets and the high degree of similarity in appearance between similar objects, it is required that global feature information be fully correlated in image detection to accurately uncover the feature correlations and subtle differences between targets in different regions of the image. Traditional convolution operations are limited to local feature information and cannot effectively take into account the feature correlations of a large area of the image, which easily leads to incomplete feature extraction and insufficient representation capabilities.
[0031] To enhance the ability to capture global information, this invention employs multi-head self-attention for feature extraction, replacing the traditional 3×3 convolution operation. The multi-head self-attention mechanism adaptively calculates the correlation between any two locations in the feature map, effectively focusing on key features within the feature map's scope.
[0032] Preferred features of multi-head self-attention include: By uniformly dividing the query, key, and value into several parts along the feature dimension, and performing dot product and normalization on each part based on the corresponding query, key, and value, the corresponding single-head self-attention features are obtained. After concatenating multiple single-head self-attention features, a linear transformation is performed to obtain multi-head self-attention features.
[0033] Input shallow feature map The query feature Q, key feature K, and value feature V are obtained through linear transformation: , in, This is the learnable weight matrix used to generate Q, K, and V.
[0034] Then Q, K, and V are uniformly divided into i parts along the feature dimension, represented as: , Among them, i , The total number of bullish self-attention. Represents the partitioning function. Indicates the first division A separate query sub-feature, Indicates the first division A unique key feature Indicates the first division Each independent value sub-feature.
[0035] Calculate the single-head self-attention features separately for each part. , represented as: , in, The dimension of the key feature. express The transpose of .
[0036] Then, multiple single-head self-attention features are concatenated and subjected to a linear transformation to obtain multi-head self-attention features. , represented as: , in, The output projection weight matrix maps the concatenated multi-head self-attention features to the shallow features. Figure 1 Channel Dimensions This ensures dimensional consistency in subsequent feature processing.
[0037] Multi-head self-attention features and coordinate attention enhancement features are fused in parallel to obtain shallow hybrid attention features. , .
[0038] Preferably, parallel fusion includes: The multi-head self-attention features and coordinate attention enhancement features are dimensionally aligned, and then element-wise weighted fusion, convolution, and normalization are performed sequentially.
[0039] The multi-head self-attention features and coordinate attention-enhanced features are dimensionally aligned, and then element-wise weighted fusion, convolution, and normalization operations are performed sequentially on the multi-head self-attention features and coordinate attention-enhanced features. This approach can ensure the global dependency capture capability of multi-head self-attention while taking into account the spatial location sensitivity of coordinate attention-enhanced features, thus achieving effective fusion of global and spatial location features.
[0040] S2. Continue convolution and downsampling on the shallow feature map to obtain the high-level feature map; apply the processing steps for the shallow feature map in S1 to the high-level feature map to obtain the high-level hybrid attention features.
[0041] At the same time, the shallow feature map corresponding to the warehouse machine items will be generated. Local features are extracted through multiple convolution operations, and then progressive downsampling is used to continuously compress the feature size and expand the receptive field, ultimately outputting a high-level feature map with strong representational capabilities.
[0042] The high-level feature map is then subjected to coordinate attention enhancement and multi-head self-attention feature extraction in sequence. The coordinate attention enhancement and multi-head self-attention features are then fused in parallel to obtain high-level hybrid attention features. , .
[0043] S3. Divide the shallow mixed attention features and the high-level mixed attention features into subsets along the channel dimension and assign learnable weights. Adaptively optimize the weight allocation based on the backpropagation algorithm. After weighted fusion of the features of each subset, obtain the multi-scale weighted fusion features.
[0044] In traditional feature pyramid networks, multi-scale feature weighted fusion operations typically employ indiscriminate splicing or element-wise addition. These methods indiscriminately retain all channel information, resulting in a large amount of background noise and redundant features entering subsequent networks. Furthermore, the fixed fusion method cannot adjust the contribution weights of each source feature according to the dynamic changes in the input content, making it difficult to adapt to scenarios in warehouse object detection where targets are small in size and similar in shape.
[0045] To further improve the fusion quality of multi-scale features, this invention proposes a learnable weighted fusion method. The core idea is to gradually regularize the feature information during processing, moving it from a state of random distribution across channels. Features of similar importance are grouped into adjacent channels, and the overall feature vector is then divided into several groups for weighted fusion. By transforming indiscriminate concatenation into fine-grained weighted fusion, fine-grained feature modulation is achieved.
[0046] Shallow blended attention features and high-level hybrid attention features The feature vectors are uniformly divided into n groups, each assigned a learnable parameter, and finally weighted and concatenated together. Taking n = 2 as an example, the calculation process is as follows: Suppose there are two feature vectors to be fused. and First, they are divided into two subsets along the channel dimension: , in, .
[0047] Subsequently, an independent learnable weight scalar is assigned to each sub-feature map. The sub-feature maps are weighted and modulated: , , , in, This represents the learnable weights, initialized with uniformly distributed random values. This represents the weighted sub-feature map. The weight allocation of different sub-feature maps is automatically iteratively optimized through backpropagation, assigning greater weight to feature vector groups with high feature relevance and reducing the weight of feature vector groups with low feature relevance.
[0048] Preferably, the adaptive optimization weight allocation is as follows: Learnable weights are assigned to each sub-feature map and normalized to obtain learnable weight scalars; the current detection error is calculated based on the loss function to obtain the gradient information of the loss with respect to each weight scalar; based on the gradient information, the gradient of each learnable weight is further calculated, and then each learnable weight is iteratively updated according to the gradient descent strategy.
[0049] Specifically, let the loss function be... , The formula for calculating the backpropagation gradient is: , The parameter update formula is: , in, This represents the learning rate during the backpropagation process.
[0050] Finally, the four weighted sub-features Multi-scale weighted fusion features are output by stitching and fusing along the channel dimension. : , In summary, by designing channel grouping, learnable weight allocation, and channel weighted splicing, the traditional feature fusion method of indiscriminate splicing is transformed into adaptive fine-grained weighted fusion.
[0051] S4. Input the multi-scale weighted fusion features into the detection head, and simultaneously enter the one-to-one branch and the one-to-many branch for classification detection and regression detection. Embed the residual module in the one-to-one branch and output the detection results of the machine object.
[0052] In image detection, the detection head's role is to predict the category, location, and confidence level of objects in the image from the extracted feature maps. The benchmark algorithm uses a lightweight, one-to-one detection head based on YOLOv10 to optimize the classification branch; however, its detection performance is not ideal in tasks involving densely stacked objects and distinguishing highly similar objects, such as... Figure 1 As shown, green boxes represent correct predictions, red boxes represent incorrect predictions, and blue boxes represent unpredicted predictions. This rule applies to all experimental images in this scheme.
[0053] Depend on Figure 1 It can be seen that when detecting images 1, 2, and 4, the baseline algorithm's classification ability was insufficient because the bolts in the images were different in type but highly similar in shape, leading to incorrect detection of some bolt categories. When detecting images 3 and 4, the baseline algorithm classified two bolts that were closely stacked together as if they were only one, and incorrectly suppressed the predicted bounding box of the other bolt, resulting in missed detections.
[0054] Therefore, the baseline algorithm uses a lightweight structure with a one-to-one detection head for classification, which makes it difficult to accurately identify targets with high similarity and incorrectly suppresses the confidence of stacked positive samples. The low confidence of positive samples makes the corresponding predicted boxes easy to be judged as redundant boxes in the post-processing process, thus missing the target.
[0055] Therefore, it is necessary to increase the depth of classification branches and the ability of nonlinear expression on the basis of lightweight detection head to improve the classification and discrimination ability of the detection head.
[0056] However, deep networks can exhibit degradation issues during convergence due to non-overfitting. As the model depth increases, the accuracy may first saturate and then plummet. Adding more layers to a sufficiently deep model can lead to an increase in training error.
[0057] To address the aforementioned issues, this solution employs a deep residual module. Two feature transfer branches are configured: one performs an identity mapping to preserve the original feature information, while the other focuses on learning hidden layer features and residual information that are difficult for shallow networks to fit. However, when stacking non-residual convolutional layers in deep convolutional neural networks, it becomes difficult to fit the stacked structure to an identity mapping form, and parameter degradation, where gradients approach zero, easily occurs. This is the underlying reason why blindly increasing the number of layers in deep networks easily leads to performance degradation, and it is also the core principle behind how the residual module can effectively alleviate network degradation problems.
[0058] Inspired by this, in order to avoid network degradation caused by increasing the number of network layers in the detection head, this invention introduces residual learning in the classification branch of the one-to-one branch of the detection head, which improves the classification and discrimination ability while suppressing the network degradation problem.
[0059] Preferred, detection head: A non-nonmaximum suppression detection head is used, and the detection results are output in parallel through one-to-one branches and one-to-many branches.
[0060] When small parts are stacked in a warehouse, a non-maximum suppression (NMS) detection head will directly delete adjacent overlapping boxes, easily leading to missed detections and false detections. Therefore, this invention uses a non-NMS detection head, inputting multi-scale weighted fusion features into the one-to-many branch and one-to-one branch of the detection head, extracting features and making predictions in parallel, and finally fusing the outputs of the two branches as the overall detection output to ensure detection accuracy.
[0061] Preferred types of branches: one-to-one and one-to-many branches Multi-scale weighted fusion features are used as input feature vectors and processed in parallel through one-to-many and one-to-one branches. The one-to-many branch is processed by regular convolution, batch normalization and activation. The one-to-one branch is processed by depthwise separable convolution, feature enhancement and residual module in sequence to output accurate classification results.
[0062] Preferably, residual modules are embedded in one-to-one branches: The residual module directly passes the input features to the output, and is cascaded with depthwise separable convolutions with one-to-one branches and 1×1 convolutions to further enhance the features.
[0063] Multi-scale weighted fusion features The process then proceeds to either one-to-many or one-to-one branching. The calculation flow for the one-to-one branch can be represented as follows: , , , , , , in, This indicates a depthwise separable convolution with a kernel of 3. BN represents batch normalization, and ResNetBlock() handles residual module processing. The number of categories. For activation function, For a convolution with a kernel of 1, - Characteristics of intermediate processes, Output category confidence for one-to-one branch.
[0064] The calculation process for a one-to-many branch classification can be represented as follows: , , , , , , in, - Characteristics of intermediate processes, Class confidence of the output of a one-to-many branch.
[0065] Will and Weighted fusion is performed to obtain the comprehensive category confidence score, and the final detection result is output based on the comprehensive category confidence score.
[0066] The machine object image detection results based on this scheme are as follows: Figure 2 As shown. With Figure 1 The results show that this scheme still demonstrates high image detection accuracy even when the target similarity is high and the targets are stacked. Meanwhile, the baseline algorithm processes 206.64 frames per second, while this scheme processes 192.02 frames per second, a reduction of only about 7%. This proves that this scheme effectively improves the model's classification and discrimination capabilities without significantly reducing inference speed.
[0067] Furthermore, to verify the robustness of this solution, comparative experiments were conducted on a self-built industrial warehouse machine object inspection dataset, such as... Figure 3 As shown.
[0068] Method 1 uses YOLOv8 and a feature pyramid network to achieve multi-scale feature extraction, but does not consider coordinate attention and multi-head attention mechanisms. Method 2 uses YOLOv10 and a feature pyramid network to achieve feature extraction using multi-head self-attention, and employs one-to-many and one-to-one non-maximum suppression detection heads, but does not consider coordinate attention and multi-head self-attention feature fusion. Method 3, based on Method 1, incorporates coordinate attention to enhance feature perception, but multi-scale feature fusion still uses a simple concatenation method and does not employ an adaptive weighted fusion strategy. This solution simultaneously introduces coordinate attention and multi-head self-attention for feature enhancement and employs a weighted fusion strategy to achieve multi-scale feature aggregation; it also combines one-to-one and one-to-many parallel classification branch structures to efficiently complete deep image feature extraction and complementary fusion, effectively improving detection performance in complex scenes.
[0069] Depend on Figure 3 As can be seen from the images in the first row, where bolts are obscured by each other and the lighting is low, methods 1 and 3 resulted in missed detections of bolts, while method 2 resulted in false detections. In the images in the second row, where bolts and nuts are densely stacked, methods 1 and 2 resulted in false detections, while method 3 resulted in missed detections. In the images in the third row, where the shapes of the objects are varied and they are obscured by each other, methods 1 and 3 resulted in false detections, while method 2 resulted in missed detections. In the images in the fourth row, where there is a mixture of slender objects and small parts, methods 1-3 all resulted in varying degrees of false detections, while methods 2 and 3 both resulted in missed detections. This solution demonstrated satisfactory detection results in the above experiments, proving that the detection method designed in this invention can effectively alleviate the problems of false detections and missed detections of objects in scenarios such as dense stacking, high similarity, and low lighting conditions.
[0070] An image-based inspection system for machine parts in industrial warehousing includes: The shallow hybrid attention module performs convolution and downsampling on the object image to obtain a shallow feature map. The shallow feature map is then averaged along the horizontal and vertical directions, and then sequentially convolved, normalized, and activated to obtain horizontal and vertical attention weights. These weights are then multiplied element-wise with the shallow feature map to obtain coordinate attention enhancement features. Query, key, and value are generated from the shallow feature map, and after being evenly divided along the feature dimensions, multi-head self-attention features are obtained. These features are then fused in parallel with the coordinate attention enhancement features to obtain the shallow hybrid attention features.
[0071] The high-level hybrid attention module further convolves and downsamples the shallow feature map to obtain a high-level feature map; the high-level feature map is then processed using the same steps as the shallow feature map in the shallow hybrid attention module to obtain the high-level hybrid attention features.
[0072] The weighted fusion module divides the shallow and high-level mixed attention features into subsets along the channel dimension and assigns learnable weights. Based on the backpropagation algorithm, the weight allocation is adaptively optimized, and the features of each subset are weighted and fused to obtain multi-scale weighted fusion features.
[0073] The detection output module inputs multi-scale weighted fusion features into the detection head, and simultaneously performs classification and regression detection in one-to-one and one-to-many branches. The residual module is embedded in the one-to-one branch, and the detection results of the machine object are output.
[0074] Preferably, the weighted fusion module includes: The adaptive optimization weight allocation module assigns learnable weights to each sub-feature map and normalizes them to obtain learnable weight scalars; it calculates the current detection error based on the loss function to obtain the gradient information of the loss with respect to each weight scalar; based on the gradient information, it further calculates the gradient of each learnable weight, and then iteratively updates each learnable weight according to the gradient descent strategy.
Claims
1. A method for detecting machine parts in industrial warehousing, characterized in that, Includes the following steps: S1. Perform convolution and downsampling on the machine object image to obtain a shallow feature map; perform average pooling on the shallow feature map along the horizontal and vertical directions respectively, and then perform convolution, normalization and activation processing in sequence to obtain horizontal attention weights and vertical attention weights, and multiply them element-wise with the shallow feature map to obtain coordinate attention enhancement features. The query, key, and value are generated from the shallow feature map. After being evenly divided along the feature dimension, multi-head self-attention features are obtained. These features are then fused in parallel with coordinate attention enhancement features to obtain shallow hybrid attention features. S2. Continue convolution and downsampling on the shallow feature map to obtain the high-level feature map; apply the same processing steps to the shallow feature map as in S1 to the high-level feature map to obtain the high-level hybrid attention features. S3. Divide the shallow mixed attention features and the high-level mixed attention features into subsets along the channel dimension and assign learnable weights. Adaptively optimize the weight allocation based on the backpropagation algorithm. After weighted fusion of the features of each subset, obtain the multi-scale weighted fusion features. S4. Input the multi-scale weighted fusion features into the detection head, and simultaneously enter the one-to-one branch and the one-to-many branch for classification detection and regression detection. Embed the residual module in the one-to-one branch and output the detection results of the machine object.
2. The method for detecting machine parts in industrial warehousing according to claim 1, characterized in that... Adaptive optimization of weight allocation in S3: Learnable weights are assigned to each sub-feature map and normalized to obtain learnable weight scalars; the current detection error is calculated based on the loss function to obtain the gradient information of the loss with respect to each weight scalar. Based on the gradient information, the gradient of each learnable weight is further calculated, and then the learnable weight is iteratively updated according to the gradient descent strategy.
3. The method for detecting machine parts images in industrial warehousing according to claim 1, characterized in that... One-to-one and one-to-many branches in S4: Multi-scale weighted fusion features are used as input feature vectors and processed in parallel through one-to-many and one-to-one branches. The one-to-many branch is processed by regular convolution, batch normalization and activation. The one-to-one branch is processed by depthwise separable convolution, feature enhancement and residual module in sequence to output accurate classification results.
4. The method for detecting machine parts in industrial warehousing according to claim 1, characterized in that... In S4, residual modules are embedded in one-to-one branches: The residual module directly passes the input features to the output, and is cascaded with depthwise separable convolutions with one-to-one branches and 1×1 convolutions to further enhance the features.
5. The method for detecting machine parts in industrial warehousing according to claim 1, characterized in that... The multi-head self-attention features in S1 include: By uniformly dividing the query, key, and value into several parts along the feature dimension, and performing dot product and normalization on each part based on the corresponding query, key, and value, the corresponding single-head self-attention features are obtained. After concatenating multiple single-head self-attention features, a linear transformation is performed to obtain multi-head self-attention features.
6. The method for detecting machine parts images in industrial warehousing according to claim 1, characterized in that... Parallel fusion in S1 includes: The multi-head self-attention features and coordinate attention enhancement features are dimensionally aligned, and then element-wise weighted fusion, convolution, and normalization are performed sequentially.
7. The method for detecting machine parts in industrial warehousing according to claim 1, characterized in that... The horizontal and vertical attention weights in S1 include: The horizontal and vertical attention weights are expanded in dimension to match the spatial size of the shallow feature map. The expanded horizontal and vertical attention weights are then added element by element and fused together, and finally weighted with the shallow feature map to obtain the coordinate attention-enhanced feature.
8. The method for detecting machine parts in industrial warehousing according to claim 1, characterized in that... Detection head in S4: A non-nonmaximum suppression detection head is used, and the detection results are output in parallel through one-to-one branches and one-to-many branches.
9. A machine part image detection system for industrial warehousing, characterized in that, include: The shallow hybrid attention module performs convolution and downsampling on the machine object image to obtain a shallow feature map; The shallow feature map is average pooled along the horizontal and vertical directions respectively, and then processed by convolution, normalization and activation to obtain the horizontal attention weights and vertical attention weights. These are then multiplied element-wise with the shallow feature map to obtain the coordinate attention enhancement features. The query, key, and value are generated from the shallow feature map. After being evenly divided along the feature dimension, multi-head self-attention features are obtained. These features are then fused in parallel with coordinate attention enhancement features to obtain shallow hybrid attention features. The high-level hybrid attention module further convolves and downsamples the shallow feature maps to obtain high-level feature maps; The high-level feature map is processed using the shallow feature map processing steps in the shallow hybrid attention module to obtain high-level hybrid attention features; The weighted fusion module divides the shallow and high-level mixed attention features into subsets along the channel dimension and assigns learnable weights. It adaptively optimizes the weight allocation based on the backpropagation algorithm and obtains multi-scale weighted fusion features by weighted fusion of the features of each subset. The detection output module inputs multi-scale weighted fusion features into the detection head, and simultaneously performs classification and regression detection in one-to-one and one-to-many branches. The residual module is embedded in the one-to-one branch, and the detection results of the machine object are output.
10. The machine object image detection system for industrial warehousing according to claim 9, characterized in that... The weighted fusion module includes: The adaptive optimization weight allocation module assigns learnable weights to each sub-feature map and normalizes them to obtain learnable weight scalars; it calculates the current detection error based on the loss function to obtain the gradient information of the loss with respect to each weight scalar; based on the gradient information, it further calculates the gradient of each learnable weight, and then iteratively updates each learnable weight according to the gradient descent strategy.