Complex scene target detection method, system, storage medium and device

Abstract

The present application relates to a complex scene target detection method, system, storage medium and equipment, an image to be detected is acquired and preprocessed to obtain a blocked image and a corresponding label; the blocked image and the corresponding label are processed based on a hierarchical feature extraction network to obtain feature maps of different levels; spatial attention and channel attention are extracted from the feature maps of different levels respectively, and interlayer interaction is performed to obtain a feature spectrum containing multi-level key information, and after processing by a detection head, a detection result containing a target boundary and a category is obtained. The spatial and channel double attention mechanism is introduced into the traditional target detection algorithm, and a cross-channel information fusion mechanism is used, thereby specifically solving the problems of uneven spatial distribution of detected objects and easy confusion in a complex scene, and the present application has the characteristics of high reusability and strong robustness.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to methods, systems, storage media, and devices for target detection in complex scenes. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Automated vending machines or vending robots typically incorporate object detection systems. These systems acquire images of goods and, based on object detection algorithms, perform functions such as product storage and sales processing. Object detection algorithms, such as YOLO, R-CNN, and Vision Transformer, are usually developed to meet specific object detection requirements.

[0004] Taking a robot that sells tobacco as an example, in this scenario, tobacco is distributed along the rows of shelves in different packaging. The density of the target to be detected differs significantly between the shelf display area and the non-display area. This is a scenario with small target detection and easy confusion. Therefore, most target detection algorithms in the existing technology have poor generalization performance and need to be fine-tuned with a large amount of data for specific scenarios, which increases the data acquisition cost and prolongs the development cycle. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a method, system, storage medium, and device for object detection in complex scenes. It dynamically adjusts the image segmentation scale, gradually bringing the block segmentation scheme to a globally optimal level, thus mitigating the accuracy decline caused by differences in target size and reducing reliance on large amounts of training data. By introducing a spatial and channel dual-attention mechanism into traditional object detection algorithms and employing a cross-channel information fusion mechanism, it specifically addresses the problems of uneven spatial distribution and easy confusion of detected objects in complex scenes, exhibiting high reusability and robustness.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for target detection in complex scenes, comprising the following steps: The image to be detected is acquired and preprocessed to obtain image blocks and corresponding labels; Based on a hierarchical feature extraction network, block images and corresponding labels are processed to obtain feature maps at different levels; Spatial attention and channel attention are extracted from feature maps at different levels, and inter-layer interaction is performed to obtain a feature spectrum containing key information at multiple levels. After processing by the detection head, a detection result containing target boundaries and categories is obtained.

[0007] Furthermore, the preprocessing includes converting the acquired image to be tested into a matrix composed of pixels, and dividing the image into blocks according to the pixel-level size of the matrix.

[0008] Furthermore, image segmentation processing includes: if the combination of image size and segment size parameters satisfies the divisibility requirement, then the number of segments along the width and height dimensions is: ; ; in, , These represent the width and height of the image, respectively. and These represent the width, height, and corresponding sliding step size of the block, respectively.

[0009] Furthermore, image segmentation processing also includes: if the combination of parameters for image size and segment size does not meet the divisibility requirement, dynamically adjusting the size of the input image, as shown in the following formula: ; ; in, , These represent the width and height of the image, respectively. and These represent the width, height, and corresponding sliding step size of the block, respectively.

[0010] Furthermore, the hierarchical feature extraction network has convolutional layers of different levels for the input image. Each convolutional layer maps the input image. Assuming the pre-processed, block-based image passed to the convolutional layer is... The first layer obtained through the convolutional layer Layer feature map is .

[0011] Furthermore, during training, the hierarchical feature extraction network guides the feature extraction process on the feature map through the weight branches corresponding to the spatial attention branch and the channel attention branch.

[0012] Furthermore, the weight branches corresponding to the spatial attention branch and the channel attention branch are fused between layers and scaled to enable attention information at different semantic levels to be shared.

[0013] A second aspect of the present invention provides a system for implementing the above-described method, comprising: The preprocessing module is configured to: acquire the image to be detected and preprocess it to obtain image blocks and corresponding labels; The feature extraction module is configured to process block images and corresponding labels based on a hierarchical feature extraction network to obtain feature maps at different levels. The attention fusion module is configured to extract spatial attention and channel attention from feature maps at different levels, and perform inter-layer interaction to obtain a feature spectrum containing key information at multiple levels. After processing by the detection head, a detection result containing target boundaries and categories is obtained.

[0014] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described complex scene target detection method.

[0015] A fourth aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the above-described complex scene target detection method.

[0016] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects: 1. The introduction of spatial and channel dual attention mechanism can capture key image information at different levels of abstraction and introduce it between the backbone network of the detector and the detection head, playing a connecting role and better capturing local contextual information. It forms a cross-channel information fusion mechanism, which specifically solves the problems of uneven spatial distribution and easy confusion of detected objects in complex scenes. It has the characteristics of high reusability and strong robustness.

[0017] 2. In traditional image recognition methods, images are typically divided into 3×3 or 5×5 grids. However, for the specific characteristics of cigarette pack detection, the image segmentation method used results in a certain degree of overlap between blocks. Since the size of the detected cigarette pack images is relatively small compared to the overall image and their arrangement on the image is relatively dense, if the traditional method is used for segmentation, a large number of the target objects located at the edge of the segment will span multiple sub-blocks, making it difficult for any sub-block to completely identify them, thus increasing the false negative rate. Therefore, partial overlap can solve this problem.

[0018] 3. During training, the image segmentation scale is dynamically adjusted so that the segmentation scheme gradually reaches the global optimal level. This effectively alleviates the problem of decreased accuracy caused by differences in the size of the detected target, as well as the dependence on a large amount of training data, improves data utilization, and makes full use of image features under limited data. This can significantly reduce the data acquisition cost and development threshold of downstream detection tasks. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a schematic diagram of a complex scene target detection process provided by one or more embodiments of the present invention; Figure 2 This is a schematic diagram of overlapping spatial blocks during target detection in complex scenes provided by one or more embodiments of the present invention; Figure 3 This is a schematic diagram of a multi-scale fusion attention mechanism involved in complex scene target detection provided by one or more embodiments of the present invention; Figure 4 This is a schematic diagram of the attention fusion process in the multi-scale fusion attention mechanism provided by one or more embodiments of the present invention; Figure 5 This is a schematic diagram of the feature fusion process in the multi-scale fusion attention mechanism provided by one or more embodiments of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] As introduced in the background section, object detection algorithms for product images are often developed according to different needs and scenarios, such as: In the context of product inspection, most products are distributed along the rows and columns of the display cabinet, and there is a significant difference in the density of inspection targets between the shelf display area and the non-display area. In detection scenarios such as aviation and remote sensing, the objects to be detected are often distributed in the form of points, clusters, and axes, and there are many small targets in the images to be tested, making it very easy to confuse the targets.

[0024] Currently, most object detection algorithms are based on improvements to convolutional neural networks (CNNs), including designing convolutional kernels of different sizes and adding residual connections between different convolutional layers. These improvements have alleviated the problem of poor detection algorithm performance to some extent. However, in scenarios where the spatial distribution of objects is uneven and there are many small, low-feature, and easily confused samples, the existing algorithms are insufficient to meet the requirements.

[0025] Specifically, in the tobacco retail scenario, when detection robots identify cigarette pack images, the size of the target (cigarette pack, carton, price tag, etc.) differs significantly from the overall image size. At the same time, cigarette packaging from different brands exhibits a certain degree of similarity in appearance, especially for categories with similar colors; the differences often lie only in the font and labels. To identify products at this level of fine detail, the detection model needs to possess strong recognition capabilities for small and easily confused targets. Therefore, the following embodiments take the complex scene target detection method of identifying cigarette packs as an example, and propose a detection scheme based on sliding window and dynamic multi-scale block segmentation. This scheme maintains a certain degree of overlap between blocks. During training, the image segmentation scale is dynamically adjusted so that the block segmentation scheme gradually reaches the global optimum, effectively alleviating the problem of accuracy decline caused by differences in the size of detected targets and the dependence on a large amount of training data. In addition, this scheme introduces a spatial and channel dual attention mechanism into the traditional target detection algorithm, and also adopts a cross-channel information fusion mechanism to specifically solve the problems of uneven spatial distribution and easy confusion of detected objects in complex scenes. It has the characteristics of high reusability and strong robustness.

[0026] Example 1: like Figures 1-5 As shown, the object detection method for complex scenes includes the following steps: The input image is preprocessed through multi-scale block segmentation and overlap calculation; The segmented image is fed into a backbone network built on hierarchical convolution to extract feature maps at each level. The feature maps at different levels are processed by the spatial and channel attention modules to highlight important information and complete inter-layer interactions, forming a processed feature spectrum containing key information at multiple levels so that the detection head can make predictions and obtain the final detection result containing the target boundary and category.

[0027] In this embodiment, the target detection method receives images of cigarette packs to be detected.

[0028] Typically, the image to be tested is a matrix composed of pixels, and the shape of the matrix is... ,in ， These represent the width and height of the image, respectively, and 3 represents the number of channels, which is the RGB color value of the corresponding pixel.

[0029] After receiving the image to be tested, determine the pixel size of the image, assuming the size of the input image is [value missing]. , and These represent the width, height, and sliding step size of the block, respectively. In this embodiment, the required number of blocks along the width and height dimensions is: ; ; After segmentation, the original image is divided into multiple sub-images, each of which is a sub-image of size in the original matrix. Part of it, the difference between the horizontal and vertical coordinates between adjacent subgraphs is .

[0030] If the combination of image size and block size parameters does not meet the divisibility requirement of the above formula, the size of the input image needs to be dynamically adjusted: ; The formula for adjusting dimensions in the height direction is similar.

[0031] ; After dynamic adjustment, the dimensional parameters and sliding step parameters In the new image size If the value is divisible by the given value, the input image can be divided into blocks. For a target spanning different detection blocks, the detection results of all blocks near its corresponding region are selected and stitched together to obtain the complete range of the target across regions.

[0032] In this embodiment, the block division mechanism is as follows: Figure 2 As shown. For the sake of neatness, Figure 2 Only the vertical-scale block division is shown. Numerically, All ,from arrive The longitudinal offset is . Figure 2 Typical boundary objects spanning different regions, namely P1~P3, appeared in the data. When detecting these objects, the method of taking the bounding rectangle of the low-threshold portion of the image from different blocks was used. Assume the detected target range for each block is... The confidence threshold is Rect represents the operation of taking the bounding rectangle, and a global threshold is set. By controlling the minimum confidence level of the detection results, the final range of candidate boxes is: ； The images obtained from the block processing and their corresponding labels are fed into the backbone network of the detection algorithm. Different levels of convolutional layers are stacked to abstract the features of different levels of the image.

[0033] Specifically, for the input image The backbone network consists of multiple convolutional layers, each mapping the input image. The first convolutional layer... The mapping representation of a convolutional layer is as follows Its actual mechanism of action can be expressed as: ; in This represents the convolutional mapping level. The higher the level, the higher the degree of abstraction of the extracted features, the less original information the features contain, and the more abstract semantics they contain.

[0034] Each convolutional layer has a corresponding feature map. Assume the image passed to the convolutional layer after block preprocessing is... The first layer obtained through the convolutional layer Layer feature map is The multi-scale fusion attention mechanism processes feature maps at different levels.

[0035] like Figures 3-5 As shown, during training, two different attention branches—spatial attention branch and channel attention branch—are optimized by the deep neural network and used to guide the feature extraction process on the feature map. Assume... As attention weights, the feature map update process can be represented as: ; Among them, operators This indicates element-wise multiplication. By trained spatial attention weight branch Channel attention weight branch The result of fusion: ; in, This represents the Sigmoid function. After obtaining the multi-branch attention weights, inter-layer fusion is needed to enable the sharing of attention information at different semantic levels. Assume... These are feature information from feature layers at different scales. Through scale transformation operations, the feature map of a certain layer is transformed. and neighboring layer feature maps The dimensions are equal. After the scaling operation, the inter-layer attention fusion formula can be obtained: ; Among them This represents the normalization operation.

[0036] The aforementioned multi-scale attention mechanism module based on fusion and information interaction can be integrated with any feedforward convolutional neural network to infer attention mappings along two different paths (channel and space). Placing the module at the connection between the target detector backbone network and the detection head allows for better capture of local contextual information.

[0037] In this embodiment, the attention module connects the backbone network of the detector and the detection head, containing key image information at different levels of abstraction, thus playing a crucial role in bridging the gap between them. The detection head receives the feature map from the backbone network and the information from the attention module, and provides the final prediction result, including the location and category information of the detected target.

[0038] Targeted optimizations were made to different functional stages of the detection algorithm, which significantly improved the problem of low accuracy of traditional detection algorithms in scenarios with small target concentrations and uneven sample spatial distribution.

[0039] Based on the block-based training method, an improved method based on dynamic sliding window and detection region fusion is proposed.

[0040] From the perspective of algorithmic behavior, the introduction effectively alleviates the problem that small targets and easily confused targets are difficult to detect accurately in traditional methods.

[0041] From a data perspective, this method significantly improves data utilization. The block-based approach greatly optimizes data utilization, making full use of image features under limited data, and can significantly reduce the data acquisition cost and industrial development threshold of downstream detection tasks.

[0042] In practical deployment, this method is plug-and-play, highly scalable and reusable, supports customized development, and is easy to adapt to different specific task scenarios.

[0043] From the perspective of application cost, the improvement proposed in this invention can significantly reduce data acquisition costs and save manual annotation time, while maintaining considerable reliability and robustness, making it suitable for large-scale industrial application scenarios.

[0044] Example 2: A system for implementing the above method includes: The preprocessing module is configured to: acquire the image to be detected and preprocess it to obtain image blocks and corresponding labels; The feature extraction module is configured to process block images and corresponding labels based on a hierarchical feature extraction network to obtain feature maps at different levels. The attention fusion module is configured to extract spatial attention and channel attention from feature maps at different levels, and perform inter-layer interaction to obtain a feature spectrum containing key information at multiple levels. After processing by the detection head, a detection result containing target boundaries and categories is obtained.

[0045] The system is formed based on the method in Example 1, ensuring a certain degree of overlap between blocks. During training, the image segmentation scale is dynamically adjusted, gradually bringing the segmentation scheme to a globally optimal level. This effectively alleviates the accuracy decline caused by differences in target size and reduces reliance on large amounts of training data. Furthermore, this scheme introduces a spatial and channel dual attention mechanism into traditional target detection algorithms and employs a cross-channel information fusion mechanism, specifically addressing the issues of uneven spatial distribution and easy confusion of detected objects in complex scenes. It features high reusability and robustness.

[0046] Example 3: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the complex scene target detection method described in Embodiment 1 above.

[0047] Example 4: This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the complex scene target detection method described in Embodiment 1 above.

[0048] The steps involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method for complex scenes, characterized in that, Includes the following steps: The image to be detected is acquired and preprocessed to obtain image blocks and corresponding labels; Based on a hierarchical feature extraction network, block images and corresponding labels are processed to obtain feature maps at different levels; Spatial attention and channel attention are extracted from feature maps at different levels, and inter-layer interaction is performed to obtain a feature spectrum containing key information at multiple levels. After processing by the detection head, a detection result containing target boundaries and categories is obtained. Preprocessing includes converting the acquired image to be tested into a matrix composed of pixels, and dividing the image into blocks according to the pixel-level size of the matrix; Image segmentation processing includes: dynamically adjusting the size of the input image if the combination of parameters for image size and segment size does not meet the divisibility requirement, as shown in the following formula: ； ； , These represent the width and height of the image, respectively. and These represent the width, height, and corresponding sliding step of the block, respectively. s 1 and s 2 represents the number of blocks in the width and height dimensions.

2. The complex scene target detection method as described in claim 1, characterized in that, Image segmentation processing includes: if the combination of image size and segment size parameters satisfies the divisibility requirement, then the number of segments along the width and height dimensions is: ； ； , These represent the width and height of the image, respectively. and These represent the width, height, and corresponding sliding step size of the block, respectively.

3. The complex scene target detection method as described in claim 1, characterized in that, Hierarchical feature extraction networks have convolutional layers of different levels for the input image. Each convolutional layer maps the input image. Assuming the pre-processed, block-based image passed to the convolutional layer is... The first layer obtained through the convolutional layer Layer feature map is .

4. The complex scene target detection method as described in claim 1, characterized in that, During training, the hierarchical feature extraction network guides the feature extraction process on the feature map through the weight branches corresponding to the spatial attention branch and the channel attention branch.

5. The complex scene target detection method as described in claim 4, characterized in that, The weighted branches corresponding to the spatial attention branch and the channel attention branch are fused between layers and scaled to enable attention information at different semantic levels to be shared.

6. A target detection system for complex scenes, characterized in that, include: The preprocessing module is configured to: acquire the image to be detected and preprocess it to obtain image blocks and corresponding labels; The feature extraction module is configured to process block images and corresponding labels based on a hierarchical feature extraction network to obtain feature maps at different levels. The attention fusion module is configured to extract spatial attention and channel attention from feature maps at different levels, and perform inter-layer interaction to obtain a feature spectrum containing key information at multiple levels. After processing by the detection head, a detection result containing target boundaries and categories is obtained. Preprocessing includes converting the acquired image to be tested into a matrix composed of pixels, and dividing the image into blocks according to the pixel-level size of the matrix; Image segmentation processing includes: dynamically adjusting the size of the input image if the combination of parameters for image size and segment size does not meet the divisibility requirement, as shown in the following formula: ； ； , These represent the width and height of the image, respectively. and These represent the width, height, and corresponding sliding step of the block, respectively. s 1 and s 2 represents the number of blocks in the width and height dimensions.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps in the complex scene target detection method as described in any one of claims 1-5.

8. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the complex scene target detection method as described in any one of claims 1-5.

Images