Target detection method, device, equipment and medium

Abstract

The present application discloses a target detection method, device, equipment, and medium, relating to the field of industrial production technology. The target detection method includes: obtaining a target image to be detected, the target image being an image of a product to be boxed and / or a box body in a product boxing scene captured by a camera; inputting the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, the target detection result including position information of the product and / or box body; wherein the target detection model includes a backbone network, a neck network, and a detection head connected in sequence, the backbone network including at least N first convolution modules for feature extraction based on parallel convolution and a variable convolution kernel, M of the N first convolution modules being convolution modules including wavelet convolution submodules, M being less than or equal to N, and the wavelet convolution submodule being used to increase the weight of the contour features of the product in the target image.

Description

Technical Field

[0001] The present application relates to the field of industrial production technology, and more specifically, to a target detection method, device, equipment, and medium. Background Art

[0002] In industrial production, product assembly and packaging are crucial, and the inspection of the product packing process is a key step in these processes. Currently, with the rapid development of computer vision technology, object detection technology is gradually being applied to industrial production to achieve inspection of the product packing process.

[0003] However, in industrial production scenarios, the actual application of object detection technology faces complex and changing environments. For example, small products can lead to small targets, and there can be occlusion between products. This results in low detection accuracy in traditional object detection methods. Summary of the Invention

[0004] One purpose of this application is to provide a new technical solution for target detection.

[0005] According to a first aspect of the present application, a target detection method is provided, the method comprising:

[0006] Acquire a target image to be detected, where the target image is an image of a product to be boxed and / or a box body captured by a camera in a product boxing scene;

[0007] Inputting the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, wherein the target detection result includes position information of the product and / or the box;

[0008] In which, the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network is used to perform feature extraction on the target image to obtain multiple first feature maps of different scales; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head is used to generate the target detection results corresponding to each second feature map; the backbone network includes at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels, M of the N first convolution modules are convolution modules containing wavelet convolution submodules, M is less than or equal to N, and the wavelet convolution submodule is used to increase the weight of the contour features of the product in the target image.

[0009] Optionally, the backbone network also includes multiple second convolution modules for performing convolution operations, normalization operations and activation operations, at least one of the second convolution modules is used to perform convolution operations, normalization operations and activation operations on the target image to obtain a third feature map, and input the third feature map into the wavelet convolution submodule in the first convolution module to generate the first feature map.

[0010] Optionally, the wavelet convolution submodule includes at least a first wavelet transform unit and a second wavelet transform unit, the first wavelet transform unit being used to decompose the third feature map to obtain low-frequency subband features and high-frequency subband features, and input the low-frequency subband features into the second wavelet transform unit; the low-frequency subband features are used to characterize the contour features of the product in the third feature map, and the high-frequency subband features are used to characterize the local detail features of the product in the third feature map; the second wavelet transform unit is used to decompose the low-frequency subband features to obtain first low-frequency subband features and second low-frequency subband features to increase the feature weight of the contour features in the generated first feature map.

[0011] Optionally, the M is smaller than the N, and the other NM first convolution modules except the M first convolution modules among the N first convolution modules do not include the wavelet convolution submodule.

[0012] Optionally, the neck network includes a feature fusion extraction module and a spatial enhancement attention module,

[0013] The feature fusion extraction module is used to perform feature fusion extraction based on the first feature maps of multiple different scales to obtain a fourth feature map of multiple different scales;

[0014] The spatial enhanced attention module is used to perform spatial attention weighting and multi-scale feature fusion processing on each of the fourth feature maps, generate the second feature map, and input the second feature map into the detection head.

[0015] Optionally, the pre-generated target detection model is pre-generated based on the following method:

[0016] Acquire sample images and training labels corresponding to the sample images, wherein the sample images are images of products to be packed and / or boxes captured by a camera in a product packing scene;

[0017] Training a preset first detection model according to the sample image and the training label to generate a trained second detection model;

[0018] Pruning is performed on the first detection model to generate the target detection model.

[0019] Optionally, the target image is a plurality of temporally continuous target images; the method further comprises:

[0020] Get the target detection results corresponding to each frame of image;

[0021] Assigning product identifications to products in the target detection results based on a BOT-SORT algorithm;

[0022] determining a packing state of the product according to a product identification of the product and a change in an area of ​​the product in the multiple frames of target images;

[0023] The quantity of packed products is determined according to the packing status of the products.

[0024] According to a second aspect of the present application, a target detection device is provided, the device comprising:

[0025] An acquisition module, configured to acquire a target image to be detected, wherein the target image is an image of the product to be boxed and / or the box body in a product boxing scene captured by a camera;

[0026] a detection module, configured to input the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, wherein the target detection result includes position information of the product and / or the box;

[0027] In which, the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network is used to perform feature extraction on the target image to obtain multiple first feature maps of different scales; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head is used to generate the target detection results corresponding to each second feature map; the backbone network includes at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels, M of the N first convolution modules are convolution modules containing wavelet convolution submodules, M is less than or equal to N, and the wavelet convolution submodule is used to increase the weight of the contour features of the product in the target image.

[0028] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute a method as described in any one of the first aspects.

[0029] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0030] The present application provides a target detection method, which includes: obtaining a target image to be detected, the target image being an image containing products and / or boxes to be packed in a product packing scene captured by a shooting device; inputting the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, the target detection result including position information of the products and / or boxes; wherein the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network being used to perform feature extraction on the target image to obtain multiple first feature maps of different scales; the neck network being used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head being used to generate target detection results corresponding to each second feature map; the backbone network including at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels, M of the N first convolution modules being convolution modules including wavelet convolution submodules, M being less than or equal to N, and the wavelet convolution submodule being used to increase the weight of the contour features of the product in the target image. In this way, by introducing the wavelet convolution submodule into the target detection model, the weight of the contour features of the product in the target image can be increased. When there is occlusion and dense arrangement of products in the product packaging scenario, the position of the product can still be accurately detected, thereby improving the detection accuracy.

[0031] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0033] Figure 1 This is a hardware configuration framework of an electronic device for implementing a target detection method according to an embodiment of the present application. Figure 1 ;

[0034] Figure 2 This is a flow chart of a method for implementing target detection according to an embodiment of the present application;

[0035] Figure 3 is a schematic structural diagram of a target detection model provided according to an embodiment of the present application;

[0036] Figure 4 Schematic diagram of the structure of a wavelet convolution submodule provided according to an embodiment of the present application;

[0037] Figure 5 is a structural diagram of a first convolution module provided according to an embodiment of the present application;

[0038] Figure 6 is a structural diagram of a spatial attention module provided according to an embodiment of the present application;

[0039] Figure 7 1 is a flow chart of a method for generating a target detection model according to an embodiment of the present application;

[0040] Figure 8 This is a schematic diagram of a structure of a target detection device according to an embodiment of the present application;

[0041] Figure 9 This is a hardware configuration framework of an electronic device for implementing a target detection method according to an embodiment of the present application. Figure 2 . DETAILED DESCRIPTION

[0042] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0043] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0044] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above technologies, methods, and equipment should be considered part of the specification.

[0045] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0046] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0047] Figure 1 This is a hardware configuration framework of an electronic device for implementing a target detection method according to an embodiment of the present application. Figure 1 .

[0048] The electronic device 1000 can be a terminal or a server. Furthermore, the terminal can be a hardware device (such as an AR device, an MR device, and a VR device), a portable computer, a tablet computer, a PDA, etc. The server can be a cloud server, etc.

[0049] Electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. Processor 1100 may be a central processing unit (CPU), a microprocessor (MCU), or the like. Memory 1200 may include, for example, ROM (read-only memory), RAM (random access memory), or non-volatile memory such as a hard disk. Interface device 1300 may include, for example, a USB port or a headphone jack. Communication device 1400 may be capable of wired or wireless communication. Display device 1500 may be, for example, an LCD display or a touchscreen display. Input device 1600 may include, for example, a touchscreen or a keyboard. Users may input and output voice information via speaker 1700 and microphone 1800.

[0050] Despite Figure 1 Multiple devices are shown for the electronic device 1000, but the embodiments of the present application may only involve some of the devices, for example, the electronic device 1000 only includes a memory 1200 and a processor 1100.

[0051] In the embodiments of the present application, the memory 1200 of the electronic device 1000 may be used to store instructions, which are used to control the processor 1100 to execute the target detection method provided in the embodiments of the present application.

[0052] In the above description, a person skilled in the art can design instructions according to the solution disclosed in this application. How instructions control the processor to operate can be described in the relevant art and will not be repeated here.

[0053] This application provides a target detection method, which is applied to Figure 1 The electronic equipment shown. Figure 2 As shown, the target detection method provided in this application includes the following steps S2100 to S2200.

[0054] Step S2100: Acquire the target image to be detected.

[0055] The target image is an image of the product to be packed and / or the box body in a product packing scene captured by a shooting device.

[0056] For example, a box may be used to load products, and a camera may be located above the box to capture a scene of product packing, such as a packing video of the product being packed into the box. The target image may be one or more frames of images in the packing video.

[0057] It should be noted that in product packaging scenarios, there are complex and changeable scenarios such as occlusion, densely arranged products, and small detection targets. The detection accuracy of related technologies is often difficult to meet the requirements.

[0058] Step S2200: Input the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model.

[0059] The target detection result includes the location information of the product and / or the box. Optionally, the target detection result may also include parameters such as the type and area of ​​the product and / or the box.

[0060] The target detection model includes a backbone network, a neck network and a detection head connected in sequence, wherein the backbone network is used to extract features of the target image to obtain multiple first feature maps of different scales, and input the first feature maps into the neck network; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales, and input the second feature maps into the detection head; the detection head is used to generate target detection results corresponding to each second feature map.

[0061] In one embodiment of the present application, the backbone network may include at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels, wherein M of the N first convolution modules are convolution modules including wavelet convolution submodules, where M is less than or equal to N, and both N and M are positive integers. The wavelet convolution submodule is used to increase the feature weight of the contour features of the product in the target image. The feature weight can be used to characterize the proportion of each feature in the first feature map. The larger the feature weight of the contour feature, the more accurate the contour of the product can be detected. In complex scenarios such as product packaging, the accuracy of the target detection model for product detection can be improved.

[0062] Figure 3 This is a schematic diagram of the structure of a target detection model provided according to an embodiment of the present application. Figure 3 As shown, the target detection model can be an improved YOLO11 model.

[0063] It should be noted that while the YOLO11 model in related technologies can provide rapid detection, it still suffers from insufficient accuracy when faced with the complex scenarios described above during the product packing process, resulting in missed or false detections. This embodiment improves the YOLO11 model by introducing a wavelet convolution submodule to increase the accuracy of the object detection model for product detection, effectively reducing false and missed detections during the product packing process.

[0064] like Figure 3As shown in the figure, the target detection model (improved YOLO11 model) can include a backbone network (backbone), a neck network (Neck), and a detection head (Head). The following are introduced respectively:

[0065] The backbone network can be used for feature extraction, for example, to extract features from a target image to obtain multiple first feature maps of different scales. The backbone network may include multiple first convolution modules and multiple second convolution modules. The first convolution module may be the C3K2 module in YOLO11, and the second convolution module may be the CBS (Convolution-BatchNorm-Silu) module in YOLO11, that is, the second convolution module is used to perform convolution operations, normalization operations, and activation operations in sequence. Figure 3 As shown, the backbone network in this embodiment may include multiple layers connected in series, for example, including an input layer, a CBS module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, an SPPF (Spatial Pyramid Pooling-Fast) module, and a C2PSA (Global Guided Pathway Attention) module, a total of 12 layers.

[0066] In one embodiment of the present application, the above-mentioned M may be smaller than N, and the other NM first convolution modules except the M first convolution modules among the N first convolution modules do not include a wavelet convolution submodule.

[0067] In some examples, the backbone network includes four C3K2 modules. Wavelet convolution submodules can be added to the C3K2 modules (i.e., the first convolution modules) at the eighth and tenth layers, i.e., they can be transformed into C3K2_WTConv. Optionally, the first convolution module including the wavelet convolution submodule and the first convolution module not including the wavelet convolution submodule can be distinguished in name. For example, the first convolution module including the wavelet convolution submodule can be called the third convolution module (e.g., Figure 3 The first convolution module that does not contain the wavelet convolution submodule can be called the fourth convolution module (e.g. Figure 3 C3K2 in ).

[0068] It should be noted that in the backbone network of the YOLO11 model, the scale feature maps processed by the eighth and tenth layers are primarily responsible for extracting high-level semantic information. The wavelet convolution submodule uses wavelet transforms to decompose the input into multi-scale components and applies small convolution kernels to low-frequency subbands. This significantly expands the receptive field while only increasing the number of parameters logarithmically, while enhancing the ability to capture low-frequency features (such as shape and structure). This design improves the efficiency of extracting complex features. In the product packaging tracking and counting system, WTConv's large receptive field and sensitivity to low-frequency features enable the model to extract structural information from occluded objects and infer their location, thereby improving recognition robustness. At the same time, by expanding the receptive field, the model can detect small objects in a wider context and leverage surrounding information to improve counting accuracy. This improvement significantly enhances the system's performance in complex scenarios, ensuring tracking and counting accuracy.

[0069] The shallow layers of the backbone network (such as the C3K2 modules in the fourth and sixth layers) mainly extract low-level features such as edges and textures. These features are relatively simple and can be efficiently processed by traditional convolution. Therefore, there is no need to add wavelet convolution submodules to improve the computational efficiency of the model.

[0070] At least one second convolution module (i.e., CBS module) is configured to perform convolution, normalization, and activation operations on the target image to obtain a third feature map, and then input the third feature map into the wavelet convolution submodule in the first convolution module to generate the first feature map. The at least one second convolution module can be a module that inputs feature maps to a C3K2 module including a wavelet convolution submodule, such as the CBS modules in the seventh and ninth layers.

[0071] Figure 4 Schematic diagram of the structure of a wavelet convolution submodule provided according to an embodiment of the present application. Figure 4 As shown, the wavelet convolution submodule C3K2_WTConv of this embodiment may include at least a first wavelet transform unit and a second wavelet transform unit. The first wavelet transform unit is used to decompose the input third feature map to obtain low-frequency subband features and high-frequency subband features, and input the low-frequency subband features into the second wavelet transform unit; the low-frequency subband features are used to characterize the contour features of the product in the third feature map, and the high-frequency subband features are used to characterize the local detail features of the product in the third feature map; the second wavelet transform unit is used to further decompose the low-frequency subband features to obtain first low-frequency subband features and second low-frequency subband features, so as to improve the feature weight of the contour features in the generated first feature map.

[0072] Specifically, the wavelet convolution submodule uses a two-dimensional discrete wavelet transform to decompose the input feature map into low-frequency and high-frequency subband features. The low-frequency subband features are used to capture global structural features (such as the outline of a product), while the high-frequency subband features are used to preserve local details. After applying standard convolution to the low-frequency subband features, the features are reconstructed using an inverse wavelet transform and added to the input standard convolution result to generate the output. To enhance multi-scale capabilities, multiple layers of wavelet transforms can be nested to expand the receptive field. Compared to traditional convolution (e.g., increasing the convolution kernel or stacking the layers), the parameter growth of wavelet convolution is logarithmic rather than quadratic. Wavelet convolution significantly reduces the number of parameters and computational complexity, laying the foundation for a lightweight model. It also significantly expands the receptive field and improves object detection accuracy.

[0073] In this embodiment, the second wavelet transform unit further decomposes the low-frequency sub-band features to form a hierarchical abstract structure: through cascaded decomposition, the perceptual range of the underlying convolution kernel is gradually multiplied (for example, the second-level decomposition enables a 3×3 kernel to cover a 12×12 area in the original input). Under the constraint that the parameters increase only linearly, the network is forced to focus on the underlying geometric structure of the target, thereby achieving more robust global feature association in occluded scenes. In other words, by repeatedly decomposing and strengthening low-frequency sub-band features (such as the outline of a product), the ability to capture the outline of occluded objects can be significantly improved, and the computational efficiency is close to that of standard convolution.

[0074] Figure 5 : is a structural diagram of a first convolution module provided according to an embodiment of the present application. Figure 5 As shown, the first convolution module includes a CBS module, a splitting module, M first submodules (the first submodule can be called a C3K module), a dimension fusion module and a CBS module connected in series.

[0075] The CBS module consists of three parts: convolution, batch normalization, and SiLU activation. It performs convolution on the input data, followed by batch normalization and activation, to output a feature map. The split module splits the feature map generated by CBS into multiple parts for parallel processing in subsequent convolution operations, helping the model learn richer feature representations.

[0076] M (e.g., 2 or 3) first submodules can be used to extract features and enhance their expressiveness. The dimension fusion module is used to fuse the multiple feature maps output by the splitting module to restore them to their original dimensions. This can be achieved through feature map concatenation, addition, or other fusion techniques to integrate feature information learned from different paths. Following dimension fusion, the CBS module performs further convolution, batch normalization, and activation operations on the fused feature maps to enhance the model's expressiveness.

[0077] The first submodule includes a CBS module, N bottleneck units (Bottleneck), dimension fusion and CBS module connected in sequence. The bottleneck unit (Bottleneck) includes a CBS module and a wavelet convolution submodule connected in sequence. In other words, Figure 5 The first convolution module shown includes a wavelet convolution submodule. The bottleneck unit may also include an optional residual connection, based on which the input of the bottleneck unit may be directly superimposed on the output of the wavelet convolution submodule to obtain the output of the bottleneck unit. The bottleneck unit in the first convolution module including the wavelet convolution submodule may be provided with the residual connection, i.e., the input of the bottleneck unit is allowed to be directly superimposed on the output of the wavelet convolution submodule. For the bottleneck unit in the first convolution module that does not include the wavelet convolution submodule, the residual connection may not be provided, i.e., the input of the bottleneck unit does not need to be superimposed again. It should be noted that the user may also determine whether the bottleneck unit in each first convolution module is provided with the residual connection according to different requirements, and this embodiment does not limit this.

[0078] In some examples, the backbone network may perform feature extraction on the target image to obtain at least three first feature maps at different scales, and input the three first feature maps at different scales into the feature fusion extraction module of the neck network. For example, the at least three first feature maps may be output by the C3K2 module in the sixth layer, the C3K2_WTConv module in the eighth layer, and the C2PSA module in the twelfth layer, respectively.

[0079] The neck network can be used to perform feature fusion processing on multiple first feature maps of different scales to obtain multiple second feature maps of different scales, and input the second feature maps into the detection head.

[0080] In one embodiment of the present application, the neck network includes a feature fusion extraction module and a spatially enhanced attention module (SEAM).

[0081] The feature fusion extraction module is used to perform feature fusion extraction based on the first feature maps of multiple different scales to obtain the fourth feature maps of multiple different scales. The feature fusion extraction module can adopt the structure of the neck network of YOLO11 in the related art.

[0082] The spatially enhanced attention module is used to perform spatial attention weighting and multi-scale feature fusion processing on each fourth feature map, generate a second feature map, and input the second feature map into the detection head. The spatially enhanced attention module can be a module added to the neck network of YOLO11 in the related art.

[0083] like Figure 3As shown, the neck network can include an upsampling module, a dimension fusion module, a C3K2 module, an upsampling module, a dimension fusion module, a C3K2 module, a SEAM module, a CBS module, a dimension fusion module, a C3K2 module, a SEA module, a CBS module, a dimension fusion module, a C3K2 module, and a SEA module connected in sequence. Three of the SEAM modules are spatially enhanced attention modules, each of which is used to output a second feature map to the detection head. The other modules constitute a feature fusion extraction module, which is used to perform feature fusion extraction on multiple first feature maps of different scales to obtain multiple fourth feature maps of different scales, and input the fourth feature maps into the SEAM module respectively, so that the SEAM module generates a second feature map based on the fourth feature map.

[0084] It should be noted that if Figure 3 As shown in Figure 2, the spatial attention module (SEAM) can be applied to the neck network before outputting to the detection head. This helps the model adjust feature map weights at different scales (e.g., large-scale, medium-scale, and small-scale object detection layers) and optimize occlusion handling. For example, in product packaging scenarios, SEAM can effectively reduce missed detections due to partial occlusions and improve detection robustness.

[0085] Figure 6 Schematic diagram of the structure of a spatial attention module SEAM provided according to an embodiment of the present application. Figure 6 As shown, the spatial attention module may include an input layer, a CSMM module (Channel and Spatial Mixing Module), an average pooling layer (AveragePooling), a linear layer (Linear), a ReLU activation function (ReLU), a linear layer (Linear), a Sigmoid activation function (Sigmoid), and a channel expansion (Channel exp) connected in sequence, wherein:

[0086] The Channel and Spatial Mixing Module (CSMM) processes input data with different patch sizes (e.g., Patch = 6, 7, and 8) to extract multi-scale features. The CSMM module processes input data using patches of varying sizes to capture feature information at different scales. Patches are small, square or rectangular regions extracted from the target image that can be used to capture local features within the image. When processing an image, dividing a large image into multiple patches allows the model to focus on image details, thereby improving the efficiency and accuracy of feature extraction. In this embodiment, using different patch sizes means that the model extracts features from local regions of varying sizes from the input data. This allows for the capture of features at different scales, enabling the model to better understand various details and structures within the image. For example, smaller patches can capture finer details of a product, while larger patches can capture broader contextual information, such as its outline. The Average Pooling layer performs average pooling on features processed by the CSMM module to reduce the dimensionality of the feature map while retaining key information. The Linear layer performs a linear transformation on the pooled features to adjust their dimensionality. The ReLU activation function introduces nonlinearity to enhance the model's expressiveness. The Linear layer performs another linear transformation on the features to further adjust their dimensionality. The Sigmoid activation function maps feature values ​​to the (0, 1) range to facilitate subsequent channel expansion operations. Channel expansion (Channel exp) is used to enhance the expressiveness of features by increasing the number of channels.

[0087] In one embodiment of the present application, the CSMM module may include at least three CSMMs, for example, three CSMMs with patches 6, 7, and 8, respectively. Each CSMM includes sequentially connected modules such as patch embedding, GELU activation function (GELU), batch normalization, depthwise convolution, GELU activation function (GELU), batch normalization, pointwise convolution, GELU activation function (GELU), and batch normalization. Among them:

[0088] Patch Embedding is used to split the input data into patches (regions) of different sizes and embed them to capture local features. The GELU activation function (GELU) is used to introduce the GELU activation function after Patch Embedding to enhance nonlinear characteristics. Batch Normalization is used to perform batch normalization on features to accelerate model convergence and improve stability. Depthwise Convolution is used to perform convolution operations on features by channel to extract spatial features and learn the correlation between spatial dimensions and channels. Pointwise Convolution is used to fuse channel information through 1x1 convolution to learn the correlation between channels.

[0089] In this way, the SEAM module, through its CSMM module, extracts multi-scale features from different patches and uses depthwise separable convolution to extract spatial features by channel and learn the correlation between spatial dimensions and channels, thereby reducing computational complexity. Furthermore, SEAM combines residual connections to enhance feature transfer to avoid information loss, 1x1 convolutions to fuse channel information to learn correlations, and fully connected layers to highlight the feature expression of unoccluded areas through a two-layer network. This adaptively enhances the feature response of unoccluded areas and compensates for feature loss caused by occlusion.

[0090] In one embodiment of the present application, the detection head in the target detection model can be used to generate target detection results corresponding to each second feature map. For example, each second feature map can be detected to obtain a target detection result. It should be noted that the specific implementation methods of the backbone network, neck network, and some modules in the detection head of the target detection model can also adopt methods in related technologies, and this embodiment does not limit this.

[0091] In one embodiment of the present application, a method for generating a target detection model is further provided. The method may include the following steps S11 to S12.

[0092] Step S11: Obtain a sample image and a training label corresponding to the sample image.

[0093] The sample image is an image of the product to be packed and / or the box body in a product packing scene captured by a photographing device.

[0094] Step S12: training the preset first detection model according to the sample image and the training label to generate a trained second detection model.

[0095] Step S13: Pruning the first detection model to generate a target detection model.

[0096] For example, the first detection model can be pruned using the Layer-wise Adaptive Magnitude Pruning (LAMP) method to generate an object detection model. LAMP adaptively removes redundant parameters based on the importance of each layer's weights, successfully maintaining high model accuracy while reducing computational complexity. This optimization results in a smaller and faster model, making it ideal for deployment on edge devices and meeting the demands of real-time applications.

[0097] In this way, the target detection model is pre-generated using the above method, and the model is optimized based on pruning technology, which reduces redundant parameters and improves the inference speed. This target detection model can be applied to Figure 2 The target detection method shown in FIG. 3 improves the efficiency of target detection.

[0098] The present application also provides a target detection method that can be applied to Figure 1 The electronic equipment shown. Figure 7 As shown, the target detection method may further include the following steps S7100 to S7500.

[0099] Step S7100, obtaining multiple frames of target images.

[0100] For example, it may be based on multiple frames of target images containing products to be packed and / or boxes in a product packing scene captured by a photographing device.

[0101] The shooting device can be located above the box to shoot the scene of product packing, such as a packing video of the process of products being packed into the box, thereby obtaining multiple frames of target images in the packing video.

[0102] Step S7200: Obtain the target detection result corresponding to each frame of the target image.

[0103] The above target image includes multiple frames of target images that are continuous in time, which can be based on Figure 2 The method of the illustrated embodiment obtains the target detection result corresponding to each frame of the target image. For example, a target detection model can be used to detect each frame of the target image to obtain the target detection result.

[0104] Step S7300: assign product identifications to products in the target detection results based on a preset tracking algorithm.

[0105] The preset tracking algorithm may be a BOT-SORT (Bag of Tricks for SORT) algorithm.

[0106] Step S7400: Determine the packing status of the product based on the product identification and the change in the area of ​​the product in the multiple frames of target images.

[0107] Step S7500: Determine the quantity of packed products according to the packing status of the products.

[0108] In some examples, the product's packing status may include one or more of: packing started, packing in progress, or packing completed. For example, when a product corresponding to a product identifier is first detected, the product's packing status may be packing started, followed by packing in progress. Changes in the product's area across multiple frames of target images may determine whether the product's packing status has changed to packing completed. Upon completion of a product packing, the packed product data count is incremented.

[0109] In one embodiment of the present application, the packed product data may also be output and displayed to the user. For example, an image of the product being packed, a product identification, and the current number of packed products may be displayed on a display device.

[0110] This method, while improving detection accuracy and efficiency based on the target detection model, combines it with the BoT-SORT algorithm for target tracking, further improving stability and accuracy in multi-target environments. This multi-layered optimization approach effectively reduces false positives and missed detections, improving the overall performance of product packaging tracking and counting.

[0111] Figure 8 Schematic diagram of a target detection device provided in an embodiment of the present application. Figure 8 As shown, the target detection device 800 may include:

[0112] An acquisition module 810 is configured to acquire a target image to be detected, wherein the target image is an image of a product to be boxed and / or a box body in a product boxing scene captured by a camera;

[0113] A detection module 820 is configured to input the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, wherein the target detection result includes position information of the product and / or the box;

[0114] In which, the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network is used to perform feature extraction on the target image to obtain multiple first feature maps of different scales; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head is used to generate the target detection results corresponding to each second feature map; the backbone network includes at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels, M of the N first convolution modules are convolution modules containing wavelet convolution submodules, M is less than or equal to N, and the wavelet convolution submodule is used to increase the weight of the contour features of the product in the target image.

[0115] In one embodiment of the present application, the backbone network also includes multiple second convolution modules for performing convolution operations, normalization operations, and activation operations, at least one of which is used to perform convolution operations, normalization operations, and activation operations on the target image to obtain a third feature map, and input the third feature map into the wavelet convolution submodule in the first convolution module to generate the first feature map.

[0116] In one embodiment of the present application, the wavelet convolution submodule includes at least a first wavelet transform unit and a second wavelet transform unit, the first wavelet transform unit being used to decompose the third feature map to obtain low-frequency subband features and high-frequency subband features, and input the low-frequency subband features into the second wavelet transform unit; the low-frequency subband features are used to characterize the contour features of the product in the third feature map, and the high-frequency subband features are used to characterize the local detail features of the product in the third feature map; the second wavelet transform unit is used to decompose the low-frequency subband features to obtain first low-frequency subband features and second low-frequency subband features to increase the feature weight of the contour features in the generated first feature map.

[0117] In one embodiment of the present application, M is smaller than N, and the other NM first convolution modules except the M first convolution modules among the N first convolution modules do not include the wavelet convolution submodule.

[0118] In one embodiment of the present application, the neck network includes a feature fusion extraction module and a spatial enhancement attention module.

[0119] The feature fusion extraction module is used to perform feature fusion extraction based on the first feature maps of multiple different scales to obtain a fourth feature map of multiple different scales;

[0120] The spatial enhanced attention module is used to perform spatial attention weighting and multi-scale feature fusion processing on each of the fourth feature maps, generate the second feature map, and input the second feature map into the detection head.

[0121] In one embodiment of the present application, the device also includes a generation module, which is used to pre-generate the target detection model based on the following method: obtaining a sample image and a training label corresponding to the sample image, wherein the sample image is an image of the product to be packed and / or the box body in a product packing scene captured by a shooting device; training a preset first detection model according to the sample image and the training label to generate a trained second detection model; and pruning the first detection model to generate the target detection model.

[0122] In one embodiment of the present application, the target image is a plurality of temporally continuous target images; the apparatus further comprises:

[0123] The processing module is used to obtain the target detection results corresponding to each frame of the image; assign product identifications to the products in the target detection results based on the BOT-SORT algorithm; determine the packing status of the products based on the product identifications and the changes in the areas of the products in the multiple frames of the target image; and determine the number of packed products based on the packing status of the products.

[0124] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0125] The present application also provides an electronic device, which includes any one of the target detection devices 800 provided in the above device embodiments.

[0126] Or, as Figure 9 As shown, the electronic device 1000 includes a memory 1200 and a processor 1100. The memory 1200 is used to store computer instructions, and the processor 1100 is used to call computer instructions from the memory 1200 to execute any method provided in the above method embodiments.

[0127] The present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements any of the methods described in the aforementioned embodiments of the present application. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto and may also be a transient storage medium.

[0128] An embodiment of the present application further provides a computer program product, which may include a computer program. When the computer program is executed by a processor, it can implement any of the methods in the aforementioned embodiments of the present application.

[0129] The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement any of the methods in the aforementioned embodiments of the present application.

[0130] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or raised-in-groove structure on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0131] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0132] The computer program instructions used to perform the operations of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, which may include object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present application.

[0133] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of an instruction, and the module, program segment or part of the instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It should be noted that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0137] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a target image to be detected, where the target image is an image of a product to be boxed and / or a box body captured by a camera in a product boxing scene; Inputting the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, wherein the target detection result includes position information of the product and / or the box; Wherein, the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network is used to extract features from the target image to obtain multiple first feature maps of different scales; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head is used to generate the target detection results corresponding to each second feature map; the backbone network includes at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels and multiple second convolution modules for performing convolution operations, normalization operations and activation operations, M of the N first convolution modules are convolution modules including wavelet convolution submodules, M is less than or equal to N, and the wavelet convolution submodule is used to increase the weight of the contour features of the product in the target image; The wavelet convolution submodule includes at least a first wavelet transform unit and a second wavelet transform unit. The first wavelet transform unit is used to decompose low-frequency subband features and high-frequency subband features from a third feature map obtained after at least one of the second convolution modules processes the target image, and input the low-frequency subband features into the second wavelet transform unit; the low-frequency subband features are used to characterize the contour features of the product in the third feature map, and the high-frequency subband features are used to characterize the local detail features of the product in the third feature map; the second wavelet transform unit is used to decompose the low-frequency subband features to obtain first low-frequency subband features and second low-frequency subband features, so as to increase the feature weight of the contour feature in the generated first feature map; the feature weight is used to characterize the proportion of each feature in the first feature map. The larger the feature weight of the contour feature, the more accurate the contour of the product is detected.

2. The method according to claim 1, characterized in that At least one of the second convolution modules is used to perform convolution operations, normalization operations, and activation operations on the target image to obtain a third feature map, and input the third feature map into the wavelet convolution submodule in the first convolution module to generate the first feature map.

3. The method according to claim 1, characterized in that The M is smaller than the N, and the other NM first convolution modules except the M first convolution modules among the N first convolution modules do not include the wavelet convolution submodule.

4. The method according to claim 1, wherein The neck network includes a feature fusion extraction module and a spatial enhancement attention module. The feature fusion extraction module is used to perform feature fusion extraction based on the first feature maps of multiple different scales to obtain a fourth feature map of multiple different scales; The spatial enhanced attention module is used to perform spatial attention weighting and multi-scale feature fusion processing on each of the fourth feature maps, generate the second feature map, and input the second feature map into the detection head.

5. The method according to claim 1, wherein The pre-generated target detection model is pre-generated based on the following method: Acquire sample images and training labels corresponding to the sample images, wherein the sample images are images of products to be packed and / or boxes captured by a camera in a product packing scene; Training a preset first detection model according to the sample image and the training label to generate a trained second detection model; Pruning is performed on the first detection model to generate the target detection model.

6. The method according to any one of claims 1 to 5, characterized in that The target image is a multi-frame target image that is continuous in time; the method further includes: Get the target detection results corresponding to each frame of the target image; Assigning product identifications to products in the target detection results based on a BOT-SORT algorithm; determining a packing state of the product according to a product identification of the product and a change in an area of ​​the product in the multiple frames of target images; The quantity of packed products is determined according to the packing status of the products.

7. A target detection device, characterized in that: The device comprises: An acquisition module, configured to acquire a target image to be detected, wherein the target image is an image of the product to be boxed and / or the box body in a product boxing scene captured by a camera; a detection module, configured to input the target image into a pre-generated target detection model to obtain a target detection result output by the target detection model, wherein the target detection result includes position information of the product and / or the box; In which, the target detection model includes a backbone network, a neck network and a detection head connected in sequence, the backbone network is used to extract features from the target image to obtain multiple first feature maps of different scales; the neck network is used to perform feature fusion processing on the multiple first feature maps of different scales to obtain multiple second feature maps of different scales; the detection head is used to generate the target detection results corresponding to each second feature map; the backbone network includes at least N first convolution modules for feature extraction based on parallel convolution and variable convolution kernels and multiple second convolution modules for performing convolution operations, normalization operations and activation operations, M of the N first convolution modules are convolution modules containing wavelet convolution submodules, M is less than or equal to N, and the wavelet convolution submodule is used to increase the weight of the contour features of the product in the target image; the wavelet convolution The submodule includes at least a first wavelet transform unit and a second wavelet transform unit. The first wavelet transform unit is used to decompose the third feature map obtained after at least one of the second convolution modules processes the target image to obtain low-frequency subband features and high-frequency subband features, and input the low-frequency subband features into the second wavelet transform unit; the low-frequency subband features are used to characterize the contour features of the product in the third feature map, and the high-frequency subband features are used to characterize the local detail features of the product in the third feature map; the second wavelet transform unit is used to decompose the low-frequency subband features to obtain first low-frequency subband features and second low-frequency subband features, so as to increase the feature weight of the contour feature in the generated first feature map; the feature weight is used to characterize the proportion of each feature in the first feature map. The larger the feature weight of the contour feature, the more accurate the contour of the product is detected.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, which implements the method according to any one of claims 1 to 6 when executed by a processor.

Images