Target object detection method and device, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of image detection, and provides a target object detection method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring an acquired image, and segmenting the acquired image according to a preset image size to obtain a to-be-detected image; inputting the to-be-detected image into a stage cycle module in a preset depth residual network to obtain a target to-be-detected feature map corresponding to the to-be-detected image; inputting the target to-be-detected feature map into a comprehensive processing module to obtain a feature vector corresponding to the to-be-detected image; and when the value of the feature vector is greater than a preset value, determining the current to-be-detected image as a target image including a target object, and detecting the target object based on the target image. Through the target object detection method provided by the invention, the speed of target object detection is improved, and meanwhile, the cost performance of target object detection is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image detection technology, and in particular to a target object detection method, device, electronic device and readable storage medium. Background Art

[0002] In the prior art, image detection algorithms are usually performed based on target detection or image segmentation, with high detection accuracy, especially for the detection of a small number of single objects. However, the execution of high-precision image detection algorithms has certain requirements on the clarity of the image to be detected, and the clarity of the image to be detected must be high, resulting in slow detection speed, high time cost and labor cost when detecting large-area targets in the image to be detected, causing unnecessary losses. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a target detection method, device, electronic device and readable storage medium to solve the problem of slow detection speed when detecting targets in a large area in an image to be detected in the prior art.

[0004] According to a first aspect of the present disclosure, a method for detecting an object is provided, comprising:

[0005] Acquire the acquired image, and divide the acquired image according to a preset image size to obtain an image to be detected;

[0006] Input the image to be detected into the stage cycle module in the preset deep residual network to obtain the target feature map to be detected corresponding to the image to be detected;

[0007] Input the target feature map to be detected into the comprehensive processing module to obtain the feature vector corresponding to the image to be detected;

[0008] When the value of the feature vector is greater than a preset value, the current image to be detected is determined as a target image including the target object, and the target object is detected based on the target image;

[0009] The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, which is used to process the image to be detected to obtain the target feature map to be detected, so as to improve the accuracy of the feature map to be detected;

[0010] The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules. The preset convolution modules are connected in sequence. The preset self-attention mechanism is configured in the first preset convolution module for processing the target feature map to be detected. The multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

[0011] According to a second aspect of the present disclosure, there is provided a target detection device, comprising:

[0012] A preprocessing module is configured to obtain the acquired image, segment the acquired image according to a preset image size, and obtain an image to be detected;

[0013] A first processing module is configured to input the image to be detected into a stage cycle module in a preset deep residual network to obtain a target feature map to be detected corresponding to the image to be detected;

[0014] The second processing module is configured to input the target feature map to be detected into the comprehensive processing module to obtain a feature vector corresponding to the image to be detected;

[0015] A detection module, configured to determine the current image to be detected as a target image including the target object when the value of the feature vector is greater than a preset value, and detect the target object based on the target image;

[0016] The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, which is used to process the image to be detected to obtain the target feature map to be detected, so as to improve the accuracy of the feature map to be detected;

[0017] The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules. The preset convolution modules are connected in sequence. The preset self-attention mechanism is configured in the first preset convolution module for processing the target feature map to be detected. The multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0020] Compared with the prior art, the disclosed embodiment has the following beneficial effects: acquiring a captured image, dividing the captured image according to a preset image size to obtain an image to be detected, inputting the image to be detected into a stage cycle module in a preset deep residual network, obtaining a target feature map to be detected corresponding to the image to be detected, inputting the target feature map to be detected into a comprehensive processing module, obtaining a feature vector corresponding to the image to be detected, and when the value of the feature vector is greater than the preset feature vector, determining the current image to be detected as a target image including the target object, and detecting the target object according to the target image, wherein the stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, which improves the network learning. The preset separation convolution attention module is used to process the image to be detected to obtain the target feature map to be detected, thereby improving the accuracy of the feature map to be detected, improving the ability to consider global information, adaptability and learning long-range dependencies, the comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules, the preset convolution modules are connected in sequence, the preset self-attention mechanism is connected in the first preset convolution module, and is used to process the target feature map to be detected. The multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected, so that the detection accuracy can be improved through multiple preset convolution modules and the preset self-attention mechanism therein, the speed of target detection is improved, and the cost performance of target detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 is a flow chart of a target object detection method provided by an embodiment of the present disclosure;

[0023] Figure 2 is a processing flow chart of a preset deep residual network provided by an embodiment of the present disclosure;

[0024] Figure 3 is a processing flow chart of a preset separation convolution attention module provided in an embodiment of the present disclosure;

[0025] Figure 4 is a processing flow chart of LSKA provided by an embodiment of the present disclosure;

[0026] Figure 5 is a processing flow chart of a preset self-attention mechanism provided by an embodiment of the present disclosure;

[0027] Figure 6 is a schematic diagram of a target object detection device provided by an embodiment of the present disclosure;

[0028] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0030] A method and device for detecting an object according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0031] In the prior art, the method of detecting a target in an image is usually based on target detection or image segmentation. The detection accuracy of the above two methods is too high, and the image clarity requirement is high, that is, when the collected image is obtained by the camera of a drone, the distance between the camera and the ground needs to be close, which leads to slow detection speed and high time and labor costs. Therefore, the present disclosure provides a target detection method, which can segment the collected image with a fixed size and detect the target based on a classification model, so that the detection process is faster, and the accuracy of the detection result is not pursued, saving time and labor costs.

[0032] Figure 1 It is a flow chart of a target object detection method provided in an embodiment of the present disclosure. Figure 1 The target object detection method can be executed by the terminal device or the server. Figure 1 As shown, the target object detection method includes:

[0033] S101, acquiring a collected image, dividing the collected image according to a preset image size, and obtaining an image to be detected;

[0034] S102, inputting the image to be detected into a stage cycle module in a preset deep residual network to obtain a target feature map to be detected corresponding to the image to be detected;

[0035] S103, inputting the target feature map to be detected into the comprehensive processing module to obtain the feature vector corresponding to the image to be detected;

[0036] S104, when the value of the feature vector is greater than a preset value, determining the current image to be detected as a target image including the target object, and detecting the target object based on the target image;

[0037] The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, which is used to process the image to be detected to obtain the target feature map to be detected, so as to improve the accuracy of the feature map to be detected;

[0038] The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules. The preset convolution modules are connected in sequence. The preset self-attention mechanism is configured in the first preset convolution module for processing the target feature map to be detected. The multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

[0039] Specifically, the collected image is obtained, wherein the collected image can be collected by an image collection device, and the image collection device can be a high-definition camera or a high-definition camera installed on a drone.

[0040] The captured image is segmented according to a preset image size to obtain an image to be detected, so that an image to be detected with a uniform size suitable for processing by a deep learning model can be obtained, so that the deep learning model can reduce the amount of calculation and maintain the consistency of the model input, wherein the preset image size can be 224×224 (pixels).

[0041] In addition, the image to be detected is input into a preset deep residual network to obtain a target feature map to be detected corresponding to the image to be detected, wherein the deep residual network includes a stage cycle module, and the stage cycle module includes multiple stage cycle submodules, each stage cycle submodule is connected in sequence, and different stage cycle submodules are used to extract image features at different abstract levels, and each stage cycle submodule includes a preset separation convolution attention module, which can reduce the computational complexity while enhancing the representation ability of image features and improve the accuracy of the target feature map to be detected by combining separation convolution operation and attention mechanism. Among them, the preset separation convolution attention module can improve the accuracy and robustness of the target feature map to be detected by modeling the association of global features.

[0042] It should be noted that improving the accuracy of the target feature map to be detected here means improving the accuracy of the target feature map to be detected compared with the accuracy of the feature map obtained by the deep residual network in the conventional technology.

[0043] Among them, the deep residual network can be ResNet (Residual Network).

[0044] In addition, the target feature map to be detected is input into the comprehensive processing module to obtain a feature vector corresponding to the image to be detected, wherein the comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules connected in series, and the preset self-attention mechanism is configured in the first preset convolution module to capture the key information in the target feature map to be detected, and process the target feature map to be detected to enhance the expression ability of the target feature map to be detected. Subsequently, through the nonlinear transformation of multiple preset convolution modules, the features are gradually abstracted and refined to generate a feature vector corresponding to the image to be detected.

[0045] In addition, the value corresponding to the calculated feature vector is compared with a preset value. When the value of the feature vector is greater than the preset value, the current image to be detected is determined to be a target image including a target object, wherein the preset feature vector can be a preset feature vector obtained through multiple experiments and can represent the typical characteristics of the target object.

[0046] In addition, by detecting the target object based on the target image, the target object in the target image can be accurately located and identified through target detection algorithms (such as bounding box regression) and the proportion of the target object in the target image can be determined.

[0047] For example, the target object may be a weed, and the image to be detected may be a cornfield. The embodiments provided in the present disclosure may be used to detect the proportion of weeds in each cornfield image, and then, based on the detected weed proportion, the herbicide and weed control costs for a certain cornfield may be reduced and the corn yield may be increased.

[0048] In another embodiment, weeds can also be determined as target objects, corn fields can be determined as images to be detected, target images with manually marked weed areas exceeding a preset threshold can be determined as "target images with target objects" in the training data, and other target images can be determined as "target images without target objects" in the training data. The preset deep residual network is trained so that during the application process, after receiving the image to be detected, the deep residual network can determine whether the image to be detected is a target image or not a target image.

[0049] Furthermore, Figure 2 is a processing flow chart of a preset depth residual network provided by an embodiment of the present disclosure, such as Figure 2As shown, the preset deep residual network includes a stage cycle module, a comprehensive processing module, an average pooling layer and a fully connected layer. The stage cycle module includes 5 stage cycle sub-modules, which can be recorded as the first stage cycle sub-module stage0, the second stage cycle sub-module stage1, the third stage cycle sub-module stage2, the fourth stage cycle sub-module stage3, and the last stage cycle sub-module stage4, among which stage0-stage3 all include a preset separation convolution attention module.

[0050] The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules. The types of preset convolution modules include a first convolution type and a second convolution type. The preset convolution module corresponding to the first convolution type includes a preset convolution layer, a normalization layer and an activation function. The preset convolution module corresponding to the second convolution type includes a preset convolution layer and a normalization layer. The preset convolution module corresponding to the first convolution type can be recorded as CBS, and the preset convolution module corresponding to the second convolution type can be recorded as CB. The number of preset convolution modules can be four, namely, the first preset convolution module, the second preset convolution module, the third preset convolution module and the fourth preset convolution module, wherein the convolution type corresponding to the third preset convolution module is the second convolution type, and the convolution type corresponding to the preset convolution modules except the third preset convolution module is the first convolution type. The preset self-attention mechanism is configured in the first preset convolution module (i.e., the first preset convolution module).

[0051] According to the technical solution provided by the embodiments of the present disclosure, by integrating a preset separation convolutional attention module and a preset self-attention mechanism into a preset deep residual network, the ability of the network to learn long-range dependencies can be improved, thereby improving the detection accuracy of the target object and reducing false detections and missed detections. By integrating a preset self-attention mechanism, the network's global information consideration ability, adaptability and learning of long-range dependencies can be improved, thereby improving the adaptability of the preset deep residual network to different scenes and lighting conditions when detecting targets.

[0052] In some embodiments, the phase cycle module includes a first phase cycle submodule, a second phase cycle submodule, a third phase cycle submodule, a fourth phase cycle submodule, and a final phase cycle submodule;

[0053] The image to be detected is input into the stage cycle module in the preset deep residual network to obtain the feature map to be detected corresponding to the image to be detected, including:

[0054] Inputting the image to be detected into the first-stage periodic submodule to process the image to be detected through the preset separation convolution attention module corresponding to the first-stage periodic submodule to obtain a first feature map;

[0055] Processing the first feature map through a preset separation convolution attention module corresponding to the second-stage periodic submodule to obtain a second feature map;

[0056] Processing the second feature map through a preset separation convolution attention module corresponding to the third-stage periodic submodule to obtain a third feature map;

[0057] Processing the third feature map through a preset separation convolution attention module corresponding to the fourth-stage periodic submodule to obtain a fourth feature map;

[0058] The fourth feature map is processed by the final periodic submodule to obtain the target feature map to be detected.

[0059] In some embodiments, the stage cycle module includes a first stage cycle sub-module, a second stage cycle sub-module, a third stage cycle sub-module, a fourth stage cycle sub-module and a final stage cycle sub-module. The image to be detected is input into the first stage cycle sub-module. In this stage, the image to be detected passes through a preset separation convolution attention module included in the first stage cycle sub-module to obtain a first feature map.

[0060] Among them, the preset separation convolutional attention module includes a preset backbone network, a large core separation convolutional attention processing module and a fully connected feedforward network. The preset separation convolutional attention module is at the tail end of the first stage periodic submodule. For example, the preset deep residual network is a ResNet18 network. After the large separation attention mechanism is inserted into the stage0 in the prior art, the stage0 in the prior art and the large separation attention mechanism are collectively referred to as the first stage periodic submodule, that is, the image to be detected is input into the first stage periodic submodule, and is processed by the stage0 in the prior art to obtain output data, and the output data is input into the large separation attention mechanism for processing to obtain the first feature map, thereby reducing the computational complexity and memory occupancy of extracting the first feature map.

[0061] In addition, after the first feature map is processed by the second-stage periodic submodule, the first feature map will be processed by a preset separation convolution attention module, which can further refine the features in the first feature map and generate the second feature map through deeper convolution operations and the guidance of the attention mechanism. Compared with the first feature map, the second feature map is more refined and rich in feature expression, and contains more image details and feature information.

[0062] In addition, the second feature map is processed by the third stage periodic submodule, and then processed by the third preset separation convolution attention module to generate the third feature map. The third feature map is more in-depth and comprehensive in terms of feature level, not only including the feature information of each local area in the image, but also beginning to show the global feature structure.

[0063] In addition, the third feature map is processed by the fourth-stage periodic sub-module and then processed by the fourth preset separation convolution attention module, which can further enhance the expressiveness of the feature map. The fourth feature map is generated through more advanced convolution operations and the guidance of the attention mechanism, and is more refined and comprehensive in feature expression.

[0064] Furthermore, the preset separation convolution attention module involved in the process of the second-stage periodic sub-module, the third-stage periodic sub-module and the fourth-stage periodic sub-module processing the corresponding feature map is consistent with the processing flow of the preset separation convolution attention module corresponding to the first-stage periodic sub-module processing the image to be detected, and the input data and the output data are different.

[0065] In addition, the fourth feature map is processed by the final periodic submodule to obtain the target feature map to be detected, which contains the feature information of each local area in the image to be detected and integrates the global context information.

[0066] According to the technical solution provided by the embodiment of the present disclosure, each stage cycle module can effectively extract rich and accurate feature information from the image to be detected through the collaborative work of its internal sub-modules, thereby improving the efficiency of feature extraction and enhancing the expressiveness of features.

[0067] In some embodiments, the preset separation convolution attention module includes a preset backbone network, a large core separation convolution attention processing module and a fully connected feedforward network, and the convolution layer in the fully connected feedforward network is a full convolution;

[0068] The image to be detected is input into the first stage periodic submodule, so as to process the image to be detected through the preset separation convolution attention module corresponding to the first stage periodic submodule to obtain a first feature map, including:

[0069] Input the image to be detected into the preset backbone network to obtain the first-stage feature map;

[0070] The first-stage feature map is processed by a large-kernel separated convolution attention module to obtain the second-stage feature map;

[0071] According to the first preset weight corresponding to the first stage feature map and the second preset weight corresponding to the second stage feature map, performing weighted averaging on the first stage feature map and the second stage feature map to obtain a third stage feature map;

[0072] The third-stage feature map is processed by a preset backbone network and a fully connected feedforward network to obtain a first feature map corresponding to the image to be detected.

[0073] In some embodiments, the preset separation convolution attention module includes a preset backbone network, a large core separation convolution attention processing module and a fully connected feedforward network, wherein the preset backbone network can be used to preliminarily extract the basic features of the image, the large core separation convolution attention processing module can be used to further refine the features through separation convolution operations and attention mechanisms, especially the important areas in the image to be detected, and the fully connected feedforward network is used to integrate and transform the features to obtain a feature map. The preset separation convolution attention module can optimize the image processing and feature extraction capabilities.

[0074] Among them, the convolutional layer in the fully connected feedforward network is designed as a fully convolutional form, which helps to maintain the integrity of spatial information while enhancing the nonlinear expression ability of features.

[0075] The image to be detected is input into the preset backbone network. Through the deep structure of the preset backbone network, low-level visual features can be effectively extracted from the image to be detected to obtain the first-stage feature map, which includes the basic contour, color, texture and other information of the image.

[0076] In addition, the first-stage feature map is processed by a large-kernel separated convolution attention module, and the amount of computation is reduced by separating the trick operations (i.e., performing convolutions separately in different dimensions). At the same time, the weights of the first-stage feature map are dynamically adjusted using the attention mechanism, so that the model can focus on the key areas in the image to obtain the second-stage feature map. Compared with the first-stage feature map, the second-stage feature map retains important information while removing redundancy and noise.

[0077] In addition, by performing weighted averaging according to the first preset weight corresponding to the first-stage feature map and the second preset weight corresponding to the second-stage feature map, it is possible to fully utilize the information of the two-stage feature maps, balance the contribution between the original features and the refined features, and generate a third-stage feature map that combines the advantages of the first-stage feature map and the second-stage feature map.

[0078] In addition, the third-stage feature map is processed by presetting the backbone network and the fully connected feedforward network, so that the features can be nonlinearly transformed and integrated through the deep feature extraction capability of the backbone network and the fully convolutional layer of the fully connected feedforward network to obtain the first feature map corresponding to the image to be detected. This makes the first feature map contain rich image information and is optimized through deep learning and attention mechanism.

[0079] Among them, the large kernel separation convolution attention processing module includes: Large kernel separation convolution attention LSKA (LargeSeparable Kernel Attention), which can decompose the traditional 2D convolution kernel into two cascaded 1D convolution kernels, where the order of the 1D convolution kernel is horizontal first and then vertical, which can reduce the computational complexity and memory usage of the 2D large kernel convolution, avoid the high computational complexity caused by the large convolution kernel of the deep convolution layer in the large kernel attention LKA (Large Kernel Attention), and can obtain the long-range dependency in the preset deep residual network, and the computational complexity is smaller than that of LKA.

[0080] Figure 3 is a processing flow chart of a preset separation convolution attention module provided by an embodiment of the present disclosure, such as Figure 3 As shown, taking the preset deep residual network as ResNet18 as an example, the initial output data of stage0 output in the prior art is input into the preset backbone network, and the first stage feature map is obtained after being processed by the preset backbone network, and the first stage feature map is input into the large core separation convolution attention processing module, wherein the processing process of the above data in the large core separation convolution attention processing module is as follows: 1×1 convolution, activation function GELU (Gaussian Error Linear Unit), LSKA, 1×1 convolution, and the second stage feature map is obtained, and the first stage feature map is assigned a corresponding first preset weight, and the second stage feature map is assigned a second preset weight, and the first stage feature map and the second stage feature map are weighted averaged according to the first preset weight corresponding to the first stage feature map and the second stage feature map corresponding to the second preset weight, and the third stage feature map is obtained. The third stage feature map is input into the preset backbone network to obtain the fourth stage feature map, and the fourth stage feature map is input into the fully connected feed-forward neural network layer FFN (Feed-Forward Neural Network) Network), obtain a fifth-stage feature map, assign a corresponding third preset weight to the third-stage feature map, assign a corresponding fourth preset weight to the fifth-stage feature map, perform weighted averaging on the third-stage feature map and the fifth-stage feature map according to the third preset weight corresponding to the third-stage feature map and the fourth preset weight corresponding to the fifth-stage feature map, and obtain a first feature map.

[0081] Among them, the processing process of the fourth stage feature map in FFN is: 1×1 convolution, 3×3 convolution, GELU, 1×1 convolution. In the processing process of LSKA, the processing order of the convolution kernel is horizontal first and then vertical.

[0082] Among them, BN is used to characterize the preset backbone network, Attention is used to characterize the large kernel separation convolution attention processing module, and ConV is used to characterize convolution.

[0083] Figure 4 is a processing flow chart of LSKA provided by an embodiment of the present disclosure, such as Figure 4 As shown, the processing order of LSKA is depth-separable convolution of the first preset convolution step size and direction, depth-separable convolution of the second preset convolution step size and direction, depth-separable dilated convolution of the first preset step size, depth-separable dilated convolution of the second preset step size, and 1×1 convolution to obtain the LSKA result, and the initial input data input to the LSKA is weighted with the LSKA result to obtain the final result.

[0084] Among them, 2d-1 represents the size of the convolution kernel in the preset direction, k / d represents the size of the convolution kernel, k represents the convolution kernel parameter, d represents the dilation rate, DW-Conv represents depthwise separable convolution, and DW-D-Conv represents depthwise separable dilated convolution.

[0085] According to the technical solution provided by the embodiment of the present disclosure, the ability of the preset deep residual network to learn long-range dependencies can be improved, thereby improving detection accuracy.

[0086] In some embodiments, the first stage feature map is processed by a large kernel separation convolution attention module to obtain a second stage feature map, including:

[0087] The large kernel separation convolution attention module processes the first stage feature map in the order of horizontal convolution and vertical convolution to obtain the second stage feature map.

[0088] Specifically, the large kernel separated convolution attention module is decomposed into two main parts for operation, namely horizontal convolution and vertical convolution. Smaller convolution kernels can be used to simulate the effect of large convolution kernels while reducing the amount of computation and memory consumption.

[0089] The module applies horizontal convolution to process the first-stage feature map. Horizontal convolution focuses on capturing the contextual information of the feature map in the horizontal direction. It can perform convolution operations on each row of the feature map in a sliding window manner to extract key features in the horizontal direction and strengthen the correlation of the feature map in the horizontal dimension.

[0090] In addition, based on the horizontal convolution, the module continues to perform vertical convolution. The role of vertical convolution is to capture the contextual information of the feature map in the vertical direction, which can also be done by sliding the window, but this time the convolution operation is performed on each column of the feature map, which enhances the feature representation ability of the feature map in the vertical dimension, so that the feature map can more comprehensively reflect the spatial structure information of the input data.

[0091] According to the technical solution provided by the embodiments of the present disclosure, through the sequential processing of horizontal convolution and vertical convolution, the large kernel separation convolution attention module can effectively extract and integrate more refined and discriminative features from the first-stage feature map, thereby generating a second-stage feature map, so that the second-stage feature map contains the key information in the original feature map, and also enhances the correlation between features through the convolution attention mechanism.

[0092] In some embodiments, the target feature map to be detected is input into the comprehensive processing module to obtain the feature vector corresponding to the image to be detected, including:

[0093] Processing the target feature map to be detected by a first preset convolution module and a preset self-attention mechanism included in the first preset convolution module to obtain a first convolution feature map;

[0094] Add the first convolution feature map and the target feature map to be detected to obtain a second convolution feature map;

[0095] Inputting the second convolution feature map into other multiple preset convolution modules, so that the other multiple preset convolution modules process the feature map output by the previous preset convolution module to obtain a third convolution feature map;

[0096] The second convolution feature map and the third convolution feature map are added to obtain a feature vector corresponding to the image to be detected.

[0097] Specifically, the target feature map to be detected is processed by the first preset convolution module and the preset self-attention mechanism included in the first preset convolution module, wherein the preset self-attention mechanism can capture the correlation between different positions in the feature map, and strengthen or suppress specific features by calculating the attention weight of each position to other positions, thereby improving the feature representation capability.

[0098] In the first preset convolution module, the self-attention mechanism is combined with the convolution operation, which can work together on the target feature map to be detected, thereby obtaining the first convolution feature map containing richer information and stronger feature representation capabilities.

[0099] In addition, in order to retain the original information in the target feature map to be detected and integrate the new features in the first convolution feature map, the first convolution feature map and the target feature map to be detected are added element by element. The preliminarily processed features can be combined with the original features to form a second convolution feature map that contains both the original information and the ability to represent new features. This can introduce new feature representations while maintaining the key information in the feature map to obtain a more comprehensive feature basis.

[0100] In addition, the second convolution feature map is input into other multiple preset convolution modules for further processing, that is, other multiple convolution modules process the feature map output by the previous preset convolution module. For example, the second convolution feature map is input into the second preset convolution module to obtain the third convolution feature map, and the third convolution feature map is input into the third preset convolution module to obtain the fourth convolution feature map, and so on, gradually extracting and integrating higher-level feature information. Through the sequential processing of multiple convolution modules, a third convolution feature map containing higher-level feature representation is obtained.

[0101] In addition, in order to combine the features after preliminary processing with the features after final processing, the second convolution feature map and the third convolution feature map are added element by element, so that the feature information of different stages can be fused together to form a feature vector corresponding to the image to be detected. The feature vector corresponding to the image to be detected contains the key information in the target image to be detected, and through the processing of multiple convolution modules and self-attention mechanism, it has stronger feature representation ability and higher discrimination.

[0102] According to the technical solution provided by the embodiments of the present disclosure, the target feature map to be detected is processed by a comprehensive processing module, and the target feature map to be detected can be converted into a feature vector corresponding to the image to be detected, wherein the preset self-attention mechanism can capture the relationship between the elements within the feature map, and generate a more accurate and globally perceptual target feature map to be detected by calculating the similarity and dependency between the elements. The target feature map to be detected contains the feature information of each local area in the image to be detected, and also integrates the global context information.

[0103] In some embodiments, the target feature map to be detected is processed by a first preset convolution module and a preset self-attention mechanism to obtain a first convolution feature map, including:

[0104] Adjust the dimension to be detected of the feature map to be detected according to the self-attention dimension corresponding to the preset self-attention mechanism;

[0105] Based on the feature graph to be detected, the query vector, key vector and value vector corresponding to the preset self-attention mechanism are obtained;

[0106] Based on the query vector, key vector and value vector, self-attention processing is performed to obtain the initial first convolution feature map;

[0107] The convolution dimension corresponding to the initial first convolution feature map is adjusted to the dimension to be detected to obtain the first convolution feature map.

[0108] Specifically, according to the self-attention dimension required by the preset self-attention mechanism, the dimension to be detected of the feature map to be detected is adjusted, so as to ensure that the dimension of the feature map to be detected matches the dimension processed by the preset self-attention mechanism, so as to perform subsequent self-attention calculations.

[0109] Among them, adjusting the dimension may involve operations such as reshaping, expanding or reducing the feature map, depending on the dimensionality requirements required by the preset self-attention mechanism.

[0110] In addition, based on the adjusted feature map to be detected, the query vector (Query), key vector (Key) and value vector (Value) corresponding to the preset self-attention mechanism are calculated, where the query vector, key vector and value vector are the core components of the self-attention mechanism, which are used to represent different parts of the feature map and to calculate the correlation between different parts. The query vector is used to query the information in the feature map, the key vector is used to match the query vector, and the value vector contains the feature information after matching with the key vector.

[0111] In addition, the attention weight between the query vector and the key vector is determined, and the value vector is weighted summed with the determined weight to obtain an initial first convolutional feature map. The initial first convolutional feature map can capture the correlation between different parts of the feature map and enhance the feature information that is highly matched with the query vector.

[0112] However, the convolution dimension of the initial first convolution feature map obtained after self-attention processing may be inconsistent with the dimension to be detected. Therefore, after obtaining the initial first convolution feature map, its corresponding convolution dimension is adjusted back to the dimension to be detected, wherein the dimension adjustment may involve operations such as reshaping, interpolation or cropping of the feature map to ensure that the dimension of the first convolution feature map matches the dimension of the feature map to be detected.

[0113] Figure 5 is a processing flow chart of a preset self-attention mechanism provided by an embodiment of the present disclosure, such as Figure 5As shown, the feature map to be detected is passed through a convolution layer and a normalization layer (collectively referred to as CB), and the dimension to be detected of the feature map to be detected is adjusted according to the self-attention dimension corresponding to the preset self-attention mechanism by means of a reshaping function. According to the adjusted feature map to be detected, the query vector q, key vector k and value vector v (implemented by the Split function) corresponding to the preset self-attention mechanism are obtained, q is transposed (transposed), and matrix multiplication (@ represents matrix multiplication) is performed on q and k, and the result is scaled to avoid extreme cases when the softmax function calculates the attention allocation, and then the similarity is converted into the attention allocation value through the softmax function to ensure that the sum of all allocation values ​​is 1, and then the transpose operation is performed, the transposed data is recorded as attn, matrix multiplication is performed between the value vector and attn, reshape is performed on the result of the matrix multiplication, and reshape is performed on the value vector at the same time, and the above two reshape results are added to obtain the initial first convolution feature map.

[0114] According to the technical solution provided in the embodiment of the present disclosure, the target feature map to be detected is processed into a first convolution feature map through a first preset convolution module and a preset self-attention mechanism therein. In the process of obtaining the first convolution feature map, the key information in the feature map is extracted, and the correlation between features is enhanced through the preset self-attention mechanism.

[0115] In some embodiments, the collected image is a crop growth image; after detecting the target object based on the target image, the method further includes:

[0116] When the value corresponding to the target detection result of the target object is greater than the preset detection threshold, the crop growth ground corresponding to the target detection result on the current target image is determined as the area to be processed, so as to process the area to be processed;

[0117] When the value corresponding to the target detection result is not greater than the preset detection threshold, the current target image is discarded.

[0118] Specifically, the collected images can be images of the growth conditions of crops, which can be obtained through advanced image acquisition equipment, such as drones, so as to comprehensively and accurately reflect the growth status of crops in farmland, and can also evaluate the growth status, health status or specific growth stage of crops. That is, the target detection result can be the area of ​​crops, etc., or whether the crops are similar to the target image of "crops present" in the training data.

[0119] When the value corresponding to the target detection result is greater than the preset detection threshold, it means that the growth condition of the crops in the current image meets or exceeds the expected attention standard, which may involve abnormal growth conditions, such as signs of pests and diseases, excessive luxuriance or slow growth. For example, when the crops are weeds growing in a corn field, it indicates that the corn field needs weeding.

[0120] At this time, the crop growing ground area corresponding to the target detection result on the current target image is marked as the area to be processed. The area to be processed is regarded as a place that needs special attention or corresponding management measures, such as pest control, adjustment of irrigation and fertilization strategies, or preparation for harvesting. According to the specific agricultural management needs, the corresponding agricultural technical means or automated equipment will be used to carry out subsequent processing on the area to be processed in order to optimize the growth environment of crops and improve yield and quality.

[0121] On the contrary, if the value corresponding to the target detection result is not greater than the preset detection threshold, it indicates that the growth condition of the crops in the current image is normal or expected, and there is no obvious abnormality or management demand. In this case, in order to avoid unnecessary waste of data processing resources, the current target image can be discarded and no further analysis or processing is performed on it.

[0122] According to the technical solution provided in the embodiment of the present disclosure, the growth status of crops is carefully monitored and intelligent decision support is integrated, which can promote the efficient and healthy growth of crops through precision agricultural practices.

[0123] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0124] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0125] Figure 6 Schematic diagram of a target object detection device provided by an embodiment of the present disclosure. Figure 6 As shown, the target object detection device includes: a pre-processing module 601, a first processing module 602, a second processing module 603, and a detection module 604, wherein:

[0126] The preprocessing module 601 is configured to obtain the acquired image, segment the acquired image according to a preset image size, and obtain an image to be detected;

[0127] The first processing module 602 is configured to input the image to be detected into the stage cycle module in the preset deep residual network to obtain the target feature map to be detected corresponding to the image to be detected;

[0128] The second processing module 603 is configured to input the target feature map to be detected into the comprehensive processing module to obtain a feature vector corresponding to the image to be detected;

[0129] The detection module 604 is configured to determine the current image to be detected as a target image including the target object when the value of the feature vector is greater than a preset value, and detect the target object based on the target image;

[0130] The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, which is used to process the image to be detected to obtain the target feature map to be detected, so as to improve the accuracy of the feature map to be detected;

[0131] The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules. The preset convolution modules are connected in sequence. The preset self-attention mechanism is configured in the first preset convolution module for processing the target feature map to be detected. The multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

[0132] In some embodiments, the phase cycle module includes a first phase cycle submodule, a second phase cycle submodule, a third phase cycle submodule, a fourth phase cycle submodule, and a final phase cycle submodule;

[0133] The first processing module 602 is configured to:

[0134] Inputting the image to be detected into the first-stage periodic submodule to process the image to be detected through the preset separation convolution attention module corresponding to the first-stage periodic submodule to obtain a first feature map;

[0135] Processing the first feature map through a preset separation convolution attention module corresponding to the second-stage periodic submodule to obtain a second feature map;

[0136] Processing the second feature map through a preset separation convolution attention module corresponding to the third-stage periodic submodule to obtain a third feature map;

[0137] Processing the third feature map through a preset separation convolution attention module corresponding to the fourth-stage periodic submodule to obtain a fourth feature map;

[0138] The fourth feature map is processed by the final periodic submodule to obtain the target feature map to be detected.

[0139] In some embodiments, the preset separation convolution attention module includes a preset backbone network, a large core separation convolution attention processing module and a fully connected feedforward network, and the convolution layer in the fully connected feedforward network is a full convolution;

[0140] The first processing module 602 is configured to:

[0141] Input the image to be detected into the preset backbone network to obtain the first-stage feature map;

[0142] The first-stage feature map is processed by a large-kernel separated convolution attention module to obtain the second-stage feature map;

[0143] According to the first preset weight corresponding to the first stage feature map and the second preset weight corresponding to the second stage feature map, performing weighted averaging on the first stage feature map and the second stage feature map to obtain a third stage feature map;

[0144] The third-stage feature map is processed by a preset backbone network and a fully connected feedforward network to obtain a first feature map corresponding to the image to be detected.

[0145] In some embodiments, the first processing module 602 is configured to:

[0146] The large kernel separation convolution attention module processes the first stage feature map in the order of horizontal convolution and vertical convolution to obtain the second stage feature map.

[0147] In some embodiments, the second processing module 603 is configured to:

[0148] Processing the target feature map to be detected by a first preset convolution module and a preset self-attention mechanism included in the first preset convolution module to obtain a first convolution feature map;

[0149] Add the first convolution feature map and the target feature map to be detected to obtain a second convolution feature map;

[0150] Inputting the second convolution feature map into other multiple preset convolution modules, so that the other multiple preset convolution modules process the feature map output by the previous preset convolution module to obtain a third convolution feature map;

[0151] The second convolution feature map and the third convolution feature map are added to obtain a feature vector corresponding to the image to be detected.

[0152] In some embodiments, the second processing module 603 is configured to:

[0153] Adjust the dimension to be detected of the feature map to be detected according to the self-attention dimension corresponding to the preset self-attention mechanism;

[0154] Based on the feature graph to be detected, the query vector, key vector and value vector corresponding to the preset self-attention mechanism are obtained;

[0155] Based on the query vector, key vector and value vector, self-attention processing is performed to obtain the initial first convolution feature map;

[0156] The convolution dimension corresponding to the initial first convolution feature map is adjusted to the dimension to be detected to obtain the first convolution feature map.

[0157] In some embodiments, the image captured by the detection module 604 is an image of crop growth; after detecting the target object based on the target image, the detection module 604 is further configured to:

[0158] When the value corresponding to the target detection result of the target object is greater than the preset detection threshold, the crop growth ground corresponding to the target detection result on the current target image is determined as the area to be processed, so as to process the area to be processed;

[0159] When the value corresponding to the target detection result is not greater than the preset detection threshold, the current target image is discarded.

[0160] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0161] Figure 7 Schematic diagram of an electronic device 7 provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0162] The electronic device 7 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 7 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 The electronic device 7 is merely an example and does not limit the electronic device 7 , and may include more or less components than those shown in the figure, or different components.

[0163] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0164] The memory 702 may be an internal storage unit of the electronic device 7, for example, a hard disk or memory of the electronic device 7. The memory 702 may also be an external storage device of the electronic device 7, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. The memory 702 may also include both an internal storage unit of the electronic device 7 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0165] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0167] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A method for detecting a target object, characterized in that: include: Acquire the acquired image, and divide the acquired image according to a preset image size to obtain an image to be detected; Inputting the image to be detected into a stage cycle module in a preset deep residual network to obtain a target feature map to be detected corresponding to the image to be detected; Inputting the target feature map to be detected into a comprehensive processing module to obtain a feature vector corresponding to the image to be detected; When the value of the feature vector is greater than a preset value, determining the current image to be detected as a target image including a target object, and detecting the target object based on the target image; The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, and the preset separation convolution attention module is used to process the image to be detected to obtain a target feature map to be detected, so as to improve the accuracy of the feature map to be detected; The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules, the preset convolution modules are connected in sequence, the preset self-attention mechanism is configured in the first preset convolution module, and is used to process the target feature map to be detected, and the multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

2. The method according to claim 1, characterized in that The phase cycle module includes a first phase cycle submodule, a second phase cycle submodule, a third phase cycle submodule, a fourth phase cycle submodule and a final phase cycle submodule; Inputting the image to be detected into a stage cycle module in a preset deep residual network to obtain a feature map to be detected corresponding to the image to be detected, including: Inputting the image to be detected into the first stage periodic submodule, so as to process the image to be detected through the preset separation convolution attention module corresponding to the first stage periodic submodule to obtain a first feature map; Processing the first feature map by the preset separation convolution attention module corresponding to the second stage periodic submodule to obtain a second feature map; Processing the second feature map by the preset separation convolution attention module corresponding to the third stage periodic submodule to obtain a third feature map; Processing the third feature map by the preset separation convolution attention module corresponding to the fourth stage periodic submodule to obtain a fourth feature map; The fourth feature map is processed by the final stage periodic submodule to obtain the target feature map to be detected.

3. The method according to claim 2, characterized in that The preset separation convolution attention module includes a preset backbone network, a large core separation convolution attention processing module and a fully connected feedforward network, and the convolution layer in the fully connected feedforward network is a full convolution; Inputting the image to be detected into the first stage periodic submodule, so as to process the image to be detected by the preset separation convolution attention module corresponding to the first stage periodic submodule to obtain a first feature map, including: Inputting the image to be detected into the preset backbone network to obtain a first-stage feature map; Processing the first-stage feature map through the large-kernel separation convolution attention module to obtain a second-stage feature map; According to the first preset weight corresponding to the first stage feature map and the second preset weight corresponding to the second stage feature map, performing weighted averaging on the first stage feature map and the second stage feature map to obtain a third stage feature map; The third stage feature map is processed by the preset backbone network and the fully connected feed-forward network to obtain a first feature map corresponding to the image to be detected.

4. The method according to claim 3, characterized in that Processing the first stage feature map through the large kernel separation convolution attention module to obtain the second stage feature map includes: The large kernel separation convolution attention module processes the first stage feature map in the order of horizontal convolution and vertical convolution to obtain the second stage feature map.

5. The method according to claim 1, characterized in that Inputting the target feature map to be detected into the comprehensive processing module to obtain the feature vector corresponding to the image to be detected, including: Processing the target feature map to be detected by the first preset convolution module and the preset self-attention mechanism to obtain a first convolution feature map; Adding the first convolution feature map and the target feature map to be detected to obtain a second convolution feature map; Inputting the second convolution feature map to the other plurality of the preset convolution modules, so that the other plurality of the preset convolution modules process the feature map output by the previous preset convolution module to obtain a third convolution feature map; The second convolution feature map and the third convolution feature map are added to obtain a feature vector corresponding to the image to be detected.

6. The method according to claim 5, characterized in that Processing the target feature map to be detected by the first preset convolution module and the preset self-attention mechanism to obtain a first convolution feature map, including: Adjusting the dimension to be detected of the feature map to be detected according to the self-attention dimension corresponding to the preset self-attention mechanism; Based on the feature graph to be detected, obtaining a query vector, a key vector and a value vector corresponding to the preset self-attention mechanism; Based on the query vector, the key vector and the value vector, perform self-attention processing to obtain an initial first convolution feature map; The convolution dimension corresponding to the initial first convolution feature map is adjusted to the dimension to be detected to obtain the first convolution feature map.

7. The method according to claim 1, characterized in that The collected images are crop growth images; After detecting the target object based on the target image, the method further includes: When the value corresponding to the target detection result of the target object is greater than a preset detection threshold, the crop growth ground corresponding to the target detection result on the current target image is determined as a to-be-processed area, so as to process the to-be-processed area; When the value corresponding to the target detection result is not greater than the preset detection threshold, the current target image is discarded.

8. A target object detection device, characterized in that: include: A preprocessing module is configured to obtain the acquired image, segment the acquired image according to a preset image size, and obtain an image to be detected; A first processing module is configured to input the image to be detected into a stage cycle module in a preset deep residual network to obtain a target feature map to be detected corresponding to the image to be detected; A second processing module is configured to input the target feature map to be detected into a comprehensive processing module to obtain a feature vector corresponding to the image to be detected; a detection module, configured to determine the current image to be detected as a target image including a target object when the value of the feature vector is greater than a preset value, and detect the target object based on the target image; The stage cycle submodules in the stage cycle module are connected in sequence, and each stage cycle submodule includes a preset separation convolution attention module, and the preset separation convolution attention module is used to process the image to be detected to obtain a target feature map to be detected, so as to improve the accuracy of the feature map to be detected; The comprehensive processing module includes a preset self-attention mechanism and multiple preset convolution modules, the preset convolution modules are connected in sequence, the preset self-attention mechanism is configured in the first preset convolution module, and is used to process the target feature map to be detected, and the multiple preset convolution modules are used to obtain the feature vector corresponding to the image to be detected based on the target feature map to be detected.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images