Method, device and equipment for performing target detection on image, medium and product
By adding attention modules to the backbone model, the feature learning of small objects in the image is enhanced, and the problem of blurred features of small objects in the existing technology is solved, and the accuracy of detection of small objects in the image is improved.
Patent Information
- Application Number
- CN202411258628.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-27
AI Technical Summary
In the prior art, after multiple convolution and pooling operations of the image, the features of the small target may become blurred or even disappear, resulting in inaccurate recognition of small targets with insignificant features in the image.
The attention module is added to the residual module of the backbone model. Through global average pooling and weighting and activation operations of the fully connected layer, the learning of important spatial and channel features in the feature map is enhanced, and the accuracy of small and small object detection in the image is improved. Spend.
Through the use of attention module, the network can better grasp important features, improve the accuracy of detection of small and medium-sized objects in the image, and reduce the risk of feature blur.
Smart Images
Figure CN119206173A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to a method, device, equipment, medium and product for performing target detection on an image. Background Art
[0002] In the field of image processing and computer vision, object detection in images is an important means of extracting and understanding image information. In actual application scenarios, the targets detected in images appear in the form of small targets. The physical size of small targets in the image is extremely small, which directly leads to the limited number of pixels they occupy in the digital image. This limitation makes the feature representation of small targets extremely sparse, reduces the distinction between features, and increases the difficulty of pattern recognition.
[0003] The existing technology mainly uses convolutional neural networks to detect targets in images. After the image enters the neural network, it undergoes multiple convolution and pooling operations. The convolution operation extracts features in the image, and the pooling operation downsamples the feature map to reduce the spatial dimension of the data while retaining important features, and finally outputs the detection result.
[0004] With existing technologies, after an image enters a convolutional neural network, it will undergo multiple convolution and pooling operations. After multiple convolution and pooling operations, the features of small targets may become blurred or even disappear, making it more difficult to detect small targets in the image, and thus leading to inaccurate recognition of small targets with unclear features in the image. Summary of the invention
[0005] The embodiments of the present application provide a method, apparatus, device, medium and product for performing target detection on an image, so as to achieve the effect of accurately identifying small targets in an image.
[0006] In a first aspect, the present application provides a method for performing object detection on an image, comprising:
[0007] Acquire target image data, wherein the target image data includes pixel values of each pixel point in the image on multiple channels;
[0008] Inputting the target image data into a backbone model to extract a first image feature, the backbone model comprising the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module comprising a first branch, the second branch and a merging module, the first branch comprising the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, a CBM module, the second branch comprising: a CBM module, the input of the second branch being connected to the output of the first CBM in the first branch, the merging module being used to merge the outputs of the first branch and the second branch, the residual module comprising a CBM module, an attention module and a superposition module, the CBM module being used to perform convolution operations, normalization and mish activation processing;
[0009] Inputting the first image feature into a neck model to perform context analysis and feature fusion on the first image feature to obtain a second image feature;
[0010] The second image feature is input into a prediction model to predict target object information in the image, wherein the prediction model comprises a cascaded CBL module and a convolution module, and the CBL module is used for performing convolution, normalization and LeakyReLU activation processing.
[0011] In a possible implementation, the CBM module and the attention module in the residual module are located on the same branch or on different branches, and the input of the superposition module is connected to the outputs of the two branches for merging the outputs of the two branches.
[0012] In a possible implementation, the CBM module and the attention module in the residual module are cascaded in sequence to form a third branch, and the input of the superposition module is connected to the output of the third branch and the input of the residual module.
[0013] In a possible implementation, the CBM module in the residual module forms a third branch, the input of the attention module is the input of the residual module, and the input of the superposition module is connected to the output of the third branch and the output of the attention module.
[0014] In a possible implementation, the attention module includes a first submodule and / or a second submodule. When the first submodule and the second submodule exist at the same time, the attention module further includes a superposition submodule for superimposing outputs of the first submodule and the second submodule;
[0015] The processing process of the first submodule includes:
[0016] Receiving image feature data input into the attention module;
[0017] For each channel, performing global average pooling on the image feature data of all pixels corresponding to the channel to obtain an average pixel value of each channel;
[0018] Inputting the vector composed of the average pixel values corresponding to the plurality of channels into the first fully connected layer, performing weighting, activation and re-weighting operations in sequence, to obtain a first fully connected feature;
[0019] Normalizing the first fully connected features to obtain output data of the first submodule;
[0020] The processing process of the second submodule includes:
[0021] Receiving image feature data input into the attention module;
[0022] For each pixel, inputting the image feature data of the three channels of the pixel into the second fully connected layer for weighted operation to obtain a second fully connected feature;
[0023] The second fully connected features are normalized to obtain output data of the second submodule.
[0024] In a possible implementation, the method further includes:
[0025] During the training of the backbone model, the image feature data input into the attention module is corrected through the first fully connected feature and / or the second fully connected feature.
[0026] In a second aspect, the present application provides a device for performing object detection on an image, comprising:
[0027] An acquisition module, used to acquire target image data, wherein the target image data includes pixel values of each pixel point in the image on multiple channels;
[0028] A processing module, used for inputting the target image data into a backbone model to extract a first image feature, wherein the backbone model comprises the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module comprises a first branch, the second branch and a merging module, the first branch comprises the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, a CBM module, the second branch comprises: a CBM module, the input of the second branch is connected to the output of the first CBM in the first branch, the merging module is used to merge the outputs of the first branch and the second branch, the residual module comprises a CBM module, an attention module and a superposition module, and the CBM module is used to perform convolution operation, normalization and mish activation processing;
[0029] The processing module is further used to input the first image feature into the neck model to perform context analysis and feature fusion on the first image feature to obtain a second image feature;
[0030] The processing module is further used to input the second image feature into a prediction model to predict the target object information in the image, wherein the prediction model includes a cascaded CBL module and a convolution module, and the CBL module is used to perform convolution, normalization and LeakyReLU activation processing.
[0031] In a device for performing target detection on an image provided by the present application:
[0032] The CBM module and the attention module in the residual module are located on the same branch or on different branches, and the input of the superposition module is connected to the outputs of the two branches for merging the outputs of the two branches.
[0033] In a device for performing target detection on an image provided by the present application:
[0034] The CBM module and the attention module in the residual module are cascaded in sequence to form a third branch, and the input of the superposition module is connected to the output of the third branch and the input of the residual module.
[0035] In a device for performing target detection on an image provided by the present application:
[0036] The CBM module in the residual module forms a third branch, the input of the attention module is the input of the residual module, and the input of the superposition module is connected to the output of the third branch and the output of the attention module.
[0037] In a device for performing target detection on an image provided by the present application:
[0038] The processing process of the first submodule includes:
[0039] An acquisition module, used to receive image feature data input into the attention module;
[0040] A processing module, for each channel, for performing global average pooling on the image feature data of all pixels corresponding to the channel to obtain an average pixel value of each channel;
[0041] The processing module is further used to input the vector composed of the average pixel values corresponding to the plurality of channels into the first fully connected layer, and perform weighting, activation and re-weighting operations in sequence to obtain a first fully connected feature;
[0042] A processing module, used for normalizing the first fully connected features to obtain output data of the first submodule;
[0043] The processing process of the second submodule includes:
[0044] An acquisition module, used to receive image feature data input into the attention module;
[0045] For each pixel, the processing module is used to input the image feature data of the three channels of the pixel into the second fully connected layer for weighted operation to obtain a second fully connected feature;
[0046] A processing module is used to normalize the second fully connected features to obtain output data of the second submodule.
[0047] In a device for performing target detection on an image provided by the present application:
[0048] A processing module is used to correct the image feature data input into the attention module through the first fully connected feature and / or the second fully connected feature during the training of the backbone model.
[0049] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;
[0050] The memory stores computer-executable instructions;
[0051] The processor executes the computer-executable instructions stored in the memory to implement a method for performing object detection on an image as described in any one of the first aspects.
[0052] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement a method for performing target detection on an image as described in any one of the first aspects.
[0053] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements a method for detecting an object in an image as described in any one of the first aspects.
[0054] The method, apparatus, device, medium and product for performing target detection on an image provided by the embodiments of the present application obtain target image data, the target image data includes pixel values of each pixel point in the image on multiple channels, and then inputs the target image data into a backbone model to extract a first image feature, the backbone model includes the following modules cascaded in sequence: a CBM module, multiple CSP modules, the CSP module includes a first branch, a second branch and a merging module, the first branch includes the following modules cascaded in sequence: two CBM modules, multiple residual modules, a CBM module, the second branch includes: a CBM module, the input of the second branch is connected to the The output of the first CBM in the first branch, the merging module is used to merge the outputs of the first branch and the second branch, the residual module includes a CBM module, an attention module and an overlay module, the CBM module is used to perform convolution operations, normalization and mish activation processing, the first image feature is input into the neck model to perform context analysis and feature fusion on the first image feature to obtain the second image feature, the second image feature is input into the prediction model to predict the target object information in the image, the prediction model includes a cascaded CBL module and a convolution module, the CBL module is used to perform convolution, normalization and LeakyReLU activation processing. Compared with the prior art, the convolutional neural network is used to detect targets in images, so that the features of small targets may become blurred or even disappear after multiple convolution and pooling operations, thereby resulting in inaccurate recognition of small targets with unclear features in the image. In this application, an attention module is added to the residual module of the backbone model, and the attention module can enhance the learning of important spatial and channel features in the feature map, so that the network can learn to focus on the key points during training, thereby improving the accuracy of detecting small targets in the image. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0056] Figure 1 A schematic diagram of a flow chart of a method for performing target detection on an image provided in this embodiment;
[0057] Figure 2 It is a schematic diagram of the backbone model;
[0058] Figure 3 This is a schematic diagram of the CSP module;
[0059] Figure 4 A schematic diagram showing that the CBM module and the attention module in the residual module are located on the same branch;
[0060] Figure 5Schematic diagram of the CBM module and attention module in the residual module located on different branches;
[0061] Figure 6 A device for performing target detection on an image provided in this embodiment;
[0062] Figure 7 A schematic diagram of the structure of an electronic device embodiment 1 provided in an embodiment of the present application.
[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0066] In the prior art, after the image undergoes multiple convolution operations and pooling operations, the features of small targets may become blurred or even disappear, which increases the difficulty of detecting small targets in the image, resulting in inaccurate recognition of small targets with unclear features in the image.
[0067] Based on this, the inventive concept of this application is to provide a method for target detection in images, by embedding an attention module in the network to enhance the learning of important spatial and channel features in the feature map, so that the network can learn to grasp the key points during training and improve the accuracy of detecting small targets in the image.
[0068] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0069] Figure 1 A flowchart of a method for detecting an object in an image provided by this embodiment is shown in FIG. Figure 1 As shown, the method comprises the following steps:
[0070] S101: Acquire target image data, where the target image data includes pixel values of each pixel point in the image on multiple channels.
[0071] In this embodiment, an image for target detection, i.e., target image data, is first obtained. The target image data includes pixel values of each pixel in the image on multiple channels. For example, if the target image is an RGB image, the pixel values of each pixel in the image on multiple channels refer to the pixel value of each pixel in the red channel, the pixel value of the green channel, and the pixel value of the blue channel.
[0072] S102: Input the target image data into the backbone model to extract the first image feature, the backbone model includes the following modules cascaded in sequence: a CBM module, multiple CSP modules, the CSP module includes a first branch, a second branch and a merging module, the first branch includes the following modules cascaded in sequence: two CBM modules, multiple residual modules, a CBM module, the second branch includes: a CBM module, the input of the second branch is connected to the output of the first CBM in the first branch, the merging module is used to merge the outputs of the first branch and the second branch, the residual module includes a CBM module, an attention module and a superposition module, and the CBM module is used to perform convolution operations, normalization and mish activation processing.
[0073] In this embodiment, the target image data is input into the backbone model to extract the first image feature. The backbone model is composed of a CBM module and a plurality of CSP modules cascaded in sequence.
[0074] For example, if Figure 2 Shown is a schematic diagram of a backbone model, which may be composed of a CBM module, a CSP1 module, a CSP2 module, a CSP8 module, a CSP8 module and a CSP4 module that are cascaded in sequence.
[0075] like Figure 3The figure shows a schematic diagram of a CSP module. The CSP module consists of two branches and a merging module. The first branch includes two CBM modules, multiple residual modules, and a CBM module that are cascaded in sequence. The second branch includes a CBM module. The input of the second branch is connected to the output of the first CBM in the first branch. The merging module is used to merge the outputs of the first branch and the second branch. Among them, CSPX represents the number of residual modules in the CSP module, CSP1 represents that there is 1 residual module in the CSP module, CSP2 represents that there are 2 residual modules in the CSP module, CSP8 represents that there are 8 residual modules in the CSP module, and CSP4 represents that there are 4 residual modules in the CSP module. Among them, the residual module includes a CBM module, an attention module, and a superposition module. The CBM module is used to perform convolution operations, normalization, and mish activation processing.
[0076] In the above step S102, the residual module includes a CBM module, an attention module and a superposition module. The CBM module and the attention module in the residual module are located on the same branch or on different branches. The input of the superposition module is connected to the outputs of the two branches for merging the outputs of the two branches.
[0077] In this embodiment, there are multiple residual modules in the CSP module of the backbone model, and by adding an attention module to the residual module, the network's ability to learn important features can be enhanced, while suppressing weak features and improving detection accuracy. In this embodiment, the residual module includes a CBM module, an attention module, and an overlay module. The CBM module and the attention module in the residual module are located on the same branch, or the CBM module and the attention module in the residual module are located on different branches. The input of the overlay module is connected to the output of the two branches for merging the outputs of the two branches.
[0078] In a possible implementation, the CBM module and the attention module in the residual module are cascaded in sequence to form a third branch, and the input of the superposition module is connected to the output of the third branch and the input of the residual module.
[0079] In this embodiment, if Figure 4 The figure shows that the CBM module and the attention module in the residual module are located on the same branch. Figure 4 As shown, the residual module includes two branches, one of which has a CBM module, a CBM module and an attention module cascaded in sequence, and the input of this branch is the input of the residual module. The other branch has no other modules and the input is the input of the residual module. Both branches are connected to the superposition module as the input of the superposition module.
[0080] In another possible implementation, the CBM module in the residual module forms a third branch, the input of the attention module is the input of the residual module, and the input of the superposition module is connected to the output of the third branch and the output of the attention module.
[0081] In this embodiment, if Figure 5 The figure shows a schematic diagram of the CBM module and the attention module in the residual module located on different branches. Figure 5 As shown, the residual module includes two branches, one of which has a CBM module and a CBM module cascaded in sequence, and the input of this branch is the input of the residual module, and the other branch is an attention module and the input is the input of the residual module. Both branches are connected to the superposition module as the input of the superposition module.
[0082] The attention module added to the residual module includes a first submodule and / or a second submodule. When the first submodule and the second submodule exist at the same time, the attention module also includes a superposition submodule that superimposes the outputs of the first submodule and the second submodule.
[0083] The processing of the first submodule includes:
[0084] Receive image feature data that is input into the attention module.
[0085] In this embodiment, the image feature data of the first submodule input into the attention module is received,
[0086] U a =[u1,u2,…u c ]
[0087] Among them, U a represents the image feature data of the first submodule input to the attention module, u i ∈R w×h , w represents the width of the target image, h represents the height of the target image, and c represents the number of channels.
[0088] For each channel, the image feature data of all pixels corresponding to the channel are globally averaged and pooled to obtain the average pixel value of each channel.
[0089] In this embodiment, the obtained image feature data is globally averaged pooled according to the channel, that is, for each channel, the image feature data of all pixels corresponding to the channel are globally averaged pooled to obtain the average pixel value of each channel. Among them, the image feature data of all pixels corresponding to the channel are globally averaged pooled as follows:
[0090]
[0091] Among them, z krepresents the average pixel value of the kth channel, u k (i, j) represents the pixel value at the (i, j) position on the k-th channel, i represents the height of each parameter on the feature map of the k-th channel, and j represents the width of each parameter on the feature map of the k-th channel.
[0092] The vector composed of the average pixel values corresponding to multiple channels is input into the first fully connected layer, and weighted, activated and weighted again are performed in sequence to obtain the first fully connected feature.
[0093] In this embodiment, the average pixel value z corresponding to multiple channels is k The vector form z is formed, and the vector composed of the average pixel values corresponding to multiple channels is input into the first fully connected layer for weighting, activation and re-weighting operations to obtain the first fully connected feature, where the first fully connected layer is:
[0094]
[0095] in, They represent the weights of the fully connected layer, d(g) represents the activation, and the optional ReLU activation function. represents the first fully connected feature.
[0096] The first fully connected feature is normalized to obtain the output data of the first submodule.
[0097] In this embodiment, the first fully connected feature After normalization by sigmoid function, we get The value range of is changed to [0,1].
[0098] In this embodiment, the first submodule is a channel attention mechanism, which can automatically learn the importance of each channel and adjust the representation of input data according to the contribution of the channel. By assigning different weights to each channel, the channel attention mechanism can enhance useful features and suppress useless or redundant features, thereby improving the detection accuracy of small objects in the image.
[0099] The processing of the second submodule includes:
[0100] Receive image feature data that is input into the attention module.
[0101] In this embodiment, the image feature data of the second submodule input into the attention module is received,
[0102] U b =[u 1,1 ,u 1,2 ,…,u i,j ,…,uH,W ]
[0103] Among them, U b represents the image feature data of the second submodule input to the attention module, u i,j ∈R 1×1×C , H represents the height of the input feature map, W represents the width of the input feature map, i∈{1, 2, ..., H}, j∈{1, 2, ..., W}, u i,j Represents the feature information of all channels at the position (i, j) of the feature map.
[0104] For each pixel, the image feature data of the three channels of the pixel are input into the second fully connected layer for weighted operation to obtain the second fully connected feature.
[0105] In this embodiment, for each pixel, the image feature data of the three channels of the pixel are input into the second fully connected layer for weighted operation, where the second fully connected layer is:
[0106] q=W sq *U b
[0107] Among them, W sq ∈R 1×1×C×1 represents the weight of the second fully connected layer, and q represents the second fully connected feature.
[0108] The second fully connected feature is normalized to obtain the output data of the second submodule.
[0109] In this embodiment, the second fully connected feature q is normalized by a sigmoid function to obtain s(q), and the value range of s(q) is transformed to [0, 1].
[0110] In this embodiment, the second submodule is a spatial attention mechanism. The second submodule can focus on local features. By dynamically adjusting the importance of different spatial positions of the input data, the second submodule can enhance useful information and suppress useless information, thereby improving the model's detection accuracy for small targets in the image.
[0111] After obtaining the first fully connected feature and the second fully connected feature as described above, during the training of the backbone model, the image feature data input into the attention module is corrected using the first fully connected feature and / or the second fully connected feature.
[0112] In this embodiment, after the image feature is processed by the first module to obtain the first fully connected feature, the first fully connected feature is normalized to obtain the output data of the first submodule for re-correcting the image feature data input to the first submodule:
[0113]
[0114] in, represents the image feature data of the first submodule after correction, F cSE represents the first submodule correction function, It represents the normalized value of the first fully connected feature, and σ represents the sigmoid function.
[0115] After the image features are processed by the second module to obtain the second fully connected features, the second fully connected features are normalized to obtain the output data of the second submodule for re-correcting the image feature data input to the second submodule:
[0116]
[0117] in, represents the image feature data of the second submodule after correction, F sSE represents the second submodule correction function, σ(q 1,1 ) represents the normalized value of the second fully connected feature, and σ represents the sigmoid function.
[0118] The attention module also includes a superposition submodule for superimposing the outputs of the first submodule and the second submodule. By superimposing the outputs of the first submodule and the second submodule, the input image feature data can be recalibrated in terms of space and channel at the same time. The recalibration of channel features by the first submodule and the enhancement of spatial information by the second submodule work together to make feature expression more accurate and efficient:
[0119]
[0120] in, Indicates the superposition value of the corrected image feature data of the first submodule and the image feature data of the second submodule.
[0121] S103: Input the first image feature into the neck model to perform context analysis and feature fusion on the first image feature to obtain a second image feature.
[0122] In this embodiment, the first image feature output by the backbone model is input into the neck model to perform context analysis and feature fusion on the first image feature to obtain the second image feature. The main purpose of the neck model is to fuse the features of different levels extracted by the backbone model to enhance the semantic information and spatial information of the features.
[0123] S104: Inputting the second image feature into the prediction model to predict the target object information in the image, the prediction model includes a cascaded CBL module and a convolution module, and the CBL module is used for convolution, normalization and LeakyReLU activation processing.
[0124] In this embodiment, the second image feature obtained by the neck model is input into the prediction model, and the prediction model is used to output the target object information, wherein the prediction model is composed of a cascaded CBL module and a convolution module, wherein the CBL module is used to perform convolution, normalization and LeakyReLU activation processing.
[0125] In this embodiment, target image data is obtained, the target image data includes pixel values of each pixel point in the image on multiple channels, and then the target image data is input into the backbone model to extract the first image feature, the backbone model includes the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module includes a first branch, a second branch and a merging module, the first branch includes the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, a CBM module, the second branch includes: a CBM module, the input of the second branch is connected to the output of the first CBM in the first branch The merging module is used to merge the outputs of the first branch and the second branch. The residual module includes a CBM module, an attention module and an overlay module. The CBM module is used to perform convolution operations, normalization and mish activation processing. The first image feature is input into the neck model to perform context analysis and feature fusion on the first image feature to obtain the second image feature. The second image feature is input into the prediction model to predict the target object information in the image. The prediction model includes a cascaded CBL module and a convolution module. The CBL module is used to perform convolution, normalization and LeakyReLU activation processing. Compared with the prior art, the convolutional neural network is used to detect targets in images, so that the features of small targets may become blurred or even disappear after multiple convolution and pooling operations, thereby resulting in inaccurate recognition of small targets with unclear features in the image. In this application, an attention module is added to the residual module of the backbone model. The attention module can enhance the learning of important spatial and channel features in the feature map, so that the network can learn to focus on the key points during training and improve the accuracy of detecting small targets in the image.
[0126] Figure 6 The present embodiment provides a device for performing target detection on an image. The present application provides a device for performing target detection on an image, comprising: an acquisition module 61 and a processing module 62; wherein:
[0127] The acquisition module 61 is used to acquire target image data, and the target image data includes pixel values of each pixel point in the image on multiple channels; the processing module 62 is used to input the target image data into the backbone model to extract the first image feature, and the backbone model includes the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module includes a first branch, a second branch and a merging module, the first branch includes the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, and a CBM module, the second branch includes: a CBM module, the input of the second branch is connected to the output of the first CBM in the first branch, and the merging module The block is used to merge the outputs of the first branch and the second branch. The residual module includes a CBM module, an attention module and an overlay module. The CBM module is used to perform convolution operations, normalization and mish activation processing; the processing module 62 is also used to input the first image feature into the neck model to perform context analysis and feature fusion on the first image feature to obtain the second image feature; the processing module 62 is also used to input the second image feature into the prediction model to predict the target object information in the image. The prediction model includes a cascaded CBL module and a convolution module. The CBL module is used to perform convolution, normalization and LeakyReLU activation processing.
[0128] Figure 7 A schematic diagram of the structure of an electronic device embodiment 1 provided in the present application. Figure 7 The electronic device provided in this embodiment includes: at least one processor 71 and a memory 72. The processor 71 and the memory 72 are connected via a bus 73.
[0129] The specific implementation process of the processor 71 can be found in the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here in this embodiment.
[0130] In the above Figure 7 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0131] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0132] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0133] The present application also provides a computer-readable storage medium having computer-executable instructions stored thereon. When a processor executes the computer-executable instructions, a method for detecting an object in an image as described above is implemented.
[0134] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general or special purpose computer.
[0135] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0136] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0138] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0139] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0140] The present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the above method for detecting an object in an image.
[0141] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0142] It should be further noted that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flow chart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0143] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.
[0144] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.
[0145] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.
[0146] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0148] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0149] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for detecting an object in an image, characterized in that: include: Acquire target image data, wherein the target image data includes pixel values of each pixel point in the image on multiple channels; Inputting the target image data into a backbone model to extract a first image feature, the backbone model comprising the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module comprising a first branch, a second branch and a merging module, the first branch comprising the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, a CBM module, the second branch comprising: a CBM module, the input of the second branch being connected to the output of the first CBM in the first branch, the merging module being used to merge the outputs of the first branch and the second branch, the residual module comprising a CBM module, an attention module and a superposition module, the CBM module being used to perform convolution operations, normalization and mish activation processing; Inputting the first image feature into a neck model to perform context analysis and feature fusion on the first image feature to obtain a second image feature; The second image feature is input into a prediction model to predict target object information in the image, wherein the prediction model comprises a cascaded CBL module and a convolution module, and the CBL module is used for performing convolution, normalization and LeakyReLU activation processing.
2. The method according to claim 1, characterized in that The CBM module and the attention module in the residual module are located on the same branch or on different branches, and the input of the superposition module is connected to the outputs of the two branches for merging the outputs of the two branches.
3. The method according to claim 2, characterized in that The CBM module and the attention module in the residual module are cascaded in sequence to form a third branch, and the input of the superposition module is connected to the output of the third branch and the input of the residual module.
4. The method according to claim 2, characterized in that: The CBM module in the residual module forms a third branch, the input of the attention module is the input of the residual module, and the input of the superposition module is connected to the output of the third branch and the output of the attention module.
5. The method according to any one of claims 1 to 4, characterized in that The attention module includes a first submodule and / or a second submodule. When the first submodule and the second submodule exist at the same time, the attention module further includes a superposition submodule for superimposing outputs of the first submodule and the second submodule; The processing process of the first submodule includes: Receiving image feature data input into the attention module; For each channel, performing global average pooling on the image feature data of all pixels corresponding to the channel to obtain an average pixel value of each channel; Inputting the vector composed of the average pixel values corresponding to the plurality of channels into the first fully connected layer, performing weighting, activation and re-weighting operations in sequence, to obtain a first fully connected feature; Normalizing the first fully connected features to obtain output data of the first submodule; The processing process of the second submodule includes: Receiving image feature data input into the attention module; For each pixel, inputting the image feature data of the three channels of the pixel into the second fully connected layer for weighted operation to obtain a second fully connected feature; The second fully connected features are normalized to obtain output data of the second submodule.
6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: During the training of the backbone model, the image feature data input into the attention module is corrected through the first fully connected feature and / or the second fully connected feature.
7. A device for performing target detection on an image, characterized in that: include: An acquisition module, used to acquire target image data, wherein the target image data includes pixel values of each pixel point in the image on multiple channels; A processing module, used for inputting the target image data into a backbone model to extract a first image feature, wherein the backbone model comprises the following modules cascaded in sequence: a CBM module, a plurality of CSP modules, the CSP module comprises a first branch, a second branch and a merging module, the first branch comprises the following modules cascaded in sequence: two CBM modules, a plurality of residual modules, a CBM module, the second branch comprises: a CBM module, the input of the second branch is connected to the output of the first CBM in the first branch, the merging module is used to merge the outputs of the first branch and the second branch, the residual module comprises a CBM module, an attention module and a superposition module, and the CBM module is used to perform convolution operation, normalization and mish activation processing; The processing module is further used to input the first image feature into the neck model to perform context analysis and feature fusion on the first image feature to obtain a second image feature; The processing module is further used to input the second image feature into a prediction model to predict the target object information in the image, wherein the prediction model includes a cascaded CBL module and a convolution module, and the CBL module is used to perform convolution, normalization and LeakyReLU activation processing.
8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.
10. A computer program product, characterized in that The method comprises a computer program, which is used to implement the method according to any one of claims 1 to 6 when the computer program is executed by a processor.