Image detection method, device, equipment and readable medium

Abstract

The embodiment of the present application provides a kind of image detection method, device, equipment, readable medium.It relates to computer image detection field.The image detection method includes: step 1, obtaining the image to be detected;Step 2, the image is processed through the backbone network of YOLO model, and the extracted feature is generated;The backbone network includes C3_CA module;Step 3, the feature is used as the input of the neck network of YOLO model, and feature fusion processing is carried out, to obtain feature map;Step 4, the feature map is detected by the detection head of YOLO model, and the attribute of the target to be detected is predicted.The present application can improve the detection accuracy of image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer image detection, and particularly relates to an image detection method, device and equipment and readable medium. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicles and aerospace technology, the detection and identification of targets in remote sensing images (RSI) has become a key research field for collecting ground information, and has important military, security and environmental protection application values.

[0003] Existing remote sensing image detection algorithms can be divided into traditional algorithms and deep learning-based algorithms.

[0004] Traditional target detection algorithms often need to perform complex feature engineering on input remote sensing images in advance, and design convolution kernels containing a large number of hyperparameters, which greatly challenges the ease of use of the algorithm.

[0005] And the deep learning-based target detection algorithm is usually end-to-end, and the model can autonomously learn and update its own parameters, and no longer needs human setting of the parameters of the model. Therefore, in recent years, the deep learning-based target detection algorithm has become popular and a batch of excellent models have emerged. Among them, YOLO is widely used because of its high detection accuracy and response speed.

[0006] However, the basic YOLO model has a large number of model parameters while improving the detection accuracy, and with the increase of network layers, the features of small targets are easy to be lost, leading to missed detection and false detection, so that the small target detection performance on remote sensing images is poor. SUMMARY

[0007] Embodiments of the present application provide an image detection method, device and equipment and readable medium in the field of computer image detection, and provide the accuracy of image detection.

[0008] In one aspect, an image detection method is provided, comprising:

[0009] Step 1, obtaining an image to be detected;

[0010] Step 2, processing the image through a backbone network of a YOLO model to generate extracted features; the backbone network comprises a C3_CA module;

[0011] Step 3, taking the features as input of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map;

[0012] Step 4, detecting the feature map through a detection head of the YOLO model to predict attributes of the target to be detected;

[0013] Optionally, step 2 is specifically:

[0014] Step 21, the image is sequentially processed by two convolution modules Conv and a C3_CA module to obtain the first output of the backbone network as the shallow feature of the image;

[0015] Step 22, the extracted shallow feature is sequentially processed by a convolution module and a C3_CA module to obtain the second output of the backbone network as the intermediate feature of the image;

[0016] Step 23, the intermediate feature is sequentially processed by a convolution module, a C3_CA module, a convolution module, a C3_CA module and a SPPF module to obtain the third output of the backbone network as the deep feature of the image;

[0017] The shallow feature, the intermediate feature and the deep feature of the image are taken as the extracted features.

[0018] Optionally, the C3_CA module is specifically:

[0019] Suppose the input of the C3_CA module is f, the first branch of the input of the C3_CA module is sequentially processed by a CBS module and a CA module to obtain the output That is:

[0020]

[0021] The second branch of the input of the C3_CA module is processed by a CBS module alone to obtain the output That is:

[0022]

[0023] Then, And After being spliced by a Concat module and then processed by a CBS module, the final output P is obtained 2 That is:

[0024]

[0025] Optionally, the CA module is specifically:

[0026] Suppose the input of the CA module is the feature tensor p extracted by the preceding module, the result after the calculation of the coordinate attention of the feature tensor p is residual connected with the feature tensor p, and the obtained result is the output O of the CA module, that is:

[0027] O = Residual(p, Coordinate_attention(p)).

[0028] Optionally, the CA module specifically is:

[0029] Suppose the size of the feature tensor p is C x H x W, the feature tensor p is first subjected to an average pooling in the x direction and an average pooling in the y direction respectively, to obtain an output O x and O y , that is:

[0030] O x = XAP(p)

[0031] O y = YAP(p)

[0032] O x , the size of O y is C x 1 x W; then O x and O y are spliced and subjected to a convolution module to obtain an output O1, the size of O1 is C / r x 1 x (W+H), r is a scaling factor, that is:

[0033] O1 = Conv2d(Concat(O x ,O y ));

[0034] Then, O1 is subjected to a batch normalization module and a nonlinear activation module to obtain O2, the size of O2 is C / r x 1 x (W+H), that is:

[0035] O2 = BN(NL(O1))

[0036] Next, O2 is split into x-direction and y-direction feature tensors O' x and O' y with sizes of C / r x H x 1 and C / r x 1 x W respectively, then O' x and O' y are respectively subjected to a convolution module and a Sigmoid activation module to obtain and , that is:

[0037]

[0038] After the above processing, and are the attention weight matrices corresponding to the X direction and the Y direction respectively, with sizes of C x 1 x H and C x 1 x W respectively;

[0039] Finally, the generated weight matrices are applied to the X and Y directions of the original input features, respectively, and pixel-by-pixel multiplication is performed through a broadcast mechanism to achieve feature reweighting; that is:

[0040]

[0041] The coordinate attention calculation result is residually concatenated with the original input to obtain the final output O of the CA module:

[0042] O=Residual(p,Coordinate_attention(p))

[0043] The size of O is C×H×W, consistent with the original input features, and includes features enhanced by channel attention and spatial coordinate information. Here, C, H, and W are the three dimensions of the output O.

[0044] Optionally, step 3 specifically includes:

[0045] Suppose that shallow feature f1, intermediate feature f2, and deep feature f3 are the inputs to the three branches of the neck network, respectively;

[0046] First, it is processed by a convolutional module to obtain the output O. 1 ,Right now:

[0047]

[0048] and Simultaneously serving as input to the ASFF2 module for both the first and second branches, it yields output O. 2 ,Right now:

[0049]

[0050] Outputs of the two ASFF2 modules Simultaneously serving as the input to module C3 for both the first and second branches, it yields output O. 3 ,Right now:

[0051]

[0052] It is then processed again by a convolutional module to obtain... Right now:

[0053]

[0054] at last, Simultaneously, as input to the ASFF3 module in the three branches, after fusion, the final output F of the neck network is obtained, namely:

[0055]

[0056] Optionally, the attribute of the to-be-detected target includes one or more of a position, a size, and a category of the to-be-detected target.

[0057] In another aspect, an image detection device is provided, comprising:

[0058] an acquisition module configured to acquire an image to be detected;

[0059] a first processing module configured to process the image through a backbone network of a YOLO model to generate extracted features;

[0060] a second processing module configured to take the features as an input of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map;

[0061] a detection module configured to detect the feature map through a detection head of the YOLO model to predict an attribute of a to-be-detected target;

[0062] The first processing module comprises:

[0063] a first processing submodule configured to sequentially process the image through two convolution modules Conv and a C3_CA module to obtain a first output of the backbone network as a shallow feature of the image;

[0064] a second processing submodule configured to sequentially process the extracted shallow feature through a convolution module and a C3_CA module to obtain a second output of the backbone network as an intermediate feature of the image;

[0065] a third processing submodule configured to sequentially process the intermediate feature through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain a third output of the backbone network as a deep feature of the image;

[0066] a fourth processing submodule configured to take the shallow feature, the intermediate feature, and the deep feature of the image as the extracted features.

[0067] In another aspect, an intelligent device is provided, comprising the device.

[0068] In another aspect, an electronic device is provided, comprising a memory, a processor, a communication interface, and a communication bus, the memory storing a computer program executable on the processor, the memory, the processor, and the communication interface being in communication through the communication bus, and the processor implementing the data storage and reading method when executing the computer program.

[0069] In another aspect, a computer readable medium having non-transitory program code executable by a processor is provided, the program code causing the processor to perform the method.

[0070] As can be seen from the technical solutions provided by the above-mentioned embodiments of the present application, step 1, an image to be detected is acquired; step 2, the image is processed through a backbone network of a YOLO model to generate extracted features; the backbone network comprises a C3_CA module; step 3, the features are taken as inputs of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map; and step 4, the feature map is detected through a detection head of the YOLO model to predict attributes of the target to be detected.

[0071] Step 2 specifically comprises: step 21, the image is sequentially processed through two convolution modules Conv and a C3_CA module to obtain a first output of the backbone network as shallow features of the image; step 22, the shallow features are sequentially processed through a convolution module and a C3_CA module to obtain a second output of the backbone network as intermediate features of the image; and step 23, the intermediate features are sequentially processed through a convolution module, a C3_CA module, a convolution module, a C3_CA module and an SPPF module to obtain a third output of the backbone network as deep features of the image; and the shallow features, the intermediate features and the deep features of the image are taken as the extracted features. By introducing the C3_CA module, the present application alleviates feature disappearance in some scenes and improves the accuracy of image detection.

[0072] Additional aspects and advantages of the present application will be described in the following description, will become apparent from the following description, or will be learned by practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0074] Figure 1 It is a flowchart of the image detection method in the present application.

[0075] Figure 2 It is a first schematic diagram of the overall processing flow of the YOLO model in the present application.

[0076] Figure 3 It is a second schematic diagram of the overall processing flow of the YOLO model in the present application.

[0077] Figure 4 This is a schematic diagram illustrating the specific implementation of the backbone network of the YOLO model of the present invention;

[0078] Figure 5 This is a schematic diagram illustrating the specific implementation of the neck network in the YOLO model of the present invention;

[0079] Figure 6 This is a schematic diagram illustrating the specific implementation of the C3_CA module in the YOLO model of this invention;

[0080] Figure 7 This is a schematic diagram illustrating the specific implementation of the CA module in the C3_CA module of the YOLO model of the present invention;

[0081] Figure 8 This is a schematic diagram of the connection of the image detection device in this invention. Detailed Implementation

[0082] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0083] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0084] Figure 1 This is a schematic flowchart of the image detection method in this invention; Figure 2 This is a first schematic diagram of the overall processing flow of the YOLO model in this invention; Figure 3 This is a second schematic diagram illustrating the overall processing flow of the YOLO model in this invention; Figure 4 This is a schematic diagram illustrating the specific implementation of the backbone network of the YOLO model of the present invention; Figure 5 This is a schematic diagram illustrating the specific implementation of the neck network in the YOLO model of the present invention; Figure 6 This is a schematic diagram illustrating the specific implementation of the C3_CA module in the YOLO model of this invention; Figure 7 This is a schematic diagram illustrating the specific implementation of the CA module, a component in the C3_CA module of the YOLO model of the present invention. Figure 8 This is a schematic diagram showing the connection of the image detection device in this invention. The following description is in conjunction with the figures.

[0085] like Figure 1 As shown, an image detection method according to the present invention includes:

[0086] Step 1, obtaining an image to be detected; wherein the image can be a remote sensing image or the like.

[0087] Step 2, as shown in the figure, processing the image through a backbone network of a YOLO model to generate extracted features; the backbone network comprises a C3_CA module; this step specifically comprises extracting rich semantic information of the image through the backbone network, and outputting a feature tensor. Figure 4

[0088] Step 3, as shown in the figure, taking the features as input of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map; this step specifically comprises performing fusion and deep processing of the features through the neck network, and outputting the feature map. Figure 5

[0089] Step 4, detecting the feature map through a detection head of the YOLO model to predict attributes of the target to be detected; optionally, the attributes of the target to be detected comprise one or more of the position, size and category of the target to be detected in any combination; the target to be detected can be a vehicle or the like, which is set according to actual conditions. This step specifically comprises predicting and outputting the position, size and category of the detected target according to the information of the feature map and the related information of the target to be detected set in advance.

[0090] Optionally, as shown in the figure, step 2 specifically comprises: Figure 4

[0091] Step 21, processing the image through two convolution modules Conv and a C3_CA module in sequence to obtain a first output of the backbone network as a shallow feature of the image;

[0092] Step 22, processing the extracted shallow feature through a convolution module and a C3_CA module in sequence to obtain a second output of the backbone network as an intermediate feature of the image;

[0093] Step 23, processing the intermediate feature through a convolution module, a C3_CA module, a convolution module, a C3_CA module and a SPPF module in sequence to obtain a third output of the backbone network as a deep feature of the image;

[0094] Taking the shallow feature, the intermediate feature and the deep feature of the image as the extracted features.

[0095] As shown in the figure, this step specifically comprises: Figure 4

[0096] ​​​​1. The image is a tensor k with a size of 640*640*3; 2. k is extracted by a convolution kernel to obtain basic edge and texture information of the image, and a tensor k1 with a size of 64*320*320 is output;

[0097] 3. k1 is deepened by a convolution kernel to obtain shallow feature information of the image, and a tensor k2 with a size of 128*160*160 is output;

[0098] 4. k2 is strengthened by introducing coordinate attention to express position information of the shallow features in the image, and a tensor f1 with a size of 128*160*160 is output;

[0099] 5. f1 is extracted by a convolution kernel to obtain more intermediate features of the image, such as the shape and contour of the target, and a tensor k3 with a size of 256*80*80 is output;

[0100] 6. k3 is strengthened by introducing coordinate attention to express position information of the intermediate features in the image, and a tensor f2 with a size of 256*80*80 is output;

[0101] 7. f2 is extracted by a convolution kernel to obtain high-level semantic features of the image, such as the category and relationship of the target, and a tensor k4 with a size of 512*40*40 is output;

[0102] 8. k4 is strengthened by introducing coordinate attention to express position information of the high-level semantic features in the image, and a tensor k5 with a size of 512*40*40 is output;

[0103] 9. k5 is extracted by a convolution kernel to obtain deep abstract semantic features of the image, and a tensor k6 with a size of 1024*20*20 is output;

[0104] 10. k6 is strengthened by introducing coordinate attention to express position information of the deep abstract semantic features in the image, and a tensor k7 with a size of 1024*20*20 is output;

[0105] 11. k7 is aggregated by an SPPF module to obtain semantic feature information in the image, and a tensor f3 with a size of 1024*20*20 is output.

[0106] Optionally, as shown in Figure 6 , the C3_CA module specifically comprises:

[0107] Suppose the input of the C3_CA module is f, the first branch of the input of the C3_CA module is processed by a CBS module and a CA module in turn, and the output is , that is:

[0108]

[0109] The second branch of the input of the C3_CA module is processed by a CBS module alone to obtain an output That is,

[0110]

[0111] Then, And After being spliced by a Concat module and then processed by a CBS module, the final output P is obtained 2 That is,

[0112]

[0113] As shown in Figure 6 , the C3_CA module specifically comprises:

[0114] 1. The feature tensor f further extracts target semantic information in the CBS module, normalizes the feature tensor to reduce changes in data distribution, and then introduces a nonlinear factor through an activation function to output two feature tensors that contain more semantic information, have more stable data distribution, and contain more complex information;

[0115] 2. The feature tensor output by the CBS module is processed by the CA module, the spatial information of the feature tensor is enhanced by calculating the coordinate attention, and a feature map that can better reflect the spatial distribution and global position of the target after coordinate weighting is output ;

[0116] 3. And After being adjusted in dimension by a Concat operation, a feature map containing more information is output; 4. The feature map further extracts target semantic information in the CBS module, normalizes the feature tensor to reduce changes in data distribution, and then introduces a nonlinear factor through an activation function to output a feature tensor P2 that contains more semantic information, has more stable data distribution, and contains more complex information.

[0117] Optionally, the CA module specifically comprises:

[0118] Suppose the input of the CA module is the feature tensor p extracted by the preceding module, the result after the feature tensor p is processed by calculating the coordinate attention is residual connected with the feature tensor p, and the result is the output O of the CA module, that is,

[0119] O = Residual (p, Coordinate_attention (p)).

[0120] Optionally, as shown in Figure 7 , the CA module specifically comprises:

[0121] Suppose the size of the feature tensor p is C x H x W, the feature tensor p is first respectively subjected to an x-direction average pooling and a y-direction average pooling to obtain an output O x and O y , that is:

[0122] O x = XAP(p)

[0123] O y = YAP(p)

[0124] O x , the size of O y is C x 1 x W; then O x and O y are spliced and subjected to a convolution module to obtain an output O1, the size of O1 is C / r x 1 x (W+H), r is a scaling factor, that is:

[0125] O1 = Conv2d(Concat(O x , O y ));

[0126] Then, O1 is subjected to a batch normalization module and a nonlinear activation module to obtain O2, the size of O2 is C / r x 1 x (W+H), that is:

[0127] O2 = BN(NL(O1))

[0128] Next, O2 is split into x-direction and y-direction feature tensors O' x and O' y with sizes of C / r x H x 1 and C / r x 1 x W respectively, then O' x and O' y are respectively subjected to a convolution module and a Sigmoid activation module to obtain and , that is:

[0129]

[0130] After the above processing, and are the attention weight matrices corresponding to the X direction and the Y direction respectively, the sizes are C x 1 x H and C x 1 x W respectively;

[0131] Finally, the generated weight matrices are respectively applied to the X and Y directions of the original input feature, and the pixel-by-pixel multiplication is performed through the broadcast mechanism to realize the reweighting of the feature; that is:

[0132]

[0133] The calculation result of coordinate attention is connected with the original input in residual connection to obtain the final output O of the CA module, that is:

[0134] O = Residual (p, Coordinate_attention (p))

[0135] The size of O is CxHxW, which is consistent with the original input feature and contains channel attention and spatial coordinate information enhanced features, wherein C, H and W are three dimensions of the output O.

[0136] Optionally, the step 3 is specifically:

[0137] Suppose that the shallow feature f1, the intermediate feature f2 and the deep feature f3 are respectively the inputs of the three branches of the neck network;

[0138] Firstly, the output O is obtained through the processing of a convolution module 1 That is:

[0139]

[0140] And At the same time, the output O is obtained as the input of the ASFF2 module of the first branch and the second branch 2 That is:

[0141]

[0142] The outputs of the two ASFF2 modules are At the same time, the output O is obtained as the input of the C3 module of the first branch and the second branch 3 That is:

[0143]

[0144] Directly again through the processing of a convolution module, the output is obtained That is:

[0145]

[0146] Finally, At the same time, the output F of the neck network is obtained as the input of the ASFF3 module in the three branches through fusion, that is:

[0147]

[0148] As shown in Figure 5 The step 3 is specifically:

[0149] 1.ASFF2 module accepts and as input, fuses and through an adaptive weighting mechanism, and outputs a comprehensive feature map fused with multi-scale information and ;

[0150] 2.C3 module accepts and as input, extracts deeper semantic information through multiple bottleneck structures, and retains more detailed information, and outputs a feature map containing original information of the input features and deep information enhanced by the bottleneck structure and ;

[0151] 3.The feature tensors f1, f2, and f3 are processed by a convolution kernel, and the input features are further encoded, and a feature tensor with more semantic information is output ;

[0152] 4.The feature tensor is processed by a convolution kernel, and the abstract deep semantic information is extracted again while the spatial dimension is adjusted, and a feature tensor with the same spatial dimension as and is output , facilitating the fusion of the next stage;

[0153] 5.ASFF3 module accepts , and as input, fuses , and through an adaptive weighting mechanism, and outputs a comprehensive feature map F1, F2, and F3 fused with more multi-scale information, containing richer spatial and semantic features.

[0154] The application scenario of the present application is described below.

[0155] The present application provides a small vehicle remote sensing image detection technology based on network reconstruction, which relates to deep learning and remote sensing image processing, belongs to the cross-application technology of computer vision and image processing, and can be applied to the algorithm optimization of small vehicle target detection.

[0156] As shown in Figure 2 , the deep learning-based target detection algorithm model mainly includes three parts:

[0157] 1) backbone network (Backbone);

[0158] 2) neck network (Neck);

[0159] 3) Prediction Head.

[0160] Wherein, the backbone network is a basic component in the target detection model, mainly responsible for extracting rich semantic information from the input image, such as basic texture, color, target contour, edge and more abstract high-level semantic features, therefore, the performance of the backbone network can directly affect the accuracy and performance of the downstream task.

[0161] The neck network is located between the backbone network and the detection head, which plays a role in further processing and fusion of the features extracted by the backbone network.

[0162] The detection head is the last link in the target detection model, responsible for predicting the location, size and category of the target according to the feature map output by the neck network.

[0163] YOLO series model (take YOLOv5 as an example), the input image is extracted by the backbone network, and then the feature fusion is carried out by the neck network, finally three feature maps with resolution of 20x20, 40x40, 80x80 are output, and then three detection heads are used to detect and classify the target on the feature map.

[0164] However, when detecting targets on remote sensing images, small targets are easily ignored because they occupy a small area of pixels and have no obvious pixel changes. With the increase of the depth of the backbone network, on the one hand, the features of small targets will gradually disappear, on the other hand, the resolution of the feature map will gradually decrease, both of which will lead to false detection and missed detection.

[0165] In addition, the C3 module is the core component of the backbone network in the YOLOv5 model, which connects the low-level feature map to the high-level feature map through a series of convolution layers and bottleneck structures, thereby giving the model the ability to extract high-level features and low-level features. This design can make the model better capture context information and improve detection accuracy. Although the C3 module has strong feature extraction capability, it can capture context information, making the model perform well in processing large targets and complex backgrounds. However, when processing small targets, it does not prioritize the preservation of spatial information, which leads to insufficient feature extraction capability for small targets, resulting in missed detection of targets.

[0166] Finally, with the development of the YOLO series, although the new model has higher detection accuracy (such as YOLOv8), the parameter amount of the model also expands; and in the application scenarios of unmanned aerial vehicles and aerospace fields, there is no sufficient hardware resource, so the compromise between model size and detection accuracy is essential. After the above analysis, although YOLO has high detection accuracy in target detection, it still faces challenges in detecting small targets: 1) the feature representation ability is insufficient, which makes it difficult to accurately detect the target; 2) small targets are prone to overlap, which affects the detection accuracy; 3) under the condition of limited hardware, it is difficult to optimize the model accuracy and size at the same time.

[0167] In view of the problems of large feature extraction difficulty and low detection accuracy of the existing remote sensing image target detection algorithm in small vehicle detection, the present application proposes a lightweight and high-precision detection network, which aims to optimize the feature extraction of small targets, alleviate the problems of target overlap and information loss, and reduce the model parameters while maintaining the detection accuracy.

[0168] The present application is improved on the basis of the YOLOv5 basic structure, and the basic process is as shown in Figure 1 , which includes:

[0169] Firstly, the input image is extracted by the backbone network;

[0170] Then, the features are taken as the input of the neck network, and the feature fusion is performed to obtain the feature map;

[0171] Finally, the position, size and category of the target are predicted by the detection head.

[0172] The structure of the backbone network is shown in Figure 1 , which is composed of 10 modules, and the input is a remote sensing image, which has three outputs. The input image is sequentially processed by two convolution modules (Conv) and a C3_CA module to obtain the first output of the backbone network, i.e. the shallow feature of the input image;

[0173] Then, the extracted shallow feature is sequentially processed by a convolution module and a C3_CA module to obtain the second output of the backbone network, i.e. the intermediate feature of the input image;

[0174] Finally, the intermediate feature is sequentially processed by a convolution module, a C3_CA module, a convolution module, a C3_CA module and a SPPF module to obtain the last output of the backbone network, i.e. the deep feature.

[0175] The structure of the neck network is as shown in Figure 2The neck network is composed of 11 modules. The shallow, intermediate and deep features extracted by the backbone network are input into the neck network and three feature maps with sizes of 40x40, 80x80 and 160x160 are output. Unlike the backbone network, the neck network has three branches; assuming that the shallow feature f1, the intermediate feature f2 and the deep feature f3 are input into the three branches of the neck network, the output O 1 i.e.

[0176]

[0177] and The output O 2 i.e.

[0178]

[0179] The outputs of the two ASFF2 modules are The output O 3 i.e.

[0180]

[0181] Unlike the first branch and the second branch, the third branch does not need to pass through the ASFF2 module and the C3 module, but directly passes through a convolution module again to obtain i.e.

[0182]

[0183] Finally, The output F of the neck network is obtained by inputting the outputs of the three branches into the ASFF3 module, i.e.

[0184]

[0185] F1 is a feature map with a size of 160x160; F2 is a feature map with a size of 80x80; and F3 is a feature map with a size of 40x40. These feature maps are input into the detection head network and finally predict the size, position and category of the target.

[0186] Since the C3 module in the backbone network of the YOLOv5 model loses some potential information (e.g., location features) when processing complex backgrounds, dense targets, and small objects, the detection performance of the model is poor. In order to alleviate the feature disappearance problem in this scenario, the C3_CA module is introduced in the backbone network, as shown in the C3_CA part in Figure 6 The features extracted by the upper convolutional layer are processed by two branches, one branch is only processed by the CBS module, and the other branch is processed by the CBS module and then extracted by the CA module, and the outputs of the two branches are processed by another CBS module to restore them to the original dimension. The input of the C3_CA module is f, the first branch needs to be processed by a CBS module and a CA module in turn, and the output is That is,

[0187]

[0188] The second branch is processed by a CBS module alone, and the output is That is,

[0189]

[0190] Subsequently, and will be spliced by a Concat module and then processed by a CBS module to obtain the final output P 2 That is,

[0191]

[0192] The specific implementation of the CA module in the C3_CA module is shown in Figure 7 The input of the CA module is the feature tensor p extracted by the preceding module, and the result after calculating the coordinate attention is connected with the feature tensor p in residual, and the result is the output O of the CA module, that is,

[0193] O = Residual (p, Coordinate_attention (p))

[0194] The calculation of the coordinate attention is shown in Figure 7 Assuming that the size of the feature tensor p is CxHxW, the feature tensor p will first be processed by an x-direction average pooling and a y-direction average pooling, respectively, to obtain the outputs O x and O y That is,

[0195] O x = XAP (p)

[0196] O y

[0197] O x of size C x H x 1, O y of size C x 1 x W, then O x and O y are concatenated and passed through a convolution module to obtain an output O1, O1 of size C / r x 1 x (W+H), r being a scaling factor, the purpose being to reduce the amount of calculation, that is:

[0198] O1 = Conv2d(Concat(O x ,O y ))

[0199] After that, O1 will pass through a batch normalization module and a nonlinear activation module to obtain O2, O2 of size still C / r x 1 x (W+H), that is:

[0200] O2 = BN(NL(O1));

[0201] Where BN (batch norm) is batch normalization, NL (non-linear) is nonlinear activation.

[0202] Next, O2 will be split into x-direction and y-direction feature tensors O x ′ and O y ′ of sizes C / O x H x 1 and C / r x 1 x W respectively, then O x ′ and O y ′ will be processed through a convolution module and a Sigmoid activation module respectively to obtain and that is:

[0203]

[0204] After the above processing, and are the attention weight matrices corresponding to the X direction and the Y direction respectively, of sizes C x 1 x H and C x 1 x W respectively; and after processing by the Sigmoid activation module, the weights are also normalized to the range [0, 1]. Finally, the generated weight matrices are applied to the X and Y directions of the original input features respectively, and the pixel-by-pixel multiplication is performed through the broadcast mechanism to realize the reweighting of the features. That is:

[0205]

[0206] ​By calculating the coordinate attention, the features in X and Y directions can be modeled separately, so as to retain richer spatial information. Then, the calculation result of the coordinate attention is connected with the original input in residual, and the final output O of the CA module is obtained, that is:

[0207] O = Residual (p, Coordinate_attention (p))

[0208] The size of O is C*H*W, which is consistent with the original input feature, but contains the enhanced features of channel attention and spatial coordinate information. The introduction of the CA mechanism can enable the model to capture long-distance dependencies and internal relationships within the channel. Not only can the original feature information be retained, but also the model can locate the target with higher precision due to the preservation of the position relationship. The features extracted by the backbone network usually need to be further fused in the neck network, and multi-scale features are important for detecting targets of different sizes. Multi-scale features usually use top-down and bottom-up strategies, such as the feature pyramid network PAFPN. However, as the feature extraction network deepens, the surface features of small targets in remote sensing images will be ignored by the feature fusion network, because the feature fusion network cannot effectively combine high-level features with low-level attributes, which ultimately leads to the loss of semantic information related to small vehicles during the propagation and interaction process. In addition, when fusing the features of small targets, the original feature network is prone to be affected by model redundancy when fusing high-level features, which also increases the complexity of deployment. In order to solve this problem, the present application proposes a module named Lite-Asymptotic Feature Pyramid Network (L-AFPN) for reconstructing the neck network in combination with the small target detection layer. This method aims to improve the detection accuracy of the model while reducing the number of parameters in the model.

[0209] This module realizes feature fusion by fusing the output of the backbone network. In order to reduce the number of parameters in the neck network to realize lightweight and target detection, the intermediate feature fusion part in the multi-layer fusion structure of AFPN is deleted, so that the two bottom layer features are directly subjected to convolution operation. This method helps to realize lightweight and promotes efficient fusion with top layer features during feature fusion. Subsequently, the activeldapy Spatial Feature Fusion (ASFF) module is used to adjust the two shallow layer features to feature maps with sizes of 80*80 and 160*160 respectively. Then they are weighted and added bit by bit, and finally fused with the top layer features of the backbone network. Figure 1The Neck part shows an example of a neck network. The module automatically learns weight parameters and gradually fuses semantic and detailed information of low-level and high-level attributes, thereby avoiding information loss and degradation in multi-level transfer.

[0210] As shown in Figure 8 An image detection device is provided.

[0211] An acquisition module 901 acquires an image to be detected.

[0212] A first processing module 902 processes the image through a backbone network of a YOLO model to generate extracted features.

[0213] A second processing module 903 takes the features as input of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map.

[0214] A detection module 904 detects the feature map through a detection head of the YOLO model to predict attributes of a target to be detected.

[0215] The first processing module includes:

[0216] A first processing submodule is configured to sequentially process the image through two convolution modules Conv and a C3_CA module to obtain a first output of the backbone network as shallow features of the image.

[0217] A second processing submodule is configured to sequentially process the extracted shallow features through a convolution module and a C3_CA module to obtain a second output of the backbone network as intermediate features of the image.

[0218] A third processing submodule is configured to sequentially process the intermediate features through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain a third output of the backbone network as deep features of the image.

[0219] A fourth processing submodule is configured to take the shallow features, the intermediate features, and the deep features of the image as the extracted features.

[0220] In another aspect, an intelligent device is provided, which includes the device.

[0221] In another aspect, an electronic device is provided, comprising a memory, a processor, a communication interface and a communication bus, the memory storing a computer program executable on the processor, the memory, the processor and the communication interface being in communication through the communication bus, and the processor implementing the data storage and reading method when executing the computer program.

[0222] In another aspect, a computer readable medium having non-volatile program code executable by a processor is provided, the program code causing the processor to execute the method.

[0223] The present application has the following beneficial effects:

[0224] 1. The RA-YOLO algorithm experiments on DOTA, NWPU VHR-10 and DIOR data sets show that compared with the existing YOLOv5, the mAP is increased from 61.3% to 68.8%, the model parameters are reduced from 7.02M to 5.22M, and the inference speed and detection accuracy are also better. The balance between lightweight design and accuracy is especially suitable for hardware resource limited scenes.

[0225] 2. The present application proposes a RA-YOLO algorithm based on network reconstruction, increases the design and application of coordinate attention mechanism in C3_CA module, can combine coordinate attention mechanism and residual structure, ensures the accurate reservation of position information of small target, and improves the detection accuracy without significantly increasing the model parameters.

[0226] 3. The present application increases the lightweight design and multi-layer feature fusion strategy of L-AFPN module, can optimize the multi-scale feature fusion process, reduce the loss of shallow information, and improve the multi-scale feature fusion effect.

[0227] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of detecting an image, characterized by, The method comprises the following steps: Step 1, obtaining an image to be detected; Step 2, processing the image through a backbone network of a YOLO model to generate extracted features; the backbone network comprises a C3_CA module; Step 3, taking the features as inputs of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map; Step 4, detecting the feature map through a detection head of the YOLO model to predict attributes of a target to be detected; Step 2 specifically comprises the following steps: Step 21, processing the image through two convolution modules Conv and a C3_CA module in sequence to obtain a first output of the backbone network as shallow features of the image; Step 22, processing the extracted shallow features through a convolution module and a C3_CA module in sequence to obtain a second output of the backbone network as intermediate features of the image; Step 23, processing the intermediate features through a convolution module, a C3_CA module, a convolution module, a C3_CA module and a SPPF module in sequence to obtain a third output of the backbone network as deep features of the image; The shallow features, the intermediate features and the deep features of the image are taken as the extracted features; The C3_CA module specifically comprises the following steps: Assuming that the input of the C3_CA module is f, the first branch of the input of the C3_CA module is processed by a CBS module and a CA module in turn, and the output is That is: The second branch of the input of the C3_CA module is processed by a CBS module alone, and the output is That is: Then, and After concatenation by a Concat module and processing by a CBS module, the final output P is obtained 2 That is: The step 3 specifically comprises the following steps: Supposing that the shallow features f1, the intermediate features f2 and the deep features f3 are inputs of three branches of the neck network respectively; Firstly, the output O is obtained through a convolution module 1 That is: and Also as input to the ASFF2 module as the first branch and the second branch, the output O is obtained 2 i.e.: The output of the two ASFF2 modules The input of the C3 module as the first branch and the second branch, resulting in output O 3 That is: directly again through a convolution module processing, get that is: Finally, At the same time as the input of the ASFF3 module in the three branches, after fusion, the final output F of the neck network is obtained, that is:

2. The method of claim 1, wherein, The CA module specifically comprises the following steps: Supposing that an input of the CA module is a feature tensor p extracted by a preceding module, the result after the feature tensor p is processed through coordinate attention is connected in residual with the feature tensor p to obtain an output O of the CA module, that is: O = Residual (p, Coordinate_attention (p) ).

3. The method of claim 1, wherein, The CA module is specifically: assuming that the size of the feature tensor p is CxHxW, the feature tensor p first respectively undergoes an average pooling in an x direction and an average pooling in a y direction to obtain an output O x and O y That is: O x = XAP(p) O y = YAP(p) O x of size C x H x 1, O y of size C x 1 x W; then O x and O y are spliced and passed through a convolution module to obtain an output O1, O1 has a size of C / r x 1 x (W+H), r is a scaling factor, namely: O1 = Conv2d(Concat(O x ,O y )) Then, O1 is processed through a batch normalization module and a nonlinear activation module to obtain O2, the size of O2 is C / r x 1 x (W+H), that is: O2 = BN (NL (O1) ). Next, O2 is split into x-direction and y-direction feature tensors O of sizes C / r x H x 1 and C / r x 1 x W, respectively x ′ and O y ′ , and then O x ′ and O y ′ are processed by a convolution module and a Sigmoid activation module, respectively, to obtain and that is: After the above processing, and are attention weight matrices corresponding to the X direction and the Y direction, respectively, with sizes of Cx1xH and Cx1xW, respectively. Finally, the generated weight matrix is applied to the X and Y directions of the original input features respectively, and the features are reweighted through a broadcast mechanism; that is: The result of the coordinate attention is connected in residual with the original input to obtain the final output O of the CA module, that is: O = Residual (p, Coordinate_attention (p) ). The size of O is C x H x W, which is consistent with the original input features and contains features enhanced by channel attention and spatial coordinate information, wherein C, H and W are three dimensions of the output O.

4. The method of claim 1, wherein, The attributes of the target to be detected comprise one or more of any combination of the position, size and category of the target to be detected.

5. An apparatus for detecting an image, characterized by comprising: The method comprises the following steps: An acquisition module acquires an image to be detected; A first processing module processes the image through a backbone network of a YOLO model to generate extracted features; A second processing module takes the features as inputs of a neck network of the YOLO model to perform feature fusion processing to obtain a feature map; A detection module detects the feature map through a detection head of the YOLO model to predict attributes of a target to be detected; The first processing module comprises: The first processing submodule is configured to sequentially process the image through two convolution modules Conv and a C3_CA module to obtain a first output of the backbone network as a shallow feature of the image; The second processing submodule is configured to sequentially process the extracted shallow feature through a convolution module and a C3_CA module to obtain a second output of the backbone network as an intermediate feature of the image; The third processing submodule is configured to sequentially process the intermediate feature through a convolution module, a C3_CA module, a convolution module, a C3_CA module and an SPPF module to obtain a third output of the backbone network as a deep feature of the image; The fourth processing submodule is configured to take the shallow feature, the intermediate feature and the deep feature of the image as the extracted features. The C3_CA module specifically comprises: Assuming that the input of the C3_CA module is f, the first branch of the input of the C3_CA module is processed by a CBS module and a CA module in turn, and the output is That is: The second branch of the input of the C3_CA module is processed by a CBS module alone, and the output is That is: Then, and After a Concat module splicing, then again through a CBS module processing, get the final output P 2 That is: The second processing module specifically comprises: Suppose that the shallow feature f1, the intermediate feature f2 and the deep feature f3 are inputs of three branches of the neck network respectively; First, the output O is obtained after processing by a convolution module 1 That is: and Also as input to the ASFF2 module as the first branch and the second branch, resulting in output O 2 i.e.: The output of the two ASFF2 modules The input of the C3 module as the first branch and the second branch, resulting in output O 3 That is: directly again through a convolution module processing, get that is: Finally, At the same time as the input of the ASFF3 module in the three branches, after fusion, the final output F of the neck network is obtained, that is:

6. A smart device, comprising: The apparatus of claim 5.

7. An electronic device comprising a memory, a processor, a communication interface and a communication bus, the memory having stored therein a computer program executable on the processor, the memory, the processor being in communication via the communication bus and the communication interface, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 4.

8. A computer readable medium having a non-transitory program code executable by a processor, the program code comprising instructions for: The program code causes the processor to execute the method of any one of claims 1 to 4.

Images