Image detection method and device, equipment and readable medium

Abstract

The embodiment of the invention provides an image detection method and device, equipment and a readable medium. Relates to the field of computer image detection. The image detection method comprises the following steps: step 1, acquiring an image to be detected; 2, the image is processed through a backbone network of a YOLO model, and extracted features are generated; the backbone network comprises a C3CA module; 3, taking the features as input of a neck network of a YOLO model, and performing feature fusion processing to obtain a feature map; and 4, detecting the feature map through a detection head of the YOLO model, and predicting the attribute of the to-be-detected target. According to the invention, the image detection precision can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer image detection, and in particular to an image detection method, device, equipment, and readable medium. Background Art

[0002] With the rapid development of UAV and aerospace technology, the detection and recognition of targets in remote sensing imagery (RSI) has become a key research area for collecting surface information, with important application value in military, security and environmental protection.

[0003] Existing remote sensing image detection algorithms can be divided into traditional algorithms and deep learning-based algorithms.

[0004] Traditional target detection algorithms often require complex feature engineering of input remote sensing images in advance and the design of convolution kernels containing a large number of hyperparameters, which poses a great challenge to the usability of the algorithm.

[0005] Deep learning-based object detection algorithms are typically end-to-end, and the models can autonomously learn and update their own parameters, eliminating the need for manual parameter configuration. Consequently, deep learning-based object detection algorithms have become increasingly popular in recent years, and a number of excellent models have emerged. Among them, YOLO is widely used for its high detection accuracy and fast response time.

[0006] However, while the basic YOLO model improves detection accuracy, the number of model parameters is large, and as the number of network layers increases, the features of small targets are easily lost, resulting in missed detections and false detections, resulting in poor small target detection performance in remote sensing images. Summary of the Invention

[0007] The embodiments of the present invention provide a method, apparatus, device, and readable medium for detecting an image, which relates to the field of computer image detection and improves the accuracy of image detection.

[0008] In one aspect, a method for detecting an image is provided, comprising:

[0009] Step 1: Obtain the image to be detected;

[0010] Step 2: Process the image through the backbone network of the YOLO model to generate extracted features; the backbone network includes a C3_CA module;

[0011] Step 3: Using the features as input to the neck network of the YOLO model, performing feature fusion processing to obtain a feature map;

[0012] Step 4: Detect the feature map using the detection head of the YOLO model to predict the attributes of the target to be detected;

[0013] Optionally, step 2 is as follows:

[0014] Step 21: The image is processed by two convolutional modules Conv and one C3_CA module in sequence to obtain the first output of the backbone network as the shallow feature of the image;

[0015] Step 22: The extracted shallow features are processed by a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image;

[0016] Step 23: Process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain the third output of the backbone network as the deep features of the image;

[0017] The shallow features, intermediate features and deep features of the image are used as extracted features.

[0018] Optionally, the C3_CA module is specifically:

[0019] Assume that the input of C3_CA module is f, the first branch of the input of C3_CA module is processed by a CBS module and a CA module in sequence, and the output is Right now:

[0020]

[0021] The second branch of the input of the C3_CA module is processed by a CBS module alone, and the output is Right now:

[0022]

[0023] Then, and After being spliced by a Concat module and then processed by a CBS module, the final output P is obtained. 2 ,Right now:

[0024]

[0025] Optionally, the CA module is specifically:

[0026] Assume that the input of the CA module is the feature tensor p extracted by the pre-module. The result of calculating the coordinate attention of the feature tensor p is residually connected with the feature tensor p. The result is the output O of the CA module, that is:

[0027] O=Residual(p,Coordinate_attention(p)).

[0028] Optionally, the CA module is specifically:

[0029] Assuming that the size of the feature tensor p is C×H×W, the feature tensor p first undergoes an average pooling in the x direction and an average pooling in the y direction to obtain the output O x and O y ,Right now:

[0030] O x =XAP(p)

[0031] O y =YAP(p)

[0032] O x The size is C×H×1, O y The size of C×1×W; then O x and O y After concatenation and passing through a convolution module, the output O1 is obtained. The size of O1 is C / r×1×(W+H), where r is the scaling factor, that is:

[0033] O1=Conv2d(Concat(O x ,O y ));

[0034] After that, O1 goes through a batch normalization module and a nonlinear activation module to obtain O2. The size of O2 is C / r×1×(W+H), that is:

[0035] O2=BN(NL(O1))

[0036] Next, O2 is split into feature tensors O′ in the x-direction and y-direction with sizes of C / r×H×1 and C / r×1×W respectively. x and O′ y , then O′ x and O′ y After being processed by a convolution module and a Sigmoid activation module respectively, we get and Right now:

[0037]

[0038] After the above processing, and The attention weight matrices corresponding to the X and Y directions are C×1×H and C×1×W respectively;

[0039] Finally, the generated weight matrix is applied to the X and Y directions of the original input features respectively, and pixel-by-pixel multiplication is performed through the broadcast mechanism to achieve feature re-weighting; that is:

[0040]

[0041] The calculation result of the coordinate attention is residually connected with the original input to obtain the final output O of the CA module, namely:

[0042] O=Residual(p,Coordinate_attention(p))

[0043] The size of O is C×H×W, which is consistent with the original input features and includes the features enhanced by channel attention and spatial coordinate information. Among them, C, H, and W are the three dimensions of output O respectively.

[0044] Optionally, the step 3 is specifically as follows:

[0045] Assume that the shallow feature f1, the intermediate feature f2, and the deep feature f3 are the inputs of the three branches of the neck network;

[0046] First, after processing through a convolution module, the output O is obtained 1 ,Right now:

[0047]

[0048] and As the input of the ASFF2 module of the first branch and the second branch, the output O 2 ,Right now:

[0049]

[0050] Output of two ASFF2 modules At the same time, it serves as the input of the C3 module of the first branch and the second branch, and the output is O 3 ,Right now:

[0051]

[0052] Directly pass through a convolution module again to obtain Right now:

[0053]

[0054] at last, At the same time, it is used as the input of the ASFF3 module in the three branches. After fusion, the final output F of the neck network is obtained, that is:

[0055]

[0056] Optionally, the attributes of the target to be detected include: one or more of the position, size, and category of the target to be detected.

[0057] In another aspect, an image detection device is provided, comprising:

[0058] An acquisition module acquires the image to be detected;

[0059] The first processing module processes the image through the backbone network of the YOLO model to generate extracted features;

[0060] The second processing module uses the features as input to the neck network of the YOLO model, performs feature fusion processing, and obtains a feature map;

[0061] The detection module detects the feature map through the detection head of the YOLO model and predicts the attributes of the target to be detected;

[0062] The first processing module includes:

[0063] The first processing submodule is used to process the image in sequence through two convolution modules Conv and one C3_CA module to obtain the first output of the backbone network as the shallow feature of the image;

[0064] The second processing submodule is used to process the extracted shallow features through a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image;

[0065] A third processing submodule is used to process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain a third output of the backbone network as a deep feature of the image;

[0066] The fourth processing submodule is configured to use the shallow features, intermediate features, and deep features of the image as extracted features.

[0067] On the other hand, a smart device is provided, comprising the apparatus.

[0068] On the other hand, an electronic device is provided, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the processor implements the above-mentioned data storage and reading method when executing the computer program.

[0069] In another aspect, a computer-readable medium having non-volatile program code executable by a processor is provided, the program code causing the processor to perform the method described.

[0070] It can be seen from the technical solutions provided by the above embodiments of the present invention that, in step 1, an image to be detected is obtained; in step 2, the image is processed by the backbone network of the YOLO model to generate extracted features; the backbone network includes a C3_CA module; in step 3, the features are used as input to the neck network of the YOLO model to perform feature fusion processing to obtain a feature map; in step 4, the feature map is detected by the detection head of the YOLO model to predict the attributes of the target to be detected;

[0071] Step 2 is specifically as follows: Step 21, the image is processed sequentially through two convolution modules Conv and a C3_CA module to obtain the first output of the backbone network as the shallow features of the image; Step 22, the extracted shallow features are processed sequentially through a convolution module and a C3_CA module to obtain the second output of the backbone network as the intermediate features of the image; Step 23, the intermediate features are processed sequentially through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain the third output of the backbone network as the deep features of the image; the shallow features, intermediate features, and deep features of the image are used as the extracted features. By introducing the C3_CA module, the present invention alleviates the disappearance of features in some scenarios and improves the accuracy of image detection.

[0072] Additional aspects and advantages of the present invention will be set forth in part in the following description, will become apparent from the following description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0074] Figure 1 Schematic diagram of the process of the image detection method of the present invention;

[0075] Figure 2 This is a first schematic diagram of the overall processing flow of the YOLO model in the present invention;

[0076] Figure 3 This is a second schematic diagram of the overall processing flow of the YOLO model in the present invention;

[0077] Figure 4 Schematic diagram of the specific implementation of the backbone network of the YOLO model of the present invention;

[0078] Figure 5 Schematic diagram of a specific implementation of the neck network of the YOLO model of the present invention;

[0079] Figure 6 Schematic diagram of the specific implementation of the C3_CA module of the YOLO model of the present invention;

[0080] Figure 7 Schematic diagram of a specific implementation of the CA module in the C3_CA module of the YOLO model of the present invention;

[0081] Figure 8 Schematic diagram of the connection of the image detection device in the present invention. DETAILED DESCRIPTION

[0082] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0083] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings. However, each embodiment does not constitute a limitation on the embodiments of the present invention.

[0084] Figure 1 Schematic diagram of the process of the image detection method of the present invention; Figure 2 This is a first schematic diagram of the overall processing flow of the YOLO model in the present invention; Figure 3 This is a second schematic diagram of the overall processing flow of the YOLO model in the present invention; Figure 4 Schematic diagram of the specific implementation of the backbone network of the YOLO model of the present invention; Figure 5 Schematic diagram of a specific implementation of the neck network of the YOLO model of the present invention; Figure 6 Schematic diagram of the specific implementation of the C3_CA module of the YOLO model of the present invention; Figure 7 Schematic diagram of a specific implementation of the CA module in the C3_CA module of the YOLO model of the present invention; Figure 8 FIG1 is a schematic diagram of the connection of the image detection device of the present invention.

[0085] like Figure 1 FIG. 1 is a method for detecting an image according to the present invention, comprising:

[0086] Step 1: Acquire an image to be detected; the image may be a remote sensing image, etc.

[0087] Step 2, such as Figure 4 As shown, the image is processed by the backbone network of the YOLO model to generate extracted features; the backbone network includes a C3_CA module; this step is specifically, the image is processed by the backbone network to extract rich semantic information and output it as a feature tensor.

[0088] Step 3, such as Figure 5 As shown, the features are used as the input of the neck network of the YOLO model, and feature fusion processing is performed to obtain a feature map; this step specifically includes that the feature tensor is subjected to feature fusion and deep processing through the neck network, and output as a feature map; this step specifically includes that the feature tensor is subjected to feature fusion and deep processing through the neck network, and output as a feature map.

[0089] Step 4: The detection head of the YOLO model detects the feature map and predicts the attributes of the target to be detected. Optionally, the attributes of the target to be detected include one or more of its location, size, and category. The target to be detected can be a vehicle, etc., depending on the actual situation. Specifically, this step involves the detection head predicting and outputting the location, size, and category of the detected target based on the feature map information and pre-set information about the target to be detected.

[0090] Optional, such as Figure 4 As shown, step 2 is specifically as follows:

[0091] Step 21: The image is processed by two convolutional modules Conv and one C3_CA module in sequence to obtain the first output of the backbone network as the shallow feature of the image;

[0092] Step 22: The extracted shallow features are processed by a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image;

[0093] Step 23: Process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain the third output of the backbone network as the deep features of the image;

[0094] The shallow features, intermediate features and deep features of the image are used as extracted features.

[0095] like Figure 4 As shown, the specific steps are:

[0096] 1. The image is a tensor k of size 640*640*3; 2. k is convolved with a kernel to extract basic edge and texture information from the image, outputting a tensor k1 of size 64*320*320;

[0097] 3. k1 is further processed through the convolution kernel to deepen the shallow feature information of the image and output a tensor k2 of size 128*160*160;

[0098] 4. k2 strengthens the representation of shallow feature position information in the image by introducing coordinate attention, and outputs a tensor f1 of size 128*160*160;

[0099] 5. f1 extracts more intermediate features of the image through the convolution kernel, such as the shape and outline of the target, and outputs a tensor k3 of size 256*80*80;

[0100] 6. k3 strengthens the representation of the position information of the central feature of the image by introducing coordinate attention, and outputs a tensor f2 of size 256*80*80;

[0101] 7.f2 extracts high-level semantic features of the image, such as the category and relationship of the target, through the convolution kernel, and outputs a tensor k4 of size 512*40*40;

[0102] 8.k4 introduces coordinate attention, which strengthens the expression of high-level semantic feature position information of the image and outputs a tensor k5 of size 512*40*40;

[0103] 9. k5 extracts deep abstract semantic features of the image through the convolution kernel and outputs a tensor k6 of size 1024*20*20;

[0104] 10.k6 introduces coordinate attention, which strengthens the expression of deep abstract semantic feature position information of the image and outputs a tensor k7 of size 1024*20*20;

[0105] 11.k7 aggregates the semantic feature information in the image through the SPPF module and outputs a tensor f3 of size 1024*20*20.

[0106] Optional, such as Figure 6 As shown, the C3_CA module is specifically:

[0107] Assume that the input of C3_CA module is f, the first branch of the input of C3_CA module is processed by a CBS module and a CA module in sequence, and the output is Right now:

[0108]

[0109] The second branch of the input of the C3_CA module is processed by a CBS module alone, and the output is Right now:

[0110]

[0111] Then, and After being spliced by a Concat module and then processed by a CBS module, the final output P is obtained. 2 ,Right now:

[0112]

[0113] like Figure 6 As shown, the C3_CA module is specifically:

[0114] 1. The feature tensor f is further extracted from the CBS module to extract target semantic information. The feature tensor is then normalized to mitigate data distribution variations. An activation function is then used to introduce nonlinear factors, outputting two feature tensors that contain more semantic information, a more stable data distribution, and more complex information.

[0115] 2. The feature tensor output by the CBS module is processed by the CA module. By calculating the coordinate attention, the spatial information of the feature tensor is enhanced, and a feature map that can better reflect the spatial distribution and global position of the target after coordinate weighting is output. ;

[0116] 3. and After the Concat operation, the dimensions are adjusted to output a feature map containing more information. 4. The feature map further extracts target semantic information in the CBS module and normalizes the feature tensor to reduce changes in data distribution. Nonlinear factors are then introduced through the activation function to output a feature tensor P2 that contains more semantic information, a more stable data distribution, and more complex information.

[0117] Optionally, the CA module is specifically:

[0118] Assume that the input of the CA module is the feature tensor p extracted by the pre-module. The result of calculating the coordinate attention of the feature tensor p is residually connected with the feature tensor p. The result is the output O of the CA module, that is:

[0119] O=Residual(p,Coordinate_attention(p)).

[0120] Optional, such as Figure 7 As shown, the CA module is specifically:

[0121] Assuming that the size of the feature tensor p is C×H×W, the feature tensor p first undergoes an average pooling in the x direction and an average pooling in the y direction to obtain the output O x and O y ,Right now:

[0122] O x =XAP(p)

[0123] O y =YAP(p)

[0124] O x The size is C×H×1, O y The size of C×1×W; then O x and O y After concatenation and passing through a convolution module, the output O1 is obtained. The size of O1 is C / r×1×(W+H), where r is the scaling factor, that is:

[0125] O1=Conv2d(Concat(O x ,O y ));

[0126] After that, O1 goes through a batch normalization module and a nonlinear activation module to obtain O2. The size of O2 is C / r×1×(W+H), that is:

[0127] O2=BN(NL(O1))

[0128] Next, O2 is split into feature tensors O′ in the x-direction and y-direction with sizes of C / r×H×1 and C / r×1×W respectively. x and O′ y , then O′ x and O′ y After being processed by a convolution module and a Sigmoid activation module respectively, we get and Right now:

[0129]

[0130] After the above processing, and The attention weight matrices corresponding to the X and Y directions are C×1×H and C×1×W respectively;

[0131] Finally, the generated weight matrix is applied to the X and Y directions of the original input features respectively, and pixel-by-pixel multiplication is performed through the broadcast mechanism to achieve feature re-weighting; that is:

[0132]

[0133] The calculation result of the coordinate attention is residually connected with the original input to obtain the final output O of the CA module, namely:

[0134] O=Residual(p,Coordinate_attention(p))

[0135] The size of O is C×H×W, which is consistent with the original input features and includes the features enhanced by channel attention and spatial coordinate information. Among them, C, H, and W are the three dimensions of output O respectively.

[0136] Optionally, the step 3 is specifically as follows:

[0137] Assume that the shallow feature f1, the intermediate feature f2, and the deep feature f3 are the inputs of the three branches of the neck network;

[0138] First, after processing through a convolution module, the output O is obtained 1 ,Right now:

[0139]

[0140] and As the input of the ASFF2 module of the first branch and the second branch, the output O 2 ,Right now:

[0141]

[0142] Output of two ASFF2 modules At the same time, it serves as the input of the C3 module of the first branch and the second branch, and the output is O 3 ,Right now:

[0143]

[0144] Directly pass through a convolution module again to obtain Right now:

[0145]

[0146] at last, At the same time, it is used as the input of the ASFF3 module in the three branches. After fusion, the final output F of the neck network is obtained, that is:

[0147]

[0148] like Figure 5 As shown, the step 3 is specifically as follows:

[0149] 1.ASFF2 module accepts and As input, the adaptive weighting mechanism and Fusion, output a comprehensive feature map that integrates multi-scale information and ;

[0150] 2.C3 module acceptance and As input, it extracts deeper semantic information through multiple bottleneck structures and retains more detailed information, and outputs a feature map containing the original information of the input features and the deep information enhanced by the bottleneck structure. and ;

[0151] 3. The feature tensors f1, f2, and f3 are processed by the convolution kernel, and the input features are further encoded to output a feature tensor with more semantic information. ;

[0152] 4. Feature Tensor After convolution kernel processing, the deep semantics are extracted again, and the spatial dimension is adjusted to output a and Feature tensors with consistent spatial dimensions , to facilitate the next stage of integration;

[0153] 5.ASFF3 module accepts 、 and As input, the adaptive weighting mechanism 、 and Fusion,outputs a comprehensive feature map F1, F2, and F3 that integrates more multi-scale information,,including richer spatial and semantic features.

[0154] The following describes the application scenarios of the present invention.

[0155] The present invention provides a detection technology for small vehicle remote sensing images based on network reconstruction, which involves deep learning and remote sensing image processing. It is an interdisciplinary application technology of computer vision and image processing and can be applied to algorithm optimization for detection of targets such as small vehicles.

[0156] like Figure 2 As shown in the figure, the target detection algorithm model based on deep learning mainly consists of three parts:

[0157] 1) Backbone network;

[0158] 2) Neck network (Neck);

[0159] 3) Prediction Head.

[0160] Among them, the backbone network is the basic component of the target detection model, which is mainly responsible for extracting rich semantic information from the input image, such as basic texture, color, target contour, edge and more abstract high-level semantic features. Therefore, the performance of the backbone network can directly affect the accuracy and performance of downstream tasks.

[0161] The neck network is located between the backbone network and the detection head, and plays the role of further processing and fusing the features extracted by the backbone network.

[0162] The detection head is the last link in the object detection model, responsible for predicting the location, size and category of the target based on the feature map output by the neck network.

[0163] The YOLO series of models (taking YOLOv5 as an example) uses the backbone network to extract features from the input image, and then uses the neck network for feature fusion. It will eventually output three feature maps with resolutions of 20×20, 40×40, and 80×80. The three detection heads then detect and classify targets on the feature maps respectively.

[0164] However, when performing target detection on remote sensing images, small targets are easily overlooked during detection because they occupy a small pixel area and pixel changes are not obvious. In addition, as the depth of the backbone network increases, the features of small targets will gradually disappear, and the resolution of the feature map will gradually decrease. Both will lead to false detections and missed detections.

[0165] In addition, the C3 module is the core component of the backbone network in the YOLOv5 model. It connects low-level feature maps to high-level feature maps through a series of convolutional layers and bottleneck structures, giving the model the ability to extract the intrinsic connection between high-level features and low-level features. This design enables the model to better capture contextual information and improve detection accuracy. Although the C3 module has powerful feature extraction capabilities, thanks to the capture of contextual information, the model performs well when dealing with large targets and complex backgrounds. However, when dealing with small targets, it does not prioritize the preservation of spatial information, which leads to insufficient feature extraction capabilities for small targets and thus causes missed detections.

[0166] Finally, as the YOLO family has evolved, while new models have achieved higher detection accuracy (such as YOLOv8), their parameters have also expanded. However, in drone and aerospace applications, where hardware resources are limited, a compromise between model size and detection accuracy is essential. The above analysis shows that while YOLO achieves high detection accuracy in object detection, it still faces challenges when detecting small objects: 1) Insufficient feature representation capabilities make it difficult to accurately detect objects; 2) Small objects are prone to overlap, affecting detection accuracy; and 3) Under limited hardware conditions, it is difficult to simultaneously optimize model accuracy and size.

[0167] To address the problems of difficult feature extraction and low detection accuracy in small vehicle detection using existing remote sensing image target detection algorithms, this paper proposes a lightweight and high-precision detection network that aims to optimize feature extraction of small targets, alleviate target overlap and information loss problems, and reduce model parameters while maintaining detection accuracy.

[0168] The present invention is improved on the basic structure of YOLOv5. The basic process is as follows: Figure 1 Shown, including:

[0169] First, the input image is passed through the backbone network to extract features;

[0170] Subsequently, the features are used as the input of the neck network and feature fusion is performed to obtain the feature map;

[0171] Finally, the location, size and category of the target are predicted by the detection head.

[0172] The structure of the backbone network is as follows Figure 1 As shown in the Backbone in Figure 1, it consists of 10 modules, takes a remote sensing image as input, and has three outputs. The input image passes through two convolutional modules (Conv) and a C3_CA module to obtain the first output of the backbone network, which is the shallow features of the input image.

[0173] Subsequently, the extracted shallow features are processed by a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network, which is the intermediate features of the input image;

[0174] Finally, the intermediate features are processed in sequence by a convolution module, a C3_CA module, a convolution module, a C3_CA module and an SPPF module to obtain the final output of the backbone network, namely the deep features.

[0175] The structure of the neck network is as follows Figure 2As shown in Neck, it consists of 11 modules. The shallow features, intermediate features, and deep features of the original image extracted by the backbone network serve as the input of the neck network and output three feature maps of size 40×40, 80×80, and 160×160 respectively. Unlike the backbone network, the neck network has three branches; assuming that the shallow features f1, intermediate features f2, and deep features f3 are respectively used as the input of the three branches of the neck network, they are first processed by a convolution module to obtain the output O 1 ,Right now:

[0176]

[0177] and As the input of the ASFF2 module of the first branch and the second branch, the output O 2 ,Right now:

[0178]

[0179] Afterwards, the outputs of the two ASFF2 modules At the same time, it serves as the input of the C3 module of the first branch and the second branch, and the output is O 3 ,Right now:

[0180]

[0181] Unlike the first and second branches, It does not need to be processed by the ASFF2 module and the C3 module, but is directly processed by a convolution module again to obtain Right now:

[0182]

[0183] at last, At the same time, it serves as the input of the ASFF3 module in the three branches, and the final output F of the neck network is obtained after fusion, that is:

[0184]

[0185] F1 is a feature map of size 160 × 160; F2 is a feature map of size 80 × 80; and F3 is a feature map of size 40 × 40. These feature maps are used as input to the detection head network and ultimately predict the size, location, and category of the object.

[0186] Since the C3 module in the backbone network of the YOLOv5 model loses some potential information (such as position features) when processing complex backgrounds, dense targets and small objects, the detection performance of the model is poor. In order to alleviate the problem of feature disappearance in this scenario, the present invention introduces the C3_CA module in the backbone network, such as Figure 6 The features extracted by the upper convolutional layer are processed by two branches. One branch is only processed by the CBS module, and the other branch is processed by the CA module for feature extraction. The outputs of the two branches are merged and spliced before being processed by another CBS module to restore them to their original dimensions. The input of the C3_CA module is assumed to be f. The first branch needs to be processed by a CBS module and a CA module in sequence, and the output is Right now:

[0187]

[0188] The second branch is processed by a CBS module alone, and the output is Right now:

[0189]

[0190] Then, and It will be spliced through a Concat module and then processed by a CBS module to obtain the final output P 2 ,Right now:

[0191]

[0192] The specific implementation of the key component CA module in the C3_CA module is as follows Figure 7 As shown in Figure 2, the input of the CA module is the feature tensor p extracted by the pre-module. After calculating the coordinate attention of the feature tensor p, the result is residually connected with the feature tensor p. The result is the output O of the CA module, that is:

[0193] O=Residual(p,Coordinate_attention(p))

[0194] The calculation of coordinate attention is as follows Figure 7 As shown, assuming that the size of the feature tensor p is C×H×W, the feature tensor p will first undergo an average pooling in the x direction and an average pooling in the y direction to obtain the output O x and O y ,Right now:

[0195] O x =XAP(p)

[0196] O y =YAP(p)

[0197] O x The size is C×H×1, O y The size of C×1×W, then O x and O y After concatenation, the output O1 is obtained through a convolution module. The size of O1 is C / r×1×(W+H), where r is the scaling factor. The purpose is to reduce the amount of calculation, that is:

[0198] O1=Conv2d(Concat(O x ,O y ))

[0199] After that, O1 will go through a batch normalization module and a nonlinear activation module to obtain O2. The size of O2 is still C / r×1×(W+H), that is:

[0200] O2=BN(NL(O1));

[0201] Among them, BN (batch norm) is batch normalization, NL (non-linear) is nonlinear activation.

[0202] Next, O2 is split into feature tensors O in the x-direction and y-direction with sizes of C / O×H×1 and C / r×1×W respectively. x ′ and O y ′ , then O x ′ and O y ′ Will be processed by a convolution module and a Sigmoid activation module respectively to obtain and Right now:

[0203]

[0204] After the above processing, and The attention weight matrices corresponding to the X and Y directions are C×1×H and C×1×W, respectively. After being processed by the Sigmoid activation module, the weights are also normalized to the range [0,1]. Finally, the generated weight matrices are applied to the X and Y directions of the original input features, and pixel-by-pixel multiplication is performed through the broadcast mechanism to achieve feature re-weighting. That is:

[0205]

[0206] By calculating the coordinate attention, the features in the X and Y directions can be modeled separately, thereby retaining richer spatial information. The result of the coordinate attention calculation is then residually connected with the original input to obtain the final output O of the CA module, which is:

[0207] O=Residual(p,Coordinate_attention(p))

[0208] The size of O is C × H × W, consistent with the original input features, but incorporates features enhanced with channel attention and spatial coordinate information. The introduction of the CA mechanism enables the model to capture long-range dependencies and intrinsic connections within the channel, preserving not only the original feature information but also, thanks to the preservation of positional relationships, enabling the model to localize objects with higher accuracy. Features extracted by the backbone network are often further fused in the neck network. Multi-scale features are crucial for object detection of varying sizes, and are typically implemented using top-down and bottom-up strategies, such as the feature pyramid network (PAFPN). However, as the feature extraction network deepens, surface features of smaller objects in remote sensing images are often overlooked by the feature fusion network. This is because the feature fusion network cannot effectively combine high-level features with low-level attributes, ultimately resulting in the loss of semantic information related to small vehicles during propagation and interaction. Furthermore, when fusing features of small objects, the original feature network is susceptible to model redundancy when fusing high-level features, which increases deployment complexity. To address this issue, we propose a module called Lite-Asymptotic Feature Pyramid Network (L-AFPN), which reconstructs the neck network while combining it with a small object detection layer. This approach aims to improve the detection accuracy of the model while reducing the number of parameters in the model.

[0209] This module achieves feature fusion by fusing the outputs of the backbone network. To reduce the number of parameters in the neck network for lightweight and object detection, the intermediate feature fusion portion of the multi-layer fusion structure of the AFPN is removed, allowing the two bottom-level features to be directly convolved. This approach facilitates lightweighting and promotes efficient integration with top-level features during the feature fusion process. Subsequently, the activeldapy Spatial Feature Fusion (ASFF) module is used to resize the two shallow-level features into feature maps of 80*80 and 160*160, respectively. These features are then weighted bit by bit, added, and finally fused with the top-level features of the backbone network. Figure 1The Neck section of the Neck Network demonstrates an example. This module automatically learns weight parameters and gradually integrates the semantics and details of low-level and high-level attributes, avoiding information loss and degradation during multi-level transfer. This approach reduces computational overhead and implementation complexity while ensuring feature scale invariance.

[0210] like Figure 8 FIG. 1 is an image detection device according to the present invention, comprising:

[0211] An acquisition module 901 acquires an image to be detected;

[0212] The first processing module 902 processes the image through the backbone network of the YOLO model to generate extracted features;

[0213] The second processing module 903 uses the features as input to the neck network of the YOLO model, performs feature fusion processing, and obtains a feature map;

[0214] The detection module 904 detects the feature map using the detection head of the YOLO model and predicts the attributes of the target to be detected;

[0215] The first processing module includes:

[0216] The first processing submodule is used to process the image in sequence through two convolution modules Conv and one C3_CA module to obtain the first output of the backbone network as the shallow feature of the image;

[0217] The second processing submodule is used to process the extracted shallow features through a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image;

[0218] A third processing submodule is used to process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain a third output of the backbone network as a deep feature of the image;

[0219] The fourth processing submodule is configured to use the shallow features, intermediate features, and deep features of the image as extracted features.

[0220] On the other hand, a smart device is provided, comprising the apparatus.

[0221] On the other hand, an electronic device is provided, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the data storage and reading method is implemented when the processor executes the computer program.

[0222] In another aspect, a computer-readable medium having non-volatile program code executable by a processor is provided, the program code causing the processor to perform the method described.

[0223] The present invention has the following beneficial effects:

[0224] 1. Experiments on the DOTA, NWPU VHR-10, and DIOR datasets show that the RA-YOLO algorithm improves mAP from 61.3% to 68.8% compared to the existing YOLOv5s, reduces model parameters from 7.02M to 5.22M, and demonstrates superior performance in inference speed and detection accuracy. Its lightweight design and balance of accuracy make it particularly suitable for scenarios with limited hardware resources.

[0225] 2. This paper proposes a RA-YOLO algorithm based on network reconstruction, which adds the design and application of the coordinate attention mechanism in the C3_CA module. It can combine the coordinate attention mechanism with the residual structure to ensure the accurate retention of the position information of small targets and improve the detection accuracy without significantly increasing the model parameters.

[0226] 3. The present invention adds a lightweight design and multi-layer feature fusion strategy to the L-AFPN module, which can optimize the multi-scale feature fusion process, reduce shallow information loss, and improve the multi-scale feature fusion effect.

[0227] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting an image, characterized in that: include: Step 1: Obtain the image to be detected; Step 2: Process the image through the backbone network of the YOLO model to generate extracted features; the backbone network includes a C3_CA module; Step 3: Using the features as input to the neck network of the YOLO model, performing feature fusion processing to obtain a feature map; Step 4: Detect the feature map using the detection head of the YOLO model to predict the attributes of the target to be detected; Step 2 is as follows: Step 21: The image is processed by two convolutional modules Conv and one C3_CA module in sequence to obtain the first output of the backbone network as the shallow feature of the image; Step 22: The extracted shallow features are processed by a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image; Step 23: Process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain the third output of the backbone network as the deep features of the image; The shallow features, intermediate features and deep features of the image are used as extracted features.

2. The method according to claim 1, characterized in that The C3_CA module is specifically: Assume that the input of C3_CA module is f, the first branch of the input of C3_CA module is processed by a CBS module and a CA module in sequence, and the output is Right now: The second branch of the input of the C3_CA module is processed by a CBS module alone, and the output is Right now: Then, and After being spliced by a Concat module and then processed by a CBS module, the final output P is obtained. 2 ,Right now:

3. The method according to claim 2, characterized in that The CA module is specifically: Assume that the input of the CA module is the feature tensor p extracted by the pre-module. The result of calculating the coordinate attention of the feature tensor p is residually connected with the feature tensor p. The result is the output O of the CA module, that is: O=Residual(p, Coordinate_attention(p)).

4. The method according to claim 2, characterized in that The CA module is specifically: Assuming that the size of the feature tensor p is C×H×W, the feature tensor p first undergoes an average pooling in the x direction and an average pooling in the y direction to obtain the output O x and O y ,Right now: Oh x =XAP(p) HE y =DO(p) O x The size is C×H×1, O y The size of C×1×W; then O x and O y After concatenation and passing through a convolution module, the output O1 is obtained. The size of O1 is C / r×1×(W+H), where r is the scaling factor, that is: O1=Conv2d(Concat(O x ,O y )); After that, O1 goes through a batch normalization module and a nonlinear activation module to obtain O2. The size of O2 is C / r×1×(W+H), that is: O2=BN(NL(O1)) Next, O2 is split into feature tensors O′ in the x-direction and y-direction with sizes of C / r×H×1 and C / r×1×W respectively. x and O′ y , then O′ x and O′ y After being processed by a convolution module and a Sigmoid activation module respectively, we get and Right now: After the above processing, and The attention weight matrices corresponding to the X and Y directions are C×1×H and C×1×W respectively; Finally, the generated weight matrix is applied to the X and Y directions of the original input features respectively, and pixel-by-pixel multiplication is performed through the broadcast mechanism to achieve feature re-weighting; that is: The calculation result of the coordinate attention is residually connected with the original input to obtain the final output O of the CA module, namely: O=Residual(p, Coordinate_attention(p)) The size of O is C×H×W, which is consistent with the original input features and includes the features of channel attention and spatial coordinate information enhancement, where C, H, and W are the three dimensions of output O respectively.

5. The method according to claim 1, characterized in that The step 3 is specifically as follows: Assume that the shallow feature f1, the intermediate feature f2, and the deep feature f3 are the inputs of the three branches of the neck network; First, after processing through a convolution module, the output O is obtained 1 ,Right now: and As the input of the ASFF2 module of the first branch and the second branch, the output O 2 ,Right now: Output of two ASFF2 modules At the same time, it serves as the input of the C3 module of the first branch and the second branch, and the output is O 3 ,Right now: Directly pass through a convolution module again to obtain Right now: at last, At the same time, it is used as the input of the ASFF3 module in the three branches. After fusion, the final output F of the neck network is obtained, that is:

6. The method according to claim 1, wherein The attributes of the target to be detected include: one or more of the position, size, and category of the target to be detected.

7. An image detection device, characterized in that: include: An acquisition module acquires the image to be detected; The first processing module processes the image through the backbone network of the YOLO model to generate extracted features; The second processing module uses the features as input to the neck network of the YOLO model, performs feature fusion processing, and obtains a feature map; The detection module detects the feature map through the detection head of the YOLO model and predicts the attributes of the target to be detected; The first processing module includes: The first processing submodule is used to process the image in sequence through two convolution modules Conv and one C3_CA module to obtain the first output of the backbone network as the shallow feature of the image; The second processing submodule is used to process the extracted shallow features through a convolution module and a C3_CA module in sequence to obtain the second output of the backbone network as the intermediate features of the image; A third processing submodule is used to process the intermediate features in sequence through a convolution module, a C3_CA module, a convolution module, a C3_CA module, and an SPPF module to obtain a third output of the backbone network as a deep feature of the image; The fourth processing submodule is configured to use the shallow features, intermediate features, and deep features of the image as extracted features.

8. A smart device, characterized in that: Comprising the device according to claim 7.

9. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the data storage and reading method according to any one of claims 1 to 6 is implemented.

10. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that The program code enables the processor to execute the method according to any one of claims 1 to 6.

Images