Method for detecting traffic signs in foggy weather

Abstract

The invention provides a method for detecting traffic signs in foggy days, and belongs to the field of image processing. By adopting the lightweight network model, the problem that the traffic sign in the foggy day cannot be accurately detected and identified due to whitening distortion and blurring of the traffic sign caused by foggy day noise is solved. Meanwhile, compared with the prior art, the method has the advantages that the parameter quantity is smaller in the training process, and the requirement for hardware needing to deploy equipment is lower. In the detection and recognition process, the accuracy is higher, and the speed is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting traffic signs in foggy conditions. Background Art

[0002] Research on traffic sign detection in foggy conditions aims to enable autonomous driving systems to operate effectively in foggy conditions, thereby improving driving safety and convenience. Currently, there are two main approaches to traffic sign detection: traditional methods and deep learning-based methods. Traditional methods primarily employ an exhaustive strategy, using classic techniques such as sliding windows and image scaling for object detection. Subsequently, object features are extracted by combining information about color and shape. Finally, methods such as support vector machines are used to classify the detected objects. However, traditional object detection methods still face several challenges, including uncertainty in object position and variations in object size and shape. To address these issues, rectangular bounding boxes are often used to define objects. However, this approach introduces new challenges due to the varying aspect ratios of objects. The high time and space requirements of the detection model make the detection process complex and cumbersome, hindering widespread adoption. Furthermore, traditional methods suffer from low detection and recognition accuracy, computational complexity, bloated models, and poor feature robustness, all of which hinder practical application. Therefore, they are not suitable for meeting the current needs of traffic sign detection in foggy conditions.

[0003] With technological advancements and the increasing computing power of computers and other devices, deep learning-based methods for foggy traffic sign detection have gradually gained attention and become a hot topic in current detection tasks. This approach eliminates the tedious steps involved in defining features and achieves end-to-end object detection. These methods can be broadly categorized into two types: one-stage and two-stage models. Two-stage methods for foggy traffic sign detection, such as R-CNN and Faster R-CNN, rely on a complete convolutional neural network for object detection and feature extraction through convolution. Their most notable feature is that their training consists of two steps: the first involves training the Recurrent Neural Network (RPN) network, and the second involves training the object detection region network. This algorithm generates candidate bounding boxes and then uses a convolutional neural network to classify these samples. Compared to traditional methods, this type of object detection algorithm offers the advantages of simplicity and higher accuracy. However, its disadvantage is its slower speed. On the other hand, single-stage detection methods, such as SSD and YOLO, do not require the involvement of an RPN network. Instead, they directly obtain object location and category information through the main network. This difference primarily benefits from faster implementation, but the disadvantage is lower accuracy compared to two-stage models.

[0004] Therefore, to better identify traffic signs in foggy road scenes, it is necessary to develop an object detection method that can reduce interference caused by background noise, achieve high detection accuracy, and utilize a lightweight and uncomplicated model. The YOLO family of object detection algorithms offers high detection efficiency, but performs poorly when detecting noisy images. The original model still suffers from several shortcomings that could be improved, such as false positives and missed detections, large model size, and a small detection scale. Summary of the Invention

[0005] In response to the technical problems mentioned in the background art, a method for detecting traffic signs in foggy conditions is provided. This method can accurately and quickly detect traffic signs in foggy conditions. This method not only ensures a lightweight model but also improves detection accuracy and generalization capabilities, making it more suitable for traffic sign detection tasks in foggy conditions.

[0006] The present invention further integrates the Atrous Selective Kernel Attention (ASKA) mechanism into the deep learning method, adjusts the receptive field range in the feature map, and adds a lightweight progressive feature pyramid network C-AFPN to alleviate the conflict of information fusion between different scales.

[0007] The technical means adopted by the present invention are as follows: a traffic sign detection method under foggy conditions based on YOLOv8, comprising the following steps:

[0008] S1. Construct a foggy traffic sign dataset: obtain a public dataset, preprocess the images in the public dataset, and divide the preprocessed dataset into a training set and a test set according to a certain ratio;

[0009] S2. Build a network model for traffic sign detection in foggy weather;

[0010] S3, training the network model constructed in S2: inputting the training set data into the network model, training the network model, and obtaining a trained model;

[0011] S4. Complete the test by testing the model trained in S3 using the test set.

[0012] Furthermore, the S1 includes the following steps:

[0013] S11: Download the traffic sign dataset CCTSDB 2021 from GitHub.

[0014] S12: Use the Mosaic data enhancement method to enhance the traffic sign dataset;

[0015] S13: The dataset CCTSDB 2021 contains foggy images and other weather images. Using the OpenCV library in Python, fog the fog-free images in the dataset CCTSDB 2021, while leaving the foggy images unprocessed. A new dataset containing the fogged images and foggy images is obtained.

[0016] S14: All images in the new dataset are randomly divided into a training set and a test set in a ratio of 9:1 to obtain a final foggy traffic sign dataset.

[0017] Furthermore, the step S2 includes the following steps:

[0018] S21: Design the hole selection kernel attention mechanism ASKA and add the attention mechanism ASKA to the last layer of the backbone network Backbone; the hole selection kernel attention mechanism ASKA includes: large kernel convolution, hole convolution, standard convolution, pooling operation and Sigmoid activation function;

[0019] S22: Replace some convolution blocks in the backbone layer with group shuffle convolution GSConv to speed up model training.

[0020] S23: Based on the idea of ​​lightweight progressive feature fusion, a lightweight progressive feature pyramid network C-AFPN based on the neck network Neck is introduced;

[0021] S24: Implement the multi-head detection structure of the Head layer, that is, use four convolutional layers with scales of 20×20, 40×40, 80×80, and 160×160 respectively to detect traffic signs of different scales;

[0022] S25: Set the SIOU loss function as the regression loss function of the YOLOv8 network; the calculation formula of the SIOU loss function is:

[0023]

[0024] Among them, Δ represents distance loss and Ω represents shape loss;

[0025] S26: Use the Backbone layer, Neck layer, Head layer and SIOU loss function to form a complete foggy traffic sign detection model.

[0026] Furthermore, the step S3 includes the following steps:

[0027] S31: Set the image input to the network model to a fixed size of 640×640 pixels;

[0028] S32: Selecting the stochastic gradient descent function SGD as the optimization function in the network model; setting the training round Epoch to 300, the batch size Bachsize to 16, the initial learning rate to 0.001 and the learning rate adjustment strategy. After each training round, the network model automatically saves the corresponding weight information;

[0029] S33: Determine the size of the training epoch;

[0030] When the training epoch is less than 50, freeze the Backbone layer, that is, set the learning rate of the Backbone layer to 0 and no longer update it;

[0031] When the epoch is greater than or equal to 50, the Backbone layer is no longer frozen and the initial learning rate is restored;

[0032] When the epoch reaches 300, the final model weights are saved, model training is completed, and the model effect evaluation is output, including the number of parameters, accuracy P, recall R, and mean average precision (mAP). The specific calculation formula is as follows:

[0033]

[0034]

[0035]

[0036] Among them, TP represents the number of positive examples predicted as positive examples, FP represents the number of negative examples predicted as positive examples, and FN represents the number of positive examples predicted as negative examples.

[0037] Furthermore, the S4 includes the following steps:

[0038] S41: Set the image resolution to 640 pixels × 640 pixels;

[0039] S42: Predict the target of the image in the test set. During the prediction process, the model will generate a target box to capture the target, use the target box to represent the range of the target object in the image, and assign a category and target confidence to the target box.

[0040] S43: Output the predicted result, i.e., the category, confidence level, and target frame coordinate position information of the traffic sign in the image, and the detection is completed.

[0041] Furthermore, the step S21 includes the following steps:

[0042] S211: Obtain background information features of different regions through large kernel convolution and void convolution, reducing the number of model parameters;

[0043] S212: Standard convolution is used to coordinate the channel dimensions of various feature maps, and average pooling and maximum pooling operations are performed to extract spatial relationships;

[0044] S213: Obtain weights from different convolutions through the Sigmoid activation function; perform weighted processing on the weights and the feature map obtained above to obtain the attention feature Y; the calculation formula is:

[0045]

[0046] in, represents convolution, Represents features from different convolutions, Represents spatial pooling features.

[0047] Furthermore, the step S23 includes the following steps:

[0048] S231: First, obtain the two features extracted from the image by the third and fifth layers in the Backbone layer; perform upsampling and downsampling operations on these two features respectively;

[0049] S232: Adaptively perform spatial fusion on the two features after the sampling operation. The fusion calculation formula is:

[0050]

[0051] in, and represents the initial eigenvector, and Indicates the spatial weights of the features at two levels at different levels, Represents the fused feature vector;

[0052] S233: extract visual features from the spatially fused features by fusing the phantom convolution block;

[0053] S234: Adaptively spatially fuse the features obtained in S233 with the seventh-layer visual features in the Backbone layer, and extract visual features through the phantom convolution block;

[0054] S235: The features obtained in S234 are adaptively spatially fused with the tenth layer visual features in the Backbone layer, and the visual features are extracted through the phantom convolution block.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] 1. This paper proposes a dilated selective kernel attention mechanism (ASKA). This mechanism dynamically adjusts the receptive field of a feature map, reducing the impact of foggy noise on traffic signs. It uses dilated convolution to expand the receptive field and a selective attention mechanism to enhance feature extraction.

[0057] 2. This paper proposes a lightweight progressive feature pyramid network (C-AFPN) that addresses the suboptimal fusion problem caused by non-adjacent scales in feature pyramid networks. Features are gradually fused from low to high scales, alleviating semantic conflicts between non-adjacent levels. The subsequent multi-head detection architecture can adapt to the diversity of traffic sign scales and improve detection accuracy.

[0058] 3. This paper proposes an improved YOLOv8-based foggy traffic sign detection model, which has high precision and lightweight effect. It can be deployed in practical applications and has excellent detection effect on traffic signs in foggy conditions.

[0059] Based on the above reasons, the present invention can be widely used in multiple devices such as drones, cars, and monitoring devices in foggy conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 This is a flow chart of a method for detecting traffic signs in foggy weather based on improved YOLOv8 in the present invention;

[0062] Figure 2 This is a schematic diagram of the YOLOv8 structure of the present invention;

[0063] Figure 3 Schematic diagram of the ASKA attention mechanism structure of the present invention;

[0064] Figure 4 Schematic diagram of the C-AFPN network structure of the present invention. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0066] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] like Figure 1-4 As shown, the present invention provides a traffic sign detection method under foggy conditions based on YOLOv8, comprising the following steps:

[0068] S1. Construct a foggy traffic sign dataset: obtain a public dataset, preprocess the images in the public dataset, and divide the preprocessed dataset into a training set and a test set according to a certain ratio.

[0069] Preferably, in the present application, S1 includes the following steps:

[0070] S11: Download the traffic sign dataset CCTSDB 2021 from the GitHub website; it can be understood that in other implementations, different public datasets can also be downloaded to realize traffic sign detection.

[0071] S12: Mosaic data augmentation is used to enhance the traffic sign dataset. Four images are randomly selected and randomly scaled and cropped. Random scaling is controlled by the scale factor, a random number between 0.6 and 0.8. Random cropping involves cropping at random locations, cropping the image to a fixed size of 640 × 640 pixels. The four processed images are then concatenated into a new image at the top left, top right, bottom left, and bottom right positions.

[0072] S13: The dataset contains foggy images and other weather images. Using the OpenCV library in Python, fogging is performed on the fog-free images in the dataset, while foggy images are not processed.

[0073] S14: Divide all images in the dataset into a training set and a test set at a ratio of 9:1 to obtain the final foggy traffic sign dataset. It should be understood that in other implementations, the specific division ratio can be determined based on the actual amount of data in the dataset, as long as accurate and clear line of sight data is required for training and testing.

[0074] Furthermore, S2: constructing a network model for foggy traffic sign detection. S2 includes the following steps:

[0075] S21: Design a hole selection kernel attention mechanism ASKA, and add the attention mechanism ASKA to the last layer of the backbone network Backbone; the hole selection kernel attention mechanism ASKA includes: large kernel convolution, hole convolution, standard convolution, pooling operation and Sigmoid activation function; S21 includes the following steps:

[0076] S211: Image data is convolved with large kernel convolution and dilated convolution to obtain background information features of different regions, reducing the number of model parameters.

[0077] S212: Standard convolution is used to coordinate the channel dimensions of various feature maps, and average pooling and maximum pooling operations are performed to extract spatial relationships;

[0078] S213: Obtain weights from different convolutions through the Sigmoid activation function; perform weighted processing on the weights and the feature map obtained above to obtain the attention feature Y. The calculation formula is:

[0079]

[0080] in, represents convolution, Represents features from different convolutions, Represents spatial pooling features.

[0081] S22: Replace some convolution blocks in the backbone layer with group shuffle convolution GSConv to speed up model training.

[0082] S23: Based on the idea of ​​lightweight progressive feature fusion, a lightweight progressive feature pyramid network C-AFPN of the neck network Neck is designed; S23 includes the following steps:

[0083] S231: First, obtain the two features extracted from the image by the third and fifth layers of the Backbone layer. The features mainly include basic visual elements such as color, texture, shape, and edge. Then, perform upsampling and downsampling operations on these two features respectively.

[0084] S232: Adaptively perform spatial fusion on the two features after the sampling operation. The fusion calculation formula is:

[0085]

[0086] in, and represents the initial eigenvector, and Indicates the spatial weights of the features at two levels at different levels, Represents the fused feature vector;

[0087] S233: extract visual features from the spatially fused features by fusing the phantom convolution block;

[0088] S234: The features obtained in S233 are adaptively spatially fused with the seventh-layer visual features in the Backbone layer, and the visual features are extracted through the phantom convolution block.

[0089] S235: The features obtained in S234 are adaptively spatially fused with the tenth layer visual features in the Backbone layer, and the visual features are extracted through the phantom convolution block.

[0090] S24: Implement the multi-head detection structure of the Head layer, that is, use four convolutional layers with scales of 20×20, 40×40, 80×80, and 160×160 to detect objects of different scales;

[0091] S25: Set the SIOU loss function as the regression loss function of the improved YOLOv8 network; the calculation formula of the SIOU loss function is:

[0092]

[0093] Among them, Δ represents distance loss and Ω represents shape loss;

[0094] S26: A complete foggy traffic sign detection model is constructed using the Backbone, Neck, and Head layers, along with the SIOU loss function. Image data first enters the Backbone layer for feature extraction, then the Neck layer for feature fusion, and finally the Head layer. The Head layer generates candidate bounding boxes to predict objects in the image. The SIOU loss function uses distance loss and shape loss to adjust the prediction results, achieving an optimal balance.

[0095] As a preferred embodiment, S3, training the network model constructed in S2: inputting the training set data into the network model to train the network model; S3 includes the following steps:

[0096] S31: Set the image input to the network to a fixed size of 640×640 pixels;

[0097] S32: Select the stochastic gradient descent function SGD as the optimization function in the network, set the epoch to 300, the batch size to 16, the initial learning rate to 0.001, and the learning rate adjustment strategy. The model will save the model weights once after each training round.

[0098] S33: The network starts training. When the epoch is less than 50, the Backbone layer is frozen, that is, the learning rate of the Backbone layer is set to 0 and no longer updated.

[0099] S34: When the epoch is greater than or equal to 50, the Backbone layer is no longer frozen and the learning rate is restored.

[0100] S35: When the epoch reaches 300, the final model weights are saved, the model training is completed, and the model effect evaluation is output, including the number of parameters, accuracy P, recall R, and mean average precision (mAP). The specific calculation formula is as follows:

[0101]

[0102]

[0103]

[0104] Among them, TP represents the number of positive examples predicted as positive examples, FP represents the number of negative examples predicted as positive examples, and FN represents the number of positive examples predicted as negative examples.

[0105] Preferably, S4 is to complete the test by testing the model trained in S3 with a test set. S4 includes the following steps:

[0106] S41: Set the image resolution to 640×640 pixels;

[0107] S42: During training, the higher the mean average precision (mAP) value of the model, the better the model's detection performance, meaning it accurately detects more objects and has fewer missed or false detections. The best model during training is selected and used to predict objects on images in the test set. During the prediction process, the model generates a target box to capture the object, using the target box to represent the range of the target object in the image, and assigns a category and target confidence to the target box.

[0108] S43: Output the predicted result, i.e., the category, confidence level, and target frame coordinate position information of the traffic sign in the image, and the detection is completed.

[0109] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0110] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0112] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0113] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A traffic sign detection method in foggy weather based on YOLOv8, characterized in that: The following steps are involved: S1. Construct a foggy traffic sign dataset: obtain a public dataset, preprocess the images in the public dataset, and divide the preprocessed dataset into a training set and a test set according to a certain ratio; S2, building a network model for traffic sign detection in foggy weather; S3, training the network model constructed in S2; inputting the data of the training set into the network model, training the network model, and obtaining a trained model; S4. Test the model trained in S3 using a test set, complete the test, and obtain the final model.

2. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S1 comprises the following steps: S11: Download the traffic sign dataset CCTSDB 2021 from the GitHub website; S12: Use the Mosaic data enhancement method to enhance the traffic sign dataset; S13: The dataset CCTSDB 2021 includes foggy images and other weather images; fogging the fog-free images in the dataset CCTSDB 2021 using the OpenCV library in Python, while leaving the foggy images unprocessed; obtaining a new dataset including fogged images and foggy images; S14: All the images in the new data set are randomly divided into a training set and a test set in a ratio of 9:1 to obtain a final foggy traffic sign data set.

3. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S2 comprises the following steps: S21: Design a hole selection kernel attention mechanism ASKA, and add the attention mechanism ASKA to the last layer of the backbone network Backbone; the hole selection kernel attention mechanism ASKA includes: large kernel convolution, hole convolution, standard convolution, pooling operation and Sigmoid activation function; S22: Replace some convolution blocks in the backbone network Backbone layer with group shuffle convolution GSConv to speed up model training; S23: Based on the idea of ​​lightweight asymptotic feature fusion, a lightweight asymptotic feature pyramid network C-AFPN based on the neck network Neck is introduced; S24: Implement the multi-head detection structure of the Head layer, that is, use four convolutional layers with scales of 20×20, 40×40, 80×80, and 160×160 to detect traffic signs of different scales; S25: Set the SIOU loss function as the regression loss function of the YOLOv8 network; the calculation formula of the SIOU loss function is: Among them, Δ represents distance loss, Ω represents shape loss; S26: The foggy traffic sign detection model is represented by the Backbone layer, Neck layer, Head layer and SIOU loss function.

4. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S3 includes the following steps: S31: Set the image input to the network model to a fixed size of 640×640 pixels; S32: Selecting the stochastic gradient descent function SGD as the optimization function in the network model; setting the training round Epoch to 300, the batch size Bachsize to 16, the initial learning rate to 0.001 and the strategy for adjusting the learning rate, and the network model automatically saves the corresponding weight information after each training round; S33: Determine the size of the training epoch; When the training epoch is less than 50, freeze the Backbone layer, that is, set the learning rate of the Backbone layer to 0 and no longer update it; When the Epoch is greater than or equal to 50, the Backbone layer is no longer frozen and the initial learning rate is restored; When the epoch reaches 300, the final model weight is saved, the model training is completed, and the evaluation of the model effect is output, including the number of parameters, accuracy P, recall rate R and average precision mAP. The calculation formula is as follows: Among them, TP represents the number of positive examples predicted as positive examples, FP represents the number of negative examples predicted as positive examples, and FN represents the number of positive examples predicted as negative examples.

5. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S4 includes the following steps: S41: set the image resolution to 640 pixels × 640 pixels; S42: Predict the target for the images in the test set. During the prediction process, the model generates a target box to capture the target, uses the target box to represent the range of the target object in the image, and assigns a category and target confidence to the target box. S43: Output the predicted result, i.e., the category, confidence level, and target frame coordinate position information of the traffic sign in the image, and the detection is completed.

6. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S21 includes the following steps: S211: Obtain background information features of different regions through large kernel convolution and hole convolution to reduce the number of model parameters; S212: Standard convolution is used to coordinate the channel dimensions of various feature maps, and average pooling and maximum pooling operations are performed to extract spatial relationships; S213: Obtain weights from different convolutions through the Sigmoid activation function; perform weighted processing on the weights and the feature map obtained above to obtain the attention feature Y; the calculation formula of the attention feature Y is: in, represents convolution, represents features from different convolutions, Represents spatial pooling features.

7. The method for detecting traffic signs in foggy weather based on YOLOv8 according to claim 1, characterized in that: The S23 includes the following steps: S231: first obtain two features extracted from the image by the third and fifth layers in the Backbone layer; perform upsampling and downsampling operations on the two features respectively; S232: Adaptively spatially fuse the two features after the sampling operation. The fusion calculation formula is: in, and represents the initial eigenvector, and Represents the spatial weights of the features at two levels at different levels, Represents the fused feature vector; S233: extract visual features from the spatially fused features by fusing the phantom convolutional blocks; S234: Adaptively spatially fuse the features obtained in S233 with the seventh layer visual features in the Backbone layer, and extract visual features through the phantom convolution block; S235: Adaptively spatially fuse the features obtained in S234 with the tenth layer visual features in the Backbone layer, and extract visual features through the phantom convolution block.

Images