Traffic scene target detection method and system

By combining CenterNet with multi-scale feature fusion module, attention aggregation module and boundary feature extraction module in traffic scene object detection, the balance problem between detection accuracy and speed is solved, and efficient object detection in complex traffic scenes is achieved.

CN113869165BActive Publication Date: 2024-11-01SHANDONG NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111101011.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-18
Publication Date
2024-11-01
Estimated Expiration
2041-09-18

AI Technical Summary

Technical Problem

The existing traffic scene object detection methods are difficult to balance between detection accuracy and speed. The anchor-based method has not been ideal for detection speed, while the anchor-free method sacrifices accuracy in pursuit of speed, especially in complex traffic scenarios.

Method used

A traffic scene object detection method is designed, using CenterNet as the baseline, features are extracted through the backbone network, combined with the scale fusion module and attention aggregation module, and further enhance the feature representation of the detection head through the boundary feature extraction module to achieve the final classification and Regression prediction.

Benefits of technology

Through multi-scale feature fusion and boundary feature extraction, the accuracy and speed of detection are improved, and problems such as low detection accuracy, slow speed, false detection, and missed detection are solved, and have good generalization capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113869165B_ABST
    Figure CN113869165B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for detecting a target in a traffic scene, comprising: obtaining an image to be detected in a traffic scene; extracting features from the image to be detected to extract a plurality of features; fusing the extracted features to obtain fused features; performing feature enhancement processing on the fused features to obtain preliminary classification features and preliminary regression features; extracting boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; fusing the preliminary classification features with the boundary classification features to obtain final classification features; fusing the preliminary regression features with the boundary regression features to obtain final regression features; obtaining a final classification prediction result based on the final classification features and a trained classifier; obtaining a final regression prediction result based on the final regression features and a trained regressor. The method solves the problems of low detection accuracy, slow speed, false detection, missed detection, etc. existing in the previous methods, and has good generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method and system for detecting a target in a traffic scene. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] Traffic scene target detection is a basic task in intelligent transportation systems. It is necessary to locate and classify traffic scene targets in real time and accurately. Therefore, improving traffic scene target detection has great practical significance and application value. The traditional target detection method is to first select some candidate regions on a given image, then extract features from these regions, and finally use a classifier for classification. However, this sliding window-based region selection strategy will lead to excessive time complexity and window redundancy problems, and the manually designed features are not very robust to changes in diversity.

[0004] With the rapid development of deep convolutional neural networks, the performance of target detection has been significantly improved, but there are still some problems. At present, target detection methods based on convolutional neural networks can be divided into anchor-based methods and anchor-free methods. In the anchor-based method, Faster R-CNN uses a two-stage structure to generate target candidate boxes first, and then classify and regress the candidate boxes. This type of method uses a large number of anchors as prior boxes, which not only introduces additional hyperparameters, but also causes an extreme imbalance between positive and negative samples. Although its detection accuracy has improved, the detection speed is not ideal. In the anchor-free method, CenterNet uses key point estimation to find the center point and regress to other target attributes. This type of method does not use any predefined prior boxes as a reference, but solves the detection problem by determining the key points of the target and predicting the four bounding boxes. However, the anchor-free detection method sacrifices accuracy in pursuit of speed, which means that while its detection speed is improved, its detection accuracy is not as good as that of the anchor-based detection method, and when it faces some more complex traffic scenes, the detection effect is not ideal. Therefore, there is still a lot of room for development in the research on target detection in traffic scenes, and achieving a trade-off between detection accuracy and speed is an important issue that the present invention needs to solve. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the present invention provides a method and system for detecting traffic scene targets;

[0006] In a first aspect, the present invention provides a method for detecting objects in traffic scenes;

[0007] A traffic scene target detection method, comprising:

[0008] Acquire the image to be detected in the traffic scene; perform feature extraction on the image to be detected and extract several features;

[0009] Perform feature fusion on several extracted features to obtain fused features; perform feature enhancement processing on the fused features to obtain preliminary classification features and preliminary regression features;

[0010] Extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features; fuse preliminary classification features with boundary classification features to obtain final classification features; fuse preliminary regression features with boundary regression features to obtain final regression features;

[0011] Based on the final classification features and the trained classifier, the final classification prediction result is obtained; based on the final regression features and the trained regressor, the final regression prediction result is obtained.

[0012] In a second aspect, the present invention provides a traffic scene target detection system;

[0013] A traffic scene target detection system, comprising:

[0014] The feature extraction module is configured to: obtain an image to be detected in a traffic scene; perform feature extraction on the image to be detected to extract a number of features;

[0015] The feature fusion and enhancement module is configured to: perform feature fusion on the extracted features to obtain fusion features; perform feature enhancement processing on the fusion features to obtain preliminary classification features and preliminary regression features;

[0016] The boundary feature extraction module is configured to: extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features; fuse the preliminary classification features with the boundary classification features to obtain final classification features; fuse the preliminary regression features with the boundary regression features to obtain final regression features;

[0017] The prediction module is configured to: obtain a final classification prediction result based on the final classification feature and the trained classifier; and obtain a final regression prediction result based on the final regression feature and the trained regressor.

[0018] In a third aspect, the present invention further provides an electronic device, comprising:

[0019] a memory for non-transitory storage of computer-readable instructions; and

[0020] a processor for executing the computer readable instructions,

[0021] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0022] In a fourth aspect, the present invention further provides a storage medium that non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] Richer semantic information is obtained by fusing multi-scale features, and head classification and regression are enhanced by extracting boundary features.

[0025] By aggregating multiple attention modules, the representation ability of the network is improved, and the problems of low detection accuracy, slow speed, false detection, missed detection, etc. in previous methods are solved, and it has good generalization ability.

[0026] Advantages of additional aspects of the present invention will be partially given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0028] Figure 1 A network structure diagram of SABNet of the first embodiment;

[0029] Figure 2 It is a structural diagram of the scale fusion module of the first embodiment;

[0030] Figure 3 is a structural diagram of the attention aggregation module of the first embodiment;

[0031] Figure 4 It is a structural diagram of the boundary feature extraction module of the first embodiment;

[0032] Figure 5(a) to Figure 5(p) The detection results of our method in the first embodiment on the BDD100K test set;

[0033] Figure 6(a) to Figure 6(c) This is the PR curve of our method in the first embodiment on the KITTI test set;

[0034] Figure 7 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0038] In this embodiment, all data is obtained in compliance with laws and regulations and based on the user's consent, and is used legally.

[0039] In order to balance the detection accuracy and speed of traffic scene targets, this paper designs an anchor-free detection method based on CenterNet as the baseline to detect traffic scene targets. Figure 1 As shown. First, the present invention inputs the image into the backbone network ResNet-101 to extract features, and the extracted features are generated by the scale fusion module (SFM) to generate fused features with high resolution and high-level semantic information. Secondly, the fused features are passed through the attention aggregation module (AAM) to enhance the feature representation of the target detection head. In addition, the attention aggregation module uses the output preliminary classification features and preliminary regression features as the input of the boundary feature extraction module (BFEM) to generate boundary classification features and boundary regression features. Finally, the preliminary features and boundary features are simply fused to obtain the final classification prediction and regression prediction, thereby obtaining the final detection result.

[0040] Embodiment 1

[0041] This embodiment provides a method for detecting traffic scene targets;

[0042] like Figure 7 As shown, a traffic scene target detection method includes:

[0043] S101: Acquire an image to be detected in a traffic scene; perform feature extraction on the image to be detected to extract a number of features;

[0044] S102: performing feature fusion on the extracted features to obtain fused features; performing feature enhancement processing on the fused features to obtain preliminary classification features and preliminary regression features;

[0045] S103: extracting boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; fusing the preliminary classification features with the boundary classification features to obtain final classification features; fusing the preliminary regression features with the boundary regression features to obtain final regression features;

[0046] S104: Based on the final classification features and the trained classifier, a final classification prediction result is obtained; based on the final regression features and the trained regressor, a final regression prediction result is obtained.

[0047] Furthermore, in S101, feature extraction is performed on the image to be detected to extract a number of features; specifically, the RestNet 101 network is used to extract the feature of the image to be detected to extract a number of features.

[0048] The RestNet 101 network includes: a convolutional layer a1, a convolutional layer a2, a convolutional layer a3, and a convolutional layer a4 connected in sequence;

[0049] The input value of the convolution layer a1 is the image to be detected; the output value of the convolution layer a1 is the feature B2;

[0050] The input value of the convolution layer a2 is feature B2; the output value of the convolution layer a2 is feature B3;

[0051] The input value of the convolution layer a3 is feature B3; the output value of the convolution layer a3 is feature B4;

[0052] The input value of the convolution layer a4 is feature B4; the output value of the convolution layer a4 is feature B5.

[0053] Furthermore, the S102: performing feature fusion on the extracted features to obtain fused features; specifically, using a scale fusion module (SFM) to perform feature fusion on the extracted features to obtain fused features.

[0054] Furthermore, the scale fusion module SFM is adopted, and its network structure includes: a convolution layer b1, an upsampling layer c1, an adder d1, a convolution layer b2, an upsampling layer c2, an adder d2, a convolution layer b3, an upsampling layer c3, an adder d3 and a convolution layer b4 connected in sequence;

[0055] Convolutional layer b1 is also connected to convolutional layer a4 through convolutional layer f1;

[0056] Adder d1 is also connected to convolutional layer a3 through convolutional layer f2;

[0057] Adder d2 is also connected to convolutional layer a2 through convolutional layer f3;

[0058] Adder d3 is also connected to convolutional layer a1 through convolutional layer f4;

[0059] Convolutional layer b2 and convolutional layer b3 are also connected to adder d4;

[0060] Convolution layer b1, convolution layer b4 and adder d4 are also connected to the splicer.

[0061] Furthermore, the scale fusion module SFM is used to fuse the extracted features to obtain fused features; the working principle includes:

[0062] Features B2, B3, B4, and B5 of different scales are obtained through the convolution operation of ResNet-101;

[0063] Perform 1×1 convolution operations on features B2, B3, B4, and B5, respectively, to obtain features F2, F3, F4, and F5;

[0064] After the feature F5 is subjected to a 3×3 convolution operation, feature S5 is obtained; feature S5 is upsampled;

[0065] The feature F4 is fused with the feature S5 after the upsampling operation to obtain a first fusion result, and the first fusion result is subjected to a 3×3 convolution operation to obtain a feature S4; the feature S4 is upsampled;

[0066] The feature F3 is fused with the feature S4 after the upsampling operation to obtain a second fusion result, and the second fusion result is subjected to a 3×3 convolution operation to obtain a feature S3; the feature S3 is upsampled;

[0067] The feature F2 is fused with the feature S3 after the upsampling operation to obtain a third fusion result, and the third fusion result is subjected to a 3×3 convolution operation to obtain feature S2;

[0068] The feature S3 is fused with the feature S4 to obtain a fourth fusion result, and the fourth fusion result is connected with the feature S5 and the feature S2 to obtain a final fusion feature.

[0069] It should be understood that in order to solve the problem of multi-scale target detection in traffic scene target detection, the present invention designs a scale fusion module to integrate features of different scales. The structure of the scale fusion module is as follows: Figure 2 shown.

[0070] First, the input image is subjected to the convolution operation of the ResNet-101 convolution block to obtain features of different scales {B2, B3, B4, B5}, and these features are subjected to the 1×1 convolution operation to obtain features of corresponding sizes {F2, F3, F4, F5}. The main function of the 1×1 convolution kernel here is to reduce the number of feature maps without changing the size of the feature maps. Secondly, the upsampling result is fused with the features of the same size generated from the bottom up, and the fused features are subjected to a 3×3 convolution operation to obtain features {S2, S3, S4, S5} with the same size and channel as the features {F2, F3, F4, F5}. Among them, S2 and S5 are responsible for predicting small-scale targets and large-scale targets, respectively, and S3 and S4 are responsible for predicting medium-scale targets. Finally, the present invention first fuses S3 and S4, and then concats the fused result with S2 and S5 to obtain the final fused feature.

[0071] The scale fusion module mainly uses the idea of ​​feature pyramid network to obtain more powerful semantic features by fusing high-resolution low-level features and high-level features with high-level semantic information. Therefore, the scale fusion module can well cope with the problem of multi-scale object detection.

[0072] Furthermore, the S102: performing feature enhancement processing on the fused features to obtain preliminary classification features and preliminary regression features; specifically: using an attention aggregation module (AAM, Attention Aggregation Module) to perform feature enhancement processing on the fused features to obtain preliminary classification features and preliminary regression features.

[0073] Furthermore, the network structure of the attention aggregation module AAM includes:

[0074] Several attention aggregation submodules connected in series;

[0075] like Figure 3 As shown, each attention aggregation submodule includes: a scale attention unit, a spatial attention unit and a channel attention unit connected in series in sequence; wherein the output end of the channel attention unit of each submodule is also connected to the input end of the scale attention unit of the current submodule.

[0076] Furthermore, the scale attention unit enhances feature representation by identifying three scales in the fused features.

[0077] Exemplarily, the scale attention unit is expressed as:

[0078]

[0079]

[0080] in, represents the Hard Sigmoid function. δ(·) represents the ReLU function, f 1×1 (·) is a linear function of 1×1 convolution. It represents the global average pooling of the fusion feature T in the scale attention unit. Represents a scaled attention unit.

[0081] Furthermore, the spatial attention unit not only pays attention to each spatial position, but also adaptively aggregates multiple feature layers together to learn more discriminative representations.

[0082] Exemplarily, the spatial attention unit is expressed by the formula:

[0083]

[0084]

[0085] in, It represents the Sigmoid function, f 3×3 (·) is the linear function of 3×3 convolution. and Respectively represent the scale refinement feature T′ in the spatial attention unit Global average pooling and global maximum pooling performed in; Indicates that the two pooling results are concat-operated.

[0086] Furthermore, the channel attention unit is used to focus on each task (classification and regression) of the channel and to indicate the importance of each channel in the next stage.

[0087] Exemplarily, the channel attention unit is expressed as:

[0088]

[0089]

[0090] in, Represents two fully connected layers. x is a scaling parameter, which is mainly used to reduce the computational complexity and parameter amount of the network. Represents the spatial refinement feature T" in the channel attention unit The global average pooling is performed in , and T"′ represents the channel refinement feature.

[0091] It should be understood that, generally speaking, after the backbone network extracts the features, it can obtain the final three prediction results through some simple convolution operations. However, although this method of directly obtaining the prediction results is simple, the final detection effect is not ideal. Therefore, the present invention proposes an attention aggregation module to enhance the prediction ability of the head.

[0092] As an enhancement module of the head, the attention aggregation module improves the representation ability of the detection head by aggregating scale attention, spatial attention and channel attention. Experimental results show that the module improves the detection accuracy with very little time consumption. Since the attention aggregation module is composed of multiple blocks, the output of the previous block is also the input of the next block. Through multiple experiments, the present invention found that superimposing four blocks can achieve the best detection performance.

[0093] Furthermore, the S103: extracting boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; specifically includes: using a boundary feature extraction module (BFEM, Boundary Feature Extraction Module) to extract boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features.

[0094] Furthermore, the network structure of the boundary feature extraction module BFEM includes:

[0095] The convolutional layer g1, the maximum pooling layer and the convolutional layer g2 are connected in sequence.

[0096] Furthermore, the boundary feature extraction module BFEM is used to extract boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; the specific working principle includes:

[0097] Find the four boundary points and center point of the target, and determine the position of the target bounding box based on the four boundary points and center point;

[0098] According to the position of the target bounding box, each edge is evenly divided into six points, and the feature value of each point is subjected to the maximum pooling operation to obtain the output boundary feature map;

[0099] Then perform convolution operation on the output boundary feature map to obtain boundary classification features and boundary regression features.

[0100] In fact, there is a large amount of complex background information in daily traffic scenes, and the existence of this information will cause great interference to traffic scene target detection. However, there is usually a small amount of foreground information on the boundary of the target. Therefore, in order to further enhance the classification and regression of the head, the present invention proposes a boundary feature extraction module to effectively extract boundary features. The present invention determines the position of the target bounding box by finding the four critical points and the center point of the target. Then, according to the position of the target bounding box, each boundary is evenly divided into six points, and the feature values ​​of these six points are subjected to the maximum pooling operation to obtain the output boundary feature map. Output boundary feature map The available formula is:

[0101]

[0102] in Represents the eigenvalue of the output boundary feature map at the (x, y) point of the cth channel. V c (x0, y0) is the feature value at the center of the target. (x1, y1) and (x2, y2) are the lower left and lower right corner coordinates of the target bounding box, respectively.

[0103] The structure of the boundary feature extraction module is as follows: Figure 4 As shown. The boundary feature extraction module mainly obtains the output boundary feature map by performing the maximum pooling operation on each k channel in the boundary perception feature map. Then the output boundary feature map of 5k channels is subjected to the convolution operation to obtain the final boundary feature. The present invention fuses the obtained boundary features (cls2 and reg) with the classification features (cls1) and the regression features (cp1 and wh1), respectively, and uses the fused results (cls+, cp+ and wh+) to detect the final prediction of the head.

[0104] Furthermore, the preliminary classification features are fused with the boundary classification features to obtain the final classification features; the specific fusion method adopted is product fusion.

[0105] Furthermore, the preliminary regression features are fused with the boundary regression features to obtain the final regression features; the specific fusion method adopted is splicing fusion.

[0106] Furthermore, the step S104: obtaining a final classification prediction result based on the final classification feature and the trained classifier; wherein the training step of the trained classifier includes:

[0107] Construct a classifier; construct a training set; the training set includes: image classification features with known classification labels; use the training set to train the classifier to obtain a trained classifier.

[0108] Furthermore, in S104: obtaining a final regression prediction result based on the final regression feature and the trained regressor; wherein the training step of the trained classifier includes:

[0109] Construct a regressor; construct a training set; the training set includes: image regression features with known regression labels; use the training set to train the regressor to obtain a trained classifier.

[0110] The regression label refers to the real bounding box and center point of the target. The loss function is used to measure the inconsistency between the model's predicted value and the real value. The smaller the loss function, the better the robustness of the model. The loss function in the present invention consists of classification loss and regression loss, where the classification loss refers to the class score loss, and the regression loss refers to the center point offset loss and the bounding box size offset loss.

[0111] The present invention trains and evaluates the model of the present invention on the BDD100K and KITTI target detection benchmarks, and demonstrates the superiority of the present invention method by comparing it with other advanced methods. In the experiment, the data sets used by the present invention are all in PASCAL VOC format, and mAP and FPS are used as evaluation indicators for the accuracy and speed of traffic scene target detection. The test results are shown in the following table, and the method of the present invention achieves the best detection performance.

[0112] The visualization of the detection results of the method of the present invention on the BDD100K test set is as follows: Figure 5(a) to Figure 5(p) The present invention shows the detection results of the method in daytime, rainy daytime, night and rainy night. The results show that the method of the present invention can effectively detect traffic scene targets of different scales under the interference of complex background information.

[0113] Figure 6(a) to Figure 6(c) The precision-recall curve of the method of the present invention on the KITTI dataset is shown.

[0114] Table 1 Comparison with other advanced methods on the BDD100K test set

[0115]

[0116] Table 2 Comparison with other traffic scene target detection methods on the KITTI test set

[0117]

[0118] In summary, the method of the present invention can accurately locate and classify traffic scene targets, which not only overcomes the interference of complex background information, but also solves the multi-scale detection problem, and finally achieves a trade-off between detection accuracy and detection speed.

[0119] Embodiment 2

[0120] This embodiment provides a traffic scene target detection system;

[0121] A traffic scene target detection system, comprising:

[0122] The feature extraction module is configured to: obtain an image to be detected in a traffic scene; perform feature extraction on the image to be detected to extract a number of features;

[0123] The feature fusion and enhancement module is configured to: perform feature fusion on the extracted features to obtain fusion features; perform feature enhancement processing on the fusion features to obtain preliminary classification features and preliminary regression features;

[0124] The boundary feature extraction module is configured to: extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features; fuse the preliminary classification features with the boundary classification features to obtain final classification features; fuse the preliminary regression features with the boundary regression features to obtain final regression features;

[0125] The prediction module is configured to: obtain a final classification prediction result based on the final classification feature and the trained classifier; and obtain a final regression prediction result based on the final regression feature and the trained regressor.

[0126] It should be noted that the feature extraction module, feature fusion and enhancement module, boundary feature extraction module and prediction module correspond to steps S101 to S104 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0127] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0129] Embodiment 3

[0130] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.

[0131] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0132] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0133] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0134] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0135] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0136] Embodiment 4

[0137] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A traffic scene target detection method, characterized in that: include: Obtain the image to be detected in the traffic scene; perform feature extraction on the image to be detected, and extract several features, specifically using the ResNet-101 network to extract features from the image to be detected; The extracted features are fused to obtain fused features, specifically by using a scale fusion module; the fused features are enhanced to obtain preliminary classification features and preliminary regression features, specifically by using an attention aggregation module to enhance the fused features to obtain preliminary classification features and preliminary regression features; The attention aggregation module is an attention aggregation module AAM, and its network structure includes: a plurality of attention aggregation submodules connected in series; each attention aggregation submodule includes: a scale attention unit, a spatial attention unit and a channel attention unit connected in series; wherein the output end of the channel attention unit of each submodule is also connected to the input end of the scale attention unit of the current submodule; The preliminary classification features are subjected to boundary feature extraction to obtain boundary classification features and boundary regression features, specifically including: using a boundary feature extraction module to extract boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; using the boundary feature extraction module to extract boundary features from the preliminary classification features to obtain boundary classification features and boundary regression features; the specific working principle includes: Find the four boundary points and center point of the target, and determine the position of the target bounding box based on the four boundary points and center point; According to the position of the target bounding box, each edge is evenly divided into six points, and the feature value of each point is subjected to the maximum pooling operation to obtain the output boundary feature map; Then perform convolution operation on the output boundary feature map to obtain boundary classification features and boundary regression features; The preliminary classification features are combined with the boundary classification features to obtain the final classification features; the preliminary regression features are combined with the boundary regression features to obtain the final regression features; Based on the final classification features and the trained classifier, the final classification prediction result is obtained; based on the final regression features and the trained regressor, the final regression prediction result is obtained.

2. A traffic scene target detection method as claimed in claim 1, characterized in that: Perform feature extraction on the image to be detected and extract several features; Specifically, the ResNet-101 network is used to extract features of the image to be detected and extract several features; The ResNet-101 network includes: a convolutional layer a1, a convolutional layer a2, a convolutional layer a3 and a convolutional layer a4 connected in sequence; The input value of the convolution layer a1 is the image to be detected; the output value of the convolution layer a1 is the feature B2; The input value of the convolution layer a2 is feature B2; the output value of the convolution layer a2 is feature B3; The input value of the convolution layer a3 is feature B3; the output value of the convolution layer a3 is feature B4; The input value of the convolution layer a4 is feature B4; the output value of the convolution layer a4 is feature B5.

3. A traffic scene target detection method as claimed in claim 1, characterized in that: Perform feature fusion on several extracted features to obtain fused features; Specifically, a scale fusion module is used to fuse several extracted features to obtain fused features; The scale fusion module SFM is adopted, and its network structure includes: a convolution layer b1, an upsampling layer c1, an adder d1, a convolution layer b2, an upsampling layer c2, an adder d2, a convolution layer b3, an upsampling layer c3, an adder d3 and a convolution layer b4 connected in sequence; Convolutional layer b1 is also connected to convolutional layer a4 through convolutional layer f1; Adder d1 is also connected to convolutional layer a3 through convolutional layer f2; Adder d2 is also connected to convolutional layer a2 through convolutional layer f3; Adder d3 is also connected to convolutional layer a1 through convolutional layer f4; Convolutional layer b2 and convolutional layer b3 are also connected to adder d4; Convolutional layer b1, convolutional layer b4 and adder d4 are also connected to the splicer.

4. A traffic scene target detection method as claimed in claim 3, characterized in that: The scale fusion module is used to fuse several extracted features to obtain fused features. The working principle includes: Features B2, B3, B4, and B5 of different scales are obtained through the convolution operation of ResNet-101; Perform 1×1 convolution operations on features B2, B3, B4, and B5, respectively, to obtain features F2, F3, F4, and F5; After the feature F5 is subjected to a 3×3 convolution operation, feature S5 is obtained; feature S5 is upsampled; The feature F4 is fused with the feature S5 after the upsampling operation to obtain a first fusion result, and the first fusion result is subjected to a 3×3 convolution operation to obtain a feature S4; the feature S4 is upsampled; The feature F3 is fused with the feature S4 after the upsampling operation to obtain a second fusion result, and the second fusion result is subjected to a 3×3 convolution operation to obtain a feature S3; the feature S3 is upsampled; The feature F2 is fused with the feature S3 after the upsampling operation to obtain a third fusion result, and the third fusion result is subjected to a 3×3 convolution operation to obtain feature S2; The feature S3 is fused with the feature S4 to obtain a fourth fusion result, and the fourth fusion result is connected with the feature S5 and the feature S2 to obtain a final fusion feature.

5. A traffic scene target detection system, characterized in that: include: The feature extraction module is configured to: obtain an image to be detected in a traffic scene; perform feature extraction on the image to be detected to extract a number of features, specifically using a ResNet-101 network to extract features from the image to be detected; The feature fusion and enhancement module is configured to: perform feature fusion on the extracted features to obtain fusion features, specifically using a scale fusion module; perform feature enhancement processing on the fusion features to obtain preliminary classification features and preliminary regression features, specifically: use an attention aggregation module to perform feature enhancement processing on the fusion features to obtain preliminary classification features and preliminary regression features; The attention aggregation module is an attention aggregation module AAM, and its network structure includes: a plurality of attention aggregation submodules connected in series; each attention aggregation submodule includes: a scale attention unit, a spatial attention unit and a channel attention unit connected in series; wherein the output end of the channel attention unit of each submodule is also connected to the input end of the scale attention unit of the current submodule; The boundary feature extraction module is configured to: extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features, specifically including: using the boundary feature extraction module to extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features; using the boundary feature extraction module to extract boundary features from preliminary classification features to obtain boundary classification features and boundary regression features; the specific working principle includes: Find the four boundary points and center point of the target, and determine the position of the target bounding box based on the four boundary points and center point; According to the position of the target bounding box, each edge is evenly divided into six points, and the feature value of each point is subjected to the maximum pooling operation to obtain the output boundary feature map; Then perform convolution operation on the output boundary feature map to obtain boundary classification features and boundary regression features; The preliminary classification features are combined with the boundary classification features to obtain the final classification features; the preliminary regression features are combined with the boundary regression features to obtain the final regression features; The prediction module is configured to: obtain a final classification prediction result based on the final classification feature and the trained classifier; and obtain a final regression prediction result based on the final regression feature and the trained regressor.

6. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; as well as a processor for executing the computer readable instructions, When the computer-readable instructions are executed by the processor, the method described in any one of claims 1 to 4 is executed.

7. A storage medium, characterized in that it non-temporarily stores computer-readable instructions, wherein: When the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1 to 4 are executed.