Road scene object detection method based on optimized network structure and mixed confidence
By optimizing network structure and mixed confidence, the feature extraction and feature alignment modules are improved, and the problem of difficulty in selecting fixed position reliability and low multi-scale detection accuracy in road scene object detection is solved, achieving higher detection accuracy and quality Characterization.
Patent Information
- Application Number
- CN202210197708.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-03-01
AI Technical Summary
In the prior art, it is difficult to select the optimal position reliability of road scene object detection, resulting in low detection accuracy, especially in the multi-scale object detection, with unreasonable quality characterization.
Using a method based on optimization of network structure and mixed confidence, feature extraction is performed through the Dilated-ResNeSt network, feature fusion is performed using feature pyramids, and dynamic fusion of attention mechanism is achieved through feature alignment enhancement strategies, and different functions are selected according to the sensitivity of the target scale. The mechanism is mixed confidence to improve detection quality characterization in the NMS stage.
It improves feature extraction ability and detection accuracy, solves the problem of mismatch in attention mechanism migration, enhances the effect of feature alignment module, and improves the accuracy of multi-scale object detection in road scenes.
Smart Images

Figure CN114663847B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine vision analysis, and in particular to a road scene target detection method based on optimized network structure and mixed confidence. Background Art
[0002] As the research foundation of autonomous driving, road scene object detection has always been a hot and difficult problem in the field of computer vision. Road scene object detection uses computer vision to determine whether there are vehicles, pedestrians and other targets on the road and to identify the type of target and its specific location. The accuracy and real-time performance of the algorithm will directly affect the safety performance of autonomous vehicles. In recent years, with the breakthrough progress of computer hardware technology and deep learning algorithms in computer vision, deep learning-based
[0003] Road scene target detection algorithms have become mainstream and can be divided into three types of algorithms: dual-stage, single-stage, and 1.5-stage. The dual-stage and its improved algorithms have complex structures and long inference times. They focus on improving accuracy but sacrifice real-time performance. Single-stage target detection algorithms represented by the YOLO series and SSD series significantly shorten the detection time, but the accuracy of the algorithm is low and it is difficult to directly apply to complex road scenes. The 1.5-stage target detection algorithm RepPoints combines the advantages of single-stage and dual-stage algorithms, but RepPoints also has the problem of unreasonable detection quality representation in the NMS (non-maximum suppression) stage. The positioning confidence directly determines the direction of the results in the NMS stage, which in turn affects the accuracy of road scene target detection. Therefore, the key to the problem lies in how to select the optimal positioning confidence for road scene target detection, and the detection accuracy of multi-scale targets in road scenes in related technologies is not high. Summary of the invention
[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and propose a road scene target detection method based on optimized network structure and hybrid confidence, so as to solve the technical problems in the prior art that it is difficult to select the optimal positioning confidence for road scene target detection and the detection accuracy of multi-scale targets in road scenes is also low.
[0005] To achieve the above technical objectives, in a first aspect, the technical solution of the present invention provides a road scene target detection method based on optimized network structure and hybrid confidence, comprising the following steps:
[0006] Acquire a road scene image;
[0007] Using a Dilated-ResNeSt network to extract features from the road scene image to obtain a feature map, and using a feature aggregation operation to replace a fully connected layer in ResNeSt;
[0008] Using a feature pyramid to perform feature fusion on the feature map;
[0009] Processing the feature map using a feature alignment enhancement strategy to achieve dynamic fusion of the attention mechanism;
[0010] The targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and enhance the detection accuracy of multi-scale targets in the road scene.
[0011] Compared with the prior art, the road scene target detection method based on optimized network structure and mixed confidence provided by the present invention has the following beneficial effects:
[0012] The road scene target detection method based on optimized network structure and hybrid confidence improves the feature extraction module and feature alignment module respectively to optimize the overall network structure. The proposed feature extraction improvement strategy solves the structural incompatibility problem in the feature extraction network ResNeSt. The improved algorithm has stronger feature extraction ability and can provide better features for the network, thereby improving the detection accuracy. The proposed feature alignment enhancement strategy solves the migration incompatibility problem of the attention mechanism, can further calibrate the feature information corresponding to the target, and enhance the effect of the feature alignment module. The proposed hybrid confidence improvement strategy for road scene target detection solves the problem of unreasonable detection quality representation of the prediction box in the NMS stage and improves the detection accuracy of the algorithm.
[0013] According to some embodiments of the present invention, before using the feature pyramid to perform feature fusion on the feature map, the method further includes stacking convolutional layers to increase the number of input channels of the feature aggregation operation.
[0014] According to some embodiments of the present invention, the step of using a feature pyramid to perform feature fusion on the feature map comprises the following steps:
[0015] The feature pyramid includes a bottom-up part, a top-down part and a horizontal connection part, wherein the bottom-up part is used to reduce the original image;
[0016] The top-down part is used to enlarge the low-resolution feature map;
[0017] The lateral connection part is used to fuse the feature maps with the same resolution.
[0018] According to some embodiments of the present invention, the feature alignment enhancement strategy includes a parallel attention part, a dynamic weight calculation part, and an attention weighting part.
[0019] According to some embodiments of the present invention, the process of processing the feature map using a feature alignment enhancement strategy to achieve dynamic fusion of an attention mechanism comprises the steps of:
[0020] The parallel attention part includes a channel attention module and a spatial attention module, and uses the channel attention module and the spatial attention module in parallel to input the feature map to obtain a channel attention output and a spatial attention output;
[0021] Dynamic weight calculation and dynamic attention weighting processing are performed on the channel attention output and the spatial attention output to obtain dynamic weights and output features.
[0022] According to some embodiments of the present invention, the dynamically weighting the channel attention output and the spatial attention output comprises the steps of:
[0023] Element-by-element summation is performed to fuse the channel attention output and the spatial attention output to obtain a first output;
[0024] Processing the first output using an average pooling layer and a maximum pooling layer to obtain corresponding channel information and summing the information to obtain sum information;
[0025] Processing the summed information through a fully connected layer to obtain weight information;
[0026] Dynamic weight calculation is performed on the weight information to obtain the dynamic weight.
[0027] According to some embodiments of the present invention, the road scene target detection method based on optimized network structure and mixed confidence also includes the steps of:
[0028] A localization confidence subnetwork is added to the output part of the mixed confidence network, and the feature map is processed by 1×1 convolution to output a mixed confidence prediction value.
[0029] According to some embodiments of the present invention, the road scene target detection method based on optimized network structure and mixed confidence also includes:
[0030] Post-processing stage: The hybrid confidence prediction value is combined with the classification confidence to characterize the detection quality and used as the ranking score in the NMS stage.
[0031] In the second aspect, the technical solution of the present invention provides a road scene target detection system based on optimized network structure and hybrid confidence, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the road scene target detection method based on optimized network structure and hybrid confidence as described in any one of the first aspects is implemented.
[0032] In a third aspect, the technical solution of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a road scene target detection method based on optimized network structure and mixed confidence as described in any one of the first aspects.
[0033] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, wherein the abstract drawings are identical to one of the drawings in the specification:
[0035] Figure 1 A flowchart of a road scene target detection method based on optimized network structure and mixed confidence provided by an embodiment of the present invention;
[0036] Figure 2 A flowchart of a road scene target detection method based on optimized network structure and mixed confidence provided by another embodiment of the present invention;
[0037] Figure 3 A flowchart of a road scene target detection method based on optimized network structure and mixed confidence is provided in another embodiment of the present invention. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0039] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0040] The present invention provides a road scene target detection method based on optimized network structure and hybrid confidence, and improves the feature extraction module and the feature alignment module respectively to optimize the overall network structure. The proposed feature extraction improvement strategy solves the structural incompatibility problem in the feature extraction network ResNeSt. The improved algorithm has a stronger feature extraction capability and can provide the network with better features, thereby improving the detection accuracy. The proposed feature alignment enhancement strategy solves the migration incompatibility problem of the attention mechanism, can further calibrate the feature information corresponding to the target, and enhance the effect of the feature alignment module. A hybrid confidence improvement strategy for road scene target detection is proposed, which solves the problem of unreasonable detection quality representation of the prediction box in the NMS stage and improves the detection accuracy of the algorithm.
[0041] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.
[0042] Reference Figure 1 , Figure 1 A flowchart of a road scene target detection method based on optimized network structure and mixed confidence is provided for one embodiment of the present invention. The road scene target detection method based on optimized network structure and mixed confidence includes but is not limited to steps S110 to S150.
[0043] Step S110, acquiring a road scene image;
[0044] Step S120, using the Dilated-ResNeSt network to extract features from the road scene image to obtain a feature map, and using a feature aggregation operation to replace the fully connected layer in ResNeSt;
[0045] Step S130, using a feature pyramid to perform feature fusion on the feature map;
[0046] Step S140, using a feature alignment enhancement strategy to process the feature map to achieve dynamic fusion of the attention mechanism;
[0047] Step S150, classify and regress the targets in the road scene image, and select mixed confidences of different action mechanisms according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and enhance the detection accuracy of multi-scale targets in the road scene.
[0048] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of an attention mechanism; finally, objects in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and enhance the detection accuracy of multi-scale objects in road scenes.
[0049] The road scene target detection method based on optimized network structure and hybrid confidence provided in this embodiment improves the feature extraction module and feature alignment module respectively to optimize the overall network structure. The proposed feature extraction improvement strategy solves the structural incompatibility problem in the feature extraction network ResNeSt. The improved algorithm has a stronger feature extraction capability and can provide the network with better features, thereby improving the detection accuracy. The proposed feature alignment enhancement strategy solves the migration incompatibility problem of the attention mechanism, can further calibrate the feature information corresponding to the target, and enhance the effect of the feature alignment module. A hybrid confidence improvement strategy for road scene target detection is proposed to solve the problem of unreasonable detection quality representation of the prediction box in the NMS stage, and improve the detection accuracy of the algorithm.
[0050] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, a feature aggregation operation is used to replace the fully connected layer in ResNeSt, and convolutional layers are stacked to increase the number of input channels of the feature aggregation operation. Then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the objects in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different object scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and improve the detection accuracy of multi-scale objects in the road scene.
[0051] The feature extraction improvement strategy Dilated-ResNeSt further increases the number of input channels aggregated by the convolution operation, allowing the convolution to take into account more channel information. On the one hand, stacking convolutional layers can effectively increase the number of aggregated input channels with only a very small increase in the number of parameters; on the other hand, using dilated convolutions instead of ordinary convolutions can further increase the information aggregated by convolutions. The feature extraction improvement strategy Dilated-ResNeSt uses feature aggregation operations to replace the fully connected layers in ResNeSt, further simplifying the network structure, while improving the incompatibility issues in ResNeSt, and improving the rationality of the channel weights of the Split-Attention module, thereby providing the network with better features and improving the accuracy of road scene target detection.
[0052] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the NMS stage detection quality characterization method and improve the detection accuracy of multi-scale targets in the road scene. The feature pyramid is used to fuse the feature map, including the following steps: the feature pyramid includes a bottom-up part, a top-down part, and a lateral connection part, the bottom-up part is used to reduce the original image; the top-down part is used to enlarge the low-resolution feature map; the lateral connection part is used to fuse the feature maps of the same resolution.
[0053] In one embodiment, a road scene image is first obtained; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the NMS stage detection quality characterization method and improve the detection accuracy of multi-scale targets in road scenes. The feature alignment enhancement strategy includes a parallel attention part, a dynamic weight calculation part, and an attention empowerment part.
[0054] Reference Figure 2 , Figure 2A flowchart of a road scene target detection method based on optimized network structure and mixed confidence is provided for another embodiment of the present invention. The road scene target detection method based on optimized network structure and mixed confidence includes but is not limited to steps S210 to S220.
[0055] Step S210, the parallel attention part includes a channel attention module and a spatial attention module, and the channel attention module and the spatial attention module are used in parallel on the input feature map to obtain a channel attention output and a spatial attention output;
[0056] Step S220, perform dynamic weight calculation and dynamic attention weighting processing on the channel attention output and the spatial attention output to obtain dynamic weights and output features.
[0057] In one embodiment, a road scene image is first obtained; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the NMS stage detection quality characterization method and improve the detection accuracy of multi-scale targets in road scenes. The feature alignment enhancement strategy includes a parallel attention part, a dynamic weight calculation part, and an attention empowerment part.
[0058] The feature map is processed using a feature alignment enhancement strategy to achieve dynamic fusion of the attention mechanism, including the following steps: the parallel attention part includes a channel attention module and a spatial attention module, the channel attention module and the spatial attention module are used in parallel on the input feature map to obtain the channel attention output and the spatial attention output; the channel attention output and the spatial attention output are dynamically weighted and dynamically weighted to obtain dynamic weights and output features.
[0059] Reference Figure 3 , Figure 3 A flowchart of a road scene target detection method based on optimized network structure and mixed confidence is provided for another embodiment of the present invention. The road scene target detection method based on optimized network structure and mixed confidence includes but is not limited to steps S310 to S340.
[0060] Step S310, summing and fusing the channel attention output and the spatial attention output element by element to obtain a first output;
[0061] Step S320, using an average pooling layer and a maximum pooling layer to process the first output to obtain corresponding channel information and summing them to obtain sum information;
[0062] Step S330, the sum information is processed by a fully connected layer to obtain weight information;
[0063] Step S340, performing dynamic weight calculation on the weight information to obtain a dynamic weight.
[0064] In one embodiment, a road scene image is first obtained; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the NMS stage detection quality characterization method and improve the detection accuracy of multi-scale targets in road scenes. The feature alignment enhancement strategy includes a parallel attention part, a dynamic weight calculation part, and an attention empowerment part.
[0065] The feature map is processed using a feature alignment enhancement strategy to achieve dynamic fusion of the attention mechanism, including the following steps: the parallel attention part includes a channel attention module and a spatial attention module, the channel attention module and the spatial attention module are used in parallel on the input feature map to obtain the channel attention output and the spatial attention output; the channel attention output and the spatial attention output are dynamically weighted and dynamically weighted to obtain dynamic weights and output features.
[0066] Dynamic weight calculation is performed on the channel attention output and the spatial attention output, including the steps of: element-by-element summing and fusing the channel attention output and the spatial attention output to obtain a first output; using an average pooling layer and a maximum pooling layer to process the first output to obtain corresponding channel information and summing up to obtain sum information; processing the sum information through a fully connected layer to obtain weight information; and performing dynamic weight calculation on the weight information to obtain a dynamic weight.
[0067] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences with different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and improve the detection accuracy of multi-scale targets in the road scene. A fixed position confidence subnetwork is added to the output part of the mixed confidence network, and the feature map is processed by 1×1 convolution to output the mixed confidence prediction value.
[0068] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of the attention mechanism; finally, the targets in the road scene image are classified and regressed, and mixed confidences with different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and improve the detection accuracy of multi-scale targets in the road scene. Post-processing stage: The mixed confidence prediction value is combined with the classification confidence to characterize the detection quality and used as the ranking score in the NMS stage.
[0069] In one embodiment, a road scene image is first acquired; then, a Dilated-ResNeSt network is used to extract features from the road scene image to obtain a feature map, and a feature aggregation operation is used to replace the fully connected layer in ResNeSt; then, a feature pyramid is used to fuse the feature map; secondly, a feature alignment enhancement strategy is used to process the feature map to achieve dynamic fusion of an attention mechanism; finally, objects in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and enhance the detection accuracy of multi-scale objects in road scenes.
[0070] Step S1: Feature extraction
[0071] The Dilated-ResNeSt network is used to improve feature extraction. The feature extraction improvement strategy Dilated-ResNeSt uses feature aggregation operations instead of the fully connected layer in ResNeSt, which can further simplify the network structure. While improving the incompatibility problem in ResNeSt, it improves the rationality of the channel weights of the Split-Attention module, thereby providing the network with better features and improving the accuracy of road scene target detection.
[0072] Step S2: Feature fusion
[0073] Feature pyramid: FPN is mainly composed of three parts: bottom-up, top-down, and horizontal connections. The ResNet network as the bottom-up part appropriately reduces the original image, and the top-down appropriately enlarges the minimum resolution feature map, and then uses horizontal connections to fuse the feature maps of the same resolution. Use a 1*1 convolution kernel to reduce the features of each layer of the bottom-up part to 256 dimensions, and then fuse the reduced feature maps with the feature maps of the same resolution in the top-down, and the fusion adopts the element-by-element addition method. Finally, a 3*3 convolution is used to further eliminate the aliasing effect after fusion.
[0074] Step S3: Feature alignment
[0075] The feature alignment enhancement strategy Selective Attention Module (SAM) is used to further calibrate the features corresponding to the target: SAM consists of three parts: parallel attention, dynamic weight calculation, and attention empowerment. The dynamic weight is calculated using the image information summarized by the parallel attention module, and then the two types of attention are weighted to achieve dynamic fusion of the attention mechanism.
[0076] Step S31: Parallel attention: Use the channel attention module and the spatial attention module in parallel on the input feature map to obtain the corresponding weights c and s:
[0077] c=σ(MLP(AP(X))) (1)
[0078] s=σ(conv 7×7 (concat(AP(X),MP(X)))) (2)
[0079] In the formula, AP is the average pooling layer, MP is the maximum pooling layer, MLP is the multi-layer fully connected layer, and is the sigmoid activation function. The output X of channel attention and spatial attention is calculated using the attention weights c and s C and X S :
[0080] X C =X×c (3)
[0081] X S =X×s (4)
[0082] Step S32: Dynamic weight calculation: The information of XC and XS is obtained by element-by-element summation, and then the corresponding channel information is obtained by using MP and AP and summed. The summed information is passed through the fully connected layer to obtain the weight w:
[0083] w=FC1[MP(X′)+AP(X′)] (5)
[0084] Where FC1 is a fully connected layer. The dynamic weights m and n are further calculated.
[0085]
[0086] In the formula, FC2 is a fully connected layer. The obtained matrix n is a redundant matrix, that is, n = 1-m. The dynamic weight is dynamically calculated based on the channel attention information and spatial attention information of different inputs, which can capture the complex and rich information around the target in the road scene.
[0087] Step S33: Dynamic attention weighting:
[0088] Output X of parallel attention C and X S After weighting, the final output feature Y is obtained, namely:
[0089] Y=m·X C +n·X S (7) Step S4: Network output
[0090] For large, medium and small targets in road scenes, the targets are first classified and regressed into large, medium and small. The hybrid confidence improvement strategy combines the advantages of IoU and centerness, and selects hybrid confidence with different action mechanisms according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage, further improving the detection accuracy of multi-scale targets in road scenes.
[0091] The present invention also provides a road scene target detection system based on optimized network structure and hybrid confidence, comprising: a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the road scene target detection method based on optimized network structure and hybrid confidence as described above is implemented.
[0092] The processor and the memory may be connected via a bus or other means.
[0093] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0094] It should be noted that the road scene target detection system based on optimized network structure and hybrid confidence in this embodiment may include a business processing module, an edge database, a server version information register, and a data synchronization module. When the processor executes the computer program, it implements the road scene target detection method based on optimized network structure and hybrid confidence as applied to the road scene target detection system based on optimized network structure and hybrid confidence as described above.
[0095] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0096] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor or controller, for example, by a processor in the above-mentioned terminal embodiment, so that the above-mentioned processor can execute the road scene target detection method based on optimized network structure and mixed confidence in the above-mentioned embodiment.
[0097] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0098] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.
[0099] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A road scene target detection method based on optimized network structure and mixed confidence, characterized in that: The following steps are involved: Acquire a road scene image; Using a Dilated-ResNeSt network to extract features from the road scene image to obtain a feature map, and using a feature aggregation operation to replace a fully connected layer in ResNeSt; Using a feature pyramid to perform feature fusion on the feature map; Processing the feature map using a feature alignment enhancement strategy to achieve dynamic fusion of the attention mechanism; The targets in the road scene image are classified and regressed, and mixed confidences of different action mechanisms are selected according to different target scale sensitivities to improve the rationality of the detection quality characterization method in the NMS stage and enhance the detection accuracy of multi-scale targets in the road scene.
2. The road scene target detection method based on optimized network structure and mixed confidence according to claim 1, characterized in that: Before using the feature pyramid to perform feature fusion on the feature map, the method further includes the following steps: Convolutional layers are stacked to increase the number of input channels for the feature aggregation operation.
3. The road scene target detection method based on optimized network structure and mixed confidence according to claim 1, characterized in that: The method of using a feature pyramid to perform feature fusion on the feature map comprises the following steps: The feature pyramid includes a bottom-up part, a top-down part and a horizontal connection part, and the bottom-up part is used to reduce the original image; The top-down part is used to enlarge the low-resolution feature map; The lateral connection part is used to fuse the feature maps with the same resolution.
4. The road scene target detection method based on optimized network structure and mixed confidence according to claim 1, characterized in that: The feature alignment enhancement strategy includes a parallel attention part, a dynamic weight calculation part, and an attention empowerment part.
5. The road scene target detection method based on optimized network structure and mixed confidence according to claim 4, characterized in that: The feature map is processed using a feature alignment enhancement strategy to achieve dynamic fusion of an attention mechanism, comprising the steps of: The parallel attention part includes a channel attention module and a spatial attention module, and uses the channel attention module and the spatial attention module in parallel to input the feature map to obtain a channel attention output and a spatial attention output; Dynamic weight calculation and dynamic attention weighting processing are performed on the channel attention output and the spatial attention output to obtain dynamic weights and output features.
6. The road scene target detection method based on optimized network structure and mixed confidence according to claim 5, characterized in that: The dynamically weighting the channel attention output and the spatial attention output comprises the steps of: Element-by-element summation is performed to fuse the channel attention output and the spatial attention output to obtain a first output; Processing the first output using an average pooling layer and a maximum pooling layer to obtain corresponding channel information and summing the information to obtain sum information; Processing the summed information through a fully connected layer to obtain weight information; Dynamic weight calculation is performed on the weight information to obtain the dynamic weight.
7. The road scene target detection method based on optimized network structure and mixed confidence according to claim 1, characterized in that: Also includes the steps: A localization confidence subnetwork is added to the output part of the mixed confidence network, and the feature map is processed by 1×1 convolution to output a mixed confidence prediction value.
8. The road scene target detection method based on optimized network structure and mixed confidence according to claim 1, characterized in that: Also includes: Post-processing stage: The hybrid confidence prediction value is combined with the classification confidence to characterize the detection quality and used as the ranking score in the NMS stage.
9. A road scene target detection system based on optimized network structure and mixed confidence, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the road scene target detection method based on optimized network structure and mixed confidence as described in any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the road scene target detection method based on optimized network structure and mixed confidence as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Method and system for detecting artificial small target in SAR (Synthetic Aperture Radar) image
CN113567984A
Target detection algorithm for vehicles and pedestrians in road scene
CN114004966A
Detection method using fusion network based on attention mechanism, and terminal device
US11222217B1
Parallel scene primitive detection using a surround camera system
US20170300767A1
