Traffic sign detection method applied to haze weather
By using the joint training of two-way branch models and defog network in traffic sign detection, the problem of reduced traffic sign detection accuracy in smog weather is solved, and more efficient traffic sign recognition and detection is achieved.
Patent Information
- Application Number
- CN202411067280.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-24
AI Technical Summary
In smog weather, the performance of traffic sign detection methods declines, resulting in blurred image and insufficient lighting, making it difficult to identify detailed features, which in turn leads to missed and missed detection and reduced monitoring accuracy.
Two-way branch models are used to extract features of the detection image and the image after defogging treatment, and detect it by combining the fusion feature images, combined with the joint training of the defogging network and the detection model, and calculate the haze robust loss to optimize the model.
It improves the accuracy and real-time nature of traffic sign detection, avoids the impact of haze on detection, and enhances the detection performance of small targets.
Smart Images
Figure CN119181074A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of traffic sign detection, and in particular to a traffic sign detection method applied to haze weather. Background Art
[0002] Traffic sign detection is crucial to autonomous driving. Traffic signs contain a lot of key information such as road conditions, driving environment, speed limit warnings, etc., which provide rich navigation information for intelligent vehicles, which is conducive to ensuring their safety and stability during driving. It is of great significance to the realization of intelligent transportation. However, related technologies will face varying degrees of performance degradation when detecting traffic signs in haze weather, including but not limited to: the quality of the collected images is reduced due to blurring and insufficient lighting caused by haze, making it difficult to identify the detailed features in the image, which in turn causes problems of missed detection and false detection, significantly reducing the performance of the detection method; since the distance between the vehicle and the traffic sign is uncertain when the image is taken, the traffic sign occupies a smaller area when the distance is far, and is more easily obscured by haze, resulting in reduced monitoring accuracy.
[0003] At present, in order to reduce the interference of haze weather on target detection tasks, in addition to the traditional improvements on the feature extraction or feature fusion part of the target detection algorithm, the target detection algorithm and the image restoration algorithm can also be combined. For example, the detection model is directly trained on the blurred image without restoration labels. This method relies on the feature extraction ability of the detection model. In addition, the restoration model can be trained with clear image labels, and then the enhanced restored image is input into the pre-trained detection model. However, there is a potential conflict between the current restoration model and the detection model. The image generated by the restoration model may contain noise that is invisible to the human eye, which will interfere with the subsequent detection model.
[0004] Therefore, there is currently a lack of a method that can accurately and efficiently identify and detect traffic signs in haze weather. Summary of the invention
[0005] The embodiments of the present application provide a method, device, equipment and storage medium for detecting traffic signs in haze weather, so as to solve the defects of the above-mentioned related technologies. The technical solution is as follows:
[0006] In a first aspect, an embodiment of the present application provides a traffic sign detection method applied to haze weather, comprising:
[0007] Acquire a first image to be processed, and input the first image into a first branch and a second branch of a trained traffic sign detection model respectively;
[0008] Obtaining a first feature corresponding to the first image through the extraction module of the first branch;
[0009] The defogging module of the second branch performs defogging processing on the first image to obtain a second image, and the extraction module of the second branch obtains a second feature corresponding to the second image;
[0010] Fusing the first feature and the second feature to obtain a fused feature image;
[0011] The detection module based on the trained traffic sign detection model detects and classifies the fused feature image and outputs the detection result.
[0012] In an optional solution of the first aspect, the defogging module of the second branch performs defogging on the first image to obtain the second image, including:
[0013] The first image is input into the defogging network model, the residual between the first image and the clear image after defogging is predicted by the defogging network model, and the residual is added to the first image to obtain the second image after defogging.
[0014] In another optional solution of the first aspect, the fusing the first feature and the second feature to obtain a fused feature image includes:
[0015] Processing the first feature and the second feature extracted based on the same first image through a concat operation to obtain three fused feature images;
[0016] Among them, the sizes of the three fused feature images fall into the first size range, the second size range and the third size range respectively, the lower limit of the first size range is greater than or equal to the upper limit of the second size range, and the lower limit of the second size range is greater than or equal to the upper limit of the third size range.
[0017] In another optional solution of the first aspect, after obtaining the first feature corresponding to the first image through the extraction module of the first branch and obtaining the second feature corresponding to the second image through the extraction module of the second branch, the method further includes:
[0018] Performing global average pooling based on the first feature and the second feature, calculating a difference between a distribution of the first feature and a distribution of the second feature based on a pooling result of the first feature and a pooling result of the second feature, and calculating a haze robust loss;
[0019] Combining the haze robust loss and the detection loss of the detection module to obtain a total loss;
[0020] The traffic sign detection method further includes:
[0021] The trained traffic sign detection model is obtained based on the total loss training.
[0022] In another optional solution of the first aspect, after the trained traffic sign detection model is obtained based on the total loss training, the method further includes:
[0023] When it is determined based on the total loss that the trained traffic sign detection model has converged, all model parameters and corresponding hardware parameters of the trained traffic sign detection model are saved, and a model configuration file is generated based on all model parameters, the corresponding hardware parameters and the total loss;
[0024] The model configuration file is used to configure a new traffic sign detection model.
[0025] In a second aspect, the embodiment of the present application further provides a traffic sign detection device for use in haze weather, comprising:
[0026] An image input unit, used for acquiring a first image to be processed, and inputting the first image into a first branch and a second branch of a trained traffic sign detection model respectively;
[0027] A first extraction unit, configured to obtain a first feature corresponding to the first image through the extraction module of the first branch;
[0028] A second extraction unit, configured to obtain a second image by performing a defogging process on the first image through the defogging module of the second branch, and obtain a second feature corresponding to the second image through the extraction module of the second branch;
[0029] a fusion unit, configured to fuse the first feature and the second feature to obtain a fused feature image;
[0030] The monitoring unit is used to detect and classify the fused feature image based on the detection module of the trained traffic sign detection model, and output the detection result.
[0031] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method provided in the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect is implemented.
[0032] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.
[0033] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:
[0034] The embodiment of the present application provides a traffic sign detection method for use in haze weather. In terms of application, by setting two branches in the model, feature extraction is performed based on a first image to be detected and a clear second image after defogging of the first image, respectively. This can better identify the difference between the first image and the second image, thereby avoiding the influence of haze on target recognition in the image, and can improve the detection performance of small targets such as traffic signs while meeting the real-time and lightweight requirements of traffic sign detection tasks.
[0035] In terms of model training, by setting two branches in the model, feature extraction is performed based on the first image to be detected and the clear second image after defogging the first image, so that the model can better learn the difference between the first image and the second image, that is, the entire model is optimized, and the defogging and detection models are jointly trained on blurry images with or without restoration labels, thereby avoiding the defect of potential conflict in fitting image enhancement tasks and detection tasks in the prior art, thereby improving the problem of decreased traffic sign detection accuracy in haze conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 It is a flow chart of a traffic sign detection method applied to haze weather according to an embodiment of the present application;
[0038] Figure 2 It is a flow chart of a traffic sign detection method applied to haze weather provided by an embodiment of the present application;
[0039] Figure 3 It is a structural schematic diagram of a traffic sign detection device applied to haze weather provided by an embodiment of the present application;
[0040] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0042] The terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.
[0043] It should be noted that the terms "first\second" involved in the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.
[0044] It should be noted that one or more embodiments of the present application can be applied to the field of traffic sign detection. The corresponding methods can be based on carriers such as vehicles, mobile terminals, monitoring equipment, and traffic signal equipment. They can be used to improve the accuracy and sensitivity of related equipment in detecting traffic signs in haze weather, and avoid the impact of haze weather on the recognition of traffic signs by related equipment.
[0045] It should be noted that traffic signs are unique due to their inherent colors (red, yellow, blue) and specific shapes (such as triangles, rectangles, circles). These distinctive features make them easy to identify from the surrounding environment. Based on this, related technologies often use color features, shape features, and fusion features to detect and identify traffic signs. However, the detection method based on color features is easily affected by weather conditions and changes in lighting, and the detection method based on shape is easily affected by occlusion caused by snow, fog, etc. The detection method based on fusion features has superior detection accuracy compared to single features, but it increases complexity, resulting in slow detection speed and poor real-time performance.
[0046] Therefore, the traffic sign recognition method in the related art is not robust enough, the detection performance is poor, the detection type is monotonous, and the detection speed is slow, which needs further improvement.
[0047] The present application is described in detail below with reference to specific embodiments.
[0048] Next, combine Figure 1 Taking the traffic sign detection method applied to haze weather executed by the terminal as an example, the traffic sign detection method applied to haze weather provided by the embodiment of the present application is introduced. Figure 1 A flow chart of a traffic sign detection method for haze weather provided in an embodiment of the present application is shown, and the method comprises the following steps:
[0049] S101, obtaining a first image to be processed, and inputting the first image into a first branch and a second branch of a trained traffic sign detection model respectively.
[0050] S102: Obtain a first feature corresponding to the first image through the extraction module of the first branch.
[0051] S103, performing defogging processing on the first image through the defogging module of the second branch to obtain a second image, and obtaining a second feature corresponding to the second image through the extraction module of the second branch.
[0052] S104: Fusing the first feature and the second feature to obtain a fused feature image.
[0053] S105, performing detection and classification on the fused feature image based on the detection module of the trained traffic sign detection model, and outputting a detection result.
[0054] It should be noted that the main body of the trained traffic sign detection model in the embodiment of the present application is the YOLOv7 network.
[0055] Specifically, the first image input in S101 may be an image captured by any shooting device. The shooting device may be a terminal held by a user such as a mobile phone, or may be a sensor unit of a vehicle such as a camera, which is not limited in this embodiment of the present application.
[0056] Specifically, the trained traffic sign detection model can perform recognition based on the input first image, eliminate the influence of haze, determine whether there is a traffic sign therein, and mark the position of the traffic sign in the form of a box in the output image.
[0057] Specifically, the first image is the original image obtained by shooting, and the first image is input into two branches respectively, such as Figure 2 As shown, specifically including:
[0058] In S102, the first image is received by the extraction module of the first branch, so as to extract the first feature corresponding to the first image, that is, the original feature of the image.
[0059] Among them, the extraction module is the feature extraction module in the original YOLOv7 network.
[0060] In S103, the first image is first received through the defogging module of the second branch, and the defogging module predicts the residual between the first image and the clear image after defogging, and the residual is added to the first image to obtain the second image after defogging, that is, the clear image after removing the haze, and then the clear second image is input into the same extraction module as the first branch to extract the corresponding second feature, that is, the defogging feature of the image.
[0061] Specifically, the dehazing module can use the GCANet model, which can be composed of an encoder, a decoder, and a feature fusion module including a smooth dilated module and a gated fusion subnetwork. Through the collaborative work of the internal convolution blocks, the image originally containing haze features can be converted into a feature image after eliminating the haze features.
[0062] In the training process of the above defogging module, the difference between the defogging image after defogging and the original image not affected by haze can be adaptively learned, thereby improving the accuracy of the defogging image obtained after defogging the haze image.
[0063] Based on this, through the above defogging module, the residual between the defogging second image and the original hazy first image can be accurately predicted, and then the defogging second image can be output.
[0064] Specifically, Figure 2 As shown, after the original first feature is obtained through the first branch and the second feature after defogging is obtained through the second branch, S104 includes:
[0065] The first feature and the second feature are fused to obtain a fused feature image.
[0066] Specifically, a concat operation may be performed through the original YOLOv7 network to obtain a preset number of images based on the fusion of the first feature and the second feature, for example, three fused feature images.
[0067] Specifically, the sizes of the three fused feature images fall into the first size range, the second size range and the third size range respectively, the lower limit of the first size range is greater than or equal to the upper limit of the second size range, and the lower limit of the second size range is greater than or equal to the upper limit of the third size range.
[0068] Therefore, the three fused feature images cover the feature information of large, medium and small scales respectively, which is conducive to detecting targets of different scales, where the scale can be understood as the size and resolution of the image.
[0069] In some embodiments, after S104, more effective multi-scale and small target high-dimensional features can be obtained based on the multi-scale spatial feature pyramid pooling module in the original YOLOv7 network, and combined with the BiFormer attention mechanism, specifically including:
[0070] The SPP module, which combines SPP (spatial pyramid pooling) and CSP (cross-stage partial connections) technologies, can improve the efficiency of feature extraction and feature fusion. First, the input fused feature image can be pyramid pooled based on the SPP technology, and multiple fused feature images of different scales can be spliced. In this way, the features of the traffic signs to be detected in the image at different scales can be extracted without losing spatial information, thereby enhancing the ability to detect traffic signs. Then, the CSP technology is used to perform cross-stage partial connections on the feature image after SPP pooling, and the feature image of the previous layer and the feature image of the next layer are spliced together, and the number of channels of the feature image is halved, thereby improving the efficiency and accuracy of the network. The CSP technology can effectively alleviate the gradient disappearance and information bottleneck problems caused by the increase in network depth, thereby enhancing the network's expression and generalization capabilities. Finally, the convolutional layer is used to process the fused feature image. Through this mechanism, the traffic sign detection model can expand its receptive field range and significantly enhance its feature expression ability, thereby more accurately capturing the key information in the image.
[0071] Specifically, BiFormer adopts a four-level feature pyramid structure, each level consists of multiple BiFormer Block modules. In the initial stage, BiFormer uses an overlapping patch embedding strategy. In the second to fourth stages, by introducing a patch merging module, the spatial resolution of the input is gradually reduced and the number of channels is correspondingly increased. This design aims to more effectively extract and integrate features, thereby improving the performance of the model.
[0072] Specifically, the BiFormer Block module is built based on two-level routing attention. First, the relative position information is obtained through a 3×3 depth-separable convolution, and then normalized using LN (Layer Norm). Next, the feature image to be processed is divided into several non-overlapping regions using a two-layer routing attention mechanism (BRA), and cross-region relationship modeling is performed. In some regions, key-value pairs that do not contain traffic sign-related features are removed, and only regions containing traffic sign-related features are retained to reduce redundant information.
[0073] Specifically, BRA achieves efficient and accurate processing of the input image by dividing the image region, locating the attention region, calculating the affinity between regions, screening the most relevant regions and collecting their information for self-attention calculation. First, the input feature image is divided into multiple non-overlapping small regions, and then each region is linearly mapped to obtain the corresponding query matrix (Q), key matrix (K) and value matrix (V). Based on the query matrix, key matrix and value matrix, the attention region of each region is located, and the adjacency matrix between these regions is calculated. Only the K regions with the largest attention to the current region are retained to form a routing index matrix. Finally, according to this routing index matrix, the key and value information of these K regions are collected, and self-attention calculation is performed, thereby achieving efficient processing of the image and reducing unnecessary computational overhead.
[0074] Further, in S105, detection and recognition are performed based on the detection module of the original YOLOv7 network, the type of the traffic sign in the first image is output, and the position of the traffic sign is marked in the corresponding image.
[0075] It should be noted that in the training process of the traffic sign detection model in the embodiment of the present application, it is necessary to calculate the loss function, including the haze robust loss in the image fusion process and the loss of the detection module of the original YOLOv7 network, specifically including:
[0076] Perform global average pooling based on the first feature and the second feature, calculate the difference between the distribution of the first feature and the distribution of the second feature based on the pooling result of the first feature and the pooling result of the second feature, and calculate the haze robust loss, applying the formula:
[0077]
[0078] Among them, L hr is the haze robust loss, F h is the first feature, F deh is the second feature, GAP(F h ) is the pooling result of the first feature, GAP(F deh ) is the pooling result of the second feature.
[0079] It should be noted that the loss term of image fusion, i.e., the haze robust loss, can be used to learn the restoration of image quality. In the process of image fusion restoration based on the original first feature of the image and the second feature after dehazing, the background noise information needs to be taken into account to avoid the influence of traffic sign detection caused by image fusion. Therefore, the haze robust loss should be before the detection loss. Specifically, the formula of the total loss is as follows:
[0080] Ltotal =L det +αL hr ;
[0081] Among them, L det is the detection loss of the detection module in the original YOLOv7 network, and α is used to balance L det and L hr The weight, L total is the total loss.
[0082] Specifically, during the training process, parameters such as the number of training rounds, learning rate, batch size, etc. can be set according to actual implementation requirements to train traffic sign detection.
[0083] In some embodiments, the present application can generate a model configuration file based on all model parameters, the corresponding hardware parameters and the total loss by saving all model parameters and corresponding hardware parameters of the trained traffic sign detection model.
[0084] Among them, model parameters may include the number of training rounds, learning rate, batch size, model weights, image size, etc., and hardware parameters may include GPU parameters, etc.
[0085] By saving the model configuration file, you do not need to retrain the model when using it on a new device or updating the model. You can reuse the model by directly reusing the saved model configuration file.
[0086] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
[0087] See next Figure 3 , is a schematic diagram of the structure of a traffic sign detection device for haze weather provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a terminal through software, hardware, or a combination of both, and can also be integrated on a server as an independent module. The traffic sign detection device for haze weather in the embodiment of the present application can be applied to a terminal or a cloud. The device 30 includes an image input unit 310, a first extraction unit 320, a second extraction unit 330, a fusion unit 340, and a monitoring unit 350, wherein:
[0088] The image input unit 310 is used to obtain a first image to be processed, and input the first image into a first branch and a second branch of a trained traffic sign detection model respectively;
[0089] The first extraction unit 320 is used to obtain a first feature corresponding to the first image through the extraction module of the first branch;
[0090] The second extraction unit 330 is used to obtain a second image by performing a defogging process on the first image through the defogging module of the second branch, and obtain a second feature corresponding to the second image through the extraction module of the second branch;
[0091] The fusion unit 340 is used to fuse the first feature and the second feature to obtain a fused feature image;
[0092] The monitoring unit 350 is used to detect and classify the fused feature image based on the detection module of the trained traffic sign detection model, and output the detection result.
[0093] It should be noted that the device 30 provided in the above embodiment only uses the division of the above functional modules as an example when executing the traffic sign detection method applied to haze weather. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the embodiment of the traffic sign detection method applied to haze weather belong to the same concept, and its implementation process is detailed in the method embodiment, which will not be repeated here.
[0094] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the program.
[0095] See also Figure 4 , which is a structural block diagram of an electronic device provided in an embodiment of the present application.
[0096] like Figure 4 As shown, the electronic device 400 includes: a processor 401 and a memory 402 .
[0097] In the embodiment of the present application, the processor 401 is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine. The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).
[0098] The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also called a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.
[0099] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the method in the embodiment of the present application.
[0100] In some embodiments, the electronic device 400 further includes: a peripheral device interface 403 and at least one peripheral device 404. The processor 401, the memory 402 and the peripheral device interface 403 can be connected via a bus or a signal line. Each peripheral device 404 can be connected to the peripheral device interface 403 via a bus, a signal line or a circuit board. Specifically, the peripheral device 404 includes: a display screen, a camera and an audio circuit. The peripheral device interface 403 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 401 and the memory 402.
[0101] In some embodiments of the present application, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 can be implemented on a separate chip or circuit board. This embodiment of the present application does not specifically limit this.
[0102] The electronic device structure block diagram shown in the embodiment of the present application does not constitute a limitation on the electronic device 400. The electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.
[0103] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.
[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A traffic sign detection method applied to haze weather, characterized in that: include: Acquire a first image to be processed, and input the first image into a first branch and a second branch of a trained traffic sign detection model respectively; Obtaining a first feature corresponding to the first image through the extraction module of the first branch; The defogging module of the second branch performs defogging processing on the first image to obtain a second image, and the extraction module of the second branch obtains a second feature corresponding to the second image; Fusing the first feature and the second feature to obtain a fused feature image; The detection module based on the trained traffic sign detection model detects and classifies the fused feature image and outputs the detection result.
2. The traffic sign detection method for haze weather according to claim 1, characterized in that: The defogging module of the second branch performs defogging on the first image to obtain a second image, comprising: The first image is input into the defogging network model, the residual between the first image and the clear image after defogging is predicted by the defogging module, and the residual is added to the first image to obtain the second image after defogging.
3. The method for detecting traffic signs in haze weather according to claim 1, characterized in that: The fusing the first feature and the second feature to obtain a fused feature image includes: Processing the first feature and the second feature extracted based on the same first image through a concat operation to obtain three fused feature images; Among them, the sizes of the three fused feature images fall into the first size range, the second size range and the third size range respectively, the lower limit of the first size range is greater than or equal to the upper limit of the second size range, and the lower limit of the second size range is greater than or equal to the upper limit of the third size range.
4. The method for detecting traffic signs in haze weather according to claim 1, characterized in that: After obtaining the first feature corresponding to the first image through the extraction module of the first branch and obtaining the second feature corresponding to the second image through the extraction module of the second branch, the method further includes: Performing global average pooling based on the first feature and the second feature, calculating a difference between a distribution of the first feature and a distribution of the second feature based on a pooling result of the first feature and a pooling result of the second feature, and calculating a haze robust loss; Combining the haze robust loss and the detection loss of the detection module to obtain a total loss; The traffic sign detection method further includes: The trained traffic sign detection model is obtained based on the total loss training.
5. The method for detecting traffic signs in haze weather according to claim 4, characterized in that: After the trained traffic sign detection model is obtained based on the total loss training, the method further includes: When it is determined based on the total loss that the trained traffic sign detection model has converged, all model parameters and corresponding hardware parameters of the trained traffic sign detection model are saved, and a model configuration file is generated based on all model parameters, the corresponding hardware parameters and the total loss; The model configuration file is used to configure a new traffic sign detection model.
6. A traffic sign detection device used in haze weather, characterized in that: include: An image input unit, used for acquiring a first image to be processed, and inputting the first image into a first branch and a second branch of a trained traffic sign detection model respectively; A first extraction unit, configured to obtain a first feature corresponding to the first image through the extraction module of the first branch; A second extraction unit, configured to obtain a second image by performing a defogging process on the first image through the defogging module of the second branch, and obtain a second feature corresponding to the second image through the extraction module of the second branch; a fusion unit, configured to fuse the first feature and the second feature to obtain a fused feature image; The monitoring unit is used to detect and classify the fused feature image based on the detection module of the trained traffic sign detection model, and output the detection result.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.