Traffic sign detection method and system for autonomous driving

Abstract

The present invention provides a traffic sign detection method and system for autonomous driving, comprising: obtaining a raw data stream in real-time autonomous driving, and preprocessing the raw data stream to obtain data to be detected; constructing a detection model, performing traffic sign detection on the data to be detected, and obtaining a detection result; processing the detection result and the raw data stream to generate data to be classified; constructing a classification model, extracting traffic sign categories and location information from the data to be classified, and obtaining a classification result; and combining a priori information database to post-process each target in the detection results and classification results to obtain a final detection result of the traffic sign. The present invention adopts a multi-stage processing method for small target detection, which not only solves the problem of computing resource occupation but also improves the detection effect. Assisted by data closed-loop optimization, the long-tail problem is greatly alleviated. Through the priori-based post-processing method, the safety redundancy when applied in the field of autonomous driving is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and more particularly to a traffic sign detection method and system for autonomous driving. A corresponding terminal and medium are also provided. Background Art

[0002] Autonomous driving technology has advanced rapidly in recent years, gradually transitioning from assisted autonomous driving to advanced autonomous driving. This has posed new challenges to autonomous driving perception technology. Existing traffic sign detection methods are struggling to meet the increasingly stringent detection standards. Furthermore, as a typical small-target detection problem, existing data-driven methods suffer from low recall and accuracy compared to conventional target detection. Furthermore, as a typical long-tail problem, existing technologies for detecting a small number of categories lag significantly behind those for more common categories. Even though some methods achieve good detection results, they inevitably require significant computing resources, making them unsuitable or even unusable for full-stack autonomous driving solutions. Summary of the Invention

[0003] The present invention addresses the above-mentioned deficiencies in the prior art and provides a traffic sign detection method and system for autonomous driving, as well as a corresponding terminal and medium.

[0004] According to one aspect of the present invention, a traffic sign detection method for autonomous driving is provided, comprising:

[0005] Obtaining a raw data stream in real-time autonomous driving, and preprocessing the raw data stream to obtain data to be detected;

[0006] Constructing a detection model, performing traffic sign detection on the data to be detected, and obtaining a detection result;

[0007] Processing the detection result and the original data stream to generate data to be classified;

[0008] Constructing a classification model, extracting traffic sign categories and location information from the data to be classified, and obtaining classification results;

[0009] Combined with the prior information database, post-processing is performed on each target in the detection result and the classification result to obtain the final detection result of the traffic sign.

[0010] Optionally, preprocessing the original data stream includes:

[0011] Acquire the position information in the raw data stream in real time, and locate the target area where the traffic sign is likely to appear in real time based on the position information;

[0012] Acquire image data containing traffic signs by real vehicle acquisition, and mark the traffic signs in the image data to obtain sample data;

[0013] The sample data is statistically analyzed to obtain location information of traffic signs in the sample data. The target area is scaled and / or cropped to obtain data to be detected of a specified size, which is used as input data for a detection model.

[0014] Optionally, the constructing of a detection model to perform traffic sign detection on the data to be detected includes:

[0015] Construct a sample dataset containing labeled traffic signs;

[0016] Based on a deep learning neural network, the deep learning neural network is trained using the sample data set to obtain a detection model;

[0017] The data to be detected is used as input data of the detection model, and traffic sign detection is performed on the data to be detected.

[0018] Optionally, the processing the detection result and the original data stream includes:

[0019] Using the detection results, locate multiple image locations that need to be classified and record the locations as proposals;

[0020] Through DSP acceleration or CPU algorithm acceleration, a specific sampling and splicing operation is performed on the position proposals to obtain the data to be classified as the input data of the classification model; wherein:

[0021] The specific sampling splicing method includes:

[0022] The original image regions corresponding to the n detection results obtained by the detection model are cropped respectively and scaled to a fixed pixel size of w×h in width×height;

[0023] Splicing the images of fixed size w×h into one picture along the h direction;

[0024] Add a black image of size wxh between every two pictures to obtain an image of size wx(nxhx2);

[0025] When the number of detection results n obtained in a certain frame is less than the set maximum number of targets N max When the black image of wⅹh is used to supplement the image of wⅹ(nⅹhⅹ2) until wⅹ(N max ⅹhⅹ2) is the picture to be classified, which is the input data of the classification model.

[0026] Optionally, the constructing of a classification model to extract traffic sign categories and location information from the data to be classified includes:

[0027] Build a deep learning classification model based on deep learning neural network;

[0028] Using the data to be classified as input to the classification model, extracting high-level semantic information of the data to be classified, classifying the traffic signs, and obtaining the categories of the traffic signs;

[0029] Combined with the detection results obtained by the detection model, the position of the classified traffic sign is corrected to obtain the position information of the traffic sign.

[0030] Optionally, the post-processing of each target in the detection result and the classification result in combination with the prior information includes:

[0031] The detection result and the classification result are post-processed using the prior information in the prior information database. The post-processing includes target matching, target tracking, category fusion and / or reporting logic screening to obtain the final detection result of the traffic sign.

[0032] Optionally, the target matching includes: quantifying multiple information and applying different weights to comprehensively determine the matching result; wherein the multiple information includes: category, location, high-level image features and / or low-level image features.

[0033] Optionally, the target tracking includes: tracking the traffic sign by filtering to obtain a tracking result, and compensating the tracking result with the aid of real-time posture information.

[0034] Optionally, the category fusion includes: constructing a category fusion model through a probabilistic method in combination with big data statistical information to ensure the accuracy and stability of the categories.

[0035] Optionally, the reporting logic screening includes: constructing different reporting logics according to different driving scenarios and driving speeds, wherein the driving scenarios include: conventional urban roads, elevated roads, expressways and ramps.

[0036] Optionally, the method further includes:

[0037] Obtaining a true result of the traffic sign and comparing it with the final detection result of the traffic sign to obtain conflicting data; and / or running the traffic sign detection method in an offline state, and annotating the final detection result of the traffic sign with category and location information and performing data augmentation to obtain basic data;

[0038] Optimizing the lightweight detection model and the deep learning classification model using the conflicting data and / or the basic data;

[0039] The conflicting data and / or the basic data are used as new prior information to update the prior information database.

[0040] According to a second aspect of the present invention, there is provided a traffic sign detection system for autonomous driving, comprising:

[0041] A data acquisition module, which is used to acquire the original data stream in real-time autonomous driving and pre-process the original data stream to obtain the data to be detected;

[0042] A detection module, which is used to build a detection model, perform traffic sign detection on the data to be detected, and obtain a detection result;

[0043] a conversion module, configured to process the detection result and the original raw data stream to generate data to be classified;

[0044] A classification module, which is used to build a classification model, extract traffic sign categories and location information from the data to be classified, and obtain classification results;

[0045] A post-processing module is used to combine the prior information database to post-process each target in the detection result and the classification result to obtain the final detection result of the traffic sign.

[0046] Optionally, the system further comprises:

[0047] An optimization module is provided, which is used to obtain the real results of traffic signs and compare them with the final detection results of the traffic signs to obtain conflicting data; and / or, to run the traffic sign detection method in an offline state, and to classify and annotate the final detection results of the traffic signs and perform data augmentation to obtain basic data; to optimize the lightweight detection model and the deep learning classification model using the conflicting data and / or the basic data; and to update the prior information database using the conflicting data and / or the basic data as new prior information.

[0048] According to a third aspect of the present invention, a computer terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to execute any of the above-described methods, or to execute any of the above-described systems.

[0049] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute any of the methods described above, or run any of the systems described above.

[0050] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0051] The traffic sign detection method and system for autonomous driving provided by the present invention adopt a multi-stage processing approach for small target detection of traffic signs, which not only solves the problem of computing resource occupation but also improves the detection effect.

[0052] The traffic sign detection method and system for autonomous driving provided by the present invention, supplemented by data closed-loop optimization of the entire system, greatly alleviate the long-tail problem.

[0053] The traffic sign detection method and system for autonomous driving provided by the present invention ensure further improvement in detection effect through post-processing based on prior information, and correspondingly improve safety redundancy when applied in the field of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0055] Figure 1 4 is a flowchart of a traffic sign detection method for autonomous driving according to an embodiment of the present invention.

[0056] Figure 2 Schematic diagram of the components of a traffic sign detection system for autonomous driving according to an embodiment of the present invention;

[0057] Figure 3 Schematic diagram of the working framework of a traffic sign detection method and system for autonomous driving in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention.

[0059] An embodiment of the present invention provides a traffic sign detection method for autonomous driving. This method uses data-driven technology to construct a traffic sign detection technology suitable for the field of autonomous driving by accumulating data in the optimization stage, a multi-stage processing method, and a post-processing method based on prior information, thereby greatly improving detection accuracy and stability.

[0060] like Figure 1 As shown, the traffic sign detection method for autonomous driving provided by this embodiment may include:

[0061] S1, obtains the data stream in real-time autonomous driving and preprocesses the original data stream to obtain the data to be detected.

[0062] In a preferred embodiment of S1, preprocessing the original data stream may include:

[0063] Acquire the position information in the raw data stream in real time, and locate the target area where the traffic sign is likely to appear in real time based on the position information;

[0064] Acquire image data containing traffic signs by real vehicle acquisition, and mark the traffic signs in the image data to obtain sample data;

[0065] The sample data is statistically analyzed to obtain location information of traffic signs in the sample data. The target area is scaled and / or cropped to obtain data to be detected of a specified size, which is used as input data for a detection model.

[0066] Furthermore, in a specific application example of S1, real-time pose information is used to identify a target area where traffic signs are most likely to appear. For example, when a vehicle is jolted, the location of a traffic sign can shift from the upper portion of the image to the center or even the lower portion. Adjusting the position of the target area using pose information can significantly improve the detection rate of traffic signs. Using internal and external camera parameters and / or statistical data collected from actual vehicles, information about the locations where traffic signs frequently appear in images can be obtained, eliminating the need to detect all input data. The size of this area can also be adjusted based on different algorithm requirements and hardware support. Ultimately, this minimizes input data while ensuring detection accuracy, improving resource utilization, and ensuring low latency. Real-time pose information, namely the vehicle's ego motion, can be obtained using an IMU or a specific algorithm for calculating ego motion.

[0067] S2, build a detection model, perform traffic sign detection on the data to be detected, and obtain the detection results.

[0068] In a preferred embodiment of S2, constructing a detection model to perform traffic sign detection on the data to be detected may include:

[0069] Construct a sample dataset containing labeled traffic signs;

[0070] Based on a deep learning neural network, the deep learning neural network is trained using the sample data set to obtain a detection model;

[0071] The data to be detected is used as input data of the detection model, and traffic sign detection is performed on the data to be detected.

[0072] In a specific application example of S2, a detection model is trained using deep learning methods combined with labeled data. The subsequent steps significantly reduce the performance requirements for the detection model, allowing the corresponding model structure to be compressed to ensure detection speed. Furthermore, the preceding steps significantly reduce the size of the input data. Given the same total computing power, the model depth can be increased, ensuring detection accuracy. By balancing these two aspects, an optimal model is achieved, striking a balance between timeliness and accuracy to ensure detection quality.

[0073] S3, processes the detection results and the original data stream to generate data to be classified.

[0074] In a preferred embodiment of S3, processing the detection results and the original data stream may include:

[0075] Using the detection results, locate multiple image locations that need to be classified and record the locations as proposals;

[0076] Through DSP acceleration or CPU algorithm acceleration, a specific sampling and splicing operation is performed on the position proposals to obtain the data to be classified as the input data of the classification model; wherein:

[0077] The specific sampling splicing method includes:

[0078] The original image regions corresponding to the n detection results obtained by the detection model are cropped and scaled to a fixed pixel size of w x h;

[0079] Splicing the images of fixed size w×h into one picture along the h direction;

[0080] When n is less than the set maximum target number N max When , add a black image of size w x h between each two pictures until the width x height is fixed to w x (N maxⅹhⅹ2) is the image to be classified, which is the input data of the classification model.

[0081] In a specific application example of S3, a DSP operator or a CPU is used to process the detection results and the original data stream. Specifically:

[0082] The corresponding area obtained from the original data stream is accelerated through DSP operators or CPU algorithms.

[0083] By processing the input data using special methods, such as stitching the images obtained in the previous step, the number of runs of the classification network can be reduced, thus shortening the classification time. The special organization of the input data can also ensure the accuracy of the classification network and reduce interference.

[0084] The specific splicing method is:

[0085] The original image areas corresponding to the obtained n detection results are cropped and scaled to a fixed size w x h image;

[0086] Splice the above wxh images into one picture along the h direction;

[0087] In order to prevent the two images from affecting each other, a black image of size w x h is added between each two images;

[0088] Finally, the image wⅹ(nⅹhⅹ2) is sent to the classification network;

[0089] Because the input needs to be a fixed value, the maximum number of targets is set to N. max , fixed input wⅹ(N max The image of ⅹhⅹ2) is used as input. When the number of targets in a certain frame is not enough, a black image can be used to supplement it.

[0090] S4, build a classification model, extract traffic sign categories and location information from the classified data, and obtain classification results.

[0091] In a preferred embodiment of S4, building a classification model to extract traffic sign categories and location information from the classified data may include:

[0092] S41, build a deep learning classification model based on deep learning neural network;

[0093] S42, using the data to be classified as input to the classification model, extracting high-level semantic information of the data to be classified, classifying the traffic sign, and obtaining a category of the traffic sign;

[0094] S43, combining the detection results obtained by the detection model, correcting the position of the classified traffic sign to obtain more accurate position information of the traffic sign.

[0095] S5, combined with the prior information database, post-processes each target in the detection results and classification results to obtain the final detection result of the traffic sign.

[0096] In a preferred embodiment of S5, post-processing each target in the detection results and classification results in combination with prior information may include:

[0097] The detection and classification results are post-processed using the prior information in the prior information database. The post-processing includes target matching, target tracking, category fusion and / or reporting logic screening to obtain the final detection results of traffic signs.

[0098] In a preferred embodiment of S5, target matching may include:

[0099] Target matching includes quantifying multiple pieces of information and applying different weights to comprehensively determine the matching result; wherein the multiple pieces of information include category, location, high-level image features, and / or low-level image features.

[0100] Furthermore, in a specific application example of the above embodiment, the obtained features can be used to match traffic signs. These features include traditional image features, such as category, location, optical flow features, histogram features, etc., and also include features obtained through deep learning methods, such as matching features obtained through ReId networks. The above features are quantitatively fused according to information weights to ultimately obtain a matching relationship. Specifically:

[0101] Using image histograms, ReId networks, or the location information of objects in the image, features that can be used to indicate whether two objects are the same object are extracted through various dimensions;

[0102] Quantify these features to obtain information weights;

[0103] These features are quantitatively fused to obtain the final matching features and the correspondence between the objects in the previous and next frames.

[0104] This target matching serves as the basis for subsequent target tracking.

[0105] In a preferred embodiment of S5, target tracking may include:

[0106] Traffic signs are tracked by filtering to obtain tracking results, and the tracking results are compensated with the aid of real-time posture information.

[0107] Furthermore, in a specific application example of the above embodiment, various filtering methods, such as Kalman filtering and its derivatives, are used to achieve target prediction and tracking. The filtering is compensated by the vehicle's position information. Through target tracking, the position state information (position, velocity, etc.) of an object in the current frame and its possible position state information in the next frame are obtained.

[0108] During target tracking, specific pose information refers to ego motion, which can be obtained using an IMU sensor or a specific algorithm for calculating ego motion. Using this pose information and the position of the traffic sign in the image from the previous frame, the following calculation can be used to determine the approximate location of the target object in the current frame. This can then be used to compensate for filtering during tracking, resulting in more accurate predictions. A more accurate prediction improves the accuracy of target matching in the previous step. The calculation method involves triangulating the target's position in the image and the camera's intrinsic and extrinsic parameters. Using kinematic methods combined with ego motion information, the relative position of the target and camera in the current frame can be determined. The camera's imaging principle can then be used to determine the target's position in the image for the current frame.

[0109] In a preferred embodiment of S5, category fusion may include:

[0110] Through probabilistic methods and combined with big data statistical information, a category fusion model is constructed to ensure the accuracy and stability of categories.

[0111] Furthermore, in a specific application example of the above embodiment, image data containing traffic signs is obtained by real vehicle collection, and the traffic signs in the image data are marked to obtain sample data; the sample data are statistically analyzed, and the results of the aforementioned detection model and classification model are combined with probabilistic methods, such as Bernoulli distribution and evidence distribution, to model the fusion results. For example, a DS evidence theory model is constructed using statistical methods, and the categories are fused. By setting different thresholds for different categories, the final category accuracy is guaranteed to reach 99%.

[0112] The purpose of category fusion is to keep the category of an object unchanged and make the final fixed category as accurate as possible to ensure the stability of the category and prevent category jumps.

[0113] In a preferred embodiment of S5, the reporting logic screening may include:

[0114] Different reporting logics are constructed according to different driving scenarios and driving speeds, wherein the driving scenarios may include: conventional urban roads, elevated roads, expressways and ramps.

[0115] Furthermore, in a specific application example of the above embodiment, different reporting logics are implemented through a hierarchical reporting method. For example, when a vehicle is on different road structures, the reporting logic is different. On urban roads, because the roads are narrow, reporting when the traffic sign appears near the center of the image can satisfy the requirement of providing a prompt before the traffic sign's effective range takes effect. On expressways or elevated roads, because the roads are wide, reporting when the traffic sign is near the edge of the image can satisfy the requirement of providing a prompt before the traffic sign's effective range takes effect. The same logic can also be applied to the classification of speed, etc.

[0116] In the reporting logic screening, the reporting logic refers to the feedback of detected traffic signs to the downstream (regulatory and control side or vehicle dashboard display, etc.) under the set rules; multiple rules are set according to different scenarios, vehicle speeds, and usage.

[0117] In a preferred embodiment, the traffic sign detection method may further include:

[0118] S61, obtaining a real result of the traffic sign and comparing it with the final detection result of the traffic sign to obtain conflicting data; and / or

[0119] The traffic sign detection method is run offline, and the final detection results of the traffic signs are annotated with category and location information and data augmented to obtain basic data. Furthermore, the above-mentioned data augmentation method includes but is not limited to using computer vision algorithms to achieve seamless replacement of traffic signs, which can perfectly imitate the lighting, contrast, etc. of the replaced scene, achieving an effect that is exactly the same as the data collected from the actual vehicle.

[0120] S62, optimizing the lightweight detection model and the deep learning classification model using conflicting data and / or basic data;

[0121] S63: Use the conflicting data and / or basic data as new prior information to update the prior information database.

[0122] An embodiment of the present invention further provides a traffic sign detection system for autonomous driving.

[0123] like Figure 2 As shown, the traffic sign detection system for autonomous driving provided by this embodiment may include:

[0124] Data acquisition module, which is used to obtain the original data stream in real-time autonomous driving and pre-process the original data stream to obtain the data to be detected;

[0125] Detection module, which is used to build a detection model, perform traffic sign detection on the data to be detected, and obtain detection results;

[0126] The conversion module is used to process the detection results and the original data stream to generate data to be classified;

[0127] Classification module, which is used to build a classification model, extract traffic sign categories and location information from the classified data, and obtain classification results;

[0128] The post-processing module is used to combine the prior information database to post-process each target in the detection results and classification results to obtain the final detection results of traffic signs.

[0129] In a preferred embodiment, the traffic sign detection system may further include:

[0130] An optimization module is provided, which is used to obtain the real results of traffic signs and compare them with the final detection results of traffic signs to obtain conflicting data; and / or, to run the traffic sign detection method in an offline state, and to perform category and position labeling and data augmentation on the final detection results of traffic signs to obtain basic data; to use the conflicting data and / or basic data to optimize the lightweight detection model and the deep learning classification model; and to use the conflicting data and / or basic data as new prior information to update the prior information database.

[0131] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules in the system, and those skilled in the art can refer to the technical solution of the method to implement the composition of the system. That is, the embodiments in the method can be understood as preferred examples of constructing the system, which will not be elaborated here.

[0132] The traffic sign detection method and system provided by the above embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0133] Figure 3 Schematic diagram of the working framework of the traffic sign detection method and system for autonomous driving provided in the above embodiment of the present invention. Figure 3 As shown, the overall working framework is divided into two parts, among which the outside of the dotted box is divided into the workflow framework of the system applied to autonomous driving products; the inside of the dotted box is divided into the offline or online (online: refers to the normal working state of the system) data accumulation workflow framework.

[0134] like Figure 3As shown in the figure, when a traffic sign detection system is implemented in a product, the data stream for the autonomous driving process is provided by another system (the image data from the sensor (camera) passes through the ISP, undergoes dedistortion, and finally becomes a usable RGB or YUV image as the system input). Upon receiving the data stream, it undergoes preprocessing to convert the data into a state suitable for subsequent processing. It then undergoes a detection process, which uses a lightweight detection model to detect the presence of traffic signs in the data, ensuring detection speed and reducing computing power requirements. After obtaining the detection results, the system uses its own DSP operators or CPU to process the detection results and the original data stream to generate input for the classification module, completing the conversion process. The classification process then yields the final traffic sign category and location information. This classification process uses a classification model based on a deep learning network to extract high-level semantic information for traffic sign classification, ensuring classification accuracy. The conversion process also balances detection accuracy and computing power, achieving high detection accuracy while utilizing less computing power and ensuring fast operation.

[0135] Finally, the outputs of both the detection and classification processes are passed to post-processing. This process combines prior information from a self-built database with the detection and classification results to perform post-processing operations on each target. This includes target matching, target tracking, category fusion, and reporting logic screening, ultimately outputting the final detection results. The category fusion step improves the stability of the detection results, while the reporting logic screening step significantly reduces false detections, further enhancing the system's capabilities.

[0136] Furthermore, when the system is in the offline process, data accumulation will be carried out according to the workflow within the dotted box. First, the detection results obtained from post-processing are compared with the actual results obtained from other systems or the driving judgment logic, and the conflicting parts are retained as data for subsequent model optimization. The data obtained from this step are mostly difficult examples in the long-tail problem, which greatly alleviates the long-tail problem. At the same time, this process can also use the offline model to perform a detection-based method on the entire data stream to obtain basic data for model optimization. After manual labeling, the long-tail problem is further alleviated through data augmentation methods, and finally a priori information database is constructed. This database is used not only to optimize the classification model and / or detection model, but also to provide priori information to support each step of post-processing.

[0137] An embodiment of the present invention provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the terminal can be used to execute any one of the methods described in the foregoing embodiments of the present invention, or to execute any one of the systems described in the foregoing embodiments of the present invention.

[0138] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above-mentioned methods), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. In addition, the above-mentioned computer programs, computer instructions, data, etc. can be called by the processor.

[0139] The aforementioned computer programs, computer instructions, etc. may be partitioned and stored in one or more memories, and the aforementioned computer programs, computer instructions, data, etc. may be called by a processor.

[0140] The processor is configured to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0141] The processor and memory can be independent structures or integrated structures. When the processor and memory are independent structures, the memory and processor can be coupled via a bus.

[0142] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute any one of the methods in the above-mentioned embodiments of the present invention, or to run any one of the systems in the above-mentioned embodiments of the present invention.

[0143] The traffic sign detection method and system for autonomous driving provided by the above-mentioned embodiments of the present invention utilize a multi-stage processing approach, particularly the conversion process connecting the detection and classification stages. This differs from traditional two-stage networks, achieving decoupling of detection and classification while also ensuring lightweight network implementation and improved accuracy. DSP operators or CPU processing optimization and conversion logic are particularly important. The post-processing phase utilizes detection, classification, and prior information for integrated logical processing. The category fusion method and reporting logic screening algorithm are core components of the entire system, further improving the output accuracy of the entire detection method and system, ensuring its suitability for autonomous driving. The offline method for addressing long-tail problems significantly alleviates the inherent category imbalance of traffic signs. A proprietary data augmentation algorithm is used during data construction to ensure overall system accuracy and enhance the product's competitiveness in the autonomous driving field. The overall process, from offline optimization and updates to actual use, ensures the proper operation of the detection method and system and provides a foundation for continuous iteration and update of the detection method and system.

[0144] Matters not mentioned in the above embodiments of the present invention are well known in the art.

[0145] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A traffic sign detection method for autonomous driving, characterized in that: include: Obtaining a raw data stream in real-time autonomous driving, and preprocessing the raw data stream to obtain data to be detected; Constructing a detection model, performing traffic sign detection on the data to be detected, and obtaining a detection result; Using the detection results, multiple image locations that need to be classified are located and recorded as proposals; using DSP acceleration or CPU algorithm acceleration, a specific sampling and splicing operation is performed on the location proposals to obtain data to be classified as input data for the classification model; wherein the specific sampling and splicing method includes: The original image regions corresponding to the n detection results obtained by the detection model are cropped respectively and scaled to a fixed pixel size of w×h in width×height; Splicing the images of fixed size w×h into one picture along the h direction; Add a black image of size wxh between every two pictures to obtain an image of size wx(nxhx2); When the number of detection results n obtained in a certain frame is less than the set maximum number of targets N max When the black image of wⅹh is used to supplement the image of wⅹ(nⅹhⅹ2) until wⅹ(N max ⅹhⅹ2) pictures, that is, the data to be classified as the input data of the classification model; Constructing a classification model, extracting traffic sign categories and location information from the data to be classified, and obtaining classification results; Combined with the prior information database, post-processing is performed on each target in the detection result and the classification result to obtain the final detection result of the traffic sign.

2. The traffic sign detection method for autonomous driving according to claim 1, characterized in that: Preprocessing the original data stream includes: Acquire the position information in the raw data stream in real time, and locate the target area where the traffic sign is likely to appear in real time based on the position information; Acquire image data containing traffic signs by real vehicle acquisition, and mark the traffic signs in the image data to obtain sample data; The sample data is statistically analyzed to obtain location information of traffic signs in the sample data. The target area is scaled and / or cropped to obtain data to be detected of a specified size, which is used as input data for a detection model.

3. The traffic sign detection method for autonomous driving according to claim 1, wherein: The construction of the detection model and the detection of traffic signs on the data to be detected include: Construct a sample dataset containing labeled traffic signs; Based on a deep learning neural network, the deep learning neural network is trained using the sample data set to obtain a detection model; The data to be detected is used as input data of the detection model, and traffic sign detection is performed on the data to be detected.

4. The traffic sign detection method for autonomous driving according to claim 1, wherein: The construction of the classification model and the extraction of traffic sign categories and location information from the data to be classified include: Build a deep learning classification model based on deep learning neural network; Using the data to be classified as input to the classification model, extracting high-level semantic information of the data to be classified, classifying the traffic signs, and obtaining the categories of the traffic signs; Combined with the detection results obtained by the detection model, the position of the classified traffic sign is corrected to obtain the position information of the traffic sign.

5. The traffic sign detection method for autonomous driving according to claim 1, wherein: The post-processing of each target in the detection result and the classification result in combination with the prior information includes: The detection result and the classification result are post-processed using the prior information in the prior information database. The post-processing includes target matching, target tracking, category fusion and / or reporting logic screening to obtain the final detection result of the traffic sign.

6. The traffic sign detection method for autonomous driving according to claim 5, characterized in that: Also includes any one or more of the following: - The target matching includes quantifying multiple pieces of information and applying different weights to comprehensively determine the matching result; wherein the multiple pieces of information include category, location, high-level image features, and / or low-level image features; - The target tracking includes: tracking the traffic sign by filtering to obtain a tracking result, and compensating the tracking result with real-time posture information; - The category fusion includes: using probabilistic methods and combining big data statistical information to build a category fusion model to ensure the accuracy and stability of the categories; - The reporting logic screening includes: constructing different reporting logics according to different driving scenarios and driving speeds, wherein the driving scenarios include: conventional urban roads, elevated roads, highways and ramps.

7. The traffic sign detection method for autonomous driving according to any one of claims 1 to 6, characterized in that: Also includes: Obtaining the actual result of the traffic sign and comparing it with the final detection result of the traffic sign to obtain conflicting data; and / or, running the traffic sign detection method in an offline state, and performing category and location information labeling and data augmentation on the final detection results of the traffic signs to obtain basic data; Optimizing the detection model and the classification model using the conflicting data and / or the basic data; The conflicting data and / or the basic data are used as new prior information to update the prior information database.

8. A traffic sign detection system for autonomous driving, characterized in that: include: A data acquisition module, which is used to acquire the original data stream in real-time autonomous driving and pre-process the original data stream to obtain the data to be detected; A detection module, which is used to build a detection model, perform traffic sign detection on the data to be detected, and obtain a detection result; A conversion module, which uses the detection results to locate multiple image positions that need to be classified and records the positions as proposals; performs a specific sampling and splicing operation on the position proposals through DSP acceleration or CPU algorithm acceleration to obtain data to be classified as input data of the classification model; wherein the specific sampling and splicing method includes: cropping the original image areas corresponding to the n detection results obtained by the detection model respectively, and scaling them to images with a fixed size of wxh in width×height; splicing the images with a fixed size of wxh in the h direction into one image; adding a black image of size wxh between each two images to obtain an image of wx(nxhx2); when the number n of detection results obtained in a certain frame is less than the set maximum number of targets N max When the black image of wⅹh is used to supplement the image of wⅹ(nⅹhⅹ2) until wⅹ(N max ⅹhⅹ2) pictures, that is, the data to be classified as the input data of the classification model; A classification module, which is used to build a classification model, extract traffic sign categories and location information from the data to be classified, and obtain classification results; A post-processing module is used to combine the prior information database to post-process each target in the detection result and the classification result to obtain the final detection result of the traffic sign.

9. The traffic sign detection system for autonomous driving according to claim 8, characterized in that: Also includes: an optimization module, which is used to obtain the real result of the traffic sign and compare it with the final detection result of the traffic sign to obtain conflicting data; and / or, running the traffic sign detection method in an offline state, and performing category and location information labeling and data augmentation on the final detection results of the traffic signs to obtain basic data; Optimizing the detection model and the classification model using the conflicting data and / or the basic data; The conflicting data and / or the basic data are used as new prior information to update the prior information database.

10. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it can be used to perform the method according to any one of claims 1 to 7, or to run the system according to any one of claims 8 to 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to perform the method according to any one of claims 1 to 7, or to run the system according to any one of claims 8 to 9.

Images