Interactive labeling method and system for improving ADAS perception capability, medium and equipment

Abstract

The invention discloses an interactive labeling method and system for improving ADAS perception capability, a medium and equipment. The method comprises the following steps: acquiring a real-time driving state of a vehicle; controlling a data collector to collect driving scene data in real time, and storing the driving scene data in a circular cache region; when it is detected that the vehicle is in the auxiliary driving state, target driving scene data are collected from the circular cache region according to a driver instruction, and the target driving scene data are stored to a cloud end for labeling; and when it is detected that the vehicle is in a manual driving state, marking the driving scene data based on a marking neural network model, correcting a marking result, and uploading the corrected marking result to a cloud. Based on the data processing flow, efficient data acquisition and processing can be realized when the vehicle is in different driving states, and meanwhile, the safety and driving experience of a driver are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle driving annotation, in particular to an interactive annotation method and system for improving ADAS perception capability, a medium and equipment. BACKGROUND

[0002] In the visual perception module of the current assisted driving system, the phenomenon of missing detection, misjudgment or insufficient understanding of environmental elements often occurs, one of the core bottlenecks is that the scale of the annotation data used for training the model is still insufficient. The existing data covers limited scene dimensions, and it is difficult to cover complex and variable traffic conditions, extreme weather, special obstacles and rare long-tail scenes in the real world. This leads to a decline in the generalization ability of the perception model when encountering scenes that have not been fully learned. To build a more reliable and adaptable perception system, it is necessary to continuously invest in large-scale and diversified data collection and annotation work, and gradually fill the gaps in model cognition by continuously expanding the breadth and depth of data.

[0003] In the assisted driving state, the vehicle owner is easy to over-rely on the safety of the assisted driving. Thus, the attention to the road state is reduced, and the traffic accident caused by the untimely manual intervention is easy to occur, and the similar traffic accident is usually fatal. Therefore, it is urgent to design a technical scheme to solve the above technical problems. SUMMARY

[0004] The present application provides an interactive annotation method and system for improving ADAS perception capability, a medium and equipment, which can realize efficient data collection and processing while improving the safety and driving experience of the driver when the vehicle is in different driving states. In a first aspect, an interactive annotation method for improving ADAS perception capability is provided, comprising: acquiring a real-time driving state of a vehicle; controlling a data collector to collect driving scene data in real time and storing the driving scene data in a circular buffer area; when it is detected that the vehicle is in an assisted driving state, collecting target driving scene data from the circular buffer area according to a driver's instruction and storing the target driving scene data in cloud annotation; when it is detected that the vehicle is in a manual driving state, annotating the driving scene data based on an annotation neural network model and uploading the annotation result to the cloud after correction.

[0005] In some embodiments, the collecting target driving scene data from the circular buffer area according to the driver's instruction comprises: acquiring a driving scene collection instruction issued by the driver; controlling the central control screen to display a scene classification list according to the driving scene collection instruction; Obtain the scene classification results issued by the driver based on the scene classification list, and collect target driving scene data from the circular buffer based on the scene classification results.

[0006] In some embodiments, the step of collecting target driving scenario data from the circular buffer based on the scenario classification result includes: Record the time point at which the driving scenario acquisition command is received; Extract data slices from N seconds before the acquisition time point to N seconds after the acquisition time point from the circular buffer; By combining the scene classification results with the data slices, the target driving scene data is obtained.

[0007] In some embodiments, the annotation of the driving scene data based on the labeled neural network model includes: The labeled neural network model includes a sequentially connected multimodal feature extraction backbone network, a dynamic sparse attention module, a pruning module, and a multi-task output head; The multimodal feature extraction backbone network constructed using neural architecture search technology is used to extract multimodal features from the driving scene data; The dynamic sparse attention module is used to suppress redundant information in the extracted features; The pruning module is used to perform structured pruning on the shallow convolutional layers of the multimodal feature extraction backbone network, and the number of output channels of the convolutional layers after pruning is reconstructed. The multi-task output head is used to decouple the driving scene features processed by the multimodal feature extraction backbone network, the dynamic sparse attention module, and the pruning module, and output the annotation results.

[0008] In some embodiments, the correction of the annotation results includes: The annotation results are stored in the memory of the cockpit domain controller; When the vehicle is detected to be parked, the control screen will display the annotation results. Obtain the result processing instructions issued by the driver based on the annotation results, and correct the annotation results according to the result processing instructions.

[0009] In some embodiments, correcting the annotation results according to the result processing instructions includes: When the result processing instruction is a labeling result correct instruction, the labeling result corresponding to the correct instruction is confirmed and stored; When the result processing instruction is a labeling result error instruction, the labeling result corresponding to the error instruction will be corrected according to the driver's processing result.

[0010] In some embodiments, storing the annotation results in the cockpit domain controller's memory includes: The annotation results, along with the corresponding driving scenario data, timestamps, and GPS location information, are packaged into a data packet, which is then stored in the memory of the cockpit domain controller.

[0011] Secondly, an interactive annotation system for enhancing ADAS perception capabilities is provided, including: The driving status acquisition module is used to acquire the real-time driving status of the vehicle. The driving data acquisition module is used to control the data acquisition device to collect driving scene data in real time and store it in a circular buffer. The assisted driving annotation module, communicatively connected to the driving data acquisition module, is used to collect target driving scene data from the cyclic buffer according to the driver's instructions when the vehicle is detected to be in assisted driving mode, and to store the target driving scene data in the cloud for annotation; and, The manual driving annotation module is communicatively connected to the driving data acquisition module. When the vehicle is detected to be in manual driving mode, the module annotates the driving scene data, corrects the annotation results, and uploads them to the cloud.

[0012] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the interactive annotation method for improving ADAS perception capabilities as described above.

[0013] Fourthly, embodiments of the present invention provide an electronic device, including a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein the processor, when executing the computer program, implements the interactive annotation method for improving ADAS perception capabilities as described above.

[0014] Compared with existing technologies, the advantages of this invention are as follows: This invention, through an intelligent data acquisition and annotation mechanism, enables efficient data acquisition and processing under different driving conditions. In assisted driving mode, target driving scenario data is collected from the cyclic buffer according to driver instructions and uploaded to the cloud for annotation. This process avoids storing and processing excessive irrelevant data, improving data processing efficiency. In manual driving mode, the system automatically annotates the collected driving scenario data based on an annotation neural network model, corrects the annotation results, and uploads them to the cloud, ensuring data accuracy and reliability. Furthermore, cloud storage makes data management and analysis more convenient and provides rich training data for subsequent intelligent driving functions. This invention enables more precise driving assistance for vehicles, while improving driver safety and driving experience. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of an embodiment of an interactive annotation method for improving ADAS perception capabilities according to the present invention; Figure 2 This is a schematic diagram of an embodiment of an interactive annotation system for enhancing ADAS perception capabilities according to the present invention. Figure 3 This is a schematic diagram of another embodiment of the interactive annotation system for improving ADAS perception capabilities according to the present invention. Detailed Implementation

[0016] Referring now to specific embodiments of the invention, examples of which are illustrated in the accompanying drawings. Although the invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0017] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Note: The examples described below are merely specific examples and are not intended to limit the embodiments of the present invention to the specific steps, values, conditions, data, order, etc. Those skilled in the art can utilize the concept of the present invention to construct more embodiments not mentioned herein by reading this specification.

[0019] Please see Figure 1The present invention provides a flowchart illustrating an interactive annotation method for improving ADAS perception capabilities, applied to a driving domain controller. The method includes: Step S100: Obtain the real-time driving status of the vehicle; Vehicle driving states generally include assisted driving state and manual driving state, with assisted driving state corresponding to ADAS (Advanced Driver Assistance Systems).

[0020] Step S200: Control the data acquisition device to collect driving scenario data in real time and store it in the circular buffer area; During normal vehicle operation, the underlying data-driven modules (camera data driver, LiDAR data driver, millimeter-wave radar data driver) of the autonomous driving domain controller continuously write raw sensor data (video streams, point clouds, etc.) into a high-speed loop buffer of fixed duration. This buffer is typically stored in memory or high-speed storage and is designed to temporarily retain historical data from a recent period.

[0021] Step S300: When the vehicle is detected to be in assisted driving mode, target driving scene data is collected from the cyclic buffer according to the driver's instructions, and the target driving scene data is stored in the cloud for annotation. The step of collecting target driving scenario data from the cyclic buffer according to driver instructions includes: Obtain driving scenario collection instructions issued by the driver; The central control screen is controlled to display a list of scene categories according to the driving scene acquisition command; Obtain the scene classification results issued by the driver based on the scene classification list, and collect target driving scene data from the circular buffer based on the scene classification results.

[0022] Specifically, in this embodiment of the invention, since the vehicle is in assisted driving mode, the driver has enough energy to pay attention to road conditions and special scenarios. When the driver pays attention to special road conditions, he can trigger scenario recording through physical buttons, central control screen or voice, such as: "hi, <voice wake-up word>, record special scenario", which corresponds to the driving scenario collection command issued by the driver.

[0023] The central control screen will prompt you to select a list of special scenarios to upload, including: "Road construction", "Slow-moving construction vehicle ahead", "Foreign object on the road", "Incorrect vehicle type recognition ahead", "Inaccurate obstacle recognition", "Other types", etc. This list can be configured in the backend, and the voice system will also provide corresponding prompts.

[0024] The driver selects from the list, for example, by saying "first one" via voice. After receiving the instruction, the system triggers subsequent data collection actions. Therefore, it obtains the scenario classification results issued by the driver based on the scenario classification list, and collects the target driving scenario data from the circular buffer based on the scenario classification results.

[0025] The step of collecting target driving scenario data from the circular buffer based on the scenario classification result includes: Record the time point at which the driving scenario acquisition command is received; Extract data slices from N seconds before the acquisition time point to N seconds after the acquisition time point from the circular buffer; By combining the scene classification results with the data slices, the target driving scene data is obtained.

[0026] Specifically, the first step is to locate the time point: mark the moment when the car owner presses the button—that is, the moment when the driving scenario collection command is obtained—as the collection time point T0.

[0027] Next, the slice range is determined: based on a preset strategy (e.g., the first 6 seconds, the last 6 seconds), the complete sensor data sequence from T0-6s to T0+6s is extracted from the circular buffer. This "before and after N seconds" design is crucial because it not only captures the instant when the event occurs but also preserves the cause and subsequent development of the event, providing a complete context for algorithmic analysis, making the data extremely valuable.

[0028] The extracted data slices are then transferred from the circular buffer for data storage, specifically written to the domain controller's non-volatile storage space, forming a complete data packet—the target driving scenario data. This data packet contains multimodal sensor data throughout the entire event.

[0029] In step S400, when the vehicle is detected to be in a manual driving state, the driving scene data is labeled based on the labeled neural network model, and the labeling results are corrected and uploaded to the cloud.

[0030] The annotation of the driving scene data based on the labeled neural network model includes: S410, the labeled neural network model includes a sequentially connected multimodal feature extraction backbone network, a dynamic sparse attention module, a pruning module, and a multi-task output head; S420, The multimodal feature extraction backbone network constructed using neural architecture search technology is used to extract multimodal features from the driving scene data; S430, the dynamic sparse attention module is used to suppress redundant information of the extracted features; S440, The pruning module is used to perform structured pruning on the shallow convolutional layers of the multimodal feature extraction backbone network, and the number of output channels of the convolutional layers after pruning is reconstructed. S450, the multi-task output head is used to decouple the driving scene features processed by the multimodal feature extraction backbone network, the dynamic sparse attention module and the pruning module, and output the annotation results.

[0031] Specifically, in this embodiment of the invention, the labeled neural network model includes: a multimodal feature extraction backbone network, a dynamic sparse attention module, a pruning module, and a multi-task output head. The multimodal feature extraction backbone network is used to simultaneously process camera images and LiDAR point cloud data and extract multimodal features. Addressing the sparse target distribution (e.g., distant small targets, occluded targets) in driving scenarios, the dynamic sparse attention module is superimposed on the feature map output by the backbone network to suppress redundant information in the feature map and avoid the high computational overhead of fully connected attention. The dynamic sparse attention module generates a sparse mask by calculating the gradient magnitude and spatial entropy of the feature map and sets the response of non-critical region features to zero. Then, for the retained critical region features, grouped fully connected channel attention is used to suppress redundant channel activation.

[0032] The pruning module is connected to the dynamic sparse attention module and is used to perform mixed precision quantization and structured pruning on the model parameters. Specifically, it performs KL divergence-based dynamic range calibration on the feature maps output by the backbone network, and performs structured pruning on the shallow convolutional layers of the multimodal feature extraction backbone network. It also reconstructs the number of output channels of the convolutional layers after pruning, such as adjusting the original 256 channels to 182 channels to an integer multiple of 256 channels, namely 128 channels.

[0033] The multi-task output head is connected to the pruning module and is used to synchronously output annotation results such as 2D / 3D bounding box coordinates, category labels, and confidence scores. The core functionality is to synchronously output multiple types of annotation information—2D / 3D bounding boxes, category labels, and confidence scores—through a single forward inference. Specifically, this corresponds to three independent but collaborative sub-branches: a 2D / 3D bounding box generation branch, a semantic segmentation branch, and a confidence score calculation branch.

[0034] In step S400, correcting the annotation results includes: The annotation results are stored in the memory of the cockpit domain controller; When the vehicle is detected to be parked, the control screen will display the annotation results. Obtain the result processing instructions issued by the driver based on the annotation results, and correct the annotation results according to the result processing instructions.

[0035] The step of correcting the annotation results according to the result processing instructions includes: When the result processing instruction is a labeling result correct instruction, the labeling result corresponding to the correct instruction is confirmed and stored; When the result processing instruction is a labeling result error instruction, the labeling result corresponding to the error instruction will be corrected according to the driver's processing result.

[0036] Specifically, in this embodiment of the invention, real-time data streams from sensors such as cameras and LiDAR are used, on the one hand, to ensure basic safety during manual driving, and on the other hand, are input in parallel into the aforementioned lightweight annotation neural network model. This model performs real-time inference on the data stream, executing automated annotation tasks such as 2D / 3D bounding box generation, semantic segmentation, and lane detection.

[0037] The generated automated annotation results are packaged together with the corresponding original data frames / fragments and metadata such as timestamps and GPS locations to form a structured data packet. The packaged data is then uploaded to the local storage of the cockpit domain controller for temporary storage via a communication middleware. To save resources, the upload strategy can be periodic or based on data value filtering, such as only uploading difficult samples with confidence levels above or below a certain threshold, i.e., samples where the model is "uncertain".

[0038] When the vehicle is detected to be parked or the owner is in a non-driving, idle period, the cockpit interaction system will prompt the owner through an "idle time task pop-up." Instead of the original video, the system presents an image or short video clip with automatically generated annotation boxes from the model, possibly accompanied by simple interactive tasks, such as: a. Confirm Correction: If the annotations are correct, click "Confirm." b. Correct Errors: Drag to adjust incorrect annotation boxes or select the correct category from the list.

[0039] The high-quality labeled data, corrected by the car owner, is reported to the cloud. The cloud platform's quality management module will then give the data a final quality score. This score can be used as a basis for awarding points and rewards to car owners, thus forming a positive feedback loop for continuous optimization.

[0040] See also Figure 2 As shown, this embodiment of the invention also provides an interactive annotation system for improving ADAS perception capabilities, including: The driving status acquisition module is used to acquire the real-time driving status of the vehicle. The driving data acquisition module is used to control the data acquisition device to collect driving scene data in real time and store it in a circular buffer. The assisted driving annotation module, communicatively connected to the driving data acquisition module, is used to collect target driving scene data from the cyclic buffer according to the driver's instructions when the vehicle is detected to be in assisted driving mode, and to store the target driving scene data in the cloud for annotation; and, The manual driving annotation module is communicatively connected to the driving data acquisition module. When the vehicle is detected to be in manual driving mode, the module annotates the driving scene data, corrects the annotation results, and uploads them to the cloud.

[0041] In summary, the main advantages of this invention are as follows: 1. Improved data labeling quality. Even professional data collectors cannot cover all special scenarios. As frontline data creators, car owners can effectively help collect data for special scenarios.

[0042] 2. It enhances the driver's focus on the road while in assisted driving mode, thereby improving the safety of assisted driving.

[0043] 3. In manual driving mode, the computing power of the autonomous driving system during idle time is utilized to obtain the calibration data at the scene and perform automated annotation, reducing annotation errors caused by distortion during data transmission.

[0044] 4. The interactive labeling system allows non-professional car owners to participate in the labeling of driver assistance systems, thus improving the earning potential of drivers and other members of the public.

[0045] 5. The indirect and continuous interaction between vehicle drivers and algorithm researchers creates positive feedback.

[0046] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.

[0047] See also Figure 3 As shown, this embodiment of the invention also provides an interactive annotation system to enhance ADAS perception capabilities. The core advantage of this system lies in the in-vehicle cockpit calibration interaction system and the automated annotation system within the assisted driving domain controller. Traditional annotation systems rely on data collection vehicles, requiring significant manpower to collect data and perform cleaning and annotation for training. This system allows non-professional vehicle owners to participate in data processing, effectively improving the quality of the data collection and annotation processes. Improved data quality consequently enhances the perception quality of assisted driving. Since the current technological bottleneck in assisted driving lies in perception capabilities, this indirectly improves the overall assisted driving experience. Figure 3 As shown, the overall structure is as follows: Inside the vehicle, based on the current mainstream electronic and electrical architecture, a calibration interaction system for the driver will be deployed in the cockpit domain. This system can operate in both assisted driving and manual driving modes. In both modes, the calibration system can intervene in the calibration process in different forms, while also helping the driver improve driving safety.

[0048] The autonomous driving domain controller also deploys calibration-related components, including data acquisition components that operate in autonomous driving mode and data annotation components that operate in non-autonomous driving mode. These correspond to the assisted driving mode and manual driving mode in the interactive system, respectively.

[0049] When the vehicle is in assisted driving mode, the driver has sufficient focus to pay attention to road conditions and special scenarios. Upon detecting a special scenario, data collection is triggered via buttons or physical controls in the cockpit interface. After receiving the data collection command, the autonomous driving domain controller stores a certain amount of cached data (e.g., 6 seconds before and 6 seconds after the event). This data represents the special scenario and possesses unique value for algorithm research. After being aggregated by the cockpit domain controller, the data is ultimately uploaded to the cloud, contributing to algorithm iteration.

[0050] The automated calibration system under manual driving conditions is designed as follows: Since the vehicle is in manual driving mode, the interactive interface will not provide any calibration prompts during manual driving (to avoid distracting the driver and ensuring safety). Instead, the system continuously performs automated image annotation calculations in the background using the idle computing power of the autonomous driving domain controller. The calculation results are ultimately stored in the cockpit domain controller through certain strategies. When the driver is at leisure, the cockpit interaction system will prompt the driver to process the annotation data. Based on the annotation prompts, the data is classified. When the driver provides high-quality annotation data, the algorithm research team can provide corresponding rewards, forming a positive feedback loop and continuously iterating the algorithm's evolution.

[0051] In summary, the overall functions are as follows: 1. In assisted driving mode, the cockpit offers a degree of entertainment while requiring the driver to constantly monitor road conditions and provide feedback. This feedback process typically yields important, specific scenario data. Such data is of particular value to algorithm researchers. Simultaneously, the driver benefits from providing this data, is motivated to continuously monitor road conditions, and this contributes to safer vehicle operation. Furthermore, it allows for earlier intervention in driving, ensuring the safety of assisted driving.

[0052] 2. In manual driving mode, the autonomous driving domain controller has sufficient computing power. At this time, an autonomous driving annotation component runs on the autonomous driving domain controller, automatically collecting and annotating data during vehicle operation in real time. Since automated annotation systems usually have certain quality defects, vehicle owners can participate in the data quality screening to improve the data quality. When vehicle owners continuously provide high-quality annotated data, they benefit from it and thus continuously participate in the cause of helping to develop assisted driving.

[0053] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the method steps of the above method.

[0054] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0055] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program that runs on the processor. When the processor executes the computer program, it implements all or part of the method steps described above.

[0056] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.

[0057] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0058] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0059] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An interactive labeling method for improving ADAS perception capability, characterized in that, The method comprises: acquiring real-time driving state of a vehicle; controlling a data collector to collect driving scene data in real time and store the driving scene data in a circular buffer area; when it is detected that the vehicle is in an assisted driving state, collecting target driving scene data from the circular buffer area according to a driver instruction and storing the target driving scene data in a cloud terminal for labeling; when it is detected that the vehicle is in a manual driving state, labeling the driving scene data based on a labeled neural network model and uploading the labeling result to the cloud terminal after correction.

2. The interactive labeling method for boosting ADAS perception capability of claim 1, wherein, The collecting of the target driving scene data from the circular buffer area according to the driver instruction comprises: acquiring a driving scene collection instruction issued by the driver; controlling a central control screen to display a scene classification list according to the driving scene collection instruction; acquiring a scene classification result issued by the driver according to the scene classification list and collecting target driving scene data from the circular buffer area according to the scene classification result.

3. The interactive labeling method for boosting ADAS perception capability of claim 2, wherein, The collecting of the target driving scene data from the circular buffer area according to the scene classification result comprises: recording a collection time point at which the driving scene collection instruction is acquired; extracting a data slice between N seconds before the collection time point and N seconds after the collection time point in the circular buffer area; and combining the scene classification result and the data slice to obtain the target driving scene data.

4. The interactive labeling method for boosting ADAS perception capability of claim 1, wherein, The labeling of the driving scene data based on the labeled neural network model comprises: the labeled neural network model comprises a multi-modal feature extraction backbone network, a dynamic sparse attention module, a pruning module and a multi-task output head connected in sequence; the multi-modal feature extraction backbone network constructed by a neural architecture search technology is used to perform multi-modal feature extraction on the driving scene data; the dynamic sparse attention module is used to suppress redundant information of the extracted features; the pruning module is used to perform structural pruning on shallow convolutional layers of the multi-modal feature extraction backbone network and reconstruct the output channel number of the convolutional layers after pruning; the multi-task output head is used to perform task decoupling on the driving scene features processed by the multi-modal feature extraction backbone network, the dynamic sparse attention module and the pruning module, and output the labeling result.

5. The interactive labeling method for boosting ADAS perception capability of claim 1, wherein, The correction of the labeling result comprises: storing the labeling result in a memory of a cabin domain controller; when it is detected that the vehicle is in a parking state, controlling a central control screen to display the labeling result; acquiring a result processing instruction issued by the driver according to the labeling result and correcting the labeling result according to the result processing instruction.

6. The interactive labeling method for boosting ADAS perception capability of claim 5, wherein, The correction of the labeling result according to the result processing instruction comprises: when the result processing instruction is a correct instruction, confirming and storing one labeling result corresponding to the correct instruction; when the result processing instruction is an incorrect instruction, correcting one labeling result corresponding to the incorrect instruction according to a processing result of the driver.

7. The interactive labeling method for boosting ADAS perception capability of claim 5, wherein, The storage of the labeling result in the memory of the cabin domain controller comprises: The labeling result, the driving scene data corresponding to the labeling result, the timestamp, and the GPS position information are packaged to obtain a data packet, and the data packet is stored in a memory of the cockpit domain controller.

8. An interactive labeling system for improving ADAS perception capability, comprising: a driving state acquisition module configured to acquire real-time driving state of a vehicle; a driving data acquisition module configured to control a data collector to acquire driving scene data in real time and store the driving scene data in a circular buffer area; an assisted driving labeling module in communication connection with the driving data acquisition module, configured to, when detecting that the vehicle is in an assisted driving state, acquire target driving scene data from the circular buffer area according to a driver's instruction, and store the target driving scene data in a cloud labeling; and a manual driving labeling module in communication connection with the driving data acquisition module, configured to, when detecting that the vehicle is in a manual driving state, label the driving scene data, and upload the labeling result to the cloud after correction.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the interactive labeling method for improving ADAS perception capability as claimed in any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and operable on the processor, characterized in that, The processor, when running the computer program, implements the interactive labeling method for improving ADAS perception capability as claimed in any one of claims 1 to 7.

Images