Large model-based autonomous driving scene mining method, device and equipment

Abstract

The application discloses a large model-based automatic driving scene mining method, device and equipment, relates to the field of automatic driving, and comprises the following steps: collecting automatic driving images, performing frame extraction processing on the collected automatic driving video, obtaining single-frame images, labeling the single-frame images by using labels, generating enhanced labeled images, the labels comprising lane lines, road edges, guide arrows, obstacle three-dimensional boxes and a self-vehicle future driving track, the self-vehicle future driving track being used for representing a predicted driving path of the vehicle in a subsequent time period, inputting the enhanced labeled images and data verification prompt words and data mining prompt words into a large model, performing data verification and scene mining on the enhanced labeled images, outputting scene mining results, and performing data warehousing based on the scene mining results. The application significantly improves the automatic driving data processing efficiency and scene coverage capability, reduces the labor cost, and is suitable for a large-scale automatic driving data closed-loop system.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a method, apparatus, and device for autonomous driving scene mining based on a large model. Background Technology

[0002] As autonomous driving technology evolves from Level 2 assisted driving to Level 4 advanced autonomous driving, its research and development paradigm has shifted from algorithm-driven to data-driven. Against this backdrop, efficiently and accurately identifying low-frequency, high-risk long-tail scenarios from massive amounts of roadside data has become a core requirement for building a data closed loop and improving the safety and reliability of autonomous driving systems.

[0003] In related technologies, methods typically involve filtering and mining collected driving video data based on manually defined rules or by combining multimodal large models. The former relies on engineers' experience to build a rule base and uses automated scripts to match suspected abnormal segments, while the latter attempts to leverage the powerful understanding capabilities of large models to directly identify and classify complex driving scenarios from video data.

[0004] However, existing technologies suffer from the following technical problems: rule-based methods heavily rely on prior knowledge and struggle to adapt to dynamically changing real-world traffic environments. Directly using large models to process video data faces issues of slow processing speed and high computational costs. Both rule-based methods and large-model methods that directly input raw video primarily rely on judgments based on already occurred footage, failing to effectively understand and utilize the vehicle's future trajectory information, thus making it difficult to identify and uncover potential hazardous scenarios. Summary of the Invention

[0005] This application provides a method, apparatus, and device for mining autonomous driving scenarios based on large models, which can solve the problems of difficulty in understanding large model videos, slow video processing, inability to predict upcoming scenarios, low efficiency, high cost, and incomplete scenario coverage in the existing technology for large-scale autonomous driving data mining.

[0006] In a first aspect, embodiments of this application provide a method for mining autonomous driving scenarios based on a large model, characterized in that the method includes: The system acquires autonomous driving images and performs frame extraction on the acquired autonomous driving videos to obtain single-frame images. A single frame image is labeled with tags to generate an enhanced labeled image. The tags include lane lines, road edges, guide arrows, 3D boxes of obstacles, and the vehicle's future driving trajectory. The vehicle's future driving trajectory is used to characterize the vehicle's predicted driving path in subsequent time periods. Input the augmented labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the augmented labeled image, and output the scene mining results; Data is entered into the database based on the results of scene mining.

[0007] In one implementation, the frame extraction process for the acquired autonomous driving video specifically includes: The frame extraction frequency is determined based on the vehicle's speed, and the autonomous driving images are then extracted at equal intervals based on the determined frame extraction frequency.

[0008] In one implementation, the data verification prompt includes a description of a valid sample and verification criteria; The description of qualified samples includes clear annotations, reasonable trajectories, accurate 3D bounding boxes, and no obstructions or missing annotations. The verification criteria include whether the labeling is complete, whether the location is accurate, whether the trajectory is reasonable, and whether the scenario is valid.

[0009] In one implementation, the input augmented image, data verification prompts, and data mining prompts are fed into a large model to perform data verification and scene mining on the augmented image, and output scene mining results, specifically including: Input the enhanced labeled images, data verification tips, and data mining tips into the large model; The large model performs data verification on the current enhanced labeled image based on data verification prompts and outputs the data verification results: If the data verification result is qualified, the large model will perform scene mining on the current enhanced labeled image based on the data mining prompt words to obtain the scene mining result; If the data verification result is unqualified, the current enhanced labeled image is actively filtered. If the active filtering result is qualified, the large model performs scene mining on the current enhanced labeled image based on data mining prompts to obtain scene mining results. If the active filtering result is unqualified, the current enhanced labeled image is discarded.

[0010] In one implementation, the large model performs scene mining on the current enhanced labeled image based on data mining prompts to obtain scene mining results, specifically including: The large model determines the upcoming driving scenario based on the vehicle's future trajectory in the current enhanced and annotated image, as well as data mining prompts. Based on the determined driving scenario, the scenario mining results are obtained, which include scenario type, scenario risk level, key element judgment, and scenario confidence level.

[0011] In one implementation, the scene types include turning, lane changing, construction, road obstruction, ghosting, sudden braking, and driving in the wrong direction; The key elements to be judged include whether the trajectory is reasonable and whether the obstacles are correct.

[0012] In one implementation, the data storage based on the scene mining results specifically includes: Based on scene confidence scores from scene mining results: If the scene confidence level is lower than the preset threshold, the scene mining results will be actively reviewed and then stored in the database. If the scene confidence level is not lower than the preset threshold, the scene mining results will be directly stored in the database.

[0013] In one implementation, after the scene mining results are stored in the database, the method further includes: Based on the selected time length of the vehicle's future driving trajectory, the autonomous driving images after the enhanced and annotated images for a set time are labeled with the same tags as the enhanced and annotated images, and the labeled data is categorized and stored.

[0014] Secondly, embodiments of this application provide an autonomous driving scene mining device based on a large model, the device comprising: The data preprocessing module is used to acquire autonomous driving images and perform frame extraction processing on the acquired autonomous driving videos to obtain single-frame images. The single-frame images are then labeled with tags to generate enhanced labeled images. The mining module is used to input the enhanced labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the enhanced labeled image and output the scene mining results. The data storage module is used to store data based on the results of scene mining.

[0015] Thirdly, embodiments of this application provide an autonomous driving scene mining device based on a large model. The autonomous driving scene mining device based on a large model includes a processor, a memory, and an autonomous driving scene mining program based on a large model stored in the memory and executable by the processor. When the autonomous driving scene mining program based on a large model is executed by the processor, it implements the steps of the autonomous driving scene mining method based on a large model as described in any one of claims 1 to 8.

[0016] The beneficial effects of the technical solutions provided in this application include: This application obtains single-frame images by extracting frames from autonomous driving videos. These single-frame images are then augmented with annotations including lane lines, 3D bounding boxes of obstacles, and the vehicle's future trajectory. A large-scale model is used to verify and mine scenes from these augmented images. The scene mining results are then stored in a database. This addresses the technical problems in related technologies, such as the low efficiency of manual scene mining in autonomous driving, the limited representation of future driving intentions by single-frame annotations, and the inconsistent data quality due to a lack of standardized verification. By introducing labels for the vehicle's future trajectory, this application allows the large-scale model to achieve equivalent video temporal prediction by processing only a single frame, significantly improving the efficiency and accuracy of scene mining. Furthermore, the integrated data verification and mining approach achieves standardized output and automated integration, ensuring the validity of the stored data and providing high-quality long-tail scene data for training autonomous driving models. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the autonomous driving scene mining method based on a large model according to this application; Figure 2 This is a functional module diagram of an embodiment of the autonomous driving scene mining device based on a large model according to this application; Figure 3 This is a schematic diagram of the hardware structure of the autonomous driving scene mining device based on a large model involved in the embodiments of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] In a first aspect, embodiments of this application provide a method for mining autonomous driving scenarios based on a large model.

[0020] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the autonomous driving scene mining method based on a large model according to this application. Figure 1 As shown, the methods for mining autonomous driving scenarios based on large models include: S1: Acquire autonomous driving images and perform frame extraction on the acquired autonomous driving video to obtain a single frame image; S2: Label the single frame image to generate an enhanced labeled image. The labels include lane lines, road edges, guide arrows, obstacle 3D boxes, and the vehicle's future driving trajectory. S3: Input the enhanced labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the enhanced labeled image, and output the scene mining results; S4: Data is imported into the database based on the results of scene mining.

[0021] In this embodiment, step S1 collects autonomous driving images and videos through vehicle-mounted sensors, and performs equal-interval frame extraction on the autonomous driving images and videos according to preset time intervals or driving distance intervals to extract discrete, time-representative single-frame images to form an input image sequence for subsequent scene analysis and model inference. Each single-frame image contains static environmental information and vehicle motion state information at the corresponding time, thereby solving the problems of slow speed and high cost when large models process video data.

[0022] Step S2 involves overlaying multi-layered data labels, including lane lines, road edges, guide arrows, obstacle 3D boxes, and the vehicle's future driving trajectory, onto a single-frame image to form an enhanced labeled image. The vehicle's future driving trajectory is based on pre-collected and labeled labels. The trajectory ground truth is located in the vehicle's coordinate system. A coordinate transformation matrix is ​​used to transform the coordinate system and project the trajectory points onto the image plane, ultimately obtaining a visualized future driving trajectory on the image. This process solidifies the dynamic driving information, originally distributed in the video time sequence, into a single-frame static image, solving the shortcomings of large models in processing videos. Equivalent dynamic scene judgment can be achieved using a single-frame image, improving processing efficiency.

[0023] Step S3 inputs the enhanced labeled image, data verification prompts, and data mining prompts into the multimodal large model. The data verification prompts guide the large model to verify the validity of the enhanced labeled image and output a verification conclusion indicating whether the enhanced labeled image is qualified. The data mining prompts tell the large model the lines in the enhanced labeled image representing the vehicle's future driving trajectory and instruct the large model to combine the current static image with prior trajectory information to infer and judge the upcoming driving scenario. Then, according to the specified scenario category (such as turning, lane changing, construction, lane obstruction, unexpected pedestrian appearance, sudden braking, wrong-way driving, etc.), the model outputs the scenario mining results. This step, through a dual-prompt word system, enables large models to simultaneously perform data quality screening and long-tail scene recognition based on single-frame images without processing video streams. This achieves efficient mapping from raw data to high-value scene labels, avoiding the high computational cost and latency issues of video inference. Furthermore, by using trajectory priors, the model is endowed with the ability to predict future situations, providing accurate candidate criteria for subsequent manual review and data entry. This significantly improves the automation level and coverage of scene mining in the autonomous driving data closed loop.

[0024] In one implementation, the frame extraction process for the acquired autonomous driving video specifically includes: The frame extraction frequency is determined based on the vehicle's speed, and the autonomous driving images are then extracted at equal intervals based on the determined frame extraction frequency.

[0025] In this embodiment, the frame extraction frequency is dynamically adjusted according to the real-time driving speed of the vehicle, and images are extracted at equal time intervals according to the determined frame extraction frequency. In high-speed driving scenarios, high-frequency frame extraction is performed to ensure that sufficient image density can be obtained to guarantee perception accuracy when the scene changes drastically due to the high speed of the vehicle. In low-speed driving scenarios, low-frequency frame extraction is performed to save computing resources. Through the above processing, the amount of computation is further reduced while ensuring accuracy.

[0026] In one implementation, the data verification prompt includes a description of a valid sample and verification criteria; The description of qualified samples includes clear annotations, reasonable trajectories, accurate 3D bounding boxes, and no obstructions or missing annotations. The verification criteria include whether the labeling is complete, whether the location is accurate, whether the trajectory is reasonable, and whether the scenario is valid.

[0027] In this embodiment, when using a large model to verify autonomous driving data, a set of dedicated data verification prompts is designed. These prompts consist of two parts: a description of qualified samples (clearly informing the model what constitutes a qualified labeled sample, including clear annotations, reasonable trajectories, accurate 3D bounding boxes, and no occlusions or missing labels) and verification criteria (requiring the model to sequentially check whether the annotations are complete, the positions are accurate, the trajectories are reasonable, and the scene is effective). Through these prompts, the large model can determine the qualification of output data based on a single enhanced image, thereby achieving automated quality screening of massive amounts of collected data. Data that fails verification is then transferred to manual review, significantly reducing the amount of manual review and standardizing the evaluation criteria.

[0028] In one implementation, the input augmented image, data verification prompts, and data mining prompts are fed into a large model to perform data verification and scene mining on the augmented image, and output scene mining results, specifically including: Input the enhanced labeled images, data verification tips, and data mining tips into the large model; The large model performs data verification on the current enhanced labeled image based on data verification prompts and outputs the data verification results: If the data verification result is qualified, the large model will perform scene mining on the current enhanced labeled image based on the data mining prompt words to obtain the scene mining result; If the data verification result is unqualified, the current enhanced labeled image is actively filtered. If the active filtering result is qualified, the large model performs scene mining on the current enhanced labeled image based on data mining prompts to obtain scene mining results. If the active filtering result is unqualified, the current enhanced labeled image is discarded.

[0029] In this embodiment, after the enhanced labeled image, along with data verification prompts and data mining prompts, is input into the large model, the large model first performs a quality check on the current enhanced labeled image based on the data verification prompts and outputs a qualified or unqualified verification result.

[0030] If the verification result is satisfactory, the large model continues to mine and output information such as scene type and risk level based on data mining prompts. If the verification result is unsatisfactory, the image will not directly enter the mining process. Instead, it will be screened out and manually reviewed. Only images that pass manual review will be sent to the large model for scene mining. Images that still fail manual review will be discarded. This tiered process of verification first, then mining, and manual intervention for unsatisfactory images ensures that all data entering the mining stage meets quality requirements. Manual review also helps to recover some initially deemed unsatisfactory but actually valid samples, thus achieving a balance between automation efficiency and mining recall.

[0031] In one implementation, the large model performs scene mining on the current enhanced labeled image based on data mining prompts to obtain scene mining results, specifically including: The large model determines the upcoming driving scenario based on the vehicle's future trajectory in the current enhanced and annotated image, as well as data mining prompts. Based on the determined driving scenario, the scenario mining results are obtained, which include scenario type, scenario risk level, key element judgment, and scenario confidence level.

[0032] In this embodiment, after receiving an enhanced labeled image showing the vehicle's future driving trajectory, the large model uses specially designed data mining prompts (which explicitly inform the model that the lines in the image represent the future trajectory and instruct it to determine the upcoming driving scenario) to comprehensively reason about the road structure, obstacle distribution, and the vehicle's planned path in the current image. This allows it to predict the type of driving scenario that will occur (such as turning, changing lanes, passing through a construction zone, avoiding obstacles, sudden stops, or driving against traffic). Based on this, the large model further outputs structured scenario mining results, including scenario type, scenario risk level, key element judgments (such as whether the trajectory is reasonable and whether the obstacle position is correct), and scenario confidence level. Output results with a confidence level below a preset threshold are marked and submitted for manual review to improve the accuracy of the mining.

[0033] In one implementation, the data storage based on the scene mining results specifically includes: Based on scene confidence scores from scene mining results: If the scene confidence level is lower than the preset threshold, the scene mining results will be actively reviewed and then stored in the database. If the scene confidence level is not lower than the preset threshold, the scene mining results will be directly stored in the database.

[0034] In this embodiment, after the large model completes scene mining and outputs results including scene confidence levels, the system performs triage processing on the mining results according to a preset confidence threshold: if the confidence level of the current scene mining result is lower than the threshold, it indicates that the large model is not confident enough in its judgment, and it will not be used directly. Instead, it will be sent to the active review process, and only after the review confirms its validity will it be stored in the database. If the confidence level is not lower than the preset threshold, it indicates that the large model's judgment is relatively reliable, and scene mining can proceed directly without manual intervention. This hierarchical storage strategy ensures that high-confidence scene data can be automatically archived efficiently and at low cost, while manual review provides a safety net for low-confidence samples, effectively preventing erroneous or invalid data from polluting the dataset.

[0035] In one implementation, after the scene mining results are stored in the database, the method further includes: Based on the selected time length of the vehicle's future driving trajectory, the autonomous driving images after the enhanced and annotated images for a set time are labeled with the same tags as the enhanced and annotated images, and the labeled data is categorized and stored.

[0036] In this embodiment, after the scene mining results are successfully stored in the database, the system further automatically labels all autonomous driving image frames within the corresponding number of seconds after the current enhanced and labeled image with the same scene tags (such as "turning" or "lane change") as the current enhanced and labeled image, based on the pre-selected time length of the vehicle's future driving trajectory (i.e., the duration of the trajectory extension coverage). This achieves the propagation of tags from single-frame mining results to the entire continuous video segment. After labeling, these data are uniformly classified and stored, expanding the scene information that was originally only valid for a single frame to a complete time series segment. This provides richer time series samples for subsequent model training and also realizes a closed loop of automated mining and cleaning of large-scale data.

[0037] Secondly, see Figure 2 This application provides an autonomous driving scene mining device based on a large model, the device comprising: The data preprocessing module is used to acquire autonomous driving images and perform frame extraction processing on the acquired autonomous driving videos to obtain single-frame images. The single-frame images are then labeled with tags to generate enhanced labeled images. The mining module is used to input the enhanced labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the enhanced labeled image and output the scene mining results. The data storage module is used to store data based on the results of scene mining.

[0038] The functions of each module in the above-mentioned large-model-based autonomous driving scene mining device correspond to the steps in the above-mentioned large-model-based autonomous driving scene mining method embodiment, and their functions and implementation processes will not be described in detail here.

[0039] Thirdly, embodiments of this application provide an autonomous driving scene mining device based on a large model. The autonomous driving scene mining device based on a large model can be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0040] Reference Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of the autonomous driving scene mining device based on a large model involved in the embodiments of this application. In the embodiments of this application, the autonomous driving scene mining device based on a large model may include a processor, a memory, a communication interface, and a communication bus.

[0041] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0042] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the large-model-based autonomous driving scene mining equipment, as well as interfaces used for interconnecting the large-model-based autonomous driving scene mining equipment with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can be displays, keyboards, etc.

[0043] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0044] The processor can be a general-purpose processor, which can call the autonomous driving scene mining program based on a large model stored in memory and execute the autonomous driving scene mining method based on a large model provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the autonomous driving scene mining program based on a large model is called can be referred to the various embodiments of the autonomous driving scene mining method based on a large model of this application, and will not be repeated here.

[0045] Those skilled in the art will understand that Figure 3 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0046] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0047] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0048] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0049] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0050] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0051] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for mining autonomous driving scenarios based on large models, characterized in that, The method includes: The system acquires autonomous driving images and performs frame extraction on the acquired autonomous driving videos to obtain single-frame images. A single frame image is labeled with tags to generate an enhanced labeled image. The tags include lane lines, road edges, guide arrows, 3D boxes of obstacles, and the vehicle's future driving trajectory. The vehicle's future driving trajectory is used to characterize the vehicle's predicted driving path in subsequent time periods. Input the augmented labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the augmented labeled image, and output the scene mining results; Data is entered into the database based on the results of scene mining.

2. The autonomous driving scene mining method based on a large model as described in claim 1, characterized in that, The frame extraction process for the collected autonomous driving video specifically includes: The frame extraction frequency is determined based on the vehicle's speed, and the autonomous driving images are then extracted at equal intervals based on the determined frame extraction frequency.

3. The autonomous driving scene mining method based on a large model as described in claim 1, characterized in that, The data verification prompts include descriptions of qualified samples and verification standards; The description of qualified samples includes clear annotations, reasonable trajectories, accurate 3D bounding boxes, and no obstructions or missing annotations. The verification criteria include whether the labeling is complete, whether the location is accurate, whether the trajectory is reasonable, and whether the scenario is valid.

4. The autonomous driving scene mining method based on a large model as described in claim 1, characterized in that, The input augmented and annotated image, along with data verification prompts and data mining prompts, is fed into a large model to perform data verification and scene mining on the augmented and annotated image, outputting scene mining results, specifically including: Input the enhanced labeled images, data verification tips, and data mining tips into the large model; The large model performs data verification on the current enhanced labeled image based on data verification prompts and outputs the data verification results: If the data verification result is qualified, the large model will perform scene mining on the current enhanced labeled image based on the data mining prompt words to obtain the scene mining result; If the data verification result is unqualified, the current enhanced labeled image is actively filtered. If the active filtering result is qualified, the large model performs scene mining on the current enhanced labeled image based on data mining prompts to obtain scene mining results. If the active filtering result is unqualified, the current enhanced labeled image is discarded.

5. The autonomous driving scene mining method based on a large model as described in claim 4, characterized in that, The large model performs scene mining on the current enhanced labeled image based on data mining prompts, and obtains scene mining results, specifically including: The large model determines the upcoming driving scenario based on the vehicle's future trajectory in the current enhanced and annotated image, as well as data mining prompts. Based on the determined driving scenario, the scenario mining results are obtained, which include scenario type, scenario risk level, key element judgment, and scenario confidence level.

6. The autonomous driving scene mining method based on a large model as described in claim 5, characterized in that: The types of scenes include turning, changing lanes, construction, lane obstruction, sudden appearance of pedestrians, emergency braking, and driving in the wrong direction; The key elements to be judged include whether the trajectory is reasonable and whether the obstacles are correct.

7. The autonomous driving scene mining method based on a large model as described in claim 6, characterized in that, The data entry based on the scene mining results specifically includes: Based on scene confidence scores from scene mining results: If the scene confidence level is lower than the preset threshold, the scene mining results will be actively reviewed and then stored in the database. If the scene confidence level is not lower than the preset threshold, the scene mining results will be directly stored in the database.

8. The autonomous driving scene mining method based on a large model as described in claim 7, characterized in that, After the scene mining results are entered into the database, the following are also included: Based on the selected time length of the vehicle's future driving trajectory, the autonomous driving images after the enhanced and annotated images for a set time are labeled with the same tags as the enhanced and annotated images, and the labeled data is categorized and stored.

9. An autonomous driving scene mining device based on a large model, characterized in that, The device includes: The data preprocessing module is used to acquire autonomous driving images and perform frame extraction processing on the acquired autonomous driving videos to obtain single-frame images. The single-frame images are then labeled with tags to generate enhanced labeled images. The mining module is used to input the enhanced labeled image, data verification prompts, and data mining prompts into the large model to perform data verification and scene mining on the enhanced labeled image and output the scene mining results. The data storage module is used to store data based on the results of scene mining.

10. An autonomous driving scene mining device based on a large model, characterized in that, The large-model-based autonomous driving scene mining device includes a processor, a memory, and a large-model-based autonomous driving scene mining program stored in the memory and executable by the processor, wherein when the large-model-based autonomous driving scene mining program is executed by the processor, it implements the steps of the large-model-based autonomous driving scene mining method as described in any one of claims 1 to 8.

Images