Method and device for generating driving sample data, and vehicle

Abstract

This application provides a method, apparatus, and vehicle for generating driving sample data. The method includes: acquiring multi-source data, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle state data representing the vehicle state; performing task detection based on the image sequence data to obtain task perception information, including at least one of target detection and tracking results, lane line detection results, and drivable area detection results; determining environmental assessment information based on the environmental perception data; determining driving event information representing driving behavior based on the vehicle state data, task perception information, and environmental assessment information; and associating the task perception information, environmental assessment information, driving event information, and multi-source data to generate driving sample data for autonomous driving learning. This allows for the timely capture of truly valuable, high-quality demonstration segments.

Description

Technical Field

[0001] This application relates to the field of driving perception, and more particularly to a method, apparatus, and vehicle for generating driving sample data. Background Technology

[0002] With the rapid development of internet and artificial intelligence technologies, autonomous driving is rapidly evolving from assisted driving to advanced autonomous driving. Constructing high-value, high-quality, and interpretable driving sample datasets is a crucial foundation for training multimodal autonomous driving models (including vision, language, and action models) and achieving safe, comfortable, and human-logical autonomous driving decisions.

[0003] In related technologies, existing methods for generating driving sample data mainly rely on mass-produced fleets or open-source autonomous driving datasets to collect data over long periods of time and continuously. Representative segments are then selected through preset rules (such as whether acceleration exceeds a threshold) or manual screening to generate driving sample data. However, this method, which relies solely on simple rules or manual experience for screening, struggles to systematically identify and accumulate high-quality driving sample data. Consequently, multimodal autonomous driving models can only learn average or even low-quality driving behaviors, limiting the performance ceiling and practical application of autonomous driving models. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, and vehicle for generating driving sample data, in order to solve the technical problem that existing technologies rely solely on simple rules or human experience for screening, making it difficult to systematically identify and accumulate high-quality driving sample data. This results in multimodal autonomous driving models only learning average or even low-quality driving behaviors, thus limiting the performance ceiling and practical application effectiveness of autonomous driving models. The specific technical solution is as follows: In a first aspect of this application, a method for generating driving sample data is provided, the method comprising: Acquire multi-source data, including environmental perception data characterizing the external climate environment, image sequence data characterizing the external traffic environment, and vehicle state data characterizing the vehicle state. Task detection is performed based on the image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results. Based on the environmental perception data, environmental assessment information is determined; Based on the vehicle status data, the task perception information, and the environmental assessment information, driving event information characterizing driving behavior is determined; The task perception information, the environmental assessment information, the driving event information, and the multi-source data are correlated to generate driving sample data for autonomous driving learning.

[0005] In an optional implementation, task detection is performed based on the image sequence data to obtain task-aware information, including: The image sequence data is subjected to image quality detection to obtain the image quality detection result; Scene detection is performed on the image sequence data to obtain road condition perception results; The image quality detection results and the road condition perception results are integrated to obtain the task perception information.

[0006] In an optional implementation, determining environmental assessment information based on the environmental perception data includes: Based on the environmental sensing data, at least one environmental feature is obtained that characterizes the climate level, the amount of climate change, or the climate risk level. The environmental assessment information is determined based on at least one of the environmental characteristics.

[0007] In an optional implementation, based on the vehicle state data, the task perception information, and the environmental assessment information, driving event information characterizing driving behavior is determined, including: Based on the task perception information and the environmental assessment information, the driving scenario is determined; The driving scenario is evaluated based on the vehicle status data to determine the driving quality assessment result; The driving scenario and the driving quality assessment results are integrated to obtain the driving event information that characterizes driving behavior.

[0008] In an optional implementation, the step of evaluating the driving scenario based on the vehicle state data to determine the driving quality assessment result includes: Determine the vehicle type; Obtain the driving evaluation parameter set corresponding to the vehicle type, wherein the driving evaluation parameter set includes at least one of a safe distance threshold, a comfort acceleration threshold, and an efficiency evaluation benchmark; Based on the driving evaluation parameter set and the vehicle status data, the driving scenario is evaluated to determine the driving quality evaluation result.

[0009] In an optional implementation, the task perception information includes dirt detection results. Before correlating the multi-source data based on the task perception information, the environmental assessment information, and the driving event information to generate driving sample data for autonomous driving learning, the method further includes: An image quality score is determined based on the dirt detection results; If the image quality score is greater than a preset quality threshold, a trajectory quality score is determined based on the task perception information and the environmental assessment information, and a data quality level is determined based on the trajectory quality score. Based on the task perception information, the environmental assessment information, and the driving event information, the multi-source data is correlated to generate driving sample data for autonomous driving learning, including: The multi-source data, the data quality level, the task perception information, the environmental assessment information, and the driving event information are correlated to generate the driving sample data used for autonomous driving learning.

[0010] In an optional implementation, after associating the multi-source data, the data quality level, the task perception information, the environmental assessment information, and the driving event information to generate the driving sample data for autonomous driving learning, the method further includes: The driving sample data is anonymized to obtain anonymized driving sample data; Based on the data quality level, determine the upload priority of the anonymized driving sample data; According to the upload priority, the de-identified driving sample data is uploaded to the preset storage area.

[0011] In an optional implementation, the method further includes: At least one obstruction is identified based on the task perception information; For any of the aforementioned occlusions, based on the image quality detection results in the task perception information, the type and degree of occlusion corresponding to the occlusion are identified; Obtain the occlusion threshold corresponding to the type of occlusion; If the degree of occlusion exceeds the occlusion threshold corresponding to the type of occlusion, the image quality score in the image quality detection result is used to determine whether to trigger a cleaning reminder signal; When the cleaning reminder signal is triggered, a prompt message is output through the vehicle's human-machine interface to remind the driver to clean the camera or windshield.

[0012] In a second aspect of this application, an apparatus for generating driving sample data is also provided, the apparatus comprising: The data acquisition module is used to acquire multi-source data, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle status data representing the vehicle status. The task detection module is used to perform task detection based on the image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results. An environmental assessment module is used to determine environmental assessment information based on the environmental sensing data. The driving event information determination module is used to determine driving event information characterizing driving behavior based on the vehicle status data, the task perception information, and the environmental assessment information. The driving sample data generation module is used to associate the task perception information, the environmental assessment information, the driving event information, and the multi-source data to generate driving sample data for autonomous driving learning.

[0013] In a third aspect of the embodiments of this application, a vehicle is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method for generating driving sample data as described in any of the first aspects above.

[0014] In a fourth aspect of the embodiments of this application, a storage medium is also provided, the storage medium storing instructions that, when run on a computer, cause the computer to execute the method for generating driving sample data as described in any of the first aspects above.

[0015] In a fifth aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the method for generating driving sample data as described in any of the first aspects above.

[0016] The technical solution provided in this application involves acquiring multi-source data, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle state data representing the vehicle state; performing task detection based on the image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results; determining environmental assessment information based on the environmental perception data; determining driving event information representing driving behavior based on the vehicle state data, task perception information, and environmental assessment information; and associating the task perception information, environmental assessment information, driving event information, and multi-source data to generate driving sample data for autonomous driving learning. This technical solution acquires environmental perception data, image sequence data, and vehicle status data to perform image task detection, environmental assessment, and driving event recognition. It then correlates multi-source data based on task perception information, environmental assessment information, and driving event information to generate high-quality driving sample data. This eliminates the need for offline cloud-based filtering or manual annotation; it utilizes only the existing sensors and computing resources in mass-produced vehicles to achieve efficient data acquisition, significantly reducing bandwidth costs and the burden of manual annotation. Simultaneously, it avoids redundant acquisition of numerous events without events, low information density, or poor image quality, saving on-vehicle storage and upload bandwidth while promptly capturing truly valuable, high-quality demonstration segments. Furthermore, by fusing information from three different modalities—image perception results, environmental assessment information, and vehicle status data—to determine driving events and generate driving sample data, it offers higher real-time performance and accuracy compared to single-signal or offline post-processing methods. This significantly improves acquisition efficiency and reduces system costs while ensuring the quality of driving sample data. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1A schematic diagram illustrating the implementation process of a method for generating driving sample data provided in this application embodiment; Figure 2 A schematic diagram illustrating the implementation process of a method for determining task-aware information provided in this application embodiment; Figure 3 A schematic diagram illustrating the implementation process of a method for determining driving event information provided in this application embodiment; Figure 4 A schematic diagram illustrating the implementation process of another method for generating driving sample data provided in this application embodiment; Figure 5 A schematic diagram of a device for generating driving sample data provided in an embodiment of this application; Figure 6 This is a structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0023] The method for generating driving sample data provided in this application will be further explained below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this invention.

[0024] To address the technical problem that existing technologies rely solely on simple rules or human experience for screening, making it difficult to systematically identify and accumulate high-quality driving sample data. This results in multimodal autonomous driving models only learning average or even low-quality driving behaviors, thus limiting the performance ceiling and practical application of autonomous driving models. This application provides a method for generating driving sample data. The method involves acquiring multi-source data, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle state data representing the vehicle state. Task detection is performed based on the image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results. Environmental assessment information is determined based on the environmental perception data. Driving event information representing driving behavior is determined based on the vehicle state data, task perception information, and environmental assessment information. Finally, the task perception information, environmental assessment information, driving event information, and multi-source data are correlated to generate driving sample data for autonomous driving learning.

[0025] This technical solution acquires environmental perception data, image sequence data, and vehicle status data to perform image task detection, environmental assessment, and driving event recognition. It then correlates multi-source data based on task perception information, environmental assessment information, and driving event information to generate high-quality driving sample data. This eliminates the need for offline cloud-based filtering or manual annotation; it utilizes only the existing sensors and computing resources in mass-produced vehicles to achieve efficient data acquisition, significantly reducing bandwidth costs and the burden of manual annotation. Simultaneously, it avoids redundant acquisition of numerous events without events, low information density, or poor image quality, saving on-vehicle storage and upload bandwidth while promptly capturing truly valuable, high-quality demonstration segments. Furthermore, by fusing information from three different modalities—image perception results, environmental assessment information, and vehicle status data—to determine driving events and generate driving sample data, it offers higher real-time performance and accuracy compared to single-signal or offline post-processing methods. This significantly improves acquisition efficiency and reduces system costs while ensuring the quality of driving sample data.

[0026] like Figure 1 The diagram shown illustrates the implementation flow of a method for generating driving sample data according to an embodiment of this application, which may specifically include the following steps: S101, acquire multi-source data, which includes environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle state data representing the vehicle state.

[0027] The aforementioned environmental perception data refers to information characterizing the external climate environment collected by environmental sensors installed on the vehicle (such as rain sensors, light sensors, temperature sensors, humidity sensors, etc.). This data is used to assist in judging weather, lighting conditions, etc., and to provide semantic background for subsequent environmental assessments and analysis of the rationality of driving behavior. For example, a rain sensor outputs rainfall intensity from 0 to 100%, a light sensor outputs illuminance values ​​from 0 to 100,000 Lux, and a temperature sensor outputs temperature values ​​from -40 to 80°C.

[0028] The aforementioned image sequence data refers to a sequence of video frames continuously captured by an onboard camera (e.g., a forward-facing camera). It is used to characterize the external traffic environment (such as the number of vehicles), provide visual perception input, and detect lane lines, targets (vehicles, pedestrians), drivable areas, dirt, and obstructions.

[0029] The aforementioned vehicle status data refers to vehicle dynamics and control status signals acquired via the CAN (Controller Area Network) bus. These signals characterize the vehicle's state and may include vehicle speed, longitudinal acceleration, lateral acceleration, steering wheel angle, accelerator / brake pedal opening, ESP status, wheel speed, etc. They reflect the driver's intentions and the vehicle's actual motion response. For example, vehicle status data might include vehicle speed (e.g., 120 km / h), steering wheel angle (e.g., 120°), and pedal opening (e.g., 50%).

[0030] In this embodiment, the vehicle-side domain controller / high-performance electronic control unit can use the CAN bus and Ethernet interface to acquire multi-source data in parallel with a unified timestamp. The multi-source data includes environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle status data representing the vehicle status.

[0031] As an optional implementation, to reduce the pressure on vehicle-side storage and communication bandwidth while ensuring data integrity, multi-source data can be acquired based on different sampling frequencies. Specifically, for environmental quantities that change slowly (such as temperature and humidity) in environmental perception data, low-frequency sampling (e.g., 1Hz) is used; for environmental quantities that change rapidly (such as rainfall and illumination), medium-frequency sampling (e.g., 10Hz) is used, and the sampling frequency is dynamically adjusted according to the rate of change. For image sequence data, the frame rate can be dynamically adjusted according to the complexity of the vehicle driving scene. For example, when driving straight at high speed and without events, it can be reduced to 10-15fps; when potential critical events are detected (such as navigation prompts for upcoming ramps or fluctuations in target detection confidence), it can be automatically increased to 30fps or higher. For critical signals in vehicle status data (such as steering wheel angle and pedal opening), high-frequency sampling (50-100Hz) is maintained, while non-critical signals (such as average wheel speed) can be reduced to 10Hz.

[0032] S102, Perform task detection based on image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results.

[0033] The aforementioned task detection refers to the process of performing multiple parallel computer vision tasks on each or multiple frames of an image sequence to extract structured environmental semantic information from the original image sequence data. These task detections may include, but are not limited to, dirt detection, object detection and tracking, lane detection, and drivable area detection.

[0034] The aforementioned task-aware information refers to the structured semantic information output after performing multiple parallel perception tasks on image sequence data. This information can include at least one of the following: target detection and tracking results (vehicles, pedestrians, non-motorized vehicles, etc.), lane detection results, and drivable area detection results. For example, target detection and tracking results: There is a car 30 meters ahead, traveling at approximately 40 km / h. Lane detection results: The left lane line is a white dashed line with a curvature of 0.002; the right lane line is a white solid line. Drivable area detection results: The road surface within 50 meters ahead is a drivable area; there are construction cones marking it as non-drivable to the right.

[0035] In this embodiment of the application, task detection can be performed based on image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results.

[0036] The specific methods for performing task detection and obtaining task-aware information based on image sequence data will be explained below. Figure 2 The process shown will be explained in detail here.

[0037] S103, Based on environmental perception data, determine environmental assessment information.

[0038] The aforementioned environmental assessment information refers to a qualitative or quantitative description reflecting the current external environmental state of the vehicle. This may include weather type (e.g., sunny, rainy, snowy, foggy), lighting conditions (e.g., daytime, nighttime, tunnel, backlighting, strong sunlight), and road surface conditions (e.g., dry, slippery, flooded, icy). It is used to assist in the rationality analysis of driving behavior and to provide environmental context for the identification of driving events.

[0039] In the embodiments of this application, environmental assessment information can be determined based on environmental perception data.

[0040] S104, based on vehicle status data, task perception information and environmental assessment information, determines driving event information that characterizes driving behavior.

[0041] The aforementioned driving event information refers to a structured description of the maneuvering behavior performed by the driver within a specific time window, which has certain semantic meaning and is used to characterize driving behavior, such as changing lanes to the left, slowing down while following another vehicle, and turning right after yielding to pedestrians.

[0042] In this embodiment of the application, driving event information can be determined based on vehicle status data, task perception information, and environmental assessment information.

[0043] As for how driving event information is determined based on vehicle status data, task perception information, and environmental assessment information, it will be explained in the following text. Figure 3 The process shown will be explained in detail here.

[0044] S105 correlates task perception information, environmental assessment information, driving event information, and multi-source data to generate driving sample data for autonomous driving learning.

[0045] The aforementioned autonomous driving learning refers to the process of acquiring knowledge and refining behavioral patterns with the goal of improving the autonomous driving capabilities of vehicles in various traffic environments. This process includes, but is not limited to: optimizing the perception capabilities of autonomous driving systems, learning decision-making and planning strategies, imitating control behaviors, understanding driving styles, transferring end-to-end driving skills, and adaptively improving driving behavior for specific scenarios (such as complex ramps, inclement weather, and urban congestion).

[0046] In this embodiment, task perception information, environmental assessment information, driving event information, and multi-source data can be correlated to generate driving sample data for autonomous driving learning. Here, driving sample data refers to structured multimodal samples that can be directly used for autonomous driving learning.

[0047] Task detection can also include dirt detection, with the corresponding detection result being the dirt detection result. Therefore, task perception information can include the dirt detection result. For example, the dirt detection result might be that there are raindrops obstructing 8% of the area of ​​the front of the windshield, the type is "raindrops," and the severity is "medium." Before executing step S105, the data quality level can be determined based on the dirt detection result. Based on the data quality level, task perception information, environmental assessment information, driving event information, and multi-source data can be correlated to generate driving sample data for autonomous driving learning. The specific method of correlating task perception information, environmental assessment information, driving event information, and multi-source data based on the dirt detection result to generate driving sample data for autonomous driving learning will be explained below. Figure 4 The process shown will be explained in detail here.

[0048] Based on the above description of the technical solution provided in the embodiments of this application, multi-source data is acquired, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle state data representing the vehicle state; task detection is performed based on the image sequence data to obtain task perception information, which includes at least one of target detection and tracking results, lane line detection results, and drivable area detection results; environmental assessment information is determined based on the environmental perception data; driving event information representing driving behavior is determined based on the vehicle state data, task perception information, and environmental assessment information; and the task perception information, environmental assessment information, driving event information, and multi-source data are correlated to generate driving sample data for autonomous driving learning.

[0049] This technical solution acquires environmental perception data, image sequence data, and vehicle status data to perform image task detection, environmental assessment, and driving event recognition. It then correlates multi-source data based on task perception information, environmental assessment information, and driving event information to generate high-quality driving sample data. This eliminates the need for offline cloud-based filtering or manual annotation; it utilizes only the existing sensors and computing resources in mass-produced vehicles to achieve efficient data acquisition, significantly reducing bandwidth costs and the burden of manual annotation. Simultaneously, it avoids redundant acquisition of numerous events without events, low information density, or poor image quality, saving on-vehicle storage and upload bandwidth while promptly capturing truly valuable, high-quality demonstration segments. Furthermore, by fusing information from three different modalities—image perception results, environmental assessment information, and vehicle status data—to determine driving events and generate driving sample data, it offers higher real-time performance and accuracy compared to single-signal or offline post-processing methods. This significantly improves acquisition efficiency and reduces system costs while ensuring the quality of driving sample data.

[0050] As an optional embodiment, for Figure 1 In step S103, based on environmental perception data, environmental assessment information is determined, which may specifically include the following steps: Step 11: Based on environmental perception data, obtain at least one environmental feature that characterizes the climate level, the amount of climate change, or the climate risk level.

[0051] The aforementioned climate levels refer to the results of qualitative or quantitative grading of environmental sensing data according to preset standards, used to describe the static category and intensity of the current environment. For example, based on rainfall sensor data in the environmental sensing data, rainfall intensity is divided into levels such as "no rain, light rain, moderate rain, heavy rain, and rainstorm"; based on light sensor data, ambient brightness is divided into levels such as "strong light during the day, normal during the day, dusk / dawn, streetlights at night, and no streetlights at night"; and based on temperature and humidity data, road surface conditions are divided into levels such as "dry, wet, waterlogged, snow-covered, and icy".

[0052] The aforementioned climate change metrics refer to the magnitude or rate of change in environmental sensing data within a certain time window. Examples include changes in rain sensor readings over the past minute (sudden increases or decreases in rainfall); abrupt changes in light intensity before and after entering a tunnel (the magnitude of the change from strong to weak light); and the rate of temperature decrease (used to determine the likelihood of road icing). Climate change metrics are used to capture dynamic environmental changes and help identify driving risks or adaptive behaviors caused by sudden environmental changes.

[0053] The aforementioned climate risk levels refer to a classification of the degree of risk that may affect driving safety or driving behavior quality, based on a comprehensive assessment of environmental perception data. For example, "heavy rain + low visibility + slippery road surface" is rated as "high risk"; "light snow + no snow accumulation on the road surface" is rated as "low risk"; and "strong sunlight and backlight + entering and exiting tunnels" is rated as "medium risk". Climate risk levels can be directly used to adjust the strictness of driving quality evaluation (such as giving higher weight to safe distance scores in high-risk environments) or to determine whether to trigger critical event recording.

[0054] In this embodiment of the application, at least one environmental feature characterizing the climate level, the amount of climate change, or the climate risk level can be obtained based on environmental sensing data. Specifically, the environmental sensing data can be parsed to obtain environmental information, which includes at least one of rainfall information, light intensity information, temperature information, and humidity information. Feature extraction is performed on the environmental information to obtain at least one environmental feature characterizing the climate level, the amount of climate change, or the climate risk level.

[0055] The aforementioned environmental characteristics refer to feature values ​​with stronger discriminative power or physical significance obtained from environmental information (including rainfall information, illumination information, temperature information, and humidity information) through mathematical transformation, time series analysis, or combination. Examples include rainfall change rate, illumination change trend, icing risk index, backlight intensity, and fog visibility estimation.

[0056] As an optional implementation, the original sequence corresponding to rainfall information can be differentially calculated to obtain the rainfall change rate as an environmental feature. This feature is used to determine whether the rainfall suddenly intensifies or gradually weakens, thereby distinguishing between two different risk levels: sudden heavy rain and continuous light rain.

[0057] The rainfall information mentioned above refers to the intensity of rainfall detected by rain sensors (such as infrared or optical sensors) on the outer surface of the windshield or the environment around the vehicle. This information is used to determine whether it is raining and the severity of the rainfall, providing a basis for subsequent environmental assessments (such as rain scene recognition), image quality assessments (the impact of raindrop obstruction), and analysis of the rationality of driving behavior (such as whether the following distance is appropriate in rainy weather). For example, rainfall information indicates 65% or moderate rain (5~10 mm / h).

[0058] The aforementioned light intensity information refers to the light intensity value of the environment surrounding the vehicle detected by a light sensor (such as a photoresistor or photodiode). It is used to determine whether the current lighting conditions are daytime, nighttime, dawn / dusk, inside a tunnel, or backlighting. For example, the light intensity information is 120 Lux or 5000 Lux.

[0059] The temperature information mentioned above refers to the external ambient temperature value of the vehicle detected by temperature sensors (such as thermistors or semiconductor sensors). It is used to determine whether there are extreme weather conditions such as icing, snow accumulation, or high temperatures, to help identify environmental conditions such as snowy weather and the risk of road icing, and to provide a reference for the rationality analysis of driving behavior (such as whether to slow down in advance on icy or snowy roads). For example, the temperature information is 3℃ or -5℃.

[0060] The humidity information mentioned above refers to the relative humidity of the vehicle's external environment detected by humidity sensors (such as capacitive or resistive sensors). It is used to assist in judging weather phenomena such as rain, fog, and snow. Especially when the temperature is close to 0°C, high humidity (>80%) indicates a potential risk of icy or slippery road surfaces, providing a basis for environmental assessment and driving safety evaluation. For example, humidity information of 85% or 95%.

[0061] Step 12: Determine environmental assessment information based on at least one environmental characteristic.

[0062] In the embodiments of this application, environmental assessment information can be determined based on at least one environmental characteristic.

[0063] Specifically, a combination of rule engines and feature fusion can be used to combine the extracted environmental features (such as rainfall change rate, light change trend, icing risk index, backlight intensity, fog visibility estimation, etc.) into an environmental feature vector, which is then input into an environmental classifier to output structured environmental assessment information.

[0064] In one alternative embodiment, the current weather type can be determined based on the rainfall change rate, icing risk index, and rainfall-related features in the environmental feature vector.

[0065] For example, if the "Rainfall Change Rate" is greater than 0.2 (indicating that rainfall is continuously increasing) and the current rainfall characteristic value (such as normalized rainfall intensity) exceeds 0.3, while the "Icing Risk Index" is less than 0.2 (indicating no icing risk), it is judged as a "rainy day," and further subdivided into "light rain," "moderate rain," and "heavy rain" based on the rainfall characteristic value. If the "Icing Risk Index" is higher than 0.7 (obtained by fusing temperature and humidity characteristics), and the rainfall change rate is at a low level, it is judged as a "snowy day or high risk of road icing."

[0066] In another alternative embodiment, the lighting environment can be determined based on the lighting change trend (such as the first difference of lighting intensity), backlight intensity (calculated from the angle between the lighting direction and the vehicle's orientation, the absolute value of lighting, etc.) in the environmental feature vector, and the prior information of the tunnel entrance provided by the navigation system.

[0067] For example, if the "lighting change trend" is negative for 5 consecutive frames and the absolute value is large, combined with the current "backlight intensity" being below the threshold and the navigation information indicating that we are about to enter a tunnel, then it is determined to be "inside a tunnel". If the "backlight intensity" is higher than 0.8 (indicating strong backlight) and the absolute value of the light is greater than 70,000 Lux, then it is determined to be "strong backlight". If the "lighting change trend" is stable and the absolute value of the light is lower than 50 Lux, then it is determined to be "night".

[0068] like Figure 2 The diagram shown is a schematic representation of the implementation flow of a method for determining task-aware information provided in an embodiment of this application. Figure 2 exist Figure 1 Building upon this foundation, the paper details how to perform task detection based on image sequence data to obtain task-aware information, which may include the following: S201, Perform image quality detection on the image sequence data to obtain the image quality detection result.

[0069] The aforementioned image quality detection refers to the quality analysis and evaluation of image sequence data to determine whether the acquired multi-source data is authentic.

[0070] The above image quality detection results are used to characterize the quality of the current image sequence data.

[0071] In this embodiment of the application, image quality detection is performed on the image sequence data to obtain the image quality detection result.

[0072] Specifically, sharpness (Laplacian variance), contrast (root mean square contrast), noise level (wavelet transform estimation), exposure (histogram analysis), and motion blur (gradient magnitude distribution) can be calculated in parallel for each frame of the image sequence data. It can also determine whether there are obstructions such as raindrops, mud spots, and insect corpses and their percentage coverage area. The scores of the above dimensions are then weighted and fused (e.g., sharpness weight 0.3, obstruction weight 0.3, exposure weight 0.2, noise weight 0.1, and blur weight 0.1) to obtain the image quality detection result.

[0073] S202, Scene detection is performed on the image sequence data to obtain road condition perception results.

[0074] The aforementioned scene detection refers to the parallel execution of multiple visual perception tasks based on image sequence data, which may include target detection and tracking, lane line detection, drivable area detection, etc.

[0075] The aforementioned road condition perception results refer to the structured information output by scene detection, which may include target information: the category, 2D / 3D position, speed, and direction of movement of vehicles, pedestrians, and non-motorized vehicles ahead and around. Lane line information: lane line type (solid line / dashed line / double yellow line), geometric parameters (curvature, width), and number of lanes. Driving area: pixel-level semantic segmentation results of the safe driving area, as well as annotations of non-drivable areas such as obstacles and construction zones.

[0076] In this embodiment of the application, scene detection is performed on image sequence data to obtain road condition perception results.

[0077] Specifically, image sequence data can be used as input, and tasks such as target detection and tracking, lane line detection, and drivable area detection can be performed in parallel on the image processor / neural network processing unit of the vehicle domain controller, outputting a structured road condition perception result containing target information, lane line information, and drivable area information.

[0078] S203 integrates the image quality detection results and road condition perception results to obtain task perception information.

[0079] The aforementioned task-aware information refers to the unified structured output formed by aligning image quality detection results with road condition perception results by timestamp. It includes both the quality level and quality score of the current frame image, as well as all semantic information detected in that frame image (targets, lane lines, drivable areas, etc.).

[0080] In this embodiment of the application, image quality detection results and road condition perception results can be integrated to obtain task perception information.

[0081] like Figure 3The diagram shown is a schematic representation of the implementation process of a method for determining driving event information provided in an embodiment of this application. Figure 3 exist Figure 1 Building upon this foundation, the paper details how to determine driving event information based on vehicle status data, task perception information, and environmental assessment information. Specifically, this may include the following: S301 determines the driving scenario based on task perception information and environmental assessment information.

[0082] The aforementioned driving scenarios refer to the comprehensive driving situations in which the vehicle is currently located, such as dense traffic scenarios, complex ramp scenarios, construction detour scenarios, and low visibility scenarios (rain / snow / fog / nighttime), etc.

[0083] In this embodiment of the application, the driving scenario can be determined based on task perception information and environmental assessment information.

[0084] As an optional implementation, a set of typical multimodal feature templates can be predefined for each driving scenario requiring identification (such as dense traffic, complex ramps, construction detours, and low visibility). Each template contains the feature range of each task perception information and environmental assessment information in that scenario. For example, for the "dense traffic" scenario, the template can be defined as: the number of detected objects > 5. For the "low visibility" scenario, the template can be defined as: the "weather" in the environmental assessment information is "rainy" or "foggy," and the "clarity" score in the image quality assessment is lower than a first threshold, or the "contrast" score is lower than a second threshold. The task perception information (such as the number of vehicles ahead, average distance between vehicles, and lane curvature) and environmental assessment information (such as light intensity and rainfall) aligned at the current time are numerically processed to form a multi-dimensional feature vector. For example, feature vector = [number of vehicles ahead, average distance between vehicles, lane curvature, light intensity, rainfall]. The currently generated feature vector is matched with each template in the feature template library. The matching algorithm can calculate the Mahalanobis distance between the feature vector and the central feature vector of the template, or determine whether each dimension of the vector falls within the threshold range defined by the template. The driving scenario corresponding to the template with the highest matching score is selected as the recognition result at the current moment. If the highest score is lower than the preset confidence threshold, it can be determined as a "normal driving scenario" or an "unrecognized scenario".

[0085] S302 assesses driving scenarios based on vehicle status data to determine driving quality assessment results.

[0086] In this embodiment of the application, the driving scenario can be evaluated based on vehicle status data to determine the driving quality assessment result. The driving quality assessment result refers to the comprehensive result obtained after multi-dimensional quantitative evaluation of the current driving behavior (i.e., the driving scenario determined in step S301 above), including scores for dimensions such as safety, comfort, legality, and efficiency, as well as comprehensive quality labels (such as "high-quality skilled demonstration", "general demonstration", "counterexample", etc.).

[0087] S303 integrates driving scenarios and driving quality assessment results to obtain driving event information that characterizes driving behavior.

[0088] In this embodiment of the application, driving scenarios and driving quality assessment results are integrated to obtain driving event information characterizing driving behavior, wherein the driving event information is used to indicate the degree of standardization of driving behavior in the corresponding driving scenario.

[0089] Specifically, driving scenarios and driving quality assessment results can be integrated through structured splicing and semantic tag mapping to obtain driving event information that represents driving behavior.

[0090] For example, when the driving scenario is a left turn at an urban intersection, and the driving quality assessment results are: Safety 95 points, Comfort 92 points, Efficiency 90 points, and the overall quality label "High-quality and Skilled Demonstration," the integrated driving event information is: Left Turn at Urban Intersection - High-quality and Skilled Demonstration (Excellent in Safety / Comfort / Efficiency); when the driving scenario is following another vehicle on a rainy highway, and the driving quality assessment results are: Safety 65 points, Comfort 80 points, Efficiency 78 points, and the overall quality label "Average Demonstration," the integrated driving event information is: Following another vehicle on a rainy highway - Average Demonstration (Safety meets standards, Comfort / Efficiency is good); when the driving scenario is detouring through a construction zone, and the driving quality assessment results are: Safety 50 points, Comfort 55 points, Efficiency 45 points, and the overall quality label "Counterexample," the integrated driving event information is: Detouring through a Construction Zone - Counterexample (Safety / Comfort / Efficiency do not meet standards).

[0091] As an optional embodiment, step S302 above, for evaluating the driving scenario based on vehicle state data and determining the driving quality evaluation result, may specifically include the following steps: Step 21, determine the vehicle type.

[0092] In this embodiment, the vehicle type can be determined by matching the vehicle identification code or model code obtained through the vehicle bus or interface with a preset model configuration table; or by directly reading preset vehicle dynamic parameters (such as curb weight, maximum power, maximum torque, braking system parameters, etc.) from the vehicle control unit, thereby determining the vehicle type (such as sports sedan, commercial vehicle, etc.). The vehicle type serves as a differentiated reference for driving quality assessment, i.e., different evaluation benchmarks (such as acceleration capability, braking distance, and expected thresholds for steering response) are set for different vehicle types (such as sports sedan, commercial vehicle) to ensure that the driving quality score reflects driving behavior performance adapted to the vehicle's own physical characteristics.

[0093] Step 22: Obtain the driving evaluation parameter set corresponding to the vehicle type. The driving evaluation parameter set includes at least one of the following: safe distance threshold, comfort acceleration threshold, and efficiency evaluation benchmark.

[0094] In this embodiment, a set of driving evaluation parameters corresponding to the vehicle type is obtained. This set includes at least one of a safe distance threshold, a comfort acceleration threshold, and an efficiency evaluation benchmark. The driving evaluation parameters refer to a pre-set set of evaluation thresholds and benchmarks for different vehicle types. The safe distance threshold refers to the minimum permissible distance or minimum permissible time interval that should be maintained between the vehicle and the vehicle in front, surrounding obstacles, or pedestrians under different driving scenarios to ensure driving safety. The comfort acceleration threshold refers to the upper limit values ​​set for the longitudinal acceleration, longitudinal jerk (i.e., the rate of change of acceleration), and lateral acceleration during vehicle movement to ensure the comfort of occupants. The efficiency evaluation benchmark refers to a reference standard established for specific driving operations (such as merging on a highway, turning left at an intersection, changing lanes on urban roads), under specific scenarios (such as dense traffic or smooth traffic), used to measure the completion time or traffic efficiency of the operation. For example, the efficiency evaluation benchmarks are as follows: in the scenario of merging on a highway, the reference merging time for a sports sedan is 5-8 seconds, for a family SUV it is 7-10 seconds, and for a commercial vehicle it is 10-15 seconds; in the scenario of turning left at an urban intersection, under smooth conditions, the reference passage time for a sports sedan is 6-9 seconds, for a family SUV it is 8-12 seconds, and for a commercial vehicle it is 12-18 seconds.

[0095] Step 23: Based on the driving evaluation parameter set and vehicle status data, evaluate the driving scenario and determine the driving quality evaluation result.

[0096] In this embodiment of the application, the driving scenario is evaluated based on the driving evaluation parameter set and vehicle status data to determine the driving quality evaluation result.

[0097] Specifically, vehicle status data such as vehicle speed, longitudinal acceleration, lateral acceleration, steering wheel angle, following distance, and operation time can be compared and quantified item by item with the corresponding safe distance threshold, comfort acceleration threshold, and efficiency evaluation benchmark in the driving evaluation parameter set. The driving behavior in the current driving scenario is scored and compliance is judged from multiple dimensions such as safety, comfort, legality, and efficiency. The evaluation results of each dimension are combined and weighted to finally obtain a driving quality assessment result that includes scores of each dimension, a comprehensive score, and quality labels.

[0098] For example, when a vehicle performs a highway merging operation, the actual operation time (e.g., 6.3 seconds) is extracted and compared with the efficiency evaluation benchmark corresponding to the current vehicle type (e.g., a sports sedan) (reference merging time 5-8 seconds). Since 6.3 seconds falls within the reference range, the efficiency score is 100 points; if the actual time is 4.2 seconds (below the lower limit), it is reduced to 93.6 points according to the deduction rules; if the actual time is 10 seconds (above the upper limit), it is reduced to 87.5 points. In this way, scores for dimensions such as safety (e.g., comparison of minimum following distance with the safe distance threshold) and comfort (e.g., comparison of maximum acceleration with the comfort acceleration threshold) are calculated separately. After weighting and merging the scores for each dimension, the driving quality assessment result is output.

[0099] like Figure 4 The diagram shown is a schematic representation of the implementation flow of another method for generating driving sample data provided in this application. Figure 4 exist Figure 1 Building upon this foundation, the paper details how to correlate task perception information, environmental assessment information, driving event information, and multi-source data based on contamination detection results to generate driving sample data for autonomous driving learning. Specifically, this driving sample data can include the following: S401, determine the image quality score based on the dirt detection results.

[0100] In this embodiment, an image quality score is determined based on the contamination detection results. The image quality score is used to quantitatively evaluate the usability of the current image sequence data. The image quality score can be derived from the image quality detection results of step S201 above, and can be a weighted fusion score of dimensions such as sharpness, contrast, noise, exposure, blur, and occlusion, or it can be a discrete level (Excellent / Good / Usable / Unusable).

[0101] S402, if the image quality score is greater than the preset quality threshold, determine the trajectory quality score based on task perception information and environmental assessment information, and determine the data quality level based on the trajectory quality score.

[0102] The aforementioned trajectory quality score is used to quantitatively evaluate the vehicle's trajectory performance, serving as a core metric for driving proficiency.

[0103] The aforementioned data quality level refers to the degree of data quality used to characterize image sequence data.

[0104] In this embodiment, if the image quality score is greater than a preset quality threshold, it indicates that the sharpness, contrast, exposure, and occlusion level of the current image sequence data meet the basic requirements for data quality, and the accuracy of the driving sample data will not be affected by quality issues in the image sequence data. Therefore, the trajectory quality score can be determined based on task perception information and environmental assessment information, and the data quality level can be determined based on the trajectory quality score. This avoids including low-quality images such as overexposed, underexposed, motion-blurred, and severely occluded images in the training dataset, thereby ensuring the validity of the driving sample data.

[0105] To determine the trajectory quality score based on task perception information and environmental assessment information, the driving scenario can be determined based on the task perception information and environmental assessment information. The driving scenario can then be evaluated based on vehicle state data to determine the driving quality assessment result. (For details, please refer to steps S302 above.) Then, scores including dimensions such as safety, comfort, legality, and efficiency can be obtained from the driving quality assessment result. These multi-dimensional scores are then weighted to determine the trajectory quality score.

[0106] For example, if the driving quality assessment results are: safety 95 points, comfort 92 points, efficiency 90 points, and the overall quality label "high-quality and skilled demonstration", then the safety score of 95 points, comfort score of 92 points, and efficiency score of 90 points are obtained from the driving quality assessment results. The scores of each dimension are then weighted and integrated according to preset weights (e.g., safety weight 0.4, comfort weight 0.3, efficiency weight 0.3) to calculate the trajectory quality score as: 95×0.4+92×0.3+90×0.3=92.6 points.

[0107] In another embodiment of this application, if the image quality score is less than or equal to a preset quality threshold, it indicates that the current image has serious overexposure / underexposure, motion blur, lens dirt, or occlusion, and cannot provide effective visual information. Any of the following strategies can be adopted: (1) discard the event data directly and do not generate subsequent driving sample data; (2) mark the data quality level of the event as "unavailable" or the lowest level (such as level 1), and only retain the metadata for statistical analysis; (3) if the image quality score is lower than the threshold but has not completely failed (e.g., only partially occluded), the upload priority of the event can be reduced, or a cleaning reminder can be triggered and the next event can be waited for.

[0108] As an optional implementation, a data quality scoring rule table can be preset, storing the mapping relationship between trajectory quality scoring intervals and data quality levels, thereby determining the data quality level based on the trajectory quality score.

[0109] For example, the data quality scoring rule table could be as follows: when the trajectory quality score is 90-100, the corresponding data quality level is 5 (the highest level), marked as "high-quality proficient demonstration"; when the trajectory quality score is 75-89, the corresponding data quality level is 4, marked as "general proficient demonstration"; when the trajectory quality score is 60-74, the corresponding data quality level is 3, marked as "usable demonstration"; and when the trajectory quality score is below 60, the corresponding data quality level is 2 or 1, marked as "low quality" or "counterexample".

[0110] S403 correlates multi-source data, data quality levels, task perception information, environmental assessment information, and driving event information to generate driving sample data for autonomous driving learning.

[0111] The aforementioned association refers to organizing multi-source data, data quality levels, task perception information, environmental assessment information, and driving event information into a unified data object using a preset format (such as JSON, Protobuf, or HDF5), thereby generating driving sample data for autonomous driving learning. Specifically, image sequence data from the multi-source data can be stored as compressed video files or image file paths stored frame by frame, with an index recorded in the data object. Vehicle status data and environmental perception data are stored aligned to timestamps.

[0112] In this embodiment, multi-source data, data quality level, task perception information, environmental assessment information, and driving event information can be correlated to generate driving sample data for autonomous driving learning.

[0113] For example, multi-source data, data quality levels, task perception information, environmental assessment information, and driving event information can be encapsulated into a single JSON-formatted related entry to generate driving sample data for autonomous driving learning.

[0114] After performing step S403 above, the driving sample data can also be stored, which may include the following: Step 31: De-identify the driving sample data to obtain de-identified driving sample data.

[0115] In this embodiment, the driving sample data is anonymized to obtain anonymized driving sample data. Specifically, the anonymization process may include one or more of the following operations: License plate blurring: Identify the license plate areas of other vehicles in the image sequence and perform Gaussian blurring, mosaic, or pixelation on these areas to make the specific numbers unrecognizable; Face blurring: Detect the face areas of pedestrians and drivers (e.g., visible through rearview mirrors or side windows) in the image and blur them; GPS accuracy downgrading: Round or offset the original precise latitude and longitude coordinates (e.g., centimeter or meter level), for example, retaining them to 3 decimal places (approximately 100-meter accuracy), or replacing them with road segment identifiers instead of precise coordinates; Timestamp anonymization: Adjust the timestamps accurate to milliseconds to retain only the date and hour, or add a random offset to prevent tracing specific trips through time correlation; Filtering of other sensitive information: Such as in-vehicle voice messages and addresses, which are removed or hashed and anonymized according to the configuration.

[0116] Step 32: Determine the upload priority of the anonymized driving sample data based on the data quality level.

[0117] In this embodiment of the application, the upload priority of the anonymized driving sample data is determined based on the data quality level.

[0118] Specifically: For driving sample data with high data quality levels (such as Level 5 and Level 4) ("high-quality proficiency demonstration"), the highest upload priority should be assigned, and the data should be uploaded immediately when the network is available, with higher bandwidth allocated to ensure transmission speed; for data with medium data quality levels (such as Level 3), a normal priority should be assigned, and the data should be uploaded in batches during periods of network downtime (such as at night or when vehicles are parked), with bandwidth usage appropriately reduced; for data with low data quality levels (such as Level 2 and Level 1), the lowest priority should be assigned, and the data can be uploaded with a delayed upload, or it can be processed locally with a high compression rate before being uploaded in batches, or even cleaned up periodically without being uploaded.

[0119] Step 33: Upload the anonymized driving sample data to the preset storage area according to the upload priority.

[0120] In this embodiment, the anonymized driving sample data is uploaded to a preset storage area according to the upload priority. The preset storage area can be a cloud object storage bucket, a distributed file system, or a data warehouse service.

[0121] Specifically, an adaptive upload strategy can be initiated based on the data quality level and upload priority of the anonymized driving sample data: when network bandwidth is sufficient and the vehicle is parked or stationary, high-quality proficiency demonstration data of level 4 and level 5 are prioritized for transmission; when network bandwidth is limited or the vehicle is in motion, fragmented transmission, dynamic bitrate compression, and breakpoint resumption mechanisms are used to ensure transmission stability, while low-priority data undergoes lightweight processing such as frame extraction and bitrate reduction before uploading; TLS / HTTPS encrypted transmission is used during the upload process to ensure data security and compliance, and finally all qualified anonymized driving sample data are stably uploaded to the cloud object storage bucket, distributed file system, or dedicated data warehouse service according to priority, completing the closed-loop collection of high-quality driving sample data from the vehicle to the cloud.

[0122] Furthermore, according to the above Figure 1 The method for generating driving sample data can also determine cleaning reminders based on task perception information, specifically including the following steps: Step 41: Identify at least one occlusion based on task perception information.

[0123] In this embodiment, at least one obstruction is determined based on task perception information. The obstruction can be any object attached to the camera lens or windshield that affects image acquisition, such as raindrops, mud, insect carcasses, ice, snow, frost, fog, stickers, or stains.

[0124] Step 42: For any occlusion, based on the image quality detection results in the task perception information, identify the type and degree of occlusion corresponding to the occlusion.

[0125] In this embodiment, for any occlusion, the type and degree of occlusion are identified based on the image quality detection results in the task-aware information. The degree of occlusion can be represented as a percentage of the total image area covered by the occlusion, or as the coverage ratio of the occlusion in key areas (such as lane line areas or areas of vehicles ahead).

[0126] Step 43: Obtain the occlusion threshold corresponding to the occlusion type.

[0127] In this embodiment of the application, since different types of occlusions have different degrees of impact on image quality, different occlusion thresholds can be set in advance for different types of occlusions, thereby obtaining the occlusion thresholds corresponding to the types of occlusions.

[0128] For example, raindrops (such as light rain): the occlusion area threshold can be set to 15%, because raindrops are usually small and scattered, and have limited impact on perception; mud spots / insect corpses: the occlusion area threshold can be set to 5%, because these obstructions are usually opaque and will severely obstruct local areas; ice and snow: the occlusion area threshold can be set to 3%, because ice and snow can easily cause large-scale visual obstruction.

[0129] Step 44: If the degree of occlusion exceeds the occlusion threshold corresponding to the type of occlusion, determine whether to trigger a cleaning reminder signal by combining the image quality score in the image quality detection results.

[0130] In this embodiment, if the degree of occlusion exceeds the occlusion threshold corresponding to the type of occlusion, it indicates that the current occlusion has reached or exceeded the preset level, which may have a significant impact on image quality and perception. Therefore, it is necessary to combine the image quality score in the image quality detection result to determine whether to trigger a cleaning reminder signal.

[0131] In another embodiment of this application, if the degree of occlusion does not exceed the occlusion threshold corresponding to the type of occlusion, it indicates that the current occlusion area is small or located in a non-critical area, and its impact on the overall image quality and perception task is within an acceptable range. In this case, regardless of the image quality score (because occlusion itself is not the main cause of quality degradation), a cleaning reminder signal is not triggered, and the trend of occlusion degree changes can continue to be monitored: if the occlusion degree continues to rise and is about to exceed the threshold, the sensitivity of the event trigger can be reduced in advance, or the above logic can be applied after the occlusion degree exceeds the threshold.

[0132] Step 45: When the cleaning reminder signal is triggered, output a prompt message through the vehicle human-machine interface to remind the driver to clean the camera or windshield.

[0133] In this embodiment of the application, when a cleaning reminder signal is triggered, a prompt message is output through the vehicle human-machine interface to remind the driver to clean the camera or windshield.

[0134] Corresponding to the above method embodiments, this application also provides a device for generating driving sample data, such as... Figure 5 As shown, the device may include a data acquisition module 501, a task detection module 502, an environmental assessment module 503, a driving event information determination module 504, and a driving sample data generation module 505.

[0135] The data acquisition module 501 is used to acquire multi-source data, including environmental perception data representing the external climate environment, image sequence data representing the external traffic environment, and vehicle status data representing the vehicle status. The task detection module 502 is used to perform task detection based on image sequence data to obtain task perception information, which includes at least one of dirt detection results, target detection and tracking results, lane line detection results, and drivable area detection results. Environmental assessment module 503 is used to determine environmental assessment information based on environmental sensing data; The driving event information determination module 504 is used to determine driving event information based on vehicle status data, task perception information, and environmental assessment information. The driving sample data generation module 505 is used to associate task perception information, environmental assessment information, driving event information and multi-source data to generate driving sample data for autonomous driving learning.

[0136] This application also provides a vehicle, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604. Memory 603 is used to store computer programs; In one embodiment of this application, when the processor 601 executes a program stored in the memory 603, it performs the following steps: Acquire multi-source data, including environmental perception data, image sequence data, and vehicle status data; perform task detection based on image sequence data to obtain task perception information; determine environmental assessment information based on environmental perception data; determine driving event information based on vehicle status data, task perception information, and environmental assessment information; process the multi-source data based on task perception information, environmental assessment information, and driving event information to generate driving sample data.

[0137] The communication bus mentioned in the above vehicles can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0138] The communication interface is used for communication between the aforementioned vehicle and other devices.

[0139] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0140] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0141] In another embodiment provided in this application, a storage medium is also provided, which stores instructions that, when run on a computer, cause the computer to execute the method for generating driving sample data as described in any of the above embodiments.

[0142] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the method for generating driving sample data as described in any of the above embodiments.

[0143] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0144] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0146] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the protection scope of this application.

Claims

1. A method for generating driving sample data, characterized by, The method comprises: acquiring multi-source data, the multi-source data comprising environment perception data representing an external climate environment of a vehicle, image sequence data representing an external traffic environment of the vehicle, and vehicle state data representing a state of the vehicle; performing task detection based on the image sequence data to obtain task perception information, the task perception information comprising at least one of target detection and tracking results, lane line detection results, and drivable area detection results; determining environment evaluation information based on the environment perception data; determining driving event information representing a driving behavior based on the vehicle state data, the task perception information, and the environment evaluation information; associating the task perception information, the environment evaluation information, the driving event information, and the multi-source data to generate driving sample data for automatic driving learning.

2. The method of claim 1, wherein, The task detection based on the image sequence data to obtain task perception information comprises: performing image quality detection on the image sequence data to obtain image quality detection results; performing scene detection on the image sequence data to obtain road condition perception results; integrating the image quality detection results and the road condition perception results to obtain the task perception information.

3. The method of claim 1, wherein, The determination of the environment evaluation information based on the environment perception data comprises: obtaining at least one environment feature representing a climate level, a climate change amount, or a climate risk level based on the environment perception data; determining the environment evaluation information based on the at least one environment feature.

4. The method of claim 1, wherein, The determination of the driving event information representing a driving behavior based on the vehicle state data, the task perception information, and the environment evaluation information comprises: determining a driving scene based on the task perception information and the environment evaluation information; evaluating the driving scene based on the vehicle state data to determine a driving quality evaluation result; integrating the driving scene and the driving quality evaluation result to obtain the driving event information representing the driving behavior.

5. The method of claim 4, wherein, The evaluation of the driving scene based on the vehicle state data to determine a driving quality evaluation result comprises: determining a vehicle type; acquiring a driving evaluation parameter set corresponding to the vehicle type, the driving evaluation parameter set comprising at least one of a safety distance threshold, a comfort acceleration threshold, and an efficiency evaluation benchmark; evaluating the driving scene based on the driving evaluation parameter set and the vehicle state data to determine the driving quality evaluation result.

6. The method of claim 1, wherein, The task perception information comprises a dirt detection result, and before the association of the multi-source data based on the task perception information, the environment evaluation information, and the driving event information to generate driving sample data for automatic driving learning, the method further comprises: determining an image quality score based on the dirt detection result; in a case where the image quality score is greater than a preset quality threshold, determining a trajectory quality score based on the task perception information and the environment evaluation information, and determining a data quality level according to the trajectory quality score; the association of the multi-source data based on the task perception information, the environment evaluation information, and the driving event information to generate driving sample data for automatic driving learning comprises: Correlate the multi-source data, the data quality level, the task perception information, the environment assessment information and the driving event information to generate the driving sample data for automatic driving learning.

7. The method of claim 6, wherein, After the multi-source data, the data quality level, the task perception information, the environment assessment information and the driving event information are correlated to generate the driving sample data for automatic driving learning, the method further comprises: Desensitizing the driving sample data to obtain desensitized driving sample data; According to the data quality level, determine the upload priority corresponding to the desensitized driving sample data; According to the upload priority, upload the desensitized driving sample data to a preset storage area.

8. The method of claim 1, wherein, The method further comprises: According to the task perception information, determine at least one occlusion; For any occlusion, based on the image quality detection result in the task perception information, identify the occlusion type and the occlusion degree corresponding to the occlusion; Obtain the occlusion threshold corresponding to the occlusion type; In the case where the occlusion degree exceeds the occlusion threshold corresponding to the occlusion type, combine the image quality score in the image quality detection result to determine whether to trigger a cleaning reminder signal; In the case where the cleaning reminder signal is triggered, output prompt information through a vehicle-mounted human-computer interaction interface, and the prompt information is used to remind the driver to clean the camera or windshield.

9. A driving sample data generation device characterized by comprising: The device comprises: A data acquisition module for acquiring multi-source data, the multi-source data comprising environment perception data representing the external climate environment of the vehicle, image sequence data representing the external traffic environment of the vehicle, and vehicle state data representing the state of the vehicle; A task detection module for performing task detection based on the image sequence data to obtain task perception information, the task perception information comprising at least one of target detection and tracking results, lane line detection results, and drivable area detection results; An environment assessment module for determining environment assessment information based on the environment perception data; A driving event information determination module for determining driving event information representing driving behavior based on the vehicle state data, the task perception information and the environment assessment information; A driving sample data generation module for correlating the task perception information, the environment assessment information, the driving event information and the multi-source data to generate driving sample data for automatic driving learning.

10. A vehicle characterized by comprising: It comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor is used to execute the programs stored on the memory to realize the method of any one of claims 1-8.

Images