Vehicle-mounted driving data video generation method and device and electronic equipment

Abstract

The invention discloses a vehicle-mounted driving data video generation method and device and electronic equipment, and relates to the technical field of vehicles, and the method comprises the steps: collecting multi-source data of a vehicle-mounted edge layer in a driving process, carrying out the fusion and preprocessing of the collected multi-source data, extracting driving feature data, and recognizing a key event; uploading the extracted driving feature data and the identified key event to a cloud service layer, and performing analysis based on a driving behavior scoring model to generate a driving behavior report; and a video generation engine of the cloud service layer automatically generates a driving record video according to the driving behavior report and the original data collected by the vehicle-mounted edge layer in combination with a scene label and a video template.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a method for generating vehicle driving data video, a device for generating vehicle driving data video, an electronic device, and a storage medium. Background Technology

[0002] With the development of AI technology and hardware, people have increasingly higher requirements and expectations for the functions of in-vehicle systems. Meanwhile, modern technology has made progress in data recording, driving monitoring, and video editing. For example, it records data such as mileage, speed, and energy consumption, and displays it visually on the dashboard. Systems that record vehicle status, environmental perception, and decision-making control improve the driving experience and control. Some dashcams may even have automatic emergency event editing functions. However, these functions lack coordination. Driving behavior data, video data, and driver status are usually stored independently, making it impossible to process this data uniformly and generate ideal observation data. Furthermore, video generation systems lack intelligent storytelling capabilities, requiring manual editing and failing to automatically generate personalized travel reports. This invention can combine data from OBD, cameras, radar, etc., to build a unified time-series database, deploy a lightweight AI model in the in-vehicle system, and combine reinforcement learning to automatically adjust the editing according to user needs, intelligently generating driving videos. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method for generating vehicle driving data video, a device for generating vehicle driving data video, an electronic device and a storage medium, in order to solve the technical problem that the prior art cannot uniformly process and generate ideal observation data.

[0004] This invention provides the following solution: According to one aspect of this application, a method for generating vehicle driving data video is provided, comprising the following steps:

[0005] Collect multi-source data during the driving process, fuse and preprocess the collected multi-source data, extract driving feature data and identify key events;

[0006] The extracted driving feature data and identified key events are uploaded to the cloud service layer, analyzed based on the driving behavior scoring model, and a driving behavior report is generated.

[0007] The video generation engine in the cloud service layer automatically generates driving recording videos based on driving behavior reports and raw data collected from the vehicle edge layer, combined with scene tags and video templates.

[0008] Furthermore, the collected multi-source data includes: driving behavior data, environmental perception data, vehicle bus data, and driver monitoring data.

[0009] Furthermore, including:

[0010] The driving behavior scoring model includes: feature extraction, multi-dimensional scoring, and weight fusion;

[0011] Feature extraction includes: extracting feature values ​​of acceleration, speed, following behavior, and driver attention based on driving feature data and key event identification;

[0012] Multi-dimensional scoring includes: performing multi-dimensional scoring based on extracted feature values ​​to obtain scoring results;

[0013] Weighted fusion includes: weighting the scores according to preset weights to output the final comprehensive driving behavior score.

[0014] Furthermore, including:

[0015] Multiple dimensions include safety, fuel consumption, and stability.

[0016] Furthermore, including:

[0017] Based on the analysis results, the cloud-based intelligent analysis platform identifies non-compliant driving behaviors and generates warning messages;

[0018] Simultaneously, by combining driving environment data, the system analyzes driving status under different environments and provides driving suggestions.

[0019] Furthermore, including:

[0020] The vehicle edge layer also includes: a real-time data processing engine;

[0021] When the real-time data processing engine identifies a critical event, it issues a real-time warning through the in-vehicle display terminal, voice prompts, or haptic feedback, and marks and uploads the warning information to the cloud service layer.

[0022] Furthermore, including:

[0023] The collected sensor data is subjected to noise filtering, missing value completion, and outlier detection and removal.

[0024] Perform timestamp synchronization and time sequence alignment on multi-source data.

[0025] According to two aspects of this application, an in-vehicle driving data video generation device is provided, comprising:

[0026] Data acquisition unit, model analysis unit, and record generation unit;

[0027] The data acquisition unit is used to collect multi-source data during the driving process, fuse and preprocess the collected multi-source data, extract driving feature data, and identify key events;

[0028] The model analysis unit is used to upload extracted driving feature data and identified key events to the cloud service layer, analyze them based on the driving behavior scoring model, and generate a driving behavior report.

[0029] The recording generation unit, used by the video generation engine in the cloud service layer, automatically generates driving recording videos based on driving behavior reports and raw data collected from the vehicle edge layer, combined with scene tags and video templates.

[0030] According to three aspects of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0031] The memory stores a computer program, which, when executed by a processor, causes the processor to perform the steps of a method for generating vehicle driving data video.

[0032] According to four aspects of the present invention, a computer-readable storage medium is provided that stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a method for generating vehicle driving data video.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] This application constructs a multimodal data fusion framework and uses timestamp alignment technology to achieve millisecond-level synchronization of CAN bus data, video stream, and IMU sensor data.

[0035] This application generates a 3D accident reconstruction report by automatically associating multi-source data. The report includes vehicle dynamics data and surround-view video, which saves a lot of time compared to traditional manual analysis. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a method for generating vehicle driving data video according to one or more embodiments of the present invention.

[0038] Figure 2 This is a structural diagram of an in-vehicle driving data video generation device provided in one or more embodiments of the present invention.

[0039] Figure 3 This is an architectural design diagram of an in-vehicle driving data video generation system according to a specific embodiment of the present invention.

[0040] Figure 4 This is a video generation flowchart of an in-vehicle driving data video generation device according to a specific embodiment of the present invention.

[0041] Figure 5 This is a block diagram of an electronic device for generating vehicle driving data video according to one or more embodiments of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0044] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0045] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0046] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0047] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0048] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0049] Figure 1 This is a flowchart of a method for generating vehicle driving data video according to one or more embodiments of the present invention.

[0050] like Figure 1 As shown, it includes the following steps:

[0051] Step S1: Collect multi-source data from the vehicle edge layer during the driving process, fuse and preprocess the collected multi-source data, extract driving feature data and identify key events;

[0052] Specifically, the multi-source data includes: driving behavior data, environmental perception data, vehicle bus data, and driver monitoring data.

[0053] The vehicle edge layer includes: driving behavior sensors, environmental perception sensors, vehicle bus data acquisition module, and driver monitoring module;

[0054] The driving behavior sensors include: steering wheel angle and torque sensors, accelerator and brake pedal travel sensors, and an inertial measurement unit.

[0055] Environmental perception sensors include: forward-facing cameras, surround-view cameras, and millimeter-wave radar;

[0056] The vehicle bus data acquisition module includes: acquiring real-time data on vehicle speed, engine speed, gear position, and fuel consumption via the OBD-II / CAN bus.

[0057] The driver monitoring module includes an infrared camera and a microphone.

[0058] Step S2 involves uploading extracted driving feature data and identified key events to the cloud service layer, analyzing them based on the driving behavior scoring model, and generating a driving behavior report.

[0059] Specifically, this includes an intelligent analysis platform: this platform uses driving behavior models to comprehensively analyze driving behavior and generate corresponding driving behavior reports. It issues warnings for non-compliant driving behaviors, analyzes the driving environment, analyzes driving behavior under different driving conditions, and provides reasonable suggestions.

[0060] In step S3, the video generation engine in the cloud service layer automatically generates driving recording videos based on the driving behavior report and the raw data collected by the vehicle edge layer, combined with scene tags and video templates.

[0061] Specifically, it receives data collected from the edge and combines it with the results of the intelligent analysis platform to generate videos based on different scene tags and video templates.

[0062] Furthermore, including:

[0063] The driving behavior scoring model includes: feature extraction, multi-dimensional scoring, and weight fusion;

[0064] Feature extraction includes: extracting feature values ​​of acceleration, speed, following behavior, and driver attention based on driving feature data and key event identification;

[0065] Multi-dimensional scoring includes: performing multi-dimensional scoring based on extracted feature values ​​to obtain scoring results;

[0066] Weighted fusion includes: weighting the scores according to preset weights to output the final comprehensive driving behavior score.

[0067] Furthermore, including:

[0068] Multiple dimensions include safety, fuel consumption, and stability.

[0069] Furthermore, including:

[0070] Based on the analysis results, the cloud-based intelligent analysis platform identifies non-compliant driving behaviors and generates warning messages;

[0071] Simultaneously, by combining driving environment data, the system analyzes driving status under different environments and provides driving suggestions.

[0072] Furthermore, the in-vehicle edge layer also includes: a real-time data processing engine;

[0073] When the real-time data processing engine identifies a critical event, it issues a real-time warning through the in-vehicle display terminal, voice prompts, or haptic feedback, and marks and uploads the warning information to the cloud service layer.

[0074] Furthermore, including:

[0075] The collected sensor data is subjected to noise filtering, missing value completion, and outlier detection and removal.

[0076] Perform timestamp synchronization and time sequence alignment on multi-source data.

[0077] Furthermore, including:

[0078] The key events are: sudden acceleration / deceleration, sharp turns, and sudden steering wheel movements that significantly deviate from safe, stable, and standardized driving behavior in an instant or short period of time.

[0079] Specifically, by integrating driving behavior data (steering wheel, pedal, inertial measurement), environmental perception data (camera, radar), vehicle bus data (vehicle speed, RPM, etc.), and driver monitoring data (infrared camera, microphone), it comprehensively covers the four-dimensional driving scenario of people, vehicles, roads, and environment, solving the problem that traditional single data sources (such as relying solely on cameras) cannot fully capture the driving state, and ensuring the comprehensiveness of driving behavior analysis.

[0080] By employing preprocessing techniques such as noise filtering, missing value completion, and outlier removal, combined with timestamp synchronization and time sequence alignment technologies, this approach effectively addresses industry pain points such as noise interference, time sequence misalignment, and data incompleteness in multi-source sensor data. It improves data consistency and effectiveness, providing accurate data support for driving feature extraction and critical event identification.

[0081] The video generation engine combines driving behavior reports, raw data, scene tags, and templates to automatically generate driving record videos containing key events, scoring results, and warning information. It transforms abstract sensor data and scoring results into intuitive and visual content, solving the problem of traditional driving data being "obscure and difficult to trace," making it convenient for drivers to review and for regulators to verify (such as fleet management and insurance claims).

[0082] The video focuses on key events, non-compliant behaviors, and driving advice, avoiding the drawbacks of traditional in-vehicle video recording and information redundancy, improving the efficiency of driving data utilization, and reducing storage and viewing costs.

[0083] By scoring safety, fuel consumption, and stability from multiple dimensions, it not only ensures driving safety but also helps drivers reduce fuel consumption and improve driving stability, achieving the dual value of safety and economy. Combined with the analysis of driving status in different environments, it makes driving suggestions more relevant to actual scenarios and improves practicality.

[0084] Figure 2 This is a structural diagram of an in-vehicle driving data video generation device provided in one or more embodiments of the present invention.

[0085] like Figure 2 As shown, it includes:

[0086] Data acquisition unit, model analysis unit, and record generation unit;

[0087] The data acquisition unit is used to collect multi-source data during the driving process, fuse and preprocess the collected multi-source data, extract driving feature data, and identify key events;

[0088] The model analysis unit is used to upload extracted driving feature data and identified key events to the cloud service layer, analyze them based on the driving behavior scoring model, and generate a driving behavior report.

[0089] The recording generation unit, used by the video generation engine in the cloud service layer, automatically generates driving recording videos based on driving behavior reports and raw data collected from the vehicle edge layer, combined with scene tags and video templates.

[0090] It is worth noting that although only some basic functional modules are disclosed in this embodiment, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules does not mean that the scope of protection of the claims of this invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0092] Figure 3 This is an architectural design diagram of an in-vehicle driving data video generation system according to a specific embodiment of the present invention.

[0093] Specifically, this includes: the vehicle edge layer, which is mainly responsible for data collection, real-time data processing, and local data storage.

[0094] Data Acquisition Layer: Acquires key data through the following systems and hardware.

[0095] Driving behavior sensors: collect data from steering wheel angle and torque sensors, accelerator and brake pedal travel sensors, and inertial measurement units to capture data related to rapid acceleration, emergency braking, and sharp turns.

[0096] Environmental perception system: forward-facing camera records road conditions, surround-view camera, and millimeter-wave radar monitors vehicle distance.

[0097] Vehicle bus data: Real-time data such as vehicle speed, RPM, gear position, and fuel consumption are obtained via OBD-II / CAN bus.

[0098] Driver monitoring: infrared camera, microphone.

[0099] Real-time data processing engine: Based on the vehicle's AI processor, it processes the collected data in real time, integrates sensor data, analyzes and processes driving behavior based on the driving behavior scoring model, and monitors key events.

[0100] Local storage: A unified time-series database is built by combining OBD, camera, and radar data. The real-time data processing engine stores the processed data.

[0101] Cloud service layer: mainly responsible for intelligent data analysis, generating intelligent reports and videos, and storing the generated reports and videos to the user terminal system.

[0102] Intelligent Analysis Platform: This platform uses driving behavior models to comprehensively analyze driving behavior and generate corresponding driving behavior reports. It issues warnings for non-compliant driving behaviors, analyzes the driving environment, examines driving states under different driving conditions, and provides reasonable suggestions.

[0103] Video Generation Engine: The intelligent content production pipeline is the core component of this module. It receives data collected from the edge and combines it with the results of the intelligent analysis platform to generate videos based on different scene tags and video templates. See the processing flow for details. Figure 5 .

[0104] User terminal: Users have an independent data management terminal, which can be used to view driving record reports stored in the cloud and driving record videos generated and stored according to user requirements.

[0105] In another embodiment of the vehicle driving data and video generation method, the following steps are included:

[0106] S1. Data acquisition at the vehicle edge layer: Driving behavior data, environmental data, vehicle status data, and driver status data are collected through driving behavior sensors, environmental perception system, vehicle bus data acquisition module, and driver monitoring module, respectively.

[0107] S2. Real-time data processing at the vehicle edge layer: Integrate the multi-source data collected in step S1, analyze and process driving behavior through a driving behavior scoring model, and monitor key events;

[0108] S3. Local storage of the vehicle edge layer: The structured data processed in step S2 is stored in a unified time-series database, wherein the structured data is fused with OBD data, camera data and radar data;

[0109] S4. Intelligent Analysis of Cloud Service Layer: The data stored in the vehicle edge layer is transmitted to the cloud through the communication module. The cloud intelligent analysis platform performs a comprehensive analysis of the data based on the driving behavior model, generates a driving behavior report, issues warnings for non-compliant driving behaviors, analyzes the driving status in different driving environments, and outputs reasonable driving suggestions.

[0110] S5. Video generation in the cloud service layer: The cloud video generation engine receives the raw data collected in step S1 and the analysis results in step S4 through the intelligent content production pipeline, and generates driving recording videos according to preset scene tags and video templates;

[0111] S6. Output and User Interaction: The driving behavior report from step S4 and the driving recording video from step S5 are stored in the cloud and displayed to users through user terminals, allowing users to view and access them as needed.

[0112] Figure 5 This is a block diagram of an electronic device for generating vehicle driving data video according to one or more embodiments of the present invention.

[0113] like Figure 5 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0114] The memory stores a computer program that, when executed by a processor, causes the processor to perform the steps of a method for generating in-vehicle driving data video.

[0115] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a method for generating vehicle driving data video.

[0116] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0117] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating vehicle driving data video, characterized in that, include: Multi-source data from the vehicle edge layer during the driving process is collected, and the collected multi-source data is fused and preprocessed to extract driving feature data and identify key events; The extracted driving feature data and identified key events are uploaded to the cloud service layer, and analyzed by the cloud intelligent analysis platform based on the driving behavior scoring model to generate a driving behavior report; The video generation engine in the cloud service layer automatically generates driving recording videos based on the driving behavior report and the raw data collected by the vehicle edge layer, combined with scene tags and video templates.

2. The method for generating vehicle driving data video according to claim 1, characterized in that, The collected multi-source data includes: driving behavior data, environmental perception data, vehicle bus data, and driver monitoring data.

3. The method for generating vehicle driving data video according to claim 1, characterized in that, The driving behavior scoring model includes: feature extraction, multi-dimensional scoring, and weight fusion; Feature extraction includes: extracting feature values ​​of acceleration, speed, following behavior, and driver attention based on the driving feature data and identified key events; Multi-dimensional scoring includes: performing multi-dimensional scoring based on extracted feature values ​​to obtain scoring results; Weighted fusion includes: weighting the scores according to preset weights to output the final comprehensive driving behavior score.

4. The method for generating vehicle driving data video according to claim 1, characterized in that, The multi-dimensional aspects include safety, fuel consumption, and stability.

5. The method for generating vehicle driving data video according to claim 1, characterized in that, The cloud-based intelligent analysis platform identifies non-compliant driving behaviors and generates warning messages based on the analysis results. Simultaneously, by combining driving environment data, the system analyzes driving status under different environments and provides driving suggestions.

6. The method for generating vehicle driving data video according to claim 1, characterized in that, The vehicle edge layer also includes: a real-time data processing engine; When the real-time data processing engine identifies a critical event, it issues a real-time warning through the in-vehicle display terminal, voice prompts, or haptic feedback, and marks and uploads the warning information to the cloud service layer.

7. The method for generating vehicle driving data video according to claim 1, characterized in that, include: The collected sensor data is subjected to noise filtering, missing value completion, and outlier detection and removal. Perform timestamp synchronization and time sequence alignment on multi-source data.

8. A vehicle-mounted driving data video generation device, characterized in that, include: Data acquisition unit, model analysis unit, and record generation unit; The data acquisition unit is used to collect multi-source data from the vehicle edge layer during the driving process, fuse and preprocess the collected multi-source data, extract driving feature data, and identify key events. The model analysis unit is used to upload the extracted driving feature data and identified key events to the cloud service layer, analyze them based on the driving behavior scoring model, and generate a driving behavior report. The recording generation unit is used by the video generation engine of the cloud service layer to automatically generate driving recording videos based on the driving behavior report and the raw data collected by the vehicle edge layer, combined with scene tags and video templates.

9. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the vehicle driving data video generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the vehicle driving data video generation method according to any one of claims 1-7.

Images