Intelligent driving recording method and system based on voice AI

Abstract

The invention discloses an intelligent driving recording method and system based on voice AI, and relates to the technical field of intelligent driving recording. The method comprises the following steps: acquiring a user voice instruction through voice acquisition equipment, and identifying the user voice instruction through a vehicle-mounted lightweight ASR model; analyzing the instruction to generate a structured control instruction containing an event type and a time reference; determining a time window of the event to be marked in combination with the time reference, the motion data of the IMU sensor and the video data; packaging, encrypting and storing the video clip in the window, the sensor data and the structured control instruction to generate an event record; the video is analyzed in real time through a local lightweight computer vision model, and when a preset dangerous scene is detected, voice early warning is triggered and an associated video is automatically stored. According to the invention, voice interaction control, accurate event marking, real-time danger early warning and efficient data management of the driving record are realized, and the convenience, accuracy and safety of the driving record are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle recording technology, and in particular to an intelligent vehicle recording method and system based on voice AI. Background Technology

[0002] Traditional dashcams rely on cameras to record video and store it locally. They require manual operation or physical buttons to trigger recording of key events and use a cyclic overwrite storage mode, enabling only basic video playback. They suffer from inconvenience, with manual operation during driving easily distracting drivers; massive amounts of video require manual browsing; they cannot quickly locate specific segments through semantics; and their level of intelligence is insufficient.

[0003] Some existing intelligent dashcam technologies attempt to be optimized by combining ADAS or AI technologies, such as vision-based event detection, mobile app integration, and basic voice control. However, these technologies suffer from drawbacks such as high false alarm rates and reliance on computing power for vision detection, manual operation for mobile app integration, inability of basic voice control to understand complex semantics, isolated voice and video data preventing semantic retrieval, poor real-time performance, reliance on cloud computing for AI analysis, and delayed warnings due to weak local processing capabilities. Furthermore, insufficient fusion of multi-sensor data makes it difficult to meet users' needs for convenient interaction and accurate intelligent analysis. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide an intelligent driving recording method and system based on voice AI to solve the problems mentioned in the background art.

[0005] In a first aspect, this application provides an intelligent driving recording method based on voice AI, characterized in that the method includes: The user's voice commands are acquired in real time through a voice acquisition device; The operation intent and event marker request in the user's voice command are parsed to generate a structured control command containing event type and time reference; Based on the aforementioned time reference, and combining the motion state data from the IMU sensor with the video data collected by the dashcam, the time window for the event to be marked is determined. The video clips, sensor data, and structured control instructions within the time window are packaged and stored to generate an event log; The system analyzes video clips in the event log in real time, and triggers a voice warning and automatically saves the associated video when a preset scene is detected.

[0006] In one embodiment, parsing the user's voice command for operational intent and event marker requests to generate structured control commands containing event types and time bases includes: Natural language processing technology is used to extract the operational intent from the user's voice commands; Based on the stated operational intent, the key information in the event tagging request is matched using a preset instruction template; Based on the key information, the event type and the time reference are determined, and the structured control instructions are generated.

[0007] In one embodiment, determining the time window for the event to be marked based on the time reference, combined with motion state data from the IMU sensor and video data collected by the dashcam, includes: Based on the time reference, the motion state data of the IMU sensor is correlated to determine the duration range of the vehicle dynamic event corresponding to the event tag request; Based on the duration range, and combined with the changes in the video data collected by the dashcam related to the vehicle's dynamic events, the start and end boundaries of the time window for the event to be marked are determined.

[0008] In one embodiment, the step of packaging and storing the video clips, sensor data, and structured control instructions within the time window to generate an event log includes: Acquire the video clips, corresponding sensor data, and structured control commands within the time window; The video segment is compressed and encoded, and the sensor data is synchronized and aligned with the time base of the video segment. The synchronized video clips, sensor data, and structured control commands are associated and integrated to form a structured data packet, wherein the event types include: critical events and regular events; The structured data packet is encrypted and then stored in the storage unit to generate an event record.

[0009] In one embodiment, the real-time analysis of video clips in the event log, triggering a voice alert and automatically saving the associated video when a preset scene is detected, includes: Obtain the video clips and corresponding motion state data contained in the event log; The video clips are analyzed in real time using a lightweight computer vision model, and the target objects and dynamic features in the scene are identified by combining the motion state data. When the recognition result matches a preset scenario, a voice warning corresponding to the preset scenario is triggered; Simultaneously extract video clips before and after the occurrence of the preset scene as associated videos, package and store the associated videos with the corresponding motion state data and the voice warning, and update the event record.

[0010] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record includes: The structured data packets corresponding to the key events are encrypted using hardware encryption methods to obtain hardware-encrypted data packets; The structured data packets corresponding to the regular events are encrypted using software encryption methods to obtain software-encrypted data packets; The hardware encrypted data packet and the software encrypted data packet are stored in different user access partitions in the storage unit, respectively.

[0011] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes: Create index information that includes the event type, time base, and video scene features; The index information is associated with the corresponding encrypted structured data packet, and the index information is stored in the index area of ​​the storage unit.

[0012] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes: Based on a dynamic threshold, the event type corresponding to the structured data packet is determined. When the event type is a critical event, the hardware-encrypted data packet preferentially occupies the high-privilege access partition in the storage unit.

[0013] In one embodiment, the method further includes an event retrieval step: Receive the user's voice search command and use natural language processing technology to parse the search intent in the voice search command; Extract the search keywords corresponding to the search intent; Based on the search keywords, match the corresponding event type, time frame, and video scene features in the index information; The corresponding event record is retrieved from the storage unit, the retrieved event record is decrypted, and the retrieved data containing video clips, sensor data and structured control instructions within the time window is returned.

[0014] Secondly, this application also provides an intelligent driving recorder system based on voice AI, characterized in that the system includes: The voice interaction module is used to acquire user voice commands in real time through a voice acquisition device; The instruction parsing module is used to parse the operation intent and event marker request in the user's voice instructions and generate structured control instructions containing event types and time bases; The multi-sensor fusion module is used to determine the time window of the event to be marked based on the time reference, combining the motion state data of the IMU sensor and the video data collected by the dashcam. The event log generation module is used to package and store video clips, sensor data, and structured control instructions within the time window to generate an event log. The associated video storage module is used to analyze video segments in the event record in real time, and trigger a voice warning and automatically save the associated video when a preset scene is detected.

[0015] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the intelligent driving recording method based on voice AI described in any embodiment of this application.

[0016] The aforementioned intelligent driving recorder method and system based on voice AI achieves hands-free operation throughout the entire process through a voice interaction module. It combines edge-side ASR and NLP technologies to parse complex commands, avoiding the safety hazards of manual operation while driving. By integrating visual, voice, and IMU data from multiple sensors, it accurately defines event time windows, solving the problem of missed detections by a single sensor. It employs hierarchical storage combining hardware encryption (for critical events) and software encryption (for routine events), along with physical isolation partitions and dynamic keys, ensuring data security and compliance with privacy regulations. It constructs a semantic index containing event type, time reference, and scene features, supporting voice retrieval for quick location of target records, replacing traditional manual browsing. Real-time scene detection and voice warnings are achieved through a local lightweight visual model, combined with a priority storage mechanism triggered by dynamic thresholds, ensuring priority storage of critical data. The system's modular design adapts to different vehicle models and can be expanded with more intelligent driving assistance functions, comprehensively improving the intelligence and practicality of driving records. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an intelligent driving recording method based on voice AI according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating the process of generating structured control instructions according to an exemplary embodiment; Figure 3 This is a structural block diagram of a voice AI-based intelligent driving recorder system according to an exemplary embodiment; Figure 4 This is an internal structural diagram of a computer device according to an exemplary embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "at least one" is used to indicate one or more; "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] like Figure 1 As shown in the figure, this application provides an intelligent driving recorder method based on voice AI, including the following steps: S1. Acquire user voice commands in real time through voice acquisition devices.

[0022] In this embodiment of the application, the voice acquisition device is a microphone array that supports far-field voice signal acquisition. The voice recognition process is completed locally on the vehicle terminal, wherein the microphone array performs noise reduction processing based on the in-vehicle acoustic environment.

[0023] In this embodiment of the application, the voice acquisition device refers to a microphone array (rather than a single microphone) deployed in the vehicle, which has the following characteristics: it can accurately capture voice commands from the driver's seat (usually 1-2 meters away from the device) and is adapted to the normal conversation volume of people in the vehicle; it performs noise reduction processing on the acoustic environment inside the vehicle (such as engine noise, air conditioning noise, road bump noise) and filters interference through beamforming, echo cancellation and other technologies to ensure clear voice signals.

[0024] S2. Parse the operation intent and event marker request in the user's voice command to generate a structured control command containing event type and time reference.

[0025] In some embodiments, such as Figure 2 As shown, the step of parsing the user's voice command for operation intent and event marker requests to generate structured control commands containing event types and time bases includes: S21. Natural language processing technology is used to extract the operation intent from the user's voice command; S22. Based on the stated operation intent, match the key information in the event marker request using a preset instruction template; S23. Determine the event type and the time base based on the key information, and generate the structured control instructions.

[0026] In this embodiment of the application, a lightweight ASR model deployed on an in-vehicle terminal is used for speech recognition, wherein the lightweight ASR model is a lightweight speech recognition model adapted to the in-vehicle scenario.

[0027] In this embodiment, the lightweight ASR model is a speech recognition model adapted to the hardware resources of the vehicle terminal (such as computing power, storage, and power consumption limitations), such as a lightweight RNN-T or Transformer model, and has the following characteristics: the model runs on the vehicle terminal (such as the main control chip of a dashcam), without relying on cloud computing, avoiding recognition failures caused by network latency or network outages; it is specifically trained for common driving scenario commands (such as "record video", "find accident footage", "warn of danger") to optimize recognition accuracy (especially the recognition accuracy of driving-related words); the model parameter size is small (usually less than 100M), the recognition latency is controlled within 500ms, and it does not occupy too much computing power of the vehicle system.

[0028] In this embodiment of the application, when a user notices a vehicle illegally changing lanes while driving, he / she says the command: "Xiao D, record the illegal lane change just now". The lightweight ASR model of the vehicle terminal (such as the optimized MobileASR) quickly converts the speech into text: "Xiao D, record the illegal lane change just now".

[0029] In this embodiment of the application, natural language processing technology is a technology for analyzing and understanding text instructions after speech recognition, including word segmentation and semantic parsing.

[0030] In this embodiment, extracting the operational intent from the user's voice command using natural language processing technology includes: first, inputting the text command acquired and converted by the voice acquisition device into the natural language processing module. This module first uses a word segmentation algorithm to break down the text command into independent semantic units, for example, breaking down "record the sudden braking incident" into "record," "just now," "sudden braking," and "incident." Then, a pre-trained semantic understanding model analyzes the word segmentation results to identify words representing core actions, such as "record," "search," and "mark," thereby determining whether the user's operational intent is event recording, information retrieval, or other types. For example, when the user command is "Xiao D, find yesterday's accident video," the word segmentation results in "find," "yesterday," "accident," and "video," and the semantic understanding model identifies "find" as corresponding to the operational intent of information retrieval.

[0031] In this embodiment, the preset instruction template is a standardized format pre-defined for common instruction types in driving scenarios. Each operation intent corresponds to a specific template. For example, the event recording template is "[Operation Intent] + [Event Description] + [Time Description]", and the information retrieval template is "[Operation Intent] + [Event Type] + [Time Range]". Key information refers to the core content in the event tagging request used to determine the event attributes and time range, such as the event type (illegal lane change, pedestrian crossing) and time description (just now, 5 minutes ago).

[0032] In this embodiment, matching key information in the event tagging request based on the operational intent and a preset instruction template includes: first, calling the corresponding preset instruction template from the template library based on the extracted operational intent. For example, when the operational intent is event recording, the template "Record + [Event Type] + [Time Description]" is called. Then, the user's event tagging request text is structurally matched with the template, and key information is extracted through keyword mapping. For example, after the user instruction "Record the illegal lane change of the vehicle in front 3 minutes ago" is matched with the event recording template, the key information is extracted: the event type description is "illegal lane change", and the time description is "3 minutes ago".

[0033] In this embodiment, the event type is a specific category based on the event description in the key information, divided into key events (such as collisions, violations, pedestrian crossings, etc., which need to be stored first) and regular events (such as scenery records, road condition observations, etc.). The time base is a specific time parameter converted from the time description in the key information, divided into absolute time (such as July 24, 2025, 15:30:00) and relative time range (such as 30 seconds before the current time to the current time). The structured control instruction is a standardized data format that integrates operation intent, event type, and time base, usually using a JSON structure, and includes fields such as "operation intent", "event type", "time base start", and "time base end".

[0034] In this embodiment, determining the event type and time base based on the key information and generating the structured control instructions includes: first, converting the event description in the key information into a standardized event type using an event type mapping table. For example, mapping "illegal lane change" to "traffic violation - lane change" in key events, and mapping "scenery along the way" to "environmental record" in regular events. Next, calculating the time base based on the time description: if the time description is an absolute time (e.g., "15:20"), it is directly converted to a timestamp; if it is a relative time (e.g., "just now" or "5 minutes ago"), the time range is calculated by combining it with the current system time (current time - 5 minutes to current time). For example, if the time description in the key information is "3 minutes ago" and the current system time is 15:30:00, the time base is calculated to be from 15:27:00 to 15:30:00. Finally, the operational intent, the determined event type, and the time base are integrated into a structured control instruction according to a preset format, for example: {"Operation Intent":"Event Log","Event Type":"Critical Event - Unauthorized Lane Change","Time Base Start":"2025-07-24 15:27:00","Time Base End":"2025-07-24 15:30:00"}.

[0035] In this embodiment, the operation intent is accurately extracted through natural language processing technology, and key information is matched with preset templates to finally generate structured control instructions. This solves the problems of semantic ambiguity and inaccurate response of voice instructions in traditional driving records, realizes a deep understanding of complex natural language instructions, ensures the accurate conversion of operation intent and event tagging requests, provides a standardized basis for subsequent time window determination and data storage, improves the accuracy and efficiency of voice interaction, and reduces recording deviations caused by instruction misunderstandings.

[0036] S3. Based on the time reference, and combining the motion state data of the IMU sensor with the video data collected by the dashcam, determine the time window for the event to be marked.

[0037] In some embodiments, determining the time window of the event to be marked based on the time reference, combined with the motion state data of the IMU sensor and the video data collected by the dashcam, includes: determining the duration range of the vehicle dynamic event corresponding to the event marking request by associating the motion state data of the IMU sensor with the time reference; and defining the start and end boundaries of the time window of the event to be marked by combining the duration range with the screen changes related to the vehicle dynamic event in the video data collected by the dashcam.

[0038] In this embodiment, the motion state data of the IMU sensor refers to the vehicle motion parameters collected by the inertial measurement unit, including acceleration, angular velocity, and heading angle. This data reflects the vehicle's dynamic changes such as acceleration, deceleration, steering, and emergency braking. The time reference is the time reference for the occurrence of an event, which may be a specific point in time or a relative time range. The duration of a vehicle dynamic event refers to the time span from the start to the end of the event.

[0039] In this embodiment, determining the duration of the vehicle dynamic event corresponding to the event marking request by associating the motion state data of the IMU sensor with the time reference includes: first, retrieving the IMU sensor motion state data corresponding to the time reference from the storage unit, establishing an association mapping between the time reference and the motion state data, i.e., determining the motion state data segment corresponding to the time reference. Then, performing feature analysis on this data segment to identify motion features related to the event marking request; for example, for a "sudden braking event," identifying a feature of a significant negative change in acceleration within a short period. Finally, calculating the duration of the vehicle dynamic event based on the start and end points of these motion features; for example, if the time interval from the start of a significant negative change in acceleration to its return to stability is 3 seconds, then the duration is 3 seconds.

[0040] In this embodiment of the application, in order to ensure the accuracy of the judgment, a motion characteristic threshold is set. For example, the acceleration threshold for an emergency braking event is set to -5 meters per second squared. When the detected acceleration is less than this threshold, it is determined to be the start of an emergency braking event. When the acceleration rises back to above -1 meters per second squared, it is determined to be the end of the event.

[0041] In this embodiment, the changes in video data related to vehicle dynamic events refer to changes in visual information in the video captured by the dashcam that reflect the occurrence and development of the event, such as changes in the distance to vehicles ahead when a vehicle brakes suddenly, or the positional shift of a vehicle when it illegally changes lanes. A time window refers to a time interval containing a complete video segment of the event to be tagged; the starting boundary is the start time of the time window, and the ending boundary is the end time of the time window.

[0042] In this embodiment, based on the duration range and combined with the image changes related to the vehicle dynamic event in the video data collected by the dashcam, the start and end boundaries of the time window for the event to be marked are defined. This includes: first, locating the corresponding time segment in the dashcam video data based on the duration range of the vehicle dynamic event; then, analyzing the video segment frame by frame to identify image change features related to the vehicle dynamic event, such as identifying image changes of the vehicle crossing lane lines for a "violation of lane change event"; next, temporally associating the image change features with the duration range determined by the IMU sensor, using the starting point of the duration range as a reference, tracing back to the moment when the event-related change first appears in the image, as the start boundary of the time window; and using the ending point of the duration range as a reference, extending forward to the moment when the event-related change completely disappears in the image, as the end boundary of the time window. For example, if the duration is 3 seconds, and the vehicle starts crossing the line 0.5 seconds before the start of the duration, and then completely returns to the normal lane 0.5 seconds after the end of the duration, then the starting boundary of the time window is 0.5 seconds before the start of the duration, and the ending boundary is 0.5 seconds after the end of the duration, with a total time window duration of 4 seconds.

[0043] In this embodiment of the application, in order to avoid redundancy in image analysis, keyframe extraction technology is used to identify image changes only in key frames of the video segment. The keyframe extraction interval is set to 0.1 seconds, which ensures both analysis accuracy and processing efficiency.

[0044] In this embodiment, the duration of an event is determined based on IMU sensor data associated with a time reference, and the time window boundary is defined by combining changes in the video frame. This solves the problem of incomplete event recording caused by traditional dashcams relying solely on timestamps to capture video. By fusing multi-source data, the complete time interval of the event is accurately locked, ensuring that the time window covers the entire event process while avoiding redundant data storage. This improves the integrity and storage efficiency of event recording and provides reliable time dimension support for subsequent data retrieval and analysis.

[0045] S4. Package and store the video clips, sensor data and structured control instructions within the time window to generate an event record.

[0046] In some embodiments, the step of packaging and storing video segments, sensor data, and structured control instructions within the time window to generate an event record includes: acquiring video segments, corresponding sensor data, and structured control instructions within the time window; compressing and encoding the video segments; synchronizing and aligning the sensor data with the time reference of the video segments; associating and integrating the synchronized video segments, sensor data, and structured control instructions to form a structured data packet, wherein the event types include: critical events and regular events; encrypting the structured data packet and storing it in a storage unit to generate an event record.

[0047] In this embodiment, the video segment within the time window refers to a portion of the video containing the complete event to be labeled, and its time range is determined by the start and end boundaries. The corresponding sensor data refers to IMU sensor motion state data matching the time range of the video segment, including acceleration, angular velocity, etc. The structured control instructions are standardized data containing the operational intent, event type, and time reference.

[0048] In this embodiment, acquiring the video segment, corresponding sensor data, and structured control instructions within the time window includes: first, calling the dashcam's storage interface to extract the video data stream within the time period based on the start and end boundaries of the time window, forming a video segment. Simultaneously, retrieving IMU sensor data corresponding to the time window through the sensor data interface to ensure the time range of the sensor data is consistent with the video segment. Then, retrieving the previously generated structured control instructions from the instruction cache unit to complete the collection of the three types of data.

[0049] In this embodiment of the application, in order to ensure the accuracy of data acquisition, the matching degree between video clips, sensor data and time windows is verified by comparing timestamps. If the deviation exceeds 0.1 seconds, the data is retrieved again.

[0050] In this embodiment, compression encoding is a processing method that reduces video data size through a specific algorithm, thereby reducing the storage footprint of video segments. Time base synchronization alignment ensures that the timestamps of sensor data and video segments are consistent, guaranteeing a one-to-one correspondence between the two in the time dimension.

[0051] In this embodiment, the video segment is compressed and encoded to synchronize the sensor data with the time reference of the video segment. This includes: compressing the video segment using the H.265 encoding standard, which can compress the video data to about 50% of its original size while maintaining image quality. Then, the timestamp sequences of the video segment and the sensor data are extracted. Using the time reference of the video segment, the timestamp of the sensor data is adjusted using an interpolation algorithm to keep the time interval between the two within 0.05 seconds, achieving synchronization. For example, if the timestamp of a frame in the video segment is 10:00:00.000, the corresponding timestamp of the sensor data is adjusted to 10:00:00.000 ± 0.05.

[0052] In this embodiment, the structured data packet is a data set formed by combining synchronized video clips, sensor data, and structured control commands according to a preset format. Critical events refer to events involving safety, violations, or other issues requiring special attention and storage, such as collisions or running red lights. Routine events refer to non-emergency events recorded daily, such as landscape photography or road condition observation.

[0053] In this embodiment, the synchronized video clips, sensor data, and structured control commands are associated and integrated to form a structured data packet. The event types include critical events and regular events. This involves first adding an association identifier to the synchronized video clips and sensor data, which includes event type and time reference information. Then, core fields such as event type and time reference from the structured control commands are embedded in the data packet header as index information. Finally, the three types of data are integrated according to the format "header index + video data + sensor data + control commands" to form a structured data packet. For example, the header of a critical event data packet might be marked with "Event type: illegal lane change, time reference: 10:00:00-10:00:05," while a regular event data packet would be marked with corresponding regular type information.

[0054] In this embodiment, encryption is a technical means to protect structured data packets, preventing unauthorized access or tampering. The storage unit is a hardware device used to store the encrypted data packets, such as an in-vehicle SD card or a solid-state drive. The event log is the encrypted and stored structured data packet, containing complete event-related data.

[0055] In this embodiment, the structured data packet is encrypted and then stored in a storage unit to generate an event record. This includes selecting an encryption method based on the event type in the data packet: hardware encryption is used for critical events, where a unique key generated by the vehicle-mounted security chip is used to encrypt the data packet; software encryption is used for regular events, employing the AES-128 algorithm. After encryption, the hardware-encrypted critical event data packets are stored in the encrypted partition of the storage unit, while the software-encrypted regular event data packets are stored in the ordinary partition. A data verification code is generated during storage to verify data integrity during subsequent readings, ultimately forming an event record containing complete event information.

[0056] In this embodiment, the storage unit adopts a partition management mechanism, with 20% of the storage space reserved in the critical event partition to ensure that emergency event data is stored first and to avoid data loss due to insufficient space.

[0057] In this embodiment, by means of compression encoding, time synchronization, association integration and encrypted storage, the problems of isolated storage of video and sensor data, time asynchrony and easy data leakage in traditional driving records are solved. It realizes the structured integration of video clips, sensor data and control commands, reduces storage occupation by compression encoding, ensures data correlation by time synchronization, protects data security by encryption processing, and distinguishes between storage strategies for critical events and regular events, thus taking into account both data reliability and reasonable allocation of storage resources.

[0058] In some embodiments, encrypting the structured data packet and storing it in the storage unit to generate an event record includes: encrypting the structured data packet corresponding to the key event using hardware encryption to obtain a hardware-encrypted data packet; encrypting the structured data packet corresponding to the regular event using software encryption to obtain a software-encrypted data packet; and storing the hardware-encrypted data packet and the software-encrypted data packet in different user access partitions in the storage unit.

[0059] In this embodiment, hardware encryption is a technical means of data encryption implemented through dedicated hardware devices. This process relies on an onboard security chip to perform encryption operations. This chip has an independent encryption engine and key management unit, and the key is stored in a secure, unreadable area within the chip. The structured data packets corresponding to critical events are data sets containing information about important events such as collisions and violations, and their encryption requirements are higher than those for regular events. Hardware-encrypted data packets are data packets formed after hardware encryption processing and can only be decrypted by the corresponding hardware.

[0060] In this embodiment, the structured data packets corresponding to the critical events are encrypted using hardware encryption to obtain hardware-encrypted data packets. This process includes: first, calling the encryption interface of the vehicle security chip to input the structured data packets of the critical events into the chip. The encryption engine inside the security chip calls a preset hardware encryption algorithm, such as the SM4 national cryptographic algorithm, and simultaneously retrieves a unique device-bound key from the chip's built-in key storage area. This key is associated with the vehicle identification number and cannot be exported. During encryption, the data packets are divided into fixed-size data blocks, each of which is independently encrypted using a hardware algorithm and accompanied by a message authentication code to verify data integrity. After encryption, the security chip outputs the encrypted data packets, i.e., the hardware-encrypted data packets. The entire process is completed internally within the chip, preventing the key and data from being intercepted during transmission on an external bus.

[0061] In this embodiment of the application, to enhance security, the hardware encryption adopts a dynamic session key generation mechanism. Each time encryption is performed, the security chip randomly generates a session key, which is encrypted with the device binding key and stored together with the data packet to ensure that even if a single session key is leaked, it will not affect other encrypted data.

[0062] In this embodiment, software encryption is a technical means of data encryption implemented through software programs. It relies on the onboard processor to run an encryption algorithm to complete the encryption, without relying on dedicated hardware encryption devices. The structured data packets corresponding to routine events are data sets containing information about non-emergency events such as scenery records and road condition observations; their encryption requirements are lower than those for critical events. Software-encrypted data packets are data packets formed after software encryption and can be decrypted by the corresponding software.

[0063] In this embodiment, the structured data packets corresponding to the regular events are encrypted using software encryption to obtain software-encrypted data packets. This process includes: first, the vehicle processor loads a preset software encryption algorithm, such as the AES-128 algorithm. The key for this algorithm is randomly generated by the system during the vehicle's first startup and stored in an encrypted file of the vehicle operating system. Then, the structured data packets of the regular events are divided into 128-bit data blocks according to the algorithm requirements. Each data block is then encrypted using the encryption algorithm, with an initialization vector used to ensure that the same data results in different encryption values. After encryption, the processor concatenates the encrypted data blocks and adds an algorithm identifier and initialization vector information to form the software-encrypted data packet.

[0064] In this embodiment, the software encryption key is automatically updated periodically through the vehicle system's security process. The update cycle is set to 7 days. During the update process, the validity of the new key is verified through a hash algorithm to prevent the key from being tampered with.

[0065] In this embodiment, the storage unit is an in-vehicle storage device, such as an embedded solid-state drive, used to store encrypted data packets. It is internally divided into multiple logical partitions, each with different access permissions. The user access partition is a partition within the storage unit divided according to data security levels, including a high-privilege access partition and a normal access partition. The high-privilege access partition requires hardware key verification for access, while the normal access partition can be accessed via a user password or system authorization.

[0066] In this embodiment, the hardware encrypted data packet and the software encrypted data packet are stored in different user access partitions within the storage unit. This includes: first, identifying the types of the hardware and software encrypted data packets; the hardware encrypted data packet corresponds to critical events, and the software encrypted data packet corresponds to regular events. Then, the partition management interface of the storage unit is called to write the hardware encrypted data packet to the high-privilege access partition. The access control list of this partition only allows the vehicle security chip to perform read and write operations after key verification; the user must trigger chip verification via a physical button to access it. The software encrypted data packet is written to the ordinary access partition, which is managed by the user account permissions of the vehicle system; the user can access it after entering the correct password or through biometric verification.

[0067] In this embodiment, the partition capacity of the storage unit is preset according to the frequency of event occurrence. The high-privilege access partition accounts for 30% of the total capacity, and the ordinary access partition accounts for 70%. It also supports dynamic adjustment. When the storage volume of the high-privilege partition reaches 90%, the system automatically compresses the earliest non-critical event data to free up space.

[0068] In this embodiment, differentiated encryption (hardware encryption vs. software encryption) and partitioned storage are used for critical events and routine events. This solves the problem of single data encryption methods and easy unauthorized access to important data in traditional dashcams. Hardware encryption provides a higher level of security for critical event data, while software encryption takes into account the storage efficiency of routine events. Partitioned storage enables fine-grained management of data access permissions. It not only ensures the immutability and confidentiality of critical event data (such as accidents and violations) but also meets the convenient access needs of routine events, thus balancing security and availability.

[0069] In some embodiments, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes: creating index information containing the event type, time base, and video scene features; associating the index information with the corresponding encrypted structured data packet; and storing the index information in the index area of ​​the storage unit.

[0070] In this embodiment, the index information is structured data used to quickly locate event records, containing three core types of information: event type, time reference, and video scene features. Event type is a classification of events, including critical events and routine events. The time reference is a time reference for the event's occurrence, which may be a specific point in time or a relative time range. Video scene features are visual features extracted from video clips, such as road type, weather conditions, and the presence of pedestrians.

[0071] In this embodiment, creating index information containing the event type, time reference, and video scene features includes: First, extracting the event type and time reference from the structured control instructions, ensuring that these two types of information are consistent with the structured data packet before encryption. Next, feature extraction is performed on the video segments within the time window using a lightweight image feature extraction algorithm to identify and record scene elements in the video, such as identifying information like "highway," "sunny day," and "no pedestrians" in the video frame; these elements collectively constitute the video scene features. Then, the event type, time reference, and video scene features are combined according to a preset format to form a complete index information, with each index information corresponding to a structured data packet.

[0072] In this embodiment of the application, in order to ensure the uniqueness of the index information, a unique identifier is assigned to each piece of index information. The identifier is generated by combining the event type code, the timestamp segment of the time base, and a random number. For example, the identifier of a critical event may be "K-202507241530-789", where "K" represents the critical event, "202507241530" is the timestamp segment of the time base, and "789" is a random number.

[0073] In this embodiment, associating the index information with the encrypted structured data packet establishes a correspondence between the two, enabling the corresponding encrypted data packet to be quickly located using the index information. The storage unit is a hardware device used to store data, and it has a dedicated area for storing index information, i.e., the index area. Access to the index area is faster than other storage areas, facilitating rapid retrieval.

[0074] In this embodiment, associating the index information with the corresponding encrypted structured data packet and storing the index information in the index area of ​​the storage unit includes: firstly, binding the unique identifier in the index information with the storage address of the encrypted structured data packet to form an association mapping table, which records the correspondence between identifiers and storage addresses. Then, the storage control module writes the association mapping table into the index area of ​​the storage unit, and simultaneously stores the index information itself in the corresponding position in the index area, ensuring that the index information and the mapping table are mutually associated. For example, the index information "K-202507241530-789" will be bound to the storage address "0x00012345" of the hardware encrypted data packet and stored together in the index area.

[0075] In this embodiment, the index area employs a data redundancy storage mechanism, storing each index entry and its associated mapping table simultaneously in two different physical locations within the index area. If data in one location is corrupted, it can be recovered from data in the other location, ensuring the reliability of the index information. Furthermore, the index area's storage space occupies 5% of the total storage unit capacity to meet the storage requirements of a large amount of index information.

[0076] In this embodiment, by creating index information containing event type, time base, and scene characteristics and associating it with encrypted data packets, the problem of low efficiency and reliance on a single timestamp in traditional dashcam data retrieval is solved. The index information provides multi-dimensional (type, time, scene) query basis for data retrieval, which greatly improves the retrieval speed and accuracy of event records. Users can quickly locate target events through keywords, avoiding the tedious operation of viewing videos segment by segment, and improving the convenience of data utilization.

[0077] In some embodiments, the method further includes an event retrieval step: receiving a user's voice retrieval command, parsing the retrieval intent in the voice retrieval command using natural language processing technology; extracting retrieval keywords corresponding to the retrieval intent; matching the event type, time base, and video scene features corresponding to the index information based on the retrieval keywords; retrieving the corresponding event record from the storage unit, decrypting the retrieved event record, and returning retrieval data containing video clips, sensor data, and structured control commands within the time window.

[0078] In this embodiment, the voice retrieval command is a command issued by the user via voice to search for specific event records, such as "find yesterday's illegal lane change video." Natural language processing technology, used to analyze and understand human language, is used in this step to process the voice retrieval command to clarify the user's retrieval intent. The retrieval intent is the retrieval purpose the user hopes to achieve through the voice retrieval command, such as finding events at a specific time or of a specific type.

[0079] In this embodiment, receiving a user's voice search command and parsing the search intent in the voice search command using natural language processing technology includes: first, receiving the user's voice search command through a voice acquisition device and converting it into text form. Then, inputting the text-based voice search command into a natural language processing module, which performs word segmentation processing on the text, breaking the complete command into multiple words or phrases. For example, "find pedestrian crossing records from last week" is broken down into "find," "last week," "pedestrian crossing," and "record." Next, a semantic understanding algorithm analyzes the word segmentation results to identify the user's search intent. For example, from the above word segmentation results, it can be parsed that the user's search intent is to find records of pedestrian crossing events that occurred last week.

[0080] In this embodiment of the application, to ensure the accuracy of parsing, the natural language processing module pre-sets a semantic model related to driving events. This model is trained by a large number of voice command samples in driving scenarios and can effectively understand the retrieval needs of different expressions.

[0081] In this embodiment of the application, the search keywords are core words or phrases that can characterize the search needs extracted from the search intent. These words or phrases are associated with event types, time bases, and video scene features in the index information, such as "illegal lane change", "yesterday", and "rainy day".

[0082] In this embodiment of the application, extracting the search keywords corresponding to the search intent includes: based on the parsed search intent, filtering out core information related to event records from the segmented voice search command text. For search intents involving event types, extracting corresponding event type words as search keywords, for example, extracting "running a red light" from "find all red light running records"; for search intents involving time, extracting corresponding time words as search keywords, for example, extracting "three days ago" from "find events from three days ago"; for search intents involving scene features, extracting corresponding scene feature words as search keywords, for example, extracting "foggy day" from "find driving records in foggy weather".

[0083] In this embodiment of the application, the keywords are standardized during the extraction process. For example, "yesterday" is uniformly converted to "yesterday", and "illegal lane change" and "lane change over solid lines" are uniformly classified as "illegal lane change" to ensure the accuracy of subsequent matching.

[0084] In this embodiment, the event type is a specific classification of the event, such as illegal lane changing or pedestrian crossing. The time reference is the time when the event occurs. Video scene features are scene information in the video footage at the time the event occurs, such as weather conditions and road type.

[0085] In this embodiment, matching the search keywords with the corresponding event type, time reference, and video scene features in the index information includes: comparing the extracted search keywords with the index information in the storage unit's index area. For event type keywords, matching is performed with the event type in the index information; for time keywords, matching is performed with the time reference in the index information, for example, matching "last week" with records within a week in the index information; for scene feature keywords, matching is performed with the video scene features in the index information. Through multi-dimensional matching, index information that matches the search keywords is filtered out.

[0086] In this embodiment of the application, a fuzzy matching algorithm is used in the matching process. When the search keywords and the content in the index information are not completely consistent but semantically similar, they can still be matched. For example, "foggy day" and "heavy fog weather" can be regarded as a match, so as to improve the flexibility of retrieval.

[0087] In this embodiment, decryption is the process of restoring encrypted event records to their original data. For hardware-encrypted data packets of critical events, a corresponding hardware decryption method is used; for software-encrypted data packets of regular events, a corresponding software decryption method is used. The retrieved data is data containing complete event information returned to the user after decryption, including video clips within the time window, sensor data, and structured control commands.

[0088] In this embodiment, the corresponding event record is retrieved from the storage unit, decrypted, and returned as search data containing video clips, sensor data, and structured control instructions within the time window. This includes: determining the storage location of the corresponding event record in the storage unit based on the matched index information, and then retrieving the event record from the storage unit through a storage interface. For each retrieved event record, an appropriate decryption method is selected based on its event type. If it is a hardware-encrypted data packet for a critical event, it is decrypted using the vehicle security chip; if it is a software-encrypted data packet for a regular event, it is decrypted using a preset software decryption algorithm. After decryption, the video clips, corresponding sensor data, and structured control instructions within the time window are integrated into search data and returned to the user, for example, by displaying the video clips on an in-vehicle display screen, while also providing viewing options for the sensor data and structured control instructions.

[0089] In this embodiment of the application, in order to ensure the security of data transmission, a data verification mechanism is adopted during the process of returning the retrieved data to verify the integrity of the transmitted data and ensure that the retrieved data received by the user is accurate.

[0090] In this embodiment, the complexity and inefficiency of data retrieval in traditional dashcams are solved through voice retrieval command parsing, keyword matching, index comparison, and decryption return. The technical effects are as follows: natural language processing enables convenient voice retrieval; keyword matching and index information association significantly improve retrieval accuracy; differentiated decryption ensures data access security; and finally, complete event data is quickly returned, improving the efficiency of event tracing and meeting users' needs for rapid querying of historical records.

[0091] In some embodiments, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes: determining the event type corresponding to the structured data packet based on a dynamic threshold; when the event type is a critical event, the hardware-encrypted data packet preferentially occupies the high-privilege access partition in the storage unit.

[0092] In this embodiment, the dynamic threshold is a judgment standard that is dynamically adjusted based on the characteristics of the driving scene and the degree of impact of the event. The threshold is updated in real time in combination with factors such as the potential risk level of the event and the importance of the data. For example, the judgment threshold for a collision event will be lower than the threshold for a regular landscape recording event.

[0093] In this embodiment, determining the event type corresponding to the structured data packet based on a dynamic threshold includes: first, extracting event-related feature parameters from the structured data packet, including event duration, vehicle motion state change magnitude (such as acceleration change), and frequency of abnormal targets appearing in the video footage. Then, these feature parameters are compared with a preset dynamic threshold. When the feature parameters exceed the dynamic threshold, the corresponding event type is determined to be a critical event; when the feature parameters do not exceed the dynamic threshold, it is determined to be a regular event. For example, when the vehicle acceleration change exceeds -6 meters per second squared (dynamic threshold), combined with the feature of a collision in the video footage, the event is determined to be a critical event.

[0094] In this embodiment, the dynamic threshold update mechanism is related to the vehicle driving environment. The threshold will be appropriately reduced in highway scenarios and appropriately increased in urban road congestion scenarios to adapt to the event judgment requirements in different scenarios.

[0095] In this embodiment, the hardware-encrypted data packet is a structured data packet corresponding to a critical event after being processed by hardware encryption. The storage unit is an in-vehicle storage device used to store data, and the high-privilege access partition is a storage area within the storage unit with a strict access control mechanism, accessible only through specific hardware verification or authorization. Priority occupancy means that when storage resources are limited, hardware-encrypted data packets obtain storage space in the high-privilege access partition before software-encrypted data packets.

[0096] In this embodiment, when the event type is a critical event, the hardware encrypted data packet preferentially occupies the high-privilege access partition in the storage unit. This includes: the storage unit's management module monitors the storage space usage of the high-privilege access partition in real time. When a hardware encrypted data packet to be stored is detected, if there is remaining space in the high-privilege access partition, it is directly stored in that partition; if the remaining space is insufficient, the management module releases some storage space in the order of "regular event data first, then early critical event data (non-urgently retained data)" to prioritize the storage of the current hardware encrypted data packet. For example, when the high-privilege access partition has 500MB of remaining space, and the hardware encrypted data packet to be stored is 800MB, the system will automatically delete the earliest non-essential regular event encrypted data (if any) or early low-priority critical event data in that partition until at least 800MB of space is freed up for the current hardware encrypted data packet to be stored.

[0097] In this embodiment, the priority occupancy mechanism is implemented through the underlying scheduling algorithm of the storage unit. This algorithm assigns the highest storage priority identifier to hardware encrypted data packets, ensuring that critical event data will not be lost due to insufficient space in a multi-task storage scenario.

[0098] In this embodiment, event types are determined based on dynamic thresholds, ensuring that hardware-encrypted data packets for critical events preferentially occupy high-privilege access partitions. This solves the problems of unreasonable storage resource allocation and potential loss of critical data due to insufficient space in traditional dashcams. The dynamic thresholds enable intelligent determination of event importance, and the priority allocation mechanism for high-privilege partitions guarantees the storage priority of critical event data, preventing the loss of important records due to insufficient storage resources. This ensures the reliable retention of critical event data such as accidents and violations, providing tamper-proof evidence for subsequent liability determination.

[0099] S5. Analyze the video segments in the event record in real time, and trigger a voice warning and automatically save the associated video when a preset scene is detected.

[0100] In some embodiments, the real-time analysis of video segments in the event log, triggering a voice alert and automatically saving associated videos when a preset scene is detected, includes: acquiring video segments and corresponding motion state data contained in the event log; performing real-time analysis of the video segments using a lightweight computer vision model, and identifying target objects and dynamic features in the scene by combining the motion state data; triggering a voice alert corresponding to the preset scene when the identification result matches the preset scene; synchronously extracting video segments before and after the preset scene as associated videos, packaging and storing the associated videos, corresponding motion state data, and the voice alert, and updating them in the event log.

[0101] In this embodiment, obtaining the video segments and corresponding motion state data contained in the event records includes: firstly, retrieving stored video segments from the event records in the storage unit; these video segments are video content within the time window of the previously marked event. Simultaneously, retrieving the motion state data corresponding to the video segment through time reference association ensures that the time range of the motion state data completely matches the time range of the video segment, providing a complete data foundation for subsequent real-time analysis.

[0102] In this embodiment of the application, the synchronization between the video clip and the motion state data is confirmed through a timestamp verification mechanism during the retrieval process. If the time deviation exceeds 0.1 seconds, the association retrieval is performed again to ensure data consistency.

[0103] In this embodiment, the lightweight computer vision model is a high-efficiency image recognition model suitable for vehicle-mounted terminals. Its model parameters are simplified, reducing computational resource consumption while maintaining recognition accuracy. The target object is an object related to driving in the video frame, such as other vehicles, pedestrians, and traffic signs. Dynamic features are the motion changes of the target object, such as changes in vehicle speed and the movement trajectory of pedestrians.

[0104] In this embodiment, a lightweight computer vision model is used to analyze the video clips in real time. Combined with motion state data, target objects and dynamic features in the scene are identified. This includes: inputting the video clip frame by frame into the lightweight computer vision model; the model extracts features from each frame to identify target objects in the scene; and analyzing the dynamic features of the target objects through comparison of consecutive frames. Simultaneously, motion state data is used as an auxiliary reference. For example, when the model identifies a vehicle ahead, it combines the vehicle's acceleration data to determine whether the vehicle is braking suddenly, thereby improving the accuracy of dynamic feature recognition. For instance, if motion state data shows a significant decrease in the vehicle's acceleration while a vehicle ahead is identified in the video, it can help determine that the vehicle ahead is decelerating.

[0105] In this embodiment, the preset scenarios are pre-defined driving scenarios requiring warnings, such as pedestrians crossing the road, vehicles braking suddenly ahead, traffic lights turning red, etc. Each preset scenario corresponds to specific warning content. Voice warnings are prompts issued by the in-vehicle voice broadcasting device to remind the driver to pay attention to the corresponding scenarios.

[0106] In this embodiment, when the recognition result matches a preset scenario, a voice warning corresponding to the preset scenario is triggered. This includes comparing the recognized target object and dynamic features with the feature library of the preset scenario. When the matching degree between the two exceeds a set threshold (e.g., 85%), it is determined that the recognition result matches the preset scenario. Then, a speech synthesis module is invoked to generate voice warning content corresponding to the preset scenario. For example, when the scenario of "pedestrian crossing the road" is matched, the voice message "Attention: Pedestrian crossing ahead, please slow down and give way" is generated and broadcast through the vehicle's loudspeaker.

[0107] In this embodiment, the associated video is a video segment containing a period of time before and after a preset scene occurs. Its time range is typically set according to the scene type. For example, for a pedestrian crossing scene, the associated video may contain video content from 5 seconds before the scene occurs to 10 seconds after. Packaging and storage is the process of integrating the associated video, motion status data, and voice warnings according to a certain format and then storing them.

[0108] In this embodiment, video clips before and after the occurrence of the preset scene are extracted simultaneously as associated videos. These associated videos, along with corresponding motion state data and the voice warning, are packaged and stored, and then updated in the event log. This includes: based on the occurrence time of the preset scene, a video clip of a predetermined duration (e.g., 5 seconds) is traced back as the content before the scene, and a video clip of a predetermined duration (e.g., 10 seconds) is extended forward as the content after the scene. These two parts are merged with the video clip at the time of the scene to form associated videos. Simultaneously, motion state data corresponding to the time range of the associated videos is retrieved, and the generation time and content of the voice warning are recorded. Then, the associated videos, motion state data, and voice warnings are integrated into a single data unit. This data unit is encrypted and stored in a storage unit, and the index information of the event log is updated to ensure that the associated videos and related data can be quickly retrieved subsequently.

[0109] In this embodiment, the time range of the associated video can be adjusted according to the severity of the preset scenario. For high-risk scenarios (such as collision warnings), the time range will be appropriately expanded to retain a more complete event process.

[0110] In this embodiment, a lightweight computer vision model is used to analyze video in real time, and motion state data is combined to identify preset scenes and trigger warnings. This solves the problems of high false alarm rate and untimely warnings caused by traditional dashcams relying on single visual detection. Multimodal data fusion (video + motion state) improves the accuracy of scene recognition, the lightweight model ensures real-time performance, automatic saving of associated videos enables complete tracing of dangerous scenes, and voice warnings provide timely reminders to the driver, reducing driving risks and improving active safety performance.

[0111] The aforementioned intelligent driving recorder based on voice AI achieves hands-free operation throughout the entire process through a voice interaction module. It combines edge-side ASR and NLP technologies to parse complex commands, avoiding the safety hazards of manual operation while driving. By integrating visual, voice, and IMU data from multiple sensors, it accurately defines event time windows, solving the problem of missed detections by a single sensor. It employs hierarchical storage combining hardware encryption (for critical events) and software encryption (for routine events), along with physical isolation partitions and dynamic keys, ensuring data security and compliance with privacy regulations. A semantic index containing event type, time reference, and scene features is constructed, supporting voice retrieval for quick location of target records, replacing traditional manual browsing. Real-time scene detection and voice warnings are achieved through a local lightweight visual model, combined with a priority storage mechanism triggered by dynamic thresholds, ensuring priority storage of critical data. The modular system design adapts to different vehicle models and can be expanded with more intelligent driving assistance functions, comprehensively improving the intelligence and practicality of the driving recorder.

[0112] Based on the same inventive concept, this application also provides a voice AI-based intelligent driving recorder system for implementing the aforementioned voice AI-based intelligent driving recorder method. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations of one or more voice AI-based intelligent driving recorder system embodiments provided below can be found in the limitations of the voice AI-based intelligent driving recorder method described above, and will not be repeated here.

[0113] In one embodiment, such as Figure 3 As shown, a smart driving recorder system based on voice AI is provided, including: The voice interaction module 101 is used to acquire user voice commands in real time through a voice acquisition device; The instruction parsing module 102 is used to parse the operation intent and event marker request in the user's voice instruction and generate a structured control instruction containing event type and time reference. The multi-sensor fusion module 103 is used to determine the time window of the event to be marked based on the time reference, combining the motion state data of the IMU sensor and the video data collected by the dashcam. The event log generation module 104 is used to package and store the video clips, sensor data and structured control instructions within the time window to generate an event log; The associated video storage module 105 is used to analyze video segments in the event record in real time, and trigger a voice warning and automatically save the associated video when a preset scene is detected.

[0114] The modules in the aforementioned voice AI-based intelligent driving recorder system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0115] In one embodiment, a computer device is provided, which may be a sample analyzer, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, communication interface, display screen, and input system connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input system can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0116] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: the computer program performs the steps in the above method embodiments.

[0118] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0119] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and / or features involved in this application are all information, data and / or features authorized by the user or fully authorized by all parties.

[0120] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A voice AI-based intelligent driving recording method, characterized by, The method comprises: Real-time acquisition of user voice instructions by a voice acquisition device; Parsing the operation intention and event marking request in the user voice instruction to generate a structured control instruction containing an event type and a time reference; According to the time reference, combined with the motion state data of the IMU sensor and the video data collected by the driving recorder, the time window of the event to be marked is determined; The video segment, sensor data and structured control instruction in the time window are packaged and stored to generate an event record; Real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving the associated video when a preset scene is detected.

2. The method of claim 1, wherein, The parsing of the operation intention and event marking request in the user voice instruction to generate a structured control instruction containing an event type and a time reference comprises: Using natural language processing technology to extract the operation intention in the user voice instruction; According to the operation intention, the key information in the event marking request is matched combined with the preset instruction template; According to the key information, the event type and the time reference are determined to generate the structured control instruction.

3. The method of claim 1, wherein, According to the time reference, combined with the motion state data of the IMU sensor and the video data collected by the driving recorder, the time window of the event to be marked is determined, comprising: According to the time reference, the motion state data of the IMU sensor is associated to determine the duration range of the vehicle dynamic event corresponding to the event marking request; According to the duration range, combined with the screen changes related to the vehicle dynamic event in the video data collected by the driving recorder, the start boundary and end boundary of the time window of the event to be marked are determined.

4. The method of claim 1, wherein, The video segment, sensor data and structured control instruction in the time window are packaged and stored to generate an event record, comprising: Obtaining the video segment, corresponding sensor data and structured control instruction in the time window; Compressing and encoding the video segment, synchronizing and aligning the sensor data with the time reference of the video segment; Integrating the synchronized video segment, sensor data and structured control instruction to form a structured data packet, wherein the event type includes key events and regular events; After encryption processing, the structured data packet is stored in a storage unit to generate an event record.

5. The method of claim 1, wherein, The real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving the associated video when a preset scene is detected, comprises: Obtaining the video segment and corresponding motion state data contained in the event record; Real-time analysis of the video segment by a lightweight computer vision model, combined with the motion state data, to identify target objects and dynamic characteristics in the screen; When the identification result matches the preset scene, trigger the voice warning corresponding to the preset scene; Synchronously extract the video segment before and after the occurrence of the preset scene as the associated video, package and store the associated video, corresponding motion state data and voice warning, and update to the event record.

6. The method of claim 4, wherein, The structured data packet is stored in the storage unit after encryption processing, and an event record is generated, including: The structured data packet corresponding to the key event is encrypted based on a hardware encryption method to obtain a hardware encrypted data packet; The structured data packet corresponding to the regular event is encrypted based on a software encryption method to obtain a software encrypted data packet; The hardware encrypted data packet and the software encrypted data packet are respectively stored in different user access partitions in the storage unit.

7. The method of claim 5, wherein, The structured data packet is stored in the storage unit after encryption processing, and an event record is generated, and further including: Index information containing the event type, time reference, and video scene feature is created; The index information is associated with the corresponding encrypted structured data packet, and the index information is stored in an index area in the storage unit.

8. The method of claim 6, wherein, The structured data packet is stored in the storage unit after encryption processing, and an event record is generated, and further including: Based on a dynamic threshold, the event type corresponding to the structured data packet is determined, and when the event type is a key event, the hardware encrypted data packet preferentially occupies a high-privilege access partition in the storage unit.

9. The method of claim 7, wherein, The method further includes an event retrieval step: A user's voice retrieval instruction is received, and a retrieval intent in the voice retrieval instruction is parsed using natural language processing technology; A retrieval keyword corresponding to the retrieval intent is extracted; The retrieval keyword is matched with the corresponding event type, time reference, and video scene feature in the index information; The corresponding event record is retrieved from the storage unit, the retrieved event record is decrypted, and retrieval data containing a video segment, sensor data, and structured control instructions within a time window are returned.

10. A voice AI-based intelligent driving record system, characterized by, The system includes: A voice interaction module for real-time acquisition of user voice instructions through a voice acquisition device; An instruction analysis module for analyzing operation intent and event marking requests in the user voice instructions to generate structured control instructions containing event types and time references; A multi-sensor fusion module for determining a time window of a to-be-marked event according to the time reference, combining motion state data of an IMU sensor and video data collected by a car data recorder; An event record generation module for packaging and storing video segments, sensor data, and structured control instructions within the time window to generate an event record; An associated video storage module for real-time analysis of video segments in the event record, triggering a voice warning and automatically saving associated videos when a preset scene is detected.

Images