Intelligent driving recording method and system based on voice AI

Abstract

The application discloses an intelligent driving record method and system based on voice AI, and relates to the technical field of intelligent driving record. The method comprises the following steps: acquiring a user voice instruction through a voice acquisition device, and identifying the voice instruction through a vehicle-mounted lightweight ASR model; analyzing the instruction to generate a structured control instruction containing an event type and a time reference; determining a time window of a to-be-labeled event in combination with the time reference, IMU sensor motion data and video data; packaging, encrypting and storing video clips, sensor data and the structured control instruction in the window to generate an event record; and analyzing the video in real time through a local lightweight computer vision model, and triggering a voice warning and automatically saving associated video when a preset dangerous scene is detected. The application realizes voice interaction control, accurate event labeling, real-time danger warning and efficient data management of driving record, and improves the convenience, accuracy and safety of driving record.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving record, in particular to an intelligent driving record method and system based on voice AI. BACKGROUND

[0002] Traditional driving records rely on camera video recording and local storage, and need to manually operate or physically press keys to trigger key event recording, adopt a cyclic coverage storage mode, and can only achieve basic video playback. It has the problems of inconvenient operation, easy distraction of attention during driving by manual operation, and massive video browsing by manual operation, which cannot quickly locate specific segments through semantics, and the degree of intelligence is insufficient.

[0003] Some existing intelligent driving record technologies attempt to combine ADAS or AI technologies for optimization, such as vision-based event detection, mobile APP linkage, and basic voice control. However, there are high false alarm rates and dependence on computing power for vision detection, mobile linkage still requires manual operation, and basic voice control cannot understand complex semantics; voice and video data are isolated and cannot be implemented for semantic retrieval; real-time performance is poor, AI analysis relies on the cloud, and weak local processing capability leads to early warning delay; multi-sensor data fusion is insufficient, which cannot meet the needs of users for convenient interaction and accurate intelligent analysis. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an intelligent driving record method and system based on voice AI to solve the problems raised in the background art.

[0005] In a first aspect, the present application provides an intelligent driving record method based on voice AI, characterized in that the method comprises:

[0006] real-time acquisition of user voice instructions by a voice acquisition device;

[0007] parsing the operation intention and event marking request in the user voice instruction to generate a structured control instruction containing an event type and a time reference;

[0008] determining the time window of the event to be marked according to the time reference, in combination with the motion state data of the IMU sensor and the video data collected by the driving recorder;

[0009] packaging and storing the video segments, sensor data and structured control instruction in the time window to generate an event record;

[0010] real-time analysis of the video segments in the event record, triggering a voice warning and automatically saving the associated video when a preset scene is detected.

[0011] In one of the embodiments, the operation intention and the event marking request in the user voice instruction are parsed to generate a structured control instruction containing an event type and a time reference, including:

[0012] The operation intention in the user voice instruction is extracted by using a natural language processing technology;

[0013] According to the operation intention, the key information in the event marking request is matched in combination with a preset instruction template;

[0014] According to the key information, the event type and the time reference are determined to generate the structured control instruction.

[0015] In one of the embodiments, according to the time reference, the motion state data of an IMU sensor and the video data collected by a driving recorder are combined to determine a time window of a to-be-marked event, including:

[0016] According to the time reference, the motion state data of the IMU sensor are associated to determine the duration range of a vehicle dynamic event corresponding to the event marking request;

[0017] According to the duration range, the video data collected by the driving recorder related to the vehicle dynamic event is combined to determine the start boundary and the end boundary of the time window of the to-be-marked event.

[0018] In one of the embodiments, the video segment in the time window, the sensor data and the structured control instruction are packaged and stored to generate an event record, including:

[0019] The video segment in the time window, the corresponding sensor data and the structured control instruction are obtained;

[0020] The video segment is compressed and encoded, and the sensor data are synchronized and aligned with the time reference of the video segment;

[0021] The synchronized video segment, the sensor data and the structured control instruction are associated and integrated to form a structured data packet, wherein the event type includes a key event and a regular event;

[0022] After the structured data packet is encrypted, it is stored in a storage unit to generate an event record.

[0023] In one of the embodiments, the video segment in the event record is analyzed in real time, and when a preset scene is detected, a voice warning is triggered and the associated video is automatically saved, including:

[0024] The video segment and the corresponding motion state data contained in the event record are obtained;

[0025] The video clips are analyzed in real time using a lightweight computer vision model, and the target objects and dynamic features in the scene are identified by combining the motion state data.

[0026] When the recognition result matches a preset scenario, a voice warning corresponding to the preset scenario is triggered;

[0027] Simultaneously extract video clips before and after the occurrence of the preset scene as associated videos, package and store the associated videos with the corresponding motion state data and the voice warning, and update the event record.

[0028] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record includes:

[0029] The structured data packets corresponding to the key events are encrypted using hardware encryption methods to obtain hardware-encrypted data packets;

[0030] The structured data packets corresponding to the regular events are encrypted using software encryption methods to obtain software-encrypted data packets;

[0031] The hardware encrypted data packet and the software encrypted data packet are stored in different user access partitions in the storage unit, respectively.

[0032] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes:

[0033] Create index information that includes the event type, time base, and video scene features;

[0034] The index information is associated with the corresponding encrypted structured data packet, and the index information is stored in the index area of ​​the storage unit.

[0035] In one embodiment, the step of encrypting the structured data packet and storing it in the storage unit to generate an event record further includes:

[0036] Based on a dynamic threshold, the event type corresponding to the structured data packet is determined. When the event type is a critical event, the hardware-encrypted data packet preferentially occupies the high-privilege access partition in the storage unit.

[0037] In one embodiment, the method further includes an event retrieval step:

[0038] Receive the user's voice search command and use natural language processing technology to parse the search intent in the voice search command;

[0039] extracting a search keyword corresponding to the search intention;

[0040] matching the search keyword with a corresponding event type, time reference and video scene feature in the index information;

[0041] retrieving a corresponding event record from the storage unit, decrypting the retrieved event record, and returning search data containing a video segment, sensor data and structured control instructions within a time window.

[0042] In a second aspect, the present application also provides an intelligent driving record system based on voice AI, characterized in that the system comprises:

[0043] a voice interaction module for acquiring user voice instructions in real time through a voice acquisition device;

[0044] an instruction analysis module for analyzing the operation intention and event marking request in the user voice instructions, and generating a structured control instruction containing an event type and a time reference;

[0045] a multi-sensor fusion module for determining a time window of the event to be marked according to the time reference, in combination with the motion state data of the IMU sensor and the video data collected by the driving recorder;

[0046] an event record generation module for packaging and storing the video segment, sensor data and structured control instruction within the time window, and generating an event record;

[0047] an associated video storage module for analyzing the video segment in the event record in real time, triggering a voice warning and automatically saving the associated video when a preset scene is detected.

[0048] In a third aspect, the present application also provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the intelligent driving record method based on voice AI in any embodiment of the present application.

[0049] In the intelligent driving record method and system based on voice AI, hands-free operation is realized through the voice interaction module, complex instructions are analyzed by combining end-side ASR and NLP technology, and the safety hazards of manual operation during driving are avoided; through multi-sensor fusion, visual, voice, and IMU data are integrated, the event time window is accurately determined, and the single sensor missing detection problem is solved; hierarchical storage is adopted by combining hardware encryption (key event) and software encryption (regular event), and physical isolation partition and dynamic key are used to ensure data security and comply with privacy regulations; a semantic index containing event type, time reference, and scene characteristics is constructed to support voice search to quickly locate the target record and replace traditional manual browsing; real-time scene detection and voice warning are realized through a local lightweight visual model, and a priority storage mechanism triggered by a dynamic threshold is combined to ensure that key data is saved in priority, the modular design of the system module is suitable for different vehicle models, more intelligent driving assistance functions can be expanded, and the intelligence and practicality of driving records are improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of an intelligent driving record method based on voice AI according to an exemplary embodiment;

[0051] Figure 2 is a flowchart of generating a structured control instruction according to an exemplary embodiment;

[0052] Figure 3 is a structural block diagram of an intelligent driving record system based on voice AI according to an exemplary embodiment;

[0053] Figure 4 is an internal structure diagram of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0055] The terms “first”, “second”, “third”, etc., in the embodiments of the present application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with “first”, “second”, “third” can explicitly or implicitly include at least one of the features. In the description of the present application, “at least one” is used to indicate one or more; “a plurality of” means at least two, for example, two, three, etc., unless otherwise explicitly specified. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0056] Reference to “embodiments” herein means that the specific features, structures or properties described in connection with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily a separate or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] As shown in Figure 1 The embodiments of the present application provide an intelligent driving record method based on voice AI, comprising the following steps:

[0058] S1, acquiring user voice instructions in real time through a voice acquisition device.

[0059] In the embodiments of the present application, the voice acquisition device is a microphone array that supports far-field voice signal acquisition, and the voice recognition process is completed locally on the vehicle terminal, wherein the microphone array performs noise reduction processing based on the in-vehicle acoustic environment.

[0060] In the embodiments of the present application, the voice acquisition device refers to a microphone array (not a single microphone) deployed inside the vehicle, which has the following characteristics: it can accurately capture voice instructions from the driver's seat (usually 1-2 meters away from the device) and adapt to normal conversation volume in the vehicle; noise reduction processing is performed on the in-vehicle acoustic environment (such as engine noise, air conditioner sound, road bump sound), interference is filtered through beamforming, echo cancellation and other technologies to ensure clear voice signals.

[0061] S2, analyzing the operation intent and event marker request in the user voice instructions to generate structured control instructions containing event type and time reference.

[0062] In some embodiments, as Figure 2As shown, the operation intention and event marker request in the user voice instruction are parsed, and a structured control instruction containing an event type and a time reference is generated, including:

[0063] S21, an operation intention in the user voice instruction is extracted by using a natural language processing technology;

[0064] S22, according to the operation intention, key information in the event marker request is matched in combination with a preset instruction template;

[0065] S23, according to the key information, the event type and the time reference are determined, and the structured control instruction is generated.

[0066] In the embodiment of the application, a lightweight ASR model deployed in a vehicle terminal is used for voice recognition, wherein the lightweight ASR model is a lightweight voice recognition model adapted to a vehicle scene.

[0067] In the embodiment of the application, the lightweight ASR model is a voice recognition model adapted to the hardware resources (such as computing power, storage, and power consumption limit) of the vehicle terminal, for example, a lightweight RNN-T or Transformer model, which has the following characteristics: the model runs on the vehicle terminal (such as a driving recorder main control chip), does not need to rely on cloud computing, and avoids recognition failure caused by network delay or network disconnection; special training is performed on common instructions in the driving scene (such as "record video", "find accident clip", and "warn danger"), and the recognition accuracy (especially the recognition accuracy of driving-related words) is optimized; the model parameter size is small (usually less than 100M), the recognition delay is controlled within 500ms, and the vehicle system computing power is not occupied.

[0068] In the embodiment of the application, during driving, the user finds that a vehicle in front of him changes lanes in violation of rules, and says the instruction: "Little D, record the just-mentioned illegal lane change": the lightweight ASR model of the vehicle terminal (such as the optimized MobileASR) quickly converts the voice into text: "Little D, record the just-mentioned illegal lane change".

[0069] In the embodiment of the application, the natural language processing technology is a technology for analyzing and understanding the text instruction after voice recognition, including word segmentation processing and semantic analysis.

[0070] In the embodiments of the present application, the operation intention in the user voice instruction is extracted by using natural language processing technology, which includes: first, input the text instruction converted by the voice collection device into the natural language processing module. The module first splits the text instruction into independent semantic units by a word segmentation algorithm, for example, "record the recent sudden braking event" is split into "record" "recent" "sudden braking" "event". Then, the semantic understanding model is analyzed by the pre-trained semantic understanding model, and the core action words such as "record", "find", "mark" are identified, so as to determine whether the operation intention of the user is event recording, information retrieval or other types. For example, when the user instruction is "D, find the accident video of yesterday", the word segmentation result is "find" "yesterday" "accident" "video", and the semantic understanding model identifies that "find" corresponds to the operation intention of information retrieval.

[0071] In the embodiments of the present application, the preset instruction template is a standardized format pre-set for common instruction types in the driving scene, and each operation intention corresponds to a specific template. For example, the event recording type template is "[operation intention]+[event description]+[time description]", and the information retrieval type template is "[operation intention]+[event type]+[time range]". The key information refers to the core content in the event marking request for determining the event attribute and the time range, such as the event type (illegal lane changing, pedestrian crossing), the time description (recent, 5 minutes ago), etc.

[0072] In the embodiments of the present application, the key information in the event marking request is matched according to the operation intention and the preset instruction template, which includes: first, the corresponding preset instruction template is called from the template library according to the extracted operation intention. For example, when the operation intention is event recording, the "record+[event type]+[time description]" template is called. Then, the user's event marking request text is structurally matched with the template, and the key information is extracted by keyword mapping. For example, after the user instruction "record the illegal lane changing of the vehicle in front 3 minutes ago" is matched with the event recording template, the key information is extracted: the event type related description is "illegal lane changing", and the time description is "3 minutes ago".

[0073] In the embodiments of the present application, the event type is a specific category based on the event description in the key information, which is divided into key events (such as collision, illegal behavior, pedestrian crossing, etc. which need to be stored in priority) and regular events (such as landscape recording, road condition observation, etc.). The time reference is a specific time parameter converted according to the time description in the key information, which is divided into absolute time (such as July 24, 2025 15:30:00) and relative time range (such as 30 seconds before the current time to the current time). The structured control instruction is a standardized data format integrating the operation intention, the event type and the time reference, which usually adopts the JSON structure and includes the fields of "operation intention", "event type", "time reference start", "time reference end", etc.

[0074] In the embodiments of the present application, the event type and the time reference are determined according to the key information, and the structured control instruction is generated, including: first, converting the event description in the key information into a standardized event type through an event type mapping table, for example, mapping “illegal lane changing” to “traffic violation-lane changing” in the key event, and mapping “scenery along the way” to “environmental record” in the regular event. Then, the time reference is calculated according to the time description: if the time description is absolute time (such as “15:20”), it is directly converted into a time stamp; if it is relative time (such as “just now” or “5 minutes ago”), the time range (current time-5 minutes to current time) is calculated in combination with the current system time. For example, if the time description in the key information is “3 minutes ago”, and the current system time is 15:30:00, the time reference is calculated as 15:27:00 to 15:30:00. Finally, the operation intention, the determined event type and the time reference are integrated into a structured control instruction according to a preset format, for example: { "operation intention": "event record", "event type": "key event-illegal lane changing", "time reference start": "2025-07-24 15:27:00", "time reference end": "2025-07-24 15:30:00"}.

[0075] In the embodiments of the present application, the operation intention is accurately extracted through natural language processing technology, and the key information is matched in combination with a preset template, and finally a structured control instruction is generated, solving the problems of ambiguous semantics and inaccurate response in traditional driving records, realizing deep understanding of complex natural language instructions, ensuring accurate conversion of operation intention and event marking request, providing standardized basis for subsequent time window determination and data storage, improving the accuracy and efficiency of voice interaction, and reducing the recording deviation caused by misunderstanding of instructions.

[0076] S3, according to the time reference, in combination with the motion state data of the IMU sensor and the video data collected by the driving recorder, the time window of the event to be marked is determined.

[0077] In some embodiments, according to the time reference, in combination with the motion state data of the IMU sensor and the video data collected by the driving recorder, the time window of the event to be marked is determined, including: according to the time reference, the motion state data of the IMU sensor is associated to determine the duration range of the vehicle dynamic event corresponding to the event marking request; according to the duration range, in combination with the screen changes related to the vehicle dynamic event in the video data collected by the driving recorder, the start boundary and the end boundary of the time window of the event to be marked are determined.

[0078] In the embodiments of the present application, the motion state data of the IMU sensor refers to the vehicle motion parameters collected by the inertial measurement unit, including acceleration, angular velocity, heading angle, etc., which can reflect the dynamic changes of the vehicle such as acceleration, deceleration, turning, and sudden braking. The time reference is the time reference of the event occurrence, which can be a specific time point or a relative time range. The duration range of the vehicle dynamic event refers to the time span experienced from the start to the end of the event.

[0079] In the embodiments of the present application, according to the time reference, the motion state data of the IMU sensor is associated to determine the duration range of the vehicle dynamic event corresponding to the event marking request, which includes: first, the IMU sensor motion state data corresponding to the time reference is retrieved from the storage unit, and the association mapping of the time reference and the motion state data is established, that is, the motion state data segment corresponding to the time reference is determined. Then, the data segment is analyzed for features, and the motion features related to the event marking request are identified, for example, for the "sudden braking event", the feature of the acceleration changing greatly in a short time is identified. Then, according to the starting point and the ending point of the motion features, the duration range of the vehicle dynamic event is calculated, for example, the time interval from the start of the acceleration to the recovery of the sudden braking event is 3 seconds, and the duration range is 3 seconds.

[0080] In the embodiments of the present application, in order to ensure the accuracy of the judgment, a motion feature threshold value is set, for example, the acceleration threshold value of the sudden braking event is-5 m / s2, when the acceleration is detected to be less than the threshold value, it is determined that the sudden braking event starts, and when the acceleration rises to-1 m / s2or more, it is determined that the event ends.

[0081] In the embodiments of the present application, the picture change related to the vehicle dynamic event in the video data refers to the visual information change in the video shot by the driving recorder that can reflect the occurrence and development of the event, for example, the distance change of the front vehicle when the vehicle brakes suddenly, the position offset of the vehicle when the vehicle changes lanes illegally, etc. The time window refers to the time interval of the video segment containing the complete event to be marked, the starting boundary is the start time of the time window, and the ending boundary is the end time of the time window.

[0082] In the embodiments of the present application, according to the duration range, in combination with the picture changes related to the vehicle dynamic event in the video data collected by the driving recorder, the starting boundary and the ending boundary of the time window of the event to be marked are demarcated, including: first, according to the duration range of the vehicle dynamic event, the corresponding time segment in the video data of the driving recorder is located. Then, the video segment is analyzed frame by frame, and the picture change features related to the vehicle dynamic event are identified, for example, for the “illegal lane changing event”, the picture change of the vehicle crossing the lane line will be identified. Then, the picture change features are time-correlated with the duration range determined by the IMU sensor, and the starting point of the duration range is taken as the reference, the time when the event-related change first appears in the picture is traced back to the starting boundary of the time window; the ending point of the duration range is taken as the reference, and the time when the event-related change completely disappears in the picture is extended to the ending boundary of the time window. For example, the duration range is 3 seconds, the picture change of the vehicle starting to press the line appears 0.5 seconds before the starting point of the duration range, and the vehicle completely returns to the normal lane 0.5 seconds after the termination point of the duration range, then the starting boundary of the time window is 0.5 seconds before the starting point of the duration range, the ending boundary is 0.5 seconds after the termination point of the duration range, and the total time length of the time window is 4 seconds.

[0083] In the embodiments of the present application, in order to avoid the redundancy of picture analysis, the key frame extraction technology is used to identify the picture changes only in the key frames of the video segment, and the extraction interval of the key frame is set to 0.1 seconds, which not only ensures the analysis accuracy, but also improves the processing efficiency.

[0084] In the embodiments of the present application, based on the time reference, the IMU sensor data is used to determine the duration range of the event, and the video picture changes are used to demarcate the boundaries of the time window, which solves the problem of incomplete event recording caused by the traditional driving recorder relying on time stamp to intercept the video, through multi-source data fusion, the complete time interval of the event is accurately locked, the time window is ensured to cover the whole process of the event, and redundant data storage is avoided, the completeness and storage efficiency of the event recording are improved, and reliable time dimension support is provided for subsequent data retrieval and analysis.

[0085] S4, the video segment, the sensor data and the structured control instruction in the time window are packaged and stored to generate an event record.

[0086] In some embodiments, the packaging and storing of the video clip, the sensor data and the structured control instruction in the time window, and the generating of the event record, comprises: obtaining the video clip, the corresponding sensor data and the structured control instruction in the time window; compressively encoding the video clip, and synchronously aligning the sensor data with the time reference of the video clip; associating and integrating the synchronized video clip, the sensor data and the structured control instruction to form a structured data package, wherein the event type comprises a key event and a regular event; storing the encrypted structured data package to a storage unit to generate an event record.

[0087] In the embodiments of the present application, the video clip in the time window refers to a video part containing a complete event to be marked, and the time range thereof is determined by a start boundary and an end boundary. The corresponding sensor data refers to IMU sensor motion state data matching the time range of the video clip, including acceleration, angular velocity, etc. The structured control instruction refers to standardized data containing operation intention, event type and time reference.

[0088] In the embodiments of the present application, the obtaining of the video clip, the corresponding sensor data and the structured control instruction in the time window comprises: firstly calling a storage interface of the driving recorder, extracting the video data stream in the time period according to the start boundary and the end boundary of the time window to form a video clip. At the same time, the IMU sensor data corresponding to the time window is called through a sensor data interface to ensure that the time range of the sensor data is consistent with the video clip. Then, the previously generated structured control instruction is obtained from an instruction cache unit to complete the collection of the three types of data.

[0089] In the embodiments of the present application, in order to ensure the accuracy of data acquisition, the matching degree of the video clip, the sensor data and the time window is verified through timestamp comparison. If the deviation exceeds 0.1 second, the data is re-called.

[0090] In the embodiments of the present application, the compressively encoding is a processing mode for reducing the capacity of video data through a specific algorithm, which is used herein to reduce the storage occupation of the video clip. The time reference synchronous alignment is to keep the timestamps of the sensor data and the video clip consistent, and to ensure that the two are one-to-one corresponding in the time dimension.

[0091] In the embodiment of the present application, the video segment is compressed and encoded, and the sensor data is synchronized and aligned with the time reference of the video segment, including: using the H.265 encoding standard to compress the video segment, which can compress the video data to about 50% of the original size while ensuring the picture quality. Then the timestamp sequence of the video segment and the timestamp sequence of the sensor data are extracted, and the timestamps of the sensor data are adjusted by the interpolation algorithm with the time reference of the video segment as the reference, so that the time interval of the two is controlled within 0.05 seconds, and the synchronization and alignment are realized. For example, the timestamp of a frame of the video segment is 10:00:00.000, and the corresponding timestamp of the sensor data is adjusted to 10:00:00.000±0.05.

[0092] In the embodiment of the present application, the structured data packet is a data set formed by combining the synchronized video segment, sensor data and structured control instruction in a preset format. The key event refers to an event that needs to be focused on, such as collision, red light running, etc. The regular event refers to a non-emergency event recorded daily, such as landscape shooting, road condition observation, etc.

[0093] In the embodiment of the present application, the synchronized video segment, sensor data and structured control instruction are associated and integrated to form a structured data packet, wherein the event type includes: key event and regular event, including: first adding an associated identifier to the synchronized video segment and sensor data, which contains event type and time reference information. Then the event type, time reference and other core fields in the structured control instruction are embedded in the data packet header as index information. Then the three types of data are integrated according to the format of "header index + video data + sensor data + control instruction" to form a structured data packet. For example, the header of the key event data packet will mark "event type: illegal lane changing, time reference: 10:00:00-10:00:05", and the regular event data packet will mark the corresponding regular type information.

[0094] In the embodiment of the present application, the encryption processing is to protect the structured data packet by technical means to prevent unauthorized access or tampering of data. The storage unit is a hardware device for saving the encrypted data packet, such as on-board SD card, solid state disk, etc. The event record is the structured data packet after encryption storage, which contains complete event related data.

[0095] In the embodiment of the present application, the structured data packet is stored in the storage unit after encryption processing, and an event record is generated, including: selecting an encryption method according to the event type in the data packet, using hardware encryption for critical events, generating a unique key through a vehicle-mounted security chip to encrypt the data packet; using software encryption for regular events, and performing encryption processing through an AES-128 algorithm. After encryption is completed, the hardware-encrypted critical event data packet is stored in the encrypted partition of the storage unit, and the software-encrypted regular event data packet is stored in the general partition. A data check code is generated during storage, which is used to verify data integrity during subsequent reading, and finally an event record containing complete event information is formed.

[0096] In the embodiment of the present application, the storage unit adopts a partition management mechanism, and 20% of the storage space is reserved for the critical event partition to ensure that emergency event data is stored preferentially and to avoid loss due to insufficient space.

[0097] In the embodiment of the present application, through compression encoding, time synchronization, association integration and encrypted storage, the problems of isolated storage of video and sensor data, time asynchronization, and data leakage in traditional driving records are solved, structured integration of video clips, sensor data and control instructions is realized, storage occupation is reduced through compression encoding, data correlation is ensured through time synchronization, data security is guaranteed through encryption processing, and storage strategies for critical events and regular events are distinguished, taking into account data reliability and reasonable allocation of storage resources.

[0098] In some embodiments, the structured data packet is stored in the storage unit after encryption processing, and an event record is generated, including: encrypting the structured data packet corresponding to the critical event based on a hardware encryption method to obtain a hardware-encrypted data packet; encrypting the structured data packet corresponding to the regular event based on a software encryption method to obtain a software-encrypted data packet; and storing the hardware-encrypted data packet and the software-encrypted data packet in different user access partitions of the storage unit, respectively.

[0099] In the embodiment of the present application, the hardware encryption method is a technical means for realizing data encryption through a special hardware device, and in this process, the encryption operation is completed by relying on a vehicle-mounted security chip, which has an independent encryption engine and a key management unit, and the key is stored in a secure area inside the chip and cannot be read. The structured data packet corresponding to the critical event is a data set containing important event information such as collision and illegal behavior, and its encryption requirement is higher than that of the regular event. The hardware-encrypted data packet is a data packet formed after hardware encryption processing and can only be decrypted through the corresponding hardware.

[0100] In the embodiment of the present application, the structured data packet corresponding to the key event is encrypted based on a hardware encryption mode to obtain a hardware encrypted data packet, including: first, calling an encryption interface of an on-board security chip, inputting the structured data packet of the key event into the chip. The encryption engine inside the security chip will call a preset hardware encryption algorithm, such as SM4 national encryption algorithm, and simultaneously call a unique device binding key from the key storage area built in the chip, which is associated with the vehicle identification code and cannot be derived. During the encryption process, the data packet is divided into fixed-size data blocks, each of which is independently encrypted by the hardware algorithm and attached with a message authentication code for verifying data integrity. After the encryption is completed, the security chip outputs the encrypted data packet, i.e. the hardware encrypted data packet, and the whole process is completed inside the chip to avoid the key and data being intercepted during transmission on the external bus.

[0101] In the embodiment of the present application, to enhance security, the hardware encryption adopts a session key dynamic generation mechanism, and a session key is randomly generated by the security chip each time of encryption, which is stored with the data packet after being encrypted by the device binding key, so as to ensure that even if a single session key is leaked, it will not affect other encrypted data.

[0102] In the embodiment of the present application, the software encryption mode is a technical means for realizing data encryption through a software program, which relies on the on-board processor to run the encryption algorithm to complete the encryption and does not rely on a special hardware encryption device. The structured data packet corresponding to the conventional event is a data set containing landscape records, road condition observations and other non-emergency event information, and its encryption demand is lower than that of the key event. The software encrypted data packet is a data packet formed after being encrypted by software and can be decrypted by the corresponding software.

[0103] In the embodiment of the present application, the structured data packet corresponding to the conventional event is encrypted based on a software encryption mode to obtain a software encrypted data packet, including: first, loading a preset software encryption algorithm, such as AES-128 algorithm, by the on-board processor, the key of which is randomly generated by the system when the vehicle is started for the first time and stored in the encryption file of the on-board operating system. Then, the structured data packet of the conventional event is divided into 128-bit data blocks according to the algorithm requirements, and each data block is encrypted by the encryption algorithm, and the same data is ensured to have different encryption results in the operation process by combining the initial vector. After the encryption is completed, the processor splices the encrypted data blocks and attaches the algorithm identifier and initial vector information to form the software encrypted data packet.

[0104] In the embodiment of the present application, the key of the software encryption is automatically updated regularly through the security process of the on-board system, and the update period is set to 7 days. During the update process, the validity of the new key is verified by a hash algorithm to prevent the key from being tampered.

[0105] In the embodiments of the present application, the storage unit is a vehicle-mounted storage device for storing encrypted data packets, such as an embedded solid-state hard disk, which is internally divided into multiple logical partitions, and different partitions are set with different access permissions. The user access partition is a partition in the storage unit divided according to the data security level, including a high permission access partition and a normal access partition, and the high permission access partition needs to be accessed through hardware key verification, and the normal access partition can be accessed through user password or system authorization.

[0106] In the embodiments of the present application, the hardware encrypted data packet and the software encrypted data packet are respectively stored in different user access partitions of the storage unit, including: first identifying the types of the hardware encrypted data packet and the software encrypted data packet, the hardware encrypted data packet corresponding to a key event, and the software encrypted data packet corresponding to a regular event. Then, a partition management interface of the storage unit is called, the hardware encrypted data packet is written into a high permission access partition, the access control list of the partition only allows the vehicle-mounted security chip to perform read and write operations after key verification, and the user needs to trigger the chip verification through a physical button to access. The software encrypted data packet is written into a normal access partition, and the partition is managed by the user account permission of the vehicle-mounted system, and the user can access after inputting the correct password or passing the biometric identification verification.

[0107] In the embodiments of the present application, the partition capacity of the storage unit is preset according to the event occurrence frequency, the high permission access partition accounts for 30% of the total capacity, and the normal access partition accounts for 70%, and dynamic adjustment is supported, when the storage capacity of the high permission partition reaches 90%, the system automatically compresses the earliest non-key event data to release space.

[0108] In the embodiments of the present application, the key events and the regular events are differentiated and encrypted (hardware encryption vs. software encryption) and stored in different partitions, which solves the problem of single data encryption mode in traditional driving records and important data being easily accessed illegally. The hardware encryption provides a higher security level for key event data, and the software encryption takes into account the storage efficiency of regular events, and the partition storage realizes fine management of data access permissions, which not only guarantees the non-tamperability and confidentiality of key event (such as accident, violation) data, but also meets the convenient access demand of regular events, and balances security and usability.

[0109] In some embodiments, the encrypted structured data packet is stored in the storage unit after the encryption processing, and an event record is generated, further comprising: creating index information containing the event type, time reference, and video scene features; associating the index information with the corresponding encrypted structured data packet, and storing the index information in an index area in the storage unit.

[0110] In the embodiments of the present application, the index information is structured data used for quickly locating the event record, and includes three types of core information, i.e., event type, time reference and video scene feature. The event type is a classification of the event, including a key event and a regular event. The time reference is a time reference of the event occurrence, which can be a specific time point or a relative time range. The video scene feature is a visual feature extracted from the video segment, such as road type, weather condition, and whether there are pedestrians, etc.

[0111] In the embodiments of the present application, the index information including the event type, the time reference and the video scene feature is created, including: first, extracting the event type and the time reference from the structured control instruction, to ensure that the two types of information are consistent with the structured data packet before encryption. Then, feature extraction is performed on the video segment in the time window, a lightweight image feature extraction algorithm is used to identify and record the scene elements in the video, for example, it is identified that the video picture contains “highway”, “sunny day”, “no pedestrian” and the like, and these information collectively constitute the video scene feature. Then, the event type, the time reference and the video scene feature are combined according to a preset format to form a complete index information, and each index information corresponds to a structured data packet.

[0112] In the embodiments of the present application, in order to ensure the uniqueness of the index information, a unique identifier is allocated to each index information, which is generated by combining the event type code, the timestamp segment of the time reference and a random number, for example, the identifier of the key event can be “K-202507241530-789”, in which “K” represents the key event, “202507241530” is the timestamp segment of the time reference, and “789” is the random number.

[0113] In the embodiments of the present application, the association between the index information and the encrypted structured data packet is to establish the corresponding relationship between the two, so that the corresponding encrypted data packet can be quickly found through the index information. The storage unit is a hardware device used for storing data, and an index area is specially arranged in the internal thereof for storing the index information, the access speed of the index area is faster than that of other storage areas, and the index area is convenient for quick retrieval.

[0114] In the embodiment of the present application, the index information is associated with the corresponding encrypted structured data packet, and the index information is stored in the index area of the storage unit, including: first, binding the unique identifier in the index information with the storage address of the encrypted structured data packet to form an association mapping table, which records the correspondence between the identifier and the storage address. Then, the association mapping table is written into the index area of the storage unit through the storage control module, and the index information itself is also stored in the corresponding position of the index area, ensuring the mutual association of the index information and the mapping table. For example, the index information "K-202507241530-789" is bound with the storage address "0x00012345" of the hardware encrypted data packet and stored together in the index area.

[0115] In the embodiment of the present application, the index area adopts a data redundancy storage mechanism, and each piece of index information and its associated mapping table are stored in two different physical positions of the index area. When the data in one of the positions is damaged, the data in the other position can be used for recovery, ensuring the reliability of the index information. At the same time, the storage space of the index area accounts for 5% of the total capacity of the storage unit, to meet the storage needs of a large number of index information.

[0116] In the embodiment of the present application, by creating index information containing event type, time reference, and scene characteristics and associating encrypted data packets, the problem of low efficiency of data retrieval relying on a single time stamp in traditional driving records is solved. The index information provides multi-dimensional (type, time, scene) query basis for data retrieval, greatly improving the retrieval speed and accuracy of event records. Users can quickly locate target events through keywords, avoiding the tedious operation of viewing videos segment by segment, and improving the convenience of data utilization.

[0117] In some embodiments, the event retrieval step further includes: receiving a voice retrieval instruction of a user, using natural language processing technology to analyze the retrieval intention in the voice retrieval instruction; extracting the retrieval keywords corresponding to the retrieval intention; matching the corresponding event type, time reference and video scene characteristics in the index information based on the retrieval keywords; retrieving the corresponding event record from the storage unit, and decrypting the retrieved event record to return the retrieval data containing the video segment, sensor data and structured control instruction within the time window.

[0118] In the embodiment of the present application, the voice retrieval instruction is an instruction issued by the user through voice to find a specific event record, such as "find the video of yesterday's illegal lane changing". Natural language processing technology is a technology for analyzing and understanding human language, which is used to process the voice retrieval instruction in this step to clarify the user's retrieval intention. The retrieval intention is the retrieval purpose that the user hopes to achieve through the voice retrieval instruction, such as finding an event at a specific time or a specific type of event.

[0119] In the embodiments of the present application, the voice retrieval instruction of the user is received, and a retrieval intention in the voice retrieval instruction is analyzed by using a natural language processing technology. First, the voice retrieval instruction issued by the user is received by a voice collection device and converted into a text form. Then, the voice retrieval instruction in the text form is input into a natural language processing module, the module performs word segmentation processing on the text, and splits the complete instruction into multiple words or phrases, for example, splits "find the record of pedestrian crossing in the last week" into "find", "last week", "pedestrian crossing", and "record". Then, the semantic understanding algorithm is used to analyze the word segmentation result, and the retrieval intention of the user is identified, for example, from the above word segmentation result, it can be analyzed that the retrieval intention of the user is to find the record of the pedestrian crossing event occurring in the last week.

[0120] In the embodiments of the present application, in order to ensure the accuracy of the analysis, a semantic model related to the driving event is preset in the natural language processing module, the model is trained by a large number of voice instruction samples in the driving scene, and can effectively understand the retrieval requirements in different expression modes.

[0121] In the embodiments of the present application, the retrieval keyword is a core word or phrase that can represent the retrieval requirement and is extracted from the retrieval intention. These words or phrases are associated with the event type, time reference, and video scene feature in the index information, for example, "illegal lane changing", "yesterday", and "rainy day".

[0122] In the embodiments of the present application, the retrieval keyword corresponding to the retrieval intention is extracted, including: based on the analyzed retrieval intention, the core information related to the event record is selected from the voice retrieval instruction text after word segmentation. For the retrieval intention related to the event type, the corresponding event type word is extracted as the retrieval keyword, for example, "illegal lane changing" is extracted from "find all records of running a red light"; for the retrieval intention related to the time, the corresponding time word is extracted as the retrieval keyword, for example, "three days ago" is extracted from "find the event three days ago"; for the retrieval intention related to the scene feature, the corresponding scene feature word is extracted as the retrieval keyword, for example, "foggy day" is extracted from "find the driving record in the foggy day".

[0123] In the embodiments of the present application, the keywords are standardized during the extraction process, for example, "yesterday" is uniformly converted into "yesterday", and "illegal lane changing" and "lane changing on solid line" are uniformly classified as "illegal lane changing", so as to ensure the accuracy of the subsequent matching.

[0124] In the embodiments of the present application, the event type is a specific classification of the event, such as illegal lane changing, pedestrian crossing, etc. The time reference is the time reference of the event occurrence. The video scene feature is the scene information in the video picture when the event occurs, such as weather condition, road type, etc.

[0125] In the embodiment of the present application, based on the matching of the search keywords and the corresponding event type, time reference and video scene feature in the index information, the extracted search keywords are compared with the index information in the index area of the storage unit. For the event type keyword, the event type in the index information is matched; for the time keyword, the time reference in the index information is matched, for example, "last week" is matched with the record of the time reference within one week in the index information; for the scene feature keyword, the video scene feature in the index information is matched. Through multi-dimensional matching, the index information consistent with the search keywords is screened out.

[0126] In the embodiment of the present application, a fuzzy matching algorithm is used in the matching process, when the search keywords and the content in the index information are not completely consistent but semantically similar, the matching can also be performed, for example, "foggy day" and "heavy fog weather" can be considered as matching, so as to improve the flexibility of the search.

[0127] In the embodiment of the present application, the decryption processing is the process of restoring the encrypted event record to the original data, for the hardware encrypted data packet of the key event, the corresponding hardware decryption mode is used; for the software encrypted data packet of the regular event, the corresponding software decryption mode is used. The search data is the data returned to the user after decryption processing and containing complete event information, including the video segment, sensor data and structured control instruction within the time window.

[0128] In the embodiment of the present application, the corresponding event record is called from the storage unit, the called event record is decrypted, the search data containing the video segment, sensor data and structured control instruction within the time window is returned, including: the storage position of the corresponding event record in the storage unit is determined according to the matched index information, and then the event record is called from the storage unit through the storage interface. For the called event record, the corresponding decryption mode is selected according to the event type, if it is a hardware encrypted data packet of a key event, the decryption is performed through the vehicle-mounted security chip; if it is a software encrypted data packet of a regular event, the decryption is performed through the preset software decryption algorithm. After the decryption is completed, the video segment, corresponding sensor data and structured control instruction within the time window are integrated into the search data, and returned to the user, for example, the video segment is displayed on the vehicle-mounted display screen, and the viewing options of the sensor data and structured control instruction are provided.

[0129] In the embodiment of the present application, in order to ensure the security of data transmission, in the process of returning the search data, a data verification mechanism is used to verify the integrity of the transmitted data, to ensure that the search data received by the user is accurate and correct.

[0130] In the embodiments of the present application, through voice search instruction analysis, keyword matching, index comparison and decryption return, the problems of complex data search operation and low efficiency in traditional driving record are solved. The technical effects are as follows: natural language processing realizes the convenience of voice search, keyword matching and index information association greatly improve the search accuracy, differentiated decryption ensures the security of data access, and finally the complete event data is quickly returned, which improves the efficiency of event tracing and meets the user's demand for quick query of historical records.

[0131] In some embodiments, after the structured data packet is encrypted and stored in the storage unit, an event record is generated, and the method further comprises: judging the event type corresponding to the structured data packet based on a dynamic threshold value, and when the event type is a key event, the hardware encrypted data packet preferentially occupies a high-privilege access partition in the storage unit.

[0132] In the embodiments of the present application, the dynamic threshold value is a judgment standard dynamically adjusted according to the driving scene characteristics and the event influence degree. The threshold value is updated in real time in combination with factors such as the potential risk level of the event and the importance of the data. For example, the determination threshold value of a collision event is lower than that of a regular landscape record event.

[0133] In the embodiments of the present application, judging the event type corresponding to the structured data packet based on a dynamic threshold value comprises: first extracting feature parameters related to the event in the structured data packet, including event duration, vehicle motion state change amplitude (such as acceleration change amount), and appearance frequency of abnormal targets in the video picture. Then, the feature parameters are compared with the preset dynamic threshold value. When the feature parameters exceed the dynamic threshold value, it is determined that the corresponding event type is a key event; when the feature parameters do not exceed the dynamic threshold value, it is determined to be a regular event. For example, when the vehicle acceleration change amount exceeds -6 m / s2 (dynamic threshold value), and there is a collision picture in the video picture, it is determined that the event is a key event.

[0134] In the embodiments of the present application, the update mechanism of the dynamic threshold value is associated with the vehicle driving environment. In the highway scene, the threshold value is appropriately reduced, and in the urban road congestion scene, the threshold value is appropriately increased, to adapt to the event determination requirements in different scenes.

[0135] In the embodiments of the present application, the hardware encrypted data packet is a structured data packet corresponding to a key event processed by a hardware encryption method. The storage unit is a vehicle-mounted storage device for storing data, and the high-privilege access partition is a storage area in the storage unit provided with a strict access control mechanism and can only be accessed through specific hardware verification or authorization. Preferential occupation means that when the storage resource is limited, the hardware encrypted data packet obtains the storage space of the high-privilege access partition earlier than the software encrypted data packet.

[0136] In the embodiment of the present application, when the event type is a key event, the hardware encrypted data packet preferentially occupies the high-privilege access partition in the storage unit, including: the management module of the storage unit monitors the storage space usage of the high-privilege access partition in real time. When detecting a hardware encrypted data packet to be stored, if there is remaining space in the high-privilege access partition, the hardware encrypted data packet is directly stored in the partition; if the remaining space is insufficient, the management module releases part of the storage space in the order of "first regular event data, then early key event data (non-urgent reserved data)", to preferentially guarantee the storage of the current hardware encrypted data packet. For example, when the remaining space of the high-privilege access partition is 500 MB, and the size of the hardware encrypted data packet to be stored is 800 MB, the system automatically deletes the earliest regular event encrypted data (if any) or early low-priority key event data that is not necessarily reserved in the partition, until at least 800 MB of space is freed for the storage of the current hardware encrypted data packet.

[0137] In the embodiment of the present application, the priority occupation mechanism is realized by the underlying scheduling algorithm of the storage unit, which allocates the highest storage priority identifier to the hardware encrypted data packet, ensuring that in a multi-task storage scenario, key event data will not be lost due to insufficient space.

[0138] In the embodiment of the present application, the event type is determined based on a dynamic threshold, ensuring that the hardware encrypted data packet of a key event preferentially occupies a high-privilege access partition, solving the problem of unreasonable allocation of storage resources in traditional driving records and the possible loss of key data due to insufficient space. The dynamic threshold realizes intelligent determination of the importance of events, and the priority occupation mechanism of the high-privilege access partition guarantees the storage priority of key event data, avoiding the loss of important records due to insufficient storage resources, and ensuring the reliable retention of key event data such as accidents and violations, providing tamper-proof evidence for subsequent responsibility identification.

[0139] S5, real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving the associated video when a preset scene is detected.

[0140] In some embodiments, the real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving the associated video when a preset scene is detected, includes: obtaining the video segment and the corresponding motion state data contained in the event record; performing real-time analysis on the video segment through a lightweight computer vision model, identifying the target object and dynamic characteristics in the picture in combination with the motion state data; when the identification result matches the preset scene, triggering a voice warning corresponding to the preset scene; synchronously extracting the video segments before and after the occurrence of the preset scene as associated video, packaging the associated video with the corresponding motion state data and the voice warning, and updating to the event record.

[0141] In the embodiment of the present application, the video segment and the corresponding motion state data contained in the event record are acquired, including: first, the stored video segment in the event record of the storage unit is called, and these video segments are the video content in the time window of the previously marked event. At the same time, the motion state data corresponding to the video segment is called through the time reference association, ensuring that the time range of the motion state data completely matches the time range of the video segment, providing complete data basis for subsequent real-time analysis.

[0142] In the embodiment of the present application, the synchronization of the video segment and the motion state data is confirmed through the timestamp verification mechanism during the calling process. If the time deviation exceeds 0.1 seconds, the associated calling is performed again to ensure the consistency of the data.

[0143] In the embodiment of the present application, the lightweight computer vision model is a high-efficiency image recognition model suitable for vehicle-mounted terminals. The model parameters are simplified, which can reduce the occupation of computing resources while ensuring the recognition accuracy. The target object is an object related to driving in the video picture, such as other vehicles, pedestrians, traffic signs, etc. The dynamic feature is the motion change of the target object, such as the speed change of the vehicle, the moving track of the pedestrian, etc.

[0144] In the embodiment of the present application, the video segment is analyzed in real time through the lightweight computer vision model, and the target object and the dynamic feature in the picture are identified in combination with the motion state data, including: the video segment is input into the lightweight computer vision model by frame, the model extracts features from each frame of image, identifies the target object in the picture, and analyzes the dynamic feature of the target object through the comparison of consecutive frames. At the same time, the motion state data is used as an auxiliary reference, for example, when the model identifies a vehicle in front, the acceleration data of the vehicle is combined to judge whether the vehicle is in an emergency braking state, so as to improve the accuracy of dynamic feature recognition. For example, while identifying a vehicle in front in the video picture, if the motion state data shows that the acceleration of the vehicle decreases significantly, it can assist in judging that the vehicle in front has deceleration behavior.

[0145] In the embodiment of the present application, the preset scene is a pre-set driving scene that needs to be warned, such as a pedestrian crossing the road, a vehicle in front emergency braking, a traffic signal turning red, etc. Each preset scene corresponds to specific warning content. The voice warning is a prompt information sent through the vehicle-mounted voice broadcast device, which is used to remind the driver to pay attention to the corresponding scene.

[0146] In the embodiments of the present application, when the recognition result matches the preset scene, a voice warning corresponding to the preset scene is triggered, including: comparing the target object and dynamic characteristics recognized with the feature library of the preset scene, and when the matching degree of the two exceeds a set threshold (such as 85%), it is determined that the recognition result matches the preset scene. Then, a voice synthesis module is called to generate voice warning content corresponding to the preset scene, for example, when the "pedestrian crossing the road" scene is matched, the voice information "pay attention to the front pedestrian crossing, please slow down and avoid" is generated, and the voice information is broadcast through the vehicle-mounted loudspeaker.

[0147] In the embodiments of the present application, the associated video is a video clip containing a period of time before and after the occurrence of the preset scene, and the time range is usually set according to the scene type, for example, for the pedestrian crossing scene, the associated video can contain video content 5 seconds before the scene occurs to 10 seconds after the scene occurs. Packaging and storage is a process of integrating the associated video, motion state data and voice warning according to a certain format and then storing.

[0148] In the embodiments of the present application, the video clips before and after the occurrence of the preset scene are synchronously extracted as the associated video, the associated video, the corresponding motion state data and the voice warning are packaged and stored, and are updated to the event record, including: according to the occurrence time of the preset scene, the video clip of a set time length (such as 5 seconds) before the scene occurs is traced back as the content before the scene occurs, the video clip of a set time length (such as 10 seconds) after the scene occurs is continued as the content after the scene occurs, the two parts of content and the video clip when the scene occurs are combined to form the associated video. At the same time, the motion state data corresponding to the time range of the associated video is called, and the generation time and content of the voice warning are recorded. Then, the associated video, the motion state data and the voice warning are integrated into one data unit, the data unit is encrypted and stored in the storage unit, and the index information of the event record is updated, so that the associated video and related data can be quickly retrieved in the future.

[0149] In the embodiments of the present application, the time range of the associated video can be adjusted according to the severity of the preset scene, and for high-risk scenes (such as collision warning), the time range will be appropriately expanded to retain a more complete event process.

[0150] In the embodiments of the present application, the video is analyzed in real time by a lightweight computer vision model, the preset scene is identified in combination with the motion state data, and the warning is triggered, which solves the problem of high false alarm rate and untimely warning caused by the dependence of the traditional driving recorder on single visual detection. The multi-modal data fusion (video + motion state) improves the accuracy of scene recognition, the lightweight model ensures real-time performance, the automatic saving of the associated video realizes complete tracing of dangerous scenes, and the voice warning provides timely reminders for the driver, reduces the driving risk, and improves the active safety performance.

[0151] In the above intelligent driving record method based on voice AI, hands-free operation is realized through the voice interaction module, complex instructions are analyzed by combining end-side ASR and NLP technology, and the safety hazards of manual operation during driving are avoided; the time window of the event is accurately determined by integrating visual, voice, and IMU data through multi-sensor fusion, and the single-sensor missing detection problem is solved; the hierarchical storage combining hardware encryption (key event) and software encryption (regular event) is adopted, and the physical isolation partition and dynamic key are used to ensure data security and comply with privacy regulations; a semantic index containing event type, time reference, and scene feature is constructed to support voice search to quickly locate the target record and replace traditional manual browsing; real-time scene detection and voice warning are realized through a local lightweight visual model, and a priority storage mechanism triggered by a dynamic threshold is used to ensure that key data is saved in priority, and the modular design of the system module is suitable for different vehicle models and can be extended with more intelligent driving assistance functions, thereby comprehensively improving the intelligence and practicality of the driving record.

[0152] Based on the same inventive concept, the embodiments of the present application also provide a voice AI-based intelligent driving record system for implementing the above voice AI-based intelligent driving record method. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme described in the above method, so the specific limitations in one or more voice AI-based intelligent driving record system embodiments provided below can be referred to the limitations of the voice AI-based intelligent driving record method in the foregoing, which will not be described here again.

[0153] In one embodiment, as shown in Figure 3 , a voice AI-based intelligent driving record system is provided, comprising:

[0154] The voice interaction module 101 is configured to acquire user voice instructions in real time through a voice acquisition device;

[0155] The instruction analysis module 102 is configured to analyze the operation intention and event marking request in the user voice instructions, and generate a structured control instruction containing an event type and a time reference;

[0156] The multi-sensor fusion module 103 is configured to determine the time window of the event to be marked according to the time reference, in combination with the motion state data of the IMU sensor and the video data collected by the driving recorder;

[0157] The event record generation module 104 is configured to package and store the video clips, sensor data, and structured control instructions in the time window to generate an event record;

[0158] The associated video storage module 105 is configured to analyze the video clips in the event record in real time, and trigger a voice warning and automatically save the associated video when a preset scene is detected.

[0159] The various modules in the intelligent driving record system based on voice AI can be implemented by software, hardware, and combinations thereof, in whole or in part. The various modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the various modules.

[0160] In one embodiment, a computer device is provided, which can be a sample analyzer, and an internal structure diagram thereof can be as shown in FIG. 1. Figure 4 The computer device includes a processor, a memory, a communication interface, a display screen, and an input system connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (near field communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input system of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0161] Those skilled in the art can understand that Figure 4 the structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0162] In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above method embodiments.

[0163] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the above method embodiments.

[0164] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and / or features involved in the present application are all information, data and / or features authorized by the user or authorized by all parties.

[0165] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0166] The technical features of the above embodiments can be combined in any way. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0167] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A voice AI-based intelligent driving recording method, characterized by, The method comprises: Real-time acquisition of user voice instructions by a voice acquisition device; Parsing the operation intention and event marking request in the user voice instruction to generate a structured control instruction containing an event type, an operation intention and a time reference; wherein the parsing the operation intention and event marking request in the user voice instruction to generate a structured control instruction containing an event type, an operation intention and a time reference comprises: matching key information in the event marking request according to the operation intention and a preset instruction template; determining the event type and the time reference according to the key information to generate the structured control instruction; the key information indicates the core content in the event marking request for determining event attributes and time ranges; the key information includes the event type and time description; the event type indicates a category based on the event description in the key information; the time reference indicates a time parameter converted according to the time description in the key information; Determining a time window of a to-be-marked event according to the time reference, in combination with motion state data of an IMU sensor and video data collected by a driving recorder; wherein the time window indicates a time interval containing a video segment of the to-be-marked event; Packing and storing the video segment, sensor data and the structured control instruction in the time window to generate an event record; Real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving associated video when a preset scene is detected.

2. The method of claim 1, wherein, Parsing the operation intention in the user voice instruction comprises: Extracting the operation intention in the user voice instruction by using natural language processing technology.

3. The method of claim 1, wherein, The determining a time window of a to-be-marked event according to the time reference, in combination with motion state data of an IMU sensor and video data collected by a driving recorder comprises: According to the time reference, associating the motion state data of the IMU sensor to determine the duration range of a vehicle dynamic event corresponding to the event marking request; According to the duration range, in combination with the screen changes related to the vehicle dynamic event in the video data collected by the driving recorder, the start boundary and the end boundary of the time window of the to-be-marked event are delimited.

4. The method of claim 1, wherein, The packing and storing the video segment, sensor data and the structured control instruction in the time window to generate an event record comprises: Acquiring the video segment, corresponding sensor data and the structured control instruction in the time window; Compressively encoding the video segment, synchronizing and aligning the sensor data with the time reference of the video segment; Integrating the synchronized video segment, sensor data and structured control instruction to form a structured data packet, wherein the event type includes a key event and a regular event; After encryption processing, the structured data packet is stored in a storage unit to generate an event record.

5. The method of claim 1, wherein, The real-time analysis of the video segment in the event record, triggering a voice warning and automatically saving associated video when a preset scene is detected comprises: Acquire a video clip and corresponding motion state data contained in the event record; Real-time analysis of the video clip through a lightweight computer vision model, combined with the motion state data, identifies the target object and dynamic characteristics in the picture; When the recognition result matches the preset scene, trigger the voice warning corresponding to the preset scene; Synchronously extract the video clip before and after the occurrence of the preset scene as an associated video, package the associated video with the corresponding motion state data and the voice warning, and store it in the event record.

6. The method of claim 4, wherein, The structured data packet is stored in the storage unit after encryption processing, generating an event record, including: Encrypt the structured data packet corresponding to the key event based on a hardware encryption method to obtain a hardware encrypted data packet; Encrypt the structured data packet corresponding to the regular event based on a software encryption method to obtain a software encrypted data packet; The hardware encrypted data packet and the software encrypted data packet are respectively stored in different user access partitions in the storage unit.

7. The method of claim 4, wherein, The structured data packet is stored in the storage unit after encryption processing, generating an event record, further including: Create index information containing the event type, time reference, and video scene characteristics; Associate the index information with the corresponding encrypted structured data packet, and store the index information in the index area of the storage unit.

8. The method of claim 6, wherein, The structured data packet is stored in the storage unit after encryption processing, generating an event record, further including: Determine the event type corresponding to the structured data packet based on a dynamic threshold, when the event type is a key event, the hardware encrypted data packet occupies the high-privilege access partition in the storage unit.

9. The method of claim 7, wherein, The method further includes an event retrieval step: Receive the user's voice retrieval instruction, and use natural language processing technology to analyze the retrieval intent in the voice retrieval instruction; Extract the retrieval keyword corresponding to the retrieval intent; Match the corresponding event type, time reference, and video scene characteristics in the index information based on the retrieval keyword; Retrieve the corresponding event record from the storage unit, decrypt the retrieved event record, and return the retrieval data containing the video clip, sensor data, and structured control instruction within the time window.

10. A voice AI-based intelligent driving record system, characterized by, The system includes: A voice interaction module for acquiring user voice instructions in real time through a voice acquisition device; The instruction analysis module is configured to analyze an operation intention and an event marking request in the user voice instruction, and generate a structured control instruction containing an event type, the operation intention, and a time reference; wherein the analysis of the operation intention and the event marking request in the user voice instruction, and the generation of the structured control instruction containing the event type, the operation intention, and the time reference, comprises: matching key information in the event marking request according to the operation intention and in combination with a preset instruction template; determining the event type and the time reference according to the key information, and generating the structured control instruction; the key information indicates core content for determining event attributes and a time range in the event marking request; the key information includes the event type and time description; the event type indicates a category divided based on event description in the key information; the time reference indicates a time parameter converted according to the time description in the key information; The multi-sensor fusion module is configured to determine a time window of a to-be-marked event according to the time reference, in combination with motion state data of an IMU sensor and video data collected by a driving recorder; wherein the time window indicates a time interval containing a video segment of the to-be-marked event; The event record generation module is configured to package and store the video segment, sensor data, and the structured control instruction in the time window, and generate an event record; The associated video storage module is configured to analyze the video segment in the event record in real time, and trigger a voice warning and automatically save associated video when a preset scene is detected.

Images