A video recording event analysis method and device for a driving recorder

Abstract

The purpose of the present application is to provide a video recording event analysis method and device of a driving recorder, which not only solves the technical problems that the computing power of the rear-mounted device end side is limited and cannot bear complex video analysis, and the cloud end only stores storage functions without intelligent analysis, but also analyzes the uploaded key video through the multi-modal large model deployed on the cloud end, automatically extracts and summarizes the event type and event time point of each event in the key video, realizes the user's demand for quickly knowing the core information in the key video, and improves the product use experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and device for analyzing video events from a dashcam. Background Technology

[0002] With the development of the automotive aftermarket, aftermarket consumer-grade dashcams have become mainstream in-vehicle electronic products. Their accompanying mobile apps typically feature a cloud album function, allowing users to upload emergency recordings (such as manually triggered emergency recordings) and collision detection recordings (such as gravity sensor-triggered collision recordings) to cloud storage for remote viewing. However, current technology for aftermarket dashcam mobile apps only provides video upload, storage, and preview functions, exhibiting the following key shortcomings: Users need to view each emergency video and collision detection video in the cloud album to understand the key events that happened that day. When multiple recordings are triggered in a single day, a lot of time needs to be spent filtering them, which is extremely inefficient. The video content lacks structured summaries, making it difficult for users to quickly locate the time points and event types of key events, such as when the collision occurred or when a pedestrian passed by at close range. Aftermarket dashcams have limited computing power on the device side, making it impossible to perform complex video content analysis. Existing cloud functions only focus on storage and do not leverage the advantages of cloud computing power to enable intelligent analysis.

[0003] There is currently no effective solution to the above problems. The main reason is that existing technologies have not fully taken into account the characteristics of aftermarket dashcams, such as "limited computing power on the device side and the need to control hardware costs," nor have they recognized the core user demand for "quick access to core information" in critical videos. As a result, cloud functions remain at the basic storage level and have failed to optimize the user viewing experience through technical means. Summary of the Invention

[0004] One objective of this application is to provide a method and device for analyzing video recording events of a dashcam, which solves the technical problems in the prior art where the computing power of aftermarket devices is limited and cannot support complex video analysis, and the cloud only has storage functions and does not enable intelligent analysis. By analyzing the uploaded key recordings through a multimodal large model deployed in the cloud, the method automatically extracts and summarizes the event types and event timestamps of each event in the key recordings, thereby meeting the user's need to quickly understand the core information in the key recordings and improving the product user experience.

[0005] According to one aspect of this application, a method for analyzing video recording events from a dashcam is provided. This method is applied to a cloud server, and includes: Receive key video recordings uploaded by an aftermarket dashcam in the vehicle, the key video recordings carrying a sudden attribute tag and a recording timestamp; The key video recordings are sequentially filtered and preprocessed to obtain preprocessed key video recordings; The event types in the preprocessed video are identified, event time points are extracted, and supplementary event details are obtained through a multimodal large model deployed in the cloud. A structured event summary text is generated in chronological order. The event summary text includes all events and the corresponding event time, event type, and time details for each event. The event summary text is pushed to the user terminal, wherein the cloud album module of the user terminal is provided with an event summary entry, so that the user terminal can view the event summary text in response to the user's selection operation of the event summary entry.

[0006] Furthermore, in the above method, each event in the event summary text is associated with a corresponding original video link; The step of pushing the event summary text to the user terminal so that the user terminal can view the event summary text includes: The event summary text and the original video link corresponding to each associated event are pushed to the user terminal, so that the user terminal can view the event summary text in response to the user's selection operation on the event summary entry, and jump to the video playback page corresponding to the original video link of the target event in response to the user's selection operation on the target event in the event summary text.

[0007] Furthermore, in the above method, the step of sequentially filtering and preprocessing the key video recordings to obtain preprocessed key video recordings includes: The key videos within a preset time period are selected from the key videos and determined as the selected key videos. The selected key videos are processed sequentially by extracting video frame sequences, unifying video resolution, and synchronizing the timestamps corresponding to the videos, resulting in preprocessed key videos.

[0008] Furthermore, in the above method, the key recordings include front-view key recordings recorded by the front-facing camera of the aftermarket dashcam and rear-view key recordings recorded by the rear-facing camera of the aftermarket dashcam. The process of sequentially extracting video frame sequences, unifying video resolution, and synchronizing timestamps corresponding to the selected key video recordings to obtain preprocessed key video recordings includes: The selected key videos are sequentially processed by video frame sequence extraction, video resolution unification, timestamp synchronization, and viewpoint time difference elimination to obtain preprocessed key videos. Each event in the event summary text is associated with a corresponding original pre-recorded video link and an original post-recorded video link. The user terminal supports simultaneous playback of the video corresponding to the original pre-recorded video link and the video corresponding to the original post-recorded video link.

[0009] Furthermore, the above method further includes: Identify the continuity and event correlation of video content in at least two key video clips that both carry the same suddenness attribute label. If the recorded content is continuous and all related to the same event scene, then the events of the at least two key recorded videos are determined to be events corresponding to the same event type. The events of the at least two key recorded videos are merged into one event, and the at least two key recorded videos are all associated with the original video links of the events corresponding to the same event type in the event summary text.

[0010] Furthermore, in the above method, the sudden attribute label includes an emergency attribute label and a collision attribute label, and the key video includes emergency video and collision detection video. The emergency recording is a video recording triggered manually by the user, and the collision detection recording is a video recording automatically triggered by the aftermarket dashcam after a collision is detected by the gravity sensor.

[0011] Furthermore, in the above method, the multimodal large model deployed in the cloud is a lightweight large model that has been fine-tuned and optimized for driving scenarios. It is trained by fine-tuning a large number of driving video samples containing emergency attribute labels and / or collision attribute labels. The event recognition accuracy of the multimodal large model is ≥95% and the analysis latency is ≤3 seconds / segment of 1 minute video.

[0012] According to another aspect of this application, a non-volatile storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a processor, cause the processor to implement the video recording event analysis method of the dashcam described above.

[0013] According to another aspect of this application, a recording event analysis device for a dashcam is also provided, wherein the device includes: One or more processors; Computer-readable medium for storing one or more computer-readable instructions. When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the video event analysis method of the dashcam as described above.

[0014] Compared with existing technologies, this application receives key video recordings uploaded by aftermarket dashcams in vehicles via a cloud server. These key video recordings carry sudden event attribute tags and recording timestamps. The key video recordings are sequentially filtered and preprocessed to obtain preprocessed key video recordings. A multimodal large model deployed in the cloud identifies event types, extracts event time points, and supplements event details in the preprocessed video recordings. A structured event summary text is generated in chronological order, including all events and their corresponding event time points, event types, and time details. The event summary text is then... The data is pushed to the user terminal, where an event summary entry is added to the cloud album module. This allows the user terminal to view the event summary text in response to the user's selection of the event summary entry. This not only solves the technical problems of limited computing power on the back-end device side, which cannot support complex video analysis, and the cloud only having storage functions without enabling intelligent analysis, but also analyzes the uploaded key recordings through a multimodal large model deployed in the cloud. It automatically extracts and summarizes the event type and event time of each event in the key recordings, fulfilling the user's need to quickly understand the core information in the key recordings and improving the product user experience. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A flowchart illustrating a video event analysis method for a dashcam according to one aspect of this application is shown. Figure 2 This diagram illustrates the event analysis process of dual-recording video recording in a recording event analysis method for a dashcam according to one aspect of this application. Figure 3 This diagram illustrates the system architecture corresponding to the event analysis process of dual-recording video recording in a dashcam recording event analysis method according to one aspect of this application. Figure 4 A flowchart illustrating a video event analysis method for a dashcam according to one aspect of this application is shown. Figure 5 This is a schematic diagram showing the system architecture corresponding to the flow chart of a video recording event analysis method for a dashcam according to one aspect of this application. Detailed Implementation

[0016] The present application will now be described in further detail with reference to the accompanying drawings.

[0017] In a typical configuration of this application, the terminal, the device of the service network, and the trusted party all include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0018] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0019] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0020] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a recording event analysis method for a dashcam, one aspect of this application. The method is applied to a cloud server and requires cooperation between an aftermarket dashcam installed in the vehicle, a user terminal corresponding to the driver, and a cloud server connecting the dashcam and the user terminal. The user terminal can be a user app on a mobile phone, a user app on a smartwatch, or a user app mobile terminal, etc. The method includes steps S11, S12, S13, S14, and S15, specifically including the following steps: Step S11: Receive key video recordings uploaded by an aftermarket dashcam in the vehicle, the key video recordings carrying a sudden attribute tag and a recording timestamp; The sudden attribute tags carried by the key video recording may include, but are not limited to, emergency attribute tags and collision attribute tags. The key video recording includes emergency video recording and collision detection video recording. The emergency video recording is a video recording that is manually triggered by the user, and the collision detection video recording is a video recording that is automatically triggered by the aftermarket dashcam after detecting a collision through a gravity sensor.

[0021] Before step S11, the aftermarket dashcam in the vehicle records emergency footage and collision detection footage, and uploads both types of emergency footage and collision detection footage to the cloud server so that the cloud server can analyze the key footage that needs to be analyzed.

[0022] For example, when recording emergency videos (triggered by the user pressing the emergency button, etc.) and collision detection videos (triggered when the gravity sensor detects acceleration ≥2g, etc.), the aftermarket dashcam automatically adds "emergency" and / or "collision" attribute tags to the corresponding videos in the recordings, and carries a precise timestamp (format: YYYY-MM-DD HH:MM:SS, etc.), and uploads it to the cloud server via 4G / 5G and other mobile networks.

[0023] Step S12: The key video recordings are sequentially filtered and preprocessed to obtain preprocessed key video recordings; Step S13: Identify the event types, extract event time points, and supplement event details in the preprocessed video using a multimodal large model deployed in the cloud; Here, the cloud-deployed multimodal large model is a lightweight large model fine-tuned and optimized for driving scenarios. It is trained by fine-tuning a massive number of driving video samples containing emergency attribute labels and / or collision attribute labels. The event recognition accuracy of the multimodal large model is ≥95%, and the analysis latency is ≤3 seconds per 1-minute video segment. While ensuring analysis accuracy, it reduces cloud computing power consumption and improves analysis response speed. The multimodal large model is based on the general multimodal large model CLIP (Contrastive Language-Image Pre-training). The input is the video frame sequence and timestamps from the preprocessed key videos, and the output is the event type, event time point, and detailed event description for each event.

[0024] In step S13, the preprocessed key video recordings are input into a multimodal large model deployed in the cloud. The multimodal large model performs structured analysis on the video content of the preprocessed key video recordings, specifically including: First, event type identification: Identifying key events occurring in the pre-processed key video recordings. Event types include, but are not limited to, high-frequency driving scenarios such as pedestrians passing close by, vehicle collisions, sudden braking, foreign objects approaching, and vehicles cutting in line. Second, event time point extraction: Combining the video timestamp, extracting the precise time point of each key event (accurate to the minute, such as 2:30). Third, supplementing detailed event descriptions: Providing detailed event descriptions for the identified events, such as labeling collision events with the collision object (e.g., black sedan, electric vehicle), and labeling pedestrian passing events with "pedestrians passing from left to right," etc. In step S13, leveraging the cloud computing power of the cloud server, deep structured analysis of the video content of the key video recordings to be analyzed is achieved, overcoming the computing power limitations of aftermarket dashcams and accurately extracting core information such as event type, event time point, and detailed event descriptions for each event.

[0025] Step S14: Generate a structured event summary text in chronological order. The event summary text includes all events and the event time, event type, and time details for each event. In step S14, the cloud server, based on the analysis results of the multimodal large model, generates a structured event summary text for all event types in chronological order. For example, in a preferred embodiment of this application, the format of the event summary text is: "Summary of today's key events: At 2:30 PM, a pedestrian passed by at close range from left to right; at 3:00 PM, a collision occurred with a black sedan; at 4:15 PM, there was sudden braking, but no collision." Step S14 integrates scattered event information into intuitive and easy-to-understand text, facilitating users to quickly obtain the core content from key video recordings made during emergencies.

[0026] Step S15: Push the event summary text to the user terminal. The user terminal's cloud album module includes an event summary entry point, located at the top of the cloud album module's homepage or elsewhere. Users can access the summary details page containing the event summary text by selecting the entry point. The user terminal responds to the user's selection of the entry point to view the event summary text. While viewing the event summary text, if the user wants to return to the cloud album module's page, they can select a "Return to Cloud Album" button or floating window on the summary details page to return to the cloud album module's page.

[0027] Through the above steps S11 to S15, not only are the technical problems of limited computing power on the back-end device side in the existing technology, which cannot support complex video analysis, and the cloud only has storage function and does not enable intelligent analysis, but also the multimodal large model deployed in the cloud is used to analyze the uploaded key recordings, automatically extract and summarize the event type and event time of each event in the key recordings, realize the user's need to quickly know the core information in the key recordings, and improve the product user experience.

[0028] Following the above embodiments of this application, each event in the event summary text is associated with a corresponding original video link; Specifically, step S15, which pushes the event summary text to the user terminal so that the user terminal can view the event summary text, includes: The event summary text and the original video link corresponding to each associated event are pushed to the user terminal, so that the user terminal can view the event summary text in response to the user's selection operation on the event summary entry, and jump to the video playback page corresponding to the original video link of the target event in response to the user's selection operation on the target event in the event summary text.

[0029] For example, when a user wants to view an event summary, they can select the event summary entry to access the summary details page containing the event summary text. The user terminal responds to the user's selection of the event summary entry to view the event summary text. If, while viewing the event summary text, the user also wants to view the original recordings of all the target events, they can select the "View All Recordings" button or floating window on the summary details page. This allows the user to directly jump from the summary details page to the recording playback page corresponding to the original recording link of the selected target event. This achieves the association and playback between the events in the event summary text and the original recordings, eliminating the need to search for each event individually. It enables efficient access to analysis results while simultaneously meeting the needs for quick understanding and precise viewing, thus improving the user interaction experience.

[0030] Following the above embodiments of this application, step S12 sequentially filters and preprocesses the key video recordings to obtain preprocessed key video recordings, specifically including: The key videos within a preset time period are selected from the key videos and determined as the selected key videos. The selected key videos are processed sequentially by extracting video frame sequences, unifying video resolution, and synchronizing the timestamps corresponding to the videos, resulting in preprocessed key videos.

[0031] Here, the preset time period can be the same day, the morning or afternoon of the same day, or a certain time between times, or a specific time period of a certain date, or any time period at which the user wants to view the video recording of the driving vehicle, such as every day or every few days.

[0032] In a preferred embodiment of this application, the aftermarket dashcam uploads key recordings, such as emergency recordings and collision detection recordings, to a cloud server. The cloud server automatically selects these two types of key recordings from the current day as the filtered key recordings, excluding regular loop recordings and reducing the amount of data to be analyzed. Among them, emergency recordings are videos that are manually triggered by the user, and collision detection recordings are videos that are automatically triggered by the aftermarket dashcam after a collision is detected by the gravity sensor. Both types of key recordings carry a "emergency / collision" attribute tag marked by the device and a timestamp at the time of recording, which enables precise identification of the key recordings to be analyzed, avoids invalid data from consuming cloud computing power, and thus improves analysis efficiency.

[0033] After filtering the key video recordings that need to be analyzed, the cloud server preprocesses the filtered key video recordings, including: extracting video frame sequences, unifying video resolution (to adapt to the input requirements of multimodal large models), and synchronizing the timestamp information corresponding to the recordings (accurate to the minute level, etc.). This standardizes the recordings of videos of different specifications to ensure that they meet the input requirements of multimodal large models, while synchronizing the time information to lay the foundation for subsequent event time extraction.

[0034] Most existing aftermarket dashcams support dual-view recording via a front-facing camera and a rear-facing camera. The accompanying app and cloud album only provide storage and independent preview of the dual-recorded videos. However, in the event of any traffic accident, users must separately view the front and rear recorded videos to manually reconstruct the accident, a cumbersome process. Furthermore, the limited computing power of aftermarket dashcams prevents them from supporting integrated analysis of dual-recorded videos; cloud storage is insufficient, failing to explore the complementary value of dual recording. They also lack specialized accident analysis capabilities, failing to automatically output key accident information, resulting in low efficiency for users in evidence collection and claims processing. In view of the technical deficiencies of the prior art, in one embodiment of this application, when the aftermarket dashcam is recording video, the key video recordings collected may include, but are not limited to, the following: front-facing key video recordings recorded by the front-facing camera of the aftermarket dashcam and rear-facing key video recordings recorded by the rear-facing camera of the aftermarket dashcam; wherein, step S12, which sequentially performs video frame sequence extraction, video resolution unification processing, and timestamp synchronization processing on the selected key video recordings to obtain preprocessed key video recordings, specifically includes: The selected key videos are sequentially processed by video frame sequence extraction, video resolution unification, timestamp synchronization, and viewpoint time difference elimination to obtain preprocessed key videos. Each event in the event summary text is associated with a corresponding original pre-recorded video link and an original post-recorded video link. The user terminal supports simultaneous playback of the video corresponding to the original pre-recorded video link and the video corresponding to the original post-recorded video link.

[0035] In a preferred embodiment of this application, to overcome the shortcomings of aftermarket dashcams such as cumbersome viewing of dual-recorded videos, insufficient edge computing power, and lack of cloud analysis, this application provides an automatic analysis of dual-recorded videos in scenarios such as traffic accidents, driven by a multimodal large model on a cloud server. This analysis extracts key information from traffic accidents and other events, facilitating rapid evidence collection and claims processing. In this preferred embodiment, The selected key videos are sequentially processed by video frame sequence extraction, video resolution unification, timestamp synchronization, and viewpoint time difference elimination to obtain preprocessed key videos. Each event in the event summary text is associated with a corresponding original pre-recorded video link and an original post-recorded video link. The user terminal supports simultaneous playback of the video corresponding to the original pre-recorded video link and the video corresponding to the original post-recorded video link.

[0036] In a preferred embodiment of this application, the event analysis process for dual-recording video recording is as follows: Figure 2As shown, the aftermarket dashcam uploads both the front-facing key video recording from its front camera and the rear-facing key video recording from its rear camera to a cloud server. Both the front-facing and rear-facing key videos carry recording timestamps and viewpoint tags. Upon receiving the front-facing and rear-facing key videos, the cloud server filters for dual-recorded key videos related to accidents of the same day. It then sequentially performs video frame sequence extraction, video resolution unification, and timestamp synchronization on the filtered front-facing and rear-facing key videos. The system performs processing and perspective time difference elimination to obtain preprocessed key video recordings. Then, the preprocessed key video recordings, including dual-recorded video, are input into a cloud-deployed multimodal large-scale model to identify the event type, collision location, and preliminary determination of liability. Finally, an event summary text related to the accident is generated. This summary text can be presented in report form, and is structured according to the format of "time + perspective + key information" and pushed to the user terminal. The event summary text related to the traffic accident is displayed in the user terminal's app, along with a link to the original dual-recorded video recordings, supporting synchronized playback. Furthermore, the system architecture corresponding to the event analysis process of dual-recorded video recordings is as follows: Figure 3 As shown.

[0037] In this preferred embodiment, the analysis of dual-recorded video recordings can improve accident analysis efficiency by 85%, eliminating the need to manually switch between and view the dual-recorded videos; it also ensures that the accuracy of extracting key accident information is ≥95%, facilitating rapid evidence collection and claims processing; it is more compatible with aftermarket dashcams, requiring no hardware upgrades; in addition, dual-recording fusion analysis is more comprehensive than single-view analysis, improving accident reconstruction accuracy by 40%.

[0038] Following the above embodiments of this application, one aspect of this application provides a method for analyzing video recording events of a dashcam, which further includes: Identify the continuity and event correlation of video content in at least two key video clips that all carry the same suddenness attribute label; If the recorded content is continuous and all related to the same event scene, then the events of the at least two key recorded videos are determined to be events corresponding to the same event type. The events of the at least two key recorded videos are merged into one event, and the at least two key recorded videos are all associated with the original video links of the events corresponding to the same event type in the event summary text.

[0039] In a preferred embodiment of this application, if the collision at 15:00 triggers two consecutive key video recordings (recorded at 15:00-15:01 and 15:01-15:02 respectively) due to continuous vibration, both key video recordings carry the "collision" attribute tag. During cloud-based big data model analysis, by identifying the continuity of the video content (the connection of the collision process) and the correlation of the events (both are the same collision scene) of the two key video recordings, if the video content is continuous and both are related to the same event scene, it is determined to be the same collision event. Then, this collision event is merged into one event, and only one sentence is recorded in the event summary text: "Collision with a black car at 15:00". The original video link associated with this event includes these two consecutive key video recordings. After clicking, the user can play the two video recordings in sequence to view the complete collision process. If the same event is detected to be recorded by multiple consecutive video recordings (such as multiple recordings triggered by the collision process), the multimodal big data model in the cloud server automatically identifies and merges them into one event, avoiding duplicate summaries. This not only improves the conciseness and practicality of the event summary text but also avoids redundant information from interfering with the user.

[0040] In a preferred embodiment of the actual application scenario of this application, such as Figure 4 The diagram shown is a flowchart illustrating a recording event analysis method for a dashcam according to one aspect of this application. The corresponding system architecture diagram is shown below. Figure 5 As shown, in the preferred embodiment of this application, firstly, video uploading and filtering are required: During the user's driving, emergency recording is triggered at 14:30 due to a pedestrian passing close by; collision detection recording is triggered at 15:00 due to a collision with a black sedan; and emergency recording is triggered at 16:15 due to sudden braking. The aftermarket dashcam uploads these three recordings to the cloud server. After receiving these three recordings, the video filtering module in the cloud server automatically filters out the three key recordings of the day (2026-XX-14:30:05, 2026-XX-15:00:12, and 2026-XX-16:15:30 respectively). Then, the video processing module in the cloud server preprocesses the three key video segments, extracts the key frames of the video corresponding to each key video segment (1 frame every 2 seconds), unifies the resolution of the three key video segments to 1920×1080, and synchronizes the timestamp information of each key video segment to ensure accurate time correspondence, thereby realizing the preprocessing of the selected three key video segments. Next, through cloud-based big data analysis on the cloud server, the video data corresponding to the three pre-processed key video segments are input into the multimodal big data model deployed in the cloud. The multimodal big data model analyzes the video frames in the video data corresponding to the three pre-processed key video segments by performing image recognition and timestamp matching, and outputs the following: 14:30 Pedestrians pass by at close range from left to right; 15:00 Collision with the black car occurs; 16:15 Sudden braking occurs, no collision. Next, in the summary generation module on the cloud server, a structured event summary text is generated in chronological order: "Key event summary for [Date]: 1. At 14:30, a pedestrian passed by at close range from left to right; 2. At 15:00, a collision occurred with a black sedan; 3. At 16:15, there was sudden braking, but no collision occurred. Click on the event to view the corresponding video footage." This completes the generation of the event summary text corresponding to the three key video clips. Finally, the cloud server pushes the summary text to the user's linked mobile app via push service. The user opens the app, enters the cloud album module, and clicks the "Daily Event Summary" entry in the cloud album module to view the event summary text corresponding to these three key videos. When the user clicks on the "15:00 Collision with a black sedan" event, the mobile app directly jumps to the playback page of the collision detection video corresponding to that collision event for quick viewing. Clicking the "View All Videos" button returns to the original video list in the cloud album.

[0041] In the embodiments of this application, key videos are recorded by an aftermarket dashcam and uploaded to a cloud server. The video filtering module of the cloud server filters the key videos, the video preprocessing module standardizes the video data corresponding to each video, the cloud big data model analysis module outputs event information, the summary generation module generates event summary text, and the cloud server pushes the event summary text to the user terminal's APP display and interaction module, so that the user can view the event summary text through the user terminal's APP and jump to view the original video corresponding to each event, forming a complete dynamic closed loop of "recording-upload-analysis-summary-display-interaction". The modules work together to achieve the core functions.

[0042] In the embodiments of this application, users do not need to view the emergency / collision videos in the cloud album of the user terminal one by one. They can quickly know the event type and time of all key events of the day through the structured event summary text, which can save users a lot of video viewing time and thus greatly improve user efficiency. The event summary text is concise and intuitive, allowing even non-professional users to quickly understand key driving events within a preset time period. It also features a function to jump to the original video recording, balancing the needs of "quick understanding" and "precise viewing." This reduces the learning curve for users by 90%, lowering the barrier to entry.

[0043] By fully utilizing the computing power of cloud servers, the problem of limited computing power on the end side of aftermarket dashcams can be solved. There is no need to upgrade the hardware of aftermarket dashcams. Existing aftermarket dashcam models can be directly used. The hardware cost per unit is zero, which meets the pricing requirements of consumer products and is conducive to large-scale promotion, thus adapting to the characteristics of aftermarket products. Compared to existing cloud photo albums that only have storage functions, the addition of intelligent analysis and summary functions creates a differentiated advantage. Market research shows that this function can increase users' willingness to purchase aftermarket dashcam products by more than 35%, enhance user loyalty to the brand, and improve product competitiveness. By deduplicating events, we avoid summarizing the same event repeatedly, while supplementing key details of the event. The event identification accuracy rate is ≥95%, and the analysis delay is ≤3 seconds / segment of 1 minute video recording. This ensures that the summary content is accurate and responsive, improves user trust, and thus guarantees the accuracy and practicality of the summary. By quickly presenting key driving events, it helps users efficiently handle matters such as accident claims and traffic violation reports, reducing dispute resolution time and indirectly improving driving safety awareness, thus creating a better driving environment.

[0044] According to another aspect of this application, a non-volatile storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a processor, cause the processor to implement the video recording event analysis method of the dashcam as described above.

[0045] According to another aspect of this application, a recording event analysis device for a dashcam is also provided, wherein the device includes: One or more processors; Computer-readable medium for storing one or more computer-readable instructions. When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the video event analysis method of the dashcam as described above.

[0046] For details of the various embodiments of the dashcam's recording event analysis device, please refer to the corresponding parts of the above-described embodiments of the dashcam's recording event analysis method; they will not be repeated here.

[0047] In summary, this application receives key video recordings uploaded by an aftermarket dashcam in a vehicle via a cloud server. These key video recordings carry sudden event attribute tags and recording timestamps. The key video recordings are sequentially filtered and preprocessed to obtain preprocessed key video recordings. A multimodal large model deployed in the cloud identifies event types, extracts event time points, and supplements event details in the preprocessed video recordings. A structured event summary text is generated in chronological order, including all events and the corresponding event time point, event type, and time detail description for each event. The event summary text is then pushed to the cloud. The user terminal includes an event summary entry in its cloud album module. This allows the user terminal to view the event summary text in response to the user's selection of the event summary entry. This not only solves the technical problems of limited computing power on the back-end device side, which cannot support complex video analysis, and the cloud's storage function, which does not enable intelligent analysis, but also analyzes the uploaded key recordings through a multimodal large model deployed in the cloud. It automatically extracts and summarizes the event type and time point of each event in the key recordings, fulfilling the user's need to quickly understand the core information in the key recordings and improving the product user experience.

[0048] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0049] Furthermore, a portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. The program instructions invoking the methods of this application may be stored in a fixed or removable recording medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the working memory of a computer device operating according to the program instructions. Here, one embodiment of this application includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the apparatus is triggered to operate the methods and / or technical solutions based on the foregoing embodiments of this application.

[0050] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A method for analyzing video events from a dashcam, wherein the method is applied to a cloud server, wherein, The method includes: Receive key video recordings uploaded by an aftermarket dashcam in the vehicle, the key video recordings carrying a sudden attribute tag and a recording timestamp; The key video recordings are sequentially filtered and preprocessed to obtain preprocessed key video recordings; The event types in the preprocessed video are identified, event time points are extracted, and supplementary event details are obtained through a multimodal large model deployed in the cloud. A structured event summary text is generated in chronological order. The event summary text includes all events and the corresponding event time, event type, and time details for each event. The event summary text is pushed to the user terminal, wherein the cloud album module of the user terminal is provided with an event summary entry, so that the user terminal can view the event summary text in response to the user's selection operation of the event summary entry.

2. The method according to claim 1, wherein, Each event in the event summary text is associated with a corresponding original video link; The step of pushing the event summary text to the user terminal so that the user terminal can view the event summary text includes: The event summary text and the original video link corresponding to each associated event are pushed to the user terminal, so that the user terminal can view the event summary text in response to the user's selection operation on the event summary entry, and jump to the video playback page corresponding to the original video link of the target event in response to the user's selection operation on the target event in the event summary text.

3. The method according to claim 1, wherein, The step of sequentially filtering and preprocessing the key video recordings to obtain preprocessed key video recordings includes: The key videos within a preset time period are selected from the key videos and determined as the selected key videos. The selected key videos are processed sequentially by extracting video frame sequences, unifying video resolution, and synchronizing the timestamps corresponding to the videos, resulting in preprocessed key videos.

4. The method according to claim 3, wherein, The key recordings include front-view key recordings recorded by the front-facing camera of the aftermarket dash cam and rear-view key recordings recorded by the rear-facing camera of the aftermarket dash cam. The process of sequentially extracting video frame sequences, unifying video resolution, and synchronizing timestamps corresponding to the selected key video recordings to obtain preprocessed key video recordings includes: The selected key videos are sequentially processed by video frame sequence extraction, video resolution unification, timestamp synchronization, and viewpoint time difference elimination to obtain preprocessed key videos. Each event in the event summary text is associated with a corresponding original pre-recorded video link and an original post-recorded video link. The user terminal supports simultaneous playback of the video corresponding to the original pre-recorded video link and the video corresponding to the original post-recorded video link.

5. The method according to any one of claims 1 to 4, wherein, The method further includes: Identify the continuity and event correlation of video content in at least two key video clips that both carry the same suddenness attribute label. If the recorded content is continuous and all related to the same event scene, then the events of the at least two key recorded videos are determined to be events corresponding to the same event type. The events of the at least two key recorded videos are merged into one event, and the at least two key recorded videos are all associated with the original video links of the events corresponding to the same event type in the event summary text.

6. The method according to any one of claims 1 to 4, wherein, The suddenness attribute label includes an emergency attribute label and a collision attribute label, and the key video recording includes emergency video recording and collision detection video recording. The emergency recording is a video recording triggered manually by the user, and the collision detection recording is a video recording automatically triggered by the aftermarket dashcam after a collision is detected by the gravity sensor.

7. The method according to any one of claims 1 to 4, wherein, The cloud-deployed multimodal large model is a lightweight large model that has been fine-tuned and optimized for driving scenarios. It is trained by fine-tuning a large number of driving video samples containing emergency attribute labels and / or collision attribute labels. The event recognition accuracy of the multimodal large model is ≥95% and the analysis latency is ≤3 seconds / segment of 1 minute video.

8. A non-volatile storage medium having stored computer-readable instructions thereon, which, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 7.

9. A recording event analysis device for a vehicle dashcam, wherein, The device includes: One or more processors; Computer-readable medium for storing one or more computer-readable instructions. When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors perform the method as described in any one of claims 1 to 7.

Images