Method and system for fusing intelligent driving data and driving video and sharing same

Abstract

A method and system for fusing intelligent driving data and a driving video and sharing same. The method comprises: collecting road surface image information by means of a first object; when an intelligent driving system performs a vehicle control action, performing video clipping by means of a second object; and fusing the clipped video with intelligent driving data, performing pixelization-based desensitization, and transmitting the video to a service platform by means of a network cloud service. The method does not rely on additional hardware devices, but relies only on a vehicle-mounted radar and a camera to capture images in different scenarios, and performs danger recognition and determination by means of a visual algorithm; data statistics and video clipping are fused by means of a DVR application and an intelligent driving application; and a desensitization method and a network sharing service are provided for a user to share.

Description

A method and system for integrating and sharing intelligent driving data and driving videos.

[0001] This invention claims priority to Chinese Patent Application No. 202510012942.X, filed on January 6, 2025, entitled “A Method and System for Integrating and Sharing Intelligent Driving Data and Driving Video”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of intelligent driving recorder technology, specifically to a method and system for integrating intelligent driving data and driving videos and sharing them. Background Technology

[0003] Most vehicles with intelligent driving features on the market now include a dashcam function, and the dashcam is stored in solid-state storage, such as a USB flash drive or a car infotainment system; it cannot be reused for sharing.

[0004] The intelligent driving data of user vehicles has not yet been fully released and popularized, and the statistical methods are slightly different. Only a very small number of new car manufacturers can obtain and share it through mobile phones.

[0005] Pain point: Most users are now willing to share their intelligent driving data and videos along the way to their WeChat Moments and social platforms, but the popularization of this function needs to be improved. As a social feature, it has great potential for future development. Summary of the Invention

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] Therefore, the technical problem solved by this invention is that existing intelligent driving data and roadside videos cannot be reused and shared.

[0008] To address the aforementioned technical problems, this invention provides the following technical solution: a method for integrating and sharing intelligent driving data and driving videos, comprising: acquiring road image information through a first object; cropping the video through a second object when the intelligent driving system performs vehicle control actions; integrating the cropped video with the intelligent driving data, performing mosaic decryption, and transmitting the video to a service platform via a network cloud service.

[0009] As a preferred embodiment of the method for integrating intelligent driving data and driving video and sharing them according to the present invention, after collecting road image information through the first object, the method further includes identifying the size of the object through a multi-frame rate image visual algorithm, measuring the distance between the vehicle and the object using radar, finding suspicious areas, and controlling the vehicle.

[0010] As a preferred embodiment of the method for integrating intelligent driving data and driving video and sharing it according to the present invention, the video cropping includes: when the intelligent driving system performs a vehicle control action, the intelligent driving domain controller sends a CAN signal to notify a second object in the vehicle to crop the video.

[0011] As a preferred embodiment of the method for integrating intelligent driving data and driving video and sharing it according to the present invention, after the video is cropped, the method further includes that when the user shifts into Park (P) gear, the second object saves the spliced ​​video, adds a time tag, and saves it in MP4 format to a designated folder.

[0012] As a preferred embodiment of the method for integrating intelligent driving data and driving videos and sharing them as described in this invention, the intelligent driving data includes driving mileage, duration, number of lane changes, number of parking entries, number of exits, and average time consumption data.

[0013] As a preferred embodiment of the method for integrating and sharing intelligent driving data and driving videos described in this invention, the method further includes: after integrating the cropped video with the intelligent driving data, integrating the cropped video with the visualization data to form a display format where the left side is the video and the right side is the data list; and ensuring that the video start time is consistent with the data update time based on the timestamp on the video, thereby achieving the integrated display of the video and the intelligent driving data.

[0014] As a preferred embodiment of the method for integrating intelligent driving data and driving video and sharing them as described in this invention, the mosaic decryption includes: capturing feature regions using facial key part recognition technology and license plate color recognition technology; tracking and capturing the captured feature regions to generate a unique feature code; and replacing the feature code with a mosaic image to achieve mosaic decryption.

[0015] A system for sharing intelligent driving data and driving video using any of the methods described in this invention, comprising: an identification module that collects road image information through a first object, processes and judges the high-level information processed by the visual algorithm, identifies suspicious areas, and controls the vehicle; a cropping module that receives a CAN signal sent by the intelligent driving domain controller, begins cropping the video, saves the spliced ​​video when the user shifts into Park, and merges it with the visualization data according to the timestamp to obtain a fused video; and a desensitization module that performs mosaic desensitization on the fused video and transmits the video to the service platform via a network cloud service.

[0016] A computer device includes: a memory and a processor; the memory stores a computer program, including: the steps of the processor executing the computer program to implement the method described in any one of the present invention.

[0017] A computer-readable storage medium having a computer program stored thereon, comprising the steps of implementing the method described in any one of the present invention when the computer program is executed by a processor.

[0018] The beneficial effects of this invention are as follows: The method of this invention does not rely on additional hardware devices, but only on vehicle radar and cameras to capture images in different scenarios, and uses visual algorithms to identify and judge dangers; it achieves data statistics and video cropping fusion through DVR app and intelligent driving app; and it provides de-identification methods and network sharing services for users to share. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0020] Figure 1 is an overall flowchart of a method for integrating intelligent driving data and driving video and sharing them according to an embodiment of the present invention;

[0021] Figure 2 is a video editing and data fusion logic diagram of a method for sharing intelligent driving data and driving video provided in an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of vehicle control according to a method for integrating intelligent driving data and driving video and sharing them, provided in the second embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a method for fusing intelligent driving data and driving video and sharing them, provided by the second embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] Example 1, referring to Figures 1-4, is an embodiment of the present invention, providing a method for integrating and sharing intelligent driving data and driving video, including:

[0026] In S1: Road surface image information is acquired through the first object.

[0027] In S2: When the intelligent driving system performs a vehicle control action, the video is cropped using a second object.

[0028] In S3: The cropped video is integrated with intelligent driving data, mosaic decryption is performed, and the video is transmitted to the service platform via network cloud service.

[0029] The first object can be a dual-view camera, a surround-view camera, a rear camera, etc.

[0030] Furthermore, based on the road surface image information collected from the first object (such as information about obstacles and parking spaces in the road), the size of the object is identified through a multi-frame rate image visual algorithm, and the distance between the vehicle and the object is measured using millimeter-wave radar / ultrasonic radar. The recognition system processes and judges the high-level information processed by the visual algorithm, finds suspicious areas, and controls the vehicle.

[0031] Specifically, the vehicle control system is shown in Figure 3. For example, in the driving section: if an obstacle is detected ahead, the vehicle will actively avoid it; if traffic congestion is detected, the vehicle will actively change lanes to a smooth road section. At the same time, based on the navigation map coordinates and the remembered route plan, the vehicle will actively enter / exit ramps. In the parking section: if the vehicle enters a narrow road section and wants to reverse, the vehicle will be controlled to reverse back along the original route. When the system detects a parking space, the steering wheel and accelerator will be controlled to perform actions such as parking in and out.

[0032] Furthermore, the second object is a DVR app or other object that can be connected to the smart domain controller and have video cropping functionality implemented.

[0033] When the intelligent driving system performs a vehicle control action, the intelligent driving domain controller will send a CAN signal (from the start to the end of vehicle control) to notify the DVR app in the vehicle to trim the video. When the user shifts into Park, the DVR app will save the spliced ​​video, add a time stamp, and save it in MP4 format to the designated folder.

[0034] DVR videos are recorded in duration. If a CAN signal triggering editing is received, a temporary file is created for storage. When a CAN signal indicating the end is received, the temporary file is saved and stored in the vehicle's EMMC memory. Encoding and fusion are then performed after the intelligent driving data is transmitted back. This is primarily used for scenarios such as automatic braking, obstacle avoidance, and ramp entry / exit. The key to the algorithm lies in the accuracy of the start and end markers.

[0035] Furthermore, the intelligent driving app can also collect statistics on intelligent driving data, including driving mileage, duration, number of lane changes, number of parking entries, number of exits, average time, and other key data, and generate statistical tables.

[0036] The intelligent driving app integrates the cropped video with the visualized data, combining them into a display format where the video is on the left and the data list is on the right, as shown in Figure 4.

[0037] Specifically, the video has a timestamp. As long as the video start time matches the data update time, they can be displayed together. The format is MP4, using H.264 encoding.

[0038] Furthermore, the fused video will be decrypted using facial recognition technology (key facial features) and license plate color recognition technology, and then transmitted to the service platform via cloud services. Users can remotely retrieve and save the cropped video through a mobile app, which can then be shared on social media platforms.

[0039] The camera captures facial information and divides the face into multiple feature regions, such as eyes, mouth, nose, and eyebrows. The algorithm tracks and extracts these feature regions to generate a unique facial feature code. Simultaneously, a mosaic image is used to replace the feature code, effectively desensitizing the face. License plates are categorized as blue or green, and the capture and replacement principle is the same. If a feature code is missing, the mosaic image is discarded and replaced. By increasing the number of facial recognition subjects (e.g., children, adults, and the elderly) in the training, the feature database is expanded to ensure accuracy and security.

[0040] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0041] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0042] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0043] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0044] Example 2, in an exemplary embodiment, also provides a system for integrating intelligent driving data and driving video and sharing it, including an identification module, comprising a camera device and a radar device, which is connected to the cropping module and used to acquire image information around the vehicle.

[0045] The first object can be a front-view camera, a surround-view camera, or a rear camera. The camera identifies road objects, such as obstacles or parking spaces, through a multi-frame-rate image visual algorithm. The radar equipment includes millimeter-wave radar and ultrasonic radar to measure the distance between the vehicle and the object. The recognition system judges and processes the advanced information after the visual algorithm is processed, finds suspicious areas, and controls the vehicle.

[0046] In one embodiment, a trimming module is also included;

[0047] The cropping module is connected to the recognition module and the intelligent driving domain controller, and is used to receive CAN signals sent by the intelligent driving domain controller to crop the video.

[0048] When the intelligent driving system detects vehicle control behavior, the intelligent driving domain controller sends a start signal to the trimming module via the CAN bus, notifying the DVR application in the vehicle to start the video trimming function. The video trimming function marks the start and end of the intelligent driving control behavior.

[0049] In one embodiment, a desensitization module is also included;

[0050] The desensitization module is connected to the cropping module and is used to desensitize the cropped and merged video, and transmit the video to the service platform through network cloud service.

[0051] The desensitization module includes facial recognition and license plate recognition functions. It uses a camera to capture key facial features (such as eyes, mouth, nose, and eyebrows) to divide the face into regions. The algorithm tracks features based on these regions, generates a unique facial feature code, and then replaces the feature code with a mosaic image to achieve desensitization. For license plate information, it identifies the vehicle by color (such as blue or green) and uses a similar replacement principle as for facial desensitization.

[0052] Example 3, in an exemplary embodiment, also provides a computer program product, including a computer program that, when executed by a processor, implements the following method steps, including:

[0053] Collect road surface image information of the first object;

[0054] Based on the road surface image information, visual algorithms and radar data are used to identify obstacles and suspicious areas, and to control the vehicle's driving behavior.

[0055] Based on the vehicle's driving behavior, the cropping module is triggered to crop the video;

[0056] Intelligent driving data is collected and fused with the video data to form a fused video.

[0057] In one implementation, the computer program described above can perform the following steps:

[0058] The system collects raw road surface image data of the first object, and uses visual algorithms to identify obstacles, parking spaces, and other information on the road surface based on the raw image data, generating processed high-level image information.

[0059] Obtain the road surface image information of the first object.

[0060] Based on the image information, the distance between the vehicle and obstacles is measured using millimeter-wave radar or ultrasonic radar, and the vehicle's driving behavior is controlled based on this data, including active avoidance, lane changing, entering and exiting ramps, and parking.

[0061] When the vehicle performs a vehicle control action, the cropping module is triggered to crop the video and mark the start and end times of the video.

[0062] Based on the driving data generated by the control behavior, corresponding video segments are collected and fused with the driving data to generate a fused video with a timestamp.

[0063] Sensitive information in videos (such as faces and license plates) is de-identified, and mosaic technology is used to replace sensitive areas to ensure user privacy and security.

[0064] The processed and merged video is uploaded to the service platform via cloud services, allowing users to view and share it remotely via mobile app or in-vehicle infotainment screen.

[0065] When the desensitization process is triggered, the following steps are performed:

[0066] Sensitive information captured in videos is marked using facial recognition and license plate recognition technologies;

[0067] The sensitive areas are replaced with mosaics to generate a desensitized video.

[0068] The anonymized videos are uploaded to a cloud server for users to view and share at any time.

[0069] Perform the following steps each time you upload a merged video:

[0070] Replace the previous video data with the currently generated fused video;

[0071] A new fused video is generated based on the updated data and stored in the vehicle system;

[0072] The newly generated video is uploaded to the cloud server and synchronized to the user's mobile app or in-vehicle infotainment screen.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for integrating and sharing intelligent driving data and driving videos, characterized in that, include: Road surface image information is acquired through the first object; When the intelligent driving system performs a vehicle control action, the video is cropped using a second object; The cropped video is integrated with intelligent driving data, decrypted using mosaic technology, and then transmitted to the service platform via cloud services.

2. The method for integrating intelligent driving data and driving video and sharing them as described in claim 1, characterized in that: After acquiring road surface image information through the first object, the process also includes: By using multi-frame-rate image visual algorithms to identify the size of objects, and using radar to measure the distance between the vehicle and the objects, suspicious areas can be identified and the vehicle can be controlled.

3. The method for integrating intelligent driving data and driving video and sharing them as described in claim 2, characterized in that: The video cropping includes the following: when the intelligent driving system performs a vehicle control action, the intelligent driving domain controller sends a CAN signal to notify a second object in the vehicle's infotainment system to crop the video.

4. The method for integrating intelligent driving data and driving video and sharing them as described in claim 3, characterized in that: After the video is cropped, it also includes, When the user selects P mode, the second object will save the spliced ​​video, add a time tag, and save it in MP4 format to the specified folder.

5. The method for integrating intelligent driving data and driving video and sharing them as described in claim 4, characterized in that: The intelligent driving data includes driving mileage, duration, number of lane changes, number of times the vehicle enters and exits the parking space, and average driving time.

6. The method for integrating intelligent driving data and driving video and sharing them as described in claim 5, characterized in that: After integrating the cropped video with intelligent driving data, the process also includes: The cropped video is combined with the visualized data to form a display format where the video is on the left and the data list is on the right. Based on the timestamps in the video, the video start time is kept consistent with the data update time, enabling the integrated display of video and intelligent driving data.

7. The method for integrating intelligent driving data and driving video and sharing them as described in claim 6, characterized in that: The mosaic decryption process includes capturing feature regions using facial key feature recognition technology and license plate color recognition technology, tracking and capturing the captured feature regions to generate a unique feature code, and replacing the feature code with a mosaic image to achieve mosaic decryption.

8. A system for integrating intelligent driving data and driving video and sharing them using any one of the methods described in claims 1 to 7, characterized in that, include, The recognition module collects road image information from the first object, processes and judges the advanced information after the visual algorithm, identifies suspicious areas, and controls the vehicle. The cropping module receives the CAN signal sent by the intelligent driving domain controller and begins to crop the video. When the user shifts into Park, the spliced ​​video is saved and merged with the visualization data according to the timestamp to obtain the merged video. The desensitization module performs mosaic desensitization on the merged video and transmits the video to the service platform via network cloud service.

9. A computer device, comprising: Memory and processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, it implements the steps of the method for fusing intelligent driving data and driving video and sharing them as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method for fusing intelligent driving data and driving video and sharing them as described in any one of claims 1-7.

Images