Method and system for fusing and sharing intelligent driving data and driving video

Abstract

The invention relates to the technical field of intelligent driving records, and discloses a method and system for fusing and sharing intelligent driving data and driving videos, and the method comprises the steps: collecting an intelligent driving controller signal; when the intelligent driving system makes a vehicle control behavior, performing video clipping through the first object; and fusing the cut video with the intelligent driving data, performing mosaic decryption, and transmitting the video to a service platform through network cloud service. The method does not depend on additional hardware equipment, only depends on a vehicle-mounted radar and a camera to capture images in different scenes, and carries out danger identification and makes a judgment through a visual algorithm; fusion of data statistics and video clipping is realized through a DVR app and an intelligent driving app; and providing a desensitization method and a network sharing service for users to share.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving record, and specifically to a method and system for integrating intelligent driving data and driving videos and sharing them. Background Art

[0002] Currently, most intelligent driving equipped vehicles on the market include a driving recorder function, which is stored in a solid-state memory, such as a USB flash drive or a car computer, and cannot be reused for sharing.

[0003] The intelligent driving data of users' vehicles has not been fully released and popularized, and the statistical methods are slightly different. Only a very small number of new energy vehicle manufacturers can obtain and share it through mobile phones.

[0004] Pain point: Currently, most users are willing to share their intelligent driving data and along-the-way videos on their WeChat Moments and social platforms, but the popularization of this function needs to be improved. As a social attribute, it has great development prospects in the future. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is that the existing intelligent driving data and along-the-way videos cannot be reused for sharing.

[0007] To solve the above technical problem, the present invention provides the following technical solution: A method for integrating intelligent driving data and driving videos and sharing them, including: collecting signals from an intelligent driving controller; when the intelligent driving system makes a vehicle control action, cropping the video through a first object; fusing the cropped video with the intelligent driving data, decrypting the mosaic, and transmitting the video to a service platform through network cloud services.

[0008] As a preferred solution of the method for integrating intelligent driving data and driving videos and sharing them according to the present invention, before collecting the signals from the intelligent driving controller, it further includes collecting road surface information, identifying the size of an object through a multi-frame rate image vision algorithm, measuring the distance between the vehicle and the object using a radar, finding a suspicious area, and controlling the vehicle.

[0009] As a preferred solution of the method for integrating intelligent driving data and driving videos and sharing them according to the present invention, the vehicle control actions include, but are not limited to, making a vehicle control action when an obstacle is identified ahead, vehicle congestion is identified, the vehicle enters a narrow section and wants to reverse, or a parking space is detected.

[0010] As a preferred solution of the method for fusing intelligent driving data and driving videos and sharing them according to the present invention, wherein: the fusion of the video and the intelligent driving data further includes that when the user shifts to the P gear, the first object saves the spliced video and attaches a time tag.

[0011] As a preferred solution of the method for fusing intelligent driving data and driving videos and sharing them according to the present invention, wherein: the intelligent driving data includes at least one of driving mileage, duration, number of lane changes, number of parking entries, number of exits, and average time-consuming data.

[0012] As a preferred solution of the method for fusing intelligent driving data and driving videos and sharing them according to the present invention, wherein: after fusing the cropped video with the intelligent driving data, it further includes fusing the cropped video with the visualization data to form a display format with the video on the left and the data list on the right; according to the time stamp carried by the video, keeping the start time of the video consistent with the data update time to achieve the fused display of the video and the intelligent driving data.

[0013] As a preferred solution of the method for fusing intelligent driving data and driving videos and sharing them according to the present invention, wherein: the mosaic decryption includes using face key part recognition technology and license plate color recognition technology to capture the feature area, tracking and capturing according to the captured feature area to generate a unique feature code, and at the same time using the mosaic picture to replace the feature code to achieve mosaic decryption.

[0014] A system for fusing intelligent driving data and driving videos and sharing them using any of the methods of the present invention, wherein: an identification module, which collects road surface image information through the first object, processes and judges the advanced information after visual algorithm processing, finds out the suspicious areas, and controls the vehicle; a cropping module, which receives the can signal sent by the intelligent driving domain controller, starts to crop the video, saves the spliced video when the user shifts to the P gear, and fuses it with the visualization data according to the time stamp to obtain a fused video; a desensitization module, which performs mosaic decryption on the fused video and transmits the video to the service platform through the network cloud service.

[0015] A computer device, comprising: a memory and a processor; the memory stores a computer program, including: when the processor executes the computer program, the steps of any of the methods described in the present invention are implemented.

[0016] A computer-readable storage medium, on which a computer program is stored, including: when the computer program is executed by a processor, the steps of any of the methods described in the present invention are implemented.

[0017] Advantages of the present invention: The method of the present invention does not rely on additional hardware devices. It only relies on on-vehicle radars and cameras to capture images in different scenarios, and uses vision algorithms to identify dangers and make judgments; it realizes the integration of data statistics and cropped videos through the DVR app and the intelligent driving app; it provides a desensitization method and a network sharing service for users to share. Description of the Drawings

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0019] Figure 1 It is the overall flowchart of a method for integrating intelligent driving driving data and driving videos and sharing them provided by an embodiment of the present invention;

[0020] Figure 2 It is the video editing and data fusion logic diagram of a method for integrating intelligent driving driving data and driving videos and sharing them provided by an embodiment of the present invention;

[0021] Figure 3 It is the vehicle control schematic diagram of a method for integrating intelligent driving driving data and driving videos and sharing them provided by the second embodiment of the present invention;

[0022] Figure 4 It is the fused video schematic diagram of a method for integrating intelligent driving driving data and driving videos and sharing them provided by the second embodiment of the present invention. Detailed Embodiments

[0023] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will give a detailed description of the specific embodiments of the present invention with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0024] Embodiment 1, referring to Figures 1 to 4 , which is an embodiment of the present invention, provides a method for integrating intelligent driving driving data and driving videos and sharing them, including:

[0025] In S1: Collect the signals of the intelligent driving controller.

[0026] In S2: When the intelligent driving system makes a vehicle control action, perform video cropping through the first object.

[0027] In S3: Integrate the cropped video with the intelligent driving data, perform mosaic decryption, and transmit the video to the service platform through network cloud services.

[0028] Furthermore, collect road surface image information through a collection device, which can be a binocular front camera, a surround camera, a rear camera, etc.

[0029] Furthermore, according to the road surface image information collected by the collection device (such as information about obstacles and parking spaces on the road), identify the size of the object through a multi-frame rate image vision algorithm, measure the distance between the vehicle and the object using a millimeter wave radar / ultrasonic radar, and the recognition system processes and judges the high-level information processed by the vision algorithm to find suspicious areas and control the vehicle.

[0030] Specifically, the vehicle control part is as Figure 3 shown. For the driving part: If an obstacle is detected ahead, control the vehicle to actively avoid it; if vehicle congestion is detected, actively change lanes to a smooth section, and at the same time, according to the navigation map coordinates and the memorized path planning, control the vehicle to actively enter / exit the ramp; for the parking part: When the vehicle enters a narrow section and wants to reverse, control the vehicle to reverse back along the original route, and when the system detects a parking space, control the steering wheel and throttle to perform actions such as parking in and out.

[0031] Furthermore, the first object is DVRapp or other objects that can be connected to the intelligent driving domain controller and implement the video cropping function.

[0032] When the intelligent driving system makes a vehicle control action, the intelligent driving domain controller will send a can signal (from the start to the end of vehicle control) to notify the DVR app in the car machine to crop the video. When the user hangs the P gear, the DVRapp will save the spliced video and attach a time tag, and save it in the specified folder in the MP4 format.

[0033] The DVR video is recorded by duration. If a CAN signal for triggering editing is received, a temporary file will be created for storage. When the end CAN signal is received, the temporary file will be saved in the EMMC memory of the car machine and encoded and integrated after the intelligent driving data is transmitted back, mainly for scenarios such as automatic braking, detouring and avoidance, and entering / exit ramps. The key to the algorithm lies in the accuracy of the start and end flag bits.

[0034] Even further, the intelligent driving app can also count the intelligent driving travel data, including key data such as driving mileage, duration, number of lane changes, number of parking entries, number of exits, and average time consumption, and form a statistical table-like interface.

[0035] The intelligent driving app combines the cropped video with the visualization data to form a display format with the video on the left and the data list on the right, as Figure 4 shown.

[0036] Specifically, the video has timestamps. As long as the video start time is kept consistent with the data update time, they can be combined and displayed. The format is MP4, and the encoding format is H.264.

[0037] Furthermore, the combined video will use face key part (five sense organs) recognition technology and license plate color recognition technology for mosaic decryption, and transmit the video to the service platform through network cloud services. Users can remotely retrieve the cropped video through the mobile app and save it for sharing on social platforms.

[0038] The camera captures face information and divides the face into multiple feature regions, such as eyes, mouth, nose, eyebrows, etc. The algorithm tracks and captures based on the captured feature regions to generate a unique face feature code, and at the same time uses mosaic pictures for feature code replacement to effectively perform desensitization; license plates are divided into blue and green, and the capture and replacement principle is the same. If the feature code is found to be lost, the mosaic picture replacement is discarded, and by increasing face recognition training, such as for children, adults, the elderly, etc., and strengthening the number of the feature library, the accuracy and security can be ensured.

[0039] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other various media that can store program codes.

[0040] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0041] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0042] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0043] Example 2, in an exemplary embodiment, a system for fusing intelligent driving data and driving videos and sharing them is further provided, including an identification module, including a camera device and a radar device, which is connected to the clipping module and is used to obtain image information around the vehicle.

[0044] Among them, the first object can be a front camera, a surround camera or a rear camera. The camera identifies road objects, such as obstacles or parking spaces, through a multi-frame rate image vision algorithm. The radar device includes a millimeter wave radar and an ultrasonic radar, measures the distance between the vehicle and the object, and the recognition system judges and processes the advanced information processed by the vision algorithm, finds suspicious areas, and controls the vehicle.

[0045] In one embodiment, a cropping module is further included;

[0046] The cropping module is connected to the recognition module and the intelligent driving domain controller, and is used to receive the CAN signal sent by the intelligent driving domain controller and perform video cropping.

[0047] When the intelligent driving system detects a vehicle control behavior, the intelligent driving domain controller sends a start signal to the cropping module through the CAN bus to notify the DVR application in the vehicle to start the video cropping function, and the video cropping function is marked according to the start and end of the intelligent driving control behavior.

[0048] In one embodiment, a desensitization module is further included;

[0049] The desensitization module is connected to the cropping module, and is used to perform desensitization processing on the cropped fusion video and transmit the video to the service platform through the network cloud service.

[0050] The desensitization module includes a face recognition and license plate recognition function. It uses the key parts of the face (such as eyes, mouth, nose, eyebrows, etc.) collected by the camera for region division. The algorithm tracks the features according to these regions, generates a unique face feature code, and replaces the feature code with a mosaic picture to achieve the desensitization effect. For license plate information, it is recognized according to the color (such as blue, green), and a replacement principle similar to face desensitization is adopted.

[0051] Embodiment 3, in an exemplary embodiment, a computer program product is further provided, including a computer program, and when the computer program is executed by a processor, the following method steps are implemented, including:

[0052] Collect the road surface image information of the first object;

[0053] According to the road surface image information, use the vision algorithm and radar data to identify obstacles and suspicious areas, and control the driving behavior of the vehicle;

[0054] According to the driving behavior of the vehicle, trigger the cropping module to perform video cropping;

[0055] Collect intelligent driving data and perform fusion processing with the video data to form a fusion video.

[0056] In one implementation manner, the above computer program can execute the following steps:

[0057] Collect the original road surface image data of the first object, and based on the original image data, identify information such as obstacles and parking spaces on the road surface through vision algorithms to generate advanced image information after processing.

[0058] Obtain the road surface image information of the first object.

[0059] According to the image information, use a millimeter-wave radar or an ultrasonic radar to measure the distance between the vehicle and the obstacle, and control the driving behavior of the vehicle based on these data, including active avoidance, lane change, on-ramp and off-ramp, and parking.

[0060] When the vehicle performs a vehicle control behavior, trigger the cropping module to crop the video and mark the start and end times of the video.

[0061] Collect the corresponding video clips according to the driving data generated by the control behavior, and perform fusion processing with the driving data to generate a fused video with timestamps.

[0062] Perform desensitization processing on sensitive information (such as faces and license plates) in the video, and use mosaic technology to replace sensitive areas to ensure user privacy and security.

[0063] Upload the processed fused video to the service platform through network cloud services for users to remotely view and share through a mobile app or a car infotainment system's large screen.

[0064] When triggering the desensitization processing, perform the following steps:

[0065] Use face recognition technology and license plate recognition technology to mark sensitive information captured in the video;

[0066] Replace the sensitive area with a mosaic to generate a desensitized video;

[0067] Upload the desensitized video to the cloud server for users to view and share at any time.

[0068] When uploading the fused video each time, perform the following steps:

[0069] Replace the previous video data with the currently generated fused video;

[0070] Generate a new fused video according to the updated data and store it in the in-vehicle system;

[0071] Upload the newly generated video to the cloud server and synchronize it to the user's mobile app or car infotainment system's large screen.

[0072] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for integrating intelligent driving data and driving videos and sharing them, characterized in that: include: Collect intelligent driving controller signals; When the intelligent driving system performs vehicle control, the video is cropped through the first object; The cropped video is integrated with the intelligent driving data, mosaic decryption is performed, and the video is transmitted to the service platform through network cloud services.

2. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 1, characterized in that: Before collecting the intelligent driving controller signal, the method further includes: Collect road information, identify the size of objects through multi-frame rate image vision algorithms, use radar to measure the distance between the vehicle and the object, find suspicious areas, and control the vehicle.

3. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 1, characterized in that: The vehicle control behavior includes, but is not limited to, performing vehicle control behavior when an obstacle is identified ahead, a vehicle jam is identified, a vehicle enters a narrow road section and wants to reverse, and a parking space is detected.

4. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 1, characterized in that: The fusion of the video and the intelligent driving data also includes: When the user shifts to P gear, the first object will save the stitched video and add a time tag.

5. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 4, characterized in that: The intelligent driving data includes at least one of the driving mileage, driving duration, number of lane changes, number of parking times, number of exits and average time consumption data.

6. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 5, characterized in that: After the cropped video is integrated with the intelligent driving data, the following steps are also included: Combine the cropped video with the visualized data to create a display format with the video on the left and the data list on the right. According to the timestamp of the video, the video start time is kept consistent with the data update time to achieve the integrated display of video and intelligent driving data.

7. The method for integrating intelligent driving data and driving video and sharing them as claimed in claim 6, characterized in that: The mosaic decryption includes capturing feature areas using facial key part recognition technology and license plate color recognition technology, tracking and capturing based on the captured feature areas, generating a unique feature code, and replacing the feature code with a mosaic image to achieve mosaic decryption.

8. A system for integrating intelligent driving data and driving videos and sharing them using any of the methods of claims 1 to 7, characterized in that: include, The recognition module collects road image information through the acquisition device, processes and judges the high-level information processed by the visual algorithm, finds suspicious areas, and controls the vehicle; The cropping module receives the CAN signal sent by the intelligent driving domain controller and starts to crop the video. When the user shifts to P gear, the spliced ​​video is saved and merged with the visualization data according to the timestamp to obtain the fused video. The desensitization module de-mosaic the fused video and transmits the video to the service platform through the network cloud service.

9. A computer device comprising: Memory and processor; The memory stores a computer program, which is characterized in that when the processor executes the computer program, it implements the steps of the method for integrating intelligent driving data and driving video and sharing them as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the method for integrating intelligent driving data and driving video and sharing them as described in any one of claims 1 to 7 are implemented.

Images