A method, device and storage medium for driving video recording

Abstract

The application provides a driving video recording method and device and a storage medium. The method comprises the following steps: creating a memory virtual screen; obtaining driving video data, driving audio data and driving state information; superimposing and displaying the driving video data and the driving state information on the memory virtual screen in real time; synchronously recording the driving audio data and the content displayed on the memory virtual screen to generate a driving record video file. The application simplifies the generation process of the driving record video file, reduces the requirement of the generation process on hardware, and reduces the budget cost of the vehicle machine.

Description

A method, apparatus and storage medium for recording vehicle video. Technical Field

[0001] This invention relates to the field of driving record technology, and in particular to a method, apparatus and storage medium for recording driving video. Background Technology

[0002] A dashcam is a device that uses a high-definition lens to record images, sounds, and other information during a vehicle's journey. Traditional dashcams, whether original or aftermarket, are independent modules attached to the vehicle's infotainment system, requiring separate systems, hardware, and storage devices, resulting in high hardware costs. Furthermore, the signal between this independent external dashcam and the vehicle's infotainment system is not always smooth, leading to incomplete information in the recorded video. It lacks information about the vehicle's condition at the time, especially driving information such as speed, gear, and braking. This makes it difficult to accurately reconstruct the driving scene when reviewing the recorded video, and the review process is cumbersome.

[0003] In recent years, with the development of intelligent in-vehicle systems, "safe driving" has become a crucial indicator of the intelligence of such systems. Consequently, camera-based driver assistance applications such as DMS, OMS, AVM, and ADAS have become indispensable functions of in-vehicle systems. The widespread adoption of these applications has also driven the development of in-vehicle chips and camera performance. Building on this, dashcams are increasingly being integrated into in-vehicle systems, moving beyond traditional external modules. This eliminates the need for additional hardware; it simply adds a driving video recording application to the in-vehicle system. This application records the vehicle's current driving status, such as speed, location, whether seatbelts are worn, gear position, and braking. Furthermore, video storage and playback are convenient, providing ample evidence for determining liability in traffic accidents.

[0004] However, existing driving video recording applications require two encoding processes to synthesize driving status information, video data, and audio data into a driving video file. The first encoding uses OpenGL to encode the video data from the camera and the watermark information obtained from the driving status information. The second encoding uses OpenGL to encode the audio data from the microphone and the video data carrying the watermark information. Alignment with the audio and video data is also required in the second encoding process. This results in driving video recording being time-consuming and resource-intensive, reducing the performance of the system when running other software, and consequently requiring higher-performance hardware, thus increasing costs. Summary of the Invention

[0005] To address the aforementioned technical problems of time-consuming and resource-intensive vehicle video recording, this invention provides a method, apparatus, and storage medium for vehicle video recording.

[0006] The technical solution provided by this invention is as follows:

[0007] A method for recording driving video includes:

[0008] Create a virtual screen in memory;

[0009] Acquire driving video data, driving audio data, and driving status information;

[0010] The driving video data and the driving status information are overlaid and displayed on the memory virtual screen in real time.

[0011] The driving audio data and the content displayed on the memory virtual screen are recorded simultaneously to generate a driving record video file.

[0012] In some embodiments, a memory virtual screen is created using the MediaProjection class function, a native interface of the Android framework layer; and the data buffer Surface of the MediaRecorder native interface of the Android framework layer is assigned to the memory virtual screen for use.

[0013] In some embodiments, displaying the driving video data and the driving status information overlaid on the memory virtual screen in real time includes:

[0014] The corresponding driving images are drawn based on the driving video data and displayed on the memory virtual screen;

[0015] Based on the driving status information, a corresponding driving status watermark is drawn and overlaid on the memory virtual screen;

[0016] The content displayed on the virtual screen in memory is stored in the data buffer Surface.

[0017] In some embodiments, the real-time overlay display of the driving video data and the driving status information on the memory virtual screen includes: when the driving status information meets preset conditions, the driving status information is dynamically overlaid on the driving video data on the memory virtual screen in real time; the preset conditions include: the vehicle speed exceeds a preset speed, and the driver is not wearing a seat belt.

[0018] In some embodiments, displaying the driving status information dynamically and overlaid with the driving video data on the memory virtual screen in real time includes: using Android animation technology to make the driving status information highlight and flash on the memory virtual screen, or changing the color value transparency of the driving status information on the memory virtual screen to achieve highlight and flash display.

[0019] In some embodiments, acquiring driving video data, driving audio data, and driving status information specifically includes:

[0020] The driving video data is obtained from the vehicle's camera through the Android framework layer's native interface Camera2.

[0021] The driving audio and video data recorded from the vehicle's microphone is obtained through the MediaRecorder native interface of the Android framework layer;

[0022] The driving status information recorded by the vehicle control module is obtained through the native Android framework layer interface CarService.

[0023] The process of synchronously recording the driving audio data and the content displayed on the virtual screen in memory to generate a driving recording video file includes:

[0024] The MediaRecorder interface function MediaRecorder.start() is called to record, synchronously recording the content displayed on the virtual screen in memory and the driving audio data into the driving recording video file.

[0025] In some embodiments, displaying the driving video data and the driving status information overlaid on the memory virtual screen in real time includes: displaying a watermark layer three-dimensionally on top of the driving video layer; the watermark layer is used to display the driving status information, and the driving video layer is used to display the driving video data.

[0026] In some embodiments, the method includes: projecting the content displayed on the memory virtual screen onto the dashcam screen.

[0027] The present invention also provides a device for recording driving video, comprising:

[0028] Create a module for creating a memory-based virtual screen;

[0029] The data acquisition module is used to acquire driving video data, driving audio data, and driving status information;

[0030] The virtual display module is used to overlay and display the driving video data and driving status information on the memory virtual screen in real time;

[0031] The recording module is used to simultaneously record the driving audio data and the content displayed on the memory virtual screen to generate a driving recording video file.

[0032] The present invention also provides a storage medium storing computer instructions, which, when invoked, are used to execute the aforementioned method for recording driving videos.

[0033] Compared with the prior art, the driving video recording method, apparatus and storage medium provided by the present invention can bring at least the following beneficial effects:

[0034] 1. This invention overlays driving video data and driving status information onto a virtual screen in memory, and then calls MediaRecorder to synchronously record driving audio data and the content displayed on the virtual screen in memory to generate a driving recording video file. This greatly reduces the workload of driving video recording, reduces CPU consumption, improves software performance, and at the same time reduces the requirements for vehicle hardware and lowers the vehicle system budget cost.

[0035] 2. This invention, through dynamic display, layered display, and hierarchical display, can more vividly display driving status information on driving record videos, and provide prominent reminders of driving status information such as speeding and not wearing seat belts. Attached Figure Description

[0036] The preferred embodiments will now be described in a clear and easy-to-understand manner, with reference to the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of a method, apparatus, and storage medium for recording driving video.

[0037] Figure 1 is a flowchart of an embodiment of a driving video recording method according to the present invention;

[0038] Figure 2 shows an example of a watermark effect for driving status.

[0039] Figure 3 is a schematic diagram of the structure of a vehicle video recording device according to the present invention;

[0040] Figure 4 is a flowchart of another embodiment of a driving video recording method of the present invention. Detailed Implementation

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0042] To keep the drawings concise, only the parts relevant to the invention are shown schematically in each figure, and they do not represent the actual structure of the product. Furthermore, for ease of understanding, in some figures, only one of components with the same structure or function is shown schematically, or only one is labeled. In this document, "one" can mean not only "only one" but also "more than one".

[0043] In one embodiment of the present invention, as shown in FIG1, a method for recording driving video includes:

[0044] Step S100: Create a virtual screen in memory;

[0045] Step S200: Acquire driving video data, driving audio data, and driving status information;

[0046] Step S300 displays the driving video data and driving status information in real time on the virtual screen in memory;

[0047] Step S400 synchronously records driving audio data and the content displayed on the memory virtual screen to generate a driving record video file.

[0048] Specifically, the driving recorder video files include: driving video recorded by the vehicle's cameras, ambient sound captured by microphones at various locations, and driving status information recorded by the vehicle control module. The driving status information is customizable and may include: gear position, speed, seatbelt wearing status, accelerator pedal status, brake pedal status, and time. Adding driving status information to the driving video footage allows for a more complete reconstruction of the driving process, providing reliable evidence for accident liability determination and damage compensation.

[0049] Automotive electronic products can be divided into two categories. The first category is the Electronic Control Unit (ECU), which directly participates in the control decisions for vehicle driving. The second category does not directly participate in the control decisions for vehicle driving and does not affect the vehicle's driving performance and safety, such as the instrument panel and entertainment system. The second category of electronic products can acquire driving video data, driving audio data, and driving status information. For example, driving video data can be acquired through vehicle body cameras, driving audio data through vehicle body microphones, and driving status information through vehicle control modules.

[0050] A memory-based virtual screen is a virtual screen created in memory. Unlike a physical display screen, the content displayed on a virtual screen is invisible and stored in a corresponding buffer. In this embodiment, the memory-based virtual screen is used to aggregate driving video and driving status information. In vehicle operating systems such as QNX, Linux, Android, and WinCE, the function of creating a virtual screen in memory can be realized by calling different but functionally similar instructions. For example, in an Android-based vehicle system, a memory-based virtual screen can be created by calling the Android framework layer's native interface MediaProjection.

[0051] Driving status information can be overlaid as a watermark on the driving video data and displayed on a virtual screen in memory. Then, the content on the virtual screen and the driving audio data are simultaneously recorded via the audio / video recording interface provided by the vehicle's operating system, resulting in a comprehensive driving record video file. For example, in Android systems, the native MediaRecorder interface at the framework layer can be called to simultaneously record the content on the virtual screen in memory and the driving audio data. The driving record video file can be in formats such as MP4 and AVI, and the specific video file format can be preset by the user.

[0052] In this embodiment, by projecting all driving video data and driving status information to be overlaid onto a single memory virtual screen, and then synchronously recording the memory virtual screen and driving audio, the complex encoding process of existing driving video recording applications is avoided, and the audio-video alignment process is eliminated. This greatly reduces the workload of driving video recording, lowers CPU consumption, improves software performance, and reduces the requirements for vehicle hardware, thus reducing the vehicle's budget cost.

[0053] In one embodiment, based on the Android vehicle platform, a memory virtual screen is created using the MediaProjection class function, a native interface of the Android framework layer; and the data buffer Surface of the MediaRecorder native interface of the Android framework layer is assigned to the memory virtual screen for use.

[0054] Specifically, on the Android in-vehicle platform, a virtual display is created using MediaProjection, and screen-related parameters of the virtual display are set; the virtual display is used to aggregate the interfaces that need to be recorded.

[0055] The Android framework's native interface, MediaRecorder, is used to record content from a virtual screen in memory. MediaRecorder has its own data buffer. If the MediaRecorder's data buffer is inconsistent with the virtual screen's data buffer, MediaRecorder needs to move the recorded video data from the virtual screen's data buffer to its own data buffer during recording. To improve recording efficiency and reduce data movement, when creating the virtual screen, its data buffer is set as the MediaRecorder's data buffer Surface. This allows MediaRecorder to directly retrieve the content from the virtual screen's data buffer Surface when recording.

[0056] In one embodiment, step S200 includes: obtaining driving video data collected by the vehicle body camera through the Android framework layer native interface Camera2; obtaining driving audio and video data recorded by the vehicle body microphone through the Android framework layer native interface MediaRecorder; and obtaining driving status information recorded by the vehicle body control module through the Android framework layer native interface CarService.

[0057] Step S300 includes:

[0058] Step S310: Draw the corresponding driving image based on the driving video data and display it on the virtual screen in memory;

[0059] Step S320: Draw the corresponding driving status watermark based on the driving status information and overlay it on the memory virtual screen;

[0060] Step S330 saves the content displayed on the memory virtual screen in the MediaRecorder's data buffer Surface.

[0061] Step S400 includes: calling the MediaRecorder interface function MediaRecorder.start() to record, synchronously recording the content displayed on the virtual screen in memory and the driving audio data into a driving record video file.

[0062] Specifically, a driving status watermark is an image that reflects driving status information. A driving image is drawn based on the driving video, and a driving status watermark is drawn based on the driving status information. The driving image and the driving status watermark are then overlaid and displayed on a virtual screen in memory.

[0063] The size of both the driving image and the driving status watermark does not exceed the screen size of the virtual screen in memory. Overlaying two images can be achieved by performing a bitwise OR operation on the pixel values ​​of the two images at the same location.

[0064] Driving images and driving status watermarks are two independent images, drawn separately, without the need for special encoding to combine them into a single image. This eliminates the need for encoding driving video data and driving status watermarks in existing technologies.

[0065] MediaRecorder is a media recording interface provided by the Android system, used to record media stream objects. A media stream object is a stream of media content that can contain multiple tracks, such as video and audio tracks.

[0066] Using MediaRecorder to synchronously record the memory virtual screen and driving audio eliminates the need for audio and video synthesis and encoding in existing technologies, as well as the time alignment work for audio and video.

[0067] In one embodiment, drawing the driving status watermark in step S320 includes:

[0068] Step S3210: The screen containing all driving status watermarks is pre-drawn according to the size of the memory virtual screen, and each driving status watermark is in a non-display state.

[0069] Step S3211: Adjust the corresponding driving status watermark in the screen to the display state based on the driving status information.

[0070] This allows for faster creation of driving status watermarks.

[0071] To more vividly display the driving status watermark on the virtual memory screen, we have also made the following improvements:

[0072] Different colors of vehicle speed watermarks are drawn according to different preset speed levels. For example, green is used for low speed, yellow for medium speed, and red for high speed or speeding.

[0073] In one embodiment, step S320 includes: dynamically displaying the corresponding driving status watermark on the memory virtual screen when the driving status information meets preset conditions; or step S300 includes: when the driving status information meets preset conditions, displaying the driving status information and driving video data in real time on the memory virtual screen in a dynamic manner; the preset conditions include: the vehicle speed exceeds a preset speed and the driver is not wearing a seat belt.

[0074] Dynamic displays can enhance the prompts for relevant driving statuses and attract user attention.

[0075] In one embodiment, step S320 includes: highlighting the driving status watermark; or step S300 includes: highlighting the driving status information on a memory virtual screen.

[0076] Specifically, Android animation technology can be used to make the driving status information flash on the memory virtual screen, or the flashing can be achieved by periodically or irregularly changing the color value transparency of the driving status information on the memory virtual screen.

[0077] For example, Android frame animation technology can be used to add each frame image within a highlight flashing cycle to a frame animation file and play it in a loop to achieve a continuous highlight flashing prompt effect.

[0078] Alternatively, one can continuously change the color value and transparency of the watermark while drawing the vehicle speed watermark to achieve a flashing effect.

[0079] In one embodiment, a watermark layer is displayed three-dimensionally on top of a driving video layer; the watermark layer is used to display driving status information, and the driving video layer is used to display driving video data.

[0080] For example, as shown in Figure 2, the watermark layer is located above the driving video layer. When drawing the driving status watermark, a shadow is drawn under the driving status watermark to present a three-dimensional layered effect.

[0081] In one embodiment, the method further includes projecting the content displayed on the memory virtual screen onto the dashcam screen.

[0082] Specifically, the dashcam screen can be the central control display screen, or a portion of the central control display screen.

[0083] The content displayed on the memory virtual screen is invisible to the user. By projecting the content displayed on the memory virtual screen onto the dashcam screen, the user can intuitively see the content displayed on the central control display screen.

[0084] In one embodiment of the present invention, as shown in FIG3, a vehicle video recording device includes:

[0085] Create module 100 to create a memory-based virtual screen;

[0086] Data acquisition module 200 is used to acquire driving video data, driving audio data and driving status information;

[0087] The virtual display module 300 is used to overlay driving video data and driving status information onto a memory virtual screen in real time.

[0088] The recording module 400 is used to simultaneously record driving audio data and the content displayed on the virtual screen in memory, generating a driving record video file.

[0089] In one embodiment, a memory-based virtual screen is created using the MediaProjection class function, a native interface of the Android framework layer; and the data buffer Surface of the MediaRecorder native interface of the Android framework layer is assigned to the memory-based virtual screen for use.

[0090] In one embodiment, the data acquisition module is further configured to acquire driving video data collected by the vehicle body camera through the Android framework layer native interface Camera2; acquire driving audio and video data recorded by the vehicle body microphone through the Android framework layer native interface MediaRecorder; and acquire driving status information recorded by the vehicle body control module through the Android framework layer native interface CarService.

[0091] The virtual display module includes:

[0092] The image drawing unit is used to draw corresponding driving images based on driving video data;

[0093] The watermark drawing unit is used to draw the corresponding driving status watermark based on the driving status information.

[0094] The storage unit is used to overlay the driving status watermark and driving image on the memory virtual screen, and save the content displayed on the memory virtual screen in the MediaRecorder's data buffer Surface.

[0095] The recording module is also used to call the MediaRecorder interface function MediaRecorder.start() to record, synchronously recording the content displayed on the virtual screen in memory and the driving audio data into a driving record video file.

[0096] In one embodiment, the watermark drawing unit is also used for:

[0097] The screen containing all driving status watermarks is pre-drawn according to the size of the virtual screen in memory, and each driving status watermark is in a hidden state; the corresponding driving status watermark in the screen is adjusted to be displayed according to the driving status information.

[0098] In one embodiment, the watermark drawing unit is further configured to: dynamically display the corresponding driving status watermark on a memory virtual screen when the driving status information meets preset conditions; or the virtual display module is further configured to dynamically overlay the driving status information and driving video data onto the memory virtual screen in real time when the driving status information meets preset conditions. Preset conditions include: vehicle speed exceeding a preset speed and the driver not wearing a seatbelt.

[0099] In one embodiment, the watermark drawing unit is also used to highlight and flash the driving status watermark; or the virtual display module is also used to highlight and flash the driving status information on the memory virtual screen.

[0100] In one embodiment, the virtual display module is further used to display the watermark layer in three dimensions on top of the driving video layer; the watermark layer is used to display driving status information, and the driving video layer is used to display driving video data.

[0101] In one embodiment, the virtual display module is also used to project the content displayed on the memory virtual screen onto the dashcam screen.

[0102] It should be noted that the embodiments of the vehicle video recording device provided by the present invention and the embodiments of the aforementioned vehicle video recording method are based on the same inventive concept and can achieve the same technical effects. Therefore, other specific details of the embodiments of the vehicle video recording device can be referred to the description of the embodiments of the aforementioned vehicle video recording method.

[0103] The aforementioned method for recording driving videos is applied to an Android-based in-vehicle system, as shown in Figure 4, specifically including:

[0104] First, the Android framework layer's native interface MediaProjection is called to create a VirtualDisplay memory virtual screen. At the same time, the MediaRecorder's data buffer Surface is assigned to the VirtualDisplay for use in caching the data displayed on the memory virtual screen. In this way, the MediaRecorder can obtain the aggregated content on the VirtualDisplay through this data buffer Surface for recording.

[0105] Branch 1: Call the Android native interface Camera2 to obtain driving video data captured by the vehicle's camera;

[0106] On the VirtualDisplay memory virtual screen, a native Android framework layer view SurfaceView is drawn based on the driving video data to display the driving video.

[0107] Since the driving video is constantly changing and the main content is 3D, involving many elements that need to be rendered and complex content, and requiring real-time preview of the camera footage, SurfaceView is used for rendering to avoid video stuttering.

[0108] Branch 2: Synchronously obtain driving status information recorded by the vehicle control module from the CarService of the hardware abstraction layer; draw a custom View on the VirtualDisplay memory screen based on the driving status information to display the driving status information watermark.

[0109] Although the driving status information is constantly being updated and changed, it is usually presented as 2D text and vector graphics. The content is not complicated, so drawing it using a custom view is sufficient to meet the requirements.

[0110] Branch 3: Overlay the driving video data and driving status information watermark on the VirtualDisplay memory virtual screen, and place the overlaid video data in the Surface buffer for MediaRecorder to retrieve.

[0111] Branch 4: Call MediaRecorder to automatically acquire audio data captured by the microphone; by acquiring data from the Surface buffer, synchronized recording of audio data and the virtual screen in memory is achieved;

[0112] The system checks whether the MediaRecorder has reached the user-preset recording duration. If so, it creates a driving record video file; otherwise, MediaRecorder continues recording from the virtual screen in memory.

[0113] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for recording driving video, characterized in that, include: Create a virtual screen in memory; The data buffer Surface of the Android framework layer native interface MediaRecorder is assigned to the memory virtual screen for use in caching the display content of the memory virtual screen; driving video data and driving status information are obtained. On the virtual screen in memory, a native Android framework layer view SurfaceView is drawn based on the driving video data to display the driving video; On the virtual memory screen, a custom View is drawn based on the driving status information to display a driving status information watermark; the driving video data and the driving status information watermark are overlaid on the virtual memory screen, and the overlaid video data is saved in the data buffer Surface; the Android framework layer native interface MediaRecorder is used to obtain the driving audio and video data collected by the microphone, and at the same time, the data in the data buffer Surface is obtained to realize the synchronous recording of the driving audio data and the virtual memory screen, and generate a driving recording video file.

2. The method for recording driving video according to claim 1, characterized in that, Use the MediaProjection class function, a native interface of the Android framework layer, to create a virtual screen in memory.

3. The method for recording driving video according to claim 1, characterized in that, The step of drawing the corresponding driving status watermark based on the driving status information includes: pre-drawing a screen containing all driving status watermarks according to the size of the memory virtual screen, with each driving status watermark in a non-display state; and adjusting the corresponding driving status watermark in the screen to a display state according to the driving status information.

4. The method for recording driving video according to claim 1, characterized in that, The step of displaying the driving video data and the driving status information on the memory virtual screen in real time includes: when the driving status information meets preset conditions, the driving status information and the driving video data are dynamically displayed on the memory virtual screen in real time; the preset conditions include: the vehicle speed exceeds a preset speed and the driver is not wearing a seat belt.

5. The method for recording driving video according to claim 4, characterized in that, The driving status information is displayed dynamically and overlaid with the driving video data in real time on the memory virtual screen, including: using Android animation technology to make the driving status information highlight and flash on the memory virtual screen, or by changing the color value transparency of the driving status information on the memory virtual screen to achieve highlight and flash display.

6. The method for recording driving video according to claim 1, characterized in that, The acquisition of driving video data and driving status information specifically includes: acquiring the driving video data collected by the vehicle's camera through the Android framework layer native interface Camera2; and acquiring the driving status information recorded by the vehicle control module through the Android framework layer native interface CarService.

7. The method for recording driving video according to claim 1, characterized in that, The step of overlaying and displaying the driving video data and the driving status information on the memory virtual screen in real time includes: displaying a watermark layer in three dimensions on top of the driving video layer; the watermark layer is used to display the driving status information, and the driving video layer is used to display the driving video data.

8. A method for recording driving video according to claim 1, characterized in that, include: The content displayed on the virtual screen in memory is projected onto the screen of the dashcam.

9. A device for recording vehicle video, characterized in that, include: Create a module for creating a memory-based virtual screen; The data buffer Surface of the Android framework layer native interface MediaRecorder is assigned to the memory virtual screen for use in caching the display content of the memory virtual screen; The data acquisition module is used to acquire driving video data and driving status information; the virtual display module is used to draw a native Android framework view SurfaceView on the memory virtual screen according to the driving video data to display the driving video. On the virtual screen in memory, a custom View is drawn based on the driving status information to display the driving status information watermark; the driving video data and the driving status information watermark are overlaid on the virtual screen in memory, and the overlaid video data is saved in the data buffer Surface; The recording module uses the Android framework layer native interface MediaRecorder to obtain driving audio and video data collected by the microphone, and simultaneously obtains data from the data buffer Surface, so as to realize the synchronous recording of the driving audio data and the memory virtual screen, and generate a driving record video file.

10. A storage medium, characterized in that, include: The storage medium stores computer instructions, which, when invoked, are used to execute the method for recording driving video as described in any one of claims 1-8.

Images