Information display method and apparatus, vehicle, electronic device, and storage medium

Abstract

Provided are an information display method and apparatus, a vehicle, an electronic device, and a storage medium. The information display method comprises: in response to a user operation, displaying video information and / or superimposed information, wherein the video information is information photographed by a recording device, the superimposed information is information collected when the recording device photographs the video information, the superimposed information is configured to be displayed on the video information in a superimposed manner, the video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area.

Description

Information display method and device, vehicle, electronic device, and storage medium

[0001] Related applications

[0002] The present application claims priority to a Chinese invention patent application No. 202411240475.8, filed on September 4, 2024, and entitled “Information display method and device, vehicle, electronic device, and storage medium”. TECHNICAL FIELD

[0003] The present application relates to the technical field of vehicles, driving data, and the like, and in particular to an information display method and device, a vehicle, an electronic device, and a storage medium. BACKGROUND

[0004] Currently, a driving recorder has become an essential device for every vehicle on the road. The driving recorder can record video images and sound during the entire driving process of the vehicle, and provide evidence for travel accidents and hit-and-run situations. In addition to recording images during driving through a camera, the driving recorder can also record and display the driving state of the vehicle and the operation of the driver at the time through superimposed information. SUMMARY

[0005] The present application aims to provide an information display method and device, a vehicle, an electronic device, a storage medium, and a program product. Currently, when the driving recorder starts the recording function, the superimposed information is already fixed in the video image, which cannot meet the user's needs. The present application can selectively display superimposed information in the video image according to the user's selection operation.

[0006] The present application provides an information display method, comprising: displaying video information and / or superimposed information in response to a user operation, wherein the video information is information obtained by a recording device, and the superimposed information is information collected when the recording device records the video information, and the superimposed information is used for superimposed display on the video information; wherein the video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area.

[0007] Exemplarily, the user operation includes an instruction to display superimposed information, and the displaying of the video information and / or the superimposed information in response to the user operation includes: displaying the superimposed information stored in the superimposed information storage area in response to the instruction to display the superimposed information.

[0008] Exemplarily, the displaying of the superimposed information stored in the superimposed information storage area includes: obtaining superimposed layer data according to the superimposed information; and displaying the superimposed layer data to display the superimposed information.

[0009] Exemplarily, the user operation comprises an instruction indicating to display the video information and the superimposed information, and the video information and / or the superimposed information are displayed in response to the user operation, comprising: in response to the instruction indicating to display the video information and the superimposed information, processing information stored in the superimposed information storage area to obtain the superimposed information, and processing information in the video information storage area to obtain the video information; and displaying the video information and the superimposed information, wherein the superimposed information is superimposed and displayed on the video information.

[0010] Exemplarily, the video information and the superimposed information are displayed, comprising: obtaining superimposed image information according to the image frame data and the superimposed information, wherein the image frame data is data processed according to information in the video information storage area; and displaying the superimposed image information to display the video information and the superimposed information.

[0011] Exemplarily, the superimposed image information is obtained according to the image frame data and the superimposed information, comprising: obtaining superimposed layer data according to the superimposed information; and obtaining the superimposed image information according to the image frame data and the superimposed layer data.

[0012] Exemplarily, the superimposed layer data is obtained according to the superimposed information, comprising: in a case where at least one of the video information has no frame loss and the state of the superimposed information of the current frame changes, obtaining the superimposed layer data according to the superimposed information at the current time.

[0013] Exemplarily, the superimposed layer data is obtained according to the superimposed information, further comprising: in a case where the video information has frame loss and / or the state of the superimposed information of the current frame does not change, obtaining the superimposed layer data according to the superimposed information at the last time.

[0014] Exemplarily, the image frame data and the superimposed information comprise header data, and the superimposed image information is obtained according to the image frame data and the superimposed information, comprising: determining the superimposed information corresponding to the image frame data according to the header data of the image frame data and the header data of the superimposed information; and obtaining the superimposed image information according to the image frame data and the superimposed information corresponding to the image frame data.

[0015] Exemplarily, the superimposed information corresponding to the image frame data is determined according to the header data of the image frame data and the header data of the superimposed information, comprising: when the header data of the image frame data and the header data of the superimposed information match, determining the superimposed information corresponding to the image frame data according to the superimposed information matching the image frame data; and when the header data of the image frame data and the header data of the superimposed information do not match, determining the superimposed information at the last time as the superimposed information corresponding to the image frame data.

[0016] Exemplarily, when the header data of the image frame data and the header data of the superimposed information match, the superimposed information corresponding to the image frame data is determined according to the superimposed information matching the image frame data, including: when the superimposed information matching the image frame data is not empty, the superimposed information is determined as the superimposed information corresponding to the image frame data; when the superimposed information matching the image frame data is empty, the superimposed information at the last moment is determined as the superimposed information corresponding to the image frame data.

[0017] Exemplarily, the header data includes at least one of frame number data and time data, and the matching of the header data of the superimposed information and the header data of the image frame data includes at least one of the following: the frame number data of the superimposed information matches the frame number data of the image frame data; and the time data of the superimposed information matches the time data of the image frame data.

[0018] Exemplarily, the superimposed information includes a plurality of types of superimposed information, and the method includes: in response to a user operation, determining superimposed information of a target type to display the superimposed information of the target type.

[0019] Exemplarily, the types of superimposed information include at least one of vehicle state information, various sensor information, position information, and driving function information, and the vehicle state information includes at least one of gear information, pedal information, and light information.

[0020] Exemplarily, the superimposed layer data is obtained according to the superimposed information, including: determining icon information corresponding to the superimposed information according to the superimposed information; and obtaining the superimposed layer data according to the icon information.

[0021] Exemplarily, the superimposed information includes state information, and the icon information corresponding to the superimposed information is determined according to the superimposed information, including: determining icon information corresponding to the state information in the superimposed information according to the superimposed information.

[0022] Exemplarily, the user operation includes an instruction to display video information, and the video information and / or the superimposed information are displayed in response to the user operation, including: in response to the instruction to display the video information, the video information in the video information storage area is displayed.

[0023] Exemplarily, before the video information and / or the superimposed information are displayed in response to the user operation, including: in the case that the superimposed information is target superimposed information, the superimposed information is stored to the superimposed information storage area, and the video information is stored to the video information storage area.

[0024] Exemplarily, the superimposed information is stored in the superimposed information storage area, and the method further comprises: in the case that the state of at least one of the superimposed information changes, storing the changed superimposed information in the superimposed information storage area; in the case that neither the video information nor the superimposed information changes, setting the superimposed information of the current time to be empty, or not storing the superimposed information of the current time.

[0025] Exemplarily, before displaying the video information and / or the superimposed information in response to the user operation, the method further comprises: in the case that the superimposed information is not the target superimposed information, superimposing and displaying the superimposed information in the video information, and storing the superimposed information and the video information in the video information storage area.

[0026] Exemplarily, the recording device further communicates with the plug-in software, and the method further comprises: in the case that the user operation represents a need to display the plug-in software, superimposing and displaying the plug-in software on the video information as a display window.

[0027] Another embodiment of the present application provides an information display device, which comprises: a display module configured to display video information and / or superimposed information in response to a user operation, wherein the video information is information captured by a recording device, and the superimposed information is information collected when the recording device captures the video information, and the superimposed information is configured to be superimposed and displayed on the video information; wherein the video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area.

[0028] Another embodiment of the present application provides a vehicle comprising a recording device and a processing device configured to implement the steps of the method of any one of the above embodiments.

[0029] Another embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to implement the steps of the method of any one of the above embodiments when executing the computer program.

[0030] Another embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is configured to implement the steps of the method of any one of the above embodiments when executed by a processor.

[0031] Another embodiment of the present application provides a computer program product, which comprises instructions, and the instructions are configured to enable a computer device to implement the steps of the method of any one of the above embodiments when executed by a processor of the computer device.

[0032] In the above embodiment, video information and / or superimposed information are displayed in response to a user operation, wherein the video information is information captured by the recording device, and the superimposed information is information collected when the recording device captures the video information, and the superimposed information is used to be superimposed and displayed on the video information; wherein the video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area. According to the superimposed information of the present application, the superimposed information can be selectively displayed in the video picture according to a user selection operation. BRIEF DESCRIPTION OF DRAWINGS

[0033] Fig. 1 is a schematic diagram of a superimposed information video picture of a driving recorder according to an embodiment of the present application;

[0034] Fig. 2 is a schematic diagram of superimposing superimposed information on a video picture according to an embodiment of the present application;

[0035] Fig. 3 is a flowchart of an information display method according to an embodiment of the present application;

[0036] Fig. 4 is a schematic diagram of definition and application of superimposed information according to an embodiment of the present application;

[0037] Fig. 5 is a flowchart of a recording process when superimposed information is stored in a video file self-defined extension part according to an embodiment of the present application;

[0038] Fig. 6 is a flowchart of a recording process when superimposed information is stored in a separate file according to an embodiment of the present application;

[0039] Fig. 7 is a flowchart of storing superimposed information into a superimposed information storage area according to an embodiment of the present application;

[0040] Fig. 8 is a flowchart of displaying superimposed information according to another embodiment of the present application;

[0041] Fig. 9 is a flowchart of displaying video information and superimposed information according to an embodiment of the present application;

[0042] Fig. 10 is a flowchart of displaying video information and superimposed information according to another embodiment of the present application;

[0043] Fig. 11 is a schematic diagram of a framework of a playing process according to an embodiment of the present application;

[0044] Fig. 12 is a flowchart of obtaining superimposed layer data according to an embodiment of the present application;

[0045] Fig. 13 is a flowchart of displaying superimposed information of a target type according to an embodiment of the present application;

[0046] Fig. 14 is a flowchart of a playing process when superimposed information is stored in a video file self-defined extension part according to an embodiment of the present application;

[0047] FIG. 15 is a flowchart of a playing process when the superimposed information is stored in a separate file according to an embodiment of the present application;

[0048] FIG. 16 is a flowchart of a process of determining superimposed information corresponding to image frame data according to an embodiment of the present application;

[0049] FIG. 17 is a schematic diagram of matching of image frame data and superimposed information according to an embodiment of the present application;

[0050] FIG. 18 is a flowchart of a playing process according to an embodiment of the present application;

[0051] FIG. 19 is a schematic diagram of a display device of superimposed information of a car recorder according to an embodiment of the present application;

[0052] FIG. 20 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] Embodiments of the present application are described in detail below with reference to the attached drawings, which are meant to be exemplary and not limiting. The embodiments described below are examples of how the application can be implemented and are not intended to limit the application.

[0054] Currently, a car recorder is almost a necessary device for every vehicle on the road. Installing a car recorder can record video images and sound of the whole process of the vehicle driving, and provide evidence for travel accidents and situations of encountering a collision. In addition to recording images through a camera during driving, a car recorder can also record and display information such as the driving state of the vehicle and the operation of the driver at that time through superimposed information. Currently, the superimposed information in the car recorder is fixed in the video image after the recording process of the camera is completed. Users cannot choose whether to display the superimposed information when playing.

[0055] In some examples, the superimposed information of the driving recorder is implemented in software as follows: first, prepare the icon materials (including the icons of the superimposed information and the arrangement) needed on the display interface of the driving video, as shown in Fig. 1, the icons of the superimposed information and the arrangement are shown at the top, and the icon materials are integrated in a specific path of the camera application in advance. After the development is completed, the application file is generated and built into the system. During recording, the camera normally collects images, and the images are transferred and distributed in the software. At the same time, the camera application obtains various information of the vehicle and the driving state through system data and messages. Each information and state that needs to be displayed has a specific icon corresponding to it on the application interface of the driving recorder. In the software, the corresponding icons are superimposed and drawn as watermarks on the corresponding positions of the current camera image according to the current state information obtained, as shown in Fig. 2, the icons of the superimposed information can be superimposed and drawn as watermarks on the top of the current camera image. Each frame of image is drawn with all the icons to be superimposed and displayed. The frame of image that has completed image collection and superimposed information drawing is sent to the recording unit for encoding and recording, and this step is completed frame by frame until the recording file is generated. When the user needs to view the driving recorder, the video file is opened, and the system automatically calls the default player to play.

[0056] As described above, the superimposed information in the driving recorder is drawn before the image is sent to the encoding unit in the recording process of the camera. In the setting items of the driving recorder, there is a settable item of whether to display the superimposed information, but the actual significance is not great. Because the option only determines whether to draw the superimposed information on the camera screen during recording. If the superimposed information is turned off in the option, the recording screen and the camera preview screen will not draw the superimposed information. If the superimposed information is turned on in the option, the superimposed information is fixed in the recording screen after recording is completed, and it is not possible to select whether to display the superimposed information during playback.

[0057] In addition, the driving recorder is not needed to be viewed at ordinary times except for some special situations such as driving accidents. From the system resources of the host, the driving recorder has been recording since the vehicle is powered on, and the camera image is drawn with superimposed information every frame, so that the additional image processing work consumes additional system resources and may affect the performance of the camera.

[0058] In addition, the superimposed information is generally superimposed on the camera screen, and too many icons may make the interface display not beautiful, causing poor experience, and the superimposed information also blocks the camera screen, causing a certain loss of the field of view. With product iteration, more information during driving may need to be recorded in the future, so the space available for displaying the superimposed information of the current driving recorder is very limited.

[0059] Based on this, the application provides a driving data saving and displaying method. In the daily driving state of the driving recorder, the drawing of most of the superimposed information is not needed in the video recording process, only the required information is saved as metadata in the form of formatted variables, and the metadata is parsed and applied in the playing.

[0060] FIG. 3 is a flowchart of an information displaying method according to an embodiment of the application.

[0061] As shown in FIG. 3, the information displaying method includes S301.

[0062] S301, in response to a user operation, display video information and / or superimposed information, wherein the video information is information obtained by recording equipment, and the superimposed information is information collected by the recording equipment when recording the video information, and the superimposed information is used for superimposed display on the video information.

[0063] The video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area.

[0064] For example, the recording equipment can be a driving recorder, and the driving recorder is installed on a vehicle. The driving recorder is used to record the video information around the vehicle, and the video information can be a continuous video or a photo. For example, the driving recorder records or photographs the environment around the vehicle. When the recording equipment records the environment around the vehicle, the superimposed information of the vehicle is collected synchronously. The superimposed information includes, for example, gear information, light information, and the like. The existing superimposed information of the vehicle is obtained in the recording process of the recording equipment. For example, the video unit receives image information and audio frame by frame, judges the signal corresponding to each superimposed information, and draws the icon of each superimposed information on the corresponding signal. The image with the superimposed icon is sent to the encoder, and the superimposed information in the obtained video file is fixed in the video screen, and the opening and closing of the superimposed information cannot be controlled in the playing.

[0065] For example, in the process of recording the environment around the vehicle, the recording software in the recording equipment can obtain the superimposed information of the vehicle, such as vehicle state information, sensor data, position information, intelligent driving function information, and the like, through system data and messages. Two storage areas can be divided in the software storage area for storing the superimposed information and the video information. The superimposed information is stored in the superimposed information storage area, and the video information is stored in the video information storage area. It can be understood that the superimposed information does not need to be drawn, and is saved as metadata in the form of formatted variables, which reduces the image processing work in the storage process and saves the system resources.

[0066] Exemplarily, during the video playing process, the application can display the superimposed information and / or the video information in response to the user operation. For example, when the superimposed information and the video information are displayed, the information stored in the video information storage area and the superimposed information storage area is processed so that the superimposed information is superimposed and displayed on the video information. The user operation can indicate whether the superimposed information and the video information need to be displayed. It can be understood that when the user plays the video, the user can select whether the superimposed information needs to be displayed and which information needs to be superimposed according to the situation at that time and personal needs. The display mode of the superimposed information can also be set, for example, the field of view of the image that is blocked by the superimposed information can be fully displayed because the blocked part can be turned off. On the one hand, the experience of playing the video can be increased, and on the other hand, more information is retained, which not only increases the quantity and types of information, but also can expand new functions based on the data.

[0067] The information display method of the application can selectively display the superimposed information in the video picture according to the selection operation of the user. During the storage process, the superimposed information is only saved in the file in the form of a formatted variable, instead of continuously performing the operation of drawing the icon, thereby saving the system resources of the central control host. According to the user selection, only the superimposed information selected by the user is displayed, instead of fixing all the superimposed information in the video display. Only part of the superimposed information is displayed, which improves the degree of interface beauty and reduces the blocking of the camera picture by the superimposed information.

[0068] As an example, before the video information and / or the superimposed information are displayed in response to the user operation, the method further includes: in the case that the superimposed information is target superimposed information, storing the superimposed information in the superimposed information storage area and storing the video information in the video information storage area.

[0069] Exemplarily, in the case that the superimposed information is target superimposed information, the superimposed information is stored in the superimposed information storage area and the video information is stored in the video information storage area. It can be understood that the target superimposed information is superimposed information other than the vehicle identification code, the system time and other superimposed information that needs to be always displayed on the video, for example, vehicle state information, data of the acceleration sensor and the like. It can be understood that for the vehicle identification code, the system time and other information that needs to be always superimposed and not turned off, the information is superimposed in the video picture during the recording process, and then the image with the superimposed icon is sent to the encoder and saved in the video information storage area together with the video information.

[0070] Exemplarily, in the case that the superimposed information is target superimposed information, when the video information obtained by shooting by the recording device and the superimposed information obtained by collecting are stored, the two are stored separately. It can be understood that during the recording process, the superimposed information is not drawn in the video picture, and only the required superimposed information is saved in the form of a formatted variable as metadata. When playing, the information is parsed and applied.

[0071] As an example, in response to a user operation, before displaying the video information and / or the superimposed information, including: in the case where the superimposed information is not the target superimposed information, superimposing and displaying the superimposed information in the video information, which is stored together with the video information in the video information storage area.

[0072] Exemplarily, a small part of superimposed information that needs to be always superimposed and displayed and cannot be turned off, such as system time, vehicle identification code, etc., can be directly drawn in the video information picture in the storage process, to complete superimposition and be stored together with the video information. When playing, this part of superimposed information is played and displayed together with the video information. Of course, the superimposed information such as system time and vehicle identification code can also be saved as metadata in the formatted variable, and in the playing process, according to the user selection operation, selectively display or do not display, or select to turn on or turn off which superimposed information, in response to the user operation, draw the required superimposed information into a layer, and superimpose it in the video picture when playing.

[0073] As an example, the video information further includes audio information, and storing the video information to the video information storage area includes: respectively performing encoding operations on the video information and the audio information, and storing the encoded video information and the audio information to the video information storage area.

[0074] Exemplarily, the processing module of the software receives the current frame of video information and audio information, synchronizes the audio and video, and calls the respective encoders to respectively perform encoding operations on the video information and the audio information. After completing the encoding work, the encoded video information and the audio information are stored to the video information storage area. For example, after the audio and video data are respectively encoded, the processing unit can also start the packaging and multiplexing of the audio and video data. The audio and video data can be packaged according to a predetermined format, for example, the file format required header data can be added, and the header data can be timestamp and / or frame number data, for subsequent matching with superimposed information.

[0075] As an example, the superimposed information can be written in a fixed format when stored to the superimposed information storage area, to facilitate subsequent analysis and management of the superimposed information. As shown in FIG. 4, a buffer area S401 can be applied in the flow software of the vehicle recorder to save the superimposed information required for each frame. The superimposed information required for each frame can be based on the defined superimposed information format of each frame in S402. The format of the superimposed information in S402 can include the current frame number, timestamp, gear, throttle, brake, turn signal, low beam, high beam, and various superimposed information. In addition, the structure can include a reserved definition, which can be extended. The present application adds a single-byte definition before the structure definition of the superimposed information, as the start of this frame information, as a flag indicating whether there is a frame loss when recording this frame. The buffer area S401 can be accessed and operated by both the recording flow and the playback flow, so the definition of S402 is placed in the public part of the system, so that the recording and playback flow codes can reference and include S401.

[0076] During the recording process of the vehicle recorder, the recording application S403 fills the variables with the current frame number, timestamp, gear, throttle, brake, turn signal, low beam, high beam, and various vehicle state information according to the structure format defined in S401, and writes them into the S402 buffer area. The saving method of each data is determined according to the actual situation, such as using integer or long integer to record the information represented by numerical value, using two binary digits to represent the four states of the gear, and using one binary digit to represent the on / off states of the turn signal, low beam, and high beam, so that various states are saved in two bytes of space. In addition, when the data is successfully saved, the single-byte start flag is written as 0xF1, where F is a self-defined flag in the program, and 1 indicates that there is no frame loss in this frame and the superimposed information of the current frame is recorded. If there is frame loss in the camera image collected during the recording process, the superimposed information of the current frame such as gear, throttle, brake, turn signal, low beam, and high beam is not written, only the current frame number and timestamp are written, and the single-byte start flag is written as 0xF0, indicating that the frame has been recorded but there is no data. When the recording needs to be played back, the player application S404 analyzes the recorded information of each frame and converts it into a superimposed layer. If the recorded data start flag bit is 0xF0, this frame is processed as a lost frame and does not need to create a superimposed layer.

[0077] As an example, for the recording process, the storage path of the superimposed information can also have multiple ways, for example, the superimposed information can be saved in a custom extension part of a video file, or can be saved as a separate file. It can be understood that when the superimposed information is saved as a custom extension part of a video file, the superimposed information storage area where the superimposed information is written and the video storage area where the video data is written together constitute a video file. When the superimposed information is saved as a separate file, the superimposed information storage area where the superimposed information is written is saved as a superimposed file, and the video storage area where the video data is written is saved as a video file.

[0078] The following will be described in detail the embodiments of the storage method of the superimposed information.

[0079] As an example, the superimposed information storage area can be extended and customized based on the standard of the video codec and the video information storage area format. Taking mp4 (a mainstream multimedia file format, mainly for storing digital audio and digital video) format as an example: an mp4 file is composed of many different types of boxes (referring to the arrangement unit of mp4 format), each box contains different information, and these boxes are organized in a tree structure. Among different boxes, data, meta information, media information, file information, etc. of audio and video are stored. In addition to the standard box data, the mp4 format also supports a custom box. During the recording process, the various superimposed information collected by the recording device can be managed and data stored in a structure on the software, and each frame of data is sequentially placed in a buffer. When all the image frames of the current video recording are collected, the superimposed information corresponding to all the frames of the video file has also been reserved in the buffer. This part of data is placed as metadata in the custom box. Thus, the custom metadata and the encoded audio and video data are encapsulated into the mp4 file. The player software part can parse the content of the custom box of the mp4 file. The corresponding superimposed information is drawn through the parsing result. And according to the actual needs, the setting item of the driving recorder is customized: whether to display the superimposed information when playing the driving video. In addition to the above method of storing superimposed information data into the custom box of mp4 format, another way can also be used: superimposed information data is placed frame by frame into SEI (Supplemental Enhancement Information) data. It can be implemented based on the specification framework, which is not described in detail here. The implementation methods of various containers may differ, and in actual application, they need to be customized and parsed according to the specific format. Of course, the storage method of the superimposed information of the present application can be applied to all video file formats that support extended content.

[0080] Fig. 5 is a flow chart of the recording process of the superimposed information stored in the custom extension part of the video file according to an embodiment of the present application.

[0081] Referring to FIG. 5, the recording process when the superimposed information is stored in the video file custom extension part includes:

[0082] S501: During recording, the camera transmits the images collected by the bottom layer to the upper layer frame by frame, and camera preview and recording operations are performed. At the same time, various information and driving states of the vehicle are obtained through system data and messages. The obtained superimposed information includes: general information (including vehicle identification number, system time, vehicle speed, vehicle light state, accelerator pedal, brake pedal, etc.), acceleration sensor and other sensor data obtained as needed, geographic location information, whether the intelligent driving function is started, and the like.

[0083] It should be noted that the type and quantity of superimposed information obtained can be customized and extended in the software.

[0084] S502: The real-time images collected every frame and the vehicle state data (superimposed information) are sent to the recording module in the software process together.

[0085] S503: The recording module of the software receives the current frame of video data and audio data, synchronizes the audio and video, and calls the respective encoders to complete the encoding work. This stage also performs the saving of the superimposed information. The information that needs to be displayed in the image (vehicle identification number, system time) is still superimposed on the image in the recording process as in the current superimposed process, or can be saved in the formatted variable. The remaining superimposed information is saved in the formatted variable: the vehicle information data obtained in the current frame is added to the frame number and timestamp of the current image, and is written in the buffer area (i.e., the superimposed information storage area) shown in S401 according to the structure defined in S402 in FIG. 4. Among them, the information recorded by the numerical value is an integer or a long integer, and other various states are saved in two bytes (determined whether to expand to four bytes according to the actual data size). Then write the single-byte start flag as 0xF1. If the camera image collected during the recording process is lost, the gear, accelerator, brake, turn signal, low beam, high beam, and other superimposed information positions are empty and not written, only the current frame number and timestamp are filled, and the single-byte start flag is written as 0xF0, indicating that this frame has been recorded, but there is no data. The data size to be written for each frame is the size of the structure defined in S402 plus the size of a single-byte flag (i.e., 1 byte). The writing address of the vehicle information data of each frame of image is the size value of the structure defined in S502, and then it is written until the last frame is completed. The software stores one small file at a time during the storage process, such as one minute as a small file, and the buffer area is one minute at a time, so there is a last frame in a small file during the storage process. When the recording file is generated, the superimposed information of all frames of the current recording has been written in this buffer area.

[0086] S504: After the audio and video data are encoded, the video recording unit starts to pack and multiplex the audio and video data, and adds the header required by the file format. According to the format of the video storage area, the space of the extension part is allocated, and the information of the vehicle state is written to the defined position of the extension part. Taking the mp4 format as an example, an extension box is allocated to the current video file, and sub-boxes (different data categories correspond to different sub-boxes) are allocated under the box according to the frame number and the data category (general information, acceleration sensor data, geographic position information, and started intelligent driving function) to be written, and the vehicle state information of each frame is written to the defined position of the box.

[0087] In addition to the writing method of the box, the customized vehicle information data can also be written in another way: frame by frame into the SEI data. The supplemental enhancement information SEI is a concept in the code stream category, which provides a way to add information to the video code stream, and is one of the features of the H.264 / H.265 video compression standard. Saving the superimposed information based on the SEI data is another optional way of the embodiment.

[0088] After all the boxes are prepared, they are multiplexed into the mp4 format, the recording is completed, and the recording is written to the storage.

[0089] S505: Reset the cache and variables used in the process, and prepare for the next recording.

[0090] FIG. 6 is a recording process flow chart when the superimposed information is stored according to a separate file according to an embodiment of the application.

[0091] Referring to FIG. 6, the recording process when the superimposed information is stored according to a separate file includes:

[0092] S601: During recording, the camera transmits the images collected by the underlying layer to the upper layer frame by frame, and performs camera preview and recording operations. At the same time, various information of the vehicle and the driving state are obtained through system data and messages. The superimposed information obtained includes: general information (including vehicle frame number, system time, vehicle speed, vehicle light state, accelerator pedal, brake pedal, etc.), acceleration sensor and other sensor data obtained as needed, geographic position information, whether the intelligent driving function is started, etc. The type and number of information obtained can be customized and expanded in the software.

[0093] S602: The real-time images collected by each frame and the vehicle state data (superimposed information) are sent to the recording module in the software process together.

[0094] S603: The recording module of the software receives the current frame of video data and audio data, synchronizes the audio and video, and calls the respective encoders to complete the encoding work. This stage also performs the saving of the superimposed information. The information that is always displayed in the image (the vehicle identification code and the system time) is superimposed on the image in the recording in the same way as in the current superimposition process. The remaining vehicle status information is saved in the form of a variable: the vehicle information data obtained in the current frame is added to the frame number and the timestamp of the current image, and is written in the buffer area (i.e., the superimposed information storage area) shown in S401 according to the structure defined in S402 in FIG. 4. A single-byte start flag is written as 0xF1. If the camera image collected during the recording process is lost, the information on the gear, accelerator, brake, turn signal, low beam, high beam, and other vehicle states in the current frame is not written, and only the current frame number and the timestamp are filled in, and a single-byte start flag is written as 0xF0, indicating that this frame has been recorded, but there is no data. The data size to be written for each frame is the size of the structure defined in S402 plus the size of a single-byte flag (i.e., 1 byte). The writing address of the vehicle information data for each frame of image is the size value of the structure defined in S402 plus the additional writing, until the last frame is completed. When the recording file is generated, the vehicle information of all frames of the current recording has been written in this buffer area.

[0095] S604: After the audio and video data are encoded, the recording unit starts to pack and multiplex the audio and video data, and adds the required header of the file format. In this stage, a new additional file of the recording is created in the system-specific path by the camera application, and the file name is the recording file name (without the extension.mp4) plus a specific field (such as ‘_att’). The buffer data of the vehicle status information is written into the file. The permissions of the file are configured in the system. In this scheme, the recording program has read and write permissions for the file, the player program has only read permissions for the file, and other applications and processes do not have access to the file. The permissions can also be configured according to the actual needs of the project.

[0096] S605: The cache and variables used in the process are reset, and the next recording is prepared.

[0097] As an example, during the recording process of the driving recorder, due to some unexpected situations, mechanical or electrical failure occurs in the camera module or the host, resulting in abnormal termination of the recording process, and finally no valid video file is generated, resulting in loss of recorded images and superimposed information during this period. In the implementation and development, in the S603 and S604 stages, the superimposed information data is written in the buffer area, and the operation of writing the buffer area to the file can also be optimized in the process: during the recording process, it is not necessary to wait for all the frame information to be recorded, and then save the corresponding superimposed information to the file when the video file is finally generated. Instead, every certain period of time (such as 10 seconds), the information of the specific period of time is written into the buffer area and forced to flush into the superimposed information storage area for storage. In this way, the information is written repeatedly until the complete video file is generated. In this way, the buffer area applied in the process is relatively small, and more information is guaranteed to be written to the file, avoiding the failure of vehicle state information storage due to abnormal termination of the recording process.

[0098] As an example, as shown in FIG. 7, storing the superimposed information into the superimposed information storage area also includes S701-S702.

[0099] S701, in the case that at least one state of the superimposed information changes, storing the changed superimposed information into the superimposed information storage area.

[0100] S702, in the case that the video information is lost frame, and / or, the state of the superimposed information does not change, setting the superimposed information of the current time to be empty, or not storing the superimposed information of the current time.

[0101] Exemplarily, the application can also be optimized in the process of storing superimposed information. Because there may be no change in operation for a long time during vehicle driving, i.e., the state of the superimposed information does not change during vehicle driving, for example, the vehicle lights, gear and the like in the superimposed information do not change. Therefore, it is not necessary to record all the superimposed information of each frame into the superimposed information storage area, but only record the changed superimposed information when at least one state of the superimposed information changes. In this way, system resource space can be saved, and the efficiency of superimposed information storage can be improved.

[0102] Exemplarily, when storing the superimposed information, if the video recording unit finds that the video information is missing frames, the gear, accelerator, brake, turn signal, low beam, high beam, and other various superimposed information positions of the current frame can be left empty and not written, only the current frame number, time stamp is written, and a single byte start flag is written as 0xF0, indicating that this frame is missing frames. The superimposed information including only the frame number and time stamp is still saved to the superimposed information storage area. Alternatively, if the state of the superimposed information has not changed, the gear, accelerator, brake, turn signal, low beam, high beam, and other various superimposed information positions of the current frame can be left empty and not written, only the current frame number, time stamp is written, indicating that this frame has been recorded, but there is no data, and is saved to the superimposed information storage area. In this way, the efficiency of saving the superimposed information is improved. Of course, the superimposed information can also not be stored.

[0103] After the video information and the superimposed information are stored, the information stored in the video storage area and the video information storage area is processed in response to a user operation to display the superimposed information and / or the video information.

[0104] In the present application, the playing part is functionally customized. For example, the content of the superimposed information in the custom extension part of the video file can be parsed, or the content of a separate superimposed file can be parsed. The playing software integrates the icon materials that need to be displayed in the superimposed information in the development process. When the driving video needs to be viewed, the video file is unpacked in the playing process, the vehicle state corresponding to the current video frame is determined according to the content in the extension information, and the corresponding icon is drawn at the set position of the picture. The image after the superimposed information is drawn is output to the playing window for rendering. According to actual needs, whether to display the superimposed information and which part of the superimposed information to display can be selected when playing the driving video.

[0105] As an example, the user operation can represent displaying only the video information, or displaying only the superimposed information, or displaying both. The following describes the three cases in detail.

[0106] Displaying only the video information:

[0107] As an example, the user operation includes an instruction indicating to display the video information. In response to the user operation, the video information and / or the superimposed information is displayed, including: in response to the instruction indicating to display the video information, the video information in the video information storage area is displayed.

[0108] As an example, the user operation includes an instruction indicating to display the video information, for example, there is an option on the vehicle central control display screen as to whether to display the video information. It can be understood that when the user needs to display the video information, the information in the video information storage area is processed to obtain the video information, and the video information is displayed. It should be noted that the information in the video information storage area can be metadata, and processing the information in the video information storage area includes decoding the metadata to obtain image frame data. Of course, before saving the video information, it is already processed data, and the video information is directly displayed in response to the user's instruction to display the video information.

[0109] Only display the overlay information:

[0110] As an example, the user operation includes an instruction indicating to display the overlay information, and in response to the user operation, the video information and / or the overlay information is displayed, including: in response to the instruction indicating to display the overlay information, the overlay information stored in the overlay information storage area is displayed.

[0111] As an example, the user operation includes an instruction indicating to display the overlay information, for example, there is an option on the vehicle central control display screen as to whether to display the overlay information. It can be understood that when the user needs to display the overlay information, the information in the overlay information storage area is processed to obtain the overlay information, and the overlay information is displayed. It should be noted that the overlay information can be displayed alone or together with the video information. The overlay information can be stored in the format shown in FIG. 4, or can be stored in other custom formats, and when applied, the overlay information is parsed.

[0112] As an example, as shown in FIG. 8, the overlay information stored in the overlay information storage area is displayed, including:

[0113] S801, obtaining overlay layer data according to the overlay information.

[0114] S802, displaying the overlay layer data to display the overlay information.

[0115] As an example, after the information in the overlay information storage area is parsed, the overlay layer data is obtained by drawing operation according to the overlay information. The icon material related to the overlay information can be prepared in advance, and when the overlay layer data is drawn, the icon material corresponding to the overlay information is called to obtain the overlay layer data. The user operation also includes an instruction indicating to display the overlay information. It can be understood that the user operation can indicate which overlay information needs to be displayed or which overlay information does not need to be displayed. After the overlay layer data is determined, the overlay layer data is displayed to display the overlay information.

[0116] It should be noted that the superimposed layer data can be displayed alone, that is, the superimposed information is displayed alone. When both the superimposed information and the video information need to be displayed, the superimposed layer data can be superimposed on the image frame of the video information.

[0117] The video information and the superimposed information are displayed:

[0118] As an example, as shown in FIG. 9, the user operation includes an instruction indicating to display the video information and the superimposed information, and in response to the user operation, the video information and / or the superimposed information are displayed, including:

[0119] S901, in response to the instruction indicating to display the video information and the superimposed information, processing the information stored in the superimposed information storage area to obtain the superimposed information, and processing the information in the video information storage area to obtain the video information.

[0120] S902, displaying the video information and the superimposed information, wherein the superimposed information is superimposed and displayed on the video information.

[0121] As an example, the user operation includes an instruction indicating to display the video information and the superimposed information, and it can be understood that when both the superimposed information and the video information need to be displayed, the information stored in the superimposed information storage area is processed to obtain the superimposed information, the information in the video information storage area is processed to obtain the video information, and the processing process of the information in the storage area is similar to the above and will not be repeated. The superimposed information is superimposed and displayed on the video information to display the video information and the superimposed information.

[0122] As an example, as shown in FIG. 10, the video information and the superimposed information are displayed, including:

[0123] S1001, obtaining superimposed image information according to the image frame data and the superimposed information, wherein the image frame data is data processed according to the information in the video information storage area.

[0124] S1002, displaying the superimposed image information to display the video information and the superimposed information.

[0125] As an example, the information stored in the video storage area includes video information, and the decoding operation is performed on the information stored in the video storage area to obtain the image frame data. According to the superimposed information stored in the superimposed information storage area, the drawing operation is performed to draw the superimposed layer data. The superimposed image information is obtained according to the image frame data and the superimposed layer data. The superimposed image information is displayed to display the video information and the superimposed information.

[0126] As an example, the superimposed image information is obtained according to the image frame data and the superimposed information, including: obtaining the superimposed layer data according to the superimposed information; and obtaining the superimposed image information according to the image frame data and the superimposed layer data.

[0127] Exemplarily, in a case that a user operation represents a need to display the superimposed information and the video information, a drawing operation is performed on the superimposed information stored in the superimposed information storage area, to obtain superimposed layer data, which can include icon data and arrangement of the icon data. Based on the image frame data and the superimposed layer data, the superimposed layer is superimposed on the image frame to obtain superimposed image information. It can be understood that, based on the image frame data and the superimposed layer data, the superimposed layer is matched with the image frame, and an image superimposition rendering operation is performed to obtain the superimposed image information, and the superimposed image is displayed. In this way, the superimposed information can be displayed on the video information.

[0128] FIG. 11 is a schematic diagram of a playing process according to an embodiment of the present application.

[0129] As shown in FIG. 11, the video file includes a video file and a superimposed file. The video file is subjected to a decapsulation process to obtain video encoding data and audio encoding data. The video encoding data and the audio encoding data are subjected to decoding operations respectively to obtain image frames and original audio frames, which are sent to a video playing terminal and an audio playing terminal respectively to realize synchronous playing. In the present application, the audio path is consistent with a conventional playing process, and in the video path, a customized module that is independent of and not coupled with the original process of the player is added, to perform reading and analysis of the superimposed information before the image is sent to the window of the player, and to draw the superimposed information to the corresponding position of the current image frame according to the read superimposed information. The image on which the superimposed information is drawn is sent to the window of the player for rendering and playing.

[0130] As an example, the information stored in the video storage area includes video information and audio information. The information stored in the video storage area is subjected to a decoding operation, including: performing a decoding operation on the video information to obtain image frame data; and performing a decoding operation on the audio information to obtain audio frame data.

[0131] Exemplarily, the information stored in the video storage area includes video information and audio information. The obtained video file including video and audio encoding data is saved. The video information in the video file is subjected to a decoding operation to obtain image frame data, and the audio information is subjected to a decoding operation to obtain audio frame data. The video information can be decoded by a video decoder, and the audio information can be decoded by an audio decoder.

[0132] As an example, after the superimposed image is obtained, a synchronous processing is further performed on the superimposed image and the audio frame data, to synchronously play the superimposed image and the audio frame data.

[0133] Exemplarily, the image frame data comprises header data, and the header data comprises, for example, frame number and timestamp data. The superimposed layer can be superimposed on the image frame according to the header data of the image frame data, so as to obtain a superimposed image. For example, the header data of the superimposed image and the header data of the audio frame data can be matched, so that the superimposed image and the audio frame data can be synchronized, and the superimposed image and the audio frame data can be played synchronously.

[0134] Exemplarily, of course, the video and the audio can be decoded by sending the encoded data of the video and the audio to the respective decoders respectively, and the audio-video synchronization can be completed according to the video decoder and the audio decoder. Then, the superimposed layer operation can be performed.

[0135] As an example, according to the superimposed information, the superimposed layer data is obtained by: in the case that the video information is not lost frame and at least one state of the superimposed information of the current frame changes, determining the superimposed layer data according to the superimposed information of the current time.

[0136] Exemplarily, in the case that the image frame data corresponding to the current frame is not lost frame and at least one state of the superimposed information of the current frame changes, the superimposed information stored in the superimposed information storage area is determined to be drawn to obtain the superimposed layer data corresponding to the superimposed information.

[0137] It can be understood that if the information of the current image frame stored in the video information storage area is lost frame, the superimposed layer data corresponding to the superimposed information does not need to be determined, that is, the corresponding superimposed layer does not need to be drawn. If the superimposed information of the adjacent two frames in the superimposed information storage area is the same, that is, the state of the superimposed information does not change, the superimposed layer corresponding to the current frame does not need to be drawn. In the case that the video information is not lost frame, that is, the image frame data is not lost frame, and at least one state of the superimposed information of the current frame changes, the icon information and the state information corresponding to the icon information are drawn to obtain the superimposed layer data.

[0138] As an example, according to the superimposed information, the superimposed layer data is obtained by: in the case that the video information is lost frame and / or the state of the superimposed information of the current frame does not change, the superimposed layer data is obtained according to the superimposed information of the last time. The superimposed layer data of the last time is taken as the superimposed layer data of the current time.

[0139] Exemplarily, in the case of video information frame loss, i.e. image frame data frame loss, or no change in the state of the superimposed information of the current frame, the superimposed layer data is obtained according to the superimposed information of the last moment. For example, the superimposed layer data of the last moment has been drawn, and the superimposed layer data of the current superimposed information can be directly determined according to the superimposed layer data corresponding to the last superimposed information. For example, the superimposed layer of the last moment has not been drawn, and the superimposed layer data of the current moment can be obtained by drawing according to the superimposed information of the last moment.

[0140] Exemplarily, in the storage of superimposed information, the superimposed information is stored in the format of the structure defined in S402 (single-byte start flag, frame number of the current frame, timestamp, gear, throttle, brake, turn signal, low beam, high beam). If the video information of the current frame is lost, the positions of various superimposed information such as gear, throttle, brake, turn signal, low beam, high beam of the current frame are empty and not written, only the current frame number and timestamp are written, and the single-byte start flag is written as 0xF0, indicating that this frame has been recorded but has no data. The player application program parses the recorded superimposed information of each frame and converts it into a superimposed layer. If the recorded data start flag bit is 0xF0, this frame is processed as frame loss and does not need to create a superimposed layer. The superimposed layer data of the last frame is used as the current superimposed layer data.

[0141] Exemplarily, in the storage of superimposed information, in order to save system resources, there may be a case that the superimposed information does not change for a long time during vehicle driving. If the state of the superimposed information does not change, this superimposed information can not be recorded. In the drawing of the layer, if the current frame of the superimposed information has no content, the superimposed layer data of the last moment can be processed as the superimposed layer data of the current moment. Of course, in the storage of superimposed information, the superimposed information of each moment can also be stored. In the drawing of the superimposed layer, if the content of the current frame of the superimposed information is consistent with that of the last frame, i.e. the state does not change, the superimposed layer data of the last moment is still processed as the current superimposed layer data. In this way, the step of drawing the superimposed layer is saved, system resources are saved, and processing efficiency is improved.

[0142] For example, the storage process has a frame loss situation, the current frame data start flag saved in S401 is 0xF0, indicating that the superimposed information of the current frame is lost, and the superimposed information of this frame does not need to be updated, and the superimposed layer is abandoned. If all the vehicle states of the current frame do not change, i.e. the superimposed information is the same as that of the last frame, the superimposed layer of the current frame does not need to be drawn.

[0143] As an example, as shown in FIG. 12, according to the superimposed information, the superimposed layer data is obtained, including:

[0144] S1201, determine icon information corresponding to the superimposition information according to the superimposition information.

[0145] S1202, obtain superimposition layer data according to the icon information.

[0146] Exemplarily, icon information corresponding to the superimposition information is obtained according to the superimposition information. For example, icons corresponding to each superimposition information and arrangement of the icons can be prepared in advance, and the icon corresponding to the superimposition information is obtained from an icon library according to the determined superimposition information, for example, the superimposition information includes gear information, and the gear icon is obtained from the icon library. The icon information in the icon library corresponds to the superimposition information one by one. The superimposition layer data is obtained according to the icon information, for example, the superimposition layer data is obtained by drawing operation according to the obtained icon information. For example, the superimposition layer is drawn according to the icons and the previously determined arrangement relationship of the icons.

[0147] Exemplarily, the icon materials can be integrated in a specific path of the camera application in advance, and the application file is generated and built into the system after development. Each superimposition information and superimposition information state that needs to be displayed has a specific icon corresponding thereto on the dashboard camera application interface. When drawing the superimposition layer, the corresponding icon is drawn as a watermark on the corresponding position of the current camera image according to the state information of the current superimposition information obtained, the vehicle state corresponding to the current video frame is judged according to the content in the superimposition information storage area, and the corresponding icon is drawn in the set position of the picture. The image after the superimposition information drawing is output to the rendering window for rendering.

[0148] As an example, the superimposition information includes state information, and the icon information corresponding to the superimposition information is determined according to the superimposition information, including: determining the icon information corresponding to the state information in the superimposition information according to the superimposition information.

[0149] Exemplarily, when storing the superimposition information, the variables can be filled according to the structure format defined in S401 and written in the S402 buffer, and the saving mode of each data is determined according to the actual situation, for example, the information represented by the numerical value is recorded by the integer type or the long integer type, for example, the gear has four states, which can be represented by a binary number of two bits, and the states of the turn signal, the low beam and the high beam have only two states of on and off, which can be represented by a binary number of one bit, so the states of various states are saved by two bytes of space. The state information of the icon information corresponding to the superimposition information can be understood as the change of the icon, for example, the state information of the turn signal, the low beam and the high beam has only two states of on and off. The drawing operation is performed according to the icon information and the state information corresponding to the icon information, and the superimposition layer data is obtained.

[0150] As an example, as shown in FIG. 13, the superimposed information includes multiple types of superimposed information, and the information display method includes:

[0151] S1301, in response to a user operation, determining a target type of superimposed information.

[0152] S1302, displaying the target type of superimposed information.

[0153] Exemplarily, the present application can classify the superimposed information and display it in the form of types, and the user can operate to select the type of superimposed information to be displayed. According to the user selection, the target type of superimposed information is determined. Of course, the target type of superimposed information can be one or multiple. The target type of superimposed information is displayed. For example, the target type of superimposed information includes vehicle state information type, and the target type of superimposed information includes gear information, pedal information, and light information under this type. Of course, in addition to displaying a certain type or several types of superimposed information, all determined superimposed information can also be displayed. For example, only the front light information in the light information is displayed.

[0154] As an example, the type of superimposed information includes at least one of vehicle state information type, various sensor information type, position information type, and driving function information type. The vehicle state information includes at least one of gear information, pedal information, and light information.

[0155] Exemplarily, compared with the current superimposed information, the content of the superimposed information is expanded. In addition to the current vehicle state information, more vehicle information is recorded, such as various sensor information, position information, and intelligent driving function information. The superimposed information includes multiple types of superimposed information: vehicle state information type, various sensor information type, position information type, and driving function information type. The vehicle state information includes at least one of gear information, pedal information, and light information. The gear information includes gear, the pedal information includes throttle and brake, and the light information includes turn signal, low beam, and high beam. Of course, in addition to the above-mentioned vehicle state information, the superimposed information can also include other information, such as vehicle wiper and other vehicle driving data. The type of each superimposed information can be determined by checking whether to display the type. Of course, each superimposed information can be checked respectively to determine whether to superimpose it into the playing interface.

[0156] Exemplarily, the user can determine the type of the superimposed information to be displayed according to actual needs from multiple types, and can check the type of the superimposed information to be displayed by setting the option. Of course, the user can also close the display of some superimposed information according to the position of the superimposed information. For example, the position of the display of some superimposed information is shielded from the video picture, and the user can select to close this superimposed information. The specific classification manner can be defined according to the needs of the user, such as classification according to the use function and purpose.

[0157] The following is a detailed description of the flow of playing a video.

[0158] As an example, for the storage of superimposed information according to the above two storage manners, the flow of playing a video is also different.

[0159] The playing software needs to integrate the icon material of the superimposed information. It should be noted that the icon material and the setting item of the playing software of the driving recorder have been developed.

[0160] FIG. 14 is a flowchart of the playing process of the application when the storage area of the superimposed information is stored in the self-defined extension part of the video file.

[0161] Referring to FIG. 14, the specific playing steps are as follows:

[0162] S1401: When the driving recorder is playing, the software unpacks the video file to separate the encoding data of the video and the audio and the vehicle state information packaged together.

[0163] S1402: The encoding data of the video and the audio are respectively sent to the respective decoders to complete the decoding of the video and the audio, and the audio-video synchronization work is done.

[0164] S1403: According to the setting item, if it is set not to display the superimposed information, this step does not need to be executed. If it is set to display the superimposed information, a special thread is created to process the analysis of the superimposed information and the drawing of the superimposed layer: the self-defined extension part in the video file is analyzed to obtain the vehicle state data in the metadata, the frame number and the timestamp recorded in the data are compared with the image frame number and the timestamp to be played at present by the player software, the vehicle state information to be displayed is obtained, and the superimposed layer is drawn and superimposed on the frame image.

[0165] S1404: The image with the drawn superimposed information and the audio data are respectively output to the respective rendering units to complete the playing.

[0166] FIG. 15 is a flowchart of the playing process of the application when the storage area of the superimposed information is stored in a separate file.

[0167] Referring to FIG. 15, the specific playing steps are as follows:

[0168] S1501: When the car video recorder is playing, the software unpacks the video file, separates the video and audio coding data and the vehicle state information packaged together, and plays them.

[0169] S1502: The video and audio coding data are sent to the respective decoders to complete the decoding of the video and audio and to perform the audio-video synchronization.

[0170] S1503: If the setting is not to display the superimposed information, this step is not needed. If the setting is to display the superimposed information, a special thread is created to process the superimposed information and draw the superimposed layer: the additional file (superimposed file) produced in the video recording stage is read to obtain the vehicle state data, and the player software compares the frame number and timestamp recorded in the data with the current image frame number and timestamp to be played to obtain the vehicle state information to be displayed and draw the superimposed layer to be superimposed on the frame image.

[0171] S1504: The image with the superimposed information drawn is output to the respective rendering unit with the audio data to complete the playing.

[0172] When the video playing starts, it is first determined whether the superimposed information needs to be displayed. If the superimposed information needs to be displayed, the drawing thread of the superimposed layer is started to draw the superimposed information. It can be understood that the drawing thread of the superimposed layer is a sub-thread, and the decoding and playing of the content in the video storage area are the main thread. The sub-thread and the main thread are executed asynchronously, and it can be understood that the sub-thread and the main thread have no influence on each other and do their own things, and they also send messages synchronously. The sub-thread obtains the superimposed information of the current frame, determines whether the superimposed layer needs to be drawn according to the superimposed information of the current frame. In the case where it is determined that the superimposed layer needs to be drawn, the superimposed layer of a frame is drawn according to the icon material corresponding to the superimposed information of the current frame to obtain the current superimposed layer.

[0173] As an example, the information display method further includes: after the current superimposed layer is drawn, the current superimposed layer information is cached in the layer buffer queue.

[0174] Exemplarily, the layer buffer queue caches the layer information of each frame drawn, the sub-thread caches the superimposed layer information of the current frame in the layer buffer queue after the superimposed layer of the current frame is drawn, and the main thread takes out the superimposed layer information of the current frame from the layer buffer queue and superimposes it on the current original image frame.

[0175] Exemplarily, before the playing sub-process is performed, various parameters of the overlay layer, such as the size, can also be configured, and the buffer number and the buffer size of the layer buffer queue are configured. It can be understood that one buffer stores one frame of overlay layer information, and one queue can store multiple buffers. For example, the number of buffers in the layer buffer queue is 8, that is, the layer buffer queue can store 8 buffers. It can also be understood that the layer buffer queue can store 8 frames of overlay layer information.

[0176] Exemplarily, the overlay layer information obtained by analyzing the overlay information in the overlay information storage area is sequentially placed in the layer buffer queue according to the order of frame numbers, and the layer buffer queue is managed in a first-in first-out manner.

[0177] As an example, as shown in FIG. 16, the image frame data and the overlay information include header data. According to the image frame data and the overlay information, the overlay image information is obtained, including:

[0178] S1601, according to the header data of the image frame data and the header data of the overlay information, the overlay information corresponding to the image frame data is determined.

[0179] S1602, according to the image frame data and the overlay information corresponding to the image frame data, the overlay image information is obtained.

[0180] Exemplarily, the overlay image information can include overlay layer data or not. The overlay information can be overlaid on the image frame data in the form of an image, or in other forms, for example, in the form of text overlaid on the image frame to obtain the overlay image information. If the overlay image information includes the corresponding text overlay information, the text overlay information can be converted into an image form in subsequent processing. The following is described by taking the overlay information in the form of an image overlaid on the image frame data as an example.

[0181] Exemplarily, the image frame data and the overlay layer data both include header data. When the sub-thread draws the overlay layer of the current frame, the frame number and the timestamp obtained by analyzing the overlay information storage area are filled into the header data of the overlay layer. Then, the gear, throttle, brake, and turn signal information obtained by analyzing the overlay information are obtained, the icon material corresponding to the information state is obtained, and one frame of overlay layer is drawn. This part is the tail information of the overlay layer information, and the header data information (frame number, timestamp) forms a buffer, which is stored in the layer buffer queue. The main thread obtains the current image frame obtained by analyzing the information in the video information storage area. The current image frame data also includes header data of the frame number and the timestamp.

[0182] It can be understood that the head data names the image frame data and the superimposed information, and according to the head data of the image frame data and the head data of the superimposed information, the superimposed information corresponding to the image frame data can be determined. The superimposed information corresponding to each frame of image frame information is drawn into a superimposed layer, and is correspondingly superimposed on the image frame data to obtain superimposed image information.

[0183] As an example, as shown in FIG. 17, according to the head data of the image frame data and the head data of the superimposed information, the superimposed information corresponding to the image frame data is determined, including:

[0184] S1701, when the head data of the image frame data and the head data of the superimposed information match, according to the superimposed information matched with the image frame data, the superimposed information corresponding to the image frame data is determined.

[0185] S1702, when the head data of the image frame data and the head data of the superimposed information do not match, the superimposed information at the last moment is determined as the superimposed information corresponding to the image frame data.

[0186] Exemplarily, if the head data of the image frame data and the head data of the superimposed information match, according to the superimposed information matched with the image frame data, the superimposed information corresponding to the image frame data is determined. It can be understood that the superimposed information matched with the image frame data is the superimposed information corresponding to the video information in time at the time of collection. If the head data of the image frame data and the head data of the superimposed information do not match, the image frame may exist a frame loss situation, and at this time, the superimposed information at the last moment is taken as the superimposed information corresponding to the image frame data. It should be noted that the superimposed information can be superimposed on the image frame data in the form of an image, or in other forms, for example, in the form of text superimposed on the image frame.

[0187] It can be understood that the image frame data and the superimposed information can be matched first, and the superimposed information is attached to the image frame in the form of text or other forms to obtain superimposed image information. The superimposed information can also be drawn first to obtain superimposed layer data, and the superimposed layer data is superimposed on the image frame data to obtain superimposed image information. The superimposed layer data obtained by drawing first and superimposed on the image frame data will be described in detail below.

[0188] Exemplarily, in the case that it is determined that the current overlay layer has been drawn, the current overlay layer data is obtained from the layer buffer queue. The current overlay layer data includes head data (e.g., frame number, timestamp). For example, it is judged whether the head data of the current overlay layer data and the head data of the current image frame data match. In the case of matching, it is determined that the current overlay information is the overlay information corresponding to the image frame data, and the current overlay layer is superimposed on the current image frame. It can be understood that the current overlay layer and the current image frame are merged and rendered for playing.

[0189] Exemplarily, in the layer buffer queue, the overlay layer information is arranged in the queue in the order of the layer frames, and the frame numbers are arranged in ascending order. If the head data of the current overlay layer received in the main thread and the head data of the current image frame do not match, for example, the frame number in the head data of the current overlay layer is greater than the frame number of the current image frame, or the timestamp in the head data of the current overlay layer is later than the timestamp of the current image frame, it is indicated that in the sub-thread of the overlay information drawing, since the overlay information is exactly the same as before or the image itself has a frame loss, the overlay information layer of this frame is not drawn, and this frame will not take the new layer drawn in the playing process. At this time, the current frame played does not have a new overlay layer, and the overlay layer of the previous frame is continued to be used.

[0190] As an example, when the head data of the image frame data and the head data of the overlay information match, the overlay information corresponding to the image frame data is determined according to the overlay information matching the image frame data, including:

[0191] When the overlay information matching the image frame data is not empty, the overlay information is determined as the overlay information corresponding to the image frame data.

[0192] When the overlay information matching the image frame data is empty, the overlay information at the previous moment is determined as the overlay information corresponding to the image frame data.

[0193] Exemplarily, when saving the overlay information, when the video information at the current moment has a frame loss, or the state of the overlay information at the current moment is the same as the state of the overlay information at the previous moment, the positions of various overlay information such as gear, accelerator, brake, steering light, low beam, high beam, etc. in the overlay information are empty and not written, only the current frame number and timestamp are written, and the single-byte start flag is written as 0xF0, indicating that this frame has been recorded, but there is no data. When the overlay information matching the image frame data is not empty, the overlay information is determined as the overlay information corresponding to the image frame data. If the overlay information matching the image frame data is empty, it indicates that the overlay information has been recorded, but the content of the overlay information is empty. At this time, the overlay information at the previous moment is determined as the overlay information corresponding to the image frame data.

[0194] As an example, the header data of the overlay information includes at least one of frame number data and time data, and the matching of the header data of the overlay information and the header data of the image frame data includes at least one of the following manners:

[0195] The frame number data of the overlay information matches the frame number data of the image frame data;

[0196] The time data of the overlay information matches the time data of the image frame data.

[0197] Exemplarily, the header data of the overlay layer data includes at least one of a frame number and a time stamp, and the header data of the current image frame data also includes at least one of a frame number and a time stamp, the main thread acquires the current overlay layer data from the layer buffer queue, the frame number of the current overlay layer data is the same as the frame number of the current image frame data or the time stamp of the current overlay layer data is the same as the time stamp of the current image frame data, which indicates that the current overlay layer and the current image frame match, and the current overlay layer is superimposed on the current image frame.

[0198] Exemplarily, the main thread acquires the current overlay layer data from the layer buffer queue, and when the current overlay layer is processed, the mutex lock of the current overlay layer is acquired first, and it can be understood that the frame buffer is locked to ensure that the buffer is not modified or released by other threads when the overlay layer is displayed during playing. After the current overlay layer is superimposed on the image frame, the mutex lock of the current overlay layer data is released, and the current overlay layer is put back to the layer buffer queue for cyclic use.

[0199] FIG. 18 is a flowchart of a playing process according to an embodiment of the present application.

[0200] As shown in FIG. 18, when the driving recorder starts playing the video, it is first determined whether the overlay information needs to be displayed in step 1802. If the overlay information does not need to be displayed, the original process is executed in steps 1805 and 1809 to decode the image from the video stream and display it on the window. If the overlay information needs to be displayed, the overlay information processing module developed and customized according to the present application is used.

[0201] In step 1803, the size and other parameters of the superimposed information layer are configured, and then a certain number (indicating the number of frames that the queue can accommodate) and the size of the buffer are allocated to form the superimposed layer buffer queue of Q1800. The superimposed layer buffer queue of Q1800 in the figure is obtained by parsing the saved superimposed information storage area. The superimposed layer buffer queue includes a fixed number (the number indicates the number of frames that the queue can accommodate, for example, 8, which means that the queue saves 8 frames of data) of data buffers, and the content of each buffer includes a header data information of a superimposed layer information and a superimposed layer. The header data information is filled by filling the frame number and the timestamp obtained by parsing the superimposed information in the superimposed information storage area. The gear, accelerator, brake, and turn signal information obtained by the superimposed information stored in the superimposed information storage area are used to draw a frame of superimposed layer by obtaining the icon material corresponding to the information state, and this part is the superimposed layer data. The superimposed layer data and the header information (frame number, timestamp) form a buffer, that is, one buffer stores one frame of data, and a queue can store multiple buffers. For example, the number of frames that the queue can accommodate is 8, that is, 8 buffers are stored.

[0202] Step S1804 creates a drawing thread of the superimposed layer. It can be understood that steps 1821-1825 are sub-threads for drawing the superimposed layer, and 1805-1811 are main threads for decoding the video file and merging the superimposed layer. The sub-threads and the main threads are executed together, and the two are not coupled, but can send and receive messages to each other. The sub-threads of the present application are not coupled with the original playback process, which is convenient for customization and extension. Moreover, the sub-threads and the main process are executed asynchronously, so that the drawing of the superimposed layer required for playing a frame can be completed in time.

[0203] The sub-threads put the superimposed layers obtained by parsing the content in the saved superimposed information storage area into the superimposed layer buffer queue in the order of frame number, and manage the buffer queue in a first-in first-out manner. In the sub-thread, step 1821 obtains the superimposed information storage area saved in the recording process, and parses frame by frame. In step 1822, it is judged whether the current frame needs to draw a new superimposed layer. The judgment is mainly based on two situations: 1. All vehicle states of the current frame do not change, that is, the superimposed information is the same as that of the previous frame, and there is no need to update, so the current frame does not need to draw a superimposed layer. 2. Frame loss occurs in the recording process, and the start flag of the current frame data saved in S501 is 0xF0, so the superimposed information of this frame does not need to be updated, and the superimposed layer is abandoned.

[0204] If the judgment result of the frame needs superimposed information, step 1823 is performed to parse the superimposed information of the previous frame from the superimposed information storage area, find the picture material corresponding to each vehicle state according to the information, and draw a superimposed information layer. The head information is added and placed in the superimposed layer buffer queue. In this way, each frame of superimposed information that needs to be updated is drawn in sequence, and the superimposed layer buffer queue is filled in sequence, so that the superimposed layer data is sent to the main thread.

[0205] After the superimposed layer data buffer is ready, step 1824 is performed to notify the player main thread that the superimposed layer of the previous frame is ready. It should be noted that in actual execution, the queue is not processed after all the superimposed images that need to be updated are drawn. Instead, after a frame is drawn and placed in the queue, the sub-thread 1824 sends a message to the main thread 1807, the main thread 1807 obtains the buffer in the queue, and the operation of each buffer is synchronized and managed by a mutex lock to prevent interference.

[0206] In the player main thread, step 1805 performs video decoding to obtain the current frame of image to be displayed. If the superimposed information is turned off in the setting of the driving recorder, the current image is displayed according to the original process. If the superimposed information needs to be displayed, step 1807 is performed to wait for the superimposed layer data generated by the superimposed drawing layer sub-thread. If the main thread has not received the ready message sent by the sub-thread after decoding a frame of image data, it needs to wait here. The buffer of a frame of superimposed layer received by the main thread from the superimposed layer buffer queue is determined according to the frame number and timestamp whether it is the superimposed layer that needs to be drawn. Under normal circumstances, a frame of image data corresponds to a frame of buffer in the queue. When the superimposed layer buffer of the current image frame is found, the mutex lock of the buffer is acquired to ensure that the buffer is not modified or released by other threads during the superimposed display in the playing process. Then, steps 1808 and 1809 are performed to superimpose the current image frame and the superimposed information data and send them to the player window for rendering.

[0207] Because the overlay buffer is put into the queue in the order of the image frames, the frame number is also arranged from small to large. If the frame number of the received buffer in step 1807 is larger than the frame number of the current image frame or the time stamp of the received buffer is later than the time stamp of the current image frame, it is indicated that the overlay information layer of this frame is not drawn in steps 1822 and 1823 in the overlay information drawing sub-thread because the overlay information is exactly the same as before or the image itself has a frame loss. Therefore, the new layer of this frame is not taken in steps 1807-1809 in the playing process. At this time, the current frame played has no new overlay layer, and the overlay layer of the previous frame is continuously used, and the buffer mutex lock of the overlay layer of the previous frame is continuously held.

[0208] After the current frame is completed with the overlay and is displayed, step 1811 is executed to release the mutex lock of the overlay information layer buffer of the current frame. Then in step 1825 in the overlay information drawing sub-thread, the buffer of the overlay layer of this frame is released and is put back into the overlay buffer queue. The buffer is recycled in the overlay buffer queue.

[0209] In step 1810, the overlay layer of the current frame has been used, which indicates that the buffer of this frame has been used for the merging rendering and is used up. Then the mutex lock is released, the buffer is put back into the queue, the sub-thread acquires a frame of overlay information again, and the main thread decodes a frame of image data again. The process of acquiring a frame of overlay information by the sub-thread and decoding a frame of image data by the main thread has no priority, because both are in the loop. However, in step 1807, the main thread still needs to wait for the message that the overlay layer of a frame is ready.

[0210] The steps 1805-1811 of the main thread of the player and the steps 1821-1825 of the overlay information drawing sub-thread have a life cycle of the entire playing time, and are executed in a loop until the playing is completed and the video file is closed.

[0211] As an example, the recording device also communicates with the plug-in software, and the information display method further includes: in response to a user operation representing a need to display the plug-in software, superimposing the plug-in software as a display window on the video information.

[0212] As an example, the plug-in software can be a map software, and the recording device communicates with the map software. In response to a user operation representing a need to display the plug-in software, the map software can be superimposed as a display window on the video information to display geographical position information.

[0213] Exemplarily, the scheme of the present application can obtain more rich superimposed information than the scheme of the existing driving recorder, which can be expanded in quantity and category. Exemplarily, the superimposed information function can be further expanded according to the project requirements. For example, the driving recorder also communicates with the map software, and the geographic location information is shared when the video or photo is shared, so as to enhance the interaction between the driving recorder and the map software. The geographic information can also be directly seen by using the geographic location information to superimpose a small map on the video playing window. The user clicks the small map, and a complete map application is started, so that the user can more conveniently and comprehensively view the location and environment at the time when the accident is viewed through the driving recorder. Specific data can also be provided to the application for accident analysis and processing.

[0214] The present application also provides a vehicle.

[0215] In this embodiment, the vehicle comprises a driving recorder, and the driving recorder is used to implement the steps of the information display method.

[0216] The present application also provides an information display device.

[0217] In this embodiment, as shown in FIG. 19, the information display device comprises a display module 10, which is used to display video information and / or superimposed information in response to a user operation, wherein the video information is information obtained by recording equipment, and the superimposed information is information collected when the recording equipment records the video information, and the superimposed information is used to be superimposed and displayed on the video information; wherein the video information is stored in a video information storage area, and the superimposed information is stored in a superimposed information storage area.

[0218] The present application also provides a computer readable storage medium.

[0219] In this embodiment, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the information display method.

[0220] FIG. 20 is a block diagram of an electronic device provided by an embodiment of the present application.

[0221] An embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the information display method when executing the computer program.

[0222] As shown in FIG. 20, in order to facilitate understanding, an embodiment of the present application shows a specific electronic device.

[0223] Electronic device is intended to also refer to various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic device can also refer to various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown in FIG. 20, their connections, and their functions, as well as the software implemented by the electronic device, are meant only to be examples and are not intended to limit the present disclosure to any particular implementation of the present disclosure described and / or claimed herein.

[0224] As shown in FIG. 20, the electronic device includes a computing unit 2001 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 2002 or a computer program loaded from a storage unit 2008 into a random access memory (RAM) 2003. Various programs and data required for operation of the electronic device can also be stored in the RAM 2003. The computing unit 2001, the ROM 2002, and the RAM 2003 are connected to each other through a bus 2004. An input / output (I / O) interface 2005 is also connected to the bus 2004.

[0225] Various components in the electronic device are connected to the I / O interface 2005, including an input unit 2006, such as a keyboard, a mouse, and the like; an output unit 2007, such as various types of displays, speakers, and the like; the storage unit 2008, such as a magnetic disk, an optical disk, and the like; and a communication unit 2009, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 2009 allows the electronic device to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0226] The computing unit 2001 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 2001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 2001 performs various methods described above, such as the information display method. For example, in some embodiments, the information display method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 2008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device via the ROM 2002 and / or the communication unit 2009. When the computer program is loaded onto the RAM 2003 and executed by the computing unit 2001, the information display method described above can be performed. Alternatively, in other embodiments, the computing unit 2001 can be configured to perform the information display method by any other suitable means, such as by means of firmware.

[0227] It should be noted that the logical and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For purposes of this application, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical apparatus), a portable computer diskette (magnetic apparatus), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical apparatus), and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in order to be executed.

[0228] It should be understood that various aspects of the application can be implemented in hardware, software, firmware or a combination of them. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application-specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field-programmable gate arrays (FPGA), and so on.

[0229] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0230] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0231] In addition, the terms "first", "second", etc. used in the embodiments of the present application are only for the purpose of description and can not be understood as indicating or implying relative importance or implicitly indicating the number of technical features referred to in the embodiments. Therefore, the features defined with "first", "second" and the like in the embodiments of the present application can explicitly or implicitly indicate that the embodiments include at least one of the features. In the description of the present application, the meaning of the word "plurality" is at least two or two or more, such as two, three, four, etc., unless otherwise specifically limited in the embodiments.

[0232] In the present application, unless otherwise explicitly specified or limited in the embodiments, the terms "mounting", "connecting", "connecting" and "fixing" and the like appearing in the embodiments should be understood in a broad sense, for example, the connection can be fixed connection, or detachable connection, or integral, can be understood, or mechanical connection, electrical connection, etc. Of course, it can also be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements, or the interaction relationship of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific implementation situation.

[0233] In the present application, unless otherwise explicitly specified and limited, the first feature is "on" or "under" the second feature. The first and second features can be in direct contact, or the first and second features can be indirectly contacted through an intermediate medium. Moreover, the first feature can be above, above and above the second feature, or only indicates that the first feature is higher than the second feature in horizontal height. The first feature can be below, below and below the second feature, or only indicates that the first feature is lower than the second feature in horizontal height.

[0234] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and modifications to the above embodiments within the scope of the present application.

Claims

1. An information display method comprising: displaying video information and / or overlay information in response to a user operation, wherein the video information is information captured by a recording device, and the overlay information is information collected when the recording device captures the video information, and the overlay information is used to be displayed on the video information; wherein the video information is stored in a video information storage area, and the overlay information is stored in an overlay information storage area.

2. The information display method according to claim 1, wherein The user operation includes an instruction indicating to display the overlay information, and the displaying the video information and / or the overlay information in response to the user operation comprises: displaying the overlay information stored in the overlay information storage area in response to the instruction indicating to display the overlay information.

3. The information display method according to claim 2, wherein The displaying the overlay information stored in the overlay information storage area comprises: obtaining overlay layer data according to the overlay information; displaying the overlay layer data to display the overlay information.

4. The information display method according to claim 1, wherein The user operation includes an instruction indicating to display the video information and the overlay information, and the displaying the video information and / or the overlay information in response to the user operation comprises: processing information stored in the overlay information storage area to obtain the overlay information, and processing information stored in the video information storage area to obtain the video information in response to the instruction indicating to display the video information and the overlay information; displaying the video information and the overlay information, wherein the overlay information is displayed on the video information.

5. The information display method according to claim 4, wherein The displaying the video information and the overlay information comprises: obtaining overlay image information according to image frame data and the overlay information, wherein the image frame data is data processed according to information in the video information storage area; displaying the overlay image information to display the video information and the overlay information.

6. The information display method according to claim 5, wherein The obtaining overlay image information according to image frame data and the overlay information comprises: obtaining overlay layer data according to the overlay information; obtaining overlay image information according to the image frame data and the overlay layer data.

7. The information display method according to claim 6, wherein The obtaining overlay layer data according to the overlay information comprises: obtaining overlay layer data according to overlay information of a current time in a case that at least one of the following conditions is met: the video information has no frame loss, and a state in the overlay information of a current frame changes.

8. The information display method according to claim 6, wherein The obtaining overlay layer data according to the overlay information further comprises: obtaining overlay layer data according to overlay information of a previous time in a case that the video information has frame loss, and / or a state in the overlay information of the current frame does not change.

9. The information display method according to claim 5, wherein The image frame data and the overlay information include header data, and the obtaining overlay image information according to image frame data and the overlay information comprises: determining overlay information corresponding to the image frame data according to header data of the image frame data and header data of the overlay information; obtaining overlay image information according to the image frame data and the overlay information corresponding to the image frame data.

10. The information display method according to claim 9, wherein The determining overlay information corresponding to the image frame data according to header data of the image frame data and header data of the overlay information comprises: determining the superimposition information corresponding to the image frame data according to the superimposition information matched with the image frame data when the header data of the image frame data and the header data of the superimposition information match; determining the superimposition information corresponding to the image frame data as the superimposition information of the last moment when the header data of the image frame data and the header data of the superimposition information do not match.

11. The information display method according to claim 10, wherein The method comprises: determining the superimposition information of a target type in response to a user operation, and displaying the superimposition information of the target type. The types of the superimposition information include at least one of vehicle state information, various sensor information, position information, and driving function information, and the vehicle state information includes at least one of gear information, pedal information, and light information.

12. The information display method according to claim 10, wherein According to the superimposition information, superimposition layer data is obtained, which comprises: determining icon information corresponding to the superimposition information according to the superimposition information; obtaining superimposition layer data according to the icon information.

13. The information display method according to any one of claims 1-12, wherein, The superimposition information includes state information, and the method comprises: determining icon information corresponding to the state information in the superimposition information according to the superimposition information.

14. The information display method according to claim 13, wherein The user operation includes an instruction to display the video information, and the method comprises:

15. The information display method according to claim 3 or 6, wherein displaying the video information in the video information storage area in response to the instruction to display the video information. The method comprises: storing the superimposition information to a superimposition information storage area and storing the video information to the video information storage area when the superimposition information is target superimposition information.

16. The information display method according to claim 15, wherein The method further comprises: storing the superimposition information to the superimposition information storage area when at least one of the superimposition information changes; 17. The information display method according to any one of claims 1-16, wherein, setting the superimposition information of the current moment as null or not storing the superimposition information of the current moment when the video information is lost frame and / or the state of the superimposition information does not change. ​ 18. The information display method according to any one of claims 1-17, wherein, ​ ​ 19. The information display method according to claim 18, wherein ​ ​ ​ 20. The information display method according to any one of claims 1-19, wherein, In response to a user operation, the video information and / or the overlay information are displayed, including: In the case that the overlay information is not target overlay information, the overlay information is displayed on the video information, and is stored in the video information storage area together with the video information.

21. The information display method according to any one of claims 1-20, wherein, The recording device also communicates with the plug-in software, and the method further includes: In the case that the user operation represents a need to display the plug-in software, the plug-in software is displayed as a display window on the video information.

22. An information display device, wherein, The apparatus includes: A display module, configured to display video information and / or overlay information in response to a user operation, wherein the video information is information captured by a recording device, and the overlay information is information collected when the recording device captures the video information, and the overlay information is used to be displayed on the video information. The video information is stored in a video information storage area, and the overlay information is stored in an overlay information storage area.

23. A vehicle, wherein, The apparatus includes a recording device configured to capture video information, and a processing device configured to implement the steps of the method of any one of claims 1-21.

24. An electronic device, comprising: The apparatus includes a memory storing a computer program, and a processor, wherein the processor implements the steps of the method of any one of claims 1-20 when executing the computer program.

25. A computer readable storage medium, wherein, The computer program is stored on the apparatus and is executed by a processor to implement the steps of the method of any one of claims 1-21.

26. A computer program product, wherein, The computer program product includes instructions that are executed by a processor of a computer device to enable the computer device to implement the steps of the method of any one of claims 1-21.

Images