Automobile data recorder video synthesis method, device, equipment and medium

Abstract

The invention relates to the technical field of vehicle-mounted video synthesis, in particular to an automobile data recorder video synthesis method, device and equipment and a medium, an automobile data recorder comprises a camera, the camera is connected with an intelligent driving domain controller, the method is applied to a cabin domain controller, and the method comprises the following steps: acquiring a first coded video stream transmitted by the intelligent driving domain controller, the first coded video stream is generated by performing video coding on original image data transmitted by a camera by the intelligent driving domain controller; acquiring vehicle driving information, generating a driving information image, and performing video coding on the driving information image to generate a second coded video stream; and inserting the coded data of the second coded video stream into a preset image region position of the target frame in the first coded video stream, and generating a fused driving record video stream. Therefore, the problems of high system resource consumption, poor function reusability and poor user experience in related technologies are solved, and the efficiency of synthesizing the multi-path high-definition driving video and the driving information and the user experience are improved.

Description

Technical Field

[0001] This application relates to the field of vehicle video synthesis technology, and in particular to a method, apparatus, equipment and medium for synthesizing video from a dashcam. Background Technology

[0002] With the rapid development of intelligent driving technology, Level 2 / Level 3 driver assistance systems are becoming increasingly common, leading to continuous increases in the number and resolution requirements of onboard cameras. High-definition multi-camera systems have become standard equipment in intelligent vehicles. However, cameras themselves do not have the ability to synthesize driving information (such as vehicle speed, turn signal status, and driver assistance information), and must rely on domain controllers for processing. This places higher demands on system architecture and resource allocation. Summary of the Invention

[0003] This application provides a method, apparatus, device, and medium for synthesizing video from a dashcam, in order to solve the problems of high system resource consumption, poor functional reusability, and poor user experience when synthesizing video and driving information in existing dashcam solutions. While reducing the resource consumption of the cockpit domain controller, it improves the efficiency and user experience of synthesizing multi-channel high-definition driving video and driving information.

[0004] A first aspect of this application provides a method for synthesizing video from a vehicle dashcam. The dashcam includes a camera connected to a smart driving domain controller. The method is applied to the cockpit domain controller and includes the following steps: The first encoded video stream transmitted by the intelligent driving domain controller is obtained. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. Acquire vehicle driving information, generate a driving information image based on the vehicle driving information, perform video encoding on the driving information image, and generate a second encoded video stream. The vehicle driving information is one or more of the following text content: time information, vehicle driving information, and alarm information. The driving information image is an image including the text content. The encoded data of the second encoded video stream is inserted into the preset image region position of the target frame in the first encoded video stream to generate a fused driving recorder video stream. The fused driving recorder video stream is used to generate a video frame that simultaneously includes the video image captured by the driving recorder and the driving information image corresponding to the video image.

[0005] By employing the aforementioned technical means, and by independently encoding the original camera images and the driving information images, and by directly inserting the second encoded video stream into a designated area of ​​the first encoded video stream at the macroblock level, CPU usage, DDR bandwidth, and codec load are significantly reduced, thereby improving the overall system performance and user experience while supporting multiple high-definition cameras.

[0006] According to one embodiment of this application, inserting the encoded data of the second encoded video stream into a preset image region position of the target frame in the first encoded video stream includes: The target frame is parsed, and the range of target macroblocks to be replaced in the first encoded video stream is calculated based on the parsing results and the size of the driving information image. The original encoded data of the target macroblock range to be replaced in the first encoded video stream is deleted, and the encoded data of the second encoded video stream is inserted into the target macroblock range to be replaced in the first encoded video stream.

[0007] By employing the aforementioned technical means, the target frame of the first encoded video stream is directly parsed, and the range of target macroblocks to be replaced is accurately calculated based on the size of the driving information image. Then, the original encoded data within this range is deleted, and the corresponding encoded data of the second encoded video stream is inserted. This achieves macroblock-level operation of the video stream in the encoding domain, avoiding the complex process of full-frame decoding, image overlay, and re-encoding in traditional schemes. This significantly reduces the system's computational load, memory bandwidth usage, and processing latency, thereby comprehensively improving the efficiency and performance of multi-channel high-definition video stream and driving information synthesis.

[0008] According to one embodiment of this application, the step of parsing the target frame and calculating the range of target macroblocks to be replaced in the first coded video stream based on the parsing result and the size of the driving information image includes: Obtain the resolution parameters of the target frame and the resolution parameters of the driving information image; Based on the resolution parameters of the target frame, calculate the image width and image height of the target frame in units of preset macroblocks, and calculate the image height of the driving information image in units of preset macroblocks based on the resolution parameters of the driving information image. Based on the image width of the target frame, the image height of the target frame, and the image height of the driving information image, which are represented in units of preset macroblocks, calculate the start macroblock address and end macroblock address of the target macroblock range; The target macroblock range is determined based on the start and end macroblock addresses of the target macroblock range.

[0009] Through the above technical means, a precise mathematical mapping of the image space at the encoding syntax layer is achieved, avoiding the decoding overhead caused by pixel-level operations. This provides a core positioning basis for the lossless replacement of driving information directly in the compressed bitstream, greatly improving the efficiency and accuracy of the synthesis processing.

[0010] According to one embodiment of this application, the target macroblock range to be replaced is the macroblock range from the starting macroblock address corresponding to the insertion position in the target frame to the last macroblock address of the target frame.

[0011] Through the aforementioned technical means, the cockpit domain controller can accurately stitch together the driving information image and the camera video stream by inserting the independently encoded second encoded video stream data into the target position without performing full-frame decoding and re-encoding.

[0012] According to one embodiment of this application, the width of the driving information image is equal to the width of the original image data.

[0013] By using the above technical means, by setting the width of the driving information image to be strictly equal to the width of the original image data, it is ensured that the two have the exact same number of horizontal macroblocks and alignment boundaries after encoding, avoiding macroblock misalignment, syntax errors or decoding failures caused by inconsistent widths, and ensuring the feasibility and reliability of the synthesis operation.

[0014] According to one embodiment of this application, the target frame is an I-frame or a P-frame, and the I-frame of the second encoded video stream is aligned with the I-frame of the first encoded video stream based on a first timestamp, and the P-frame of the second encoded video stream is aligned with the P-frame of the first encoded video stream based on a second timestamp.

[0015] By employing the aforementioned technical means and strictly aligning the frame types and timestamps of the first and second encoded video streams, the two video streams are ensured to be perfectly matched in terms of time synchronization and frame structure. This eliminates the need to reconstruct the GOP structure or adjust inter-frame dependencies when achieving precise frame-level replacement in the coding domain, thus guaranteeing the temporal integrity, encoding compliance, and playback smoothness of the synthesized dashcam video stream.

[0016] According to the dashcam video synthesis method provided in this application embodiment, a first encoded video stream transmitted by the intelligent driving domain controller is obtained. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. A driving information image is generated based on vehicle driving information, and the driving information image is video encoded to generate a second encoded video stream. The encoded data of the second encoded video stream is inserted into a preset image region position of the target frame in the first encoded video stream to generate a fused dashcam video stream. This solves the problems of high system resource consumption, poor functional reusability, and poor user experience in existing dashcam solutions when synthesizing video and driving information. It reduces the resource consumption of the cockpit domain controller while improving the efficiency and user experience of synthesizing multiple high-definition driving videos and driving information.

[0017] A second aspect of this application provides a video synthesis device for a vehicle dashcam, the dashcam including a camera connected to a smart driving domain controller, the device comprising: The acquisition module is used to acquire the first encoded video stream transmitted by the intelligent driving domain controller, wherein the first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera; The generation module is used to acquire vehicle driving information, generate a driving information image based on the vehicle driving information, perform video encoding on the driving information image, and generate a second encoded video stream. The vehicle driving information is one or more text contents of time information, vehicle driving information, and alarm information, and the driving information image is an image including the text content. The compositing module is used to insert the encoded data of the second encoded video stream into a preset image region position of the target frame in the first encoded video stream to generate a fused driving recorder video stream. The fused driving recorder video stream is used to generate a video frame that simultaneously includes a video image captured by the driving recorder and a driving information image corresponding to the video image.

[0018] According to one embodiment of this application, the synthesis module is used for: The target frame is parsed, and the range of target macroblocks to be replaced in the first encoded video stream is calculated based on the parsing results and the size of the driving information image. The original encoded data of the target macroblock range to be replaced in the first encoded video stream is deleted, and the encoded data of the second encoded video stream is inserted into the target macroblock range to be replaced in the first encoded video stream.

[0019] According to one embodiment of this application, the synthesis module is used for: Obtain the resolution parameters of the target frame and the resolution parameters of the driving information image; Based on the resolution parameters of the target frame, calculate the image width and image height of the target frame in units of preset macroblocks, and calculate the image height of the driving information image in units of preset macroblocks based on the resolution parameters of the driving information image. Based on the image width of the target frame, the image height of the target frame, and the image height of the driving information image, which are represented in units of preset macroblocks, calculate the start macroblock address and end macroblock address of the target macroblock range; The target macroblock range is determined based on the start and end macroblock addresses of the target macroblock range.

[0020] According to one embodiment of this application, the target macroblock range to be replaced is the macroblock range from the starting macroblock address corresponding to the insertion position in the target frame to the last macroblock address of the target frame.

[0021] According to one embodiment of this application, the width of the driving information image is equal to the width of the original image data.

[0022] According to one embodiment of this application, the target frame is an I-frame or a P-frame, and the I-frame of the second encoded video stream is aligned with the I-frame of the first encoded video stream based on a first timestamp, and the P-frame of the second encoded video stream is aligned with the P-frame of the first encoded video stream based on a second timestamp.

[0023] According to the dashcam video synthesis apparatus provided in this application embodiment, a first encoded video stream transmitted by the intelligent driving domain controller is acquired. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. A driving information image is generated based on vehicle driving information, and the driving information image is video encoded to generate a second encoded video stream. The encoded data of the second encoded video stream is inserted into a preset image region position of the target frame in the first encoded video stream to generate a fused dashcam video stream. This solves the problems of high system resource consumption, poor functional reusability, and poor user experience in existing dashcam solutions when synthesizing video and driving information. It reduces the resource consumption of the cockpit domain controller while improving the efficiency and user experience of synthesizing multiple high-definition driving videos and driving information.

[0024] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dashcam video synthesis method as described in the above embodiments.

[0025] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to perform the dashcam video synthesis method as described in the above embodiments.

[0026] A fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the dashcam video synthesis method as described in the above embodiments.

[0027] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0028] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a dashcam video synthesis method according to an embodiment of this application; Figure 2 This is a schematic diagram of the original image size of a camera according to an embodiment of this application; Figure 3 This is a schematic diagram of a camera data reading and caching process according to an embodiment of this application; Figure 4 This is a schematic diagram of H.264 / H.265 macroblock encoding according to an embodiment of this application; Figure 5 This is a schematic diagram showing the size and content of a driving information layer according to an embodiment of this application; Figure 6 This is a schematic diagram illustrating the determination of the target frame type according to an embodiment of this application; Figure 7 This is a schematic diagram showing that the target frame type is an I-frame according to an embodiment of this application; Figure 8 This is a schematic diagram showing that the target frame type is a P-frame according to an embodiment of this application; Figure 9 This is a schematic diagram of the decomposition of the macroblock Heigh according to an embodiment of this application; Figure 10 This is a schematic diagram illustrating the insertion of a corresponding driving information macroblock area according to an embodiment of this application; Figure 11 This is a schematic diagram of a dashcam video and driving information synthesis system architecture according to an embodiment of this application; Figure 12 This is a block diagram of a dashcam video synthesis device according to an embodiment of this application; Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] As those skilled in the art will understand, with the widespread application of Level 2 / Level 3 assisted driving systems, the computing power of the SOC (System on a Chip) chips used in intelligent driving domain controllers has significantly improved. Simultaneously, the number and resolution of in-vehicle cameras are constantly increasing, leading users to place higher demands on the video quality and multi-channel imaging capabilities of dashcams. However, cameras themselves do not possess the function of synthesizing driving information images. Furthermore, since the intelligent driving domain controller and the cockpit domain controller typically employ different SOC architectures, the raw video stream must be transmitted from the intelligent driving domain to the cockpit domain before information fusion can be performed. Therefore, how to achieve efficient synthesis of driving information and video data has become a key technical challenge for improving the performance of the cockpit domain controller.

[0031] Currently, there are three main technical solutions in the industry: First, the intelligent driving domain controller synthesizes driving information images and then sends them to the cockpit domain controller for storage as H.264 / H.265 video streams. However, this solution affects the ability of other applications to reuse this camera (without driving information images). Second, the intelligent driving domain controller transmits H.264 / H.265 encoded video streams (without driving information layers) to the cockpit domain, where the cockpit domain controller decodes, synthesizes, and re-encodes the video for storage. This process consumes significant SOC resources and can easily exhaust SOC computing power and memory bandwidth in multi-channel high-definition video scenarios. Third, the intelligent driving domain controller transmits H.264 / H.265 encoded video streams (without driving information layers) and the cockpit domain controller directly stores them as MP4 files, while simultaneously storing the driving information as a separate text file. While this method reduces processing overhead, it results in the separation of audio and video information during playback, making it difficult for users to view the dashcam footage on a computer or other devices.

[0032] Based on the technical problems existing in the prior art, this application proposes an efficient video synthesis technology for dashcam camera data and driving information, which is used to serve automotive in-vehicle Ethernet electronic and electrical systems. This technology may include H264 / H265 dashcam video streams, vehicle driving information images, and video stream and driving information synthesis technology.

[0033] The following description, with reference to the accompanying drawings, outlines a method, apparatus, device, and medium for synthesizing dashcam video according to embodiments of this application.

[0034] Specifically, Figure 1 This is a flowchart illustrating a dashcam video synthesis method provided in an embodiment of this application.

[0035] In this embodiment, the dashcam includes a camera, which is connected to the intelligent driving domain controller, and the method is applied to the cockpit domain controller.

[0036] like Figure 1 As shown, the dashcam video synthesis method includes the following steps: In step S101, the first encoded video stream transmitted by the intelligent driving domain controller is obtained. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera.

[0037] The intelligent driving domain controller is the core computing unit responsible for advanced driver assistance systems and autonomous driving functions, and is used to process data from sensors such as cameras, radar, and lidar. In this embodiment, the intelligent driving domain controller is responsible for camera video acquisition and encoding, and sends the raw video stream to the cockpit domain controller.

[0038] The cockpit domain controller is a computing unit that centrally manages human-machine interaction functions within the vehicle cockpit, integrating functions such as the instrument panel, central control screen, head-up display, voice recognition, multimedia entertainment, air conditioning control, and driver status monitoring. In this embodiment, the cockpit domain controller is responsible for generating driving information images (such as vehicle speed and time), encoding them into a video stream, and combining them with the camera video stream without decoding to ultimately generate a file containing driving information.

[0039] Specifically, the raw image data is the foundational data of the camera video. In this embodiment, the vehicle-mounted camera can be connected to the intelligent driving domain controller via a communication connection (either wired or wireless) to acquire the raw image data from the vehicle-mounted camera. The size of the raw image can be as follows: Figure 2 As shown, the resolution is Width1×Height1, and the raw image data from the vehicle camera is encoded and compressed using H.264 / H.265 for transmission to generate the first encoded video stream.

[0040] It should be noted that the vehicle camera parameters in this application embodiment may include camera resolution (Height×Width), frame rate (e.g., 30 FPS), and camera pixel type (e.g., YUV 4:2:0 / YUV 4:2:2).

[0041] For example, such as Figure 3As shown, firstly, raw image data is directly acquired through the vehicle-mounted camera. The intelligent driving domain controller reads the camera data into the camera buffer based on the camera resolution and frame rate. The intelligent driving domain controller then encodes and compresses the data in the camera buffer using H.264 / H.265. The encoding and compression process achieves efficient compression by configuring the corresponding compressed video image size, specifying the output frame rate, and defining the macroblock (MB) size (such as 16×16, 32×32, or 64×64 pixel blocks) parameters of H.264 / H.265. Finally, a compressed binary bitstream conforming to the standard syntax specification is output, which is the first encoded video stream.

[0042] The encoding compression parameters may include compression resolution (Height 1 × Width 1), frame transmission strategy (using IPPP strategy), and H.264 / H.265 macroblock size definition (such as 16×16, 32×32 or 64×64 pixel blocks).

[0043] It is understood that an encoded image is usually divided into several macroblocks. A macroblock can consist of a 16×16 luminance pixel and an additional 8×8 Cb and 8×8 Cr color pixel block.

[0044] It should be noted that H.264 and H.265 use a macroblock size of 16×16 as the smallest unit for image compression. Based on the pattern of each frame of data after image compression, as follows... Figure 4 As shown, a strategy of compressing data using one I-frame per second and other frames within one second as P-frames is adopted, and corresponding SPS (Sequence Parameter Set), PPS (Picture Parameter Set), IDR (Instantaneous Decoder Refresh) and other information are configured.

[0045] In step S102, vehicle driving information is acquired, and a driving information image is generated based on the vehicle driving information. The driving information image is then video encoded to generate a second encoded video stream. The vehicle driving information is text content of one or more of the following: time information, vehicle driving information, and alarm information. The driving information image is an image that includes the text content.

[0046] In some embodiments, the width of the driving information image is equal to the width of the original image data.

[0047] The time information can be the current date and time, and the vehicle driving information can include one or more of the following: vehicle speed, engine speed, gear position, turn signal status, throttle opening, brake status, and seat belt status. The warning information can include ADAS (Advanced Driver Assistance Systems) warning signals, without specific limitations.

[0048] Specifically, the cockpit domain controller acquires real-time driving information from various vehicle sensors and electronic control units via the vehicle's CAN (Controller Area Network) bus, Ethernet, or other in-vehicle network protocols. Utilizing its graphics processing capabilities, the cockpit domain controller dynamically renders and generates driving information images based on the acquired raw driving information data, such as... Figure 5 As shown, its content can be a visualization layer that overlays elements such as text, icons, and instrument graphics.

[0049] Furthermore, the generated driving information image undergoes independent video encoding and compression. The cockpit domain controller encodes and compresses the driving information image according to H.264 / H.265. To ensure seamless splicing and replacement with the first encoded video stream (camera video stream) at the bitstream level, this encoding process must maintain strict format alignment with the encoding parameters of the first video stream.

[0050] The parameters for encoding and compressing the driving information image may include the driving information resolution (Height2×Width2), the compression strategy (using the IPPP strategy), and the macroblock size definition of H.264 / H.265 (16×16 / 32×32 / 64×64) aligned with the macroblocks used for encoding and compressing the original image data.

[0051] Specifically, based on the parameters of the first encoded video stream, the width of the driving information image needs to be aligned to the width of the image output by the camera encoder to ensure that they are the same. Simultaneously, the macroblock divisions must also be aligned in size to ensure that both streams have the same frame structure and macroblock boundaries.

[0052] The calculation formula is as follows (taking 16x16 macroblocks as an example): Set the encoding resolution of the camera video stream to Width1×Heigh1, set the original resolution of the driving information image to Width2×Heigh2, and in order to achieve macroblock alignment, Width1==Width2 (the widths are absolutely equal), Width1 / 16 is an integer (the width is an integer multiple of the macroblock width), Heigh1 / 16 is an integer (the camera stream height is an integer multiple of the macroblock height), and Heigh2 / 16 is an integer (the driving information image height is an integer multiple of the macroblock height).

[0053] Furthermore, by independently encoding the driving information images with aligned widths and macroblock sizes, a second coded video stream, structurally compatible with the first coded video stream and containing only driving information graphics, is generated.

[0054] Furthermore, the cockpit domain controller acquires the H.264 / H.265 data stream, i.e. the first encoded video stream, from the intelligent driving domain controller. By analyzing the H.264 / H.265 feature data of the intelligent driving domain controller, it determines the image area location and inserts driving information image data.

[0055] In step S103, the encoded data of the second encoded video stream is inserted into the preset image region position of the target frame in the first encoded video stream to generate a fused driving recorder video stream. The fused driving recorder video stream is used to generate a video frame that simultaneously includes the video image captured by the driving recorder and the driving information image corresponding to the video image.

[0056] Furthermore, in some embodiments, inserting the encoded data of the second encoded video stream into a preset image region position of the target frame in the first encoded video stream includes: parsing the target frame and calculating the target macroblock range to be replaced in the first encoded video stream based on the parsing result and the size of the driving information image; deleting the original encoded data of the target macroblock range to be replaced in the first encoded video stream and inserting the encoded data of the second encoded video stream into the target macroblock range to be replaced in the first encoded video stream.

[0057] In some embodiments, the target frame is an I-frame or a P-frame, and the I-frames of the second encoded video stream are aligned with the I-frames of the first encoded video stream based on a first timestamp, and the P-frames of the second encoded video stream are aligned with the P-frames of the first encoded video stream based on a second timestamp.

[0058] Specifically, firstly, such as Figure 6 As shown, the target frame (I-frame or P-frame) in the first encoded video stream is parsed. By analyzing the H.264 / H.265 video data, i.e., the first encoded video stream, the SLICE data block is located according to the H.264 standard protocol format. The type of this frame is analyzed, and it is determined to be either an I-frame or a P-frame based on Slice_type == 1. For example, by parsing the target frame in the first encoded video stream, the result showing that the target frame is an I-frame is as follows. Figure 7 As shown, by parsing the target frame in the first encoded video stream, the result is that the target frame is a P-frame. Figure 8 As shown.

[0059] If the target frame is an I-frame, the corresponding macroblock location is continuously searched. For example, for a video stream with a resolution of 1920x1088, if the resolution of the driving information image is 144x1088, the image size in macroblocks must first be calculated based on the syntax elements in the sequence parameter set. 1920=(pic_width_in_mbs_minus1+1)*16, that is, pic_width_in_mbs_minus1=119.

[0060] 1088=(pic_height_in_map_units_minus1+1)*16, get pic_height_in_map_units_minus1=67.

[0061] The image width variable, defined in macroblocks, can be obtained from the following formula: PicWidthInMbs=pic_width_in_mbs_minus1+1==120 PicHeightInMapUnits = pic_height_in_map_units_minus1 + 1, from which we get PicHeightInMapUnits = 68; The starting address of the MB_layer where the insertion point is located is: PicWidthInMbs*(1088–144) / (pic_height_in_map_units_minus1+1)=120*(1088-144) / 68=1665.9.

[0062] According to the standard protocol of ITU-TH.264 / ITU-TH.265, the location of the target CurrMbAddr is determined; The PicSizeInMbs variable of the current image is obtained by the following formula: PicSizeInMbs = PicWidthInMbs * PicHeightInMbs; The image height variable in macroblocks is derived from the following formula: PicHeightInMbs = FrameHeightInMbs / (1 + field_pic_flag); `first_mb_in_slice` represents the address of the first macroblock in the slice; `CurrMbAddr = first_mb_in_slice * (1 + MbaffFrameFlag);` The NextMbAddress is obtained using the lookup formula: NextMbAddress(n), as follows: i=n+1 while(i <PicSizeInMbs&&MbToSliceGroupMap[i]!=MbToSliceGroupMap[n]) i++; nextMbAddress=i Furthermore, the specification for the mapping transformation from mapping unit to stripe group to macroblock to stripe group.

[0063] The mapping from macroblocks to stripe groups, i ranging from 0 to PicSizeInMapUnits-1 (inclusive), is generated by the following process: If frame_mbs_only_fag equals 1 or feld_pic_fag equals 1, the mapping from macroblock to stripe group is as follows: MbToSliceGroupMap[i]=mapUnitToSliceGroupMap[i].

[0064] Otherwise, if MbafFrameFlag equals 1, the mapping from macroblock to stripe group is as follows: MbToSliceGroupMap[i]=mapUnitToSliceGroupMap[i / 2].

[0065] Otherwise (frame_mbs_only_flag equals 0, and mb_adaptive_frame_field_flag equals 0, and field_pic_flag equals 0), the macroblock to stripe group mapping is as follows: MbToSliceGroupMap[i]=mapUnitToSliceGroupMap[(i / (2*PicWidthinMbs))*PicWidthInMbs+(i%PicWidthInMbs)].

[0066] In other words, based on the resolution of the camera video frame (e.g., 1920×1088) and the macroblock size in the H.264 / H.265 standard (e.g., 16×16), the image width and height in macroblock units are calculated. Based on the height of the driving information image, the number of macroblock rows corresponding to that height in the video frame is calculated, thus determining the starting row index of the insertion position in the macroblock graph. For example, if the driving information image height is 144 pixels, corresponding to 9 macroblock rows (144 / 16), then the insertion starting row is the total number of rows minus 9.

[0067] Based on the starting line index and image width, the address of the starting macroblock is calculated, which is the insertion position. All macroblocks from this address up to the last macroblock of the frame constitute the target macroblock range to be replaced.

[0068] Therefore, based on parameters such as `first_mb_in_slice` and the macroblock adaptive frame field coding flag (`MbaffFrameFlag`) in the slice header, the absolute address `CurrMbAddr` of the current macroblock is calculated. The macroblock's affiliation is determined by traversing the macroblock-to-slice group mapping table (`MbToSliceGroupMap`), and lookup functions such as `NextMbAddress(n)` are used to navigate the macroblock sequence throughout the frame. This precisely defines the number of consecutive macroblocks (determined by the height of the vehicle information image) belonging to the target macroblock range to be replaced, starting from the starting address. The target macroblock range to be replaced is the macroblock range from the starting macroblock address corresponding to the insertion position in the target frame to the last macroblock address in the target frame.

[0069] Furthermore, based on the location of the corresponding target macroblock range, the data from the insertion position to the last MB block is deleted, and the MB block data of the I-frame of the driving information is filled in, that is, the corresponding I-frame of the driving information graphic is inserted. Similarly, the corresponding P-frame of the driving information graphic is inserted. The insertion position is Heigh1-Heigh2, Width1 and Width2 are equal, and the frame rate relationship is that the I-frame is aligned and the P-frame is also aligned.

[0070] For example, the decomposition of the corresponding macroblock Heigh is as follows: Figure 9 As shown, insert the corresponding driving information macroblock area as follows: Figure 10 As shown.

[0071] In detail, the original encoded data of the identified target macroblock range in the first encoded video stream is deleted, directly removing the compressed data representing the camera image of that area. The encoded data of the corresponding frames (I-frame to I-frame, P-frame to P-frame) from the second encoded video stream (driving information video stream) are then inserted into this position. To ensure frame type matching, the I-frames of the second encoded video stream must be aligned with the I-frames of the first encoded video stream based on a first timestamp, and their P-frames must be aligned with each other based on a second timestamp. This ensures complete synchronization of the two independent video streams in terms of frame type and timing, guaranteeing that after replacing the camera background macroblock data with driving information macroblock data, the synthesized video stream is syntactically and temporally correct, and can be directly decoded and played by a standard decoder, ultimately generating the fused driving recorder video stream.

[0072] Therefore, this application significantly reduces the resource consumption of the cockpit domain controller in terms of CPU computation, DDR bandwidth, and codec load by directly embedding driving information images into the encoded video stream using macroblock-level replacement. With 3-megapixel, 5-megapixel, and even 8-megapixel high-definition cameras becoming mainstream, and the configuration of 11 external cameras per vehicle becoming increasingly common, this application can significantly reduce the performance load on the cockpit while significantly improving the overall system performance and user experience.

[0073] Further, in some embodiments, the target frame is parsed, and the range of target macroblocks to be replaced in the first coded video stream is calculated based on the parsing results and the size of the driving information image. This includes: obtaining the resolution parameters of the target frame and the resolution parameters of the driving information image; calculating the image width and image height of the target frame in units of preset macroblocks based on the resolution parameters of the target frame, and calculating the image height of the driving information image in units of preset macroblocks based on the resolution parameters of the driving information image; calculating the start macroblock address and end macroblock address of the target macroblock range based on the image width, image height, and image height of the target frame in units of preset macroblocks; and determining the target macroblock range based on the start macroblock address and end macroblock address of the target macroblock range.

[0074] Specifically, the target frame is parsed and key resolution parameters are obtained. The target frame (I-frame or P-frame) in the first encoded video stream (camera video stream) is parsed to read the frame resolution parameters, i.e., the logical width and height of the image, contained in its header information. Simultaneously, the resolution parameters (Width2 x Height2) of the pre-known driving information image (second encoded video stream) are obtained.

[0075] Furthermore, a macroblock unit standardization conversion is performed. Since the basic unit of H.264 / H.265 encoding and processing is the macroblock (usually 16x16 pixels), all pixel-based coordinates must be transformed to macroblock coordinate space. Based on the resolution parameters of the target frame, the image size in macroblock units is calculated using standard formulas: the image width of the target frame (PicWidthInMbs) = pic_width_in_mbs_minus1 + 1, and the image height of the target frame (PicHeightInMbs) = pic_height_in_map_units_minus1 + 1. Similarly, based on the resolution parameters of the driving information image (Heigh2), its corresponding macroblock height (Height2_InMbs) = Heigh2 / 16 (assuming the macroblock size is 16x16) is calculated.

[0076] Next, the address of the target macroblock range is calculated. Driving information images are typically overlaid on the bottom region of a video frame. Based on the above conversion results, the macroblock range covered by this overlaid region is determined through geometric operations: Start macroblock address (StartMbAddr): This address corresponds to the position of the first macroblock in the top left corner of the overlay area. It is calculated as: StartMbAddr = PicWidthInMbs * (PicHeightInMbs - Height2_InMbs). This means that starting from the beginning of the macroblock array, skipping the macroblocks in rows containing (total height macroblocks - driving information height macroblocks), the macroblock at the beginning of that row is the starting address.

[0077] End macroblock address (EndMbAddr): This address corresponds to the position of the last macroblock in the bottom right corner of the overlay area. It is calculated as: EndMbAddr = StartMbAddr + (PicWidthInMbs * Height2_InMbs) - 1. That is, starting from the beginning address, add (number of width macroblocks * number of height macroblocks) and then subtract one to obtain the index of the last macroblock in the entire rectangular area.

[0078] Finally, the target macroblock range is determined. Based on the calculated start macroblock address (StartMbAddr) and end macroblock address (EndMbAddr), a continuous macroblock sequence can be uniquely defined in the bitstream. This sequence precisely corresponds to the area in the video frame image that needs to be replaced by the driving information image content, thereby achieving efficient and accurate synthesis in the coding domain and avoiding the huge resource overhead of full-frame decoding and re-encoding.

[0079] Therefore, this application, based on the ITU-T H.264 / H.265 standard protocol, calculates the MB block position of the video encoded stream according to the display area size defined by the vehicle information watermark, and adopts a technical method of I-frame + P-frame video stream. It locates and inserts the MB position in the data stack of I-frame or P-frame, deletes the data after that frame, and fills in the I-frame or P-frame data of the vehicle information watermark. This overcomes the problem of excessive performance consumption caused by repeated video synthesis in traditional schemes, which leads to system lag, and significantly reduces performance consumption.

[0080] To facilitate a clearer and more intuitive understanding of the dashcam video synthesis method of this application by those skilled in the art, the following is combined with... Figure 11 A detailed explanation will be provided.

[0081] like Figure 11 As shown, the intelligent driving domain controller encodes the original camera image using H.264 / H.265 and transmits it to the cockpit domain controller via Ethernet. The cockpit domain controller generates information images from the vehicle driving information obtained from the CAN bus and encodes them independently. Then, it inserts the encoded data into the main video stream and finally synthesizes the full-function video required by the dashcam.

[0082] Therefore, this application achieves fusion on the cockpit domain controller by separately encoding camera data and driving information images, determining the insertion position on the camera data, deleting the insertion position up to the last MB block of data, and then inserting the driving information image data, thereby reducing the performance consumption of image synthesis.

[0083] According to the dashcam video synthesis method proposed in this application, a first encoded video stream transmitted by the intelligent driving domain controller is obtained. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. A driving information image is generated based on vehicle driving information, and the driving information image is video encoded to generate a second encoded video stream. The encoded data of the second encoded video stream is inserted into a preset image region position of the target frame in the first encoded video stream to generate a fused dashcam video stream. This solves the problems of high system resource consumption, poor functional reusability, and poor user experience in existing dashcam solutions when synthesizing video and driving information. It reduces the resource consumption of the cockpit domain controller while improving the efficiency and user experience of synthesizing multiple high-definition driving videos and driving information.

[0084] Next, the dashcam video synthesis apparatus according to the embodiments of this application is described with reference to the accompanying drawings.

[0085] In this embodiment, the dashcam includes a camera, which is connected to the intelligent driving domain controller.

[0086] Figure 12This is a block diagram of a dashcam video synthesis device according to an embodiment of this application.

[0087] like Figure 12 As shown, the dashcam video synthesis device 10 includes: an acquisition module 100, a generation module 200, and a synthesis module 300.

[0088] The acquisition module 100 is used to acquire the first encoded video stream transmitted by the intelligent driving domain controller. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. The generation module 200 is used to acquire vehicle driving information, generate a driving information image based on the vehicle driving information, perform video encoding on the driving information image, and generate a second encoded video stream. The vehicle driving information is text content of one or more of the following: time information, vehicle driving information, and alarm information. The driving information image is an image including the text content. The insertion module 300 is used to insert the encoded data of the second encoded video stream into a preset image area position of the target frame in the first encoded video stream to generate a fused driving recording video stream. The fused driving recording video stream is used to generate a video screen that simultaneously includes the video image captured by the driving recorder and the driving information image corresponding to the video image.

[0089] Furthermore, in some embodiments, the synthesis module 300 is configured to: parse the target frame and calculate the target macroblock range to be replaced in the first coded video stream based on the parsing result and the size of the driving information image; delete the original encoded data of the target macroblock range to be replaced in the first coded video stream and insert the encoded data of the second coded video stream into the target macroblock range to be replaced in the first coded video stream.

[0090] Further, in some embodiments, the synthesis module 300 is configured to: acquire the resolution parameters of the target frame and the resolution parameters of the driving information image; calculate the image width and image height of the target frame, represented in units of preset macroblocks, based on the resolution parameters of the target frame, and calculate the image height of the driving information image, represented in units of preset macroblocks, based on the resolution parameters of the driving information image; calculate the start macroblock address and end macroblock address of the target macroblock range based on the image width, image height, and image height of the target frame, represented in units of preset macroblocks; and determine the target macroblock range based on the start macroblock address and end macroblock address of the target macroblock range.

[0091] Furthermore, in some embodiments, the target macroblock range to be replaced is the macroblock range from the starting macroblock address corresponding to the insertion position in the target frame to the last macroblock address in the target frame.

[0092] Furthermore, in some embodiments, the width of the driving information image is equal to the width of the original image data.

[0093] Furthermore, in some embodiments, the target frame is an I-frame or a P-frame, and the I-frames of the second encoded video stream are aligned with the I-frames of the first encoded video stream based on a first timestamp, and the P-frames of the second encoded video stream are aligned with the P-frames of the first encoded video stream based on a second timestamp.

[0094] It should be noted that the foregoing explanation of the dashcam video synthesis method embodiment also applies to the dashcam video synthesis device of this embodiment, and will not be repeated here.

[0095] According to the dashcam video synthesis device proposed in this application, a first encoded video stream transmitted by the intelligent driving domain controller is acquired. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. A driving information image is generated based on vehicle driving information, and the driving information image is video encoded to generate a second encoded video stream. The encoded data of the second encoded video stream is inserted into a preset image region position of the target frame in the first encoded video stream to generate a fused dashcam video stream. This solves the problems of high system resource consumption, poor functional reusability, and poor user experience in existing dashcam solutions when synthesizing video and driving information. It reduces the resource consumption of the cockpit domain controller while improving the efficiency and user experience of synthesizing multiple high-definition driving videos and driving information.

[0096] Figure 13 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 1301, the processor 1302, and the computer program stored on the memory 1301 and executable on the processor 1302.

[0097] When the processor 1302 executes the program, it implements the dashcam video synthesis method provided in the above embodiments.

[0098] Furthermore, electronic devices also include: Communication interface 1303 is used for communication between memory 1301 and processor 1302.

[0099] The memory 1301 is used to store computer programs that can run on the processor 1302.

[0100] The memory 1301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0101] If the memory 1301, processor 1302, and communication interface 1303 are implemented independently, then the communication interface 1303, memory 1301, and processor 1302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0102] Optionally, in a specific implementation, if the memory 1301, processor 1302, and communication interface 1303 are integrated on a single chip, then the memory 1301, processor 1302, and communication interface 1303 can communicate with each other through an internal interface.

[0103] The processor 1302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0104] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described dashcam video synthesis method.

[0105] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described dashcam video synthesis method.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0108] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0110] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for synthesizing video from a dashcam, characterized in that, The dashcam includes a camera, which is communicatively connected to a smart driving domain controller. The smart driving domain controller is communicatively connected to a cockpit domain controller. The method is applied to the cockpit domain controller and includes: The first encoded video stream transmitted by the intelligent driving domain controller is obtained. The first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera. Acquire vehicle driving information, generate a driving information image based on the vehicle driving information, perform video encoding on the driving information image, and generate a second encoded video stream. The vehicle driving information is one or more of the following text content: time information, vehicle driving information, and alarm information. The driving information image is an image including the text content. The encoded data of the second encoded video stream is inserted into the preset image region position of the target frame in the first encoded video stream to generate a fused driving recorder video stream. The fused driving recorder video stream is used to generate a video frame that simultaneously includes the video image captured by the driving recorder and the driving information image corresponding to the video image.

2. The method according to claim 1, characterized in that, The step of inserting the encoded data of the second encoded video stream into the preset image region position of the target frame in the first encoded video stream includes: The target frame is parsed, and the range of target macroblocks to be replaced in the first encoded video stream is calculated based on the parsing results and the size of the driving information image. The original encoded data of the target macroblock range to be replaced in the first encoded video stream is deleted, and the encoded data of the second encoded video stream is inserted into the target macroblock range to be replaced in the first encoded video stream.

3. The method according to claim 2, characterized in that, The target macroblock range to be replaced is the range of macroblocks from the starting macroblock address corresponding to the insertion position in the target frame to the last macroblock address in the target frame.

4. The method according to claim 2, characterized in that, The step of parsing the target frame and calculating the range of target macroblocks to be replaced in the first encoded video stream based on the parsing result and the size of the driving information image includes: Obtain the resolution parameters of the target frame and the resolution parameters of the driving information image; Based on the resolution parameters of the target frame, calculate the image width and image height of the target frame in units of preset macroblocks, and calculate the image height of the driving information image in units of preset macroblocks based on the resolution parameters of the driving information image. Based on the image width of the target frame, the image height of the target frame, and the image height of the driving information image, which are represented in units of preset macroblocks, calculate the start macroblock address and end macroblock address of the target macroblock range; The target macroblock range is determined based on the start and end macroblock addresses of the target macroblock range.

5. The method according to claim 4, characterized in that, The width of the driving information image is equal to the width of the original image data.

6. The method according to claim 1, characterized in that, The target frame is an I-frame or a P-frame, and the I-frames of the second encoded video stream are aligned with the I-frames of the first encoded video stream based on a first timestamp, and the P-frames of the second encoded video stream are aligned with the P-frames of the first encoded video stream based on a second timestamp.

7. A video synthesis device for a vehicle dashcam, characterized in that, The dashcam includes a camera, which is connected to the intelligent driving domain controller. The device includes: The acquisition module is used to acquire the first encoded video stream transmitted by the intelligent driving domain controller, wherein the first encoded video stream is generated by the intelligent driving domain controller through video encoding of the original image data transmitted by the camera; The generation module is used to acquire vehicle driving information, generate a driving information image based on the vehicle driving information, perform video encoding on the driving information image, and generate a second encoded video stream. The vehicle driving information is one or more text contents of time information, vehicle driving information, and alarm information, and the driving information image is an image including the text content. The compositing module is used to insert the encoded data of the second encoded video stream into a preset image region position of the target frame in the first encoded video stream to generate a fused driving recorder video stream. The fused driving recorder video stream is used to generate a video frame that simultaneously includes a video image captured by the driving recorder and a driving information image corresponding to the video image.

8. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dashcam video synthesis method as described in any one of claims 1-6.

9. A vehicle, characterized in that, The system includes a dashcam, a smart driving domain controller, and a vehicle domain controller. The dashcam includes a camera, which is connected to the smart driving domain controller. The smart driving domain controller is connected to the vehicle domain controller, and the vehicle domain controller is used to perform the method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the dashcam video synthesis method as described in any one of claims 1-6.

Images