Multi-path vehicle-mounted camera remote monitoring method and device

Abstract

The embodiment of the invention discloses a multi-path vehicle-mounted camera remote monitoring method and device, and the method comprises the steps: obtaining a source video frame collected by each vehicle-mounted camera, adding corresponding timestamp information for each source video frame, and obtaining a video frame corresponding to each vehicle-mounted camera; copying each first frame to a backup buffer area, and moving to a target node; sequentially acquiring a current frame from other video frames corresponding to each vehicle-mounted camera, and judging whether each current frame meets a preset time requirement or not; if yes, using each current frame to replace the backup video frame stored in the backup buffer area, and moving each current frame to the target node; and when an abnormal frame which does not meet the preset time requirement exists in the current frame, for each abnormal frame, repairing the abnormal frame according to the backup video frame stored in the backup buffer area to obtain a repaired frame corresponding to the abnormal frame. By applying the scheme provided by the embodiment of the invention, the time synchronization of multiple paths of vehicle-mounted cameras can be realized.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and more specifically, to a method and apparatus for remote monitoring of multiple vehicle-mounted cameras. Background Technology

[0002] With the development of intelligent driving control technology, the requirements for vehicle monitoring in intelligent driving control systems are becoming increasingly stringent. Furthermore, vehicle monitoring typically includes multiple onboard cameras to acquire comprehensive environmental information about the vehicle's surroundings.

[0003] However, existing remote monitoring solutions are mostly used for indoor and road monitoring, typically employing a single camera for long-term remote monitoring, characterized by stable long-term monitoring. Intelligent driving control systems, on the other hand, utilize multiple vehicle-mounted cameras. Therefore, time synchronization between these cameras is crucial for obtaining accurate environmental information. Thus, achieving time synchronization among multiple vehicle-mounted cameras has become a pressing technical problem. Summary of the Invention

[0004] This application provides a method and apparatus for remote monitoring of multiple vehicle-mounted cameras to achieve time synchronization of multiple vehicle-mounted cameras. The specific technical solution is as follows.

[0005] In a first aspect, embodiments of this application provide a method for remote monitoring of multiple vehicle-mounted cameras, the method comprising:

[0006] The source video frames captured by each of the vehicle-mounted cameras are obtained, and for each of the vehicle-mounted cameras, the corresponding timestamp information is added to the source video frames captured by the vehicle-mounted camera to obtain the video frames corresponding to the vehicle-mounted camera.

[0007] The first frame of the video frame corresponding to each of the vehicle-mounted cameras is copied to the backup buffer and moved to the target node;

[0008] In the other video frames corresponding to each of the vehicle cameras, a current frame is obtained in sequence, and based on the timestamp information of each current frame, it is determined whether each current frame meets the preset time requirement.

[0009] When all the current frames meet the preset time requirement, the backup video frames stored in the backup buffer are replaced with the current frames, and the current frames are moved to the target node.

[0010] When there is an abnormal frame in the current frame that does not meet the preset time requirement, for each abnormal frame, the abnormal frame is repaired according to the backup video frame stored in the backup buffer to obtain the repair frame corresponding to the abnormal frame. The backup video frame stored in the backup buffer is replaced with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame are moved to the target node.

[0011] In one optional implementation, the step of repairing the abnormal frame based on the backup video frames stored in the backup buffer to obtain the repaired frame corresponding to the abnormal frame includes:

[0012] Among the backup video frames stored in the backup buffer, identify the target frame that corresponds to the same vehicle camera as the abnormal frame and has the closest timestamp;

[0013] Insert the target frame into the frame preceding the abnormal frame;

[0014] The target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame.

[0015] In one optional implementation, the timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time of the source video frame and the first frame captured by the vehicle-mounted camera; the step of determining whether each current frame meets the preset time requirement based on the timestamp information of each current frame includes:

[0016] Obtain the frame rate of each of the vehicle-mounted cameras;

[0017] For any given current frame, the current frame is deemed to meet the preset time requirement if it satisfies the following formula:

[0018]

[0019] The FPS is the frame rate of the vehicle camera that captures the current frame, id is the sequence number of the current frame, and T is the relative acquisition time between the current frame and the first frame captured by the corresponding vehicle camera.

[0020] In one alternative implementation, the method further includes:

[0021] Scale and stitch together the video frames that are moved to the target node in the same group into a single frame.

[0022] Determine the current transmission method;

[0023] The image is encoded into the corresponding format according to the current transmission method and then transmitted.

[0024] In one optional implementation, the current transmission method includes: local area network transmission or wide area network transmission; the step of encoding the image into a corresponding format according to the current transmission method and then transmitting it includes:

[0025] When the current transmission method is local area network transmission, the image is encoded into H264 format before transmission;

[0026] When the current transmission method is wide area network transmission, the image is encoded into H265 format before transmission.

[0027] Secondly, embodiments of this application provide a multi-channel vehicle-mounted camera remote monitoring device, the device comprising:

[0028] The source video frame acquisition module is used to acquire the source video frames captured by each of the vehicle-mounted cameras, and for each of the vehicle-mounted cameras, add corresponding timestamp information to the source video frames captured by the vehicle-mounted camera to obtain the video frame corresponding to the vehicle-mounted camera.

[0029] The first frame processing module is used to copy the first frame of the video frame corresponding to each of the vehicle-mounted cameras to the backup buffer and move it to the target node;

[0030] The time determination module is used to sequentially obtain a current frame from other video frames corresponding to each of the vehicle-mounted cameras, and determine whether each current frame meets the preset time requirement based on the timestamp information of each current frame.

[0031] The video frame processing module is used to replace the backup video frames stored in the backup buffer with each current frame when all the current frames meet the preset time requirement, and to move each current frame to the target node;

[0032] An abnormal frame processing module is used to, when there is an abnormal frame in the current frame that does not meet the preset time requirement, repair the abnormal frame according to the backup video frame stored in the backup buffer for each abnormal frame, obtain the repair frame corresponding to the abnormal frame, replace the backup video frame stored in the backup buffer with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and move the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame to the target node.

[0033] In one optional implementation, the abnormal frame processing module is specifically used for:

[0034] Among the backup video frames stored in the backup buffer, identify the target frame that corresponds to the same vehicle camera as the abnormal frame and has the closest timestamp;

[0035] Insert the target frame into the frame preceding the abnormal frame;

[0036] The target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame.

[0037] In one optional implementation, the timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time of the source video frame and the first frame captured by the vehicle-mounted camera; the time determination module is specifically used for:

[0038] Obtain the frame rate of each of the vehicle-mounted cameras;

[0039] For any given current frame, the current frame is deemed to meet the preset time requirement if it satisfies the following formula:

[0040]

[0041] The FPS is the frame rate of the vehicle camera that captures the current frame, id is the sequence number of the current frame, and T is the relative acquisition time between the current frame and the first frame captured by the corresponding vehicle camera.

[0042] In one alternative embodiment, the device further includes:

[0043] The video stitching module is used to scale and stitch together video frames that have been moved to the target node in the same group into a single frame.

[0044] The transmission mode determination module is used to determine the current transmission mode;

[0045] The video transmission module is used to encode the image into a corresponding format and then transmit it according to the current transmission method.

[0046] In one optional implementation, the current transmission method includes: local area network (LAN) transmission or wide area network (WAN) transmission; the video transmission module is specifically used for:

[0047] When the current transmission method is local area network transmission, the image is encoded into H264 format before transmission;

[0048] When the current transmission method is wide area network transmission, the image is encoded into H265 format before transmission.

[0049] In this embodiment, after acquiring the source video frames from various vehicle-mounted cameras, corresponding timestamp information can be added to each video frame. This allows for subsequent determination of whether each video frame meets the time requirements based on its timestamp information, thus ensuring the video time synchronization of each set of video frames. Furthermore, to address the time synchronization issue caused by frame loss, synchronized video frames can be stored in a backup buffer during the sequential judgment of each set of current frames. Consequently, if an abnormal frame that does not meet the time requirements exists in the next set of current frames, it can be repaired based on the video frames stored in the backup buffer. This ensures that each set of video frames that eventually moves to the target node is time-synchronized. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0051] Figure 1 A flowchart illustrating a method for remote monitoring of multiple vehicle-mounted cameras provided in an embodiment of this application is shown.

[0052] Figure 2 This application illustrates schematic diagrams of different video stream buffers in its embodiments.

[0053] Figure 3 This application illustrates a complete video frame processing flow diagram according to an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a multi-channel vehicle-mounted camera remote monitoring device provided in an embodiment of this application is shown. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0056] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0057] This application discloses a method and apparatus for remote monitoring of multiple vehicle-mounted cameras, which can achieve time synchronization of multiple vehicle-mounted cameras. The embodiments of this application are described in detail below.

[0058] Figure 1 The illustration shows a flowchart of a remote monitoring method for multiple vehicle-mounted cameras according to an embodiment of this application. This method can be applied to an algorithm server, and the process may include the following steps:

[0059] S110: Obtain the source video frames captured by each vehicle-mounted camera, and for each vehicle-mounted camera, add corresponding timestamp information to the source video frames captured by that vehicle-mounted camera to obtain the video frames corresponding to that vehicle-mounted camera.

[0060] In the field of autonomous driving, multiple onboard cameras are typically installed in vehicles to acquire video information about the surrounding environment. This information is then analyzed by the algorithm server within the System-on-Chip (SoC) to obtain complete environmental information, which is then used for operations such as path planning. It's understandable that before multiple cameras capture video, they should be synchronized in time, and each camera should capture video at the same frequency. Therefore, under normal circumstances, the timing of each video frame capture should be identical, resulting in the same number of video frames. However, during the actual video capture process, frame drops or other issues can cause time synchronization problems in the acquired video streams.

[0061] Specifically, the SOC can simultaneously send start signals to multiple vehicle-mounted cameras. Upon receiving the start signal, the threads of each vehicle-mounted camera simultaneously begin video stream acquisition, ensuring consistent acquisition timing across multiple video streams. After acquiring the source video, each vehicle-mounted camera can send its acquired source video to the algorithm server. The algorithm server then synchronizes the time of the source video acquired by each vehicle-mounted camera before sending it to the target node for subsequent analysis.

[0062] In this embodiment, after the algorithm server obtains the source video frames captured by each vehicle-mounted camera, it can add corresponding timestamp information to the source video frames captured by each vehicle-mounted camera to obtain the video frame corresponding to that vehicle-mounted camera. The timestamp information can be any information identifying the acquisition time of each source video frame.

[0063] Specifically, for any given source video frame, the timestamp information added to it can at least include: the acquisition time of that source video frame; alternatively, the source video frames acquired by each vehicle-mounted camera can be sorted. In this case, the timestamp information added to any given source video frame can include: the sequence number of that source video frame and the relative acquisition time of that source video frame and the first frame acquired by that vehicle-mounted camera. For example, for each vehicle-mounted camera, the sequence numbers of its corresponding video frames can be: 01, 02, 03, etc.

[0064] S120: Copy the first frame of the video frame corresponding to each vehicle camera to the backup buffer and move it to the target node.

[0065] Under normal circumstances, the number of video frames captured by each vehicle-mounted camera within the same time period should be the same, and the capture time of video frames with the same sequence number should also be the same within the allowable error range. In this embodiment, to achieve time synchronization of video frames captured by each vehicle-mounted camera, a group of video frames can be judged sequentially, with each group including one video frame captured by each vehicle-mounted camera. That is, each time a video frame captured by each vehicle-mounted camera is analyzed, the number of vehicle-mounted cameras is the number of video frames in each group. When each group of video frames achieves time synchronization, the time synchronization of video frames captured by each vehicle-mounted camera is achieved.

[0066] In this embodiment of the application, video frames from different processing stages can be stored using different buffers. For example... Figure 2 As shown, source video frames can be stored in multiple video stream buffers, and video frames with timestamp information added can be stored in the main video stream buffer. Synchronized video frames can be moved from the main video stream buffer to the target node. In other words, during the analysis of each group of video frames in the main video stream buffer, synchronized video frames will be removed from the main video stream buffer. When subsequent video frames become out of sync, the main video stream buffer will no longer contain the video frames previously captured for that specific video frame.

[0067] To address the issue of video frame desynchronization caused by frame loss, this embodiment of the application stores a set of newly determined time-synchronized video frames using a backup buffer. Thus, when subsequent video frames become time-synchronized, the desynchronized video frames can be repaired based on the video frames stored in the backup buffer.

[0068] Specifically, the first frame of the video frame corresponding to each vehicle camera can be directly copied to the backup buffer and moved to the target node.

[0069] S130: In the other video frames corresponding to each vehicle camera, obtain a current frame in sequence, and determine whether each current frame meets the preset time requirement based on the timestamp information of each current frame.

[0070] In the video frames corresponding to each vehicle-mounted camera, the first frame can be directly identified as synchronized, copied to the backup buffer, and moved to the target node. All subsequent frames require time synchronization checks. Specifically, one frame from each vehicle-mounted camera can be acquired each time as the current frame, and the timestamp information of each current frame can be used to determine whether each current frame meets the preset time requirements.

[0071] In one implementation, the timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time between the source video frame and the first frame captured by the vehicle-mounted camera. In this case, when determining whether each current frame meets the preset time requirement based on its timestamp information, the frame rate of each vehicle-mounted camera can be obtained first. Then, for any given current frame, if the current frame satisfies the following formula, it is determined that the current frame meets the preset time requirement:

[0072]

[0073] FPS is the frame rate of the vehicle camera capturing the current frame, id is the sequence number of the current frame, and T is the relative capture time between the current frame and the first frame captured by the corresponding vehicle camera.

[0074] Frame rate is the frequency (rate) at which bitmap images appear continuously on a camera, measured in frames. Frame rate can also be called frame frequency and is expressed in Hertz (Hz).

[0075] Under normal circumstances, the id and T values ​​of the video frames corresponding to each vehicle camera increase synchronously, and their values ​​satisfy the above formula. When any vehicle camera loses a frame, the currently acquired video frame will be a video frame acquired after the normal acquisition time, and its T value will be larger than the normal value, thus not satisfying the above formula.

[0076] S140: When each current frame meets the preset time requirement, replace the backup video frame stored in the backup buffer with each current frame, and move each current frame to the target node.

[0077] When all current frames meet the preset time requirements, it indicates that the time of the group of video frames is synchronized. In this case, each current frame can be used to replace the backup video frame stored in the backup buffer, and each current frame can be moved to the target node.

[0078] S150: When there is an abnormal frame in the current frame that does not meet the preset time requirement, for each abnormal frame, repair the abnormal frame according to the backup video frame stored in the backup buffer to obtain the repair frame corresponding to the abnormal frame. Replace the backup video frame stored in the backup buffer with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and move the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame to the target node.

[0079] When an abnormal frame that does not meet the preset time requirement exists in the current frame, it indicates that a frame loss occurred before the abnormal frame. In this case, each abnormal frame can be repaired individually. Specifically, for each abnormal frame, the abnormal frame can be repaired based on the backup video frames stored in the backup buffer to obtain the corresponding repaired frame.

[0080] In one implementation, the target frame with the same vehicle camera and the closest timestamp as the abnormal frame can first be identified from the video frames stored in the backup buffer. Then, the target frame is inserted into the frame preceding the abnormal frame. Finally, the target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame. After obtaining the repair frame, the backup video frames stored in the backup buffer can be further replaced with the repair frames corresponding to each abnormal frame and the non-abnormal frames in the current frame. The repair frames corresponding to each abnormal frame and the non-abnormal frames in the current frame are then moved to the target node.

[0081] In other words, for any abnormal frame, the nearest backup frame to the corresponding timestamp from the backup buffer can be retrieved and inserted into the main buffer along with the frame preceding that frame. Then, the abnormal frame and the inserted backup frame are all backed up to the backup buffer and sent to other processing nodes for processing. By synchronizing the startup of multiple cameras and using backup frames to re-synchronize when synchronization fails (determined by timestamp), the asynchrony problem caused by frame loss can be solved, thus enabling multiple vehicle-mounted cameras to meet synchronization requirements.

[0082] In this embodiment, after acquiring the source video frames from various vehicle-mounted cameras, corresponding timestamp information can be added to each video frame. This allows for subsequent determination of whether each video frame meets the time requirements based on its timestamp information, thus ensuring the video time synchronization of each set of video frames. Furthermore, to address the time synchronization issue caused by frame loss, synchronized video frames can be stored in a backup buffer during the sequential judgment of each set of current frames. Consequently, if an abnormal frame that does not meet the time requirements exists in the next set of current frames, it can be repaired based on the video frames stored in the backup buffer. This ensures that each set of video frames that eventually moves to the target node is time-synchronized.

[0083] It is understandable that multiple onboard cameras in autonomous vehicles, whether in remote driving or sentry mode, have high requirements for video latency in order to ensure accurate and real-time monitoring of the driving situation during remote surveillance.

[0084] As one implementation of this application, after each group of video frames is determined to be synchronized and sent to the target node, the video frames in the same group that have been moved to the target node can be scaled and stitched together into a single frame. Then, the current transmission method is determined, and finally, the image is encoded into the corresponding format and transmitted according to the current transmission method.

[0085] The aforementioned current transmission method may include: local area network transmission or network transmission; correspondingly, when the current transmission method is local area network transmission, the image can be encoded in H264 format before transmission; when the current transmission method is network transmission, the image can be encoded in H265 format before transmission.

[0086] In other words, during the image processing of the current frame, to accelerate transmission, all frames can be processed as needed before image scaling and stitching. This involves scaling and stitching the current frames from all vehicle cameras into a single frame, followed by format conversion and remote transmission. The aim is to reduce the transmission size of a single video frame and mitigate synchronization anomalies caused by packet loss during remote transmission. For example, for image transmission from n cameras, the transmission size before compression encoding can be reduced to 1 / n. Furthermore, since vehicle cameras often capture the same area from the same side, scaling and stitching the data from multiple cameras into a single frame during image encoding improves the local similarity of the single frame, thereby increasing the image compression rate of subsequent compression encoding algorithms, further reducing the image transmission size, and improving the transmission rate.

[0087] For environments using local area networks (LANs), the main latency in remote video transmission comes from the time taken by the onboard chip to process and compress the video stream. Therefore, the video stream format can be compressed and converted to H.264 (Network Device Interface Encoding Format). This format has a fast compression speed, low hardware performance consumption, and can reduce the time it takes for the onboard chip to process multiple video streams, thus reducing the latency of remote monitoring. For environments using network transmission (WANs), the main latency in remote video transmission comes from the network transmission time. Therefore, the video stream format can be converted to H.265 (High-Efficiency Video Encoding Format). This format has a higher video compression rate than H.264, which can reduce the network transmission pressure.

[0088] like Figure 3 As shown, it illustrates the complete video frame processing flow of this application embodiment. After the upper-layer application sends a synchronization start signal, multiple cameras can synchronously capture images and perform timestamp synchronization processing on multiple video frames. Then, the synchronized video stream can be processed accordingly. Specifically, this may include pre-processing the video frames according to the application settings, scaling and stitching multiple video frames into a single frame, and then encoding the single frame into H264 or H265 format based on the TDA4-VM board hardware. Furthermore, the H264 format image can be transmitted to a remote location via a local area network, and the H265 format image can be transmitted to a remote location via a network transmission method.

[0089] In this embodiment, the compression format of video transmission can be differentiated according to the different environments in which remote monitoring is used. Whether in environments where the driver is close to the vehicle, such as underground garages and parking lots where local area network transmission is applicable, or in situations where the driver needs to understand the surrounding environment of the vehicle from a distance, the transmission speed can be guaranteed to ensure the real-time monitoring. In other words, the solution proposed in this application can simultaneously acquire monitoring data from multiple cameras and transmit it to the driver in real time, which is more conducive to safe and intelligent driving of the car and the acquisition of vehicle condition and environmental information.

[0090] Figure 4 This illustration shows a structural diagram of a multi-channel vehicle-mounted camera remote monitoring device according to an embodiment of this application. The device includes:

[0091] The source video frame acquisition module 410 is used to acquire the source video frames captured by each of the vehicle-mounted cameras, and for each of the vehicle-mounted cameras, add corresponding timestamp information to the source video frames captured by the vehicle-mounted camera to obtain the video frame corresponding to the vehicle-mounted camera.

[0092] The first frame processing module 420 is used to copy the first frame of the video frame corresponding to each of the vehicle-mounted cameras to the backup buffer and move it to the target node;

[0093] The time determination module 430 is used to sequentially obtain a current frame from other video frames corresponding to each of the vehicle cameras, and determine whether each current frame meets the preset time requirement based on the timestamp information of each current frame.

[0094] The video frame processing module 440 is used to replace the backup video frame stored in the backup buffer with each current frame when each current frame meets the preset time requirement, and to move each current frame to the target node.

[0095] The abnormal frame processing module 450 is used to, when there is an abnormal frame in the current frame that does not meet the preset time requirement, repair the abnormal frame according to the backup video frame stored in the backup buffer for each abnormal frame, obtain the repair frame corresponding to the abnormal frame, replace the backup video frame stored in the backup buffer with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and move the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame to the target node.

[0096] In one optional implementation, the abnormal frame processing module 450 is specifically used for:

[0097] Among the backup video frames stored in the backup buffer, identify the target frame that corresponds to the same vehicle camera as the abnormal frame and has the closest timestamp;

[0098] Insert the target frame into the frame preceding the abnormal frame;

[0099] The target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame.

[0100] In one optional implementation, the timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time of the source video frame and the first frame captured by the vehicle-mounted camera; the time determination module 430 is specifically used for:

[0101] Obtain the frame rate of each of the vehicle-mounted cameras;

[0102] For any given current frame, the current frame is deemed to meet the preset time requirement if it satisfies the following formula:

[0103]

[0104] The FPS is the frame rate of the vehicle camera that captures the current frame, id is the sequence number of the current frame, and T is the relative acquisition time between the current frame and the first frame captured by the corresponding vehicle camera.

[0105] In one alternative embodiment, the device further includes:

[0106] The video stitching module is used to scale and stitch together video frames that have been moved to the target node in the same group into a single frame.

[0107] The transmission mode determination module is used to determine the current transmission mode;

[0108] The video transmission module is used to encode the image into a corresponding format and then transmit it according to the current transmission method.

[0109] In one optional implementation, the current transmission method includes: local area network (LAN) transmission or wide area network (WAN) transmission; the video transmission module is specifically used for:

[0110] When the current transmission method is local area network transmission, the image is encoded into H264 format before transmission;

[0111] When the current transmission method is wide area network transmission, the image is encoded into H265 format before transmission.

[0112] In this embodiment, after acquiring the source video frames from various vehicle-mounted cameras, corresponding timestamp information can be added to each video frame. This allows for subsequent determination of whether each video frame meets the time requirements based on its timestamp information, thus ensuring the video time synchronization of each set of video frames. Furthermore, to address the time synchronization issue caused by frame loss, synchronized video frames can be stored in a backup buffer during the sequential judgment of each set of current frames. Consequently, if an abnormal frame that does not meet the time requirements exists in the next set of current frames, it can be repaired based on the video frames stored in the backup buffer. This ensures that each set of video frames that eventually moves to the target node is time-synchronized.

[0113] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0114] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for remote monitoring of multiple vehicle-mounted cameras, characterized in that, The method includes: The source video frames captured by each of the vehicle-mounted cameras are obtained, and for each of the vehicle-mounted cameras, the corresponding timestamp information is added to the source video frames captured by the vehicle-mounted camera to obtain the video frames corresponding to the vehicle-mounted camera. The first frame of the video frame corresponding to each of the vehicle-mounted cameras is copied to the backup buffer and moved to the target node; In the other video frames corresponding to each of the vehicle cameras, a current frame is obtained in sequence, and based on the timestamp information of each current frame, it is determined whether each current frame meets the preset time requirement. When all the current frames meet the preset time requirement, the backup video frames stored in the backup buffer are replaced with the current frames, and the current frames are moved to the target node. When there is an abnormal frame in the current frame that does not meet the preset time requirement, for each abnormal frame, the abnormal frame is repaired according to the backup video frame stored in the backup buffer to obtain the repair frame corresponding to the abnormal frame. The backup video frame stored in the backup buffer is replaced with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame are moved to the target node.

2. The method according to claim 1, characterized in that, The step of repairing the abnormal frame based on the backup video frames stored in the backup buffer to obtain the repaired frame corresponding to the abnormal frame includes: Among the backup video frames stored in the backup buffer, identify the target frame that corresponds to the same vehicle camera as the abnormal frame and has the closest timestamp; Insert the target frame into the frame preceding the abnormal frame; The target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame.

3. The method according to claim 1, characterized in that, The timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time of the source video frame and the first frame captured by the vehicle-mounted camera; the step of determining whether each current frame meets the preset time requirement based on the timestamp information of each current frame includes: Obtain the frame rate of each of the vehicle-mounted cameras; For any given current frame, the current frame is deemed to meet the preset time requirement if it satisfies the following formula: The FPS is the frame rate of the vehicle camera that captures the current frame, id is the sequence number of the current frame, and T is the relative acquisition time between the current frame and the first frame captured by the corresponding vehicle camera.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Scale and stitch together the video frames that are moved to the target node in the same group into a single frame. Determine the current transmission method; The image is encoded into the corresponding format according to the current transmission method and then transmitted.

5. The method according to claim 4, characterized in that, The current transmission method includes: local area network transmission or wide area network transmission; the step of encoding the image into a corresponding format according to the current transmission method and then transmitting it includes: When the current transmission method is local area network transmission, the image is encoded into H264 format before transmission; When the current transmission method is wide area network transmission, the image is encoded into H265 format before transmission.

6. A multi-channel vehicle-mounted camera remote monitoring device, characterized in that, The device includes: The source video frame acquisition module is used to acquire the source video frames captured by each of the vehicle-mounted cameras, and for each of the vehicle-mounted cameras, add corresponding timestamp information to the source video frames captured by the vehicle-mounted camera to obtain the video frame corresponding to the vehicle-mounted camera. The first frame processing module is used to copy the first frame of the video frame corresponding to each of the vehicle-mounted cameras to the backup buffer and move it to the target node; The time determination module is used to sequentially obtain a current frame from other video frames corresponding to each of the vehicle-mounted cameras, and determine whether each current frame meets the preset time requirement based on the timestamp information of each current frame. The video frame processing module is used to replace the backup video frames stored in the backup buffer with each current frame when all the current frames meet the preset time requirement, and to move each current frame to the target node; An abnormal frame processing module is used to, when there is an abnormal frame in the current frame that does not meet the preset time requirement, repair the abnormal frame according to the backup video frame stored in the backup buffer for each abnormal frame, obtain the repair frame corresponding to the abnormal frame, replace the backup video frame stored in the backup buffer with the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame, and move the repair frame corresponding to each abnormal frame and the non-abnormal frame in the current frame to the target node.

7. The apparatus according to claim 6, characterized in that, The abnormal frame processing module is specifically used for: Among the backup video frames stored in the backup buffer, identify the target frame that corresponds to the same vehicle camera as the abnormal frame and has the closest timestamp; Insert the target frame into the frame preceding the abnormal frame; The target frame and the abnormal frame are used together as the repair frame corresponding to the abnormal frame.

8. The apparatus according to claim 6, characterized in that, The timestamp information of any source video frame captured by any vehicle-mounted camera includes at least: the sequence number of the source video frame and the relative acquisition time of the source video frame and the first frame captured by the vehicle-mounted camera; the time determination module is specifically used for: Obtain the frame rate of each of the vehicle-mounted cameras; For any given current frame, the current frame is deemed to meet the preset time requirement if it satisfies the following formula: The FPS is the frame rate of the vehicle camera that captures the current frame, id is the sequence number of the current frame, and T is the relative acquisition time between the current frame and the first frame captured by the corresponding vehicle camera.

9. The apparatus according to any one of claims 6-8, characterized in that, The device further includes: The video stitching module is used to scale and stitch together video frames that have been moved to the target node in the same group into a single frame. The transmission mode determination module is used to determine the current transmission mode; The video transmission module is used to encode the image into a corresponding format and then transmit it according to the current transmission method.

10. The apparatus according to claim 9, characterized in that, The current transmission method includes: Local area network (LAN) transmission or wide area network (WAN) transmission; the video transmission module is specifically used for: When the current transmission method is local area network transmission, the image is encoded into H264 format before transmission; When the current transmission method is wide area network transmission, the image is encoded into H265 format before transmission.

Images