Data processing method and device, storage medium and program product

Abstract

The invention provides a data processing method and device, a storage medium and a program product, and the method comprises the steps: a control device carries out the clock synchronization calibration of a plurality of camera devices according to a main clock, and synchronously sends a preset control signal to the plurality of camera devices; after receiving a preset control signal sent by the control device, each camera device synchronously generates a key frame in response to the preset control signal, encodes and generates video stream data carrying a key frame generation timestamp, and then sends the video stream data taking the key frame as an initial frame to the control device; and the control equipment stores the multi-channel video stream data with the same timestamp from the plurality of camera devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block, so that the corresponding multi-channel data block is queried based on the time slot index anchor point subsequently for video playback. The synchronization of the multi-channel video can be effectively guaranteed, the video playback error is reduced, and the multi-channel video playback synchronization precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring device control, and in particular, to a data processing method, device, storage medium, and program product. Background Art

[0002] With the popularization of multi-channel video acquisition applications such as smart cities, intelligent transportation, and industrial monitoring, the requirements for the playback accuracy of multi-channel video stream data across devices and across master controls are also getting higher and higher.

[0003] In the existing multi-channel video synchronous playback technology, common solutions mainly rely on post-processing alignment of video streams. Usually, it is necessary to pre-cache or store multi-channel video stream data, and then perform timeline matching and synchronous playback on the multi-channel video stream data through the client or the server side. This type of method relies on the playback end to perform complex alignment processing on the cached data of multi-channel videos. There are errors in the timestamps of multi-channel video data from different devices, and it is difficult to ensure the synchronization of multi-channel videos during multi-channel video playback. The error of synchronous playback is large, resulting in low synchronous accuracy of video playback. Summary of the Invention

[0004] The present invention provides a data processing method, device, storage medium, and program product to solve the problems in the related technology that there are errors in the timestamps of multi-channel video data, video playback depends on the post-processing of multi-channel video stream data, it is difficult to ensure the synchronization of multi-channel videos, and the synchronous accuracy of video playback is low.

[0005] In a first aspect, an embodiment of the present application provides a data processing method applied to a camera device. The method includes:

[0006] Receiving a preset control signal sent by a control device, where the preset control signal is a control signal that the control device synchronously sends to multiple camera devices after clock synchronization calibration of the multiple camera devices according to a master clock;

[0007] Responding to the preset control signal to perform a key frame generation operation, so that each camera device that receives the preset control signal synchronously generates key frames and encodes to generate video stream data carrying key frame generation timestamps;

[0008] Sending the video stream data starting from the key frame to the control device, so that the control device stores the multi-channel video stream data with the same timestamp from multiple camera devices in the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block.

[0009] In one embodiment, clock synchronization calibration of multiple camera devices according to a master clock includes:

[0010] Receive the clock synchronization message sent by the control device, where the clock synchronization message is used to instruct multiple camera devices under the control device to perform clock synchronization calibration;

[0011] Determine the clock offset of the camera device according to the time difference between the message sending time and the message receiving time in the clock synchronization message, where the message receiving time is the time when the camera device receives the clock synchronization message;

[0012] Determine the clock drift rate of the camera device according to the clock offset of the camera device, and determine the weight coefficient for dynamically compensating the clock of the camera device;

[0013] Determine the calibrated target clock time according to the clock offset, clock drift rate and weight coefficient, and update the clock time of the camera device to the calibrated target clock time.

[0014] In one embodiment, determining the calibrated target clock time according to the clock offset, clock drift rate and weight coefficient includes:

[0015] Determine the calibration interval duration between the current clock synchronization calibration and the previous clock synchronization calibration;

[0016] Take the product of the calibration interval duration, clock drift rate and weight coefficient as the clock dynamic drift amount;

[0017] Take the sum of the clock time of the camera device before calibration, clock dynamic drift amount and clock offset as the calibrated target clock time.

[0018] In one embodiment, determining the weight coefficient for dynamically compensating the clock of the camera device includes:

[0019] Obtain the historical clock drift amount of the camera device and the actual operating parameters of the camera device, where the actual operating parameters include network jitter and device temperature;

[0020] Perform weighted summation on the historical clock drift amount, network jitter and device temperature of the camera device to obtain the weight coefficient for dynamically compensating the clock of the camera device.

[0021] In one embodiment, in response to a preset control signal, perform a key frame generation operation so that each camera device receiving the preset control signal synchronously generates key frames and encodes to generate video stream data carrying the key frame generation timestamp, including:

[0022] In response to a preset control signal, perform a key frame generation operation so that each camera device receiving the preset control signal synchronously generates key frames and encodes to obtain a group of pictures starting with the key frame;

[0023] Inject the frame rate, timestamp of the starting frame, and frame type of the picture group into the picture group as supplementary enhancement information of the picture group to generate video stream data, where the frame type is used to indicate whether the starting frame of the video stream data is a true key frame.

[0024] In a second aspect, an embodiment of the present application provides a data processing method, which is applied to a control device connected to multiple camera devices. The method includes:

[0025] After clock synchronization calibration of multiple camera devices according to the master clock, synchronously send a preset control signal to the multiple camera devices. The preset control signal is used to instruct each camera device to execute a key frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates key frames and encodes to generate video stream data carrying the timestamp of key frame generation;

[0026] Receive the video stream data sent by the camera device. The key frame generated by the camera device in response to the preset control signal is the starting frame of the video stream data;

[0027] Store the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generate and store a time slot index anchor point for querying the multi-channel data block.

[0028] In an embodiment, the video stream data includes a picture group starting with the generated key frame and supplementary enhancement information of the picture group. Storing the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generating and storing a time slot index anchor point for querying the multi-channel data block includes:

[0029] Store the picture groups in the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot in sequence according to the device identifier of each camera device to obtain a multi-channel data block;

[0030] Generate index information of the multi-channel data block according to the supplementary enhancement information of each picture group and store it in the time slot. The index information includes the global timestamp of the multi-channel data block, the picture group structure information and synchronization status information of each camera device. The synchronization status information is used to indicate whether the starting frame of each camera device is a true key frame;

[0031] Generate a time slot index anchor point pointing to the multi-channel data block according to the global timestamp in the index information of the multi-channel data block and store it.

[0032] In an embodiment, the time slot also stores the index information of the multi-channel data block. After generating and storing a time slot index anchor point for querying the multi-channel data block, the method further includes:

[0033] After receiving a video playback instruction for multiple camera devices, query the target time slot whose time slot index anchor point is within the playback timestamp;

[0034] Extract the data in the target time slot to obtain the target data block corresponding to the playback timestamp and the index information of the target data block;

[0035] Decode the target data block according to the synchronization status information in the index information of the target data block to obtain the decoded video data of each camera device, where the synchronization status information is used to indicate whether the starting frame of each camera device is a real key frame;

[0036] Perform key frame alignment processing on the decoded video data of each camera device according to the group of pictures structure information in the index information to obtain the target video data of each camera device;

[0037] Based on the target video data of each camera device, control the player to perform multi-channel video playback.

[0038] In one embodiment, decoding the target data block according to the synchronization status information in the index information of the target data block includes:

[0039] When it is determined according to the synchronization status information that the starting frame of the camera device in the target data block is a real key frame, decode the group of pictures of the camera device in the target data block;

[0040] When it is determined according to the synchronization status information that the starting frame of the camera device in the target data block is a virtual key frame, perform an effective key frame generation operation on the group of pictures of the camera device in the target data block, and decode the group of pictures of the camera device based on the generated effective key frame.

[0041] In a third aspect, an embodiment of the present application provides a camera device, including:

[0042] A receiving unit, configured to receive a preset control signal sent by a control device, where the preset control signal is a control signal that the control device synchronously sends to multiple camera devices after performing clock synchronization calibration on the multiple camera devices according to a master clock;

[0043] A generating unit, configured to perform a key frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates key frames and encodes and generates video stream data carrying the key frame generation timestamp;

[0044] A sending unit, configured to send the video stream data starting with the key frame to the control device, so that the control device stores the multi-channel video stream data with the same timestamp from multiple camera devices in the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block.

[0045] In a fourth aspect, an embodiment of the present application provides a control device, including:

[0046] A sending module, configured to synchronously send a preset control signal to a plurality of camera devices after clock synchronization calibration of the plurality of camera devices according to a master clock. The preset control signal is used to instruct each camera device to execute a key frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates a key frame and encodes it to generate video stream data carrying a key frame generation timestamp;

[0047] Receiving the video stream data sent by the camera device, and the key frame generated by the camera device in response to the preset control signal is the starting frame of the video stream data;

[0048] Storing the multi-channel video stream data with the same timestamp from a plurality of camera devices into the same time slot to obtain a multi-channel data block, and generating and storing a time slot index anchor point for querying the multi-channel data block.

[0049] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above data processing method are implemented.

[0050] In a sixth aspect, an embodiment of the present application provides a readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above data processing method are implemented.

[0051] In a seventh aspect, an embodiment of the present application provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above data processing method are implemented.

[0052] In a solution provided by the above data processing method, device, storage medium, and program product, after the control device performs clock synchronization calibration on multiple imaging devices according to the master clock, it synchronously sends a preset control signal to the multiple imaging devices. After receiving the preset control signal sent by the control device, the imaging device responds to the preset control signal to perform a key frame generation operation, so that each imaging device that receives the preset control signal synchronously generates key frames, encodes them to generate video stream data carrying the key frame generation timestamps, and then sends the video stream data starting from the key frame to the control device. The control device stores the multiplexed video stream data with the same timestamps from multiple imaging devices in the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block, so as to subsequently query the corresponding multi-channel data block based on the time slot index anchor point for video playback. In this embodiment, after the control device performs clock synchronization calibration on multiple imaging devices and sends a preset control signal to each imaging device to trigger each imaging device to forcibly generate key frames, and encodes the video data accordingly, the consistency of the key frame timestamps of the multiplexed video stream data can be ensured during the acquisition stage. This not only solves the problem of errors in the timestamps of multiplexed video data, but also reduces complex multiplexed video alignment operations, and can respond to playback instructions in a timely manner to achieve the effect of low playback synchronization delay. On this basis, multiplexed video stream data from different imaging devices are uniformly written into the same time slot according to the timestamps to form a multi-channel data block, which can reduce the deviation between the data reception time and the generation time from affecting the synchronization accuracy. During subsequent video playback, based on the time slot index anchor point, the data of the corresponding multi-channel data block can be quickly located and played back, which can effectively ensure the synchronization of multi-channel videos, reduce video playback errors, and improve the multiplexed video playback synchronization accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0054] Figure 1 is a schematic structural diagram of a monitoring system in an embodiment of the present invention;

[0055] Figure 2 is a schematic signaling interaction diagram of a data processing method in an embodiment of the present invention;

[0056] Figure 3 is Figure 2 a schematic signaling interaction diagram of the clock synchronization calibration process in

[0057] Figure 4 is Figure 2 A schematic diagram of an implementation process of step S3 in

[0058] Figure 5 is Figure 2 A schematic diagram of an implementation process of step S6 in

[0059] Figure 6 Another signaling interaction schematic diagram of the data processing method in an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of the framework of the monitoring system in an embodiment of the present invention;

[0061] Figure 8 is Figure 1 A schematic diagram of the structure of the camera device in

[0062] Figure 9 is Figure 1 A schematic diagram of the structure of the control device in

[0063] Figure 10 A schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed implementation manners

[0064] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0065] It should be understood that when used in the specification and appended claims of the present invention, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations. In addition, in the description of the specification and appended claims of the present invention, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0066] References to "an embodiment" or "some embodiments" etc. described in the specification of the present invention mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0067] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0068] In order to illustrate the technical solution of the present invention, specific embodiments will be used for illustration below.

[0069] In the existing multi-channel video synchronous playback technology, common solutions mainly rely on post-processing alignment of video streams. Usually, it is necessary to pre-cache or store multi-channel video stream data, and then perform timeline matching and synchronous playback on the multi-channel video stream data through the client or the server side. This type of method relies on the complex alignment processing of the cached data of multi-channel videos by the playback end, making it difficult to ensure the synchronization of multi-channel videos, with a large error in synchronous playback, resulting in low synchronization accuracy of video playback.

[0070] For example, in some video playback solutions, taking the global frame of a certain video as a reference, searching for the key frame (i.e., I-frame) with the closest timestamp in other video streams, and making a decision to start playing from the I-frame or subsequent target frames in combination with a time difference threshold judgment. This solution relies on a large amount of cached data, and due to possible deviations in the encoding time of different devices, limited by the clock synchronization error between devices, the synchronization accuracy of video playback is poor. In some video playback solutions, a unified timeline is established for multi-channel videos, and the frame information of multiple channels is aligned according to time for video playback, but this solution still relies on post-processing judgment and does not solve the consistency problem of frame generation time from the source, resulting in poor synchronization accuracy of video playback.

[0071] In order to solve the problem of consistency in frame generation time, some solutions synchronize device time through network protocols, and then cache video data so that the above-mentioned type of data playback can be performed when playback is required later. However, this type of solution still cannot guarantee the consistent generation of frame level or even key frames (I frames), especially in scenarios with different graphic output protocols (GOP) structures or heterogeneous encoding strategies, when playing back multiple videos from different devices, it is easy to cause cross-channel frame skipping, screen distortion or picture dislocation, and the synchronization accuracy of video playback is poor. In addition, some solutions align and archive videos according to data reception time, but this solution cannot reflect the actual acquisition time of the video, and is significantly affected by network transmission jitter and cache delay, resulting in serious timing drift and difficulty in effectively aligning video timestamps, resulting in large errors in synchronous playback and low synchronization accuracy of video playback.

[0072] In response to the above problems, an embodiment of the present invention provides a data processing method, device, storage medium and program product, wherein a control device performs clock synchronization calibration on multiple camera devices according to a master clock, and then synchronously sends a preset control signal to the multiple camera devices; after the camera device receives the preset control signal sent by the control device, it performs a key frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates a key frame, and encodes and generates video stream data carrying a key frame generation timestamp, and then sends the video stream data with the key frame as the starting frame to the control device; the control device stores multiple video stream data with the same timestamp from multiple camera devices in the same time slot to obtain a multi-channel data block, generates and stores a time slot index anchor point for querying the multi-channel data block, so as to subsequently query the corresponding multi-channel data block based on the time slot index anchor point for video playback. After the clock synchronization calibration of multiple camera devices is performed through the control device, a preset control signal is sent to each camera device to trigger each camera device to force the generation of key frames, so as to encode the video data. The consistency of the key frame timestamps of the multi-channel video stream data can be guaranteed in the acquisition stage, which not only solves the problem of errors in the timestamps of the multi-channel video data, but also reduces the complex multi-channel video alignment operations, and can respond to the playback instructions in time to achieve the effect of low playback synchronization delay; on this basis, the multi-channel video stream data from different camera devices are uniformly written into the same time slot according to the timestamp to form a multi-channel data block, which can reduce the situation where the deviation between the data reception time and the generation time affects the synchronization accuracy. In the subsequent video playback, the corresponding multi-channel data block playback data can be quickly located based on the time slot index anchor point and played, which can effectively ensure the synchronization of multi-channel videos, reduce video playback errors, and improve the synchronization accuracy of multi-channel video playback.

[0073] The data processing method provided by the embodiment of the present invention can be applied in Figure 1In the monitoring system shown, the monitoring system includes a control device and a plurality of camera devices associated with the control device (including Camera Device 1... Camera Device n, where n is an integer greater than 1). Among them, each camera device communicates with the control device through a network or a cable.

[0074] During the operation of the monitoring system, the control device synchronizes and calibrates the clocks of the plurality of camera devices according to the master clock, so that the plurality of camera devices are synchronized with the master clock, and after the synchronization and calibration, a preset control signal is synchronously sent to the plurality of camera devices. After receiving the preset control signal sent by the control device, the camera device performs a key frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates a key frame, and encodes to generate video stream data carrying the key frame generation timestamp; the camera device sends the video stream data starting from the key frame to the control device. After receiving the video stream data sent by the plurality of camera devices, the control device stores the multiplexed video stream data with the same timestamp from the plurality of camera devices into the same time slot to obtain a multi-channel data block, generates and stores a time slot index anchor point for querying the multi-channel data block, so as to subsequently query the corresponding multi-channel data block based on the time slot index anchor point for video playback.

[0075] In this embodiment, after the control device synchronizes and calibrates the clocks of the plurality of camera devices, a preset control signal is sent to each camera device to trigger each camera device to forcibly generate a key frame, and video data is encoded accordingly. In the acquisition stage, the consistency of the key frame timestamps of the multiplexed video stream data can be guaranteed; on this basis, the multiplexed video stream data from different camera devices is uniformly written into the same time slot according to the timestamp to form a multi-channel data block, which can reduce the deviation between the data reception time and the generation time from affecting the synchronization accuracy. During subsequent video playback, based on the time slot index anchor point, the data of the corresponding multi-channel data block can be quickly located and played back, which can effectively ensure the synchronization of multi-channel videos, reduce video playback errors, improve the synchronization accuracy of multi-channel video playback, and achieve the effect of low synchronization playback delay.

[0076] Meanwhile, the key-frame synchronization trigger mechanism can ensure the consistency of the key-frame timestamps of multi-channel video stream data during the data acquisition phase, making the frame structures of multi-channel videos consistent at the playback starting point and controlling the playback error within 1 ms, thus avoiding phenomena such as frame skipping, screen distortion, and latency. Moreover, during the data storage phase, multi-channel video stream data with the same timestamp is stored in the same time slot, obtaining a multi-channel data block with aligned timestamps. Subsequently, during video playback, only decoding is required, and complex data processing such as caching, frame filtering, and decoding alignment is not needed during the playback phase, improving the data processing efficiency and the response speed of video playback, enabling players supporting synchronous indexing to quickly achieve synchronous playback and greatly reducing the terminal performance requirements. In addition, in this solution, multi-channel video stream data can be stored in the same time slot during the acquisition process of the imaging device, supporting the access of imaging devices with different main controls and different coding structures. Through the unified timestamp and time slot structure, the alignment ability of heterogeneous videos is achieved. At the same time, based on the time slot-based multi-channel video storage and indexing method, the function of pulling streams while collecting to synchronously display multi-channel videos can be realized, with good real-time performance and scalability.

[0077] In the embodiment of the present invention, the provided monitoring system constructs a video synchronous acquisition and playback system applicable to heterogeneous, multi-channel, and cross-device environments through a full-process collaborative design from clock synchronization → key-frame triggering → timestamp embedding → storage alignment → index playback, improving the synchronous accuracy of multi-channel video playback and the system compatibility.

[0078] Among them, the monitoring system can be a traffic monitoring system or a monitoring system applied to other scenarios, such as an industrial monitoring system, a parking lot monitoring system, and a vehicle-mounted monitoring system. Among them, multiple imaging devices can be multiple independent imaging devices located at different positions under the same local area network, such as multiple cameras; multiple imaging devices can also be multiple sensors in different channels of a multi-channel imaging device. The control device can be connected to a video recording device or various terminal devices of multiple imaging devices, including but not limited to various personal computers, laptop computers, smartphones, tablet computers, and portable wearable devices, etc.; it can also be a multi-channel imaging device with multiple sensors in different channels. In other embodiments, the control device can also be a server, and the server can be implemented by an independent server or a server cluster composed of multiple servers.

[0079] In one embodiment, as Figure 2 shown, a data processing method is provided. Taking the application of this method in the Figure 1 monitoring system as an example, the method includes the following steps:

[0080] S1: After the control device calibrates the clock synchronization of multiple imaging devices according to the master clock, it synchronously sends a preset control signal to multiple imaging devices.

[0081] During the operation of the monitoring system, the control device can perform clock synchronization calibration on multiple camera devices at a certain calibration interval, so that the clocks of multiple camera devices can remain synchronized. Among them, the control device can perform clock synchronization calibration on multiple camera devices according to the master clock in the control device, so that the clocks of multiple camera devices are aligned with the time of the master clock, thereby ensuring that the clocks of multiple camera devices remain synchronized.

[0082] Among them, the control device can adopt the Precision Time Synchronization Protocol (PTP) to perform clock synchronization calibration on multiple camera devices based on the master clock, so as to achieve sub-microsecond-level clock synchronization among multiple devices. PTP synchronizes time through message passing between the master clock (i.e., the clock of the control device) and the slave clock (i.e., the clock of the camera device). The master clock sends timestamp messages, and the slave clock receives these timestamp messages and adjusts its time to achieve clock synchronization of multiple camera devices.

[0083] After performing clock synchronization calibration on multiple camera devices to synchronize multiple camera devices with the master clock, the control device synchronously sends a preset control signal to the multiple camera devices connected to it. That is, the preset control signal is the control signal sent by the control device to each camera device after calibrating the clocks of each camera device so that multiple camera devices are synchronized with the master clock. This preset control signal is used to instruct each camera device to respond to the preset control signal to perform a key frame generation operation, so that each camera device that receives the preset control signal synchronously generates a key frame and encodes it into video stream data.

[0084] Among them, the preset control signal can be a preset pulse signal. That is, after performing clock synchronization calibration on multiple camera devices to synchronize multiple camera devices with the master clock, the control device generates a preset pulse signal through a pulse generator in the control device and synchronously sends the preset pulse signal to the multiple camera devices connected to it, so as to trigger each camera device that receives the preset pulse signal to synchronously generate a key frame and encode it into video stream data. By triggering each camera device to forcibly generate a key frame through a preset pulse signal, the pulse trigger time is short, reducing the possibility of noise or false triggering, and improving the stability and reliability of key frame generation on the basis of ensuring the accuracy of the trigger operation.

[0085] S2: The camera device receives the preset control signal sent by the control device.

[0086] The camera device receives the preset control signal sent by the control device. Since the clocks of multiple camera devices have been calibrated, the clocks of the calibrated camera devices are synchronized with the master clock. After the control device synchronously sends the preset control signal, each camera device can synchronously receive the preset control signal.

[0087] S3: The imaging device executes a key frame generation operation in response to a preset control signal, so that each imaging device receiving the preset control signal synchronously generates a key frame and encodes it to generate video stream data carrying a key frame generation timestamp.

[0088] Upon receiving the preset control signal sent by the control device, the imaging device executes a key frame generation operation in response to the preset control signal, that is, the imaging device forcibly generates a key frame in response to the preset control signal, so that each imaging device receiving the preset control signal synchronously generates a key frame, and frame encodes the key frame and subsequent picture frames to generate video stream data carrying a key frame generation timestamp.

[0089] Among them, after the imaging device forcibly generates a key frame, the imaging device performs frame encoding based on the key frame to generate a Group of Pictures (GOP), that is, a GOP group. The GOP group includes a key frame (i.e., an I frame) and one or more predicted frames (such as P frames or B frames); then, the imaging device injects the timestamp of the key frame generation (key frame generation timestamp) into the GOP group to generate video stream data. Among them, the video stream data includes GOP groups generated within a preset duration and the key frame generation timestamps of the GOP groups, that is, the video stream data includes one or more GOP groups and the corresponding key frame generation timestamps of the GOP groups.

[0090] Among them, after generating the key frame, the starting frame of the video stream data encoded by the imaging device is the generated key frame, and the starting timestamp of the video stream data is the key frame generation timestamp, that is, the key frame generated by the imaging device in response to the preset control signal is the starting frame of the video stream data.

[0091] S4: The imaging device sends the video stream data starting with the key frame to the control device.

[0092] After encoding and generating the video stream data, each imaging device sends the video stream data starting with the key frame generated in response to the preset control signal to the control device.

[0093] S5: The control device receives the video stream data sent by the imaging device.

[0094] The control device receives the video stream data sent by each imaging device. Among them, among the video stream data received by the control device from each imaging device, the starting frame of at least one video stream data is the key frame generated by the imaging device in response to the preset control signal.

[0095] S6: The control device stores the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor for querying the multi-channel data block.

[0096] After receiving the video stream data sent by multiple camera devices, the control device stores the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block. Then, the control device generates a time slot index anchor for the multi-channel data block according to the timestamps of the multi-channel video stream data stored in the time slot, and stores the time slot index anchor.

[0097] Among them, the time slot index anchor is used to query the multi-channel data block, so that when the control device has a video playback requirement subsequently, the corresponding multi-channel data block can be queried according to the time slot index anchor, decoded to obtain the multi-channel video stream and played.

[0098] In this embodiment, after the control device performs clock synchronization calibration on multiple camera devices, a preset control signal is sent to each camera device to trigger each camera device to forcibly generate a key frame, and video data is encoded accordingly. In the acquisition stage, the consistency of the key frame timestamps of the multi-channel video stream data can be ensured, which not only solves the problem of errors in the timestamps of multi-channel video data, but also reduces complex multi-channel video alignment operations, and can respond to playback instructions in a timely manner to achieve the effect of low playback synchronization delay. On this basis, the multi-channel video stream data from different camera devices is uniformly written into the same time slot to form a multi-channel data block, which can reduce the deviation between the data reception time and the generation time from affecting the synchronization accuracy. During subsequent video playback, the data of the corresponding multi-channel data block can be quickly located and played based on the time slot index anchor, which can effectively ensure the synchronization of multi-channel videos, reduce video playback errors, and improve the multi-channel video playback synchronization accuracy.

[0099] In one embodiment, as Figure 3 shown, in step S1, that is, the control device performs clock synchronization calibration on multiple camera devices according to the master clock, which specifically includes the following steps:

[0100] S11: The control device synchronously sends clock synchronization messages to multiple camera devices, and the clock synchronization messages carry the message sending time.

[0101] Before synchronously sending a preset control signal to multiple camera devices, the control device can perform clock synchronization calibration on multiple camera devices to synchronize the time of multiple camera devices with the master clock. Among them, the clock synchronization calibration of multiple camera devices by the control device is triggered by clock synchronization messages, so that each camera device performs clock dynamic compensation after receiving the clock synchronization messages, thereby completing the clock synchronization calibration work.

[0102] Among them, the control device can first synchronously send clock synchronization messages to multiple camera devices. The clock synchronization message carries the message sending time, and is used to instruct multiple camera devices under the control device to perform clock synchronization calibration. Among them, the clock synchronization message can be a synchronization message based on the CAN (Controller Area Network) bus high-level communication protocol, that is, the SYNC message based on the CANopen protocol. The clock synchronization message can be sent through optical fiber.

[0103] S12: The camera device receives the clock synchronization message sent by the control device.

[0104] The camera device receives the clock synchronization message sent by the control device to perform clock dynamic compensation according to the message sending time in the clock synchronization message, that is, calibrate the time of the clock in the camera device based on the message sending time, so that the clock of the camera device is synchronized with the master clock.

[0105] S13: The camera device determines the clock offset of the camera device according to the time difference between the message sending time and the message receiving time in the clock synchronization message.

[0106] Specifically, after receiving the clock synchronization message sent by the control device, each camera device determines the clock offset of the camera device according to the time difference between the message sending time and the message receiving time in the clock synchronization message. Among them, the message receiving time is the time when the camera device receives the clock synchronization message sent by the control device.

[0107] Specifically, after the camera device receives the clock synchronization message sent by the control device, it records the receiving time of the clock synchronization message and determines the time difference between the message sending time and the message receiving time in the clock synchronization message; at the same time, the camera device obtains the pre-calibrated path delay between the camera device and the control device, and the path delay is the time required for signal transmission between the control device and the camera device. Among them, the path delay of the camera device can be pre-stored in the memory of the camera device. Finally, the camera device subtracts the path delay from the time difference between the message sending time and the message receiving time to obtain the clock offset of the camera device, which can improve the accuracy of the clock offset.

[0108] Among them, the clock offset of the camera device can be expressed by the following formula:

[0109] Δt=t1-t0-t p ;

[0110] Among them, Δt represents the clock offset of the camera device; t0 represents the message sending time in the clock synchronization message; t1 represents the message receiving time of the camera device for the clock synchronization message; t pIndicates the path delay between the imaging device and the control device.

[0111] S14: The imaging device determines the clock drift rate of the imaging device based on the clock offset of the imaging device, and determines the weight coefficient for dynamically compensating the clock of the imaging device.

[0112] After determining the clock offset of the imaging device, the imaging device determines the clock drift rate of the imaging device based on the clock offset of the imaging device. At the same time, the imaging device can also determine the weight coefficient for dynamically compensating the clock of the imaging device; the weight coefficient is used to adjust the sensitivity of the clock drift compensation. Among them, the weight coefficient can be determined according to the actual operating parameters of the imaging device, so that the weight system can change dynamically with the operating conditions of different devices, avoiding over-compensation or under-compensation caused by using a fixed weight coefficient for clock calibration.

[0113] S15: The imaging device determines the calibrated target clock time based on the clock offset, clock drift rate, and weight coefficient, and updates the clock time of the imaging device to the calibrated target clock time.

[0114] After determining the clock drift rate of the imaging device and the weight coefficient for dynamic clock compensation, the imaging device determines the calibrated target clock time based on its clock offset, clock drift rate, and weight coefficient for dynamic clock compensation, and updates the clock time of the imaging device to the calibrated target clock time.

[0115] In this embodiment, the clock offset is determined by the message sending time and the receiving time, and then the dynamic compensation of the clock drift is performed by combining the clock offset, the drift rate, and the weight coefficient of the dynamic compensation. It does not rely on a fixed or regularly changing weight, nor does it rely on the operating system scheduling, bypassing the software delay, and the clock calibration effect is more accurate and stable. This solution realizes high-precision time synchronization between multiple imaging devices by introducing a compensation mechanism combining clock offset, drift rate, and dynamic weight, controls the clock deviation between devices within the sub-microsecond level, constructs a unified system clock basis for subsequent key frame collaborative triggering, video frame-level alignment, and cross-device synchronous playback, and improves the synchronization accuracy, stability, and adaptability of the monitoring system.

[0116] In one embodiment, after the imaging device performs dynamic clock compensation according to the message sending time, that is, after completing the clock synchronization calibration, the imaging device sends a clock synchronization success signal to the control device. The control device receives the clock synchronization success signal sent by the imaging device, and when it determines that the clock synchronization calibration of multiple imaging devices is successful according to the clock synchronization success signal, it synchronously sends a preset control signal to multiple imaging devices.

[0117] Among them, after the control device synchronously sends clock synchronization messages to multiple camera devices, if it receives the clock synchronization success signals of all camera devices among the multiple camera devices, it determines that the multiple camera devices are successfully synchronized and calibrated with the master clock, and then can synchronously send preset control signals to the multiple camera devices as needed; if it does not receive the clock synchronization success signal of any camera device among the multiple camera devices, it determines that the multiple camera devices fail to be synchronized and calibrated with the master clock, and does not send preset control signals to the multiple camera devices, but adopts the original camera device data storage and playback method to ensure the normal execution of subsequent data storage and playback.

[0118] In other embodiments, after receiving the clock synchronization message sent by the control device, the camera device sends a response notification of the clock synchronization message to the control device; the control device receives the response notification of the clock synchronization message sent by the camera device. If it receives the response notifications of all camera devices within the first time period, it determines that the multiple camera devices are successfully synchronized and calibrated with the master clock, and then can synchronously send preset control signals to the multiple camera devices as needed. If it does not receive the response notification of any camera device among the multiple camera devices within the first time period, it continues to send the clock synchronization message to it; if it receives the response notifications of all camera devices within the second time period (greater than the first time period), it determines that the multiple camera devices are successfully synchronized and calibrated with the master clock, and then can synchronously send preset control signals to the multiple camera devices as needed. If it still does not receive the response notifications of all camera devices within the second time period, it determines that the multiple camera devices fail to be synchronized and calibrated with the master clock, and does not send preset control signals to the multiple camera devices, but adopts the original camera device data storage and playback method to ensure the normal execution of subsequent data storage and playback.

[0119] In one embodiment, in step S13, that is, the camera device determines the weight coefficient for clock dynamic compensation of the camera device, which specifically includes the following steps:

[0120] S131: The camera device obtains the historical clock drift amount of the camera device, as well as the actual operating parameters of the camera device. The actual operating parameters include network jitter and device temperature.

[0121] After receiving the clock synchronization message sent by the control device, the camera device can obtain the historical clock drift amount of the camera device, as well as the actual operating parameters of the camera device. Among them, the actual operating parameters include the current network jitter and device temperature of the camera device. The current network jitter of the camera device is obtained by calculating the variance of the reception delay time of the clock synchronization message. The device temperature can affect the crystal oscillator drift rate of the camera device, causing a frequency deviation in the camera device and resulting in a clock offset of the device clock.

[0122] Among them, the historical clock drift amount can be the clock offset when the imaging device last performed clock synchronization calibration. In other embodiments, the historical clock drift amount can also be the clock offset data obtained by performing a sliding window statistics on multiple historical clock offsets of the imaging device, which can reduce the data error caused by using a single historical clock offset and improve the accuracy of the historical clock drift amount.

[0123] S132: The imaging device performs a weighted sum of the historical clock drift amount, network jitter, and device temperature of the imaging device to obtain a weight coefficient for dynamically compensating the clock of the imaging device.

[0124] After determining the clock offset of the imaging device, the imaging device obtains pre-calibrated weight data, which includes the weights of the historical clock drift amount, network jitter, and device temperature. This weight data is obtained by performing statistical analysis on the measured data of the imaging device. Then, the imaging device performs a weighted sum of the historical clock drift amount, network jitter, and device temperature of the imaging device to obtain a weight coefficient for dynamically compensating the clock of the imaging device.

[0125] Among them, the weight coefficient for dynamically compensating the clock of the imaging device can be expressed by the following formula:

[0126] α = k1 * f net + k2 * g t + k3 * h d ;

[0127] Among them, α represents the weight coefficient of the current clock synchronization calibration of the imaging device, that is, the weight coefficient for dynamically compensating the clock of the imaging device this time; f net represents the network jitter of the imaging device; g t represents the device temperature of the imaging device; h d represents the historical drift amount of the imaging device; k1, k2, and k3 respectively represent the weights of network jitter, device temperature, and historical clock drift amount; k1, k2, and k3 are all pre-calibrated constants.

[0128] In this embodiment, by analyzing data such as the network jitter and device temperature of the device to determine the clock drift trend and determining the weighted coefficient to control the sensitivity of clock compensation, it is possible to achieve the adaptive adjustment of the device to the influences of temperature drift, crystal oscillator aging, network jitter, etc., reduce the device clock offset caused by frequency deviation due to factors such as device crystal oscillation, temperature, and aging, and further reduce the video frame misalignment or synchronization failure caused by clock drift, ensuring the accuracy and stability of clock calibration.

[0129] In other embodiments, the weight coefficient for clock dynamic compensation of the imaging device can also be determined by other means. For example, determining the weight coefficient for clock dynamic compensation of the imaging device may further include the following steps:

[0130] S1301: Obtain the historical weight coefficient of the imaging device, and this historical weight coefficient can be the weight coefficient determined during the last clock synchronization calibration of the imaging device;

[0131] S1302: Based on the historical weight coefficient of the imaging device, use a deep learning algorithm to predict the weight coefficient for the current clock dynamic compensation of the imaging device, and obtain the predicted weight for the current time of the imaging device;

[0132] S1303: According to the predicted weight of the imaging device and its historical weight coefficient, use the exponential weighted moving average algorithm to perform dynamic update of the weight coefficient, and obtain the weight coefficient for the current clock dynamic compensation of the imaging device.

[0133] Among them, the weight coefficient for clock dynamic compensation of the imaging device can be expressed by the following formula:

[0134] α n =β * α n-1 +(1 - β) * α p ;

[0135] Among them, α n represents the weight coefficient for the current clock synchronization calibration of the imaging device; α n-1 represents the historical weight coefficient of the imaging device, that is, the weight coefficient determined during the last clock synchronization calibration of the imaging device; α p represents the predicted weight for the current time of the imaging device, that is, the weight value predicted by the deep learning model based on the historical weight coefficient of the imaging device; β is the weight value of the historical weight coefficient, and this value is a pre-calibrated constant.

[0136] According to the above formula, it can be known that α n represents the weighted sum of the historical weight coefficient and the predicted weight of the imaging device, and β can control the importance of the historical weight coefficient. When β is close to 1, α n is more dependent on the historical weight coefficient; when β is close to 0, α n is more dependent on the predicted weight. Therefore, the value of β can be calibrated according to actual needs to flexibly adjust the n value of α.

[0137] In this embodiment, by introducing a deep learning model to predict the weight coefficient for clock dynamic compensation, and then using the exponential weighted moving average method, the weight coefficient for this clock dynamic compensation is determined based on the predicted weight and the historical weight coefficient. It can automatically identify the drift law and fluctuation trend of the device according to the device historical data, realize the adaptive adjustment of the compensation weight varying with the device and the scenario, and combine the exponential weighted moving average update mechanism to enable the smooth update of the weight data, reduce the impact of a single anomaly on the compensation result, and ensure the accuracy of the weight coefficient.

[0138] In one embodiment, in step S14, that is, the imaging device determines the calibrated target clock time according to the clock offset, clock drift rate, and weight coefficient, which specifically includes the following steps:

[0139] S141: The imaging device determines the calibration interval duration between this clock synchronization calibration and the previous one.

[0140] After receiving the clock synchronization message sent by the control device, the imaging device can obtain the time of the previous clock synchronization calibration of this imaging device. This time can be the reception time of the clock synchronization message during the previous clock synchronization calibration, so as to determine the calibration interval duration between this clock synchronization calibration and the previous one based on the reception time of this clock synchronization message and the time of the previous clock synchronization calibration.

[0141] S142: The imaging device takes the product of the calibration interval duration, clock drift rate, and weight coefficient as the clock dynamic drift amount.

[0142] After determining the clock offset and weight coefficient of this imaging device for this time, as well as the clock drift rate, the imaging device takes the product of the calibration interval duration, the clock drift rate for this time, and the weight coefficient for clock dynamic compensation as the clock dynamic drift amount during this clock synchronization calibration of this imaging device.

[0143] S143: The imaging device takes the sum of the clock time of the imaging device before calibration, the clock dynamic drift amount, and the clock offset as the calibrated target clock time.

[0144] After determining the clock dynamic drift amount during this clock synchronization calibration of this imaging device, the imaging device takes the sum of the clock time of the imaging device before calibration (i.e., the current clock time of the imaging device), the clock dynamic drift amount, and the clock offset as the calibrated target clock time, so as to set the clock time of this imaging device to the calibrated target clock time and complete the clock synchronization calibration.

[0145] Among them, the calibrated target clock time can be expressed as follows:

[0146] Tnew = T raw + Δt + α * Draf_rate * (t1 - t n-1 );

[0147] Wherein, T new represents the calibrated target clock time; T raw represents the clock time before calibration; Δt represents the clock offset of the imaging device; α represents the weight coefficient for dynamically compensating the clock of the imaging device; Draf_rate represents the clock drift rate of the imaging device; t1 represents the time when the imaging device performs clock synchronization calibration this time, such as the reception time of the clock synchronization instruction when performing clock synchronization calibration this time; t n-1 represents the time when the imaging device last performed clock synchronization calibration, such as the reception time of the clock synchronization instruction when performing clock synchronization calibration last time; t1 - t n-1 represents the calibration interval duration of the imaging device; α * Draf_rate * (t1 - t n-1 ) represents the dynamic clock drift amount of the imaging device.

[0148] In this embodiment, the imaging device realizes the quantization and cumulative compensation of the clock dynamic offset by comprehensively considering the calibration interval duration, the clock drift rate and the weight coefficient, enabling the device clock calibration process to have the dynamic response ability of historical evolution, and improving the continuity, stability and accuracy of time synchronization among multiple devices.

[0149] In one embodiment, as Figure 4 shown, in step S3, that is, the imaging device responds to the preset control signal to perform the key frame generation operation, so that each imaging device that receives the preset control signal synchronously generates key frames and encodes to generate video stream data, specifically including the following steps:

[0150] S31: The imaging device responds to the preset control signal to perform the key frame generation operation, so that each imaging device that receives the preset control signal synchronously generates key frames and encodes to obtain a picture group starting with the key frame.

[0151] After receiving the preset control signal sent by the control device, the imaging device responds to the preset control signal to perform the key frame generation operation, so that each imaging device that receives the preset control signal synchronously generates key frames and encodes to obtain a picture group starting with the key frame.

[0152] S32: The imaging device injects the frame rate of the picture group, the time stamp of the starting frame and the frame type into the picture group as supplementary enhancement information of the picture group to generate video stream data.

[0153] After encoding a group of pictures (GOP) with a key frame as the starting frame, the imaging device injects the frame rate of the GOP, the time stamp of the starting frame, and the frame type (i.e., the generation time stamp and frame type of the key frame) into the GOP as supplementary enhancement information of the GOP, that is, performs SEI injection (Supplemental Enhancement Information Injection) on the encoded GOP group to generate video stream data. The video stream data includes one or more GOP groups and the supplementary enhancement information of the GOP group, that is, the SEI information of the GOP group.

[0154] Among them, the frame type is used to indicate whether the starting frame of the video stream data is a real key frame, and this frame type is used to mark the starting frame in this GOP as a key frame forcibly generated by the imaging device in response to a preset control signal.

[0155] Among them, the frame rate of the GOP (i.e., the GOP structure information) is the encoded data structure information when the GOP is generated in the imaging device. The frame rate of this GOP can be the frame rate of the GOP, which can represent the frame interval of the GOP, that is, represent the frame length information of the GOP.

[0156] In this embodiment, after multiple imaging devices generate key frames in response to a synchronized preset control signal, the generation time stamp, GOP structure information, and frame type are embedded in the GOP, which facilitates data decoding according to the supplementary enhancement information when playing back video data later, solves the problems of inconsistent GOP lengths among multiple devices, misalignment of key frame pairs, and asynchronization of video frame encoding time stamps themselves, and improves the accuracy, reliability, and system versatility of multi-channel video synchronous playback.

[0157] In one embodiment, the video stream data includes a GOP with a generated key frame as the starting frame and the supplementary enhancement information of the GOP. As Figure 5 shown, in step S6, that is, the control device stores multi-channel video stream data with the same time stamp from multiple imaging devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block, which specifically includes the following steps:

[0158] S61: The control device stores the GOPs in the multi-channel video stream data with the same time stamp from multiple imaging devices into the same time slot in sequence according to the device identifiers of each imaging device to obtain a multi-channel data block.

[0159] After the control device sends a preset control signal to multiple imaging devices to make the imaging devices that receive the preset control signal perform key frame generation operations in response to the preset control signal to encode and generate video stream data, the imaging devices send the video stream data to the control device.

[0160] The control device receives the video stream data sent by the camera device, and stores the picture groups in the multiple video stream data with the same timestamp from multiple camera devices into the same time slot with a preset time length in sequence according to the device identifiers of each camera device, so as to obtain a multi-channel data block.

[0161] S62: The control device generates index information of the multi-channel data block according to the supplementary enhancement information of each picture group, and stores it into the time slot.

[0162] After storing the picture groups in the multiple video stream data into the same time slot in sequence to obtain a multi-channel data block, the control device can also generate index information of the multi-channel data block according to the frame rate, the timestamp of the start frame, and the frame type of the picture group in the supplementary enhancement information of each picture group, and store it into the time slot.

[0163] Among them, the index information includes the global timestamp of the multi-channel data block, the picture group structure information of each camera device, and the synchronization status information. The synchronization status information is used to indicate whether the frame type of the start frame of the picture group is a true key frame. The global timestamp is determined according to the timestamp of the start frame in the video stream data and the preset time length of the time slot. In other embodiments, the index information may include the start timestamp (i.e., the timestamp of the start frame) of the multi-channel data block, the picture group structure information of each camera device, and the synchronization status information.

[0164] Among them, due to various reasons, it may occur that the timestamps of the start frames of the video stream data of different camera devices cannot be aligned. When the control device stores the video stream data of different camera devices, since the length of the time slot is fixed, the control device can perform video frame interception on the video stream data from some camera devices so that the timestamps of the start frames and end frames of the multiple video stream data are the same, so as to store the picture groups in the multiple video stream data with the same timestamp from multiple camera devices into the time slot. At this time, the start frame of the picture group in the multiple video stream data stored in the time slot may not be a key frame generated in response to a preset control signal. Therefore, when the control device stores each video stream data in the time slot, it can mark the frame type of the start frame of the video stream data, that is, mark the frame type of the start frame of each video stream data. Then, the control device generates index information of the multi-channel data block according to the frame rate, the timestamp of the start frame, and the frame type of the picture group in the multiple video stream data, so as to perform decoding based on the index information subsequently to play back the corresponding video data.

[0165] Among them, the physical structure of the time slot is as follows:

[0166] Time slot: Global timestamp XXX

[0167] ├── Picture group in the video stream data of camera device 1 (frame type mark of the start frame)

[0168] ├── Picture group in the video stream data of camera device 2 (frame type marker of the starting frame)

[0169] ├── Picture group in the video stream data of camera device 3 (frame type marker of the starting frame)

[0170] └── Index information

[0171] {Starting timestamp: XXX, Synchronization status information: XXX, GOP structure information: XXX}

[0172] Among them, taking a time slot with a preset time length of 1 ms as an example, Channel1, Channel2, and Channel3 respectively represent the device identifiers of camera device 1, camera device 2, and camera device 3; the GOP structure information is the frame rate of the GOP group, and the GOP group frame rates of camera device 1, camera device 2, and camera device 3 can be 30, 24, and 15 respectively. The frame types of the starting frames of camera device 1, camera device 2, and camera device 3 are true I-frame (true), true I-frame, and virtual I-frame (false) respectively; the index information is stored in the time slot in json format and is represented by metadata.json.

[0173] Then, the structure and stored data content of the time slot can be as follows:

[0174] Time slot: 12:00:00.000 - 12:00:00.001

[0175] ├── Channel1.h264 (true I-frame)

[0176] ├── Channel2.h265 (true I-frame))

[0177] ├── Channel3.av1 (virtual I-frame)

[0178] └── metadata.json

[0179] {"global_timestamp": "12:00:00.000",

[0180] "channel_sync_status": {"1": true, "2": true, "3": false},

[0181] "gop_structure": {"1": 30, "2": 24, "3": 15}}

[0182] S63: The control device generates and stores a time slot index anchor pointing to the multi-channel data block according to the global timestamp in the index information of the multi-channel data block.

[0183] After storing the picture groups in the multi-channel video stream data with the same timestamp from multiple camera devices into time slots to obtain multi-channel data blocks and generating the index information of the multi-channel data blocks, the control device generates and stores a time slot index anchor pointing to the multi-channel data block according to the global timestamp in the index information of the multi-channel data block.

[0184] In this embodiment, when storing the multi-channel video stream data, the control device can store the picture groups in the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot in sequence according to the device identifiers of each camera device to obtain multi-channel data blocks, then generate and store the index information of the multi-channel data blocks according to the supplementary enhancement information of each picture group, and then generate and store a time slot index anchor pointing to the multi-channel data block according to the global timestamp in the index information of the multi-channel data block. By constructing a structured index with the supplementary enhancement information of the picture groups, frame-level synchronous storage, second-level fast retrieval and decoding capabilities are achieved, providing a reliable data basis for high-precision synchronous playback.

[0185] In one embodiment, the control device can synchronously send a preset control signal to multiple camera devices at a preset interval, so that the multiple camera devices respond to the preset control signal and perform key frame generation operations at the preset interval to generate key frames and encode them into video stream data, and send the encoded video stream data to the control device, that is, the sending interval of the video stream data is the preset interval. Among them, the time length of the time slot can be the same as the preset interval, and the preset time length and the preset interval can be 1 ms, so that multiple camera devices can achieve microsecond-level key frame alignment, and the control device can store one video stream data generated by multiple camera devices in response to the preset control signal each time, which is convenient for more fine-grained video frame alignment during subsequent playback and improves the synchronous accuracy of multi-channel video playback.

[0186] In one embodiment, the time slot stores the multi-channel data block and the index information of the multi-channel data block, and the multi-channel data block in the time slot includes one or more picture groups of each camera device among multiple camera devices. As Figure 6 shown, after step S6, that is, after the control device generates and stores the time slot index anchor for querying the multi-channel data block, the following steps are specifically included:

[0187] S7: After receiving a video playback instruction for multiple camera devices, the control device queries the target time slot of the time slot index anchor within the playback timestamp, and extracts the data in the target time slot to obtain the target data block and index information corresponding to the playback timestamp.

[0188] After storing the video stream data of multiple camera devices into the same time slot, or during the process of storing the video stream data of multiple camera devices, the user can send a video playback instruction for multiple camera devices to the control device through the terminal device according to actual needs; the control device receives the video playback instruction for multiple camera devices.

[0189] Among them, the video playback instruction can be a video playback instruction, that is, an instruction to play back the video data collected by multiple camera devices in the past; in other embodiments, the video playback instruction can also be a video preview instruction, that is, an instruction to play the video data collected by multiple camera devices in real time.

[0190] After receiving the video playback instruction for multiple camera devices, the control device queries the target time slots within the playback timestamp where the time slot index anchor points are located. Among them, there are multiple target time slots within the playback timestamp. Then, for each target time slot, the control device extracts the multi-channel data blocks and the index information of the multi-channel data blocks in the target time slot to obtain a target data block and the index information of the target data block; by traversing all the target time slots, multiple target data blocks corresponding to the playback timestamp and the index information of each target data block can be obtained.

[0191] S8: According to the synchronization status information in the index information of the target data block, the control device decodes the target data block to obtain the decoded video data of each camera device.

[0192] Among them, the synchronization status information in the index information is used to indicate whether the starting frames of each camera device are real key frames.

[0193] After obtaining the target data blocks corresponding to the playback timestamp and the index information of the target data blocks, the control device determines whether the starting frames of each camera device in the target data block are real key frames according to the synchronization status information in the index information of the target data block, and decodes the target data block according to the determination result to obtain the decoded video data of each camera device.

[0194] Specifically, decoding the target data block according to the synchronization status information in the index information of the target data block includes:

[0195] S81: When it is determined according to the synchronization status information that the starting frame of the camera device in the target data block is a real key frame, the control device decodes the picture group of the camera device in the target data block.

[0196] For each camera device in each target data block, in the synchronization status information in the index information of the target data block, read the frame type of the starting frame of the camera device, and determine whether the frame type of the starting frame of the camera device indicates a real key frame. If the frame type of the starting frame of the camera device indicates a real key frame, determine that the starting frame of the camera device in the target data block is a real key frame; if the frame type of the starting frame of the camera device indicates a virtual key frame, determine that the starting frame of the camera device in the target data block is a virtual key frame.

[0197] When, according to the frame type of the starting frame of the camera device in the synchronization status information, it is determined that the starting frame of the camera device in the target data block is a real key frame, the control device directly decodes the picture group of the camera device in the target data block to obtain the decoded video data of the camera device.

[0198] S82: When, according to the synchronization status information, it is determined that the starting frame of the camera device in the target data block is a virtual key frame, the control device performs an effective key frame generation operation on the picture group of the camera device in the target data block, and decodes the picture group of the camera device based on the generated effective key frame.

[0199] When, according to the frame type of the starting frame of the camera device in the synchronization status information, it is determined that the starting frame of the camera device in the target data block is a virtual key frame, a virtual key frame generation algorithm is used to perform an effective key frame generation operation on the picture group of the camera device in the target data block to obtain the effective key frame of the camera device, and the picture group of the camera device is decoded based on the generated effective key frame.

[0200] Among them, using the virtual key frame generation algorithm to perform an effective key frame generation operation on the picture group of the camera device in the target data block to obtain the effective key frame of the camera device includes: obtaining the starting frame of the camera device in the target data block (i.e., the current starting frame), and obtaining the real key frame of the camera device at the previous moment (which can be obtained by querying other time slots and decoding according to the time stamp of the current starting frame); performing motion vector extrapolation calculation on the real key frame of the camera device at the previous moment and the current starting frame to obtain motion vector data, and predicting the key frame at the current moment based on the motion vector data and the real key frame at the previous moment to obtain a predicted key frame; calculating the residual data between the predicted key frame and the current starting frame, and adding the predicted key frame and the residual data to generate an effective key frame.

[0201] In this embodiment, during the process of decoding the target data block, the starting frame of the camera device in the target data block is determined as a real key frame according to the synchronization status information in its index information. When the starting frame of the camera device in the target data block is a real key frame, the picture group of the camera device in the target data block is decoded; when the starting frame of the camera device in the target data block is a virtual key frame, an effective key frame generation operation is performed on the picture group of the camera device in the target data block, and the picture group of the camera device is decoded based on the generated effective key frame. By using the synchronization status information in the stored index information, the starting frame of the device is intelligently distinguished as a real key frame and a virtual key frame, and different decoding paths are selected according to the type of the starting frame, improving the accuracy of decoding video data and enhancing the synchronization accuracy of multi-channel video playback.

[0202] S9: The control device performs key frame alignment processing on each decoded video data according to the picture group structure information in the index information to obtain the target video data of each camera device.

[0203] The control device performs key frame alignment processing on each decoded video data according to the picture group structure information in the index information, so that the time stamps of the key frames of each decoded video data are the same, in order to obtain the target video data of each camera device.

[0204] Among them, the target time slot corresponds to a global time stamp. According to the global time stamps of the target time slots corresponding to each target data block, the decoded video data of each camera device obtained by decoding multiple target data blocks are arranged in sequence, and the target video data of each camera device can be obtained.

[0205] S10: The control device controls the player to perform multi-channel video playback based on the target video data of each camera device.

[0206] After obtaining the target video data of each camera device, that is, obtaining the target video data of multiple channels, the control device sends the target video data of each camera device to the player, so that the player plays the video segments in the target video data of each camera device in time stamp order, realizing multi-channel video playback.

[0207] In this embodiment, through the time slot index anchor point mechanism, the target data block can be quickly located without traversing the entire video stream, and millisecond-level positioning and response can be achieved, improving the playback response speed and efficiency. And through the target time slot, the frame type of the starting frame and the GOP structure information, precise synchronization of video data decoding is realized, ensuring that the video frames of multiple devices are aligned on the time axis and improving the synchronization accuracy of multi-channel video playback.

[0208] Such as Figure 7As shown in the figure, the monitoring system includes a control device, a camera device A, a camera device B, a storage cluster, a global clock arbiter, and a player (not shown); the storage cluster includes multiple time slots, a metadata database, and a decoder. The global clock arbiter assigns a master clock source to the control device in the monitoring system; the control device performs clock synchronization calibration on the camera device A and the camera device B according to the assigned master clock, and after the clock synchronization calibration, the control device synchronously sends a preset control signal to the camera device A and the camera device B. After receiving the preset control signal, the camera device A and the camera device B respectively perform key frame generation operations, encode the generated key frames and inject SEI information to obtain video stream data, and send the video stream data to the control device. The control device stores the video stream data with the same time stamps from the camera device A and the camera device B into the same time slot in the storage cluster to obtain a multi-channel data block, generates index information according to the SEI information in the video stream data, and synchronously stores it in the time slot. At the same time, the control device generates a time slot index anchor pointing to the multi-channel data block according to the index information, and stores the time slot index anchor in the metadata database for subsequent multi-channel data query based on the time slot index anchor.

[0209] Among them, after receiving a video playback instruction for the multi-channel video of the camera device A and the camera device B, the control device queries the target data block of the corresponding time slot based on the time slot index anchor, and inputs the encoded data (i.e., picture group) of each camera device in the target data block into the decoder for decoding. Before inputting the target data block into the decoder, the control device can identify whether the start frames of the camera device A and the camera device B are virtual key frames according to the frame types of the start frames of the camera device A and the camera device B in the target data block. When it is determined that the start frame of the camera device is a virtual key frame, first generate an effective key frame for the start frame of the camera device to obtain an effective key frame, and input the effective key frame and the encoded data of the camera device into the decoder for decoding, so as to obtain the decoded data of the camera device. Then, perform frame alignment processing on the decoded data of the camera device A and the camera device B according to the GOP group structure information, so as to obtain the target video data of the camera device A and the target video data of the camera device B, and control the player to play, which can effectively ensure the synchronization of the multi-channel video, reduce the video playback error, improve the synchronization accuracy of the multi-channel video playback, and achieve the effect of low synchronization playback delay.

[0210] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0211] In one embodiment, a camera device is provided, and the camera device corresponds one-to-one to the data processing method in the above embodiment. As Figure 8As shown, the imaging device includes a receiving unit 801, a generating unit 802, and a transmitting unit 803. The detailed descriptions of each functional unit are as follows:

[0212] The receiving unit 801 is configured to receive a preset control signal sent by a control device, where the preset control signal is a control signal that the control device synchronously sends to multiple imaging devices after clock synchronization calibration of the multiple imaging devices according to the master clock;

[0213] The generating unit 802 is configured to perform a key frame generation operation in response to the preset control signal, so that each imaging device that receives the preset control signal synchronously generates a key frame and encodes to generate video stream data carrying a key frame generation timestamp;

[0214] The transmitting unit 803 is configured to send the video stream data starting with the key frame to the control device, so that the control device stores the multi-channel video stream data with the same timestamp from multiple imaging devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block.

[0215] In an embodiment, the receiving unit 801 is further configured to receive a clock synchronization message sent by the control device, where the clock synchronization message is used to instruct clock synchronization calibration of multiple imaging devices under the control device;

[0216] Wherein, the imaging device further includes a calibration unit, and the calibration unit is configured to: determine the clock offset of the imaging device according to the time difference between the message sending time and the message receiving time in the clock synchronization message, where the message receiving time is the time when the imaging device receives the clock synchronization message; determine the clock drift rate of the imaging device according to the clock offset of the imaging device, and determine the weight coefficient for clock dynamic compensation of the imaging device; determine the calibrated target clock time according to the clock offset, the clock drift rate, and the weight coefficient, and update the clock time of the imaging device to the calibrated target clock time.

[0217] In an embodiment, a control device is provided, and the control device corresponds one-to-one to the data processing method in the above embodiment. As Figure 9 shown, the control device includes a transmitting module 901, a receiving module 902, and a storage module 903. The detailed descriptions of each functional unit are as follows:

[0218] The transmitting module 901 is configured to synchronously send a preset control signal to multiple imaging devices after clock synchronization calibration of the multiple imaging devices according to the master clock, where the preset control signal is used to instruct each imaging device to perform a key frame generation operation in response to the preset control signal, so that each imaging device that receives the preset control signal synchronously generates a key frame and encodes to generate video stream data carrying a key frame generation timestamp;

[0219] A receiving module 902, configured to receive video stream data sent by a camera device, where a key frame generated by the camera device in response to a preset control signal is the starting frame of the video stream data;

[0220] A storage module 903, configured to store multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generate and store a time slot index anchor point for querying the multi-channel data block.

[0221] In an embodiment, the time slot further stores index information of the multi-channel data block; the control device further includes a processing module, and the processing module is configured to: after receiving a video playback instruction for multiple camera devices, query a target time slot of the time slot index anchor point within the playback timestamp, and extract data in the target time slot to obtain a target data block corresponding to the playback timestamp and index information of the target data block; decode the target data block according to the synchronization status information in the index information of the target data block to obtain decoded video data of each camera device, where the synchronization status information is used to indicate whether the starting frame of each camera device is a true key frame; perform key frame alignment processing on each decoded video data according to the picture group structure information in the index information to obtain target video data of each camera device; and control a player to perform multi-channel video playback based on the target video data of each camera device.

[0222] It should be noted that for the information interaction, execution process, etc. between the above-mentioned device / units, since they are based on the same concept as the method embodiment of the present application, their specific functions and the technical effects brought thereby can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0223] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0224] An embodiment of the present application further provides an electronic device, and the electronic device can be a camera device or a control device. For example Figure 10As shown, the electronic device 10 includes: at least one processor 101, a memory 102, and a computer program 103 stored in the memory 102 and executable on the at least one processor 101. When the processor 101 executes the computer program 103, it implements the steps in any of the above method embodiments, or when the processor 101 executes the computer program 103, it implements the functions of each module / unit in the above device embodiments. Exemplarily, the computer program 103 may be divided into one or more modules / units. The one or more modules / units are stored in the memory 102 and executed by the processor 101 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 103 in the electronic device 10.

[0225] Those skilled in the art can understand that Figure 10 merely examples of the electronic device, and do not constitute a limitation on the electronic device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may further include input / output devices, network access devices, buses, etc.

[0226] The above processor may be a central processing unit, or may also be other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc. The memory may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device, such as a plug-in hard disk, smart memory card, secure digital, flash card, etc. equipped on the electronic device. Further, the memory may also include both the internal storage unit and the external storage device of the electronic device.

[0227] The embodiment of this application also provides a readable storage medium storing a computer program, and when the computer program is executed by a processor, it can implement the steps in any of the above method embodiments.

[0228] The embodiment of this application provides a computer program product. When the computer program product runs on an electronic device, it enables the electronic device to implement the steps in any of the above method embodiments when executed.

[0229] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0230] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0231] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0232] In the embodiments provided in this application, it should be understood that the disclosed device / equipment and method can be implemented in other ways. For example, the device / equipment embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0233] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that, Applied to a camera device, the method includes: Receiving a preset control signal sent by a control device, where the preset control signal is a control signal synchronously sent by the control device to multiple camera devices after clock synchronization calibration of the multiple camera devices according to a master clock; Performing a key frame generation operation in response to the preset control signal, so that each of the camera devices receiving the preset control signal synchronously generates a key frame and encodes to generate video stream data carrying the key frame generation timestamp; Sending the video stream data starting from the key frame to the control device, so that the control device stores the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block.

2. The method according to claim 1, wherein The clock synchronization calibration of multiple camera devices according to the master clock includes: Receiving a clock synchronization message sent by the control device, where the clock synchronization message is used to instruct clock synchronization calibration of multiple camera devices under the control device; Determining the clock offset of the camera device according to the time difference between the message sending time and the message receiving time in the clock synchronization message, where the message receiving time is the time when the camera device receives the clock synchronization message; Determining the clock drift rate of the camera device according to the clock offset of the camera device, and determining a weight coefficient for clock dynamic compensation of the camera device; Determining the calibrated target clock time according to the clock offset, the clock drift rate, and the weight coefficient, and updating the clock time of the camera device to the calibrated target clock time.

3. The method according to claim 2, characterized in that The determining the calibrated target clock time according to the clock offset, the clock drift rate, and the weight coefficient includes: Determining the calibration interval duration between the current clock synchronization calibration and the previous clock synchronization calibration; Taking the product of the calibration interval duration, the clock drift rate, and the weight coefficient as the clock dynamic drift amount; Taking the sum of the clock time of the camera device before calibration, the clock dynamic drift amount, and the clock offset as the calibrated target clock time.

4. The method according to claim 2, wherein The determining the weight coefficient for clock dynamic compensation of the camera device includes: Obtaining the historical clock drift amount of the camera device and the actual operating parameters of the camera device, where the actual operating parameters include network jitter and device temperature; Performing weighted summation on the historical clock drift amount, the network jitter, and the device temperature of the camera device to obtain a weight coefficient for clock dynamic compensation of the camera device.

5. The method according to any one of claims 1-4, characterized in that, The performing a key frame generation operation in response to the preset control signal, so that each of the camera devices receiving the preset control signal synchronously generates a key frame and encodes to generate video stream data carrying the key frame generation timestamp includes: Performing a key frame generation operation in response to the preset control signal, so that each of the camera devices receiving the preset control signal synchronously generates a key frame and encodes to obtain a group of pictures starting from the key frame; Inject the frame rate of the picture group, the timestamp of the starting frame, and the frame type into the picture group as supplementary enhancement information of the picture group to generate the video stream data, where the frame type is used to indicate whether the starting frame is a true key frame.

6. A data processing method, characterized in that, Applied to a control device connected to multiple camera devices, the method includes: After clock synchronization calibration of multiple camera devices according to the master clock, send a preset control signal to the multiple camera devices synchronously, where the preset control signal is used to instruct each camera device to respond to the preset control signal to perform a key frame generation operation, so that each camera device that receives the preset control signal synchronously generates a key frame and encodes to generate video stream data carrying the key frame generation timestamp; Receive the video stream data sent by the camera device, where the key frame generated by the camera device in response to the preset control signal is the starting frame of the video stream data; Store the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generate and store a time slot index anchor for querying the multi-channel data block.

7. The method according to claim 6, wherein The video stream data includes a picture group starting with the generated key frame and supplementary enhancement information of the picture group. The storing the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generating and storing a time slot index anchor for querying the multi-channel data block includes: Store the picture groups in the multi-channel video stream data with the same timestamp from multiple camera devices into the same time slot in sequence according to the device identifier of each camera device to obtain the multi-channel data block; Generate index information of the multi-channel data block according to the supplementary enhancement information of each picture group and store it in the time slot. The index information includes the global timestamp of the multi-channel data block, the picture group structure information and synchronization status information of each camera device, and the synchronization status information is used to indicate whether the starting frame of each camera device is a true key frame; Generate a time slot index anchor pointing to the multi-channel data block according to the global timestamp in the index information of the multi-channel data block and store it.

8. The method according to claim 6, wherein The time slot also stores the index information of the multi-channel data block. After generating and storing the time slot index anchor for querying the multi-channel data block, the method further includes: After receiving a video playback instruction for multiple camera devices, query the target time slot of the time slot index anchor within the playback timestamp, and extract the data in the target time slot to obtain the target data block corresponding to the playback timestamp and the index information of the target data block; Decode the target data block according to the synchronization status information in the index information of the target data block to obtain the decoded video data of each camera device, where the synchronization status information is used to indicate whether the starting frame of each camera device is a true key frame; Perform key-frame alignment processing on each piece of the decoded video data according to the group-of-pictures structure information in the index information to obtain the target video data of each camera device; Based on the target video data of each camera device, control a player to perform multi-channel video playback.

9. The method according to claim 8, wherein The decoding of the target data block according to the synchronization status information in the index information of the target data block includes: When it is determined according to the synchronization status information that the starting frame of the camera device in the target data block is a true key frame, decode the group of pictures of the camera device in the target data block; When it is determined according to the synchronization status information that the starting frame of the camera device in the target data block is a virtual key frame, perform an effective key-frame generation operation on the group of pictures of the camera device in the target data block, and decode the group of pictures of the camera device based on the generated effective key frame.

10. A camera device, characterized in that, Including: A receiving unit, configured to receive a preset control signal sent by a control device, where the preset control signal is a control signal that the control device synchronously sends to multiple camera devices after performing clock synchronization calibration on the multiple camera devices according to a master clock; A generating unit, configured to perform a key-frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates a key frame, and encodes to generate video stream data carrying the key-frame generation timestamp; A sending unit, configured to send the video stream data starting from the key frame to the control device, so that the control device stores the multiple video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generates and stores a time slot index anchor point for querying the multi-channel data block.

11. A control device, characterized in that, Including: A sending module, configured to synchronously send a preset control signal to multiple camera devices after performing clock synchronization calibration on the multiple camera devices according to a master clock, where the preset control signal is used to instruct each camera device to perform a key-frame generation operation in response to the preset control signal, so that each camera device that receives the preset control signal synchronously generates a key frame, and encodes to generate video stream data carrying the key-frame generation timestamp; Receive the video stream data sent by the camera device, where the key frame generated by the camera device in response to the preset control signal is the starting frame of the video stream data; Store the multiple video stream data with the same timestamp from multiple camera devices into the same time slot to obtain a multi-channel data block, and generate and store a time slot index anchor point for querying the multi-channel data block.

12. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

13. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

14. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

Images