A method for synchronously playing multi-channel videos, terminal equipment and storage medium

Abstract

The present invention relates to the field of video surveillance technology, and in particular to a method for synchronous playback of multi-channel videos, a terminal device, and a computer-readable storage medium, which are applied to a video surveillance system, and adopt a synchronization algorithm and a timestamp technology to determine a reference channel by finding the key frame closest to the playback time point and the corresponding channel. The reference channel can be used as a reference for other channels to ensure that the video playback time of each channel is synchronized. A download object is created according to the key frame timestamp determined by the reference channel, and the download of the video file is realized according to the download object. Then, through precise control of the analysis, buffering, and decoding of the downloaded video data, the synchronous playback of multi-channel videos can be realized, which effectively avoids the phenomenon of asynchronous multi-channel video playback or delay between channels that is common in traditional methods, and provides a good product experience.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a synchronous playback method of multi-channel videos, terminal equipment and a computer-readable storage medium. Background Art

[0002] As we all know, in the security monitoring industry, video monitoring systems can use multi-channel technology to simultaneously download and play video streams from multiple cameras to ensure the real-time and synchronization of video playback at each monitoring point, thereby improving the efficiency and accuracy of monitoring. Currently, the real-time and synchronization of video playback at each monitoring point mainly rely on the synchronization of multiple videos on the device side. The device side writes the video data generated by different channels into the same video file. When the client plays the video data, the device side sends the video data to the client in the order of writing for multi-channel video synchronization preview. However, since there are inevitably intervals between the key frames of video data generated by different channels, the client will always be out of sync when playing multi-channel videos. It can be seen that designing a method that can synchronously play multi-channel videos is an urgent problem to be solved in the current video surveillance industry. Summary of the invention

[0003] In view of this, the present invention proposes a method for synchronously playing multi-channel videos, a terminal device and a computer-readable storage medium, aiming to solve the problem that the current video surveillance system always has a synchronization problem when playing multi-channel videos.

[0004] In order to solve the above problems, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for synchronously playing multi-channel videos, which is applied to a video surveillance system, comprising:

[0006] Obtaining a playback time point for synchronous playback of multi-channel videos; traversing the key frames and corresponding timestamps of the video files of all channels, finding the key frame and the corresponding channel closest to the playback time point, and determining the found channel as the reference channel;

[0007] Searching for a video file closest to the timestamp corresponding to the found key frame in all channels, creating a download object that starts downloading from the searched video file, and downloading the video file according to the download object;

[0008] According to the preset cache strategy, control the download progress and cache size of the video file to ensure that there is enough video data for parsing;

[0009] Parse the downloaded video file and establish a video buffer queue according to a preset buffer strategy; decode the video data of the video buffer queue to synchronously play multi-channel videos.

[0010] Furthermore, obtaining the playback time point of the multi-channel video synchronous playback includes:

[0011] A request for synchronous playback of multi-channel videos is received, and a playback time point corresponding to the request is obtained.

[0012] Furthermore, the key frames and corresponding timestamps of the video files of all channels traversed include:

[0013] The video file list of all channels on the day corresponding to the playback time point is queried, and for each video file in the video file list, the key frames and corresponding timestamps therein are traversed.

[0014] Furthermore, after controlling the download progress and cache size of the video file and before parsing the downloaded video file, the method further includes:

[0015] When the download of the current video file of the reference channel is completed, switching to the next video file for downloading according to the video file list;

[0016] Adding a preset interval duration to the end timestamp of the current video file to obtain a switching timestamp for switching video file download;

[0017] Traversing the start time and end time of the video files of all channels to find a channel that meets a preset condition; wherein the preset condition is that the switching timestamp is between the start time and end time of a video file of a channel;

[0018] If the channel is found, the found channel is set as a new reference channel;

[0019] If the channel is not found, then find the channel of the video file corresponding to the start time after the switching timestamp and closest to the switching timestamp among all channels, and set this channel as a new reference channel.

[0020] Furthermore, the preset cache strategy includes:

[0021] Cache the downloaded video data, and determine whether the size of the currently downloaded video data is greater than or equal to the preset cache data size;

[0022] If yes, the downloaded video data is put into the parsing queue; if no, the downloaded video data continues to be cached.

[0023] Further, placing the downloaded video data into a parsing queue includes:

[0024] Determine whether the parsing queue is full; if so, wait until the video data in the parsing queue is consumed, and then put the downloaded video data into the parsing queue; if not, put the downloaded video data into the parsing queue.

[0025] Further, the preset buffer strategy includes at least one of a first strategy, a second strategy and a third strategy;

[0026] The first strategy includes:

[0027] Obtain the parsed video data and determine whether the data of each frame is the reference channel data;

[0028] If yes, then update the timestamp of the current video frame to the reference timestamp, and determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames. If yes, suspend downloading and parsing the video data. If no, insert the video frame data into the video buffer queue.

[0029] If not, then obtain the difference between the timestamp of the video frame data of the current channel and the reference timestamp, if the difference is greater than the preset positive difference, suspend downloading and parsing the video data; if the difference is less than the preset negative difference, discard the video frame; if the difference is between the preset negative difference and the preset positive difference, then determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames, if so, suspend downloading and parsing the video data, if not, put the video frame data into the video buffer queue;

[0030] The second strategy includes:

[0031] Reading data from the video buffer queue of the current channel, and when the video frame data is read, determining whether the video frame data is reference channel data;

[0032] If so, the timestamp of the current video frame is updated to the reference timestamp, and all channels are traversed to obtain the timestamp of the current video frame of the channel, and the difference between the timestamp and the reference timestamp is calculated. If the difference is between the preset negative difference and the preset positive difference, the suspended downloading and parsing of video data of this channel is resumed;

[0033] The third strategy includes:

[0034] It is determined whether the number of remaining video frames in the current video buffer queue is less than the preset minimum number of frames. If so, the suspended downloading and parsing of video data is resumed, and the downloaded and parsed video frame data is inserted into the video buffer queue.

[0035] Further, the decoding of the video data of the video buffer queue so as to synchronously play the multi-channel video includes:

[0036] Obtain video frame data from the video buffer queue, and decode the video frame data according to the data type to obtain YUV data;

[0037] The YUV data is rendered using opengl to enable synchronous playback of multi-channel videos.

[0038] In a second aspect, the present invention further provides a terminal device, comprising:

[0039] a memory for storing program instructions; and

[0040] The processor is used to execute the program instructions to implement the steps of the method for synchronously playing multi-channel videos as described above.

[0041] In a third aspect, the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method for synchronous playback of multi-channel videos as described above are implemented.

[0042] Compared with the prior art, the beneficial effects of the present invention are: a method for synchronously playing multi-channel videos, a terminal device and a computer-readable storage medium, applied to a video surveillance system, including: obtaining a playback time point for synchronously playing multi-channel videos; traversing the key frames and corresponding timestamps of the video files of all channels, finding the key frames and corresponding channels closest to the playback time point, and determining the found channel as a reference channel; querying the video file closest to the timestamp corresponding to the found key frame in all channels, creating a download object that starts downloading from the queried video file, and downloading the video file according to the download object; according to a preset cache strategy, controlling the download progress and cache size of the video file to ensure that there is enough video data for parsing; parsing the downloaded video file, and establishing a video buffer queue according to a preset buffer strategy; decoding the video data of the video buffer queue, so as to synchronously play the multi-channel video. It can be seen that the method for synchronously playing multi-channel videos, the terminal device and the computer-readable storage medium adopt a synchronization algorithm and a timestamp technology to determine the reference channel by finding the key frame and the corresponding channel closest to the playback time point. The reference channel can be used as a reference for other channels to ensure that the video playback time of each channel is synchronized. A download object is created based on the key frame timestamp determined by the reference channel, and the video file is downloaded based on the download object. After precise control of the parsing, buffering and decoding of the downloaded video data, synchronous playback of multi-channel videos can be achieved, effectively avoiding the phenomenon of multi-channel video playback asynchrony or inter-channel delay that is common in traditional methods, and providing a good product experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0044] Figure 1 It is a flowchart of a first embodiment of a method for synchronously playing multi-channel videos of the present invention;

[0045] Figure 2 It is a flowchart of a second embodiment of a method for synchronously playing multi-channel videos of the present invention;

[0046] Figure 3 It is a schematic flow chart of a third embodiment of a method for synchronously playing multi-channel videos of the present invention;

[0047] Figure 4 A schematic diagram of the structure of a terminal device in a hardware operating environment involved in an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the principle of a terminal device according to an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of a synchronous control process of an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of a download control process according to an embodiment of the present invention;

[0051] Figure 8 The figure is a flowchart of the playback control class according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0054] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0055] See also Figure 1 As shown, it is a flow chart of the first embodiment of the method for synchronously playing multi-channel videos of the present invention.

[0056] In this embodiment, the method for synchronously playing multi-channel videos is applied to a video surveillance system, and includes:

[0057] Step S100: obtaining a playback time point for synchronous playback of multi-channel videos; traversing the key frames and corresponding timestamps of the video files of all channels, finding the key frame and corresponding channel closest to the playback time point, and determining the found channel as the reference channel.

[0058] Specifically, a video surveillance system includes multiple cameras for collecting video signals, a video recorder for storing video data, and a client terminal for real-time monitoring or playback of videos. These components work together to form a complete video surveillance system. Usually, if a video surveillance system has multiple channels, it means that there are multiple cameras in the system, and each camera corresponds to a channel, so as to achieve multi-channel video surveillance.

[0059] Specifically, the playback time point can be a specific time point, that is, a certain moment, such as 00:00:00 on January 1, 2024. This time point is the starting time point when multiple video files start playing at the same time. Of course, according to specific needs and scenarios, the playback time point can also be understood as a period of time, such as 00:00:00 on January 1, 2024-00:10:00 on January 1, 2024, starting playback at the start time of this period 00:00:00 and ending playback at the end time of this period 00:10:00, so as to meet the needs of different multi-channel video synchronous playback.

[0060] Specifically, in video coding, key frames (I frames, Intra-coded pictures) and predicted frames (P frames, Predictive-coded pictures) used to compress and store video data are two key frame types. Among them, I frames are independent of other frames and do not depend on the information of other frames, so the decoder can decode the entire video sequence from an I frame (a video sequence usually refers to a list or sequential arrangement of a series of video files. These video files can be arranged in a specific order or time order to describe a continuous video stream or sequence of video clips). In a video sequence, an I frame contains complete image information without referring to the data of other frames. Therefore, I frames are usually used at the starting position of the video and at the moment when a scene changes or switches in the video. P frames are predicted and compressed based on previous frames (usually I frames or previous P frames). P frames only contain the difference information between the current frame and the previous frame, but not the complete image data. By utilizing the information of previous frames, P frames can achieve higher compression rates. In video coding, efficient video compression is usually achieved by alternating between I frames and P frames. I frames provide key reference points in video sequences, while P frames exploit temporal correlations to reduce the amount of data through prediction and difference coding, thereby achieving higher compression ratios and efficient video compression and storage.

[0061] Specifically, the timestamp is used to identify the time position of each frame in the video sequence. It can represent the playback time of the video frame in the video or the time offset relative to the start time of the video. For key frames, they usually have their own corresponding timestamps to determine their exact position in the video. These timestamps can be used to ensure the sequential playback of video frames and correct time synchronization.

[0062] Specifically, the reference channel can be used as a time reference for other channels to ensure that multiple channels are played synchronously. Based on the time information of the reference channel, other channels can adjust the playback speed according to the timestamp to ensure the synchronization of multi-channel video playback.

[0063] Step S200: Searching for a video file closest to the timestamp corresponding to the found key frame in all channels, creating a download object that starts downloading from the searched video file, and downloading the video file according to the download object.

[0064] Specifically, when it comes to downloading video files, a download object is usually created to manage the download process. This download object can be a download manager, a download library, or a download control class, depending on the programming language and environment. The process of creating a download object and downloading a video file based on the download object involves steps such as creating a download object, specifying a download link, initiating a download request, downloading the video file, and completing the download. By properly managing the download object, an efficient and stable video file download process can be achieved.

[0065] Step S300: According to a preset cache strategy, the download progress and cache size of the video file are controlled to ensure that there is enough video data for parsing.

[0066] Specifically, by setting a preset cache strategy, adjustments and optimizations can be made according to specific application scenarios and needs, reducing the lag in downloading files, achieving better video download and playback effects, improving user experience and saving network resources.

[0067] Step S400: parsing the downloaded video file, and establishing a video buffer queue according to a preset buffer strategy; decoding the video data in the video buffer queue to synchronously play multi-channel videos.

[0068] Specifically, parsing video files is parsing and processing video data cached in memory or disk for playback or other operations. By caching and parsing video files, the loading and parsing speed of video data can be accelerated, and the responsiveness and smoothness of playback can be improved. The video buffer queue is used to manage the loading and playback order of video data. During video playback, video data usually needs to be loaded from the network or local storage into memory before it can be decoded and displayed. The video buffer queue is responsible for managing the loading order and buffering status of this data to ensure the continuity of video playback. It can load video data in advance as needed to cope with network delays or instability, thereby reducing freezes and waiting time during video playback.

[0069] Specifically, video decoding is a complex process that requires proper configuration and optimization according to specific application scenarios and requirements. First, you need to select a decoder that is suitable for the video data format. Common video decoders include H.264, H.265, and VP9. Initialize the selected decoder, and then obtain the video data to be decoded from the video buffer queue. Use the selected decoder to decode the video data. The decoder decompresses the compressed video data and converts it into raw pixel data for subsequent processing and display. The decoded video frames can be processed subsequently, such as image processing, rendering, etc. Finally, the decoded video frames are displayed on the screen or synthesized into a video stream. In this way, the playback control class completes the decoding and display process of the video data.

[0070] Compared with the prior art, the synchronous playback method of multi-channel videos in the embodiment of the present invention adopts a synchronization algorithm and timestamp technology to determine the reference channel by finding the key frame closest to the playback time point and the corresponding channel. The reference channel can be used as a reference for other channels to ensure the synchronization of the video playback time of each channel. A download object is created according to the key frame timestamp determined by the reference channel, and the video file is downloaded according to the download object. After precise control of the analysis, buffering and decoding of the downloaded video data, the synchronous playback of multi-channel videos can be achieved, which effectively avoids the phenomenon of asynchronous multi-channel video playback or delay between channels that is common in traditional methods, and provides a good product experience.

[0071] See also Figure 2 As shown, it is a flow chart of the second embodiment of the method for synchronously playing multi-channel videos of the present invention.

[0072] Compared with other embodiments of the method for synchronous playback of multi-channel videos, this embodiment provides detailed descriptions of the steps of obtaining the playback time point and searching for the reference channel, and the steps of decoding, rendering and playing the video, and adds a specific description of the preset caching strategy.

[0073] In this embodiment, the method for synchronously playing multi-channel videos is applied to a video surveillance system, and includes:

[0074] Step S101: receiving a request for synchronous playback of multi-channel videos, and obtaining a playback time point corresponding to the request.

[0075] Specifically, the playback time point can be a specific time point or a period of time. When the playback time point is a specific time point, the playback time point can be used as the starting time point of the video playback, and then the playback is ended by presetting the continuous playback time, such as 10 minutes. When the video playback starts, the starting time point of the playback is recorded, and then once the playback reaches the preset continuous playback time, the playback is automatically ended; the user can also manually end the video playback by operating the end button on the interface or entering a command.

[0076] Step S102: querying the video file list of all channels on the day corresponding to the playback time point, and traversing the key frames and corresponding timestamps of each video file in the video file list.

[0077] Specifically, a video file list usually refers to a list or table containing information such as the name, size, type, and creation date of a video file. This list can be used to record relevant information about video files stored in a specific location for easy management and retrieval. A video file list can be a manually created document or a list of files automatically generated by a computer system. The creation date of a video file list is usually expressed as a timestamp. A timestamp is the number of seconds or milliseconds (such as 1 second equals 1000 milliseconds) calculated from a fixed starting time point, using a numerical representation of a specific date and time, such as 2024-01-0112:00:00. In a computer system, timestamps are often used to record the creation, modification, or access time of video files, as well as other time-related information. Timestamps can be used to easily sort and compare video files, and to easily transfer and interpret time information between different systems and programs.

[0078] Step S103: Find out the key frame closest to the playback time point and the corresponding channel, and determine the found channel as the reference channel.

[0079] Specifically, to find the key frame closest to the playback time point, it is necessary to calculate the time difference between each key frame timestamp and the playback time point, and find the key frame corresponding to the timestamp with the smallest absolute value of the time difference. Once the closest key frame is found, the channel where the key frame is located can be determined as the reference channel. Usually, each channel in a video file has its own key frame sequence, and the corresponding channel can be determined based on the timestamp and channel information of the key frame.

[0080] Step S200: Searching for a video file closest to the timestamp corresponding to the found key frame in all channels, creating a download object that starts downloading from the searched video file, and downloading the video file according to the download object.

[0081] Step S300: According to a preset cache strategy, the download progress and cache size of the video file are controlled to ensure that there is enough video data for parsing.

[0082] In some embodiments of the present application, the preset cache strategy includes:

[0083] Cache the downloaded video data, and determine whether the size of the currently downloaded video data is greater than or equal to the preset cache data size;

[0084] If yes, the downloaded video data is put into the parsing queue; if no, the downloaded video data continues to be cached.

[0085] Specifically, the preset cache data size is set according to the configuration and performance of the hardware. The preset cache data size is 50KB.

[0086] Specifically, by caching the downloaded video data, streaming processing of the video data can be achieved. This means that the video data can be parsed and played while being downloaded, without waiting for the entire video file to be completely downloaded. By determining whether the size of the currently downloaded video data is greater than or equal to the preset cache data size, the size of the cache data can be controlled to avoid too much data occupying memory or storage space. Putting the downloaded video data into the parsing queue can speed up the parsing and processing of the video data, thereby improving the smoothness and responsiveness of the video playback. By streaming and parsing video data, the user's experience of watching videos can be optimized, waiting time can be reduced, and the real-time and interactivity of playback can be improved.

[0087] In some embodiments of the present application, placing the downloaded video data into a parsing queue includes:

[0088] Determine whether the parsing queue is full; if so, wait until the video data in the parsing queue is consumed, and then put the downloaded video data into the parsing queue; if not, put the downloaded video data into the parsing queue.

[0089] Specifically, by determining whether the parsing queue is full, we can avoid overflow or memory exhaustion of the parsing queue due to excessive video data. Waiting for the video data in the parsing queue to be consumed before putting the downloaded video data into the parsing queue can balance the parsing and consumption speeds and avoid performance problems caused by mismatched data processing speeds. By controlling the amount of data in the parsing queue, we can improve the stability and reliability of the system and avoid system crashes or performance degradation caused by improper video data processing, thereby improving the stability and performance of the system.

[0090] Step S401: parsing the downloaded video file and establishing a video buffer queue according to a preset buffer strategy.

[0091] Step S402: Acquire the video frame data of the video buffer queue, and decode the video frame data according to the data type to obtain YUV data.

[0092] Step S403: using opengl to render the YUV data so as to perform synchronous playback of multi-channel videos.

[0093] Specifically, video frame data is a single still image in a video file, which is usually played continuously at a certain frame rate to present dynamic images. Decoding is performed according to the data type of the video frame data (such as H.264, H.265, etc.), and the original video frame data is decoded into data in YUV format (YUV is a common video pixel format, where Y represents luminance, U and V represent chrominance, and they together constitute the color and brightness information of the video image). After obtaining the data in YUV format, OpenGL (OpenGL is a cross-platform graphics rendering API that can efficiently process and render graphics data) is used for rendering. In this step, the data in YUV format will be passed to OpenGL for processing and rendering so that the video image can be displayed on the screen. Through the YUV data rendered by OpenGL, the content of multiple video channels can be processed and displayed at the same time, realizing the synchronous playback of multi-channel videos.

[0094] The technical solution of this embodiment solves the problem that multi-channel video synchronization depends on device-side processing, simplifies the logic of the device side, and reduces the consumption of device-side resources. The method for synchronous playback of multi-channel videos also supports multiple video formats and device types, solves the problem of dependence on specific formats and devices in traditional technologies, and improves the adaptability of the system.

[0095] See also Figure 3 As shown, it is a flow chart of the third embodiment of the method for synchronously playing multi-channel videos of the present invention.

[0096] Compared with other embodiments of the method for synchronously playing multi-channel videos, this embodiment adds a step of setting a new reference channel and specifically describes the preset buffer strategy.

[0097] In this embodiment, after step S300 and before step S400, the following steps are further included:

[0098] Step S501: When the download of the current video file of the reference channel is completed, switching to the next video file for downloading according to the video file list.

[0099] Step S502: adding a preset interval duration to the end timestamp of the current video file to obtain a switching timestamp for switching video file download.

[0100] Specifically, the preset interval duration is set according to specific application scenarios and requirements. The preset interval duration is 1 second (1000 milliseconds).

[0101] Step S503: traverse the start time and end time of the video files of all channels to find a channel that meets the preset conditions.

[0102] The preset condition is that the switching timestamp is between the start time and the end time of a video file of a channel.

[0103] Step S504a: If the channel is found, the found channel is set as a new reference channel.

[0104] Step S504b: If the channel is not found, then search for a channel of the video file corresponding to a start time that is after the switching timestamp and closest to the switching timestamp among all channels, and set this channel as a new reference channel.

[0105] It should be noted that step S504a and step S504b are respectively the processing steps of finding the channel and not finding the channel. There is no order relationship between step S504a and step S504b, and either one can be implemented.

[0106] Specifically, the above are the steps for setting a new reference channel. When the current video file of the reference channel is downloaded, a new reference channel needs to be set to ensure that the system can switch to the new video source in time to maintain the continuity and stability of video playback. By setting a new reference channel, the system can seamlessly switch to the new video source immediately after the current video file is downloaded, thereby achieving the continuity of video playback and avoiding playback interruptions or freezes caused by video source interruptions or switching delays. Setting a new reference channel can also improve the user's viewing experience, ensuring that users can watch video content continuously without waiting for too long a switching time. By switching to a new reference channel in a timely manner, the system can better manage and optimize the selection and switching of video sources, thereby improving the overall performance and stability of the system to ensure continuous multi-channel video synchronous playback and a good user experience.

[0107] In some embodiments of the present application, the preset buffer strategy includes at least one of a first strategy, a second strategy, and a third strategy;

[0108] The first strategy includes:

[0109] Obtain the parsed video data and determine whether the data of each frame is the reference channel data;

[0110] If yes, then update the timestamp of the current video frame to the reference timestamp, and determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames. If yes, suspend downloading and parsing the video data. If no, insert the video frame data into the video buffer queue.

[0111] If not, obtain the difference between the timestamp of the video frame data of the current channel and the reference timestamp. If the difference is greater than the preset positive difference, suspend downloading and parsing of the video data; if the difference is less than the preset negative difference, discard the video frame; if the difference is between the preset negative difference and the preset positive difference, determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames. If so, suspend downloading and parsing of the video data; otherwise, insert the video frame data into the video buffer queue.

[0112] Specifically, the preset number of frames, the preset negative difference value, and the preset positive difference value are all set according to specific application scenarios and requirements. The preset number of frames is 10 frames. The preset negative difference value is -1, and the preset positive difference value is 1.

[0113] Specifically, the first strategy mainly checks and processes the timestamps of the video frame data of the reference channel and the video frame data of the non-reference channel on the basis of determining the reference timestamp, so as to ensure that the parsing and downloading of the video data can be synchronized with the reference time, and the amount of data in the video buffer queue is within a reasonable range. By comparing the timestamp of the current video frame of the non-reference channel with the reference timestamp, the time synchronization of the video frame data of the non-reference channel and the reference channel can be ensured, avoiding the problem of incoherent or misaligned playback caused by time difference. For the video frame data of the reference channel, it is determined whether to suspend the parsing and downloading of data according to the amount of data in the video buffer queue to balance data processing and system performance. For the video frame data of the non-reference channel, the downloading and parsing process of the video data can be controlled according to the difference of the timestamp. When the timestamp difference is greater than 1 second, the parsing and downloading of the video data are suspended to avoid excessive accumulation or unnecessary processing of data; when the timestamp difference is between -1 second and 1 second, it is determined whether to suspend the parsing and downloading of the data according to the amount of data in the video buffer queue to balance data processing and system performance. When the timestamp difference is less than -1 second, the video frame is discarded. By judging the data volume of the video buffer queue, the insertion and discarding of video frame data can be controlled to ensure that the data volume in the video buffer queue is within a reasonable range, and to avoid too much or too little data affecting the smoothness and quality of video playback. The purpose of setting the first strategy is to ensure that the video data of the non-reference channel can be synchronized with the reference time, and to reasonably control the download, parsing and buffering of the video data of the reference channel and the non-reference channel to optimize the smoothness of video playback and system performance.

[0114] The second strategy includes:

[0115] Reading data from the video buffer queue of the current channel, and when the video frame data is read, determining whether the video frame data is reference channel data;

[0116] If so, update the timestamp of the current video frame to the reference timestamp, traverse all channels to obtain the timestamp of the current video frame of the channel, calculate the difference between the timestamp and the reference timestamp, and if the difference is between the preset negative difference and the preset positive difference, resume the suspended downloading and parsing of video data for this channel.

[0117] Specifically, the second strategy is mainly to check and process the data volume of the reference channel. When it is found that the timestamp difference between the timestamp of other channel data and the timestamp of the reference channel data is within plus or minus 1 second, the synchronization control class needs to be notified to resume the data parsing and downloading of this channel. This ensures that the video data of each channel can be resumed and processed in time to avoid playback interruption or freeze caused by time difference.

[0118] The third strategy includes:

[0119] It is determined whether the number of remaining video frames in the current video buffer queue is less than the preset minimum number of frames. If so, the suspended downloading and parsing of video data is resumed, and the downloaded and parsed video frame data is inserted into the video buffer queue.

[0120] Specifically, the preset minimum frame number is set according to specific application scenarios and requirements. The preset minimum frame number is 5 frames.

[0121] Specifically, the third strategy is mainly to check and process the number of remaining data frames in the video buffer queue. By judging whether the number of remaining data frames in the video buffer queue is less than 5 frames, the shortage of video data can be discovered in time, thereby notifying the synchronization control class to resume the paused parsing and downloading to ensure that the video playback can continue and avoid playback interruptions or freezes due to insufficient data. When the number of remaining data frames in the video buffer queue is insufficient, the synchronization control class is notified to resume the paused parsing and downloading, so that the video data can be replenished in time, thereby optimizing the processing and management of the video data and ensuring the smoothness and quality of video playback. By replenishing video data in a timely manner, the overall performance of the system can be improved, and additional processing burden or performance loss caused by insufficient data can be avoided, thereby providing a better user experience.

[0122] The technical solution of this embodiment introduces a dynamic reference channel and a reference timestamp control mechanism, which can achieve the purpose of synchronously playing videos in the remaining channels with data even if there is no data in some time periods in different channels.

[0123] The following is an embodiment of the terminal device provided by the present invention. The embodiment of the terminal device and the embodiment of the synchronous playback method of multi-channel videos belong to the same concept, and the details not described in detail in the embodiment of the terminal device can refer to the embodiment of the synchronous playback method of multi-channel videos.

[0124] See also Figure 4 , which is a schematic diagram of the structure of a terminal device in a hardware operating environment involved in an embodiment of the present invention.

[0125] In this embodiment, a terminal device includes:

[0126] Memory 1005, used to store program instructions; and

[0127] The processor 1001 is used to execute the program instructions to implement the steps of the method for synchronously playing multi-channel videos as described above.

[0128] The terminal device of the embodiment of the present invention can be a computing device such as a desktop computer, a notebook, a palm computer, and a server. Figure 4 As shown, the terminal device may include: a processor 1001 (e.g., a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit, such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0129] Those skilled in the art will understand that Figure 4 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0130] like Figure 4 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a computer program.

[0131] exist Figure 4 In the terminal device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the computer program stored in the memory 1005. When the computer program is called and executed by the processor 1001, the steps of the above-mentioned multi-channel video synchronous playback method are implemented.

[0132] See also Figure 5As shown, it is a schematic diagram of the principle of a terminal device according to an embodiment of the present invention.

[0133] The terminal device provided by the embodiment of the present invention can be a client or other electronic device that establishes a connection with the client. The terminal device solves the problem that multi-channel video synchronization depends on device-side processing, simplifies the logic of the device side, and reduces the consumption of device-side resources; adopts synchronization algorithm and timestamp technology to achieve accurate synchronous playback of multiple video channels, avoiding the common problems of video asynchrony or inter-channel delay in traditional methods; also introduces dynamic reference channel and reference timestamp control mechanism, which enables different channels to play videos synchronously even if there is no data in some time periods; and supports multiple video formats and device types, which solves the problem of dependence on specific formats and devices in traditional technologies and improves the adaptability of the system.

[0134] See also Figure 6 , Figure 7 and Figure 8 As shown, it is a flow chart of the synchronization control class, download control class and playback control class of an embodiment of the present invention.

[0135] The terminal device is respectively designed with synchronization control class, download control class and playback control class. These three software designed functional modules or categories can be implemented in various programming languages, such as C++, Java, Python, etc. These three control classes work together to realize the time synchronization, data management and playback control of the video playback system, thereby providing a stable and smooth multi-channel video playback experience.

[0136] The following specifically describes the implementation of the terminal device provided in the embodiment of the present invention:

[0137] 1. Select the time point to start playing;

[0138] 2. Query the video file list of all channels on the selected playback time point on that day;

[0139] 3. Traverse the I frames and corresponding timestamps of the video files of all channels, find the channel with the I frame closest to the playback time point where playback starts, and set this channel as the reference channel;

[0140] 4. Create a corresponding number of download objects according to the number of channels, and start downloading video files from the file with the closest playback time;

[0141] 5. Cache the downloaded video data and determine whether the currently downloaded video data is 50KB. If so, put the video data into the parsing queue. If not, continue to cache the downloaded video data.

[0142] 6. Determine whether the parsing queue is full. If the queue is full, wait until the data in the data parsing queue is consumed and then put the video data into the parsing queue again;

[0143] 7. If the current video file of the reference channel has been downloaded, you need to switch to the next video file according to the video file list, use the end timestamp of the current video file plus 1 second to get the switching timestamp of the switching file, and call this time back to the synchronization control class. After the synchronization control class obtains the switching timestamp of the switching file, it traverses the start time and end time of the video files of all channels to determine whether the switching timestamp is between the start time and end time of a file of a certain channel. If such a channel is found, this channel is set as the new reference channel. If the switching timestamp is not between the start time and end time of a file of all channels, search among all channels for the channel with the start time closest to the switching timestamp after the switching timestamp, and set this channel as the new reference channel;

[0144] 8. After the video parsing is completed, a callback is sent to the synchronization control class. The synchronization control class obtains the parsed video data and determines whether the data of each frame is the reference channel data.

[0145] 9. If it is the reference channel data, the timestamp of the current video frame is updated to the reference timestamp, and it is determined whether the data in the video buffer queue of the current channel is greater than 10 frames. If it is greater than 10 frames, the download control class is informed to suspend parsing and downloading data. If the data in the video buffer queue is less than or equal to 10 frames, the video frame data is inserted into the video buffer queue;

[0146] 10. If it is non-reference channel data, the timestamp of the video frame data of the current channel is compared with the reference timestamp. If the difference between the timestamp of the current video frame and the reference timestamp is greater than 1 second, the parsing and downloading of the video data are suspended. If the difference between the timestamp of the current video frame and the reference timestamp is less than -1 second, the video frame needs to be discarded. If the difference between the timestamp of the current video frame and the reference timestamp is between -1 second and 1 second, it is determined whether the data in the video buffer queue of the current channel is greater than 10 frames. If it is greater than 10 frames, the download control class is informed to suspend the parsing and downloading of data. If it is less than or equal to 10 frames, the video frame data is stuffed into the video buffer queue.

[0147] 11. The playback control class of each channel reads data from the video buffer queue of the current channel. After reading the frame data, it determines whether it is the reference channel data. If it is the reference channel data, it notifies the synchronization control class to traverse all channels and compare the timestamp of the current video frame of the channel with the reference timestamp. If the difference is within plus or minus 1 second, the data parsing and downloading of this channel that have been suspended are resumed;

[0148] 12. In the process of obtaining video frame data from the video buffer queue, it is necessary to determine whether the number of remaining data frames in the current video buffer queue is less than 5 frames. If it is less than 5 frames, it is necessary to notify the synchronization control class, resume the suspended parsing and downloading, and put the video data into the video buffer queue;

[0149] 13. After the buffer queue gets the video frame data, it decodes it according to the data type;

[0150] 14. The decoded YUV data is called back to the upper interface, and OpenGL is used to render the YUV data, allowing users to view multi-channel video data simultaneously.

[0151] It can be seen that the terminal device provided by the embodiment of the present invention solves the problem that the current video surveillance system always has a synchronization problem when playing multi-channel videos.

[0152] The following is an embodiment of the computer-readable storage medium provided by the present invention. The embodiment of the computer-readable storage medium and the embodiment of the synchronous playback method and terminal device of the multi-channel video described above belong to the same concept, and the details not described in detail in the embodiment of the computer-readable storage medium can refer to the embodiment of the synchronous playback method and terminal device of the multi-channel video described above.

[0153] In this embodiment, a computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method for synchronously playing multi-channel videos as described above are implemented.

[0154] Since the terminal device and computer-readable storage medium of the present invention can both implement the steps of the above-mentioned method for synchronous playback of multi-channel videos, they at least have all the beneficial effects brought about by the technical solution of the above-mentioned method for synchronous playback of multi-channel videos, which will not be described in detail here.

[0155] The above description is only a partial or preferred embodiment of the present invention. Neither the text nor the drawings can limit the scope of protection of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect application in other related technical fields are included in the scope of protection of the present invention.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CDROM, optical storage, etc.) containing computer-usable program codes.

[0157] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for synchronously playing multi-channel videos, applied to a video surveillance system, characterized in that: include: Get the playback time point of multi-channel video synchronous playback; Traversing the key frames and corresponding timestamps of the video files of all channels, finding the key frame and the corresponding channel closest to the playback time point, and determining the found channel as the reference channel; Searching for a video file closest to the timestamp corresponding to the found key frame in all channels, creating a download object that starts downloading from the searched video file, and downloading the video file according to the download object; Cache the downloaded video data, and determine whether the size of the currently downloaded video data is greater than or equal to the preset cache data size; If yes, the downloaded video data is put into the parsing queue; if no, the downloaded video data is continued to be cached; Parse the downloaded video files and establish a video buffer queue according to the preset buffer strategy; Decoding the video data of the video buffer queue to synchronously play multi-channel videos; The preset buffer strategy includes a first strategy, and the first strategy includes: Obtain the parsed video data and determine whether the data of each frame is the reference channel data; If yes, then update the timestamp of the current video frame to the reference timestamp, and determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames; if yes, suspend downloading and parsing the video data, if no, insert the video frame data into the video buffer queue; If not, obtain the difference between the timestamp of the video frame data of the current channel and the reference timestamp. If the difference is greater than the preset positive difference, suspend downloading and parsing of the video data; if the difference is less than the preset negative difference, discard the video frame; if the difference is between the preset negative difference and the preset positive difference, determine whether the data in the video buffer queue of the current channel is greater than the preset number of frames; if so, suspend downloading and parsing of the video data, otherwise, insert the video frame data into the video buffer queue.

2. The method for synchronously playing multi-channel videos according to claim 1, characterized in that: The preset buffer strategy also includes a second strategy and a third strategy; The second strategy includes: Reading data from the video buffer queue of the current channel, and when the video frame data is read, determining whether the video frame data is reference channel data; If so, the timestamp of the current video frame is updated to the reference timestamp, and all channels are traversed to obtain the timestamp of the current video frame of the channel, and the difference between the timestamp and the reference timestamp is calculated. If the difference is between the preset negative difference and the preset positive difference, the suspended downloading and parsing of video data of this channel is resumed; The third strategy includes: It is determined whether the number of remaining video frames in the current video buffer queue is less than the preset minimum number of frames. If so, the suspended downloading and parsing of video data is resumed, and the downloaded and parsed video frame data is inserted into the video buffer queue.

3. The method for synchronously playing multi-channel videos according to claim 1, characterized in that: The step of obtaining the playback time point of the multi-channel video synchronous playback includes: A request for synchronous playback of multi-channel videos is received, and a playback time point corresponding to the request is obtained.

4. The method for synchronously playing multi-channel videos according to claim 1, characterized in that: The key frames and corresponding timestamps of the video files of all channels traversed include: The video file list of all channels on the day corresponding to the playback time point is queried, and for each video file in the video file list, the key frames and corresponding timestamps therein are traversed.

5. The method for synchronously playing multi-channel videos according to claim 4, characterized in that: Before parsing the downloaded video file, the method further includes: When the download of the current video file of the reference channel is completed, switching to the next video file for downloading according to the video file list; Adding a preset interval duration to the end timestamp of the current video file to obtain a switching timestamp for switching video file download; Traversing the start time and end time of the video files of all channels to find a channel that meets a preset condition; wherein the preset condition is that the switching timestamp is between the start time and end time of a video file of a channel; If the channel is found, the found channel is set as a new reference channel; If the channel is not found, then find the channel of the video file corresponding to the start time after the switching timestamp and closest to the switching timestamp among all channels, and set this channel as a new reference channel.

6. The method for synchronously playing multi-channel videos according to claim 1, characterized in that: The step of placing the downloaded video data into a parsing queue comprises: Determine whether the parsing queue is full; if so, wait until the video data in the parsing queue is consumed, and then put the downloaded video data into the parsing queue; if not, put the downloaded video data into the parsing queue.

7. The method for synchronously playing multi-channel videos according to claim 1, characterized in that: The decoding of the video data of the video buffer queue so as to synchronously play the multi-channel video includes: Obtain video frame data from the video buffer queue, and decode the video frame data according to the data type to obtain YUV data; The YUV data is rendered using opengl to enable synchronous playback of multi-channel videos.

8. A terminal device, characterized in that: include: A memory for storing program instructions; as well as A processor is used to execute the program instructions to implement the steps of the method for synchronously playing multi-channel videos as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method for synchronously playing multi-channel videos according to any one of claims 1 to 7 are implemented.

Images