A video processing device, a video processing method, and a storage medium

Abstract

Embodiments of the present application provide a video processing device, a video processing method, and a storage medium. The video processing device includes: a video processor. The video processor runs a video parsing thread, a target service thread, and a non-target service thread. The video parsing thread is used to obtain a video stream to be processed, store the video stream to be processed in a download buffer corresponding to a target video channel, read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames, and store the video frames in a video frame buffer corresponding to the target video channel. The target service thread is used to, when the parsed video frames have a target service identifier, read the video frames with the target service identifier from the video frame buffer, and decode the video frames with the target service identifier. The non-target service thread is used to read video frames from the video frame buffer, and when the video frames do not have a target service identifier, decode the video frames.

Description

Technical Field

[0001] This application relates to the technical field of video processing, and in particular, to a video processing device, a video processing method, and a storage medium. Background Art

[0002] With the increase in video surveillance services, more and more memory is required in video surveillance devices to process corresponding services. For example, devices such as NVR (Network Video Recorder) and DVR (Digital Video Recorder) can decode the I-frames in a video and display the decoded images on the interface. They can also perform JPEG encoding on the I-frames of the current channel video when decoding or detecting an alarm, and upload the encoded JPEG pictures to the hard disk of the storage device for storage, which is called the zero-channel snapshot service.

[0003] For the above zero-channel snapshot and other services, two separate memory areas need to be applied to process the video. One is the bitstream download buffer, which is used to receive the encapsulated video stream. The video stream can be encapsulated in ways such as PS (Program Stream), TS (Transport Stream), RTP (Real-time Transport Protocol), etc. Encapsulation can store the video stream in a file in a certain format, enabling the player to play according to the encapsulation format. The other is the decoding and frame-assembling buffer, which is used to store the raw stream data obtained by parsing the video stream, that is, to parse the encapsulated video stream into the original video stream data without any format encapsulation, generally in the H.264 format, for subsequent decoding use. Therefore, the number of video buffers in current video surveillance devices is relatively large, resulting in large memory occupancy. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a video processing device, a video processing method, and a storage medium to reduce the number of video buffers and save memory. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiments of this application provide a video processing device, which includes a video processor. The video processor runs a video parsing thread, a target service thread, and a non-target service thread, where:

[0006] The video parsing thread is used to obtain the video stream to be processed; store the video stream to be processed in the download buffer corresponding to the target video channel; read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames; store the video frames in the video frame buffer corresponding to the target video channel, where the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer;

[0007] The target service thread is used to, when the video frames obtained by parsing have a target service identifier, read the video frames with the target service identifier from the video frame buffer, and decode the video frames with the target service identifier;

[0008] The non-target service thread is used to read the video frames from the video frame buffer, and when the video frames do not have a target service identifier, decode the video frames.

[0009] Optionally, the video parsing thread is further used to determine whether there is a target service identifier in the packet header of the parsed video frames before the target service thread reads the video frames with the target service identifier from the video frame buffer; if so, trigger the target service thread to read the video frames with the target service identifier from the video frame buffer.

[0010] Optionally, the non-target service thread is specifically used to read the video frames from the video frame buffer; determine whether there is the target service identifier in the packet header of the video frames; if not, decode the video frames.

[0011] Optionally, the video parsing thread is further used to, before the target service thread reads the video frames with the target service identifier from the video frame buffer, move the video frames with a target service identifier in the packet header from the video frame buffer to the return buffer; and trigger the target service thread to read the video frames with a target service identifier in the packet header from the return buffer to decode the video frames with a target service identifier in the packet header.

[0012] Optionally, the video parsing thread is specifically used to receive the video stream sent by the stream sending end as the video stream to be processed, where the video stream is obtained by the stream sending end encapsulating the video data to be sent, and when the video data to be sent is for the target service, the stream sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

[0013] Optionally, the streaming end is specifically configured to add a target service identifier to the packet header corresponding to the video data to be sent every preset duration; or,

[0014] Specifically configured to add a target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or,

[0015] Specifically configured to perform target detection on the video data to be sent, and add a target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data in which a target object is detected.

[0016] Optionally, the non-target service thread is used to process video playback services;

[0017] The video parsing thread is further configured to, before the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel, in the case of an idle channel, use the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed; in the case of no idle channel, use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing video playback services.

[0018] In a second aspect, an embodiment of the present application provides a video processing method, which is applied to a video processing device. The video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread. The method includes:

[0019] The video parsing thread obtains a video stream to be processed; stores the video stream to be processed in the download buffer corresponding to the target video channel; reads the video stream to be processed from the download buffer, and parses the video stream to obtain video frames; stores the video frames in the video frame buffer corresponding to the target video channel, where the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer;

[0020] The target service thread, in the case where the parsed video frame has a target service identifier, reads the video frame with the target service identifier from the video frame buffer, and decodes the video frame with the target service identifier;

[0021] The non-target service thread reads the video frame from the video frame buffer, and decodes the video frame in the case where the video frame does not have a target service identifier.

[0022] Optionally, before the target service thread reads the video frame with the target service identifier from the video frame buffer, the method further includes:

[0023] The video parsing thread determines whether there is a target service identifier in the packet header of the parsed video frame; if so, triggers the target service thread to read the video frame with the target service identifier from the video frame buffer.

[0024] Optionally, the step of the non-target service thread reading the video frame from the video frame buffer and decoding the video frame when the video frame does not have the target service identifier includes:

[0025] The non-target service thread reads the video frame from the video frame buffer; determines whether there is the target service identifier in the packet header of the video frame; if not, decodes the video frame.

[0026] Optionally, before the target service thread reads the video frame with the target service identifier from the video frame buffer, the method further includes:

[0027] The video parsing thread moves the video frame with the target service identifier in the packet header from the video frame buffer to the back buffer; and triggers the target service thread to read the video frame with the target service identifier in the packet header from the back buffer to decode the video frame with the target service identifier in the packet header.

[0028] Optionally, the step of the video parsing thread obtaining the video stream to be processed includes:

[0029] The video parsing thread receives the video stream sent by the stream sending end as the video stream to be processed, where the video stream is obtained by encapsulating the video data to be sent by the stream sending end. When the video data to be sent is for the target service, the stream sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

[0030] Optionally, the step of adding the target service identifier to the packet header corresponding to the video data to be sent includes:

[0031] The stream sending end adds the target service identifier to the packet header corresponding to the video data to be sent every preset time period; or,

[0032] Adds the target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or,

[0033] Perform object detection on the video data to be sent, and add a target service identifier to the header of the target video data to be sent, where the target video data to be sent is the video data in which the target object is detected.

[0034] Optionally, the non-target service thread is used to process the video playback service;

[0035] Before the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel, the method further includes:

[0036] When there is an idle channel, the video parsing thread uses the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or uses the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed; when there is no idle channel, the video parsing thread uses the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing the video playback service.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the above second aspects is implemented.

[0038] Advantageous effects of the embodiments of the present application:

[0039] In the solution provided by the embodiments of the present application, the video processing device includes a video processor. The video processor runs a video parsing thread, a target service thread, and a non-target service thread. Among them, the video parsing thread is used to obtain the video stream to be processed, store the video stream to be processed in the download buffer corresponding to the target video channel, read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames, and store the video frames in the video frame buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer. The target service thread is used to, when the parsed video frames have target service identifiers, read the video frames with target service identifiers from the video frame buffer and decode the video frames with target service identifiers. The non-target service thread is used to read the video frames from the video frame buffer and, when the video frames do not have target service identifiers, decode the video frames. Since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with target service identifiers from the video frame buffer for decoding, and the non-target service thread can also read the video frames from the video frame buffer to decode the video frames without target service identifiers. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory. Of course, any product or method implementing the present application does not necessarily need to achieve all the above-mentioned advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other embodiments can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic structural diagram of a video processing device provided by an embodiment of the present application;

[0042] Figure 2 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0043] Figure 3 Based on Figure 1 It is a flowchart for determining that there is a target service identifier in the packet header of a video frame in the shown embodiment;

[0044] Figure 4 ForFigure 2 A specific flowchart of step S206 in the illustrated embodiment;

[0045] Figure 5 A service processing flowchart of the zero-channel capture service provided by the embodiment of the present application;

[0046] Figure 6 A service processing flowchart of the ordinary playback service provided by the embodiment of the present application;

[0047] Figure 7 Based on Figure 2 A schematic diagram of the video frame buffer according to the illustrated embodiment;

[0048] Figure 8 A specific flowchart of the video processing method provided by the embodiment of the present application;

[0049] Figure 9 Another structural schematic diagram of the video processing device provided by the embodiment of the present application. Detailed implementation manners

[0050] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0051] In order to reduce the number of video buffers and save memory, the embodiments of the present application provide a video processing device, a video processing method, a computer-readable storage medium, and a computer program product. Next, a video processing device provided by the embodiments of the present application will be introduced first.

[0052] As Figure 1 shown, a video processing device, the video processing device includes a video processor 110, and the video processor 110 runs a video parsing thread 111, a target service thread 112, and a non-target service thread 113, where:

[0053] The video parsing thread 111 is used to obtain a video stream to be processed; store the video stream to be processed in the download buffer corresponding to the target video channel; read the video stream to be processed from the download buffer, and parse the video stream to obtain video frames; store the video frames in the video frame buffer corresponding to the target video channel, where the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer;

[0054] The target service thread 112 is configured to, when the parsed video frame has a target service identifier, read the video frame with the target service identifier from the video frame buffer and decode the video frame with the target service identifier.

[0055] The non-target service thread 113 is configured to read the video frame from the video frame buffer and decode the video frame when the video frame does not have a target service identifier.

[0056] It can be seen that in the solution provided by the embodiments of the present application, the video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread. Among them, the video parsing thread is configured to obtain a video stream to be processed, store the video stream to be processed in the download buffer corresponding to the target video channel, read the video stream to be processed from the download buffer, parse the video stream to be processed to obtain video frames, and store the video frames in the video frame buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer. The target service thread is configured to, when the parsed video frame has a target service identifier, read the video frame with the target service identifier from the video frame buffer and decode the video frame with the target service identifier. The non-target service thread is configured to read the video frame from the video frame buffer and decode the video frame when the video frame does not have a target service identifier. Since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with the target service identifier from the video frame buffer for decoding, and the non-target service thread can also read the video frames from the video frame buffer to decode the video frames without the target service identifier. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0057] In video surveillance storage devices such as NVRs and DVRs, multiple services need to process video streams. Usually, multiple buffers are allocated for each service to store the data of the processed video streams. However, since some services have no service processing for a long time, it causes waste of buffers and thus waste of memory. In the solution provided by the embodiments of the present application, the number of buffers can be reduced by reusing the buffers.

[0058] The video parsing thread 111 can be used to obtain a video stream to be processed. Among them, the video stream is the transmission of video data and can be used as a stable and continuous stream for hard disk storage reading and processing. The video stream can be obtained by encapsulating video data. For example, it can be PS (Program Stream), etc., which is not specifically limited here. PS encapsulation is one of the current mainstream audio and video encapsulation methods and is widely used in the storage link of multimedia files such as DVDs.

[0059] In one implementation, for the video surveillance scenario, the video processing device can be an NVR. The NVR belongs to the video surveillance backend product, can access multiple network cameras, and decode and display, record and store, and transmit over the network the video stream data transmitted by the network cameras. For example, the NVR is connected to the network camera IPC (IP CAMERA). The IPC can transmit the video stream encapsulated with RTP (Real-time Transport Protocol) to the NVR. The video parsing thread of the video processor of the NVR can receive this video stream as the video stream to be processed.

[0060] After the video parsing thread 111 obtains the video stream to be processed, it can store the video stream to be processed in the download buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space. The free space is the remaining storage space of the download buffer. That is, when the download buffer has remaining storage space, it can be used to store the video stream to be processed.

[0061] The video parsing thread 111 can read the video stream to be processed from the download buffer and parse the video stream to obtain video frames. Among them, the video parsing thread 111 can call the parsing library to parse the video stream to be processed, which can be used to parse the video of the service processed by the non-target service thread 113, or can also be used to parse the video of the service processed by the target service thread 112.

[0062] Next, the video parsing thread 111 can store the parsed video frames in the video frame buffer corresponding to the target video channel. In one implementation, when the video parsing thread 111 parses the video stream to be processed to obtain video frames, it can determine whether the parsed video frames have a target service identifier. If there is a target service identifier, the target service thread 112 can be triggered to read the video frames with the target service identifier from the video frame buffer.

[0063] For example, the video processing device is an NVR. The target service thread is used to process the zero-channel snapshot capture service, and the non-target service thread is used to process the playback service. The download buffer A corresponding to the target video channel A has free space. After the video parsing thread in the NVR obtains the video stream a to be processed, it can store the video stream a to be processed in the download buffer A. Then, the video parsing thread reads the video stream a to be processed from the download buffer A, and calls the parsing library to parse the video stream a to be processed, and stores the parsed video frames A and B in the video frame buffer A corresponding to the target video channel A. Among them, it is determined that the parsed video frame A has a zero-channel snapshot capture service identifier, and the video frame B does not have a zero-channel snapshot capture service identifier, that is, the video frame A is used for the zero-channel snapshot capture service, and the video frame B is used for the playback service. Then, the video parsing thread can trigger the target service thread to read the video frame A from the video frame buffer A.

[0064] Furthermore, the target service thread 112 can, in the case where the parsed video frame has a target service identifier, read the video frame with the target service identifier from the video frame buffer and decode the video frame with the target service identifier. Specifically, the target service thread 112 is used to process the zero-channel snapshot capture service. There is an I-frame in the video frame buffer, and the I-frame is used for the zero-channel snapshot capture service. Then, the target service thread 112 can read the I-frame from the video frame buffer and decode the I-frame.

[0065] For example, the target service thread is used to process the zero-channel snapshot capture service. The video processing device NVR is connected to an IPC. The IPC sends an MJPEG-encoded picture to the NVR. The picture is encapsulated in PS. The video parsing thread of the NVR receives the video stream of the picture and stores it in the download buffer A, and stores the parsed video frame A in the video frame buffer A. In the case where the parsed video frame A has a zero-channel snapshot capture service identifier, the target service thread reads the video frame A from the video frame buffer A, and then decodes the video frame A and converts it into an RGB-format picture for display.

[0066] Another example is that the target service thread is used to process the zero-channel snapshot capture service. The video processing device NVR retrieves the first frame (i.e., the I-frame) of the video from the hard disk and stores it in the download buffer B, and stores the parsed video frame I-frame in the video frame buffer B. In the case where the parsed I-frame is used for the zero-channel snapshot capture service, the target service thread reads the I-frame from the video frame buffer B for decoding, and can be converted into an RGB format for display or saved after MJPEG encoding for subsequent use.

[0067] In one embodiment, the video parsing thread 111 and the non-target service thread 113 run in parallel. When the video parsing thread 111 stores the parsed video frames in the video frame buffer, the non-target service thread 113 can read the video frames from the video frame buffer and decode the video frames when the video frames do not have the target service identifier.

[0068] For example, the video processing device is an NVR, the target service thread is used to process the zero-channel screenshot service, and the non-target service thread is used to process the playback service. The video parsing thread in the NVR stores the parsed video frame B in the video frame buffer A. The non-target service thread can read the video frame B from the video frame buffer A. Since the video frame B does not have the zero-channel screenshot service identifier, that is, the video frame B is for the playback service, the non-target service thread can decode the video frame B.

[0069] In the solution of this embodiment, since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with the target service identifier from the video frame buffer for decoding, and the non-target service thread can also read the video frames from the video frame buffer to decode the video frames without the target service identifier. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0070] As an embodiment of this application, the above video parsing thread can also be used to determine whether there is a target service identifier in the packet header of the parsed video frame before the target service thread reads the video frame with the target service identifier from the video frame buffer; if so, trigger the target service thread to read the video frame with the target service identifier from the video frame buffer.

[0071] The video parsing thread can obtain video frames by parsing the video stream to be processed. The video frames include a packet header, and the video parsing thread can determine whether there is a target service identifier in the packet header of the video frame. The target service identifier is used to indicate whether the corresponding video frame is for the service processed by the target service thread. The target service identifier can be the channel number corresponding to the service processed by the target service thread or a preset identifier number, which is not specifically limited herein.

[0072] In one embodiment, the video stream is obtained through encapsulation. When parsing the video stream, the video frames obtained by parsing generally include a packet header and specific video data. Some parameter information of the video frame can be stored in the packet header. For example, the video stream type, the length of the video stream, the system time, etc. To characterize whether a video frame is for the target service processed by the target service thread, a target service identifier can be added to the packet header. In this way, the video parsing thread can also determine whether the video frame is for the service processed by the target service thread by determining whether there is a target service identifier in the packet header of the video frame.

[0073] If it is determined that there is a target service identifier in the packet header, that is, it can be determined that the video frame is for the target service processed by the target service thread. Then, the video parsing thread can trigger the target service thread to read the video frame with the target service identifier from the video frame buffer. If it is determined that there is no target service identifier in the packet header, that is, it can be determined that the video frame is for the service processed by a non-target service thread.

[0074] For example, the video processing device is an NVR, the target service thread is used to process the zero-channel capture service, the video stream to be processed is video stream A encapsulated in PS, and the video parsing thread of the NVR can parse video stream A to obtain video frame A and store video frame A in video frame buffer A. Video frame A includes a PS header, that is, PSH (Program Stream pack Header), and the video parsing thread can determine whether there is a target service identifier in the PSH of this video frame A. When it is determined that the PSH includes the zero-channel capture service identifier, the video parsing thread triggers the target service thread to read video frame A with the zero-channel capture service identifier from video frame buffer A.

[0075] It can be seen that in this embodiment, the video parsing thread can also be used to determine whether there is a target service identifier in the packet header of the parsed video frame before the target service thread reads the video frame with the target service identifier from the video frame buffer; if so, trigger the target service thread to read the video frame with the target service identifier from the video frame buffer. Therefore, when the video frame is stored in the video frame buffer, it can be determined whether the video frame has a target service identifier, and then when there is a target service identifier, trigger the target service thread to read the video frame, without the need to separately apply for a dedicated buffer to parse the video stream corresponding to the service processed by the target service thread, which can achieve the reuse of the buffer, and then can reduce the number of video buffers and save memory.

[0076] As an implementation manner of an embodiment of the present application, the above non-target service thread can specifically be used to read the video frame from the video frame buffer; determine whether the target service identifier exists in the packet header of the video frame; if not, decode the video frame.

[0077] Since the video parsing thread and the non-target service thread are parallel threads, when the video parsing thread stores the parsed video frame in the video frame buffer, the non-target service thread can read the video frame from the video frame buffer. After the non-target service thread reads the video frame from the video frame buffer, it can determine whether the video frame is for the service processed by the non-target service thread.

[0078] In an implementation manner, the non-target service thread can determine whether the target service identifier exists in the packet header of the video frame. If it is determined that the target service identifier does not exist in the packet header of the video frame, it can be determined that the video frame is for the service processed by the non-target service thread, and then the video frame can be decoded.

[0079] For example, the non-target service thread is used to process the playback service, the target service thread is used to process the zero-channel capture service, the video frame B is stored in the video frame buffer A, and the non-target service thread can read the video frame B from the video frame buffer A. It is determined that the zero-channel capture service identifier does not exist in the packet header of the video frame B, and it can be determined that the video frame is for the playback service. Then, the non-target service thread reads the video frame B from the video frame buffer and can decode the video frame B.

[0080] It can be seen that in this embodiment, the non-target service thread can specifically be used to read the video frame from the video frame buffer, determine whether the target service identifier exists in the packet header of the video frame, and if not, decode the video frame. Since the non-target service thread can decode the video frame when it is determined that the target service identifier does not exist in the packet header of the video frame, and the target service thread decodes the video frame when it is determined that the target service identifier exists in the packet header of the video frame, there is no need to separately apply for a buffer dedicated to the service corresponding to the target service thread, and the reuse of the buffer can be realized. Furthermore, the number of video buffers can be reduced, and thus the memory can be saved.

[0081] As an implementation manner of an embodiment of the present application, the above video parsing thread can also be used to move the video frame with the target service identifier in the packet header from the video frame buffer to the return buffer before the target service thread reads the video frame with the target service identifier from the video frame buffer; and trigger the target service thread to read the video frame with the target service identifier in the packet header from the return buffer to decode the video frame with the target service identifier in the packet header.

[0082] If the video frames parsed by the video parsing thread are relatively large, they will occupy a large amount of space in the video frame buffer. If the remaining storage space in the video frame buffer is insufficient, then the subsequently stored video frames may partially overwrite the occupied space of the video frames stored at the front, so that the video frames stored at the front are no longer complete frames. Therefore, when there is a target service identifier in the video frame header parsed, the video parsing thread can move the video frame from the video frame buffer to the look-back buffer, timely release the space of the video frame buffer, and reduce the space occupied by the video frames for the target service in the video frame buffer. Among them, the look-back buffer can store the video frames for the services processed by the target service thread.

[0083] If the video parsing thread moves the video frame with the target service identifier from the video frame buffer to the look-back buffer, the indexes of the remaining video frames stored in the video frame buffer will be updated so that the video frames can be arranged in order.

[0084] The video parsing thread moving the video frame with the target service identifier from the video frame buffer to the look-back buffer can trigger the target service thread to read the video frame with the target service identifier in the header from the look-back buffer to decode the video frame with the target service identifier in the header.

[0085] For example, the target service thread is used to process the zero-channel screenshot service. Video frame A is stored in the video frame buffer. The video parsing thread determines that there is a zero-channel screenshot service identifier in the header of the parsed video frame A, and can move video frame A from the video frame buffer to the look-back buffer. The indexes of the remaining video frames in the video frame buffer will be updated. At the same time, the video parsing thread triggers the target service thread to read video frame A with the target service identifier in the header from the look-back buffer to decode video frame A with the target service identifier in the header.

[0086] It can be seen that in this embodiment, the video parsing thread can also be used to move the video frame with the target service identifier in the header from the video frame buffer to the look-back buffer before the target service thread reads the video frame with the target service identifier from the video frame buffer; and trigger the target service thread to read the video frame with the target service identifier in the header from the look-back buffer to decode the video frame with the target service identifier in the header. Since the video parsing thread can move the video frame with the target service identifier in the header from the video frame buffer to the look-back buffer, enabling the target service thread to read the video frame with the target service identifier in the header from the look-back buffer, it can ensure the integrity of the video frame and timely release the used space of the video frame buffer, thereby ensuring the efficiency of the non-target service thread in processing services.

[0087] As an implementation manner of the embodiment of the present application, the above video parsing thread can be specifically used to receive the video stream sent by the stream sending end as the video stream to be processed. The video stream is obtained by the stream sending end encapsulating the video data to be sent. When the video data to be sent is used for the target service, the stream sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

[0088] The video parsing thread can receive the video stream sent by the stream sending end and use the video stream as the video stream to be processed. The stream sending end is the end that sends the data stream, which can be a network camera, a camera, a hard disk, etc., and is not specifically limited here.

[0089] In one implementation manner, when the video data to be sent is used for the target service, the stream sending end can add the channel number corresponding to the target service to the packet header corresponding to the video data to be sent as the target service identifier when encapsulating the video data to be sent. When the video parsing thread determines that the channel number is the channel number corresponding to the target service, it triggers the target service thread to read the video frame with the channel number from the video frame buffer.

[0090] For example, the video processing device is an NVR, the target service thread is used to process the zero-channel capture service, the zero-channel capture service corresponds to the channel number A, and the video data A to be sent is used for the zero-channel capture service. The NVR is connected to a network camera. At time t1, the network camera performs PS encapsulation on the video data A to be sent. Since the video data A to be sent is used for the zero-channel capture service, the network camera can add the channel number A corresponding to the zero-channel capture service to the packet header corresponding to the video data A to be sent as the zero-channel capture service identifier when encapsulating the video data A to be sent, and then obtain the video stream A. When the video parsing thread receives the video stream A sent by the network camera, it can use the video stream A with the zero-channel capture service identifier in the packet header as the video stream to be processed.

[0091] It can be seen that in this embodiment, the video parsing thread can be specifically used to receive the video stream sent by the stream sending end as the video stream to be processed. The video stream is obtained by the stream sending end encapsulating the video data to be sent. When the video data to be sent is used for the target service, the stream sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent. Since the stream sending end can add a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent, it provides a basis for the subsequent video parsing thread to determine whether there is a target service identifier in the packet header of the parsed video frame. Furthermore, when it is determined that the video frame has a target service identifier, the target service thread can be triggered to read the video frame from the video frame buffer for decoding.

[0092] As an implementation manner of an embodiment of the present application, the above-mentioned video sending end can be specifically used to add a target service identifier to the packet header corresponding to the video data to be sent every preset time period; or, it is specifically used to add a target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or, it is specifically used to perform target detection on the video data to be sent and add a target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data in which a target object is detected.

[0093] When adding a target service identifier to the packet header corresponding to the video data to be sent, the video sending end can add it according to a certain period or according to a certain logic.

[0094] In one implementation manner, the video sending end can add a target service identifier to the packet header corresponding to the video data to be sent every preset time period. Among them, the preset time period can be 5 minutes, 10 minutes, etc., and no specific limitation is made here.

[0095] For example, when a video processing device accesses network camera A and network camera A monitors in real time, a zero-channel snapshot service identifier can be added to the packet header corresponding to the video data to be sent every 10 minutes.

[0096] In one implementation manner, the video sending end can add a target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames. Among them, the preset number can be 10, 20, etc., and no specific limitation is made here.

[0097] For example, when a video processing device accesses network camera B and network camera B monitors in real time, a zero-channel snapshot service identifier can be added to the packet header corresponding to the video data to be sent every 20 video frames.

[0098] In one implementation manner, the video sending end can perform target detection on the video data to be sent and add a target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data in which a target object is detected.

[0099] For example, when a video processing device accesses camera A and camera A captures pictures at preset intervals, when a human or vehicle image is detected in the captured pictures, a zero-channel snapshot service identifier can be added to the packet header of the video data to be sent in which the human or vehicle image is detected.

[0100] Another example is that when a video parsing thread receives a video stream to be processed from a hard disk and the hard disk sends the video stream to be processed, a zero-channel snapshot service identifier can be added to the first frame image of the video stream used for the zero-channel snapshot service.

[0101] It can be seen that in this embodiment, the stream sending end can be specifically configured to add a target service identifier to the packet header corresponding to the video data to be sent at every preset time interval; or, it can be specifically configured to add a target service identifier to the packet header corresponding to the video data to be sent at every preset number of video frames; or, it can be specifically configured to perform target detection on the video data to be sent and add a target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data in which a target object is detected. Since the stream sending end can add a target service identifier to the packet header corresponding to the video data to be sent, the video parsing thread can determine whether there is a target service identifier in the packet header of the video frame, and then trigger the target service thread to read the video frame with the target service identifier from the video frame buffer for decoding when it is determined that there is a target service identifier in the packet header of the video frame.

[0102] As an implementation manner of the embodiment of the present application, the above non-target service thread is used to process the video playback service, where the video channel of the video playback service can be one or multiple.

[0103] The video parsing thread can also be used to, before storing the video stream to be processed in the download buffer corresponding to the target video channel, use the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed when there is an idle channel; when there is no idle channel, use the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing the video playback service.

[0104] When the video playback service has multiple video channels, there may be idle channels among the video channels of the video playback service. If there is an idle channel, then the video parsing thread can use the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed.

[0105] For example, the video playback service has multiple video channels, namely video channel A, video channel B, and video channel C. Among them, the free space of the download buffer corresponding to video channel A is 20%, the free space of the download buffer corresponding to video channel B is 60%, and video channel C is an idle channel. Then, the video parsing thread can use the download buffer corresponding to the idle channel C as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to video channel B as the download buffer for storing the video stream to be processed.

[0106] If there is no idle channel in the video channels of the video playback service, then the video parsing thread can use the download buffer corresponding to the channel with the non-lowest free space as the download buffer for storing the video stream to be processed. That is to say, the download buffer corresponding to the video channel with the highest free space can be used as the download buffer for storing the video stream to be processed, or the download buffer corresponding to the video channel with relatively high free space can be used as the download buffer for storing the video stream to be processed, and no specific limitation is made here.

[0107] For example, the video playback service has multiple video channels, namely video channel A, video channel B, and video channel C. Among them, there is no idle channel in video channel A, video channel B, and video channel C. The free space of the download buffer corresponding to video channel A is 50%, the free space of the download buffer corresponding to video channel B is 60%, and the free space of the download buffer corresponding to video channel C is 30%. Then, the video parsing thread can use the download buffer corresponding to video channel B as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to video channel A as the download buffer for storing the video stream to be processed.

[0108] It can be seen that in this embodiment, the video parsing thread can also be used to, before storing the video stream to be processed in the download buffer for parsing, in the case of having an idle channel, use the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed. In the case of having no idle channel, use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that has not processed the current video playback service. Since the download buffer corresponding to the idle channel of the video playback service, or the download buffer corresponding to the video channel with non-lowest free space can be used as the download buffer for the video stream to be processed, when the video parsing thread determines that the packet header of the video frame has the target service identifier, it triggers the target service thread to read the video frame with the target service identifier from the video frame buffer for decoding. Therefore, the target service thread can reuse the download buffer of the video channel in the non-target service thread without separately applying for a dedicated buffer to parse the video stream corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0109] Corresponding to the above first video processing device, an embodiment of the present application further provides a video processing method. The following introduces a video processing method provided by the embodiment of the present application.

[0110] Such as Figure 2As shown, a video processing method is applied to a video processing device. The video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread. The method includes:

[0111] S201, the video parsing thread obtains the video stream to be processed;

[0112] S202, the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel;

[0113] S203, the video parsing thread reads the video stream to be processed from the download buffer and parses the video stream to obtain video frames;

[0114] S204, the video parsing thread stores the video frames in the video frame buffer corresponding to the target video channel;

[0115] Wherein, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer;

[0116] S205, when the video frames obtained by parsing have a target service identifier, the target service thread reads the video frames with the target service identifier from the video frame buffer and decodes the video frames with the target service identifier;

[0117] S206, the non-target service thread reads the video frames from the video frame buffer and decodes the video frames when the video frames do not have a target service identifier.

[0118] It can be seen that in the solution provided by the embodiments of the present application, the video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread. Among them, the video parsing thread is used to obtain the video stream to be processed, store the video stream to be processed in the download buffer corresponding to the target video channel, read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames, and store the video frames in the video frame buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer. The target service thread is used to read the video frames with the target service identifier from the video frame buffer when the parsed video frames have the target service identifier, and decode the video frames with the target service identifier. The non-target service thread is used to read the video frames from the video frame buffer and decode the video frames when the video frames do not have the target service identifier. Since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with the target service identifier from the video frame buffer for decoding, and the non-target service thread can also read the video frames from the video frame buffer to decode the video frames without the target service identifier. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0119] As an implementation manner of the embodiments of the present application, as Figure 3 shown, before the target service thread reads the video frames with the target service identifier from the video frame buffer, the above method may further include:

[0120] S301, the video parsing thread determines whether there is a target service identifier in the packet header of the parsed video frame; if there is a target service identifier, execute step S302; if there is no target service identifier, execute step S303;

[0121] S302, the video parsing thread triggers the target service thread to read the video frames with the target service identifier from the video frame buffer;

[0122] S303, the video parsing thread determines that there is no target service identifier in the packet header of the parsed video frame.

[0123] As an implementation manner of the embodiments of the present application, as Figure 4As shown in the figure, the step in which the above non-target service thread reads the video frame from the video frame buffer and decodes the video frame when the video frame does not have a target service identifier may include:

[0124] S401, the non-target service thread reads the video frame from the video frame buffer;

[0125] S402, the non-target service thread determines whether the target service identifier exists in the packet header of the video frame. If the target service identifier does not exist, step S403 is executed; if the target service identifier exists, step S404 is executed;

[0126] S403, the non-target service thread decodes the video frame;

[0127] S404, the non-target service thread determines that the target service identifier exists in the packet header of the video frame.

[0128] As an implementation manner of an embodiment of the present application, before the target service thread reads the video frame with the target service identifier from the video frame buffer, the above method may further include:

[0129] The video parsing thread moves the video frame with the target service identifier in the packet header from the video frame buffer to the look-back buffer; and triggers the target service thread to read the video frame with the target service identifier in the packet header from the look-back buffer to decode the video frame with the target service identifier in the packet header.

[0130] As an implementation manner of an embodiment of the present application, the step in which the above video parsing thread obtains the video stream to be processed may include:

[0131] The video parsing thread receives the video stream sent by the video sending end as the video stream to be processed, where the video stream is obtained by encapsulating the video data to be sent by the video sending end. When the video data to be sent is used for the target service, the video sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

[0132] As an implementation manner of an embodiment of the present application, the step of adding the target service identifier to the packet header corresponding to the video data to be sent may include:

[0133] The video sending end adds the target service identifier to the packet header corresponding to the video data to be sent every preset time period; or,

[0134] Add the target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or,

[0135] Perform object detection on the video data to be sent, and add a target service identifier to the header of the target video data to be sent, where the target video data to be sent is the video data in which the target object is detected.

[0136] As an implementation manner of the embodiment of the present application, the non-target service thread is used to process the video playback service;

[0137] Before the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel, the method may further include:

[0138] When there is an idle channel, the video parsing thread uses the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or uses the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed; when there is no idle channel, the download buffer corresponding to the video channel with the non-lowest free space is used as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing the video playback service.

[0139] Among the numerous processing services of the video stream, there are zero-channel capture services and ordinary playback services. The zero-channel capture service is the service processed by the target service thread, and the ordinary playback service is the service processed by the non-target service. The zero-channel capture service and the ordinary playback service both use the same buffer to process the video stream.

[0140] Among them, the zero-channel capture service refers to a service process of an NVR or DVR, which is used to process the decoding of I frames into YUV (Luminance Chrominance Chroma) data, convert the YUV data into ARGB1555 (16 bits per pixel point) data and display it on the interface, or re-encode the YUV data after I frame decoding into MJPEG data and store it in the hard disk for subsequent viewing. To process the zero-channel capture service, a code stream download buffer for receiving the code stream and a decoded frame assembly buffer for storing the parsed code stream are applied for this service. The code stream can be a PS (Program Stream) code stream, etc. The service processing flow of the zero-channel capture service is as Figure 5 shown and may include:

[0141] S501, The APP issues a PS code stream;

[0142] For example, the APP (Application) can be the application program of the camera, and the application program of the camera can encapsulate the I frame data of the video data into a PS code stream and transmit it to the NVR device.

[0143] S502, Zero-channel capture bitstream download buffer;

[0144] The zero-channel capture bitstream download buffer can be used to receive the PS bitstream transmitted by the APP.

[0145] S503, Bitstream extraction and parsing, zero-channel capture decoding and frame assembly buffer;

[0146] The zero-channel capture service thread can read the PS bitstream from the bitstream download buffer, call the parsing library to parse the PS bitstream, and store the parsed video frames in the decoding and frame assembly buffer.

[0147] S504, Bitstream decoding and delivery, create a decoder and decode;

[0148] The video frames obtained by the zero-channel capture service thread parsing the PS bitstream are decoded and delivered, and are allocated to the decoder for decoding;

[0149] S505, Image processing, return the decoded YUV data and convert it into an RGB or JPEG encoded image and return it to the APP.

[0150] The service processing flow of the normal playback service is generally the same as that of the zero-channel capture service. Among them, the service processing flow of the normal playback service will create two threads (a thread is the smallest unit for the operating system to perform operation scheduling). One thread is used to receive the PS bitstream and perform parsing and processing. After stripping the PS bitstream header, the raw bitstream data is stored in the decoding and frame assembly buffer, and the other thread fetches the bitstream data from the decoding and frame assembly buffer and sends it to the decoder for decoding and display. As Figure 6 shown, the service processing flow of the normal playback service can include:

[0151] S601, The APP issues the PS bitstream;

[0152] For example, the application program of the camera can encapsulate the video data into a PS bitstream for transmission;

[0153] S602, Normal playback channel bitstream download buffer;

[0154] The normal playback channel bitstream download buffer can be used to receive the PS bitstream.

[0155] S603, Bitstream extraction and parsing, normal playback channel decoding and frame assembly buffer;

[0156] The normal service playback thread can read the PS bitstream from the bitstream download buffer, call the parsing library to parse the PS bitstream, and store the parsed video frames in the decoding and frame assembly buffer.

[0157] S604, Fetch the stream from the decoded frame buffer for decoding;

[0158] The normal service playback thread can read video frames from the decoded frame buffer for decoding.

[0159] S605, Create a decoder and decode;

[0160] S606, Decode and display, and display the decoded YUV picture on the video layer.

[0161] From Figure 5 and Figure 6 it can be seen that Figure 5 in the zero-channel screenshot service processing flow shown in Figure 6 the 510 part and Figure 7 in the normal playback service processing flow shown in

[0162] the 610 part are the same, and the PS stream received by the stream download buffer is the same. The decoded frame buffer stores raw stream data frame by frame. The data in the decoded frame buffer is as shown in

[0163] Among them, in front of each frame of raw stream data ES (Elementary Stream, basic stream or raw stream) data, there is a Group header, that is, Group frame information data. The relevant information of this frame of raw stream data is stored in the Group frame information data, such as the stream type (I frame, P frame, etc.), stream length, etc.

[0164] Figure 8 This is a specific flowchart of the video processing method provided by the embodiment of the present application. The following combines Figure 8 to introduce the video processing method provided by the embodiment of the present application by way of example. In the process of processing the video stream for services, three threads can be included, namely a video parsing thread, a target service processing thread, and a non-target service processing thread. Among them, the target service processing thread is used to process the zero-channel screenshot service, and the non-target service processing thread is used to process the normal playback service.

[0165] As shown in Figure 8 the video processing method provided by the embodiment of the present application may include the following steps:

[0166] S801, The APP issues a PS stream;

[0167] The APP can send the received PS stream (i.e., the video stream) to the stream download buffer corresponding to the video channel of the ordinary playback service thread. Among them, after the streaming end waits for the video data of a GOP (Group of Picture) to be sent completely, the video data encapsulated in the PS stream is stored in the stream download buffer of the ordinary playback channel.

[0168] S802, the stream download buffer of the ordinary playback channel;

[0169] The stream download buffer of the ordinary playback channel can receive the PS stream sent by the APP.

[0170] S803, fetch stream and parse, the decoding and frame-assembling buffer of the ordinary playback channel;

[0171] After the video parsing thread fetches and parses the PS stream in the stream download buffer of the ordinary playback channel, it can obtain frame-by-frame raw stream data, that is, video frames, and store the parsed video frames into the decoding and frame-assembling buffer corresponding to the video channel of the ordinary playback service thread.

[0172] S804, judge whether the stream is used for zero-channel capture through PSH;

[0173] The video parsing thread can obtain video frames by parsing the PS stream, determine whether there is a zero-channel capture service identifier in the packet header of the video frame, and then judge whether the stream is used for the zero-channel capture service thread. If so, execute step S805; if not, the processing of the next PS stream can be carried out.

[0174] S805, inform the zero-channel capture service thread to process;

[0175] If it is determined that there is a zero-channel capture service identifier in the packet header of the video frame, that is, the PS stream is used for the zero-channel capture service, the video parsing thread can trigger the zero-channel capture service thread to fetch and decode the stream from the decoding and frame-assembling buffer.

[0176] S806, the zero-channel capture service thread fetches and decodes the stream from the decoding and frame-assembling buffer;

[0177] The zero-channel capture service thread can read the video frames obtained by parsing the PS stream from the decoding and frame-assembling buffer and decode the video frames.

[0178] S807, copy the data to the backward frame-assembling buffer;

[0179] The video parsing thread can copy the video frames used for the zero-channel capture service to the backward frame-assembling buffer to release the space of the decoding and frame-assembling buffer. Furthermore, the zero-channel capture service thread can read the video frames from the backward frame-assembling buffer and decode the video frames.

[0180] S808, Create a decoder and decode;

[0181] The zero-channel screenshot service thread will create a decoder and decode the video frames used for the zero-channel screenshot service.

[0182] S809, Decode and display, display the decoded YUV picture on the video layer;

[0183] The zero-channel screenshot service thread can display the decoded YUV picture on the video layer.

[0184] S810, Normal playback retrieves the stream from the decoded frame buffer for decoding;

[0185] The video parsing thread can determine whether there is a zero-channel screenshot service flag in the packet header of the video frame. The normal playback service thread can also determine whether there is a zero-channel screenshot service flag in the packet header of the video frame. When it is determined that there is no zero-channel screenshot service flag in the packet header of the video frame, the stream is retrieved from the decoded frame buffer for decoding.

[0186] S811, Determine whether it is a playback stream by the group header. If not, wait for the zero-channel screenshot service thread to finish processing before proceeding;

[0187] The normal playback service thread can determine whether there is a zero-channel screenshot service flag in the packet header of the video frame, and then determine whether the PS stream is a playback stream. If it is determined that there is no zero-channel screenshot service flag in the packet header of the video frame, the video frame can be decoded without waiting for the screenshot service thread to finish processing.

[0188] S812, Create a decoder and decode;

[0189] If the normal playback service thread determines that the PS stream is for normal playback service, it can create a decoder and decode.

[0190] S813, Decode and display, display the decoded YUV picture on the video layer.

[0191] The normal playback service thread can display the decoded YUV picture on the video layer.

[0192] It can be seen that in this embodiment, the video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread. Among them, the video parsing thread is used to obtain the video stream to be processed, store the video stream to be processed in the download buffer corresponding to the target video channel, read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames, and store the video frames in the video frame buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer. The target service thread is used to read the video frames with the target service identifier from the video frame buffer when the parsed video frames have the target service identifier, and decode the video frames with the target service identifier. The non-target service thread is used to read the video frames from the video frame buffer and decode the video frames when the video frames do not have the target service identifier. Since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with the target service identifier from the video frame buffer for decoding, and the non-target service thread can also read the video frames from the video frame buffer to decode the video frames without the target service identifier. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0193] The embodiment of the present application also provides a video processing device, as Figure 9 shown, including:

[0194] A memory 901 for storing a computer program;

[0195] A video processor 902, when executing the program stored on the memory 901, implements the video processing method described in any of the above embodiments.

[0196] And the above video processing device may further include a communication bus and / or a communication interface, and the video processor 902, the communication interface, and the memory 901 complete communication with each other through the communication bus.

[0197] It can be seen that in the solution provided by the embodiments of the present application, the video processing device includes a video processor. The video processor runs a video parsing thread, a target service thread, and a non-target service thread. Among them, the video parsing thread is used to obtain the video stream to be processed, store the video stream to be processed in the download buffer corresponding to the target video channel, read the video stream to be processed from the download buffer, and parse the video stream to be processed to obtain video frames, and store the video frames in the video frame buffer corresponding to the target video channel. Among them, the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer. The target service thread is used to, when the parsed video frames have a target service identifier, read the video frames with the target service identifier from the video frame buffer and decode the video frames with the target service identifier. The non-target service thread is used to read video frames from the video frame buffer and, when the video frames do not have a target service identifier, decode the video frames. Since the video parsing thread can store the video stream to be processed in the download buffer and store the video frames obtained by parsing the video stream to be processed in the video frame buffer, the target service thread can read the video frames with the target service identifier from the video frame buffer for decoding, and the non-target service thread can also read video frames from the video frame buffer to decode the video frames without the target service identifier. Therefore, both the target service thread and the non-target service thread can use the download buffer and the video frame buffer corresponding to the target video channel without separately applying for a buffer dedicated to the service corresponding to the target service thread, which can reduce the number of video buffers and thus save memory.

[0198] The communication bus mentioned in the above video processing device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0199] The communication interface is used for communication between the above video processing device and other devices.

[0200] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0201] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0202] In another embodiment provided by the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video processing method described in any of the above embodiments are implemented.

[0203] In another embodiment provided by the present application, a computer program product containing instructions is further provided. When it runs on a computer, it causes the computer to execute the video processing method described in any of the above embodiments.

[0204] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0205] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0206] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for methods, computer-readable storage media, and computer program products, since they are basically similar to the device embodiments, the description is relatively simple, and reference can be made to the relevant parts of the device embodiments for the relevant content.

[0207] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included within the protection scope of the present application.

Claims

1. A video processing device, characterized in that, The video processing device includes a video processor, and the video processor runs a video parsing thread, a target service thread, and a non-target service thread, where: The video parsing thread is used to obtain a video stream to be processed; store the video stream to be processed in the download buffer corresponding to the target video channel; read the video stream to be processed from the download buffer, and parse the video stream to obtain video frames; store the video frames in the video frame buffer corresponding to the target video channel, where the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer; The target service thread is used to, when the parsed video frames have a target service identifier, read the video frames with the target service identifier from the video frame buffer, and decode the video frames with the target service identifier; The non-target service thread is used to read the video frames from the video frame buffer, and decode the video frames when the video frames do not have a target service identifier.

2. The device according to claim 1, wherein The video parsing thread is further used to determine whether a target service identifier exists in the packet header of the parsed video frames before the target service thread reads the video frames with the target service identifier from the video frame buffer; if it exists, trigger the target service thread to read the video frames with the target service identifier from the video frame buffer.

3. The device according to claim 1, wherein The non-target service thread is specifically used to read the video frames from the video frame buffer; determine whether the target service identifier exists in the packet header of the video frames; if not, decode the video frames.

4. The device according to claim 1, wherein The video parsing thread is further used to move the video frames with a target service identifier in the packet header from the video frame buffer to the return buffer before the target service thread reads the video frames with the target service identifier from the video frame buffer; and trigger the target service thread to read the video frames with the target service identifier in the packet header from the return buffer to decode the video frames with the target service identifier in the packet header.

5. The device according to any one of claims 1-4, wherein The video parsing thread is specifically used to receive the video stream sent by the stream sending end as the video stream to be processed, where the video stream is obtained by the stream sending end encapsulating the video data to be sent, and when the video data to be sent is for the target service, the stream sending end adds a target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

6. The device according to claim 5, wherein The stream sending end is specifically used to add a target service identifier to the packet header corresponding to the video data to be sent every preset time period; or, Specifically used to add a target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or, Specifically used to perform target detection on the video data to be sent, and add a target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data in which a target object is detected.

7. The device according to any one of claims 1-4, characterized in that The non-target service thread is used to process the video playback service; The video parsing thread is further used to, before the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel, in the case of having an idle channel, use the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed; in the case of not having an idle channel, use the download buffer corresponding to the video channel with non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing the video playback service.

8. A video processing method, characterized in that, The method is applied to a video processing device, the video processing device includes a video processor, the video processor runs a video parsing thread, a target service thread and a non-target service thread, and the method includes: The video parsing thread obtains the video stream to be processed; stores the video stream to be processed in the download buffer corresponding to the target video channel; reads the video stream to be processed from the download buffer, and parses the video stream to be processed to obtain video frames; stores the video frames in the video frame buffer corresponding to the target video channel, where the download buffer corresponding to the target video channel has free space, and the free space is the remaining storage space of the download buffer; The target service thread, in the case that the parsed video frame has a target service identifier, reads the video frame with the target service identifier from the video frame buffer, and decodes the video frame with the target service identifier; The non-target service thread reads the video frame from the video frame buffer, and in the case that the video frame does not have a target service identifier, decodes the video frame.

9. The method according to claim 8, characterized in that, Before the target service thread reads the video frame with the target service identifier from the video frame buffer, the method further includes: The video parsing thread determines whether there is a target service identifier in the packet header of the parsed video frame; if so, triggers the target service thread to read the video frame with the target service identifier from the video frame buffer.

10. The method according to claim 8, characterized in that The step that the non-target service thread reads the video frame from the video frame buffer, and in the case that the video frame does not have a target service identifier, decodes the video frame, includes: The non-target service thread reads the video frame from the video frame buffer; determines whether there is the target service identifier in the packet header of the video frame; if not, decodes the video frame.

11. The method according to claim 8, characterized in that, Before the target service thread reads the video frame with the target service identifier from the video frame buffer, the method further includes: The video parsing thread moves the video frame with the target service identifier in the packet header from the video frame buffer to the return buffer; and triggers the target service thread to read the video frame with the target service identifier in the packet header from the return buffer to decode the video frame with the target service identifier in the packet header.

12. The method according to any one of claims 8-11, characterized in that, The steps for the video parsing thread to obtain the video stream to be processed include: The video parsing thread receives the video stream sent by the video sending end as the video stream to be processed, where the video stream is obtained by encapsulating the video data to be sent by the video sending end. When the video data to be sent is for the target service, the video sending end adds the target service identifier to the packet header corresponding to the video data to be sent when encapsulating the video data to be sent.

13. The method according to claim 12, wherein The step of adding the target service identifier to the packet header corresponding to the video data to be sent includes: The video sending end adds the target service identifier to the packet header corresponding to the video data to be sent every preset time period; or, adds the target service identifier to the packet header corresponding to the video data to be sent every preset number of video frames; or, performs target detection on the video data to be sent, and adds the target service identifier to the packet header of the target video data to be sent, where the target video data to be sent is the video data for which a target object is detected.

14. The method according to any one of claims 8-11, characterized in that, The non-target service thread is used to process the video playback service; Before the video parsing thread stores the video stream to be processed in the download buffer corresponding to the target video channel, the method further includes: When there is an idle channel, the video parsing thread uses the download buffer corresponding to the idle channel as the download buffer for storing the video stream to be processed, or uses the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed; when there is no idle channel, uses the download buffer corresponding to the video channel with the non-lowest free space as the download buffer for storing the video stream to be processed, where the idle channel is the video channel that is not currently processing the video playback service.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 8-14 is implemented.

Images