Video frame sharing access method, storage medium and video equipment

Abstract

The invention provides a video frame sharing access method, a storage medium and video equipment. The method comprises the following steps: a producer process writes a frame of video frame data corresponding to a target video source into a target shared memory according to a production annular queue and a data writing index; the target consumption process responds to a preset judgment rule and judges whether video frame data which is not consumed by the target consumption process exists in the target shared memory or not according to a target queue index in the target consumption context data, and if yes, video frame data which is not consumed by the target consumption process are stored in the target shared memory according to the target queue index and a target consumption annular queue in the target consumption context data; and determining a target to-be-read memory identifier, accessing a target shared memory corresponding to the target to-be-read memory identifier in a read-only manner, and obtaining target video frame data. According to the method and the device, on the basis of ensuring that the production real-time performance of the video frame is not influenced by the consumption speed and different consumption task processes are not mutually influenced, the occupation amount of CPU resources and memory resources is reduced.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video frame shared access method, a storage medium and a video device. Background Art

[0002] In the field of video processing, video devices may include video acquisition devices and video processing devices. Video acquisition devices can be used to acquire real-time video streams. Video processing devices such as NVR (Network Video Recorder), AI (Artificial Intelligence) all-in-one devices, etc. can be connected to at least one video input device / video acquisition device, and decode the input real-time video stream, and use the decoded real-time video frames for different tasks such as playback display, image processing, and AI reasoning. In this process, the process of acquiring real-time video streams through video acquisition devices or the process of decoding and outputting real-time video streams through video processing devices is the production process of real-time video frames, and the corresponding acquisition module or decoding module is the production module. The tasks of playing and displaying real-time video frames, image processing, AI reasoning, etc. can be regarded as consumption processes, and the processing modules corresponding to these tasks are consumption modules or consumption applications.

[0003] Due to the limitation of hardware cost and other factors, the processing power, memory capacity and other resources of the CPU (Central Processing Unit) of the video processing device are severely limited, resulting in the number of video streams that the video processing device can decode simultaneously being limited. However, since the production and consumption of real-time video frames are in a one-to-many relationship, that is, the real-time video frames corresponding to one video stream can be consumed by N consumer tasks (N is a positive integer, and N>1), and the N consumer tasks are independent of each other and may be different applications / processes. If independent decoding is performed for each consumer task, the video processing device needs to decode the same video stream N times, which is easy to exhaust the hardware resources of the video processing device. Therefore, a separate producer process is generally used to decode the video stream uniformly, and the real-time video frames decoded by the producer process are provided to multiple consumer task processes for processing through cross-process data exchange and other methods.

[0004] However, the inventors have found through research that the prior art consumes a large amount of memory resources and CPU resources when implementing cross-process access to real-time video frames. Summary of the invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defects in the prior art that require a large amount of memory resources and CPU resources.

[0006] In a first aspect, an embodiment of the present application provides a video frame shared access method, which is applied to a video device, wherein the video device runs a producer process and N consumer task processes for the same target video source, and the N consumer task processes correspond one to one with N consumer task context data; the method comprises:

[0007] The producer process determines the memory identifier to be written according to the production circular queue and the data writing index; wherein the production circular queue includes memory identifiers corresponding to M target shared memories, and the target shared memory is a frame buffer corresponding to the target video source;

[0008] The producer process writes a frame of video frame data corresponding to the target video source into a target shared memory based on the memory identifier to be written;

[0009] In response to satisfying a preset judgment rule, the target consumption process determines whether there is video frame data in the target shared memory that has not been consumed by the target consumption process according to the target queue index in the target consumption context data; wherein the target consumption process is any of the consumption task processes, and the target consumption context data is the consumption task context data corresponding to the target consumption process;

[0010] When the target consumer process determines that there is video frame data in the target shared memory that has not been consumed by the target consumer process, the target consumer process determines the target memory identifier to be read according to the target queue index and the target consumer circular queue in the target consumer context data; wherein the queue element of the target consumer circular queue is the same as the queue element of the production circular queue;

[0011] The target consumption process accesses the target shared memory corresponding to the target to-be-read memory identifier in a read-only manner, and obtains the target video frame data.

[0012] In some of these embodiments, the queue index for consuming task context data includes a head index;

[0013] The producer process determines the memory identifier to be written according to the production ring queue and the data writing index, including:

[0014] The producer process determines whether there is at least one process that needs to drop frames among the N consumer task processes according to the data writing index and the header indexes corresponding to the N consumer task processes;

[0015] When determining that there is at least one process that needs to drop frames, the producer process performs a CAS atomic operation of comparison plus one on the head index corresponding to each process that needs to drop frames, and obtains a first atomic operation result;

[0016] The producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue.

[0017] In some embodiments, the producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue, including:

[0018] The producer process determines a first initial memory identifier in a queue element of the production ring queue based on the data write index when the result of the first atomic operation is a successful operation or when the process that needs to drop frames does not exist among the N consumer task processes;

[0019] The producer process compares the first initial memory identifier with the to-be-read memory identifiers corresponding to the N consumer task processes. If the first initial memory identifier is the same as the to-be-read memory identifier corresponding to at least one of the consumer task processes, the memory identifier of the unqueued shared memory is used as the to-be-written memory identifier; wherein the unqueued shared memory is a target shared memory whose memory identifier is different from each current queue element of the production circular queue and whose memory identifier is different from the to-be-read memory identifier corresponding to each of the consumer task processes;

[0020] When the producer process uses the memory identifier of the unqueued shared memory as the memory identifier to be written, it updates the memory identifier to be written to the production ring queue and the N consumer ring queues of the consumer task context data according to the data write index.

[0021] In some embodiments, the producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue, further comprising:

[0022] When the first initial memory identifier is different from the memory identifier to be read corresponding to each of the consumer task processes, the producer process uses the first initial memory identifier as the memory identifier to be written.

[0023] In some embodiments, determining a target memory identifier to be read according to the target queue index and the target consumption ring queue in the target consumption context data includes:

[0024] The target consumption process determines the second initial memory identifier in the target consumption circular queue according to the target header index, performs a CAS atomic operation of comparison plus one on the target header index, and obtains a second atomic operation result;

[0025] When the result of the second atomic operation is a success, the target consumer process uses the second initial memory identifier as the target memory identifier to be read; and when the result of the second atomic operation is a failure, the target consumer process uses the preset invalid identifier as the target memory identifier to be read.

[0026] In some embodiments, after obtaining the target video frame data, the method further includes:

[0027] The target consumption process performs data processing on the target video frame data, and when the processing is completed, sets the target to-be-read memory flag to the preset invalid flag.

[0028] In some embodiments, the consumption task context data is stored in a shared memory; the method further comprises:

[0029] The producer process responds to a shutdown request sent by the consumer process to be closed by deleting the consumer task context data corresponding to the consumer process to be closed from the shared memory; wherein the consumer process to be closed is a consumer task process that needs to stop accessing the video frame data corresponding to the target video source.

[0030] In some embodiments, the queue index of the consumption task context data includes a tail index; and the method further includes:

[0031] After writing a frame of video frame data corresponding to the target video source into the target shared memory, the producer process performs an increment operation on the data write index, and updates the tail indexes of the N consumer task context data based on the updated data write index.

[0032] In a second aspect, an embodiment of the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video frame shared access method described in any of the above embodiments.

[0033] In a third aspect, an embodiment of the present application provides a video device, the video device comprising: one or more processors, and a memory;

[0034] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the video frame shared access method described in any of the above embodiments are executed.

[0035] The video frame sharing method, storage medium and video device provided by some embodiments of the present application have at least the following technical effects:

[0036] (1) The producer process can write a frame of video frame data into a frame buffer. N consumer task processes correspond to N consumer task context data one by one. Each consumer task process reads the video frame data of the target video source from the same batch of shared memory through the consumer task context data corresponding to the process.

[0037] Although each consumer task context data has its own consumer ring queue, each consumer ring queue and production ring queue points to the same target shared memory. Therefore, the frame buffer corresponding to the target video source can be shared and accessed among N consumer task processes in a shared memory manner. Even in the case of N ≥ 2, for the same frame of video frame data, the producer process only needs to fill a frame buffer without adding additional memory allocation and performing additional video frame content copy operations, which can greatly reduce the usage of CPU resources and memory resources.

[0038] (2) Each of the N consumer task processes independently reads and consumes data through the consumer task context data corresponding to the process, thereby avoiding mutual influence or interference between different consumer task processes. At the same time, the consumer task process opens and accesses the shared memory of the frame buffer with read-only permission, thereby ensuring that only the producer process has the right to perform data write operations on the frame buffer, and the consumer task process can only perform data read operations, which meets the requirements of one-place production and multiple-place consumption, and can further ensure that the consumer task processes will not influence each other or tamper with the video frame content.

[0039] (3) The producer process writes data based on the production ring queue and data writing index. The real-time production of video frames will not be affected by the difference in the consumption speed of real-time video frames by the consumer task process, thus ensuring the real-time production.

[0040] In summary, the present application can reduce the usage of CPU resources and memory resources while ensuring that the real-time production of video frames is not affected by the consumption speed and that different consumption task processes do not affect each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 This is an implementation architecture diagram of the existing technology;

[0043] Figure 2This is a diagram of the implementation architecture of the cross-process shared access method for real-time video frames in this application;

[0044] Figure 3 This is one of the flowcharts of the cross-process shared access method for real-time video frames in this application;

[0045] Figure 4 This is the second flow chart of the cross-process shared access method for real-time video frames in this application;

[0046] Figure 5 This is the internal structure diagram of the video device in this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The terms involved in this application are explained below.

[0049] CAS (Compare And Swap): It is a type of atomic operation that can be used to implement uninterrupted data exchange operations in multi-threaded programming, thereby avoiding data inconsistency problems caused by uncertainty in execution order and unpredictable interruptions when multiple threads simultaneously rewrite a certain data. This operation compares the value in memory with the specified data, and replaces the data in memory with the new value when the values ​​are the same.

[0050] REST (Representational State Transfer): A software architecture style for the World Wide Web that aims to facilitate information exchange between different software / programs over a network (such as the Internet). REST is a set of constraints and properties based on the Hypertext Transfer Protocol and is a software construction style designed to provide World Wide Web services.

[0051] As mentioned in the background technology, the existing technology needs to consume a lot of memory resources and CPU resources to achieve cross-process access to real-time video frames. The inventors have found that considering the different processing performances between different consumer task processes and the possible differences in speed between production and consumption, the collaborative access mechanism needs to meet the following requirements as much as possible:

[0052] (1) The production of video frames will not be delayed due to the processing speed of the consumer task process;

[0053] (2) Different consumption task processes will not affect each other or become bottlenecks;

[0054] (3) The producer process should set up a limited video frame buffer and use a frame drop mechanism to ensure that the buffer does not overflow when the buffer is exhausted.

[0055] To meet the above requirements, in the existing technology, cross-process access of real-time video frames is mostly implemented based on the shared memory mechanism and video frame double buffer queue mechanism provided by the operating system. Figure 1 In the prior art, a certain number of video frame buffers are allocated between the producer process and the consumer task process in a shared memory manner, and each buffer is uniquely numbered for easy indexing. At the same time, the prior art establishes a double buffer queue for each consumer task process, and the double buffer queues are an idle queue and an available queue. Among them, the consumer task process puts all buffer numbers into the idle queue when initialized. During the task execution process, the consumer task process waits for and obtains the buffer number filled with video frame data from the available queue, and locates the corresponding buffer accordingly, and performs task processing according to the video frame data filled in the buffer. After processing, the consumer task process can put the buffer number back into the idle queue. After acquisition or decoding, the producer process can obtain the available buffer number from the idle queue, locate the corresponding buffer accordingly, and copy the video frame data to fill it into the located buffer. After filling, the producer process can put the buffer number into the available queue. If the producer process fails to obtain the available buffer number from the idle queue after a certain time threshold, the consumer task process can be subjected to frame loss processing.

[0056] However, the inventors have found that the prior art has the following disadvantages:

[0057] (1) When there are multiple consumer task processes, the video frame buffer shared memory can only be shared and accessed between the producer process and one consumer task process, but cannot be shared between different consumer task processes. Even if N consumer task processes consume the same video stream, when caching M video frames, the number of video frame buffer shared memories required by N consumer task processes reaches M×N blocks. This shows that the existing technology consumes a lot of memory resources.

[0058] (2) Since the shared memory of the video frame buffer cannot be shared between the consumer task processes, the same video frame needs to be filled into the video frame buffer corresponding to each consumer task process by the producer process, and N consumer task processes need to copy and fill N copies. Since the video frame data volume is large, it takes a lot of CPU cycles to complete the copying and filling of the video frame.

[0059] (3) Double-buffered queues are needed between the producer process and the consumer task process to solve the problem of speed mismatch between production and consumption. Moreover, these queues need to be protected by cross-process synchronization mechanisms such as mutex or semaphore. Access to the queues needs to be locked and protected, which involves switching between user mode and kernel mode, consuming more CPU cycles.

[0060] It can be seen that the existing technology needs to consume a lot of memory resources and CPU resources when implementing cross-process access to real-time video frames.

[0061] To solve the above problems, the embodiments of the present application provide a video frame shared access method, storage medium and video device, which can reduce the usage of CPU resources and memory resources while ensuring that the real-time production of video frames is not affected by the consumption speed and that different consumption task processes do not affect each other.

[0062] In some embodiments, the present application provides a method for shared access to video frames, so that a frame buffer corresponding to the same video source can be shared and accessed among N consumer task processes in a shared memory manner, where N is a positive integer.

[0063] The method can be applied to a video device, which can be a video acquisition device or a video processing device. The video device runs a producer process and N consumer task processes for the same target video source, where N is a positive integer, and the target video source can be a video source acquired or processed by the video device. That is, the producer process is used to generate video frame data corresponding to the target video source. The consumer task process is used to perform subsequent processing based on the video frame data corresponding to the target video source, which includes but is not limited to any one or any combination of processing methods such as playback display, image processing, and AI reasoning.

[0064] It should be noted that the video device can be used to collect / process video streams corresponding to one or more target video sources, and this document does not impose specific restrictions on this. The video device can run one or more producer processes. When the video device runs a producer process, the producer process can be used to generate video frame data corresponding to one or more target video sources. When the video device runs multiple producer processes, the target video sources corresponding to any two producer processes can be the same or different. The video device can run one or more consumer task processes, and a consumer task process can be used to process video frame data corresponding to one target video source. When the video device runs multiple consumer task processes, the target video sources corresponding to any two consumer task processes can be the same or different. For the producer process and N consumer task processes of the same target video source, the video frame data can be distributed and accessed according to the video frame shared access method provided in this application.

[0065] For example, the video device may be used to process a video stream corresponding to a first video source and a video stream corresponding to a second video source. The video device may run a first consumer task process, a second consumer task process, and one or two producer processes. The first consumer task process is used to perform subsequent processing on the video frame data corresponding to the first video source, and the number of its processes may be one or more. The second consumer task process is used to perform subsequent processing on the video frame data corresponding to the second video source, and the number of its processes may be one or more.

[0066] When the number of producer processes is one, the producer process is used to generate video frame data corresponding to the first video source and video frame data corresponding to the second video source. In this case, the producer process and each first consumer task process can distribute and access the video frame data corresponding to the first video source according to the video frame sharing access method provided in this application. The producer process and each second consumer task process can distribute and access the video frame data corresponding to the second video source according to the video frame sharing access method provided in this application.

[0067] When the number of producer processes is two, the first producer process is used to generate video frame data corresponding to the first video source, and the second producer process is used to generate video frame data corresponding to the second video source. In this case, the first producer process and each first consumer task process can distribute and access the video frame data corresponding to the first video source according to the video frame sharing access method provided in this application. The second producer process and each second consumer task process can distribute and access the video frame data corresponding to the second video source according to the video frame sharing access method provided in this application.

[0068] In some examples, such as Figure 2 As shown, the producer process may include a consumption task management module, a real-time video frame production module and a real-time video frame distribution module. Among them, the consumption task management module is responsible for communicating with the consumption task process and receiving real-time video frame shared access related requests initiated by the consumption task process. Among them, shared access related requests may include interface requests such as real-time video frame access application, life cycle maintenance (Keep alive) and active access closure. Furthermore, the communication between the consumption task management module and the consumption task process can use any communication method that supports the request / response model, such as communication through a RESTful interface method based on HTTP (Hypertext Transfer Protocol), with the consumption task management module as the server and the consumption task process as the client.

[0069] The real-time video frame production module is responsible for converting image signals into video frames by regularly collecting them according to different video source device types and video source input types, or restoring video frames by reading and decoding streaming media protocols for multiple channels connected to the network streaming media protocol such as all-in-one machines.

[0070] The real-time video frame distribution module is responsible for obtaining real-time video frames from the real-time video frame production module and sharing access and control of the consumer task process received by the consumer task management module through technologies such as shared memory. For each target video source, the real-time video frame distribution module can set up a distribution context data structure, and the producer process has read and write permissions for the distribution context data structure.

[0071] Furthermore, the distribution context data structure may include a production circular queue pr_queue, a data writing index pnext and a consumer task context set.

[0072] The production ring queue pr_queue is used to record the video frame buffer identifier (i.e., the memory identifier of the target memory) corresponding to the target video source, through which the corresponding shared memory address can be located. The queue length of the production ring queue can be a preset fixed buffer number M. That is, the production ring queue can include M queue elements, and each queue element is a memory identifier corresponding to the target shared memory, and any two queue elements are different. Among them, the target shared memory refers to the frame buffer corresponding to the target video source, which is used to store the video frame data corresponding to the target video source.

[0073] The data writing index pnext is used to determine the index value of the next target shared memory to be filled.

[0074] The consumer task context set is used to record the shared memory identifiers corresponding to N consumer task context data for subsequent distribution of video frames.

[0075] It can be understood that a target shared memory can be used to store a frame of video frame data. The memory size of a single target shared memory can be determined according to actual conditions, and this application does not impose specific restrictions on this. In one example, the memory size of the target shared memory can be [16+(video frame width×video frame height×bytes per pixel)] bytes. The first 16 bytes of each target shared memory can be used to record the following data:

[0076] (1) Bytes 1 to 8 (64 bits) are used to record the video frame number. The video frame number starts at 1 and increases by 1 for each frame.

[0077] (2) Bytes 9 to 16 (64 bits) are used to record the video frame timestamp in milliseconds.

[0078] In this application, N consumption task processes correspond to N consumption task context data one by one. In one example, for a target video source, for each additional consumption task process, the real-time video frame distribution module can create a consumption task context data, and the newly created consumption task context data corresponds to the newly added consumption task process.

[0079] The consumption task context data is used to coordinate the production and consumption of video frames, and each consumption task process can consume the video frame data in the shared memory through the consumption task context data corresponding to the process. Considering that both the producer process and the consumer task process have the need to read and write the consumption task context data, in some examples, the consumption task context data can be stored in the shared memory, and both the producer process and the consumer task process have read and write permissions to the consumption task context data to facilitate cross-process operations.

[0080] In an example, each consumption task context data may include a consumption circular queue cr_queue, a head index fhead, a tail index ftail, and a to-be-read memory identifier cblk.

[0081] The consumer ring queue cr_queue is used to record the memory identifiers of all frame buffers currently accessible to the consumer task process. The production ring queue and the consumer ring queue for the same target video source are the same.

[0082] The head index fhead is the head index of the consumer ring queue, which is used to determine the memory identifier corresponding to the next video frame data consumed by the consumer task process. In an example, the head position of the consumer ring queue can be determined according to the value of mod(fhead,M) (i.e., the value of fhead modulo M). For example, fhead is 100, M is 3, and mod(fhead,M) = 1, indicating that the first element in cr_queue is the head position.

[0083] Tail index ftail, the tail index of the consumer ring queue. When fhead and ftail are the same, it means that cr_queue is empty, that is, there is no video frame data in the target shared memory that has not been consumed by the consumer task process. When the difference between ftail and fhead is M, it means that cr_queue is full, that is, there are at least M unconsumed video frame data in the target shared memory of the consumer task process. In an example, the tail position of the consumer ring queue can be determined according to the value of mod(ftail,M) (that is, the value of ftail modulo M).

[0084] The memory identifier cblk to be read is the identifier of the video frame buffer currently being processed by the consumer task process. When the memory identifier cblk to be read is a valid number (for example, cblk is not 0), it means that the video frame buffer corresponding to cblk is being used by the consumer task process.

[0085] In some embodiments, Figure 3 As shown, the video frame shared access method provided by the present application includes the following steps:

[0086] S202: The producer process determines the memory identifier to be written according to the production ring queue and the data writing index.

[0087] The production ring queue includes memory identifiers corresponding to M target shared memories, the target shared memory is a frame buffer corresponding to the target video source, and the memory identifier to be written can be the memory identifier of the next target shared memory to be filled. For the relevant description of the production ring queue and the data writing index, please refer to the above description, which will not be repeated in this article.

[0088] In this step, the production ring queue records the memory identifiers corresponding to M target shared memories. The producer process can determine the memory identifier of the next target shared memory to be filled according to the data write index and the production ring queue, and obtain the memory identifier to be written.

[0089] In one example, the producer process can calculate the value of mod(pnext,M) and determine the memory identifier to be written accordingly. That is, the value of pnext can be modulo M, and the result is the position index that needs to be filled in the next frame of video in the production ring queue pr_queue. For example, when pnext=100, M is 3, and mod(pnext,M)=1, the producer process can fill the next frame of video into the target shared memory corresponding to the first element of pr_queue, so the first element of pr_queue can be used as the memory identifier to be written.

[0090] S204: The producer process writes a frame of video frame data corresponding to the target video source into a target shared memory based on the memory identifier to be written.

[0091] In this step, when the memory identifier to be written is determined, the producer process can write a frame of video frame data corresponding to the target video source into the target shared memory corresponding to the memory identifier to be written, so as to store the video frame data of the target video source through the target shared memory.

[0092] Exemplarily, the real-time video frame distribution module can obtain the latest video frame image data corresponding to the target video source from the real-time video frame production module, and fill the frame buffer corresponding to the memory identifier to be written with the latest video frame sequence number, timestamp and image data of the video frame. It can be understood that in this example, the video frame data includes the video frame sequence number, timestamp and image data of the video frame.

[0093] Although both the distribution context data and the consumption task context data include a circular queue, the queue elements in the circular queue are all synchronously written by the real-time video frame distribution module. Regardless of the consumption speed of each consumption task process, the real-time video frame distribution module can obtain video frames according to the production speed of the real-time video frame production module and add them to the target shared memory.

[0094] S206: In response to satisfying a preset judgment rule, the target consumer process determines whether there is video frame data in the target shared memory that has not been consumed by the target consumer process according to the target queue index in the target consumer context data.

[0095] The target consumption process is any consumption task process, and the target consumption context data is the consumption task context data corresponding to the target consumption process. Each consumption task context data may include a queue index and a consumption ring queue. The target queue index refers to the queue index included in the target consumption task context, which can be used to reflect the consumption progress of the target consumption process for the video frame data.

[0096] The preset judgment rule refers to a rule for judging whether it is necessary to detect whether there is video frame data in the target shared memory that has not been consumed by the target consumer process. In one example, the preset judgment rule may be a periodic judgment, and the detection period is less than or equal to 1 / FPS (frames per second), that is, the target consumer process may periodically detect whether there is video frame data in the M target shared memories that has not been consumed by the process.

[0097] In this step, the target consumption process can determine whether there is video frame data that has not been consumed by the process in the M target shared memories according to the queue index corresponding to the process (i.e., the target queue index). It can be understood that the target consumption process can use a variety of methods to implement the judgment, and this application does not impose specific restrictions on this. In one example, the target consumption process can determine whether the target consumption queue is empty based on the target queue index, and then determine whether there is video frame data that has not been consumed by the target consumption process in the target shared memory.

[0098] Further, in one example, if there is no video frame data in the target shared memory that is not consumed by the target consumer process, the target consumer process may wait for the next detection cycle to arrive, and re-execute step S206 when the next detection cycle arrives.

[0099] It should be noted that in the present application, when N≥2, each consumption task process independently executes steps S206~S210 according to the consumption task context data corresponding to the process to control the video frame access process of the process to avoid mutual interference between the consumption task processes.

[0100] S208: When the target consumer process determines that there is video frame data in the target shared memory that has not been consumed by the target consumer process, the target consumer process determines the target to-be-read memory identifier according to the target queue index and the target consumer circular queue in the target consumer context data.

[0101] Among them, the target consumption circular queue refers to the consumption circular queue included in the target consumption context data, and the queue element of the target consumption circular queue is the same as the queue element of the production circular queue. That is, the target consumption circular queue includes the memory identifiers of M target shared memories, and the memory identifiers of the target consumption circular queue and the production circular queue at the same queue position are the same. For example, the first queue element of the production circular queue is the same as the first queue element of the target consumption circular queue, the second queue element of the production circular queue is the same as the second queue element of the target consumption circular queue, and so on. The target to-be-read memory identifier refers to the memory identifier of the target shared memory that the target consumption process reads next.

[0102] In this step, if there is video frame data in the target shared memory that has not been consumed by the target consumer process, the target consumer process can determine the memory identifier corresponding to the next video frame to be read in the M queue elements of the consumer ring queue based on the queue index corresponding to this process and the consumer ring queue corresponding to this process, and obtain the target memory identifier to be read.

[0103] For example, the target queue index may include a target header index, and the target consumer process may determine the target queue position according to the target header index, and use the queue element at the target queue position in the target consumer ring queue as the target memory identifier to be read.

[0104] S210: The target consumption process accesses the target shared memory corresponding to the target to-be-read memory identifier in a read-only manner, and obtains the target video frame data.

[0105] In this step, the consumer task process has read-only permission for the target shared memory, and the consumer task process can only open and access the target shared memory with read-only permission. Therefore, when the target memory identifier to be read is determined, the target consumer process opens and accesses the target shared memory corresponding to the target memory identifier to be read in read-only mode, and reads the target video frame data from the target shared memory. Furthermore, the target consumer process can also perform subsequent processing on the target video frame data, such as AI reasoning, image analysis, etc.

[0106] In one example, if the target consumer process does not open the target shared memory corresponding to the target memory identifier to be read, the target consumer process opens it in read-only mode and maps it to the process address space for data access.

[0107] In summary, the present application can reduce the usage of CPU resources and memory resources while ensuring that the real-time production of video frames is not affected by the consumption speed and that different consumption task processes do not affect each other.

[0108] In some embodiments, the queue index for consuming task context data includes a head index. For relevant descriptions of the head index, please refer to the above embodiments, which will not be repeated herein.

[0109] The producer process determines the memory identifier to be written based on the production ring queue and the data write index, including:

[0110] Step A1: The producer process determines whether there is at least one process that needs to drop frames among the N consumer task processes based on the data writing index and the header indexes corresponding to the N consumer task processes;

[0111] Step A2: when the producer process determines that there is at least one process that needs to drop frames, the producer process performs a CAS atomic operation of comparison plus one on the head index corresponding to each process that needs to drop frames, and obtains a first atomic operation result;

[0112] Step A3: The producer process determines the memory identifier to be written according to the first atomic operation result and the production ring queue.

[0113] Among them, the process that needs to drop frames refers to the consumer task process that needs to perform the frame drop operation. When the unconsumed video frame data of the consumer task process in the target shared memory is greater than or equal to M frames, it indicates that the consumer ring queue corresponding to the consumer task process is full. In this case, if the producer process writes the latest frame of video frame data to the target shared memory, the video frame data that has not been consumed by the consumer task process will be overwritten. If the consumer task process consumes data according to the original header index, it is easy to cause confusion in video frame reading, affecting subsequent processing. Therefore, for the consumer task process whose consumer ring queue is full, the producer process needs to perform frame drop on it.

[0114] In this embodiment, when the producer process determines the memory identifier to be written, the producer process can determine whether the consumption ring queues corresponding to the N consumption task processes are full according to the data writing index and the head indexes corresponding to the N consumption task processes, and then determine whether there is at least one process that needs to drop frames among the N consumption task processes. If there is, the producer process can perform a CAS atomic operation of comparison plus one on the head index corresponding to each process that needs to drop frames, and determine the memory identifier to be written according to the result of the first atomic operation.

[0115] In one example, the real-time video frame distribution module traverses N consumer task context data. For each consumer task context data, if the consumer task context data satisfies: fhead < pnext and mod(fhead,M) is equal to mod(pnext,M), it can be determined that the consumer circular queue is full, and the consumer task process corresponding to the consumer task context data needs to perform a frame drop operation, so the fhead of the consumer task context data can be subjected to a CAS atomic operation of comparison plus one, and the first atomic operation result is obtained.

[0116] In this embodiment, the consumer circular queue in the consumer task context data does not need to achieve access synchronization between the producer and consumer task processes through inefficient methods such as mutex locks, but uses CAS atomic operations on fhead to achieve synchronization operations on the queue. For the producer process, this type of CAS atomic operation is only required when the consumer task process is dropped, and it will inevitably succeed after a limited number of retries (no more than M times) (because each failure means that the consumer task process is simultaneously performing CAS atomic operations of comparison and addition of 1 and succeeding, and cr_queue will inevitably be empty after a maximum of M times, and the consumer task process will no longer operate on fhead), so it will not block the producer's production and distribution of real-time video frames. At the same time, any consumer task process whose consumption speed cannot keep up with the production and distribution speed will only cause its own frame loss, and will not block or affect other consumer task processes.

[0117] In some embodiments, the producer process determines the memory identifier to be written according to the first atomic operation result and the production ring queue, including:

[0118] Step A31: when the result of the first atomic operation is successful, or when there is no process that needs to drop frames among the N consumer task processes, the producer process determines a first initial memory identifier in a queue element of the production ring queue based on the data write index;

[0119] Step A32: the producer process compares the first initial memory identifier with the to-be-read memory identifiers corresponding to the N consumer task processes. If the first initial memory identifier is the same as the to-be-read memory identifier corresponding to at least one consumer task process, the memory identifier of the unqueued shared memory is used as the to-be-written memory identifier; wherein the unqueued shared memory is a target shared memory whose memory identifier is different from each current queue element of the production circular queue, and whose memory identifier is different from the to-be-read memory identifier corresponding to each consumer task process;

[0120] Step A33: When the producer process uses the memory identifier of the unqueued shared memory as the memory identifier to be written, it updates the memory identifier to be written to the production ring queue and the N consumer ring queues of the consumer task context data according to the data write index.

[0121] In this embodiment, after performing a CAS atomic operation of comparing and adding one to the head index corresponding to the process that needs to drop frames, if the CAS atomic operation results of each process that needs to drop frames are all successful operations, it indicates that each process that needs to drop frames needs to successfully discard a frame of video frame data. If the CAS atomic operation results of some or all processes that need to drop frames are failed operations, it means that the process that needs to drop frames that failed in the operation has successfully consumed a frame of video frame data, and the queue state of the consumption ring queue corresponding to the process that needs to drop frames is updated. In this case, it can be re-determined whether there is a process that needs to drop frames among the N consumer task processes.

[0122] When the CAS atomic operation results of each process that needs to drop frames are all successful, or there is no process that needs to drop frames among the N consumer task processes, the producer process can determine the first initial memory identifier in the queue element of the production ring queue based on the data write index. For example, the producer process can calculate k=mod(pnext,M) and use the kth queue element of the production ring queue as the first initial memory identifier.

[0123] After determining the first initial memory identifier, the producer process can compare the first initial memory identifier with the memory identifiers to be read corresponding to the N consumer task processes to determine whether the first initial memory identifier is the same as the memory identifier to be read corresponding to at least one consumer task process. If so, it indicates that the target shared memory corresponding to the first initial memory identifier is being occupied by at least one consumer task process. In this case, the producer process can determine the subset of cblks (i.e., the set of unqueued shared memories) that are not in the pr_queue corresponding to the target video source and all its corresponding consumer task contexts in the target shared memory set, and use an element of the subset as the memory identifier to be written.

[0124] If the memory identifier of the unqueued shared memory is used as the memory identifier to be written, it means that the producer process needs to write the next frame of video frame data to the unqueued shared memory. To ensure that each consumer task process can read data from the correct target shared memory, the producer process can update the memory identifier to be written to the production circular queue and the N consumer task context data consumer circular queues according to the data write index, so that the production circular queue and the N consumer circular queues all include the memory identifier of the unqueued shared memory.

[0125] In one example, after determining the set of unqueued shared memory, the producer process can take out one of the buffers (preferably the buffer of the created shared memory). If the shared memory is not created, it is first created and mapped to the producer process. Then the element at the mod(pnext,M) position in pr_queue and the corresponding mod(pnext,M) position in all cr_queues (i.e. the current tail, because the values ​​of pnext and ftail are equal) is set as the memory identifier of the buffer.

[0126] In this embodiment, by adding a maximum of N blocks of target shared memory, it can be ensured that when the producer process generates a new frame of video frame data, it can select a video frame buffer that is not occupied by any consumer task process to store the video frame data, thereby ensuring production efficiency.

[0127] In some embodiments, the producer process determines the memory identifier to be written according to the first atomic operation result and the production ring queue, and further includes:

[0128] Step A34: When the first initial memory identifier is different from the memory identifier to be read corresponding to each consumer task process, the producer process uses the first initial memory identifier as the memory identifier to be written.

[0129] In this embodiment, if the first initial memory identifier is different from the memory identifier to be read corresponding to any consumer task process, it indicates that the target shared memory corresponding to the first initial memory identifier is not occupied by any consumer task process. In this case, the first initial memory identifier can be directly used as the memory identifier to be written, and the next frame of video frame data can be written to the target shared memory corresponding to the first initial memory identifier.

[0130] Furthermore, in one example, if there is not at least one process that needs to drop frames among the N consumer task processes, the producer process can directly determine the first initial memory identifier in the queue element of the production ring queue based on the data write index, and execute steps A32 and A34.

[0131] In some embodiments, determining a target to-be-read memory identifier according to a target queue index and a target consumption ring queue in target consumption context data includes:

[0132] Step B1: the target consumption process determines the second initial memory identifier in the target consumption ring queue according to the target header index, and performs a CAS atomic operation of comparison plus one on the target header index, and obtains a second atomic operation result;

[0133] Step B2: When the second atomic operation result is a success, the target consumer process uses the second initial memory identifier as the target memory identifier to be read; when the second atomic operation result is an operation failure, the target consumer process uses the preset invalid identifier as the target memory identifier to be read.

[0134] In this embodiment, when the target consumer process determines the memory identifier to be read corresponding to the process (i.e., the target memory identifier to be read), it can determine the second initial memory identifier in the target consumer circular queue according to the target head index, and then perform a CAS atomic operation of comparison plus one on the target head index. For example, the target consumer process can set the second initial memory identifier to the memory identifier at the mod(fhead, M) position in the cr_queue queue corresponding to the process, and perform a CAS atomic operation of comparison plus one on fhead.

[0135] If the CAS atomic operation is successfully performed on the target header index, it indicates that the producer process has not performed a frame drop operation on this process. Therefore, the second initial memory identifier can be directly used as the target memory identifier to be read to read the target video frame data from the target shared memory corresponding to the second initial memory identifier.

[0136] If the result of the CAS atomic operation of the target header index is a failure, it indicates that the producer process successfully performs a frame drop operation on the current process. In this case, if the video frame data corresponding to the second initial memory identifier is read, the video frame reading confusion problem is likely to occur, affecting subsequent processing. Therefore, the target consumer process can set the target memory identifier to be read to a preset invalid identifier (for example, 0), so that data access operations can be performed on the target shared memory.

[0137] In this way, the orderliness of the reading of video frame data by the consumer task process can be guaranteed, thereby improving the reliability of subsequent processing.

[0138] In some embodiments, after obtaining the target video frame data, it also includes: the target consumption process performs data processing on the target video frame data, and when the processing is completed, sets the target to-be-read memory flag to a preset invalid flag.

[0139] In this embodiment, when the target memory identifier to be read is determined, if the identifier is a valid identifier, the target consumer process can read the target video frame data from the target shared memory corresponding to the target memory identifier to be read, and perform data processing on the target video frame data. In the case where the processing is not completed, the target consumer process occupies the target shared memory corresponding to the target memory identifier to be read. Therefore, the target memory identifier to be read may not be set, so that the producer process can make an occupancy judgment based on the memory identifier to be read corresponding to each consumer task process. In the case where the processing is completed, the target consumer process ends the occupation of the shared memory. Therefore, the target memory identifier to be read can be set to a preset invalid identifier, so that the producer process can accurately judge the occupation status of the target shared memory and perform memory allocation accordingly.

[0140] In some embodiments, the consumption task context data is stored in a shared memory so that the producer process and the consumption task process can read and write data to the consumption task context data. In other words, there are at least two types of shared memories in this application: a frame buffer and a consumption task context shared memory. The frame buffer refers to a shared memory used to store video frame data, and the consumption task context shared memory refers to a shared memory used to store consumption task context data.

[0141] By introducing a consumer task context that is also a shared memory, and constructing simple data structures such as cr_queue, fhead, ftail, cblk, etc., and using lock-free CAS atomic operations to complete all accesses to cr_queue, without using technologies such as mutex that rely on kernel state, the operation is simple and efficient.

[0142] In this embodiment, the video frame shared access method of the present application may also include the following steps:

[0143] The producer process responds to the shutdown request sent by the consumer process to be closed by deleting the consumer task context data corresponding to the consumer process to be closed from the shared memory; wherein the consumer process to be closed is the consumer task process that needs to stop accessing the video frame data corresponding to the target video source.

[0144] In this embodiment, when the consumer task process needs to stop accessing the video frame data or the consumer task process needs to exit the program, the consumer task process is a consumer process to be closed. The consumer process to be closed can send a close request to the producer process. When the producer process receives the close request, it deletes the consumer task context data corresponding to the consumer process to be closed from the shared memory, and closes the shared memory occupied by the consumer process to be closed, saving memory space.

[0145] At the same time, the producer process can also notify the real-time video frame distribution module to stop distributing video frames to the consumer process to be closed. After sending the closing request, the consumer process to be closed closes all opened video frame buffer shared memories.

[0146] In some embodiments, the queue index for consuming task context data includes a tail index. For the relevant description of the tail index, please refer to the above embodiments, which will not be repeated herein.

[0147] The video frame shared access method of the present application may also include the following steps:

[0148] After writing a frame of video frame data corresponding to the target video source into a target shared memory, the producer process adds one to the data write index and updates the tail index of N consumer task context data based on the updated data write index.

[0149] In this embodiment, after completing the data writing operation, the producer process can update the value of pnext to pnext+1, and at the same time traverse all consumer task context data corresponding to the target video source, and set each tail index ftail to the same value as pnext, so that each consumer task process can accurately determine the storage location of the latest frame of video frame data through its corresponding tail index.

[0150] To facilitate understanding of the solution of the present application, a specific example is provided below for illustration.

[0151] The video frame sharing method provided in this application can be implemented and deployed on embedded terminal devices such as all-in-one machines and cameras that have audio and video decoding and AI reasoning capabilities, and the number of video source channels supported by the real-time video frame production module and the real-time video frame distribution module can be set according to the number of video channels that the hardware can support simultaneous decoding. For each video source channel, the real-time video frame distribution module can support up to N consumer task processes, where N and the number of pre-allocated video frame buffers M can be set according to business needs and hardware capabilities.

[0152] The real-time video frame production module implements video transmission and decoding for terminals supporting streaming media transmission such as external cameras connected to the all-in-one machine, and outputs video frames. The consumption task management module implements RESTful services, monitors requests from the consumption task process, and provides RESTful interfaces such as real-time video frame shared access application, closure, and session heartbeat maintenance for determining whether the consumption task process is alive for each video source; the real-time video frame distribution module implements the allocation and distribution of the corresponding video frame buffer and the consumption task context shared memory of each consumption task process for each video source according to the above process of the present invention; each consumption task process completes the real-time video frame shared access application and closure management of the specified video source through the RESTful service interface provided by the above consumption task management module, and realizes consumption processing of the current video frame in the shared memory according to the process of the technical solution of the present invention, thereby completing business functions such as image processing and AI reasoning.

[0153] The execution process of this example can be as follows Figure 4 shown. Specifically:

[0154] Step C1: The video device starts the real-time video frame production module, the consumption task management module and the real-time video frame distribution module. When initialized, the real-time video frame distribution module first pre-allocates M video frame buffers for each video source and numbers them in sequence, and fills them into the pr_queue of the corresponding distribution context data structure in the order of numbers. The number of elements in the pr_queue is M. The record data write index pnext is 1, that is, the next video frame buffer to be filled is numbered as the value corresponding to the first element in the pr_queue queue. During the shared access process, if the maximum number of consumption task processes corresponding to the video source is n, the actual number of consumption task processes is N, 0≤ N ≤ n, then the number of frame buffers actually allocated to the video source is M+N.

[0155] Step C2: After the consumer task management module receives the real-time video frame shared access request of the consumer task process, it calls the real-time video frame distribution module to create a shared memory of the consumer task context for the video source specified by the request, and fills the cr_queue with the same numbered content in the corresponding pr_queue, and then sets the head index fhead and tail index ftail corresponding to the consumer task process to pnext, that is, the cr_queue is empty.

[0156] Step C3: The consumption task management module returns the width, height, bytes per pixel, frame rate FPS and other information of the video frame, the shared memory name of the consumption task context, the maximum number of video frame buffers M+N, the pre-allocated number M and the shared memory name corresponding to each video frame buffer to the consumption task process.

[0157] Step C4: The consumer task process opens the shared memory of the consumer task context in a readable and writable manner according to the returned information and maps it to the process address space. Then, the corresponding video frame buffer shared memory is opened in a read-only manner according to the pre-allocated quantity and mapped to the process address space.

[0158] Step C5: The consumer task process periodically (interval <= 1 / FPS) checks whether there are consumable video frames from the cr_queue of the consumer task context. If there are no consumable video frames, re-execute step C5. If there are, set cblk to the buffer number corresponding to the mod(fhead, M) position in the cr_queue queue, and then perform a CAS atomic operation of comparison plus 1 on fhead. If the atomic operation fails, set cblk to an invalid number, and then repeat step C5. If the operation succeeds, execute step C6;

[0159] Step C6: If the shared memory corresponding to cblk is not opened, it is opened in read-only mode and mapped to the process address space, and then the video frame is processed as a consumer task. After completion, cblk is set to an invalid number, and then step C5 is executed.

[0160] Step C7: For each video source, the real-time video frame distribution module traverses all the corresponding consumer task context data. If one or several consumer task context data satisfy fhead < pnext and mod(fhead,M)=mod(pnext,M), it means that the cr_queue of the consumer task process is full. Then, a CAS atomic operation of comparing and adding 1 is performed on the fhead of this consumer task context until the above operations of all consumer task context data that meet the conditions are completed, or there is no consumer task process whose cr_queue is full.

[0161] Step C8: The real-time video frame distribution module finds the subset of cblks of the pr_queue and all the corresponding consumer task contexts that are not in the pr_queue corresponding to the video source from the M+N video frame buffer set, and takes out one of the buffers (preferably the buffer with created shared memory). If its shared memory is not created, it is first created and mapped to the producer process, and then the element at the mod(pnext,M) position in the pr_queue and the mod(pnext,M) position in the cr_queue of all the corresponding consumer task contexts (i.e., the tail of the current queue, because the values ​​of pnext and ftail are equal) is set to the number of the buffer;

[0162] Step C9: The real-time video frame distribution module obtains the latest video frame from the real-time video frame production module, and fills the latest video frame sequence number, timestamp and image data of the video frame into the buffer selected in the previous step, then the value of pnext is increased by 1 and updated to pnext, and at the same time, all consumer task contexts of the video source are traversed, and ftail therein is set to the same value as pnext, and then go to step C7;

[0163] Step C10: When the consumer task process needs to stop access or the program exits, it sends a close request to the consumer task management module of the producer process. The consumer task management module notifies the real-time video frame distribution module to stop the corresponding distribution, and then closes the shared memory of the corresponding consumer task context. After sending the close request, the consumer task process closes the shared memory of its consumer task context and all opened video frame buffer shared memories.

[0164] In one embodiment, the present application further provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video frame shared access method in any embodiment.

[0165] In one embodiment, the present application further provides a video device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video frame shared access method in any embodiment.

[0166] Indicatively, Figure 5 The internal structure diagram of a video device provided in an embodiment of the present application, in one example, the video device may be a video acquisition device (such as a camera). In another example, the video device may be a video processing device (such as an NVR, an AI all-in-one machine, etc.). Figure 5 The video device 900 includes a processing component 902, which further includes one or more processors, and a memory resource represented by a memory 901, for storing instructions executable by the processing component 902, such as an application. The application stored in the memory 901 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 902 is configured to execute instructions to perform the steps of the video frame shared access method described in any of the above embodiments.

[0167] The video device 900 may also include a power supply component 903 configured to perform power management of the video device 900, a wired or wireless network interface 904 configured to connect the video device 900 to a network, and an input / output (I / O) interface 905. The video device 900 may operate based on an operating system stored in the memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0168] Those skilled in the art will understand that the internal structure of the video device shown in the present application is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the video device to which the scheme of the present application is applied. The specific video device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, "one", "one", "said", "the" and "it" may also include plural forms, unless the context clearly indicates another way. A plurality refers to at least two cases, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the relevant listed items.

[0170] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0171] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video frame shared access method, characterized in that: Applied to a video device, the video device runs a producer process and N consumer task processes for the same target video source, and the N consumer task processes correspond to N consumer task context data one by one; the method comprises: The producer process determines the memory identifier to be written according to the production circular queue and the data writing index; wherein the production circular queue includes memory identifiers corresponding to M target shared memories, and the target shared memory is a frame buffer corresponding to the target video source; The producer process writes a frame of video frame data corresponding to the target video source into a target shared memory based on the memory identifier to be written; In response to satisfying a preset judgment rule, the target consumption process determines whether there is video frame data in the target shared memory that has not been consumed by the target consumption process according to the target queue index in the target consumption context data; wherein the target consumption process is any of the consumption task processes, and the target consumption context data is the consumption task context data corresponding to the target consumption process; When the target consumer process determines that there is video frame data in the target shared memory that has not been consumed by the target consumer process, the target consumer process determines the target memory identifier to be read according to the target queue index and the target consumer circular queue in the target consumer context data; wherein the queue element of the target consumer circular queue is the same as the queue element of the production circular queue; The target consumption process accesses the target shared memory corresponding to the target to-be-read memory identifier in a read-only manner, and obtains the target video frame data.

2. The method according to claim 1, characterized in that The queue index of the consumption task context data includes a head index; The producer process determines the memory identifier to be written according to the production ring queue and the data writing index, including: The producer process determines whether there is at least one process that needs to drop frames among the N consumer task processes according to the data writing index and the header indexes corresponding to the N consumer task processes; When determining that there is at least one process that needs to drop frames, the producer process performs a CAS atomic operation of comparison plus one on the head index corresponding to each process that needs to drop frames, and obtains a first atomic operation result; The producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue.

3. The method according to claim 2, characterized in that The producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue, including: The producer process determines a first initial memory identifier in a queue element of the production ring queue based on the data write index when the result of the first atomic operation is a successful operation or when the process that needs to drop frames does not exist among the N consumer task processes; The producer process compares the first initial memory identifier with the to-be-read memory identifiers corresponding to the N consumer task processes. If the first initial memory identifier is the same as the to-be-read memory identifier corresponding to at least one of the consumer task processes, the memory identifier of the unqueued shared memory is used as the to-be-written memory identifier; wherein the unqueued shared memory is a target shared memory whose memory identifier is different from each current queue element of the production circular queue and whose memory identifier is different from the to-be-read memory identifier corresponding to each of the consumer task processes; When the producer process uses the memory identifier of the unqueued shared memory as the memory identifier to be written, it updates the memory identifier to be written to the production ring queue and the N consumer ring queues of the consumer task context data according to the data write index.

4. The method according to claim 3, characterized in that The producer process determines the to-be-written memory identifier according to the first atomic operation result and the production ring queue, further comprising: When the first initial memory identifier is different from the memory identifier to be read corresponding to each of the consumer task processes, the producer process uses the first initial memory identifier as the memory identifier to be written.

5. The method according to any one of claims 2 to 4, characterized in that: Determining a target memory identifier to be read according to the target queue index and the target consumption ring queue in the target consumption context data includes: The target consumption process determines the second initial memory identifier in the target consumption circular queue according to the target header index, performs a CAS atomic operation of comparison plus one on the target header index, and obtains a second atomic operation result; When the result of the second atomic operation is a success, the target consumer process uses the second initial memory identifier as the target memory identifier to be read; and when the result of the second atomic operation is a failure, the target consumer process uses the preset invalid identifier as the target memory identifier to be read.

6. The method according to claim 5, characterized in that After obtaining the target video frame data, the method further includes: The target consumption process performs data processing on the target video frame data, and when the processing is completed, sets the target to-be-read memory flag to the preset invalid flag.

7. The method according to any one of claims 1 to 4, characterized in that: The consumption task context data is stored in a shared memory; the method further comprises: The producer process responds to a shutdown request sent by the consumer process to be closed by deleting the consumer task context data corresponding to the consumer process to be closed from the shared memory; wherein the consumer process to be closed is a consumer task process that needs to stop accessing the video frame data corresponding to the target video source.

8. The method according to any one of claims 1 to 4, characterized in that: The queue index of the consumption task context data includes a tail index; the method further includes: After writing a frame of video frame data corresponding to the target video source into the target shared memory, the producer process performs an increment operation on the data write index and updates the tail indexes of the N consumer task context data based on the updated data write index.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video frame shared access method according to any one of claims 1 to 8.

10. A video device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the video frame shared access method according to any one of claims 1 to 8 are performed.

Images