Video processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a video processing method and device, electronic equipment and storage medium, the method comprising: obtaining frame information of an input frame of a target video; storing the frame information into an input queue corresponding to a plurality of channels, the channels being created in advance according to image processing tasks of an image processing module, the channels being provided with input queues and channel attributes; obtaining the frame information of an input frame and the channel attributes of a channel corresponding to the input queue from the input queue, generating an image processing command corresponding to the frame information and the channel attributes, and sending the image processing command to the image processing module; parsing the image processing instruction by the image processing module, obtaining the input frame according to the parsed frame storage address, processing the input frame according to the parsed channel attributes to obtain an output frame, and starting an interrupt thread; and obtaining the frame information of another input frame and the channel attributes of a channel corresponding to the input queue by the interrupt thread for processing by the image processing module. The embodiments of the present application improve the video processing efficiency.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Video processing technology can be used to perform one or more operations on video image frames, such as rotation, enlargement, reduction, mirroring, flipping, and distortion correction, to meet users' needs for video playback effects. Hardware such as CPUs, DSPs (Digital Signal Processors), and image processing chips can be used to process video image frames.

[0003] In related technologies, video image frames are sent to the hardware for image processing one by one, and the hardware processes the image frames. However, due to factors such as system scheduling and system lag, image frames may not be sent to the hardware in a timely manner, resulting in low hardware utilization and low video processing efficiency. Summary of the Invention

[0004] This application provides a video processing method, apparatus, electronic device, and storage medium to solve the problem of low video processing efficiency.

[0005] In a first aspect, this application provides a video processing method, including:

[0006] Obtain the frame information of the input frame of the target video, including the frame storage address;

[0007] Frame information is stored in input queues corresponding to multiple channels. Channels are pre-created according to the image processing tasks of the image processing module. Each channel has an input queue and channel attributes. The channel attributes represent the image processing strategy of the image processing task.

[0008] Get the frame information of an input frame and the channel attributes of the corresponding channel in the input queue from the input queue, generate the image processing command with the corresponding frame information and channel attributes, and send the image processing command to the image processing module.

[0009] The image processing module parses the image processing instructions, obtains the input frame according to the parsed frame storage address, processes the input frame according to the parsed channel attributes, obtains the output frame, and starts the interrupt thread.

[0010] The frame information of another input frame and the channel attributes of the corresponding channel in the input queue are obtained by interrupting the thread, and then processed by the image processing module.

[0011] In some embodiments, the image processing command that generates corresponding frame information and channel attributes includes:

[0012] A first storage address is requested in the storage module, and the first storage address is used for the image processing module to store the output frame;

[0013] Translate the frame information into the first command of the image processing module;

[0014] Translate the channel attributes into a second command from the image processing module;

[0015] Translate the output frame address into the third command of the image processing module;

[0016] Image processing commands include the first command, the second command, and the third command.

[0017] In some embodiments, the frame information of the input frame further includes frame attributes, and the method further includes:

[0018] Replace the first attribute in the frame attributes of the input frame with the second attribute in the channel attributes to obtain the frame attributes of the output frame. The first and second attributes are attributes of the same category.

[0019] The output frame and its frame attributes are sent to the downstream image processing task by interrupting the thread.

[0020] In some embodiments, the method further includes:

[0021] After obtaining the frame information of another input frame and the channel attributes of the corresponding channel in the input queue through the interrupt thread, and sending the output frame and its frame attributes to the downstream image processing task, the interrupt thread ends.

[0022] In some embodiments, before obtaining the frame information of the input frame of the target video, the method further includes:

[0023] Create a first number of channels, where the first number is the number of image processing tasks in the image processing module;

[0024] In the storage module, an attribute memory is allocated for the channel, and the channel attributes are stored in the attribute memory.

[0025] Create the input queue corresponding to the channel and set the scheduling rules for the input queue.

[0026] In some embodiments, obtaining frame information of an input frame from the input queue includes:

[0027] Based on the polling scheduling rules, determine the input queue for obtaining the frame information of the input frames to be acquired;

[0028] According to the first-in-first-out rule, the frame information of an input frame is obtained from the determined input queue.

[0029] Secondly, this application provides a video processing apparatus, comprising:

[0030] The input module is used to obtain the frame information of the input frame, including the frame storage address;

[0031] The input buffer module is used to instantiate multiple input queues for multiple channels and store frame information into multiple input queues. Channels are pre-created according to the image processing tasks of the image processing module. Each channel has an input queue and channel attributes. The channel attributes represent the image processing strategy of the image processing task.

[0032] The pop-up module is used to obtain the frame information of an input frame and the channel attributes of the corresponding channel in the input queue from the input queue, generate the image processing command with the corresponding frame information and channel attributes, and send the image processing command to the image processing module.

[0033] The image processing module is used to parse image processing instructions, obtain input frames according to the parsed frame storage address, process the input frames according to the parsed channel attributes, obtain output frames, and start an interrupt thread. The interrupt thread is used to send a pop-up signal to the pop-up module. The pop-up module is also used to obtain the frame information of another input frame and the channel attributes of the corresponding channel in the input queue according to the pop-up signal, for the image processing module to process.

[0034] In some embodiments, the pop-up module includes:

[0035] The scheduling unit is used to retrieve frame information of one of the input frames from one of the input queues;

[0036] The storage configuration unit is used to request a first storage address in the storage module, and the first storage address is used for the image processing module to store the output frame;

[0037] The translation unit is used to translate frame information, channel attributes, and the first storage address into image processing commands from the image processing module.

[0038] In some embodiments, the apparatus further includes a push module, and the interrupt thread is further configured to send a push signal to the push module. The push module is configured to retrieve the output frame stored in the image processing module from the storage module according to the push signal, and release the storage space of the output frame.

[0039] In some embodiments, the apparatus further includes:

[0040] The output module is used to send the output frame and its frame attributes to the downstream image processing task. The frame attributes of the output frame are obtained by the pop-up module replacing the first attribute in the frame attributes of the input frame with the second attribute in the channel attributes. The first and second attributes are attributes of the same category.

[0041] Thirdly, this application provides an electronic device including one or more processors and a memory, and also includes the means of the second aspect; the memory stores a computer program, the computer program including computer instructions, and the one or more processors invoke the computer instructions to cause the electronic device to perform the video processing method of the first aspect.

[0042] Fourthly, this application provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the video processing method as described in the first aspect.

[0043] As described above, this application provides a video processing method, apparatus, electronic device, and storage medium. This application creates multiple channels, each corresponding to a different image processing task. Frame information of input frames is stored in input queues for multiple channels. Based on a set scheduling rule, frame information of one input frame is retrieved from the input queue for processing by the image processing module. The image processing module retrieves the input frame based on its frame information, processes it according to channel attributes, and after processing one input frame, interrupts the thread to schedule another input frame for processing. This allows the image processing module to continuously process input frames from multiple image processing tasks. If the input frame of one image processing task is not promptly sent to the image processing module, the module can still continue processing input frames from other tasks, improving the utilization rate of the image processing module and enhancing the overall efficiency of video processing. Attached Figure Description

[0044] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating the workflow of an image processing task provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating the video processing method provided in this application embodiment;

[0047] Figure 3 This is a schematic diagram of the data flow during the video processing process provided in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the structure of the pop-up module provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of the video processing apparatus provided in the embodiments of this application. Detailed Implementation

[0050] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0051] This application provides a video processing method, apparatus, electronic device, and storage medium. The video processing method is applicable to intelligent video processing devices, which may include security devices such as IPCs (Internet Protocol Cameras), in-vehicle video processing devices, or image processing devices.

[0052] See Figure 1 This is a schematic diagram illustrating the workflow of an image processing task provided in an embodiment of this application. Figure 1 As shown, video processing intelligent devices typically need to perform multiple image processing tasks on the target video, i.e., the video to be processed. These image processing tasks may include rotation, magnification, reduction, mirroring, flipping, distortion correction, etc. Depending on the image processing requirements, an image processing task may specifically include one subtask or multiple subtasks. For example, image processing task 1 to image processing task N are all subtasks of the current image processing task. When multiple subtasks are included, these subtasks can be tasks with the same function, such as all being tasks with the function of reduction, or they can be tasks with different functions, such as tasks with functions like rotation, magnification, and reduction. This application embodiment does not specifically limit this.

[0053] For example, Figure 1 In the process, the upstream image processing task is a rotation task, and the video obtained after processing by this rotation task is the target video of the current image processing task.

[0054] The current image processing task consists of three sub-tasks:

[0055] Image processing task 1: Adjust the target video to the first resolution (3840*2160) and the first frame rate (20fps);

[0056] Image processing task 2: Adjust the target video to the first resolution (3840*2160) and the second frame rate (10fps);

[0057] Image processing task 3: Adjust the target video to the second resolution (1280*720) and the second frame rate (10fps).

[0058] The downstream image processing task is distortion correction.

[0059] The current image processing task is handled by a video processing intelligent device. Upstream and downstream image processing tasks can also be handled by the same video processing intelligent device, or they can be handled by other devices, such as mobile terminals or servers.

[0060] In some embodiments, the current image processing task has no upstream image processing task, that is, the current image processing task processes the original video captured by the video capturing device, and the target video is the original video.

[0061] In some embodiments, the current image processing task has no downstream image processing task, that is, after the current image processing task is completed, no further image processing is performed on the output of the current image processing task.

[0062] The video processing smart device of this application creates multiple channels, each corresponding to a multiple image processing task, and schedules the image processing module of the video processing smart device to process the image frames corresponding to each channel according to the set scheduling rules. This allows the image processing module to continuously process the image frames of multiple image processing tasks, thereby improving the utilization rate of the image processing module and the overall efficiency of video processing.

[0063] See Figure 2 This is a flowchart illustrating the video processing method provided in an embodiment of this application, as shown below. Figure 2 As shown, the method may include the following steps:

[0064] Step S100: Obtain the frame information of the input frame of the target video, including the frame storage address.

[0065] In some embodiments, for a video processing smart device, the target video is a frame sequence consisting of multiple image frames output by an upstream image processing task, the image frames of the target video are input frames, and the output frames are obtained after the input frames are processed by the video processing method of this application embodiment.

[0066] The video processing intelligent device first stores the image frames output by the upstream image processing task into a storage module, such as a frame memory pool. Then, it retrieves the image frame from the frame memory pool, processes it using its image processing module to obtain an output frame, and stores the output frame back into the frame memory pool. When the efficiency of the upstream image processing task in sending image frames exceeds the processing efficiency of the image processing module, storing the image frames in an input buffer prevents frame loss. Conversely, when the efficiency of the upstream image processing task in sending image frames is less than the processing efficiency of the image processing module, if there are unprocessed image frames in the input buffer, the image processing module can process these unprocessed frames, avoiding long waits for the upstream image processing task to send image frames.

[0067] The image processing module is a hardware module, which may specifically be a CPU, DSP, image processing chip, or other components. This application does not impose specific limitations on this.

[0068] The frame memory pool includes an input buffer and an output buffer. The input buffer is used to store image frames output by the upstream image processing task, i.e., input frames. Each input frame has a frame storage address in the input buffer. The output buffer is used to store the output frames obtained after processing the input frames by the video processing method of the present application embodiment. Each output frame has a frame storage address in the output buffer.

[0069] The upstream image processing task outputs image frames along with their frame attributes. These attributes may include information such as frame width, frame height, and frame number. The image processing module can then process the image frames based on these attributes and the specific image processing task.

[0070] In some embodiments, when multiple image processing tasks exist, to improve the hardware utilization efficiency of the image processing module, a first number of channels can be pre-created, each corresponding to a different image processing task. An attribute memory is allocated for each channel in the storage module, channel attributes are stored in the attribute memory, an input queue corresponding to each channel is created, and scheduling rules for the input queues are set. Then, according to the set scheduling rules, the image processing module is scheduled to process the image frames corresponding to each channel, enabling the image processing module to continuously process image frames from multiple image processing tasks. The implementation method can be found in [reference needed]. Figure 3 This is a schematic diagram of the data flow during the video processing process provided in the embodiments of this application.

[0071] like Figure 3As shown, during video processing, the frame information of the input frame flows from the input queue to the pop-up module. The pop-up module processes the frame information to obtain input buffer information, which is then sent to the image processing module for processing. The image processing module processes the input frame to obtain output buffer information, which includes a first storage address. The push module retrieves the output frame from the first storage address based on the output buffer information and pushes the output frame to the downstream image processing task.

[0072] like Figure 3 As shown, during video processing, after the frame information of the input frame enters the input queue, the pop-up module receives a start signal. After starting according to the start signal, the pop-up module retrieves the frame information of an input frame from one of the input queues, translates the frame information into command text, and translates the channel attributes into command text. It also allocates a first storage address for the output frame in the storage module, translates the first storage address into command text, and sends the command text corresponding to the frame information, the command text corresponding to the channel attributes, and the command text corresponding to the first storage address to the image processing module. The frame attributes and channel attributes of the input frame are merged into output frame attributes, the output frame attributes are translated into command text, and the command text corresponding to the output frame attributes is sent to the push module for the push module to push to the downstream image processing task.

[0073] In some embodiments, if the downstream image processing task can identify the output frame attributes, the pop-up module may not translate the output frame attributes into command text, but instead send the output frame attributes directly to the push module.

[0074] The video processing intelligent device, based on the current image processing task of the image processing module (which includes multiple tasks), pre-establishes multiple channels. These channels transmit frame information of image frames, including frame attributes and frame storage addresses. Each channel corresponds to one of the multiple tasks, and multiple input queues are created for each channel. Multiple channel attribute memories are created in memory, such as memory 1, memory 2, memory 3, etc., storing the channel attributes of the corresponding channels. Channel attributes include the target parameters of the output frame corresponding to the image processing task. For example, channel 1 corresponds to image processing task 1, the input queue of channel 1 is queue 1, and the channel attributes of channel 1 include {ch1, 3840*2160, 20fps}. Channel 2 corresponds to image processing task 2, the input queue of channel 2 is queue 2, and the channel attributes of channel 2 include {ch2, 3840*2160, 10fps}. Channel 3 corresponds to image processing task 3, the input queue of channel 3 is queue 3, and the channel attributes of channel 3 include {ch3, 1280*720, 10fps}. Here, ch1, ch2, and ch3 are channel numbers.

[0075] In some embodiments, upon receiving an image frame and its frame attributes from the upstream image processing output, the image frame can be stored in the input buffer, and the frame information of the image frame can be obtained, including the frame attributes and the frame storage address.

[0076] Step S200: Store the frame information into the input queues corresponding to multiple channels. The channels are created in advance according to the image processing task of the image processing module. Each channel is set with an input queue and channel attributes. The channel attributes represent the image processing strategy of the image processing task.

[0077] In some embodiments, after acquiring the frame information of an image frame, the frame information of the image frame can be stored in a pre-created input queue corresponding to a first number of channels, wherein the first number is the number of image processing tasks of the image processing module. After acquiring the frame information of the next image frame, the frame information of the next image frame can be stored in multiple input queues corresponding to multiple channels.

[0078] In some embodiments, each input queue is configured with a queue depth. If the number of stored frame information is less than the queue depth, frame information of an image frame can be stored in the input queue. If the number of stored frame information is equal to the queue depth, a blocking signal is generated to prompt that no more frame information should be sent to the input queue.

[0079] In some embodiments, the upstream image processing task and the current image processing task can establish a link. After the link is established, the current image processing task can send the channel information of the current image processing task to the upstream image processing task. The channel information includes the channel number. The upstream image processing task can directly send the frame information of the image frame to each input queue.

[0080] Step S300: Obtain the frame information of an input frame and the channel attributes of the corresponding channel from the input queue, generate an image processing command with the corresponding frame information and channel attributes, and send the image processing command to the image processing module.

[0081] In some embodiments, when creating channels and queues, the video processing smart device can set scheduling rules between multiple channels and scheduling rules for each queue, so that the image processing module processes the frame information in the input queue according to the set scheduling rules.

[0082] For example, the scheduling rule among multiple channels can be a round-robin scheduling rule, which ensures that each channel has a chance to execute and avoids a situation where a channel waits for a long time. The scheduling rule for the input queue can be a first-in, first-out rule, which can guarantee the continuity of output frames.

[0083] For example, queue 1 of channel 1, queue 2 of channel 2, and queue 3 of channel 3 all contain frame information of input frame 1, frame information of input frame 2, and frame information of input frame 3. According to the scheduling rules set above, the processing order of the frame information is as follows: frame information of input frame 1 in queue 1, frame information of input frame 1 in queue 2, frame information of input frame 1 in queue 3, frame information of input frame 2 in queue 1, frame information of input frame 2 in queue 2, frame information of input frame 2 in queue 3, frame information of input frame 3 in queue 1, frame information of input frame 3 in queue 2, and frame information of input frame 3 in queue 3.

[0084] In some embodiments, frame information in each queue can be scheduled through a pop-up module. The pop-up module is a preset software function module. When the video processing smart device acquires the first frame of the target video, it sends a start signal to the pop-up module. The pop-up module starts according to the start signal and sends the image processing command corresponding to the first frame to the image processing module.

[0085] join Figure 4 This is a structural diagram of the pop-up module, such as... Figure 4 As shown, the pop-up module may include a scheduling unit, a storage configuration unit, and a translation unit.

[0086] The scheduling unit is used to retrieve frame information for one of the input frames from one of the input queues. The scheduling unit can retrieve frame information according to the scheduling rules set above.

[0087] The storage configuration unit is used to request a frame storage address in the output buffer of the frame memory pool. This frame storage address is used to store the output frame obtained by the image processing module after processing the input frame.

[0088] The translation unit is used to translate frame information, channel attributes, and the frame storage address requested by the storage configuration unit into image processing commands that the image processing module can recognize. The specific translation method can be determined according to the machine language that the image processing module can recognize. The image processing command may include multiple sub-commands, for example, it may include a first command translated from the frame information of the input frame, a second command translated from the channel attributes, and a third command translated from the first storage address. Specifically, after channel creation and before step S100, the channel attributes can be pre-translated into the second command to improve the overall generation efficiency of subsequent image processing commands.

[0089] The translation unit is also used to merge the frame attributes and channel attributes of the input frame to obtain the output frame attributes, translate the output frame attributes into a fourth command, and send the fourth command to the push module.

[0090] Figure 3In the middle, the pop-up module translates the frame information of the input frame into a first command. This first command includes input buffer information, that is, the image processing module can obtain the frame storage address of the input frame in the input buffer according to the first command.

[0091] The command text includes a second command, a third command, and a fourth command. The pop-up module sends the second and third commands, along with input buffer information, to the image processing module, and sends the fourth command to the push module. The push module is a pre-defined software function module used to distribute output frames to downstream image processing tasks.

[0092] Step S400: The image processing module parses the image processing instructions, obtains the input frame according to the parsed frame storage address, processes the input frame according to the parsed channel attributes, obtains the output frame, and starts the interrupt thread.

[0093] In some embodiments, the image processing module parses the image processing instructions to obtain the frame buffer address and frame attributes of the input frame, channel attributes, and the frame buffer address of the output frame.

[0094] Based on the target parameters corresponding to the channel attributes, the input frame is processed to obtain the output frame, and the output frame is stored in the first storage address.

[0095] Step S500: Obtain the frame information of another input frame and the channel attributes of the corresponding channel of the input queue through the interrupt thread, so that the image processing module can process them.

[0096] In some embodiments, after receiving an output frame, the image processing module starts an interrupt thread. Once started, the interrupt thread controls the pop-up module to retrieve frame information of an input frame from one of the input queues according to a set scheduling rule, and processes this frame information in the same way as described above. That is, after sending the image processing command corresponding to the first frame, the interrupt thread can control the pop-up module to continue processing subsequent image frames.

[0097] In some embodiments, the interrupt thread may send a pop-up signal to the pop-up module, causing the pop-up module to obtain the frame information of an image frame from one of the input queues based on the pop-up signal, generate the image processing command corresponding to the image frame, and generate the image processing command corresponding to the first frame in the same way. After sending the image processing command corresponding to the image frame to the image processing module, the interrupt thread is waited for to send the pop-up signal to the pop-up module again, and so on.

[0098] In some embodiments, after the interrupt thread sends the frame information of another input frame and the channel attributes of the corresponding channel in the input queue to the downstream image processing task, it can be configured to automatically terminate the interrupt thread. Alternatively, the interrupt thread can be terminated by the processor of the video processing intelligent device. The interrupt thread is started when the image processing module outputs an output frame, and terminates after controlling the pop-up module to send a new input frame to the image processing module, ensuring that the image processing tasks of the image processing module are performed in an orderly manner.

[0099] In some embodiments, after the interrupt thread starts, it also sends a push signal to the push module, which then sends the output frame to the downstream image processing task based on the push signal.

[0100] As can be seen from the above embodiments, the video processing method provided in this application creates multiple channels, each corresponding to a multiple image processing task. The frame information of the input frame is stored in the input queues of multiple channels. Based on a set scheduling rule, the frame information of an input frame is retrieved from the input queue for processing by the image processing module. The image processing module retrieves the input frame based on its frame information, processes it according to channel attributes, and after processing the frame information of one input frame, it interrupts the thread to schedule another input frame for processing. This achieves the control of the image processing module to continuously process the input frames of multiple image processing tasks. If the input frame of one image processing task is not sent to the image processing module in time, the image processing module can still continue to process the input frames of other image processing tasks, improving the utilization rate of the image processing module and the overall efficiency of video processing.

[0101] This application also provides a video processing apparatus, see [link to relevant documentation]. Figure 5 The video processing device may include an input module, an input buffer module, a pop-up module, and an image processing module.

[0102] The input module is used to obtain the frame information of the input frame, including the frame storage address;

[0103] The input buffer module is used to instantiate multiple input queues for multiple channels and store frame information into multiple input queues. Channels are pre-created according to the image processing tasks of the image processing module. Each channel has an input queue and channel attributes. The channel attributes represent the image processing strategy of the image processing task.

[0104] The pop-up module is used to obtain the frame information of an input frame and the channel attributes of the corresponding channel in the input queue from the input queue, generate the image processing command with the corresponding frame information and channel attributes, and send the image processing command to the image processing module.

[0105] The image processing module is used to parse image processing instructions, obtain input frames according to the parsed frame storage address, process the input frames according to the parsed channel attributes, obtain output frames, and start an interrupt thread. The interrupt thread is used to send a pop-up signal to the pop-up module. The pop-up module is also used to obtain the frame information of another input frame and the channel attributes of the corresponding channel in the input queue according to the pop-up signal, for the image processing module to process.

[0106] In some embodiments, the video processing apparatus further includes a push module, and the interrupt thread is further configured to send a push signal to the push module. The push module is configured to retrieve the output frame stored in the image processing module from the storage module according to the push signal, and release the storage space of the output frame.

[0107] In some embodiments, the video processing apparatus further includes an output module for sending an output frame and its frame attributes to a downstream image processing task. The frame attributes of the output frame are obtained by a pop-up module replacing a first attribute in the frame attributes of the input frame with a second attribute in the channel attributes. The first and second attributes are attributes of the same category, such as both being resolution or both being frame rate, etc.

[0108] This application also provides an electronic device, which includes one or more processors and a memory, and further includes means of the second aspect; the memory stores a computer program, the computer program including computer instructions, and one or more processors call the computer instructions to cause the electronic device to execute the video processing method of this application embodiment.

[0109] This application also provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to execute the video processing method of this application.

[0110] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the protection scope of this application.

Claims

1. A video processing method, characterized in that, include: Obtain the frame information of the input frame of the target video, wherein the frame information includes the frame storage address; The frame information is stored in input queues corresponding to multiple channels. The channels are pre-created according to the image processing task of the image processing module. Each channel is set with the input queue and channel attributes. The channel attributes characterize the image processing strategy of the image processing task. The frame information of an input frame and the channel attribute of the corresponding channel of the input queue are obtained from the input queue. A first storage address is requested in the storage module. An image processing command corresponding to the frame information, channel attribute and first storage address is generated and sent to the image processing module. The first storage address is used for the image processing module to store the output frame. The image processing module parses the image processing command, obtains the input frame according to the parsed frame storage address, processes the input frame according to the parsed channel attributes, obtains the output frame, and starts the interrupt thread. The interrupt thread obtains the frame information of another input frame and the channel attributes of the corresponding channel of the input queue for processing by the image processing module.

2. The video processing method according to claim 1, characterized in that, The image processing command that generates the corresponding frame information, channel attributes, and first storage address includes: The frame information is translated into the first command of the image processing module; Translate the channel attributes into a second command of the image processing module; The output frame address is translated into the third command of the image processing module; The image processing commands include the first command, the second command, and the third command.

3. The video processing method according to claim 1, characterized in that, The frame information of the input frame also includes frame attributes, and the method further includes: The first attribute in the frame attributes of the input frame is replaced with the second attribute in the channel attributes to obtain the frame attributes of the output frame, wherein the first attribute and the second attribute are attributes of the same category. The interrupt thread sends the output frame and its frame attributes to the downstream image processing task.

4. The video processing method according to claim 3, characterized in that, The method further includes: After obtaining the frame information of another input frame and the channel attributes of the corresponding channel of the input queue through the interrupt thread, and sending the output frame and the frame attributes of the output frame to the downstream image processing task, the interrupt thread ends.

5. The video processing method according to claim 1, characterized in that, Before obtaining the frame information of the input frame of the target video, the method further includes: Create a first number of channels, where the first number is the number of image processing tasks of the image processing module; In the storage module, an attribute memory is allocated for the channel, and the channel attributes are stored in the attribute memory. Create the input queue corresponding to the channel and set the scheduling rules for the input queue.

6. The video processing method according to claim 1, characterized in that, The step of obtaining frame information of an input frame from the input queue includes: According to the polling scheduling rules, determine the input queue for obtaining the frame information of the input frame; According to the first-in-first-out rule, the frame information of one input frame is obtained from the determined input queue.

7. A video processing apparatus, characterized in that, include: An input module is used to acquire frame information of an input frame, the frame information including the frame storage address; An input buffer module is used to instantiate multiple input queues for multiple channels and store the frame information into multiple input queues. The channels are pre-created according to the image processing task of the image processing module. The channels are set with the input queues and channel attributes. The channel attributes represent the image processing strategy of the image processing task. The pop-up module is used to obtain the frame information of an input frame and the channel attribute of the corresponding channel of the input queue from the input queue, apply for a first storage address in the storage module, generate an image processing command corresponding to the frame information, channel attribute and the first storage address, and send the image processing command to the image processing module. The first storage address is used for the image processing module to store the output frame. The image processing module is used to parse the image processing command, obtain the input frame according to the parsed frame storage address, process the input frame according to the parsed channel attributes, obtain the output frame, and start an interrupt thread. The interrupt thread is used to send a pop-up signal to the pop-up module. The pop-up module is also used to obtain the frame information of another input frame and the channel attributes of the corresponding channel of the input queue according to the pop-up signal, for the image processing module to process.

8. The video processing apparatus according to claim 7, characterized in that, The pop-up module includes: The scheduling unit is used to retrieve frame information of one of the input frames from one of the input queues; A storage configuration unit is used to request the first storage address in the storage module; The translation unit is used to translate the frame information, channel attributes, and first storage address into image processing commands of the image processing module.

9. The video processing apparatus according to claim 7, characterized in that, The device further includes a push module, and the interrupt thread is also used to send a push signal to the push module. The push module is used to retrieve the output frame stored in the image processing module in the storage module according to the push signal, and release the storage space of the output frame.

10. The video processing apparatus according to claim 7, characterized in that, The device further includes: An output module is used to send the output frame and the frame attributes of the output frame to a downstream image processing task. The frame attributes of the output frame are obtained by the pop-up module by replacing the first attribute in the frame attributes of the input frame with the second attribute in the channel attributes. The first attribute and the second attribute are attributes of the same category.

11. An electronic device comprising one or more processors and a memory, characterized in that, It also includes the apparatus of any one of claims 7-10; the memory stores a computer program, the computer program including computer instructions, and the one or more processors invoke the computer instructions to cause the electronic device to perform the video processing method of any one of claims 1-6.

12. A computer-readable storage medium, characterized in that, The device includes a computer program that, when run on an electronic device, causes the electronic device to perform the video processing method as described in any one of claims 1-6.

Images