Image processing device and method, equipment and storage medium

Abstract

The invention provides an image processing device and method, equipment and a storage medium, and the device comprises an image processing module which is used for obtaining a first indication signal, and writing an intermediate image into a first cache module based on the first indication signal; outputting a first update signal after the writing of one intermediate image is completed; the synchronous adapter is used for updating a first preset value based on the first update signal to obtain a first update value; outputting a second indication signal based on the first update value; the video encoder is used for reading an intermediate image from the first cache module based on the second indication signal; outputting a second update signal after the encoding of the intermediate image is completed; the synchronous adapter is used for updating a second preset value based on the second update signal to obtain a second update value; and outputting the first indication signal to the image processing module based on the second update value. According to the technical scheme, the signals between the image processing module and the video encoder are synchronized through hardware.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing apparatus, method, device and storage medium. Background Art

[0002] The system architecture of security camera products usually includes multiple key components that work together to achieve efficient video monitoring and security prevention.

[0003] In the related art, the system architecture of security camera products processes images through ISP (Image Signal Processor) to obtain the initial image; the initial image processing result is output offline to the image processing module; after the image processing module processes the initial image, the processed initial image is output offline to the video encoder for encoding. Among them, the data interaction involved in the above process is scheduled and processed by software. For example: the scheduling between data caches is based on pure software scheduling. The input of data is obtained from the upper level based on the software and output to the lower level.

[0004] However, during the scheduling process through software, there may be the influence of operating system scheduling, which may cause delays in obtaining data and outputting data to the lower level, thereby increasing the delay of the entire pipeline. Summary of the invention

[0005] The embodiments of the present application provide an image processing apparatus, method, device and storage medium, which use hardware to synchronize the signal between the image processing module and the video encoder, thereby solving the problem of delay in acquiring data and outputting data to the lower level. The technical solution is as follows:

[0006] According to a first aspect of an embodiment of the present application, there is provided an image processing device, the device comprising:

[0007] Image processing module, synchronization adapter and video encoder;

[0008] The image processing module is used to obtain a first indication signal, and write an intermediate image into the first cache module based on the first indication signal; after completing the writing of one of the intermediate images, output a first update signal; each of the intermediate images is obtained based on the image processing module processing an image to be processed; the first indication signal indicates a request for one of the intermediate images;

[0009] The synchronization adapter is used to update the first preset value based on the first update signal to obtain a first update value; based on the first update value, output a second indication signal; the second indication signal indicates reading one of the intermediate images;

[0010] The video encoder is used to read one of the intermediate images from the first cache module based on the second indication signal; after completing encoding of one of the intermediate images, output a second update signal;

[0011] The synchronization adapter is further used to update the second preset value based on the second update signal to obtain a second update value; and output the first indication signal to the image processing module based on the second update value.

[0012] In a possible implementation manner, each of the to-be-processed images includes a plurality of to-be-processed image blocks;

[0013] Each of the intermediate images includes a plurality of intermediate image blocks;

[0014] Each of the intermediate image blocks is obtained based on the image processing module processing one of the image blocks to be processed.

[0015] In a possible implementation, the device further includes a second cache module; the second cache module is used to store a plurality of the image blocks to be processed.

[0016] In a possible implementation, the image processing module is further configured to obtain a first sub-indication signal, read one of the to-be-processed image blocks from the second cache module based on the first sub-indication signal, process one of the to-be-processed image blocks, and obtain one of the intermediate image blocks;

[0017] Writing one of the intermediate image blocks into the first cache module, and outputting a first sub-update signal after completing writing of the one of the intermediate image blocks; the first sub-update signal is a part of the first update signal.

[0018] In a possible implementation, the first cache module includes at least one first cache unit; each of the first cache units is used to store one of the intermediate image blocks.

[0019] In a possible implementation, the synchronization adapter includes a plurality of registers and a control unit;

[0020] The control unit is used to update the first sub-preset value based on the first sub-update signal to obtain the first sub-update value; output a second sub-indication signal based on the first sub-update value; the second sub-indication signal indicates reading one of the intermediate image blocks; the first sub-preset value is the initial value of the first register; the first sub-preset value is a part of the first preset value.

[0021] In a possible implementation, the video encoder is further used to read one of the intermediate image blocks from the first cache module based on the second sub-indication signal, and output a second sub-update signal after completing encoding of one of the intermediate image blocks.

[0022] In one possible implementation, the synchronization adapter is also used to update the second sub-preset value based on the second sub-update signal to obtain a second sub-update value; based on the second sub-update value, output the first sub-indication signal to the image processing module; the second sub-preset value is the initial value of the second register; the second sub-preset value is a part of the second preset value; the second sub-update value is a part of the second update value.

[0023] According to a second aspect of an embodiment of the present application, there is provided an image processing method, the method comprising:

[0024] Obtaining a first indication signal, and writing an intermediate image into a first cache module based on the first indication signal; after completing the writing of one of the intermediate images, outputting a first update signal; each of the intermediate images is obtained based on processing an image to be processed by the image processing module; the first indication signal indicates a request for one of the intermediate images;

[0025] Based on the first update signal, updating the first preset value to obtain a first update value; based on the first update value, outputting a second indication signal; the second indication signal indicates reading one of the intermediate images;

[0026] Based on the second indication signal, read one of the intermediate images from the first cache module; after completing the encoding of one of the intermediate images, output a second update signal;

[0027] Based on the second update signal, the second preset value is updated to obtain a second update value; based on the second update value, the first indication signal is output to the image processing module.

[0028] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the above-mentioned image processing method.

[0029] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the above-mentioned image processing method.

[0030] In an embodiment of the present application, an image processing device is provided, and a synchronization adapter is provided, and the synchronization adapter is connected to an image processing module and a video encoder respectively; the image processing module obtains a first indication signal, and writes an intermediate image into a first cache module based on the first indication signal; after completing the writing of an intermediate image, a first update signal is output; the synchronization adapter is used to update a first preset value based on the first update signal to obtain a first update value; based on the first update value, a second indication signal is output; the video encoder is used to read an intermediate image from the first cache module based on the second indication signal; after completing the encoding of the intermediate image, a second update signal is output; the synchronization adapter is also used to update a second preset value based on the second update signal to obtain a second update value; based on the second update value, the first indication signal is output to the image processing module. The above technical solution realizes signal synchronization between the image processing module and the video encoder through the synchronization adapter, so that the software does not need to perform scheduling processing, thereby solving the problem of delay in obtaining data and outputting data to the lower level, and reducing the delay of the overall pipeline. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0033] Figure 2 It is a schematic diagram of a system architecture for scheduling based on software in the related art;

[0034] Figure 3 is a structural schematic diagram of a first image processing device provided according to an embodiment of the present application;

[0035] Figure 4 is a structural schematic diagram of slicing an image to be processed provided according to an embodiment of the present application;

[0036] Figure 5 is a structural schematic diagram of a second image processing device provided according to an embodiment of the present application;

[0037] Figure 6 is a flowchart of an image processing method provided according to an embodiment of the present application;

[0038] Figure 7 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0039] Figure 8 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0041] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0042] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0043] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0044] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0045] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0046] In the terminal 101, an image processing device and an ISP are provided. The image processing device includes a synchronization adapter. The synchronization adapter is a component used to synchronize data between different devices, systems or networks. It can ensure that the data remains consistent and real-time between different endpoints. The synchronization adapter is usually responsible for the transmission, conversion and synchronization control of data to achieve seamless data exchange between different systems.

[0047] The terminal 101 may be a smart phone having an image processing device and an ISP, a wearable device, a personal computer, a laptop, a tablet computer, a smart TV, a vehicle-mounted terminal, etc.

[0048] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0049] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0050] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.

[0051] Figure 2 It is a schematic diagram of a system architecture for scheduling based on software in the related art.

[0052] Combine the following Figure 2 , an exemplary description is given of the system architecture in the related technology.

[0053] The system architecture includes an image sensor, an ISP, an image processing module, a video encoder, a first cache module and a second cache module.

[0054] Among them, the image sensor is the core component of the camera, which is used to collect light signals and convert them into electrical signals. The ISP is used to process the electrical signals output by the image sensor to obtain the initial image; the ISP stores the initial image in the second cache module; the image processing module is used to read the initial image from the second cache module, process the initial image, and obtain the processing result; and then store the processing result in the first cache module. Among them, the processing of the image processing module includes but is not limited to image enhancement, feature detection, and object segmentation. Image enhancement mainly improves the visual effect of the image by means of sharpening, smoothing, contrast adjustment, etc. Feature detection is used to detect features such as edges, corners, and textures in the image, providing a basis for subsequent image analysis, recognition, and other tasks. Object segmentation is used to separate objects in the image from the background for further analysis and processing. The video encoder is used to read the processing result from the first cache module and convert the processing result into a specific encoding format for storage and transmission.

[0055] The above process involves data interaction. For example: the ISP stores the initial image in the second cache module; the image processing module reads the initial image from the second cache module, etc., all of which are scheduled and processed based on software. When scheduling and processing based on software, it is necessary to call related functions to read data through related functions. However, there is a time difference in the process of executing related functions, which leads to delays in reading the initial image and sending the initial image to the next level, which in turn leads to a larger delay in the entire pipeline. In addition, the second cache module is used to cache the entire frame of the initial image; the first cache module is also used to cache the entire frame of processing results. Therefore, bandwidth and memory are wasted.

[0056] In order to solve the above technical problems, the present application provides an image processing device for solving the problem of delay when reading an initial image and sending the initial image to the next level, as well as the problem of wasting bandwidth and memory.

[0057] Figure 3 It is a structural diagram of a first image processing device provided according to an embodiment of the present application, and the device includes: an image processing module 301, a synchronization adapter 302 and a video encoder 303.

[0058] In some embodiments, the image processing module 301, the synchronization adapter 302 and the video encoder 303 are electrically connected in sequence. In the embodiment of the present application, the synchronization adapter 302 is used to realize the signal synchronization between the image processing module 301 and the video encoder 303, so that the software does not need to perform scheduling processing, thereby solving the problem of delay caused by software scheduling processing, thereby reducing the delay of the overall pipeline. Among them, the image processing module 301 is electrically connected to the synchronization adapter 302 through two signal lines. The synchronization adapter 302 is electrically connected to the video encoder 303 through two signal lines. For example: the image processing module 301 is electrically connected to the synchronization adapter 302 through the first signal line and the second signal line. The synchronization adapter 302 is electrically connected to the video encoder 303 through the third signal line and the fourth signal line. It should be noted that the image processing module 301, the synchronization adapter 302 and the video encoder 303 are all hardware structures. The function of the synchronization adapter 302 can be set and modified by programming.

[0059] Figure 4 It is a structural schematic diagram of slicing an image to be processed provided according to an embodiment of the present application.

[0060] Combine the following Figure 4 , an exemplary description is given of the slices of the image to be processed.

[0061] In order to solve the problem of wasting bandwidth and memory in the related art, in an embodiment of the present application, a to-be-processed image is horizontally sliced ​​to obtain a plurality of to-be-processed image blocks.

[0062] In some embodiments, each image to be processed includes multiple image blocks to be processed; each intermediate image includes multiple intermediate image blocks; and each intermediate image block is obtained based on processing of one image block to be processed by the image processing module 301 .

[0063] In some embodiments, the device further includes a second cache module 305; the second cache module 305 is used to store multiple image blocks to be processed.

[0064] In one example, the ISP, the second cache module 305 and the image processing module 301 are electrically connected in sequence. The ISP outputs an image to be processed. An image to be processed is used as an example for explanation. The image to be processed is sliced ​​horizontally based on an image slicing algorithm to obtain a plurality of image blocks to be processed. The plurality of image blocks to be processed are stored in the second cache module 305. The second cache module 305 includes a plurality of second cache units, each of which is used to store an image block to be processed. For example: the image to be processed includes 4 image blocks to be processed. Then the second cache module 305 includes four second cache units. Similarly, each image to be processed can be sliced ​​horizontally, and then each image to be processed includes 4 image blocks to be processed. In addition, each time the image processing module 301 processes an image block to be processed, an intermediate image block is obtained. It should be noted that the image slicing algorithm can be obtained from the relevant technology, and the embodiments of the present application will not be repeated.

[0065] Figure 5 It is a structural schematic diagram of a second image processing device provided according to an embodiment of the present application.

[0066] Combine the following Figure 5 , the second image processing device is exemplarily described.

[0067] In some embodiments, the image processing module 301 is used to obtain a first indication signal, write an intermediate image into the first cache module 304 based on the first indication signal; after completing the writing of an intermediate image, output a first update signal; each intermediate image is obtained based on the image processing module 301 processing an image to be processed; the first indication signal indicates a request for an intermediate image. Optionally, the image processing module 301 obtains the first indication signal through a first signal line, writes the first intermediate image into the first cache module 304 based on the first indication signal; after completing the writing of an intermediate image, outputs a second update signal through a second signal line. It should be noted that the signal line can be directly obtained from the relevant technology, and the embodiments of the present application will not be repeated.

[0068] In one example, for a first image to be processed, the image processing module 301 is configured to read the first image block to be processed from the first cache module 304 , process the first image block to be processed, obtain a first intermediate image block, and write the first intermediate image block into the first cache module 304 .

[0069] In some embodiments, the first cache module 304 includes at least one first cache unit; each first cache unit is used to store an intermediate image block. Optionally, when the first cache module 304 includes a first cache unit, the image processing module 301 writes the first intermediate image block into the first cache unit. That is, the first cache module 304 does not need to cache the entire frame of the intermediate image, but only needs to cache one intermediate image block. Since the size of the first intermediate image block is much smaller than the size of the entire frame of the intermediate image, bandwidth and memory resources are saved. Optionally, when the first cache module 304 includes two first cache units, the image processing module 301 writes the first intermediate image block into one of the first cache units in the first cache module 304. The two first cache units correspond to different pipelines respectively. The first pipeline is used to control the writing and reading of the first first cache unit; the second pipeline controls the writing and reading of the second first cache unit. Optionally, while the image processing module 301 writes the first intermediate image block into the first first cache unit, the video encoder 303 writes the second intermediate image block into the second first cache unit. Optionally, while the image processing module 301 writes the first intermediate image block to the first first cache unit, the video encoder 303 reads the second intermediate image block from the second first cache unit.

[0070] Similarly, the first cache module 304 may also be provided with three first cache units, etc. And as long as the total size of the intermediate image blocks cached by the first cache module 304 is smaller than the size of the entire frame of the intermediate image, bandwidth and memory resources can be saved.

[0071] In one example, the image processing module 301 is further used to obtain a high and valid first sub-indication signal, read a to-be-processed image block from the first cache module 304 based on the high and valid first sub-indication signal, process the to-be-processed image block, and obtain an intermediate image block; write an intermediate image block into a first cache unit, and after completing the writing of an intermediate image block, output a high and valid first sub-update signal; the first sub-update signal is a part of the first update signal. Optionally, after the image processing module 301 writes an intermediate image block into the first cache unit, the image processing module 301 is in a waiting state, and after obtaining a high and valid first sub-indication signal through the first signal line, the waiting state is ended. Read an to-be-processed image block from the first cache module 304, process the to-be-processed image block, write the to-be-processed image block into a first cache unit, and output a high and valid first sub-update signal through the second signal line, and then be in a waiting state. For example: the first sub-indication signal can be expressed as "Slice_read_request". The first sub-update signal can be expressed as "Slice_buf_write_done".

[0072] In some embodiments, the synchronization adapter 302 is used to update the first preset value based on the first update signal to obtain the first update value; based on the first update value, output the second indication signal; the second indication signal indicates reading an intermediate image. Optionally, the synchronization adapter 302 obtains the first update signal through the second signal line, updates the first preset value to obtain the first update value; based on the first update value, outputs the second indication signal through the third signal line.

[0073] In one example, the synchronization adapter 302 includes a plurality of registers and a control unit; the control unit is used to update the first sub-preset value based on the first sub-update signal to obtain the first sub-update value; based on the first sub-update value, output the second sub-indication signal; the second sub-indication signal indicates reading an intermediate image block; the first sub-preset value is the initial value of the first register; the first sub-preset value is a part of the first preset value. Optionally, in the synchronization adapter 302, each register is provided with a serial number, and registers with different serial numbers match different first cache units. Each register is provided with an initial value. For example: the first register matches the first first cache unit. The initial value of the first register is "0"; the first sub-update value is "1". The second sub-indication signal can be expressed as "Slice_read_request". Optionally, after the control unit obtains a high and valid first sub-update signal through the second signal line, the initial value of the first register is updated to the first sub-update value, and a high and valid second sub-indication signal is output through the third signal line.

[0074] In some embodiments, the video encoder 303 is used to read an intermediate image from the first cache module 304 based on the second indication signal; after completing the encoding of the intermediate image, output the second update signal. Optionally, the video encoder 303 obtains the second indication signal through the third signal line, reads an intermediate image from the first cache module 304; after completing the encoding of the intermediate image, outputs the second update signal through the fourth signal line.

[0075] In one example, the video encoder 303 is further used to read an intermediate image block from the first cache module 304 based on the second sub-indication signal, and output a second sub-update signal after completing the encoding of the intermediate image block. Optionally, after the video encoder 303 reads an intermediate image block, the video encoder 303 is in a waiting state, and after obtaining a high and valid second sub-indication signal through the third signal line, the waiting state is ended. An intermediate image block is read from a first cache unit, and after completing the encoding of the intermediate image block, a high and valid second sub-update signal is output through the fourth signal line, and then the waiting state is entered. For example: the second sub-update signal can be expressed as "Slice_buf_write_done".

[0076] In some embodiments, the synchronization adapter 302 is further used to update the second preset value based on the second update signal to obtain a second update value; and output the first indication signal to the image processing module 301 based on the second update value.

[0077] In some embodiments, the synchronization adapter 302 is also used to update the second sub-preset value based on the second sub-update signal to obtain the second sub-update value; based on the second sub-update value, output the first sub-indication signal to the image processing module 301; the second sub-preset value is the initial value of the second register; the second sub-preset value is a part of the second preset value; the second sub-update value is a part of the second update value.

[0078] It should be noted that: when the image processing device provided in the above embodiment executes the corresponding steps, it only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0079] In an embodiment of the present application, a synchronization adapter is provided, and the synchronization adapter is connected to the image processing module and the video encoder respectively; the image processing module obtains a first indication signal, and writes an intermediate image into the first cache module based on the first indication signal; after completing the writing of an intermediate image, a first update signal is output; the synchronization adapter is used to update the first preset value based on the first update signal to obtain the first update value; based on the first update value, a second indication signal is output; the video encoder is used to read an intermediate image from the first cache module based on the second indication signal; after completing the encoding of the intermediate image, a second update signal is output; the synchronization adapter is also used to update the second preset value based on the second update signal to obtain the second update value; based on the second update value, the first indication signal is output to the image processing module. The above technical solution realizes the signal synchronization between the image processing module and the video encoder through the synchronization adapter, so that the software does not need to perform scheduling processing, thereby solving the problem of delay in obtaining data and outputting data to the lower level, and reducing the delay of the overall pipeline.

[0080] Figure 6 is a flow chart of an image processing method provided according to an embodiment of the present application, such as Figure 6 As shown, in the embodiment of the present application, the application is described by taking the application on a terminal having an image processing device as an example. The method comprises the following steps:

[0081] In step 601, a first indication signal is obtained, and an intermediate image is written into a first cache module based on the first indication signal; after the writing of the intermediate image is completed, a first update signal is output.

[0082] Each intermediate image is obtained based on an image to be processed by the image processing module; the first indication signal indicates a request for an intermediate image;

[0083] In step 602, based on the first update signal, the first preset value is updated to obtain a first update value; based on the first update value, a second indication signal is output.

[0084] Wherein, the second indication signal indicates reading an intermediate image;

[0085] In step 603, based on the second indication signal, an intermediate image is read from the first cache module; after the encoding of the intermediate image is completed, a second update signal is output;

[0086] In step 604, based on the second update signal, the second preset value is updated to obtain a second update value; based on the second update value, the first indication signal is output to the image processing module.

[0087] In some embodiments, each image to be processed includes a plurality of image blocks to be processed;

[0088] Each intermediate image includes a plurality of intermediate image blocks;

[0089] Each intermediate image block is obtained by processing an image block to be processed based on an image processing module.

[0090] In some embodiments, a second cache module is also included; the second cache module is used to store multiple image blocks to be processed.

[0091] In some embodiments, the method further includes: acquiring a first sub-indication signal, reading a to-be-processed image block from a second cache module based on the first sub-indication signal, processing the to-be-processed image block, and obtaining an intermediate image block;

[0092] An intermediate image block is written into the first cache module, and after the writing of the intermediate image block is completed, a first sub-update signal is output; the first sub-update signal is a part of the first update signal.

[0093] In a possible implementation, the first cache module includes at least one first cache unit; each first cache unit is used to store an intermediate image block.

[0094] In some embodiments, the synchronization adapter includes a plurality of registers and a control unit;

[0095] A control unit is used to update a first sub-preset value based on a first sub-update signal to obtain a first sub-update value; output a second sub-indication signal based on the first sub-update value; the second sub-indication signal indicates reading an intermediate image block; the first sub-preset value is an initial value of a first register; the first sub-preset value is a part of the first preset value.

[0096] In some embodiments, the method further includes: based on the second sub-indication signal, reading an intermediate image block from the first cache module, and outputting a second sub-update signal after completing the encoding of the intermediate image block.

[0097] In some embodiments, the method also includes: based on the second sub-update signal, updating the second sub-preset value to obtain a second sub-update value; based on the second sub-update value, outputting the first sub-indication signal to the image processing module; the second sub-preset value is the initial value of the second register; the second sub-preset value is a part of the second preset value; the second sub-update value is a part of the second update value.

[0098] In addition, the image processing device and the image processing method provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the device embodiments, which will not be repeated here.

[0099] In an embodiment of the present application, a synchronization adapter is provided, and the synchronization adapter is connected to the image processing module and the video encoder respectively; the image processing module obtains a first indication signal, and writes an intermediate image into the first cache module based on the first indication signal; after completing the writing of an intermediate image, a first update signal is output; the synchronization adapter is used to update the first preset value based on the first update signal to obtain the first update value; based on the first update value, a second indication signal is output; the video encoder is used to read an intermediate image from the first cache module based on the second indication signal; after completing the encoding of the intermediate image, a second update signal is output; the synchronization adapter is also used to update the second preset value based on the second update signal to obtain the second update value; based on the second update value, the first indication signal is output to the image processing module. The above technical solution realizes the signal synchronization between the image processing module and the video encoder through the synchronization adapter, so that the software does not need to perform scheduling processing, thereby solving the problem of delay in obtaining data and outputting data to the lower level, and reducing the delay of the overall pipeline.

[0100] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.

[0101] Taking computer equipment as the terminal as an example, Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Figure 7 The terminal 700 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 700 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0102] Typically, the terminal 700 includes a processor 701 and a memory 702 .

[0103] The processor 701 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0104] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one program code, which is used to be executed by the processor 701 to implement the process provided by the method embodiment of the present application for the terminal execution in the above method.

[0105] In some embodiments, the terminal 700 may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 704, a camera assembly 705, an audio circuit 706 and a power supply 707.

[0106] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0107] The display screen 704 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 704 is a touch display screen, the display screen 704 also has the ability to collect touch signals on the surface or above the surface of the display screen 704. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 704 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 704 can be one, set on the front panel of the terminal 700; in other embodiments, the display screen 704 can be at least two, respectively set on different surfaces of the terminal 700 or in a folding design; in other embodiments, the display screen 704 can be a flexible display screen, set on the curved surface or folding surface of the terminal 700. Even, the display screen 704 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 704 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0108] The camera assembly 705 is used to capture images or videos. In some embodiments, the camera assembly 705 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 705 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0109] The audio circuit 706 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 706 may also include a headphone jack.

[0110] The power supply 707 is used to power various components in the terminal 700. The power supply 707 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 707 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0111] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the terminal 700, and the terminal 700 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0112] Take the computer device as a server as an example. Figure 8 800 is a structural diagram of a server provided in an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 801 and one or more memories 802, wherein the one or more memories 802 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 801 to implement the above-mentioned image processing method. Of course, the server 800 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 800 may also include other components for implementing device functions, which will not be described in detail here.

[0113] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above method. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0114] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0115] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing device, characterized in that: include: Image processing module, synchronization adapter and video encoder; The image processing module is configured to obtain a first indication signal, and write an intermediate image into a first cache module based on the first indication signal; After completing the writing of one of the intermediate images, outputting a first update signal; each of the intermediate images is obtained based on the image processing module processing an image to be processed; The first indication signal indicates a request for one of the intermediate images; The synchronization adapter is used to update the first preset value based on the first update signal to obtain a first update value; Based on the first update value, output a second indication signal; The second indication signal indicates reading one of the intermediate images; The video encoder is used to read one of the intermediate images from the first cache module based on the second indication signal; After completing the encoding of one of the intermediate images, outputting a second update signal; The synchronization adapter is further used to update the second preset value based on the second update signal to obtain a second update value; and output the first indication signal to the image processing module based on the second update value.

2. The device according to claim 1, characterized in that Each of the images to be processed includes a plurality of image blocks to be processed; Each of the intermediate images includes a plurality of intermediate image blocks; Each of the intermediate image blocks is obtained based on the image processing module processing one of the image blocks to be processed.

3. The device according to claim 2, characterized in that The device further includes a second cache module; the second cache module is used to store a plurality of the image blocks to be processed.

4. The device according to claim 3, characterized in that The image processing module is further configured to obtain a first sub-indication signal, read one of the to-be-processed image blocks from the second cache module based on the first sub-indication signal, process one of the to-be-processed image blocks, and obtain one of the intermediate image blocks; Writing one of the intermediate image blocks into the first cache module, and outputting a first sub-update signal after completing writing of the one of the intermediate image blocks; the first sub-update signal is a part of the first update signal.

5. The device according to claim 4, characterized in that The first cache module includes at least one first cache unit; each of the first cache units is used to store one of the intermediate image blocks.

6. The device according to claim 4, characterized in that The synchronous adapter includes a plurality of registers and a control unit; The control unit is used to update the first sub-preset value based on the first sub-update signal to obtain the first sub-update value; output a second sub-indication signal based on the first sub-update value; the second sub-indication signal indicates reading one of the intermediate image blocks; the first sub-preset value is the initial value of the first register; the first sub-preset value is a part of the first preset value.

7. The device according to claim 6, characterized in that The video encoder is further configured to read one of the intermediate image blocks from the first cache module based on the second sub-indication signal, and output a second sub-update signal after completing encoding of one of the intermediate image blocks.

8. The device according to claim 7, characterized in that The synchronization adapter is also used to update the second sub-preset value based on the second sub-update signal to obtain the second sub-update value; based on the second sub-update value, output the first sub-indication signal to the image processing module; the second sub-preset value is the initial value of the second register; the second sub-preset value is a part of the second preset value; the second sub-update value is a part of the second update value.

9. An image processing method, characterized in that: include: Acquire a first indication signal, and write an intermediate image into a first cache module based on the first indication signal; After completing the writing of one of the intermediate images, outputting a first update signal; each of the intermediate images is obtained based on the image processing module processing an image to be processed; The first indication signal indicates a request for one of the intermediate images; Based on the first update signal, update the first preset value to obtain a first update value; Based on the first update value, output a second indication signal; The second indication signal indicates reading one of the intermediate images; Based on the second indication signal, reading one of the intermediate images from the first cache module; After completing the encoding of one of the intermediate images, outputting a second update signal; Based on the second update signal, update the second preset value to obtain a second update value; Based on the second update value, the first indication signal is output to the image processing module.

10. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the image processing method as claimed in claim 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the image processing method according to claim 9.

Images