Image processing method, architecture, electronic device, and storage medium

Abstract

The application discloses an image processing method, architecture, electronic equipment and storage medium, and belongs to the technical field of electronic equipment. The image processing method comprises the following steps: obtaining at least one image block based on interface image data of a first interface by a master control chip; rendering the first image block to obtain a rendered first image block, and sending the rendered first image block to an image processing chip, wherein the first image block is N image blocks in the at least one image block, N is a positive integer; processing the rendered first image block by the image processing chip to obtain a processed first image block; wherein the rendered first image block is transmitted to the image processing chip by the master control chip, or the rendered first image block is obtained from a first storage area of a memory by the image processing chip, and the first storage area is used for storing the rendered first image block.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, specifically relating to an image processing method, architecture, electronic device, and storage medium. Background Technology

[0002] Discrete display chips (hereinafter referred to as discrete graphics chips) are dedicated image processing chips that have significant advantages in improving the display effect of electronic display devices. By processing display frame rate, image resolution, color adjustment, and other aspects through discrete graphics chips, the visual experience of the human eye can be significantly enhanced.

[0003] In related technologies, a discrete graphics chip is placed between the main control chip and the screen. The main control chip first acquires all pixels of a frame of the interface image to be displayed on the screen, and then divides the frame into multiple image blocks based on all pixels. Each image block includes at least one pixel from the interface image. The main control chip then renders all image blocks of the frame, obtaining a rendered interface image, which is then cached. Further, the discrete graphics chip retrieves the rendered interface image from the cache for further processing before transmitting the processed image to the screen. Therefore, the discrete graphics chip needs to wait for the main control chip to finish rendering all image blocks of a frame before it can perform further image processing. This increases the processing time of the discrete graphics chip for the interface image, thus increasing the display latency. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, architecture, electronic device, and storage medium that can reduce the display latency of interface images.

[0005] In a first aspect, embodiments of this application provide an image processing method. The image processing apparatus includes an image processing chip and a main control chip. The method includes:

[0006] The main control chip obtains at least one image block based on the interface image data of the first interface;

[0007] The main control chip renders the first image block to obtain the rendered first image block. The first image block is N image blocks from at least one image block, where N is a positive integer.

[0008] The image processing chip processes the rendered first image block to obtain a processed first image block.

[0009] The rendered first image block is transmitted from the main control chip to the image processing chip, or the rendered first image block is obtained by the image processing chip from the first storage area of ​​the memory, and the first storage area is used to store the rendered first image block.

[0010] In a second aspect, embodiments of this application provide an image processing architecture, including an image processing chip and a main control chip. The main control chip includes a memory, a central processing unit (CPU) connected to the memory, an image processing unit (GPU) connected to the CPU and the memory, and a first interface connected to the image processing chip and the memory.

[0011] The CPU is used to control the GPU to obtain interface image data of the first interface from the memory, and to obtain at least one image block based on the interface image data.

[0012] The GPU is used to render a first image block, obtain the rendered first image block, and store the rendered first image block in a first storage area of ​​the memory; the first image block is N image blocks from at least one image block, where N is a positive integer;

[0013] An image processing chip is used to obtain a rendered first image block from a first storage area using a first interface, and to process the rendered first image block to obtain a processed first image block.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the image processing method as described in the first aspect.

[0015] Fourthly, embodiments of this application provide an electronic device that includes the image processing architecture described in any one of the second aspects.

[0016] Fifthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the image processing method as described in the first aspect.

[0017] In a sixth aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the image processing method as described in the first aspect.

[0018] In a seventh aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the image processing method as described in the first aspect.

[0019] In this embodiment, the main control chip obtains at least one image block based on the interface image data of the first interface. The main control chip renders the first image block to obtain a rendered first image block, and sends the rendered first image block to the image processing chip. The first image block consists of N image blocks from the at least one image block, where N is a positive integer. The image processing chip processes the rendered first image block to obtain a processed first image block. The rendered first image block is either transmitted from the main control chip to the image processing chip, or it is obtained by the image processing chip from a first storage area of ​​the memory, where the first storage area stores the rendered first image block. Thus, the image processing chip does not need to wait for the main control chip to finish rendering all image blocks of a frame of interface image. After the main control chip finishes rendering N image blocks, the image processing chip can perform image processing on the rendered N image blocks. This effectively reduces the processing time of the discrete graphics chip for the interface image, thereby reducing the display latency of the interface image. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0021] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0022] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0023] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0024] Figure 5 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 6A This is a schematic flowchart of an image processing procedure provided in an embodiment of this application;

[0026] Figure 6B This is a schematic flowchart of an image processing procedure provided in an embodiment of this application;

[0027] Figure 7 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of an image processing architecture provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of an image processing architecture provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of an image processing architecture provided in an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0032] Figure 12 This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0033] Figure 13 This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0034] Figure 14 This is a schematic diagram of a bus structure provided in an embodiment of this application;

[0035] Figure 15 This is a schematic diagram of a bus structure provided in an embodiment of this application;

[0036] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0037] Figure 17 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0039] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0040] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0041] It should be noted that the image processing method provided in this application can be executed by electronic devices such as mobile phones, tablets, laptops, PDAs, and in-vehicle electronic devices. Some embodiments of this application use electronic devices as the executing entity to illustrate the image processing method provided in this application.

[0042] The following will explain the terminology used in the embodiments of this application.

[0043] A System on Chip (SoC), also known as a system-on-a-chip or main control chip, is a system composed of multiple integrated circuits with specific functions. It typically includes modules such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Display Processing Unit (DPU), a Digital Signal Processor (DSP), Random Access Memory (RAM), and a Modem.

[0044] The Mobile Industry Processor Interface (MIPI) is an alliance established by mobile industry manufacturers to standardize hardware and software interfaces.

[0045] Display Serial Interface (DSI) is a protocol interface under the MIPI protocol specifically used for displaying data.

[0046] The receiver (RX) is the receiving end of the discrete graphics chip, which receives the content to be displayed.

[0047] The transmitting end (Transport, TX) is the transmitting end of the discrete graphics chip, which sends the content that has been processed by the discrete graphics chip.

[0048] Serial Communication Interface (SCI) is a general term for serial communication technology, as opposed to parallel communication. It was first proposed by Motorola.

[0049] Peripheral Component Interconnect Express (PCIe) is a high-speed serial computer expansion bus standard proposed by Intel in 2001. PCIe is a high-speed serial point-to-point dual-channel high-bandwidth transmission standard. Each connected device is allocated its own dedicated channel bandwidth and does not share the bus bandwidth. It mainly supports active power management, error reporting, end-to-end reliable transmission, hot-plugging, and quality of service functions.

[0050] The Advanced eXtensible Interface (AXI) is a bus protocol and a key component of the Advanced Microcontroller Bus Architecture (AMBA) protocol proposed by ARM. It is an on-chip bus designed for high performance, high bandwidth, and low latency. In high-performance on-chip systems, the AXI bus is commonly used to connect multiple master and slave devices, enabling address and data transfer between multiple memory-mapped devices.

[0051] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0052] The image processing method provided in this application embodiment can be applied to interface display scenarios, which can be video interfaces, game interfaces, or other interfaces. This embodiment does not impose specific limitations here.

[0053] Below, in conjunction with Figure 1The following explanation uses a game interface display scenario as an example. When a user wants to view the game application interface, they can trigger an operation on the game application icon displayed on the electronic device screen, such as double-clicking the game application icon. The CPU 111 in the main control chip 11 of the electronic device 10 can respond to the user's trigger operation on the game application icon and obtain the interface data of the game interface in real time. This interface data includes game materials and game instructions obtained by the main control chip from the background server of the game application in real time. The game materials include all the pixels that make up the game screen. The CPU 111 executes the game instructions, generates all the pixels that make up the game screen according to the game materials, and divides the game screen into at least one image block based on all the pixels that make up the game screen. Each image block has the same size. The CPU 111 then transmits the at least one image block to the GPU 112 in sequence. After the GPU 112 renders all the image blocks in the at least one image block, it stores the rendered at least one image block in the cache 113 to form the game screen to be displayed. Subsequently, triggered by a synchronization signal transmitted to screen 13, such as a TE signal, DPU 114 reads the game screen from memory 113 and transmits it to image processing chip 12 via MIPI interface 115. Image processing chip 12 then performs further processing on the game screen, such as frame interpolation and color enhancement. After further processing, image processing chip 12 transmits the processed game screen to screen 13 for display.

[0054] In this way, the graphics processing chip needs to wait for the GPU to finish rendering all the image blocks of the game screen before it can perform further processing on the game screen. Therefore, this increases the processing time of the graphics processing chip on the game screen, and thus increases the display latency of the game screen.

[0055] To address this, this application provides an image processing method. A main control chip obtains at least one image block based on interface image data of a first interface. The main control chip renders the first image block to obtain a rendered first image block, where the first image block consists of N image blocks from the at least one image block, and N is a positive integer. An image processing chip processes the rendered first image block to obtain a processed first image block. The rendered first image block is either transmitted from the main control chip to the image processing chip, or it is obtained by the main control chip from a first storage area of ​​the memory, where the first storage area stores the rendered first image block. Here, N is a positive integer. In this way, the image processing chip does not need to wait for the main control chip to finish rendering all image blocks of a frame of interface image. After the main control chip finishes rendering N image blocks, the image processing chip can perform image processing on the rendered N image blocks. This effectively reduces the processing time of the discrete graphics chip for interface images, thereby reducing the display latency of the interface images.

[0056] The image processing method provided in this application can be executed by an image processing device. For example, the image processing device can be an electronic device, or a functional component or entity within that electronic device. The following explanation uses an electronic device executing the image processing method as an example.

[0057] This application provides an image processing method applied to an image processing device, which includes a main control chip and an image processing chip. Figure 2 This is a schematic flowchart of the image processing method provided in the embodiments of this application, as shown below. Figure 2 As shown, the image processing method provided in this application embodiment may include the following steps 201 to 203:

[0058] Step 201: The main control chip obtains at least one image block based on the interface image data of the first interface.

[0059] In some embodiments of this application, the first interface described above may be the application interface of the first application.

[0060] In some embodiments of this application, the first application described above may be an application installed on an image processing device.

[0061] In some embodiments of this application, the first application described above may include, but is not limited to, any of the following: game applications, social software applications, payment applications, navigation applications, shopping applications, video applications, photo album applications, music applications, etc. The specific application can be determined based on actual usage needs, and this application embodiment does not impose any limitations.

[0062] In some embodiments of this application, the interface image data may include all pixels of a frame of interface image of the first application.

[0063] In some embodiments of this application, the backend server of the first application can obtain the interface image data and transmit the interface image data to the main control chip.

[0064] In some embodiments of this application, the main control chip can divide the interface image data into at least one image block based on the number of pixels in the interface image data by the CPU. Each image block in the at least one image block has the same size, that is, each image block contains the same number of pixels.

[0065] In some embodiments of this application, the size of the image blocks may also be set to be equal based on requirements.

[0066] For example, if the number of pixels in the above interface image data is 720*720, the main control chip can divide the interface image data into 45*45 image blocks (i.e., 2025 image blocks), and the size of each image block is 16*16, that is, each image block contains 256 pixels.

[0067] In some embodiments of this application, the main control chip can obtain at least one image block based on the interface image data of the first interface through the CPU.

[0068] Step 202: The main control chip renders the first image block to obtain the rendered first image block.

[0069] In some embodiments of this application, the first image block is N image blocks among the at least one image block, where N is a positive integer.

[0070] In some embodiments of this application, the main control chip can sequentially render N image blocks from the at least one image block, and sequentially send the N rendered image blocks from the at least one image block to the image processing chip.

[0071] For example, if the number of at least one image block is 2025, then when N is 5, the main control chip can first render the 1st to 5th image blocks out of the 2025 image blocks, and then transmit the rendered image blocks of the 1st to 5th image blocks to the image processing chip; then the main control chip can render the 6th to 10th image blocks out of the 2025 image blocks, and then transmit the rendered image blocks of the 6th to 10th image blocks to the image processing chip... Finally, the main control chip can render the 2021st to 2025th image blocks out of the 2025 image blocks, and then transmit the rendered image blocks of the 2021st to 2025th image blocks to the image processing chip.

[0072] In some embodiments of this application, the main control chip can render the first image block using a GPU.

[0073] In some embodiments of this application, the main control chip can send the rendered first image block to the image processing chip via the MIPI interface.

[0074] Step 203: The image processing chip processes the rendered first image block to obtain the processed first image block.

[0075] In some embodiments of this application, the rendered first image block is transmitted from the main control chip to the image processing chip, or the rendered first image block is obtained by the image processing chip from a first storage area of ​​the memory, wherein the first storage area is used to store the rendered first image block.

[0076] In some embodiments of this application, the aforementioned memory can be Double Data Rate SDRAM Synchronous Dynamic Random Access Memory (DDRSDRAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), etc. The specific type can be determined according to actual needs, and this embodiment does not impose specific limitations here.

[0077] In some embodiments of this application, the image processing chip can perform image processing such as frame interpolation and color enhancement on the rendered first image block. The specific processing can be determined according to actual needs, and no specific limitations are made here.

[0078] In some embodiments of this application, after processing all image blocks in at least one image block, the image processing chip can combine all the processed image blocks into a frame of processed interface image, and then transmit the pixels in the frame of processed interface image to the screen for display row by row.

[0079] In some embodiments of this application, the image processing chip may also transmit the pixels in the processed image block to the screen display row by row after processing at least one row of image blocks.

[0080] In the image processing method provided in this application embodiment, the main control chip obtains at least one image block based on the interface image data of the first interface; the main control chip renders the first image block to obtain a rendered first image block, and sends the rendered first image block to the image processing chip. The first image block consists of N image blocks from at least one image block; the image processing chip processes the rendered first image block to obtain a processed first image block. Thus, the image processing chip does not need to wait for the main control chip to finish rendering all image blocks of a frame of interface image. After the main control chip finishes rendering N image blocks, the image processing chip can perform image processing on the rendered N image blocks. This effectively reduces the processing time of the discrete graphics chip for the interface image, thereby reducing the display latency of the interface image.

[0081] In some embodiments of this application, each image patch is the same size; combined with Figure 2 ,like Figure 3 As shown, after step 203 above, the image processing method provided in this application embodiment may further include the following step 204:

[0082] Step 204: The image processing chip transmits the pixels of at least one row of the processed image block to the screen for display, row by row.

[0083] In some embodiments of this application, after processing N rendered image blocks, the image processing chip can store the processed N image blocks in its cache. Then, if the image processing chip stores at least one row of processed image blocks in its cache, the image processing chip will transmit the pixels of at least one row of processed image blocks to the screen for display, row by row.

[0084] It should be noted that the image processing chip sequentially transmits each row of pixels in at least one row of the processed image block to the screen for display.

[0085] For example, after the image processing chip processes the first to fifth image blocks after rendering, it can store these processed image blocks in the image processing chip's cache. Similarly, after processing the sixth to tenth image blocks, it can store them in the image processing chip's cache... and so on. After processing the forty-first to forty-fifth image blocks, it can store them in the image processing chip's cache. Then, with the first row of image blocks consisting of the processed first to forty-fifth image blocks stored in the image processing chip's cache, the image processing chip transmits the processed first row of image blocks to the screen for display.

[0086] In this way, the image processing chip transmits the pixels of at least one row of the processed image block to the screen for display row by row, so that the screen does not have to wait for the image processing chip to process a complete frame of image before it can display the image, thereby effectively reducing the image display latency.

[0087] In some embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, prior to step 202 above, the image processing method provided in this application embodiment may further include the following steps 205a to 205c:

[0088] Step 205a: The main control chip receives the first trigger signal generated by the screen.

[0089] In some embodiments of this application, the period of the first trigger signal is the duration required for the screen to display one frame of interface image.

[0090] In some embodiments of this application, the aforementioned frame of interface image may be a frame of interface image of the aforementioned first application.

[0091] In some embodiments of this application, the first trigger signal is used to trigger the display of a frame of interface image.

[0092] In some embodiments of this application, the first trigger signal can be a synchronization signal, such as a TE signal.

[0093] Step 205b: The main control chip multiplies the frequency of the first trigger signal to generate the second trigger signal.

[0094] In some embodiments of this application, the second period of the second trigger signal is less than the first period of the first trigger signal.

[0095] In some embodiments of this application, the second trigger signal is used to render and process the first image block.

[0096] In some embodiments of this application, the first frequency of the second trigger signal can be M times the second frequency of the first trigger signal, that is, the second period is 1 / M of the first period, where M is a positive integer.

[0097] Understandably, when the GPU in the main control chip transmits all rendered image blocks to the image processing chip based on the aforementioned first trigger signal, on the one hand, due to varying GPU rendering loads—some interface images render quickly, while others render slowly—the GPU, after rendering all image blocks of a frame of interface image, needs to wait for the next cycle of the aforementioned first trigger signal before transmitting the rendered frame of interface image to the image processing chip. On the other hand, if the time taken to render all image blocks of a frame of interface image is less than the cycle of the aforementioned first trigger signal, then the GPU in the main control chip, after rendering all images of a frame of interface image, must wait for the next trigger of the aforementioned first trigger signal before transmitting the rendered frame of interface image to the image processing chip. This increases the image transmission latency.

[0098] For example Figure 6A As shown, the F-1 frame interface image is rendered just before the previous rising edge arrives, and the memory stores a portion of the F-1 frame interface image when the previous rising edge arrives. Therefore, the main control chip can read the complete rendered F-1 frame interface image from the memory via the MIPI interface when the previous rising edge arrives and transmit it to the image processing chip, allowing the image processing chip to process the rendered F-1 frame interface image. The F-1 frame interface image is rendered very quickly; therefore, the F-1 frame interface image is rendered before the current rising edge of the TE signal arrives, and the buffer already stores the complete rendered F-1 frame interface image. However, the main control chip still needs to wait for the current rising edge of the TE signal to be triggered before it can read the complete rendered F-1 frame interface image from the memory via the MIPI interface and transmit it to the image processing chip. The image processing chip also needs to wait for the main control chip to transmit the rendered F-1 frame interface image via the MIPI interface before it can process the rendered F-1 frame interface image, resulting in a relatively large image transmission delay. F is a positive integer.

[0099] On the other hand, the workload of GPU rendering varies; some interface images are rendered quickly, while others are rendered slowly. Therefore, after the GPU finishes rendering all the image blocks of a frame, it needs to wait for the next cycle of the aforementioned first trigger signal before it can transmit all the image blocks of that frame to the image processing chip for processing. If there are still image blocks in the current frame that have not been rendered and the trigger signal of the current cycle is missed, it needs to wait for the trigger signal of the next cycle before it can transmit all the image blocks in the current frame to the image processing chip. Therefore, the image transmission delay is significant.

[0100] For example Figure 6B As shown, the F-1 frame interface image is rendered just before the previous rising edge arrives, and the memory stores a portion of the F-1 frame interface image at the arrival of the previous rising edge. Therefore, the main control chip can read the complete rendered F-1 frame interface image from the memory via the MIPI interface and transmit it to the image processing chip at the arrival of the previous rising edge, allowing the image processing chip to process the rendered F-1 frame interface image. The F-frame interface image is rendered more slowly; at the arrival of the current rising edge of the TE signal, some image blocks in the F-frame interface image have not yet been rendered. Therefore, the buffer only stores a portion of the rendered F-frame interface image at the arrival of the current rising edge, meaning the F-frame interface image slightly misses the current rising edge of the TE signal. Therefore, the main control chip needs to wait for the next rising edge of the TE signal to trigger before it can read the complete rendered F-frame interface image from the memory via the MIPI interface and transmit it to the image processing chip. The image processing chip also needs to wait for the main control chip to transmit the rendered F-frame interface image via the MIPI interface before it can process the rendered F-frame interface image, resulting in a significant delay in image transmission.

[0101] Based on this, the main control chip in this application generates a second trigger signal by multiplying the frequency of the first trigger signal. This enables the main control chip to transmit the rendered first image block to the image processing chip based on the second trigger signal. Therefore, the main control chip does not need to wait for all image blocks to be rendered before transmitting the rendered interface image, thereby greatly reducing the image transmission latency.

[0102] Step 205c: The main control chip determines the number of image blocks in the first image block based on the first ratio during the first cycle and the second cycle.

[0103] In some embodiments of this application, the first ratio is a multiple of the first frequency relative to the second frequency. In other words, the first ratio is M.

[0104] Thus, the main control chip multiplies the frequency of the first trigger signal to generate a second trigger signal, enabling it to transmit the rendered first image block to the image processing chip based on the second trigger signal. Therefore, the main control chip does not need to wait for all image blocks to finish rendering before transmitting the rendered interface image, thereby significantly reducing image transmission latency.

[0105] In some embodiments of this application, combined with Figure 4 ,like Figure 5 As shown, step 205c above can be achieved through the following steps 205c1 and 205c2:

[0106] Step 205c1: The main control chip obtains the first value corresponding to the number of at least one image block.

[0107] In some embodiments of this application, the main control chip can divide the interface image data based on the number of pixels in the interface image data by the CPU to obtain at least one image block, and then obtain a first value corresponding to the number of at least one image block.

[0108] Step 205c2: The main control chip calculates the second ratio between the first value and the first ratio, and determines the second ratio as the second value corresponding to the number of image blocks in the first image block.

[0109] In some embodiments of this application, the main control chip can calculate the aforementioned second value using the following formula:

[0110] P = L / M (1);

[0111] Where P represents the second value above, and L represents the first value above.

[0112] Thus, the main control chip obtains a first value corresponding to the number of at least one image block; the main control chip calculates a second ratio between the first value and the first ratio, and determines the second ratio as the second value corresponding to the number of image blocks in the first image block, thereby accurately determining the number of image blocks in the first image block.

[0113] In some embodiments of this application, combined with Figure 4 ,like Figure 7 As shown, prior to step 203 above, the image processing method provided in this application embodiment may further include the following step 206:

[0114] Step 206: When the level of the second trigger signal changes from the first logic level to the second logic level, the main control chip sends the rendered first image block to the image processing chip.

[0115] In some embodiments of this application, the first logic level can be a high level and the second logic level can be a low level, or the first logic level can be a low level and the second logic level can be a high level.

[0116] In some embodiments of this application, the level of the second trigger signal can be periodically switched.

[0117] In some embodiments of this application, the main control chip can send the rendered first image block to the image processing chip via the DPU.

[0118] Thus, when the level of the second trigger signal changes from the first logic level to the second logic level, the main control chip sends the rendered first image block to the image processing chip, so that the main control chip does not need to wait for all image blocks to be rendered before it can transmit the rendered interface image, thereby greatly reducing the image transmission latency.

[0119] In some embodiments of this application, after step 202 described above, the image processing method provided in this application may further include step 207; and before step 203 described above, the image processing method provided in this application may further include step 208.

[0120] Step 207: The main control chip stores the rendered first image block in the first storage area of ​​the memory.

[0121] In some embodiments of this application, the main control chip can store the rendered first image block in the first storage area of ​​the memory in the main control chip when the level of the second trigger signal changes from the first logic level to the second logic level.

[0122] In some embodiments of this application, the main control chip can store the rendered first image block in the first storage area via the GPU.

[0123] Step 208: The image processing chip reads the rendered first image block from the first storage area.

[0124] In some embodiments of this application, after the main control chip stores the rendered first image block in the first storage area of ​​the memory, it can transmit the first address of the first storage area to the image processing chip. After the image processing chip receives the first address, it can read the rendered first image block from the first storage area based on the first address.

[0125] In some embodiments of this application, the first address may be the physical address of the first storage area or the virtual address of the first storage area.

[0126] In this way, the main control chip stores the rendered first image block in the first storage area of ​​the memory; the image processing chip reads the rendered first image block from the first storage area, so that the image processing chip can sequentially read and process the N rendered image blocks in the first storage area. This allows the image processing chip to process the image without waiting for the main control chip to finish rendering all the image blocks, which can greatly reduce the image processing time and thus effectively reduce the image display latency.

[0127] In some embodiments of this application, after step 203 described above, the data processing method provided in this application may further include the following steps 209a and 209b:

[0128] Step 209a: The image processing chip stores the processed first image block in the second storage area of ​​the memory.

[0129] In some embodiments of this application, when it is necessary to send the processed first image block back to the main control chip for further processing, the image processing chip can store the processed first image block in the second storage area of ​​the memory.

[0130] Step 209b: If at least one row of processed image blocks is stored in the second storage area, the main control chip reads the at least one row of processed image blocks and transmits the pixels in the at least one row of processed image blocks to the screen for display row by row.

[0131] In some embodiments of this application, after the image processing chip stores the processed first image block in the second storage area of ​​the memory, the image processing chip can transmit the second address of the second storage area to the main control chip. Upon receiving the second address, if the second storage area stores at least one row of processed image blocks, the main control chip can retrieve the at least one row of processed image blocks from the second storage area based on the second address, further process the at least one row of processed image blocks, and then transmit the pixels in the at least one row of processed image blocks row by row to the screen for display.

[0132] In some embodiments of this application, the second storage area may also assemble all the processed image blocks into a frame of processed interface image, and the main control chip may then transmit the pixels in the frame of processed interface image to the screen for display row by row.

[0133] In some embodiments of this application, the main control chip can perform further processing on at least one row of image blocks after processing via the DPU, such as editing. The specific processing can be determined according to actual needs, and this embodiment does not impose specific limitations here.

[0134] Thus, the image processing chip stores the processed first image block in the second storage area of ​​the memory; when the second storage area stores at least one row of processed image blocks, the main control chip reads the processed at least one row of image blocks and transmits the pixels in the processed at least one row of image blocks to the screen for display row by row. The image processing chip can then send the image blocks back to the main control chip for further processing, thereby effectively improving the image quality.

[0135] In some embodiments of this application, after step 207 described above, the image processing method provided in this application may further include step 210a; and after step 203 described above, the image processing method provided in this application may further include steps 211a and 211b:

[0136] Step 210: When the level of the third trigger signal transmitted by the image processing chip changes from the third logic level to the fourth logic level, the image processing chip retrieves the rendered first image block from the first storage area.

[0137] In some embodiments of this application, the third logic level can be a high level and the fourth logic level can be a low level, or the third logic level can be a low level and the fourth logic level can be a high level.

[0138] In some embodiments of this application, the level of the third trigger signal can be periodically switched.

[0139] Step 211a: When the level of the third trigger signal changes from the fourth logic level to the third logic level, the image processing chip transmits the processed first image block to the main control chip.

[0140] In some embodiments of this application, when it is necessary to send the interface image back to the main control chip for further processing, the image processing chip can transmit the processed first image block to the main control chip.

[0141] In some embodiments of this application, the image processing chip can further process the interface image transmitted to the DPU in the main control chip, such as editing. The specific processing can be determined according to actual needs, and this embodiment does not impose specific limitations here.

[0142] Step 211b: If the main control chip stores at least one row of processed image blocks, the main control chip transmits the pixels in the at least one row of processed image blocks to the screen for display row by row.

[0143] In some embodiments of this application, when at least one row of processed image blocks is stored in the PDU's cache, the DPU can further process the at least one row of processed image blocks and transmit the pixels in the at least one row of processed image blocks to the screen for display row by row.

[0144] Thus, when the level of the third trigger signal transmitted by the image processing chip changes from the third logic level to the fourth logic level, the image processing chip retrieves the rendered first image block from the first storage area. When the level of the third trigger signal changes from the fourth logic level to the third logic level, the image processing chip transmits the processed first image block to the main control chip. If the main control chip stores at least one row of processed image blocks, it transmits the pixels in the at least one row of processed image blocks to the screen for display row by row. On the one hand, the main control chip does not need to repeatedly store image blocks, thus effectively reducing the power consumption caused by storing image blocks. On the other hand, it can effectively reduce the number of times image blocks are read and processed, thereby effectively reducing image display latency.

[0145] Figure 8 This is a schematic diagram of the image processing architecture provided in an embodiment of this application. Figure 8 As shown, the image processing architecture provided in this application embodiment may include:

[0146] The image processing chip 81 and the main control chip 82 are included. The main control chip 82 includes a memory 821, a CPU 822 connected to the memory 821, a GPU 823 connected to the CPU 822 and the memory 821, and a first interface 824 connected to the image processing chip 81 and the memory 821.

[0147] CPU 822 is used to control the GPU to obtain interface image data of the first interface from memory 821, and to obtain at least one image block based on the interface image data.

[0148] In some embodiments of this application, each image block in the at least one image block described above has the same size.

[0149] It should be noted that the specific process of CPU 822 controlling GPU memory 821 to obtain interface image data of the first interface and obtaining at least one image block based on the interface image data can be referred to the relevant description in step 201 above. To avoid repetition, this embodiment will not repeat it here.

[0150] GPU823 is used to render the first image block, obtain the rendered first image block, and store the rendered first image block in the first storage area of ​​the memory.

[0151] In some embodiments of this application, the first image block is N image blocks among the at least one image block, where N is a positive integer.

[0152] It should be noted that the specific process of GPU823 rendering the first image block and obtaining the rendered first image block can be found in the relevant description in step 202 above. To avoid repetition, this embodiment will not repeat it here.

[0153] It should be noted that the specific process by which the GPU823 stores the rendered first image block in the first storage area of ​​the memory can be found in the relevant description in step 207 above. To avoid repetition, this embodiment will not repeat it here.

[0154] The image processing chip 81 is used to read the rendered first image block from the first storage area using the first interface 824, and to process the rendered first image block to obtain the processed first image block.

[0155] In some embodiments of this application, the first interface can be a PCIE interface, or it can be an SCI interface or other communication interface. The specific interface can be determined according to actual needs, and this embodiment does not impose any specific limitations here.

[0156] It should be noted that the specific process by which the image processing chip 81 reads the rendered first image block from the first storage area using the first interface 824 can be found in the relevant description in step 208 above. To avoid repetition, this embodiment will not repeat the description here.

[0157] It should be noted that the image processing chip 81 processes the rendered first image block to obtain the processed first image block. The specific process can be found in the relevant description in step 203 above. To avoid repetition, this embodiment will not repeat it here.

[0158] In the image processing architecture provided in this application embodiment, the image processing architecture includes an image processing chip and a main control chip. The main control chip includes a memory, a CPU connected to the memory, a GPU connected to the CPU and the memory, and a first interface connected to the image processing chip and the memory. The CPU is used to control the GPU to obtain interface image data of the first interface from the memory, and to obtain at least one image block based on the interface image data, wherein each image block may have the same size. The GPU is used to render the first image block to obtain the rendered first image block, and to store the rendered first image block in a first storage area of ​​the memory. The first image block is N image blocks out of at least one image block. The image processing chip is used to read the rendered first image block from the first storage area using the first interface, and to process the rendered first image block to obtain a processed first image block. In this way, the GPU stores the rendered first image block in the first storage area of ​​the memory; the image processing chip reads the rendered first image block from the first storage area through the first interface. On the one hand, the image processing chip can read the rendered image block from the first storage area for processing, thus eliminating the need for the DPU to transfer the rendered image block to the image processing chip, and thus eliminating the need to consume the DPU's resources and power to transfer and process the image, thereby effectively saving resources and reducing power consumption. On the other hand, it enables the image processing chip to read and process N rendered image blocks sequentially from the first storage area, so that the image processing chip does not need to wait for the main control chip to finish rendering all image blocks before it can process the image, which can greatly reduce the image processing time and thus effectively reduce the image display latency.

[0159] In some embodiments of this application, combined with Figure 8 ,like Figure 9 As shown, the main control chip 82 also includes a DPU825 connected to the memory 821.

[0160] In some embodiments of this application, the image processing chip 82 is further configured to use a first interface 824 to store the processed first image block in a second storage area of ​​the memory 821.

[0161] It should be noted that the image processing chip 82 uses the first interface 824 to store the processed first image block in the second storage area of ​​the memory 821. The specific process can be found in the relevant description in step 209a above. To avoid repetition, this embodiment will not repeat it here.

[0162] In some embodiments of this application, DPU825 is used to retrieve the processed at least one row of image blocks from the second storage area when the second storage area stores such processed at least one row of image blocks, and to transmit the pixels in the processed at least one row of image blocks to the screen for display row by row.

[0163] In some embodiments of this application, each image block is the same size.

[0164] It should be noted that when the DPU825 has stored at least one row of processed image blocks in the second storage area, the specific process of obtaining the at least one row of processed image blocks from the second storage area and transmitting the pixels in the at least one row of processed image blocks to the screen for display can be found in the relevant description in step 209b above. To avoid repetition, this embodiment will not repeat the description here.

[0165] In some embodiments of this application, the size of each image block may be determined based on the actual application scenario.

[0166] Thus, the image processing chip is also used to store the processed first image block in the second storage area of ​​the memory using the first interface; the DPU is used to retrieve the processed at least one row of image blocks from the second storage area when the processed at least one row of image blocks is stored in the second storage area, and transmit the pixels in the processed at least one row of image blocks to the screen for display row by row, so that the image block can be sent back to the DPU for further processing by the image processing chip, thereby effectively improving the image quality.

[0167] In some embodiments of this application, combined with Figure 8 ,like Figure 10 As shown, the main control chip 82 also includes a DPU 825 connected to the memory 821, a first bus 826 and a matrix switch 827. The DPU 825, CPU 822, GPU 823 and the first interface 824 are connected to the first bus 826 through the matrix switch 827. The CPU 822 is also connected to the image processing chip 81.

[0168] In some embodiments of this application, the CPU 822 is further configured to control the matrix switch 827 to be in a first conducting state when the level of the third trigger signal transmitted by the image processing chip 81 is changed from a third logic level to a fourth logic level.

[0169] In some embodiments of this application, when the matrix switch 827 is in the first conducting state, the image processing chip is connected to the memory 821 through the first bus 827 and the first interface 824.

[0170] In some embodiments of this application, the image processing chip 81 is specifically used to: when the matrix switch 827 is in the first conducting state, use the first bus 826 and the first interface 824 to read the rendered first image block from the first storage area.

[0171] It should be noted that when the matrix switch 827 is in the first conducting state, the image processing chip 81 uses the first bus 826 and the first interface 824 to read the rendered first image block from the first storage area. The specific process can be found in the relevant description in step 210 above. To avoid repetition, this embodiment will not repeat it here.

[0172] In some embodiments of this application, the CPU 822 is also configured to control the matrix switch 826 to be in a second on state when the level of the third trigger signal transmitted by the image processing chip 81 changes from the aforementioned fourth logic level to the upper third logic level.

[0173] In some embodiments of this application, when the matrix switch 827 is in the second conduction state described above, the image processing chip is connected to the DPU 825 via the first bus 826.

[0174] In some embodiments of this application, the image processing chip 821 is further configured to transmit the processed first image block to the DPU 825 using the first bus 826 and the first interface 824 when the matrix switch 827 is in the second conduction state.

[0175] It should be noted that when the matrix switch 827 is in the second conduction state, the image processing chip 821 uses the first bus 826 and the first interface 824 to transmit the processed first image block to the DPU 825. The specific process can be found in the relevant description in step 211a above. To avoid repetition, this embodiment will not repeat it here.

[0176] In some embodiments of this application, the DPU825 is used to transmit the pixels in the at least one row of processed image blocks to the screen for display, in the case of storing at least one row of processed image blocks.

[0177] In some embodiments of this application, the DPU825 is used to transmit the pixels of the at least one row of processed image blocks to the screen for display, line by line, when the DPU825 cache stores at least one row of processed image blocks.

[0178] It should be noted that, for DPU825, when at least one row of processed image blocks is stored in the DPU825 cache, the specific process of transmitting the pixels of the above-mentioned at least one row of processed image blocks to the screen for display can be referred to step 211b above. To avoid repetition, this embodiment will not repeat it here.

[0179] Thus, the CPU is further configured to control the matrix switch to a first conducting state when the level of the third trigger signal transmitted by the image processing chip changes from a third logic level to a fourth logic level; the image processing chip is specifically configured to: when the matrix switch is in the first conducting state, read the rendered first image block from the first storage area using a first bus and a first interface; the CPU is further configured to control the matrix switch to a second conducting state when the level of the third trigger signal transmitted by the image processing chip changes from the fourth logic level to the third logic level; the DPU is configured to, when at least one row of processed image blocks is stored, transmit the pixels in the at least one row of processed image blocks row by row to the screen for display. On the one hand, the main control chip does not need to repeatedly store image blocks in the memory, thereby effectively reducing the power consumption caused by storing image blocks. On the other hand, it can effectively reduce the number of times image blocks are read and processed, thereby effectively reducing image display latency.

[0180] The following uses a mobile phone as an example to illustrate the interactive process involved in the image processing method provided in this application embodiment through specific implementation methods.

[0181] Implementation method 1: The main control chip of the mobile phone is based on the MIPI interface to realize the transmission of image blocks.

[0182] like Figure 11 As shown, in this embodiment, the CPU 1111 in the main control chip 111 of the mobile phone can obtain the image data of the displayed interface image from DDR 1112 (i.e., the aforementioned memory), and based on the image data of the interface image, divide the interface image to be displayed into several tiles and transmit them to the GPU 1113, for example, L tiles. The GPU 1113 then renders the tiles and stores the rendered tiles in the framebuffer of DDR 1112. Under an interrupt signal such as TE2 or other type of trigger signal (i.e., the aforementioned second trigger signal), the DPU 1114 of the mobile phone reads the rendered tiles from the framebuffer. It should be noted that, of course, an interrupt or notification can also be generated within the rendering of a single tile.

[0183] For example, the main control chip 111 packages the data into a data packet based on the tile. When the K11125 switch is turned on to MIPI RX01121, the data packet is transmitted to MIPI1 RX01121 of the image processing chip 112 via DPU 1114 using MIPI21115, and stored in the SRAM1122 of the image processing chip 112. Since the GPU 1113 does not render tiles completely sequentially, for example, tile 3 may be completed earlier than tile 2. Therefore, tile 3 is transmitted first. It can be packaged into the data packet according to its own information, such as coordinate information, or by marking it with a number. Then, the tiles are arranged and stored in the SRAM1122 of the image processing chip 112 according to the above coordinate information or number. Furthermore, according to the algorithm requirements, the image processing chip 112 can wait for a frame of interface image to be completely transmitted before processing, or it can process one tile or several tiles. This greatly reduces latency.

[0184] For example, after the image processing chip 112 finishes processing a frame of interface image, the K31123 switch can be turned on, so that the image processing chip 112 can transmit a frame of interface image to the screen display via MIPI TX01124.

[0185] It should be noted that when the mobile phone does not require the image processing chip 112 to process the frame interface image, the mobile phone can control the K11125 switch to be turned on to the bypass path 1127 and control the K31124 switch to be turned off, so that the DPU 1104 can read the rendered image from the framebuffer, and when the trigger signal TE1 (i.e. the first trigger signal mentioned above) is on the rising edge, it processes a frame interface image and uses MIPI 11116 to transmit the rendered image to the screen 113 for display through the bypass path, or processes at least one row of image blocks and uses MIPI 11116 to transmit the rendered image to the screen 113 for display through the bypass path.

[0186] It should be noted that when the mobile phone needs to send the interface image processed by the image processing chip 112 back to the main control chip 111 for processing, the mobile phone can send the processed interface image back to the MIPI CSI 1117 of the main control chip 111 through the MIPI TX 11126 of the image processing chip 112.

[0187] Implementation method 2: The main control chip of the mobile phone transmits tiles based on the PCIe interface.

[0188] like Figure 12As shown, in this embodiment, the main control chip 111 and the image processing chip 112 are connected via a PCIe interface 1118 (i.e., the first interface mentioned above). Tiles rendered by the GPU 1113 are stored in image storage area 1 (i.e., the first storage area mentioned above) in the framebuffer. Stored tiles, or several tiles packaged into a data packet, are transmitted to the image processing chip 112 via the PCIe interface 1118. After processing, the image processing chip 112 can transmit the data back to the main control chip 111 in tile format. Image storage area 2 (i.e., the second storage area mentioned above) stored in the framebuffer forms an image frame. Then, the DPU 1114 reads the image frame from image storage area 2 and sends it to the screen 113 for display. Alternatively, if image storage area 2 in the framebuffer stores at least one row of tiles transmitted back by the image processing chip 112, the DPU 1114 reads at least one row of tiles and transmits the pixels of that row of tiles to the screen for display.

[0189] The image processing method provided in this embodiment eliminates the need for the DPU to pass through the image processing chip when the tile is sent via PCIe. This avoids interference from the DPU to the tile, such as eliminating the need to consume DPU resources and power, thereby reducing latency.

[0190] Implementation Method 3: The mobile phone's main control chip transmits tiles via the bus.

[0191] In implementation method two, because the PCIe interface is connected to the DDR via a bus such as the AXI bus, the data transmitted from the processing chip to the SOC needs to be stored again in the image storage area 2 of the framebuffer before the DPU reads it, resulting in an extra storage step, increased power consumption, and increased latency. In other implementation methods, such as... Figure 13 As shown, the PCIe interface 1118 is directly connected to the DPU 1114 via the bus 1119, and the tile is directly transmitted to the DPU 1114.

[0192] Furthermore, since the delay in data transmission from the image processing chip to the main control chip is not the main component of the latency, the image processing chip can transmit the processed image to the DPU in a row-by-row or multi-row manner, for example, by storing it in the DPU's line buffer, and then the DPU can continue subsequent processing. Alternatively, the image processing chip can transmit multiple tiles of the corresponding row to the DPU together, and then the DPU can continue subsequent processing.

[0193] For example, the DPU, GPU, etc., inside the main control chip are all connected to the bus, while the traditional PCIe interface can only connect to DDR. Combined with... Figure 13 ,like Figure 14As shown, in this embodiment, a matrix switch 11191 is added. The PCIe interface 1118 can be directly connected to the DPU 1114 through the matrix switch 11191. At the same time, when the PCIe interface 1118 transmits data, it notifies the CPU 1111 through an interrupt signal (i.e., the third trigger signal mentioned above). The CPU 1111 controls the DPU 1114, thereby realizing the overall work scheduling. For details, please refer to the relevant descriptions in steps 210 to 211b above. To avoid repetition, they will not be repeated here.

[0194] For example, combined Figure 14 ,like Figure 15 As shown, matrix switch 11191 is connected to the AXI master interface 11131 of GPU 1113, the AXI master interface 11141 of DPU 1114, the AXI master interface 11121 of DDR 1112, and the AXI master interface 11111 of CPU 1111. The router 11911 inside matrix switch 1191 can be in different conduction states under the control of CPU 1111, thereby realizing different connection states between GPU 1113, DPU 1114, DDR 1112, and PCIe interface 1118.

[0195] The image processing method provided in this embodiment eliminates the need for the main control chip to repeatedly store image blocks in memory, thereby effectively reducing the power consumption associated with storing image blocks. Furthermore, it effectively reduces the number of times image blocks are read and processed, thus significantly reducing image display latency.

[0196] Optionally, such as Figure 16 As shown, this application embodiment also provides an electronic device 1600, including a processor 1601 and a memory 1602. The memory 1602 stores a program or instructions that can run on the processor 1601. When the program or instructions are executed by the processor 1601, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0197] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0198] Figure 17 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0199] The electronic device 1700 includes, but is not limited to, components such as: radio frequency unit 1701, network module 1702, audio output unit 1703, input unit 1704, sensor 1705, display unit 1706, user input unit 1707, interface unit 1708, memory 1709, and processor 1710.

[0200] Those skilled in the art will understand that the electronic device 1700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 17 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0201] The processor 1710 is used to obtain at least one image block based on the interface image data of the first interface through the main control chip, and each image block has the same size.

[0202] The main control chip renders the first image block to obtain the rendered first image block. The first image block is N image blocks among the at least one image block, where N is a positive integer.

[0203] The image processing chip processes the rendered first image block to obtain the processed first image block. The rendered first image block is transmitted to the image processing chip via the main control chip, or the rendered first image block is obtained from the first storage area of ​​the memory by the image processing chip. The first storage area is used to store the rendered first image block.

[0204] In some embodiments of this application, after the rendered first image block is processed by the image processing chip to obtain the processed first image block, the processor 1710 is further configured to transmit the pixels in at least one row of the processed image block to the screen for display by the image processing chip.

[0205] In some embodiments of this application, before the main control chip renders the first image block to obtain the rendered first image block, the processor 1710 is further configured to receive a first trigger signal generated by the screen through the main control chip, wherein the period of the first trigger signal is the duration required for the screen to display one frame of interface image.

[0206] The main control chip performs frequency multiplication on the first trigger signal to generate a second trigger signal, wherein the second period of the second trigger signal is less than the first period of the first trigger signal.

[0207] The main control chip determines the number of image blocks in the first image block based on a first ratio between the first cycle and the second cycle.

[0208] In some embodiments of this application, the processor 1710 is specifically used for:

[0209] The first value corresponding to the number of the at least one image block is obtained through the main control chip;

[0210] The main control chip calculates a second ratio between the first value and the first ratio, and determines the second ratio as the second value corresponding to the number of image blocks in the first image block.

[0211] In some embodiments of this application, before the image processing chip processes the rendered first image block to obtain the processed first image block, the processor 1710 is further configured to send the rendered first image block to the image processing chip via the main control chip when the level of the second trigger signal changes from the first logic level to the second logic level.

[0212] In some embodiments of this application, after the main control chip renders the first image block to obtain the rendered first image block, the processor 1710 is further configured to store the rendered first image block in the first storage area of ​​the memory through the main control chip.

[0213] Before the image processing chip processes the rendered first image block to obtain the processed first image block, the processor 1710 is also used to read the rendered first image block from the first storage area through the image processing chip.

[0214] In some embodiments of this application, after the image processing chip processes the rendered first image block read from the first storage area to obtain the processed first image block, the processor 1710 is further configured to store the processed first image block in a second storage area of ​​the memory through the image processing chip; when the second storage area stores at least one row of processed image blocks, the main control chip reads the at least one row of processed image blocks and transmits the pixels in the at least one row of processed image blocks to the screen for display row by row.

[0215] In some embodiments of this application, after the rendered first image block is stored in the first storage area of ​​the memory by the main control chip, the processor 1710 is further configured to obtain the rendered first image block from the first storage area by the image processing chip when the level of the third trigger signal transmitted by the image processing chip is changed from the third logic level to the fourth logic level.

[0216] After the first image block is processed by the image processing chip to obtain the processed first image block, the processor 1710 is further configured to transmit the processed first image block to the main control chip through the image processing chip when the level of the third trigger signal is changed from the fourth logic level to the third logic level.

[0217] When the main control chip stores at least one row of processed image blocks, the pixels in the at least one row of processed image blocks are transmitted to the screen for display row by row through the main control chip.

[0218] In the electronic device provided in the application embodiment, the main control chip obtains at least one image block based on the interface image data of the first interface, and each image block has the same size. The main control chip renders the first image block to obtain a rendered first image block, where the first image block consists of N image blocks from the at least one image block, and N is a positive integer. The image processing chip processes the rendered first image block to obtain a processed first image block. The rendered first image block is transmitted from the main control chip to the image processing chip, or the rendered first image block is obtained by the image processing chip from a first storage area of ​​the memory, where the first storage area is used to store the rendered first image block. In this way, the electronic device does not need to wait for the main control chip to finish rendering all the image blocks of a frame of interface image. After the main control chip finishes rendering N image blocks, it can perform image processing on the rendered N image blocks. This effectively reduces the processing time of the discrete graphics chip for the interface image, thereby reducing the display latency of the interface image.

[0219] It should be understood that, in this embodiment, the input unit 1704 may include a graphics processing unit (GPU) 17041 and a microphone 17042. The GPU 17041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1706 may include a display panel 17061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1707 includes at least one of a touch panel 17071 and other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0220] The memory 1709 can be used to store software programs and various data. The memory 1709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1709 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0221] Processor 1710 may include one or more processing units; optionally, processor 1710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1710.

[0222] This application also provides an electronic device, characterized in that it includes the image processing architecture as described above. The electronic device can implement the various processes of the above image processing architecture embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0223] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0224] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0225] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0226] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0227] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0228] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0229] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0230] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method applied to an image processing device, the image processing device comprising an image processing chip and a main control chip, characterized in that, The method includes: The main control chip obtains at least one image block based on the interface image data of the first interface; The main control chip renders the first image block to obtain the rendered first image block. The first image block is N image blocks among the at least one image block, where N is a positive integer. The image processing chip processes the rendered first image block to obtain the processed first image block; Wherein, the rendered first image block is transmitted from the main control chip to the image processing chip, or the rendered first image block is obtained by the image processing chip from the first storage area of ​​the memory, the first storage area being used to store the rendered first image block.

2. The method according to claim 1, characterized in that, The image processing chip processes the rendered first image block to obtain the processed first image block. The method further includes: The image processing chip transmits the pixels of at least one row of processed image blocks to the screen for display, row by row.

3. The method according to claim 1, characterized in that, Before the main control chip renders the first image block and obtains the rendered first image block, the method further includes: The main control chip receives a first trigger signal generated by the screen, and the period of the first trigger signal is the duration required for the screen to display one frame of interface image. The main control chip performs frequency multiplication on the first trigger signal to generate a second trigger signal, wherein the second period of the second trigger signal is less than the first period of the first trigger signal. The main control chip determines the number of image blocks in the first image block based on a first ratio between the first cycle and the second cycle.

4. The method according to claim 3, characterized in that, The main control chip determines the number of image blocks in the first image block based on a first ratio of the first cycle and the second cycle, including: The main control chip obtains a first value corresponding to the number of the at least one image block; The main control chip calculates a second ratio between the first value and the first ratio, and determines the second ratio as the second value corresponding to the number of image blocks in the first image block.

5. The method according to claim 3, characterized in that, Before processing the rendered first image block to obtain the processed first image block, the method further includes: When the level of the second trigger signal changes from the first logic level to the second logic level, the main control chip sends the rendered first image block to the image processing chip.

6. The method according to claim 1, characterized in that, After the main control chip renders the first image block and obtains the rendered first image block, the method further includes: The main control chip stores the rendered first image block in the first storage area of ​​the memory. Before the image processing chip processes the rendered first image block to obtain the processed first image block, the method further includes: The image processing chip reads the rendered first image block from the first storage area.

7. The method according to claim 6, characterized in that, After the image processing chip processes the rendered first image block read from the first storage area to obtain the processed first image block, the method further includes: The image processing chip stores the processed first image block in the second storage area of ​​the memory. When the second storage area stores at least one row of processed image blocks, the main control chip reads the at least one row of processed image blocks and transmits the pixels in the at least one row of processed image blocks to the screen for display row by row.

8. The method according to claim 6, characterized in that, After the main control chip stores the rendered first image block in the first storage area of ​​the memory, the method further includes: When the level of the third trigger signal transmitted by the image processing chip changes from the third logic level to the fourth logic level, the image processing chip obtains the rendered first image block from the first storage area. After the image processing chip processes the rendered first image block to obtain the processed first image block, the method further includes: When the level of the third trigger signal changes from the fourth logic level to the third logic level, the image processing chip transmits the processed first image block to the main control chip; When the main control chip stores at least one row of processed image blocks, the main control chip transmits the pixels in the at least one row of processed image blocks to the screen for display, row by row.

9. An image processing architecture, characterized in that, It includes an image processing chip and a main control chip. The main control chip includes a memory, a central processing unit (CPU) connected to the memory, an image processing unit (GPU) connected to the CPU and the memory, and a first interface connected to the image processing chip and the memory. The CPU is used to control the GPU to acquire the interface image data of the first interface from the memory, and to obtain at least one image block based on the interface image data. The GPU is used to render a first image block, obtain a rendered first image block, and store the rendered first image block in a first storage area of ​​the memory. The first image block is N image blocks from the at least one image block, where N is a positive integer; The image processing chip is used to read the rendered first image block from the first storage area using the first interface, and to process the rendered first image block to obtain the processed first image block.

10. The architecture according to claim 9, characterized in that, The main control chip also includes a display processing unit (DPU) connected to the memory; The image processing chip is also used to store the processed first image block in the second storage area of ​​the memory using the first interface; The DPU is used to retrieve the processed at least one row of image blocks from the second storage area when the second storage area stores the processed at least one row of image blocks, and to transmit the pixels in the processed at least one row of image blocks to the screen for display row by row.

11. The architecture according to claim 9, characterized in that, The main control chip also includes a display processing unit (DPU), a first bus, and a matrix switch connected to the memory. The DPU, the CPU, the GPU, and the first interface are connected to the first bus via the matrix switch. The CPU is also connected to the image processing chip. The CPU is further configured to control the matrix switch to a first conducting state when the level of the third trigger signal transmitted by the image processing chip changes from a third logic level to a fourth logic level. The image processing chip is specifically used to: when the matrix switch is in the first conducting state, use the first bus and the first interface to read the rendered first image block from the first storage area; When the matrix switch is in the first conducting state, the image processing chip is connected to the memory through the first bus and the first interface; The CPU is further configured to control the matrix switch to a second conduction state when the level of the third trigger signal transmitted by the image processing chip changes from the fourth logic level to the third logic level. When the matrix switch is in the second conducting state, the image processing chip is connected to the DPU via the first bus; The image processing chip is further configured to transmit the processed first image block to the DPU using the first bus and the first interface when the matrix switch is in the second conduction state. The DPU is used to transmit the pixels in the at least one row of processed image blocks to the screen for display, line by line, when at least one row of processed image blocks is stored.

12. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as described in any one of claims 1 to 8.

13. An electronic device, characterized in that, The image processing architecture includes any one of claims 9 to 11.

14. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image processing method as described in any one of claims 1 to 8.

Images