Chip system, image processing method and electronic device

Abstract

Provided in the embodiments of the present application are a chip system, an image processing method and an electronic device, belonging to the technical field of chip image processing. The chip system comprises an image processor, a neural network processor and a scheduling circuit, wherein the image processor is used for outputting, on the basis of an image sub-task, a first rendered image; the scheduling circuit is used for instructing, on the basis of a scheduling sub-task, the neural network processor to acquire the first rendered image; the neural network processor is used for, on the basis of an operation sub-task, performing neural network algorithm processing on the first rendered image and outputting an operation result; the scheduling circuit is further used for instructing, on the basis of the scheduling sub-task, the image processor to acquire the operation result; and the image processor is further used for performing, on the basis of the image sub-task, image processing on the operation result, so as to obtain a second rendered image. The embodiments of the present application perform image processing on the basis of multi-IP linkage, so as to improve the imaging quality, thus reducing the power consumption of chip systems and the processing complexity, and being applicable to different chip architectures.

Description

Chip system, image processing method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 13, 2023, with application number 202311188058.9 and application name “A chip system, image processing method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of chip image processing technology, and in particular to a chip system, an image processing method and an electronic device. Background Art

[0003] The chip system includes an image processor. The image task is performed by the image processor to render and generate the image required for imaging display. At present, in some high-power chip architectures (such as desktops based on the X86 architecture), in order to pursue better imaging quality, complex image rendering algorithms are often used to process the image in the image processor. However, complex image rendering algorithm processing will greatly increase the power consumption of the image processor. One way to improve it is to modify the design inside the image processor to add a neural network processing circuit inside the image processor. By performing image rendering and AI algorithm processing in a closed loop inside the image processor, a rendered image with higher imaging quality is obtained. Although this method can reduce power consumption, it requires changes to the IP core of the image processor and increases the area of ​​the image processor. At the same time, the neural network processing circuit in the image processor cannot be shared with other multimedia services.

[0004] On the other hand, in medium and low power chip architectures (such as mobile terminals based on ARM architecture), there are extremely high requirements for the device area and power consumption of the chip, which often requires reducing power consumption by reducing the complexity and processing volume of the image rendering algorithm. Therefore, in medium and low power chip architectures, it is difficult to improve the image quality by the above-mentioned method of increasing the complexity of image rendering algorithm processing or by adding neural network processing circuits. With the technological updates and iterations of electronic products, users' demand for the image quality of displayed images is also increasing. How to obtain a rendered image with higher image quality in chip architectures with different power consumption and different hardware designs is a major problem.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a chip system, an image processing method, and an electronic device, which improve the imaging quality of rendered images while reducing the power consumption of the chip system and are applicable to different chip architectures.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a chip system is provided, comprising an image processor, a neural network processor, and a scheduling circuit. The image processor is configured to output a first rendered image according to an image subtask. The scheduling circuit is configured to instruct the neural network processor to obtain the first rendered image according to the scheduling subtask. The neural network processor is configured to perform neural network algorithm processing on the first rendered image according to the computation subtask and output the computation result. The scheduling circuit is further configured to instruct the image processor to obtain the computation result according to the scheduling subtask. The image processor is further configured to perform image processing on the computation result according to the image subtask to obtain a second rendered image.

[0009] In an embodiment of the present application, a scheduling circuit instructs the image processor and the neural network processor to perform multi-IP coordinated processing to generate a rendered image with optimized imaging parameters. The specific operation of the multi-IP coordinated processing is as follows: the image processor executes an image subtask to render the image to obtain a first rendered image. The neural network processor performs a neural network algorithm on the first rendered image to obtain a calculation result, thereby completing the parameter optimization calculation operation. The image processor then generates a second rendered image based on the calculation result. One or more imaging parameters of the second rendered image are superior to those of the first rendered image. In this processing method, first, the neural network processor uses a neural network processing algorithm to assist in improving the imaging quality of the rendered image. Compared to using more complex image rendering algorithms in the image processor to improve imaging quality, this embodiment of the present application can significantly reduce the complexity of image processing and improve imaging quality. Second, the neural network processor and the image processor are independent devices, and the data path and control signal interaction between them is quite complex. However, the software control logic of the two is relatively simple. By providing a hardware scheduling circuit, this embodiment of the present application can achieve multi-IP coordinated image processing between the neural network processor and the image processor through a simple control method. Finally, this solution does not require hardware redesign of each chip in the chip system, and can be applied to chip architectures with different power consumption and different designs.

[0010] In one possible embodiment, the image processor is further configured to: when outputting the first rendered image, output a first trigger signal to the scheduling circuit according to the image subtask. The scheduling circuit is specifically configured to: in response to the first trigger signal, output a second trigger signal to the neural network processor according to the scheduling subtask; the second trigger signal is used to instruct the neural network processor to obtain the first rendered image. The image processor is further configured to: when outputting the calculation result, output a third trigger signal to the scheduling circuit according to the image subtask. The scheduling circuit is specifically configured to: in response to the third trigger signal, output a fourth trigger signal to the image processor according to the scheduling subtask; the fourth trigger signal is used to instruct the image processor to obtain the calculation result. In an embodiment of the present application, the scheduling of subtasks with different execution orders in multi-IP linkage processing can be completed by a few trigger signals. The scheduling method is simple and fast.

[0011] In one possible embodiment, the chip system further includes a storage circuit. Specifically, the image processor is configured to write a first rendered image into the storage circuit; and obtain a calculation result from the storage circuit in response to a fourth trigger signal. The neural network processor is configured to respond to a second trigger signal and, based on a calculation subtask, read the first rendered image from the first buffer; and write the calculation result into the storage circuit. In an embodiment of the present application, the first rendered image and calculation result exchanged between the neural network processor and the image processor are stored and transferred through a storage circuit, so that multi-IP linkage processing can be carried out in a more orderly manner.

[0012] In one possible implementation, the image processor is also used to: generate an initial rendered image according to the image subtask. Multiple first rendered images are obtained based on the initial rendered image; the multiple first rendered images correspond to different imaging parts of the initial rendered image. In an embodiment of the present application, when the image processor and the neural network processor perform image processing in conjunction with each other, there may be a situation where the processing delay of one party is large, causing the other party to wait for feedback. At this time, a complete imaging frame can be split into multiple first rendered images. By performing multi-IP linkage processing operations on multiple first rendered images in a pipeline manner by the image processor and the neural network processor, the processing performance of the image processor and the neural network processor can be effectively utilized, and the processing delay of each complete frame can be reduced.

[0013] In one possible implementation, the calculation result is an image processing coefficient. The image processor is further configured to: obtain at least one pre-processed image based on the first rendered image. Process the first rendered image and the at least one pre-processed image according to the image processing coefficient to obtain a second rendered image. In this embodiment of the present application, calculations of the relevant image processing coefficients are performed using a neural network processor. The neural network processor has a relatively small amount of data computation, requires relatively low computing power, and can quickly output the image processing coefficients as the calculation results.

[0014] In one possible implementation, the computation result is a processed image. The image processor is further configured to obtain at least one pre-processed image based on the first rendered image. The neural network processor is specifically configured to perform neural network algorithm processing on the first rendered image based on the at least one pre-processed image to obtain a processed image. In this embodiment of the present application, the image processor may perform image processing to generate one or more pre-processed images. The pre-processed images are used to assist in optimizing image parameters. The neural network processor inputs the first rendered image and at least one pre-processed image. The neural network processor performs neural network algorithm processing on the at least one pre-processed image and the first rendered image to obtain a processed image. The processed image is an image after imaging parameter optimization. The neural network processor outputs the processed image as the computation result. In this embodiment of the present application, the neural network processor may perform neural network computations to directly obtain a processed image after imaging parameter optimization, and then output the processed image as the computation result to the image processor. The image processor performs post-processing on the processed image based on the relevant rendered image to obtain a second rendered image. In this implementation, the computing power requirements of the neural network processor are slightly higher. Furthermore, the processing time of the neural network processor is also longer. However, the processing throughput of the image processor is significantly reduced.

[0015] In a possible embodiment, the chip system also includes a processor. The processor is used to: output image subtasks, operation subtasks and scheduling subtasks. At least one of the image subtasks, operation subtasks or scheduling subtasks is provided with a heterogeneous synchronization signal. The heterogeneous synchronization signal is used to indicate the execution dependency relationship between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate the execution dependency relationship between the operation subtask and the scheduling subtask. In an embodiment of the present application, when encapsulating each subtask, the corresponding heterogeneous synchronization signal is inserted according to the processing order of the subtasks, and the execution dependency relationship between different subtasks is formed through the heterogeneous synchronization signal. Under this embodiment, there is no need to make too many software-level changes to the image processor, neural network processor, etc., so that the image processor, neural network processor, etc. can directly schedule the execution of the corresponding subtasks according to the execution dependency relationship.

[0016] In one possible implementation, the processor is further configured to generate an image rendering task group in response to a rendering command input by the application; the image rendering task group includes an image subtask, a computation subtask, and a scheduling subtask. In an embodiment of the present application, the application manufacturer has coordinated the design of the rendering command. At this point, the issued rendering command has adaptively configured parameters for multiple related subtasks that the IP hardware needs to perform on its own. Based on the rendering command, a corresponding rendering image task group can be directly generated, which already includes an image subtask, a computation subtask, and a scheduling subtask.

[0017] In one possible implementation, the processor is further used to: generate an image rendering task group in response to a rendering command input by the application; the image rendering task group includes image subtasks. Operation subtasks and scheduling subtasks are set in the image rendering task group. In an embodiment of the present application, the application manufacturer did not make any changes to the rendering command. At this time, the rendering command does not include parameters related to multiple subtasks for image processing by multi-IP linkage. Because the rendering command has not been changed, the image rendering task group generated by the processor according to the rendering command only includes image subtasks related to image rendering, and does not include subtasks related to the neural network processor and the scheduling circuit. It is necessary to set the operation subtasks and scheduling subtasks through the processor.

[0018] In some examples, when setting a computation subtask in an image rendering task group, the processor is specifically configured to perform at least one of the following: modifying at least one image subtask into a computation subtask, or adding a new computation subtask to the image rendering task group. In embodiments of the present application, some image subtasks originally executed by the image processor may be modified into computation subtasks, or some computation subtasks may be added.

[0019] In some examples, the processor is further configured to add a new image subtask to the image rendering task group. In embodiments of the present application, in some application scenarios, some new image subtasks need to be added to achieve multi-IP linkage processing.

[0020] In some possible implementations, different imaging parameters of the rendered image may be optimized based on different neural network algorithm processing.

[0021] Exemplarily, the second rendered image satisfies at least one of the following conditions: a resolution of the second rendered image is greater than a resolution of the first rendered image, or a frame rate of the second rendered image is greater than a frame rate of the first rendered image.

[0022] In a second aspect, an embodiment of the present application further provides an image processing method based on a chip system. The chip system includes an image processor, a neural network processor, and a scheduling circuit. The method includes: the image processor outputs a first rendered image according to an image subtask. The scheduling circuit instructs the neural network processor to obtain the first rendered image according to the scheduling subtask. The neural network processor performs a neural network algorithm on the first rendered image according to the operation subtask and outputs the operation result. The scheduling circuit also instructs the image processor to obtain the operation result according to the scheduling subtask. The image processor also performs image processing on the operation result according to the image subtask to obtain a second rendered image.

[0023] In one possible embodiment, the method further includes: when outputting the first rendered image, the image processor outputs a first trigger signal to the scheduling circuit based on the image subtask. In response to the first trigger signal, the scheduling circuit outputs a second trigger signal to the neural network processor based on the scheduling subtask; the second trigger signal is used to instruct the neural network processor to acquire the first rendered image. When outputting the calculation result, the image processor outputs a third trigger signal to the scheduling circuit based on the image subtask. In response to the third trigger signal, the scheduling circuit outputs a fourth trigger signal to the image processor based on the scheduling subtask; the fourth trigger signal is used to instruct the image processor to acquire the calculation result.

[0024] In a possible implementation, the method further includes: an image processor generating an initial rendered image according to the image subtask; obtaining a plurality of first rendered images according to the initial rendered image; the plurality of first rendered images corresponding to different imaging parts of the initial rendered image.

[0025] In one possible implementation, the calculation result is an image processing coefficient. The method further includes: an image processor obtaining at least one pre-processed image based on the first rendered image. Outputting the second rendered image based on the calculation result includes: the image processor processing the first rendered image and the at least one pre-processed image according to the image processing coefficient to obtain the second rendered image.

[0026] In one possible implementation, the operation result is a processed image. The method further includes: an image processor obtaining at least one pre-processed image based on the first rendered image. The aforementioned performing neural network algorithm processing on the first rendered image to obtain the operation result includes: a neural network processor performing neural network algorithm processing on the first rendered image based on the at least one pre-processed image to obtain the processed image.

[0027] In one possible implementation, the method further includes: outputting an image subtask, a computation subtask, and a scheduling subtask; and at least one of the image subtask, computation subtask, or scheduling subtask is provided with a heterogeneous synchronization signal. The heterogeneous synchronization signal is used to indicate an execution dependency between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate an execution dependency between the computation subtask and the scheduling subtask.

[0028] In a possible implementation, the method further includes: generating an image rendering task group in response to a rendering command input by the application; the image rendering task group includes an image subtask, a calculation subtask, and a scheduling subtask.

[0029] In a possible implementation, the method further includes: generating an image rendering task group in response to a rendering command input by the application; the image rendering task group includes an image subtask; and setting a computation subtask and a scheduling subtask in the image rendering task group.

[0030] In a possible implementation, the setting of the operation subtask in the image rendering task group includes performing at least one of the following operations: modifying at least one image subtask into an operation subtask, or adding a new operation subtask to the image rendering task group.

[0031] In a possible implementation, the method further includes: inserting a new image subtask into the image rendering task group.

[0032] In a possible implementation, the second rendered image satisfies at least one of the following conditions: a resolution of the second rendered image is greater than a resolution of the first rendered image, or a frame rate of the second rendered image is greater than a frame rate of the first rendered image.

[0033] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a circuit board and a chip system as described in the first aspect above; the chip system is arranged on the circuit board.

[0034] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium comprising instructions; when the instructions are executed on an electronic device as described in the third aspect above, the electronic device executes the image processing method as described in the second aspect above.

[0035] Regarding the technical principles and beneficial effects of the second, third and fourth aspects mentioned above, please refer to the relevant description of the first aspect mentioned above, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0037] FIG2 is a schematic structural diagram of a first chip system provided in an embodiment of the present application;

[0038] FIG3 is a schematic structural diagram of another first chip system provided in an embodiment of the present application;

[0039] FIG4 is a first structural diagram of a second chip system provided in an embodiment of the present application;

[0040] FIG5 is a second structural diagram of another second chip system provided in an embodiment of the present application;

[0041] FIG6 is a third structural diagram of another second chip system provided in an embodiment of the present application;

[0042] FIG7 is a fourth structural diagram of another second chip system provided in an embodiment of the present application;

[0043] FIG8 is a flowchart diagram 1 of an image processing method provided in an embodiment of the present application;

[0044] FIG9 is a second flow chart of another image processing method provided in an embodiment of the present application;

[0045] FIG10 is a schematic diagram of the time delay of an image processing method without splitting imaging frames provided in an embodiment of the present application;

[0046] FIG11 is a schematic diagram of the time delay of an image processing method for splitting imaging frames provided in an embodiment of the present application;

[0047] FIG12 is a schematic diagram of an inter-chip processing flow of a neural network algorithm processing provided by an embodiment of the present application;

[0048] FIG13 is a third flow chart of another image processing method provided in an embodiment of the present application;

[0049] FIG14 is a schematic diagram of a software architecture of a processor provided in an embodiment of the present application;

[0050] FIG15 is a fourth flow chart of another image processing method provided in an embodiment of the present application;

[0051] FIG16 is a fifth flow chart of another image processing method provided in an embodiment of the present application;

[0052] FIG17 is a schematic diagram of software scheduling of a second image processor provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] It should be noted that the terms "first", "second", etc. involved in the embodiments of the present application are only used to distinguish features of the same type and cannot be understood as indicating relative importance, quantity, order, etc.

[0054] The terms "exemplary" or "for example" in the embodiments of this application are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0055] The terms "coupling" and "connection" involved in the embodiments of this application should be understood in a broad sense. For example, they may refer to a physical direct connection, or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.

[0056] First, some basic concepts involved in this application are explained:

[0057] Electronic design automation (EDA) refers to a design method that uses computer-aided design (CAD) software to complete the functional design, synthesis, verification, physical design (including layout, routing, layout, design rule checking, etc.) of integrated circuit chips.

[0058] Intellectual property (IP) cores play a crucial role in EDA. They can be understood as pre-designed circuit functional modules within integrated circuits (ICs). With the advancement of complex semiconductor and communication technologies, research and development in integrated circuits has also expanded. The rapid increase in chip integration has enabled the integration of more IP cores on a single chip, forming a system-on-chip (SoC). SoCs have different chip architectures for different applications. For high-power applications, large-area, high-power SoCs can be designed, such as SoCs based on the X86 architecture. Because high-power SoCs offer high processing performance, they also require a larger chip area and consume more power. They are typically used in computers and high-performance data processing equipment. For low- to medium-power applications, low- to medium-power SoCs can be designed, such as SoCs based on the Advanced RISC Machine (ARM) architecture. Because these chip architectures offer a smaller chip area and lower power consumption, they are often used in mobile and portable data processing devices, such as mobile phones and tablets. However, in some application scenarios, some computer products also use ARM architecture chip systems. Taking a system on chip (SoC) as an example, an SoC may integrate homogeneous or heterogeneous IP cores, including a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), a microcontroller unit (MCU), memory, an analog-to-digital converter (ADC), a digital-to-analog converter (DAC), a universal asynchronous receiver / transmitter (UART), and software and hardware modules that implement specialized custom functions. Therefore, a single chip system may include multiple different IP cores. Data communication and exchange between these different IP cores has become a key technology in SoC chip design.

[0059] Image rendering is the process of converting three-dimensional radiosity processing into a two-dimensional rendered image. Image rendering is typically achieved using a graphics processor within a chip system. During image rendering, scenes and entities are represented in three dimensions, which is closer to reality and easier to manipulate and transform. Display devices for rendered images are mostly two-dimensional rasterized displays and dot-matrix printers. Image rendering can be considered rasterization. A rasterized display can be thought of as a pixel matrix. Any image displayed on a rasterized display is actually a collection of pixels in one or more colors and grayscales. Rendering is one of the most important techniques in computer graphics. Currently, image rendering has important applications in various fields, including gaming and entertainment, film and television animation, architectural design, education and healthcare, digital twin modeling, and artificial intelligence interaction. With the advancement of technology, people have higher requirements for the quality of rendered images. For example, in gaming, rendered images are required to include features such as ray tracing (RT), deep learning super sampling (DLSS), artificial intelligence (AI), and Reflex low latency. Compared to rasterization, ray tracing makes rendered images more realistic and enables a wider range of lighting effects. To achieve even better image quality, engineers continuously optimize various rendering parameters (such as resolution, frame rate, and special effects) using software and hardware. For example, resolution can be categorized from low to high into low resolution (LR), high resolution (HR), and super resolution (SR).

[0060] An embodiment of the present application provides an electronic device, as shown in FIG1 . The electronic device 1000 includes a circuit board (not shown) and a chip system 100. The chip system 100 is disposed on the circuit board. As shown in FIG1 , the chip system 100 includes an image processor 10. The image processor 10 is configured to generate a rendered image for display imaging based on an image task.

[0061] In some possible implementations, the chip system 100 is a first chip system based on a high-power consumption chip architecture.

[0062] In some examples, to improve the quality of rendered images produced by the first chip system, as shown in FIG2 , the first chip system 100A includes a first image processor 10A. A more complex image rendering algorithm is employed in the first image processor 10A to produce a higher quality rendered image. However, in the embodiment of the present application, employing a more complex image rendering algorithm significantly increases the power consumption of the first image processor 10A. Therefore, this solution is not suitable for chip systems 100 with medium to low power consumption.

[0063] In some examples, as shown in FIG3 , the first image processor 10A includes a first image processing circuit 11A and a neural network processing circuit 12A. In the first image processor 10A, a rendered image is generated by the first image processing circuit 11A, and the rendered image is processed using an artificial intelligence (AI) algorithm by the neural network processing circuit 12A to obtain a rendered image with higher image quality. In an embodiment of the present application, a neural network processing circuit 12A for implementing the AI ​​algorithm is additionally designed and integrated into the first image processor 10A. This enables image rendering and AI algorithm processing within the first image processor 10A. Compared to the embodiment of FIG2 using a more complex image rendering algorithm, improving the quality of the rendered image through AI algorithm processing can reduce image rendering load, thereby reducing device computing power consumption, and achieve good anti-aliasing effects. Furthermore, closed-loop rendering and AI processing are performed within the first image processor 10A, resulting in low task scheduling overhead and a simple data path. However, this approach requires redesigning the IP core of the first image processor 10A, and the introduction of the neural network processing circuit 12A increases the area of ​​the first image processor 10A. Therefore, while this solution can reduce the power consumption of the first image processor 10A, it is not suitable for low- to medium-power chip architectures, which also have extremely high requirements for chip device area. Furthermore, this solution requires modification and design of the IP core of the first image processor 10A, increasing its difficulty of use and preventing its application in chip systems 100 based on other IP cores.

[0064] In some possible implementations, the chip system 100 may be a second chip system 100B that implements image rendering and AI algorithm processing based on multi-IP collaboration. In this case, the second chip system 100B may be a high-power chip architecture or a medium-to-low-power chip architecture. As shown in FIG4 , the second chip system 100B includes a second image processor 10B, a neural network processor 20B, and a scheduling circuit 30B. Among them: the second image processor 10B is used to: output a first rendered image according to an image subtask. The scheduling circuit 30B is used to: instruct the neural network processor 20B to obtain the first rendered image according to the scheduling subtask. The neural network processor 20B is used to: perform neural network algorithm processing on the first rendered image according to the operation subtask and output the operation result. The scheduling circuit 30B is also used to: instruct the second image processor 10B to obtain the operation result according to the scheduling subtask. The second image processor 10B is also used to: perform image processing on the operation result according to the image subtask to obtain a second rendered image.

[0065] In traditional chip architectures, the image processor 10 and the neural network processor 20B are independent IP cores. In typical chip designs, the image processor 10 and the neural network processor 20B primarily serve as auxiliary components to assist in image data processing and data computation. Consequently, both chips have relatively weak logic control capabilities. Furthermore, achieving coordinated image processing between the two requires a considerable amount of data exchange and complex control signals. Therefore, collaborative image data processing between the two IP cores is difficult. In the embodiment of the present application, as shown in FIG4 , a second image processor 10B generates a first rendered image based on an image subtask. After the second image processor 10B outputs the first rendered image, the scheduling circuit 30B instructs the neural network processor 20B to retrieve the first rendered image. After the neural network processor 20B retrieves the first rendered image, it performs AI algorithm processing (e.g., neural network algorithm processing) on ​​the first rendered image to obtain a computation result. At this point, the scheduling circuit 30B instructs the second image processor 10B to retrieve the computation result. Based on the computation result, the second image processor 10B generates a final rendered image, i.e., a second rendered image. The image quality of the second rendered image is higher than that of the first rendered image. By setting up a scheduling circuit 30B, the task execution between multiple IPs (including the second image processor 10B and the neural network processor 20B) is scheduled in a hardware manner. Under this embodiment, there is no need to change the IP core design of the second image processor 10B and the neural network processor 20B, and compatibility can be achieved under different chip architectures (such as X86 architecture or ARM architecture). Compared with using more complex rendering algorithms to improve imaging quality, AI algorithm processing based on the existing neural network processor 20B in the chip architecture not only improves the imaging quality of the rendered image, but also greatly reduces the power consumption of the second chip system 100B, so that the solution can be applied under chip architectures with different power consumption. At the same time, the hardware-based scheduling circuit 30B controls the execution order of the second image processor 10B to execute the image subtask and the neural network processor 20B to execute the calculation subtask. The scheduling method is simple, and the second image processor 10B and the neural network processor 20B only need to exchange the first rendered image and the calculation result, avoiding the interaction of complex data signals and control signals between multiple IPs.

[0066] In some possible implementations, as shown in FIG4 , the electronic device 1000 further includes a display 200. The display 200 is configured to display the second rendered image.

[0067] For example, the second chip system 100B may be an X86 architecture or an ARM architecture. For example, the second chip system 100B may be a very large scale integration circuit (VLSI), such as a system on chip (SoC) or a network on chip (NoC).

[0068] For example, the second image processor 10B may be a GPU, and the neural network processor 20B may be an NPU.

[0069] For example, the electronic device 1000 may be a mobile device or a terminal-side device. For example, the electronic device 1000 may be a mobile phone, a tablet computer, a smart watch, a television, an in-vehicle control device, a smart home device, etc.

[0070] In some possible implementations, the scheduling circuit 30B may instruct the second image processor 10B and the neural network processor 20B to perform related task operations by interacting with the second image processor 10B and the neural network processor 20B to execute task-related trigger signals.

[0071] Exemplarily, as shown in FIG5 , the second image processor 10B is further configured to: when outputting the first rendered image, output a first trigger signal to the scheduling circuit 30B based on the image subtask. The scheduling circuit 30B is specifically configured to: in response to the first trigger signal, output a second trigger signal to the neural network processor 20B based on the scheduling subtask. The second trigger signal is used to instruct the neural network processor 20B to acquire the first rendered image. The second image processor 10B is further configured to: when outputting a calculation result, output a third trigger signal to the scheduling circuit 30B based on the image subtask. The scheduling circuit 30B is specifically configured to: in response to the third trigger signal, output a fourth trigger signal to the second image processor 10B based on the scheduling subtask. The fourth trigger signal is used to instruct the second image processor 10B to acquire the calculation result. In this embodiment of the present application, after the second image processor 10B generates the relevant rendered image based on the image subtask, it outputs the first rendered image and then outputs the first trigger signal to the scheduling circuit 30B. In response to the received first trigger signal, the scheduling circuit 30B outputs the second trigger signal to the neural network processor 20B. The neural network processor 20B responds to the second trigger signal, receives the first rendered image output by the second image processor 10B, and performs AI algorithm processing (i.e., neural network algorithm processing) on ​​the first rendered image to obtain an operation result. While outputting the operation result, the neural network processor 20B outputs a third trigger signal to the scheduling circuit 30B. The scheduling circuit 30B responds to the third trigger signal and outputs a fourth trigger signal to the second image processor 10B. The second image processor 10B responds to the fourth trigger signal, obtains the operation result, and obtains the second rendered image based on the operation result. In an embodiment of the present application, only a few trigger signals are required to complete the scheduling of the second image processor 10B and the neural network processor 20B to realize multi-IP linkage processing. The various trigger signals involved in this embodiment may be notification signals or indication signals, which are used to control or notify other IP cores to perform the next operation. This embodiment does not limit its implementation form.

[0072] In some possible implementations, the first rendered image or the calculation result may be stored in a cache within the second image processor 10B or the neural network processor 20B.

[0073] In some possible implementations, in order to facilitate orderly linkage processing between the second image processor 10B and the neural network processor 20B, a general or dedicated storage circuit can be provided in the second chip system 100B to store the first rendered image and the calculation results in the storage circuit, so as to facilitate the transfer and interaction of the first rendered image and the calculation results between the second image processor 10B and the neural network processor 20B.

[0074] For example, as shown in FIG6 , the second chip system 100B further includes a storage circuit 40B. The second image processor 10B outputs the first rendered image to the storage circuit 40B and outputs a first trigger signal to the scheduling circuit 30B. The scheduling circuit 30B outputs a second trigger signal to the neural network processor 30B based on the first trigger signal. After the neural network processor 30B receives the second trigger signal, it responds to the second trigger signal and retrieves the first rendered image from the storage circuit 40B, performs a neural network operation on the first rendered image to obtain an operation result; the operation result is then stored in the storage circuit 40B, and a third trigger signal is output to the scheduling circuit 30B. The scheduling circuit 30B outputs a fourth trigger signal to the second image processor 10B based on the third trigger signal. When the second image processor 10B receives the fourth trigger signal, it responds to the fourth trigger signal and retrieves the operation result from the storage circuit 40B, and then obtains a second rendered image based on the operation result.

[0075] Exemplarily, the storage circuit 40B may be an on-chip system cache (SC). In some examples, the first rendered image and the computation result may be stored in different storage areas of the same storage circuit 40B. In some examples, the first rendered image and the computation result may be stored in different storage circuits 40B.

[0076] In some possible implementations, at least one of the multiple imaging parameters of the second rendered image is better than the imaging parameters of the first rendered image. Depending on the imaging parameters, different neural network algorithm processing can be performed in the neural network processor 20B to optimize the corresponding imaging parameters. In some examples, AI super-resolution algorithm processing can be performed by the neural network processor 20B to make the resolution of the second rendered image greater than the resolution of the first rendered image. In some examples, AI interpolation algorithm processing can be performed by the neural network processor 20B to make the frame rate of the second rendered image greater than the frame rate of the first rendered image. It should be understood that super-resolution or interpolation are two application scenarios of the present technical solution, and actual applications are not limited to this.

[0077] In some possible implementations, the neural network processor 20B may process the image using a neural network algorithm to obtain a calculation result as an image processing coefficient. The second image processor 10B processes the first rendered image according to the image processing coefficient to obtain a second rendered image.

[0078] Exemplarily, the second image processor 10B is also used to: obtain at least one pre-processed image based on the first rendered image. The first rendered image and at least one pre-processed image are processed according to the image processing coefficient to obtain a second rendered image. In an embodiment of the present application, the second image processor 10B can perform image processing to generate one or more pre-processed images while generating the first rendered image. The pre-processed image is used to assist in optimizing image parameters. After the neural network processor 20B outputs the image processing coefficient, the second image processor 10B can process the first rendered image and at least one pre-processed image according to the image processing coefficient to obtain a second rendered image. In an embodiment of the present application, it is necessary to perform calculations on the relevant image processing coefficients through the neural network processor 20B. The amount of data calculation of the neural network processor 20B is small, and the computing power requirements of the neural network processor 20B are low. The neural network processor 20B can quickly output the image processing coefficients as the calculation results.

[0079] In some possible implementations, the neural network processor 20B may process the image using a neural network algorithm to obtain a processed image that has been processed by the neural network algorithm and optimized for imaging parameters, and output the processed image as a calculation result. The second image processor 10B may perform some image rendering post-processing operations on the processed image to obtain a second rendered image.

[0080] Exemplarily, the second image processor 10B is further configured to obtain at least one pre-processed image based on the first rendered image. The neural network processor 20B is specifically configured to perform neural network algorithm processing on the first rendered image based on the at least one pre-processed image to obtain a processed image. In this embodiment of the present application, the second image processor 10B may also perform image processing to generate one or more pre-processed images while generating the first rendered image. The pre-processed images are used to assist in optimizing image parameters. The neural network processor 20B inputs the first rendered image and at least one pre-processed image. The neural network processor 20B performs neural network algorithm processing on the at least one pre-processed image and the first rendered image to obtain a processed image. The processed image is an image after imaging parameter optimization. The neural network processor 20B outputs the processed image as a calculation result. In this embodiment of the present application, the neural network processor 20B can directly obtain a processed image after imaging parameter optimization by performing neural network calculations, and then output the processed image as a calculation result to the second image processor 10B. The second image processor 10B performs post-processing on the relevant rendered image based on the processed image to obtain a second rendered image. In this embodiment of the present application, the computing power required of the neural network processor 20B is slightly higher. At the same time, the processing time of the neural network processor 20B will be longer, but the processing capacity of the second image processor 10B will be reduced.

[0081] In some possible implementations, each first rendered image is a complete imaging frame image. In an embodiment of the present application, the second image processor 10B generates an initial rendered image based on the image subtask, and each initial rendered image corresponds to a complete imaging frame image. Each initial rendered image is output as a first rendered image. The second image processor 10B and the neural network processor 20B process the first rendered image based on multi-IP linkage to obtain a second rendered image with optimized imaging parameters. In this implementation, for each first rendered image, it is necessary to first undergo transit processing from the second image processor 10B to the neural network processor 20B, and then transit processing from the neural network processor 20B to the second image processor 10B. Therefore, during this transit processing, due to the processing performance of the neural network processor 20B, the second image processor 10B may have a waiting delay waiting for the neural network processor 20B to output the calculation result; or, due to the processing performance of the second image processor 10B, the neural network processor 20B may have a waiting delay waiting for the second image processor 10B to output the next first rendered image or a waiting delay for outputting the first rendered image corresponding to the next initial rendered image.

[0082] In some possible implementations, multiple first rendered images correspond to a complete imaging frame image. Exemplarily, the second image processor 10B is further configured to: generate an initial rendered image based on an image subtask. Multiple first rendered images are obtained based on the initial rendered image; the multiple first rendered images correspond to different imaging portions of the initial rendered image. In an embodiment of the present application, the second image processor 10B generates a complete imaging frame image (i.e., an initial rendered image) based on the image subtask. The second image processor 10B then splits each initial rendered image into multiple sub-rendered images, i.e., multiple first rendered images. The multiple first rendered images are then output. The neural network processor 20B performs a neural network algorithm on each first rendered image in turn and outputs the corresponding calculation results. The second image processor 10B receives the calculation results corresponding to each first rendered image and, based on the calculation results, obtains the corresponding second rendered image. Finally, the multiple second rendered images corresponding to the multiple first rendered images are spliced ​​together to obtain a final complete frame image. In some examples, the multiple first rendered images corresponding to the same initial rendered image do not have the same imaging portions. In some examples, the multiple first rendered images corresponding to the same initial rendered image may have partially the same imaging portions.

[0083] In some possible implementations, as shown in Figure 7, the second chip system 100B further includes a processor 50B. An application (APP) is running on the processor 50B. When the application needs to display a rendered image, a rendering command is issued. The processor 50B is used to: output an image subtask, an operation subtask, and a scheduling subtask in response to the rendering command issued by the application. Among them, at least one of the image subtask, the operation subtask, or the scheduling subtask is provided with a heterogeneous synchronization signal; the heterogeneous synchronization signal is used to indicate the execution dependency between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate the execution dependency between the operation subtask and the scheduling subtask. In an embodiment of the present application, multiple subtasks with execution dependencies are output by the processor 50B. In the actual image processing process, the second image processor 10B, the neural network processor 20B, and the scheduling circuit 30B can contact the relevant dependencies of the subtasks to be processed according to the received trigger signal to execute and process the corresponding subtasks.

[0084] In some possible implementations, the processor 50B generates image subtasks, computation subtasks, and scheduling subtasks in different ways depending on the rendering commands issued by the application:

[0085] In some examples, the application manufacturer has coordinated the design of the rendering command. At this time, the rendering command already contains parameters related to multiple sub-tasks for image processing by multiple IPs in linkage. In this case: the processor 50B is also used to: generate an image rendering task group in response to the rendering command input by the application; the image rendering task group includes an image sub-task, an operation sub-task and a scheduling sub-task. In an embodiment of the present application, the application manufacturer can be linked with the hardware manufacturer. When designing the application, the application manufacturer makes design changes to the rendering command issued by the application. The issued rendering command has adaptively set parameters for the relevant sub-tasks that multiple IP hardware needs to perform on its own. The corresponding rendering image task group can be directly generated according to the rendering command. The rendering image task group already includes image sub-tasks, operation sub-tasks and scheduling sub-tasks. At this time, it is only necessary to insert heterogeneous synchronization signals in the corresponding sub-tasks through the SDK software program. After outputting the image sub-task, operation sub-task and scheduling sub-task, multiple IPs are linked to realize image processing based on the corresponding sub-tasks and their heterogeneous synchronization signals. In this solution, the processor 50B does not need to modify or generate some subtasks, thereby reducing the processing load of the processor 50B.

[0086] In some examples, the application manufacturer has not made any changes to the rendering command. In this case, the rendering command does not include parameters related to multiple subtasks for image processing by multi-IP linkage. In this case: the processor 50B is also used to: generate an image rendering task group in response to the rendering command input by the application; the image rendering task group includes image subtasks. Operation subtasks and scheduling subtasks are set in the image rendering task group. In an embodiment of the present application, because the rendering command has not been changed, the image rendering task group generated by the processor 50B according to the rendering command only includes image subtasks related to image rendering, and does not include subtasks related to the neural network processor 20B and the scheduling circuit 30B. It is necessary to set the operation subtasks and scheduling subtasks through the processor 50B.

[0087] In some possible implementations, the application may be a game application, a model design application, a drawing software application, a video editing application, etc.

[0088] In some examples, when the application is running, the second chip system 100B can automatically execute the contents recorded in the above embodiments. In some examples, when the application is running, in response to the user's input, the second chip system 100B starts to execute the contents recorded in the above embodiments. For example, a control switch is provided on the electronic device 1000, and the functions recorded in the above embodiments are turned on or off by the control switch. Alternatively, a human-computer interaction module (such as a display screen 200) is provided on the electronic device 1000, and the functions recorded in the above embodiments are turned on or off through the user's control operation on the human-computer interaction module.

[0089] Based on the second chip system 100B including the structures shown in FIG. 4 , FIG. 5 , FIG. 6 and FIG. 7 , the image processing method including the following steps S200 to S600 as shown in FIG. 8 may be executed:

[0090] S200 : The second image processor 10B outputs a first rendered image.

[0091] In the embodiment of the present application, as shown in FIG. 4 , FIG. 5 , FIG. 6 and FIG. 7 , the second image processor 10B performs image rendering processing according to the image subtask to obtain a first rendered image.

[0092] For example, when outputting the first rendered image, the second image processor 10B outputs a first trigger signal to the scheduling circuit 30B according to the image subtask. The first trigger signal is used to instruct the neural network processor 20B to execute the subsequent corresponding scheduling subtask.

[0093] S300: Instruct the neural network processor 20B to obtain a first rendered image.

[0094] In some possible implementations, as shown in Figures 4, 5, 6, and 7, the second image processor 10B performs preliminary image rendering processing to obtain a first rendered image. Then, the scheduling circuit 30B instructs the neural network processor 20B to obtain the first rendered image according to the scheduling subtask.

[0095] Exemplarily, the scheduling circuit 30B responds to the first trigger signal and outputs a second trigger signal to the neural network processor 20B according to the scheduling subtask; the second trigger signal is used to instruct the neural network processor 20B to obtain the first rendered image.

[0096] S400 , the neural network processor 20B performs a neural network algorithm on the first rendered image to obtain a calculation result.

[0097] In some possible implementations, as shown in Figures 4, 5, 6, and 7, the neural network processor 20B, in response to an instruction from the scheduling circuit 30B, performs a neural network algorithm on the first rendered image obtained through the preliminary rendering process by the second image processor 10B. The neural network algorithm processing yields a computation result.

[0098] Exemplarily, the neural network processor 20B obtains the first rendered image based on the second trigger signal and performs neural network algorithm processing on the first rendered image to obtain a calculation result. Upon outputting the calculation result, the second image processor 10B outputs a third trigger signal to the scheduling circuit 30B based on the image subtask. The third trigger signal is used to instruct the scheduling circuit 30B to execute the subsequent related scheduling subtask.

[0099] S500 , instructing the second image processor 10B to obtain a calculation result.

[0100] In some possible implementations, the scheduling circuit 30B instructs the second image processor 10B to obtain the operation result according to the scheduling subtask.

[0101] Exemplarily, the scheduling circuit 30B responds to the third trigger signal and outputs a fourth trigger signal to the second image processor 10B according to the scheduled subtask. The fourth trigger signal is used to instruct the second image processor 10B to obtain the operation result.

[0102] S600: The second image processor 10B performs image processing on the calculation result to obtain a second rendered image.

[0103] In some possible implementations, the second image processor 10B performs image processing on the calculation result according to the image subtask to obtain a second rendered image. The imaging parameters of the second rendered image are better than the imaging parameters of the first rendered image.

[0104] Exemplarily, the second image processor 10B executes the relevant image subtask in response to the fourth trigger signal to perform image processing on the operation result to obtain a second rendered image.

[0105] In an embodiment of the present application, the image processing method shown in FIG8 can be used to achieve linkage between the second image processor 10B and the neural network processor 20B to optimize the imaging parameters of the rendered image. During this process, using the neural network processor 20B to perform neural network algorithm processing can improve the imaging parameters of the rendered image. Compared to using more complex image rendering algorithms in the second image processor 10B to improve the imaging parameters of the rendered image, this solution can reduce the power consumption of the second chip system 100B. At the same time, because the second image processor 10B and the neural network processor 20B are independent of each other, direct linkage between them requires quite complex data path interaction and control signal interaction on the control link. However, in this embodiment of the present application, by providing a scheduling circuit 30B, the linkage between the second image processor 10B and the neural network processor 20B is scheduled through hardware scheduling. In this implementation, there is no need for complex hardware or software modifications to multiple IPs. Only a few simple trigger signals are needed to control the second image processor 10B and the neural network processor 20B to execute corresponding subtasks in an agreed order, thereby optimizing the imaging parameters of the rendered image through multi-IP linkage.

[0106] In some possible implementations, such as in step S200 of the embodiment shown in FIG8 , each first rendered image is a complete imaging frame image.

[0107] In some possible implementations, as in step S200 of the embodiment shown in FIG8 , the plurality of first rendered images correspond to a complete imaging frame image. In this case, step S200 further includes the following sub-operations of steps S210 to S220 as shown in FIG9 :

[0108] S210: Generate an initial rendering image according to the image subtask.

[0109] In some possible implementations, the second image processor 10B generates an initial rendered image according to the image subtask. The initial rendered image is a complete imaging frame image.

[0110] S220: Obtain a plurality of first rendered images according to the initial rendered image.

[0111] In the embodiment of the present application, the multiple first rendered images correspond to different imaging parts of the initial rendered image.

[0112] In the embodiment shown in FIG9 , the operations of steps S300 to S500 shown in FIG8 can be performed for each first rendered image. In step S600 , a second rendered image corresponding to a complete imaging frame image is obtained based on the multiple first rendered images corresponding to a complete imaging frame image and the calculation results thereof.

[0113] In an embodiment of the present application, as shown in FIG10 , the latency for optimizing each image frame when the first rendered image is a complete imaging frame is shown. FIG10 shows that first, the second image processor 10B performs compilation processing. Then, the second image processor 10B performs the initial rendering process in step S200 to obtain the first rendered image. The neural network processor 20B then performs the neural network algorithm in step S400 to obtain the computation result. The second image processor 10B then performs the subsequent rendering process in step S600 to obtain a parameter-optimized second rendered image based on the computation result. At this point, the processing latency for each complete imaging frame is approximately 19.5 ms. FIG11 is a schematic diagram of the processing flow corresponding to the embodiment shown in FIG9 . In the embodiment shown in FIG9 , when the initial rendered image, which is a complete imaging frame, is split into multiple first rendered images, steps S300 to S500 are sequentially performed on the multiple first rendered images. Then, in step S600, a second rendered image is obtained based on the multiple first rendered images corresponding to the same initial rendered image and the computation result. Taking the example of splitting an initial rendered image into four first rendered images, as shown in Figure 11, the four first rendered images are processed in a pipelined manner, reducing processing latency by 2ms and improving latency benefits by approximately 10.3%. When using the embodiments shown in Figures 9 and 11, the processing performance of the second image processor 10B and the neural network processor 20B can be more effectively utilized, minimizing waiting latency for processing.

[0114] In some possible implementations, the image processing method further includes: the second image processor 10B obtains at least one pre-processed image based on the first rendered image. In some examples, the at least one pre-processed image may be a pre-processed image obtained based on the first rendered image in step S100. In some examples, the at least one pre-processed image may be an image obtained by the second image processor 10B performing some image processing operations in parallel when the neural network processor 20B executes step S400, as shown in the embodiments shown in Figures 10 and 11. In some examples, the at least one pre-processed image may be a pre-processed image obtained by directly processing the first rendered image. In some examples, the at least one pre-processed image may be a pre-processed image obtained by processing the initial rendered image and then splitting the pre-processed initial rendered image according to the distribution relationship of multiple first rendered images on the initial rendered image.

[0115] In some possible implementations, as shown in FIG10 and FIG11 , when the neural network processor 20B performs step S400, the second image processor 10B may also perform some image processing operations in parallel, processing the first rendered image or the initial rendered image or at least one pre-processed image through the image processing operations to obtain some adjusted images, etc. For example, the image processing operation may be binning image processing.

[0116] In some possible implementations, in step S400, different calculation results are obtained depending on the different neural network algorithms.

[0117] In some examples, the result of the operation is an image processing coefficient. At this time, the specific operation of step S600 is: processing the first rendered image and at least one pre-processed image according to the image processing coefficient to obtain a second rendered image. In an embodiment of the present application, the operation result obtained by the neural network algorithm processing performed in step S400 is an image processing coefficient. Under this embodiment, the data processing volume of the neural network processor 20B is relatively small, and the computing power requirements and processing delay requirements of the neural network processor 20B are not high. In step S600, the second image processor 10B performs image processing on the obtained first rendered image and at least one pre-processed image according to the image processing coefficient. For example, when the image processing coefficient is a fusion coefficient, the first rendered image and at least one pre-processed image are fused based on the image processing coefficient to obtain a second rendered image with super resolution.

[0118] In some examples, the result of the operation is a processed image. The processed image is an image after imaging parameters are optimized after the neural network processor 20B performs neural network algorithm processing in step S400. At this time, the specific operation of step S400 is: the neural network processor 20B performs neural network algorithm processing on the first rendered image based on at least one pre-processed image to obtain a processed image. The specific operation of step S600 is: the second image processor 10B performs post-rendering processing on the processed image as the result of the operation to obtain a second rendered image. In an embodiment of the present application, the neural network processor 20B directly generates an image after parameter optimization in the operation of step S400, and then the second image processor 10B performs post-rendering processing on the generated processed image to obtain the final second rendered image. Under this embodiment, the data processing amount of the second image processor 10B is reduced.

[0119] In the embodiment of the present application, based on the actual application, the performance differences and processing delay differences between the second image processor 10B and the neural network processor 20B, etc., any of the above-mentioned forms of calculation results can be selected as the actual implementation plan.

[0120] Exemplarily, as shown in FIG12 , the second image processor 10B generates an initial rendered image through an image subtask. The initial rendered image is processed to obtain a first rendered image and at least one pre-processed image. The neural network processor 20B performs a neural network algorithm on the first rendered image to obtain a calculation result of the image processing coefficient. The second image processor 10B obtains a second rendered image based on the initial rendered image (or the first rendered image), the image processing coefficient (i.e., the calculation result) and at least one pre-processed image. In some examples, the second image processor 10B may also use previous historical frame images and historical frame data in the process of obtaining the first rendered image and / or obtaining at least one pre-processed image. The second image processor 10B may also use previous historical frame images in the process of obtaining the second rendered image.

[0121] Exemplarily, at least one pre-processed image may be an image directly processed by the first rendered image, or may be an image indirectly obtained by the first rendered image. In some examples, at least one pre-processed image may include an upsampling image, which is an upsampled large image without offset obtained by upsampling the first rendered image (or the corresponding initial rendered image) based on the Jitter offset coordinates. In some examples, at least one pre-processed image may include a Warp image, which is an image obtained by performing a Warp operation on the preceding complete frame image corresponding to the first rendered image based on the motion vector (MV) and depth information.

[0122] In some possible implementations, at least one of the following parameters may also be used in the processing of step S200, step S400, or step S600: Jitter offset coordinates of the current frame, color data of the current frame, color data of the historical frame (or previous frame), motion vector, depth information, and neural network filter (NNF) parameters, etc.

[0123] The above description of neural network algorithm processing is only an example. In actual applications, depending on the neural network algorithm used, a corresponding type of parameter image can be generated as at least one pre-processed image. These pre-processed images can be used in the neural network processor 20B, and can also be used in the second image processor 10B. It is also possible to add some other image processing parameters during image processing to assist in the processing. The embodiments of the present application do not limit the neural network algorithm used. In theory, any algorithm for optimizing imaging parameters can be adaptively applied to the solutions described in the embodiments of the present application.

[0124] In some possible implementations, as shown in FIG13 , the image processing method further includes the operation of step S100:

[0125] S100, output image subtask, calculation subtask and scheduling subtask.

[0126] In an embodiment of the present application, as shown in FIG7 , processor 50B outputs an image subtask, a computation subtask, and a scheduling subtask, respectively. At least one of the image subtask, computation subtask, or scheduling subtask is provided with a heterogeneous synchronization signal; the heterogeneous synchronization signal is used to indicate an execution dependency between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate an execution dependency between the computation subtask and the scheduling subtask.

[0127] Exemplarily, as shown in FIG14 , an application and a driver are running in the processor 50B. In the driver, there are callers corresponding to different IP cores, such as an image caller, an operation caller, a heterogeneous caller, and a memory allocation program. The application issues a rendering command. According to the rendering command, the application programming interface (API) required for the relevant task processing is called to call the driver. According to the called API interface, the driver encapsulates and sends the image subtask to the second image processor 10B through the image caller, encapsulates and sends the operation subtask to the neural network processor 20B through the operation caller, and encapsulates and sends the scheduling subtask to the scheduling circuit 30B through the heterogeneous caller. In this process, when it is necessary to assist in the storage of data such as the first rendered image and the operation result based on the storage circuit 40B, a storage subtask can also be sent to the storage circuit 40B through the memory allocation program to allocate corresponding storage space to different IP hardware.

[0128] Exemplarily, the API interface may include GLES services, Vulkan services, etc.

[0129] For example, taking the optimized image parameter as resolution, the driver can be a super-resolution SDK program. A deep learning super sampling (DLSS) driver can be designed as a super-resolution SDK.

[0130] In some examples, a heterogeneous synchronization signal is provided in the image subtask, and the heterogeneous synchronization signal can be used to instruct the second image processor 10B to output a first trigger signal when outputting the first rendered image, and trigger the scheduling circuit 30B to execute the relevant scheduling subtask (for example, output the second trigger signal) through the first trigger signal. Alternatively, the heterogeneous synchronization signal is used to instruct the second image processor 10B to execute the relevant image subtask to output the second rendered image according to the operation result when receiving the fourth trigger signal. In some examples, a heterogeneous synchronization signal is provided in the scheduling subtask, and the heterogeneous synchronization signal is used to instruct the scheduling circuit 30B to execute the relevant scheduling subtask to output the second trigger signal to the neural network processor 20B when receiving the first trigger signal. The heterogeneous synchronization signal can also be used to instruct the scheduling circuit 30B to execute the relevant scheduling subtask to output the fourth trigger signal to the second image processor 10B when receiving the third trigger signal. In some examples, a heterogeneous synchronization signal is set in the operation subtask, which can be used to instruct the neural network processor 20B to execute the relevant operation subtask (for example, performing AI algorithm processing on the first rendered image) when receiving the third trigger signal, or to instruct the neural network processor 20B to output the third trigger signal to the scheduling circuit 30B when outputting the operation result, etc.

[0131] In some examples, a software development kit (SDK) may be provided within the processor 50B. The SDK generates an API logic diagram based on the API interface called by the rendering command, determines the processing execution dependencies of different subtasks based on the API logic diagram, and inserts heterogeneous synchronization information into the corresponding subtasks based on the determined processing execution dependencies.

[0132] In some possible implementations, the processor 50B generates image subtasks, computation subtasks, and scheduling subtasks in different ways depending on the rendering commands issued by the application:

[0133] In some examples, the application manufacturer has designed the rendering command in a coordinated manner. In this case, the rendering command already includes parameters related to multiple subtasks of image processing for multi-IP linkage. In this case, step S100 includes the following sub-operations as shown in FIG15 , namely, steps S110 to S130:

[0134] S110: Generate an image rendering task group according to the rendering command.

[0135] In the embodiment of the present application, since the application manufacturer has coordinated the design of the rendering command, the processor 50B can directly call the relevant API interface according to the rendering command, encapsulate and generate the corresponding image subtasks, calculation subtasks, scheduling subtasks and storage subtasks, etc.

[0136] S120: Inserting a heterogeneous synchronization signal into the image rendering task.

[0137] In an embodiment of the present application, after an image rendering task group including subtasks such as image subtasks, calculation subtasks, scheduling subtasks and storage subtasks is generated according to a rendering command, the image rendering task group does not include the execution dependency relationship of the subtasks executed between different IPs. It is necessary to insert heterogeneous synchronization signals into the corresponding subtasks to construct the execution dependency relationship of the subtasks executed between different IPs.

[0138] S130, output image subtask, calculation subtask, scheduling subtask and storage subtask.

[0139] In the embodiment of the present application as shown in FIG15 , the technical principles and technical effects of the embodiment in which the application manufacturer cooperates with the design of rendering commands can be referred to the relevant description in the aforementioned hardware embodiment and will not be repeated here.

[0140] In some examples, the application manufacturer does not modify the rendering command. In this case, the rendering command does not include parameters related to multiple subtasks of image processing for multi-IP linkage. In this case, step S100 includes the following sub-operations as shown in FIG16 , namely, step S110 ′ to step S140 ′:

[0141] S110 ′: Generate an image rendering task group according to the rendering command.

[0142] In the embodiment of the present application, because the application manufacturer has not adapted the rendering command, the processor 50B generates an image rendering task group in response to the rendering command input by the application. However, the image rendering task group includes image subtasks but does not include operation subtasks and scheduling subtasks.

[0143] S120 ′: Set a computing subtask and a scheduling subtask in the image rendering task group.

[0144] Exemplarily, setting an operation subtask in the image rendering task group includes at least one of the following operations: modifying at least one image subtask into an operation subtask, or adding a new operation subtask to the image rendering task group. In an embodiment of the present application, because the second image processor 10B and the neural network processor 20B are required to perform multi-IP linkage. Therefore, some image subtasks that originally required the second image processor 10B to execute may be assigned to the neural network processor 20B for execution. At this time, some image subtasks can be modified and encapsulated into operation subtasks. At the same time, compared with traditional image rendering processing, neural network algorithm processing is added. Therefore, it is also necessary to add some new operation subtasks to instruct the neural network processor 20B to perform corresponding neural network algorithm processing, as well as related coordinated scheduling operations, etc.

[0145] For example, in order to ensure fast and easy coordinated processing between the second image processor 10B and the neural network processor 20B, some scheduling subtasks need to be added to instruct the scheduling circuit 30B to perform corresponding scheduling operations.

[0146] S130': inserting a heterogeneous synchronization signal into the image rendering task.

[0147] S140 ′, output image subtask, calculation subtask, scheduling subtask and storage subtask.

[0148] For the relevant descriptions of step S130' and step S140', reference may be made to the relevant descriptions of the aforementioned step S120 and step S130, which will not be repeated here.

[0149] In the embodiment of the present application as shown in Figure 16, because the rendering command has not been changed, the image rendering task group generated by the processor 50B according to the rendering command only includes image subtasks related to image rendering, and does not include subtasks related to the neural network processor 20B and the scheduling circuit 30B. It is necessary to set the operation subtask and the scheduling subtask through the processor 50B. Under this embodiment, there is no need to change the application, and the application can be seamless, that is, different applications can run normally and render commands below, and the SDK intercepts the API interface of the relevant image subtask, etc., and performs the encapsulation and modification operations into operation subtasks, inserting heterogeneous synchronization signals, etc. The embodiment of the present application as shown in Figure 16 can be adapted to different applications.

[0150] For example, the SDK may intercept the API interface in the following ways: field identification, view point identification, etc.

[0151] For example, assuming the application is a game and the optimized image parameter is resolution, the driver in processor 50B is a super-resolution SDK. After the super-resolution SDK begins operation, it intercepts API interfaces (such as GLES services and Vulkan services) called by the game application through rendering instructions. A logical graph of the API interfaces is generated. Based on the logical graph, the execution order between different subtasks is determined, and operation subtasks are set in the corresponding processing flow. Then, heterogeneous synchronization signals are inserted into the corresponding subtasks to generate execution dependencies between the different subtasks.

[0152] In some possible implementations, a hardware trigger circuit may be provided in the second image processor 10B to implement the generation of the first trigger signal in step S200 and to implement the initiation of the related image subtasks for generating the second rendered image in response to the fourth trigger signal in step S600.

[0153] In some possible implementations, the second image processor 10B can implement the generation of the first trigger signal in step S200 using software, and the initiation of the execution of the associated image subtask for generating the second rendered image in response to the fourth trigger signal in step S600 using software control. For example, a driver within the processor 50B encapsulates a recording command list (CmdList) based on the rendering commands issued by the application and creates a function entry. This function entry encapsulates the relevant application programming interface (API) required for executing the image subtask. Execution dependencies are then inserted into the associated image subtask in the entry buffer in the form of heterogeneous synchronization information, such as a wait-for-execution signal HTS_wait_instruction and a notification-for-execution signal HTS_notify_instruction. As shown in FIG17 , the driver development kit (DDK) uploads the service to the second image processor 10B. The task processing queue program (QueueM) in the second image processor 10B reads back the entry buffer to handle the synchronization between different entries. When the second image processor 10B executes a certain image subtask (e.g., outputting a first rendered image), the dependency on outputting the first trigger signal is released, and the first trigger signal is a notification-type signal, namely, HTS_notify_instruction. At this point, the second image processor 10B sends the first trigger signal to the scheduling circuit 30 via its internal trigger notification signal driver (Unique Signal Engine). The first trigger signal can be sent in the form of an event identifier (EventID). Similarly, the scheduling circuit 30B and the neural network processor 30B also have related heterogeneous synchronization signals. In response to the first trigger signal, the scheduling circuit 30B executes subsequent scheduling subtasks and schedules the neural network processor 30B to perform related computation subtasks. Then, after the neural network processor 30B completes the related computation subtasks, the scheduling circuit 30B can proactively send a fourth trigger signal to the second image processor 10B. The fourth trigger signal is also sent in the form of an event identifier. An event queue (event MBX) is provided within the second image processor 10B based on the entry buffer. A corresponding reference count (refcnt) is set for the enrty of each image subtask in the event queue.When the second image processor 10B has finished processing other matters and can execute subsequent image subtasks (for example, it can start executing the operation of generating the second rendered image in step S600), it reads back the event queue through the built-in trigger wait engine. When the fourth trigger signal is received, the corresponding HTS_wait_instruction dependency is released, and the event reference count value of the corresponding image subtask is 0, that is, EventID Refcnt! = 0. When the wait driver reads back the corresponding reference count value of 0, it means that the dependency of the subsequent image subtask is released, and the subsequent image subtask can be executed, such as executing the image subtask of generating the second rendered image in step S600. APB is a data transmission bus module. IRQ is an interrupt signal sending module. The count control module is used to adjust the technology in event MBX, etc.

[0154] Regarding the technical principles and technical effects of executing steps S200 to S600 after executing step S100 in the embodiments shown in Figures 13, 14, 15, 16 and 17, reference can be made to the relevant descriptions in the embodiments of Figures 8, 9, 10, 11 and 12 above, which will not be repeated here.

[0155] The embodiment of the present application provides a chip system, an image processing method and an electronic device. First, the chip system includes a hardware scheduling circuit, an image processor and a neural network processor. The scheduling circuit is used to instruct the image processor and the neural network processor to perform multi-IP linkage processing to generate a rendered image with better imaging parameters. The specific operation of the multi-IP linkage processing is as follows: the image processor executes the image subtask to render the image image to obtain a first rendered image. The neural network processor performs a neural network algorithm on the first rendered image to obtain a calculation result to complete the parameter optimization calculation operation. Finally, the image processor obtains a second rendered image based on the calculation result. One or more imaging parameters of the second rendered image are better than the first rendered image. First, the neural network processor and the image processor are independent devices, and the data path interaction and control signal interaction between the two are quite complex. And the software control logic of the two is relatively simple. The embodiment of the present application can realize multi-IP linkage image processing between the neural network processor and the image processor in a simple control manner by setting a hardware scheduling circuit. Secondly, the present application utilizes a neural network processor to assist in improving the quality of rendered images based on a neural network processing algorithm. Compared to using more complex image rendering algorithms in an image processor to improve image quality, embodiments of the present application can significantly reduce image processing complexity and improve image quality. Second, embodiments of the present application can split a complete imaging frame into multiple first rendered images. By pipeline-based multi-IP linkage processing of multiple first rendered images by the image processor and the neural network processor, the processing performance of the image processor and the neural network processor can be effectively utilized, and the processing latency of each complete frame can be reduced. Third, the chip system also includes a processor. The processor outputs image subtasks, scheduling subtasks, and computation subtasks based on rendering commands issued by the application. Embodiments of the present application can set up a corresponding SDK in the processor, allowing applications to call relevant image processing API interfaces through regular rendering commands without modifying the issued rendering commands. The set SDK intercepts the relevant API interfaces, modifies some image subtasks, and adds new image subtasks, computation subtasks, and scheduling subtasks. By inserting corresponding heterogeneous synchronization signals according to the processing order of the subtasks when encapsulating each subtask, execution dependency relationships between different subtasks are formed through the heterogeneous synchronization signals, so that the embodiment of this solution can be adapted to different applications.

[0156] An embodiment of the present application also provides a computer-readable storage medium, which includes instructions; when the instructions are run on the electronic device described in the above embodiments (for example, an electronic device including the structure shown in Figures 4, 5, 6 and 7), the electronic device executes the image processing method described in the above embodiments (for example, the image processing method described in Figures 8, 9, 13, 15 and 16).

[0157] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0158] The memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0159] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0160] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0163] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located on a single device or distributed across multiple devices. Some or all of the modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0164] In addition, the functional modules in the various embodiments of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0165] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more media that can be integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0166] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A chip system, characterized in that: It includes an image processor, a neural network processor and a scheduling circuit; wherein: The image processor is used to: output a first rendered image according to the image subtask; The scheduling circuit is used to: instruct the neural network processor to obtain the first rendering image according to the scheduling subtask; The neural network processor is used to: perform neural network algorithm processing on the first rendered image according to the operation subtask and output the operation result; The scheduling circuit is further used to: instruct the image processor to obtain the operation result according to the scheduling subtask; The image processor is further configured to: perform image processing on the operation result according to the image subtask to obtain a second rendered image.

2. The chip system according to claim 1, characterized in that: The image processor is further configured to: when outputting the first rendered image, output a first trigger signal to the scheduling circuit according to the image subtask; The scheduling circuit is specifically used to: in response to the first trigger signal, output a second trigger signal to the neural network processor according to the scheduling subtask; the second trigger signal is used to instruct the neural network processor to obtain the first rendering image; The image processor is further configured to: when outputting the operation result, output a third trigger signal to the scheduling circuit according to the image subtask; The scheduling circuit is specifically used to: in response to the third trigger signal, output a fourth trigger signal to the image processor according to the scheduling subtask; the fourth trigger signal is used to instruct the image processor to obtain the operation result.

3. The chip system according to claim 2, characterized in that: The chip system also includes a storage circuit; wherein: The image processor is specifically configured to: write the first rendered image into the storage circuit; and obtain the operation result from the storage circuit in response to the fourth trigger signal; The neural network processor is specifically used to: respond to the second trigger signal, read the first rendered image from the first buffer according to the operation subtask; and write the operation result into the storage circuit.

4. The chip system according to any one of claims 1 to 3, characterized in that: The image processor is also used for: According to the image subtask, generating an initial rendering image; A plurality of first rendering images are obtained according to the initial rendering image; the plurality of first rendering images correspond to different imaging parts of the initial rendering image.

5. The chip system according to any one of claims 1 to 4, characterized in that: The operation result is an image processing coefficient; the image processor is also used for: Obtain at least one pre-processed image according to the first rendered image; The first rendered image and the at least one pre-processed image are processed according to the image processing coefficient to obtain the second rendered image.

6. The chip system according to any one of claims 1 to 4, characterized in that: The operation result is a processed image; The image processor is further used to: obtain at least one pre-processed image according to the first rendered image; The neural network processor is specifically used to: perform neural network algorithm processing on the first rendered image according to the at least one pre-processed image to obtain the processed image.

7. The chip system according to any one of claims 1 to 6, characterized in that: The chip system also includes a processor; the processor is used for: Output the image subtask, the operation subtask and the scheduling subtask; at least one of the image subtask, the operation subtask or the scheduling subtask is provided with a heterogeneous synchronization signal; the heterogeneous synchronization signal is used to indicate the execution dependency between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate the execution dependency between the operation subtask and the scheduling subtask.

8. The chip system according to claim 7, characterized in that: The processor is further configured to: In response to a rendering command input by an application, an image rendering task group is generated; the image rendering task group includes the image subtask, the calculation subtask and the scheduling subtask.

9. The chip system according to claim 7, characterized in that: The processor is further configured to: In response to a rendering command input by an application, an image rendering task group is generated; the image rendering task group includes the image subtask; The calculation subtask and the scheduling subtask are set in the image rendering task group.

10. The chip system according to claim 9, characterized in that: The processor is specifically configured to perform at least one of the following: At least one of the image subtasks is modified into the computing subtask, or a new computing subtask is added to the image rendering task group.

11. The chip system according to claim 9 or 10, characterized in that: The processor is further configured to: Adding a new image subtask to the image rendering task group.

12. The chip system according to any one of claims 1 to 11, characterized in that: The second rendered image satisfies at least one of the following: A resolution of the second rendered image is greater than a resolution of the first rendered image, or a frame rate of the second rendered image is greater than a frame rate of the first rendered image.

13. An image processing method, characterized in that: Based on a chip system; the chip system includes an image processor, a neural network processor and a scheduling circuit; the method includes: The image processor outputs a first rendered image according to the image subtask; The scheduling circuit instructs the neural network processor to obtain the first rendered image according to the scheduling subtask; The neural network processor performs neural network algorithm processing on the first rendered image according to the operation subtask and outputs the operation result; The scheduling circuit further instructs the image processor to obtain the operation result according to the scheduling subtask; The image processor also performs image processing on the operation result according to the image subtask to obtain a second rendered image.

14. The image processing method according to claim 13, characterized in that: The method further comprises: When outputting the first rendered image, the image processor outputs a first trigger signal to the scheduling circuit according to the image subtask; The scheduling circuit outputs a second trigger signal to the neural network processor in response to the first trigger signal according to the scheduling subtask; the second trigger signal is used to instruct the neural network processor to obtain the first rendering image; When outputting the operation result, the image processor outputs a third trigger signal to the scheduling circuit according to the image subtask; The scheduling circuit responds to the third trigger signal and outputs a fourth trigger signal to the image processor according to the scheduling subtask; the fourth trigger signal is used to instruct the image processor to obtain the operation result.

15. The image processing method according to claim 13 or 14, characterized in that: The method further comprises: The image processor generates an initial rendering image according to the image subtask; obtains a plurality of first rendering images according to the initial rendering image; and the plurality of first rendering images correspond to different imaging parts of the initial rendering image.

16. The image processing method according to any one of claims 13 to 15, characterized in that: The operation result is an image processing coefficient; The method further includes: obtaining, by the image processor, at least one pre-processed image according to the first rendered image; Outputting the second rendered image according to the operation result includes: the image processor processes the first rendered image and the at least one pre-processed image according to the image processing coefficient to obtain the second rendered image.

17. The image processing method according to any one of claims 13 to 15, characterized in that: The operation result is a processed image; The method further includes: obtaining, by the image processor, at least one pre-processed image according to the first rendered image; The performing neural network algorithm processing on the first rendered image to obtain a calculation result includes: the neural network processor performing neural network algorithm processing on the first rendered image according to the at least one pre-processed image to obtain the processed image.

18. The image processing method according to any one of claims 13 to 17, characterized in that: The method further comprises: Output the image subtask, the operation subtask and the scheduling subtask; at least one of the image subtask, the operation subtask or the scheduling subtask is provided with a heterogeneous synchronization signal; the heterogeneous synchronization signal is used to indicate the execution dependency between the image subtask and the scheduling subtask, or the heterogeneous synchronization signal is used to indicate the execution dependency between the operation subtask and the scheduling subtask.

19. The image processing method according to claim 18, characterized in that: The method further comprises: In response to a rendering command input by an application, an image rendering task group is generated; the image rendering task group includes the image subtask, the calculation subtask and the scheduling subtask.

20. The image processing method according to claim 18, characterized in that: The method further comprises: In response to a rendering command input by an application, an image rendering task group is generated; the image rendering task group includes the image subtask; The calculation subtask and the scheduling subtask are set in the image rendering task group.

21. The image processing method according to claim 20, characterized in that: The step of setting the computing subtask in the image rendering task group includes: Perform at least one of the following operations: modify at least one of the image subtasks to be the computing subtask, or add a new computing subtask to the image rendering task group.

22. The image processing method according to claim 20 or 21, characterized in that: The method further comprises: Insert the new image subtask into the image rendering task group.

23. The image processing method according to any one of claims 13 to 22, characterized in that: The second rendered image satisfies at least one of the following: A resolution of the second rendered image is greater than a resolution of the first rendered image, or a frame rate of the second rendered image is greater than a frame rate of the first rendered image.

24. An electronic device, characterized in that: It comprises a circuit board and a chip system as described in any one of claims 1 to 12; the chip system is arranged on the circuit board.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions; when the instructions are executed on the electronic device as claimed in claim 24, the electronic device executes the image processing method as claimed in any one of claims 13-23.

Images