Image processing method, device and storage medium

Abstract

The present application relates to an image processing method, device and storage medium. After the target rendering operation is completed, the method immediately reads the rendering result from the rendering execution hardware to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation fails, it will be reflected in the rendering result. According to the rendering result, the anomaly of the target rendering operation can be found, so that the rendering error is located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for a complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] Current photo and video software, as well as photo editing software, all provide a variety of special effects rendering functions. Using the above software, you can overlay multiple special effects on the target object in the picture or video. During the rendering process, you need to render each special effect in turn, that is, the rendering process includes multiple rendering operations. During this process, if an error occurs in the rendering operation of a special effect, it will cause an error in the overall rendering process, and ultimately present an abnormal rendering effect.

[0003] When rendering errors occur, it is necessary to locate the erroneous special effect implementation code in order to correct the code error. However, the implementation code of special effects is large, and it is difficult to locate the erroneous code from a large amount of code. Summary of the invention

[0004] In order to solve the problem of how to quickly locate erroneous special effect implementation codes, the embodiments of the present application propose an image processing method, device and storage medium.

[0005] In a first aspect, an embodiment of the present application provides an image processing method. In an image rendering process including multiple rendering operations, the method includes:

[0006] After the target rendering operation is completed, the rendering result of the target rendering operation is written from the frame buffer unit of the graphics processor to the cache of the main processor;

[0007] Output the above rendering results in the main processor's cache.

[0008] After the target rendering operation is completed, the method provided by the embodiment of the present application immediately reads and outputs the rendering result from the graphics processor, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation makes a mistake, it will be reflected in the rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for the complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the method provided by the embodiment of the present application can not only be used for rendering anomaly detection on the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the method provided by the embodiment of the present application is used to perform frame capture processing, so as to capture the target game screen for analysis.

[0009] Optionally, the rendering result is used to perform rendering anomaly detection on the target rendering operation.

[0010] Optionally, the target rendering operation is determined according to a target control triggering event of the client human-computer interaction interface.

[0011] Since the memory overhead of reading rendering results from the graphics processor is large, in order to avoid frequently capturing rendering results, the rendering effect can be captured only when a target space trigger event occurs to reduce memory overhead.

[0012] Based on any of the above method embodiments, the implementation of outputting the above rendering result in the cache of the main processor may include at least one of the following steps:

[0013] Generate a display image on the main thread of the client according to the rendering result in the cache of the main processor, and display the display image on the display screen;

[0014] Generate a presentation image based on the rendering result in the cache of the main processor on the background thread of the client, and save the presentation image to a specified path of the first non-transient memory;

[0015] Generate a display image on the background thread of the client according to the rendering result in the cache of the main processor, and send the display image to the first receiving end device;

[0016] On the background thread of the client, save the rendering result in the cache of the main processor to a specified path of the second non-transient memory;

[0017] The rendering result in the cache of the main processor is sent to the second receiving end device on the background thread of the client.

[0018] Based on any of the above method embodiments, writing the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor includes:

[0019] The rendering result of the target rendering operation is written from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread of the client.

[0020] The method provided in the embodiment of the present application captures images in the rendering thread of the client, rather than at the operating system level, is not limited by system permissions, and can run normally on different operating systems and different operating system versions.

[0021] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0022] An image capture module is used to write a rendering result of a target rendering operation from a frame buffer unit of a graphics processor to a cache of a main processor after a target rendering operation is completed, wherein the target rendering operation is a rendering operation in an image rendering process including multiple rendering operations;

[0023] An image output module is used to output the rendering result in the cache of the main processor.

[0024] After the target rendering operation is completed, the electronic device provided by the embodiment of the present application immediately reads the rendering result from the graphics processor and outputs it, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation makes a mistake, it will be reflected in the rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for a complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the electronic device provided by the embodiment of the present application can not only be used for rendering anomaly detection on the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the electronic device provided by the embodiment of the present application is used to perform frame capture processing, so as to capture the target game screen for analysis.

[0025] Optionally, the rendering result is used to perform rendering anomaly detection on the target rendering operation.

[0026] Optionally, the target rendering operation is determined according to a target control triggering event of the client human-computer interaction interface.

[0027] Since the memory overhead of reading rendering results from the graphics processor is large, in order to avoid frequently capturing rendering results, the rendering effect can be captured only when a target space trigger event occurs to reduce memory overhead.

[0028] Based on any of the above electronic device embodiments, optionally, the image output module is used to perform at least one of the following steps:

[0029] Generate a display image on the main thread of the client according to the rendering result in the cache of the main processor, and display the display image on the display screen;

[0030] Generate a presentation image based on the rendering result in the cache of the main processor on the background thread of the client, and save the presentation image to a specified path of the first non-transient memory;

[0031] Generate a display image on the background thread of the client according to the rendering result in the cache of the main processor, and send the display image to the first receiving end device;

[0032] On the background thread of the client, save the rendering result in the cache of the main processor to a specified path of the second non-transient memory;

[0033] The rendering result in the cache of the main processor is sent to the second receiving end device on the background thread of the client.

[0034] Based on any of the above electronic device embodiments, optionally, the image capture module is used to: write the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread of the client.

[0035] The electronic device provided in the embodiment of the present application captures images in the rendering thread of the client, rather than at the operating system level, is not restricted by system permissions, and can run normally on different operating systems and different operating system versions.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory;

[0037] The memory is used to store a program for executing any method embodiment of the first aspect above;

[0038] The processor is configured to execute the program stored in the memory.

[0039] After the target rendering operation is completed, the electronic device provided by the embodiment of the present application immediately reads the rendering result from the graphics processor and outputs it, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation makes a mistake, it will be reflected in the rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for a complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the electronic device provided by the embodiment of the present application can not only be used for rendering anomaly detection on the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the electronic device provided by the embodiment of the present application is used to perform frame capture processing, so as to capture the target game screen for analysis.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a program for implementing any method embodiment of the first aspect.

[0041] After the target rendering operation is completed, the program stored in the computer-readable storage medium provided by the embodiment of the present application immediately reads and outputs the rendering result from the graphics processor, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation is wrong, it will be reflected in its rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for the complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the above program can not only be used for rendering anomaly detection of the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the above program is used for frame capture processing to capture the target game screen for analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application;

[0043] Figure 2 A flowchart of an image processing method provided by another embodiment of the present application;

[0044] Figure 3 A block diagram of an electronic device provided for one embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of a mobile terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In some processes described in the specification and claims of this application and the above-mentioned drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel.

[0048] The amount of code for a complete image rendering process is often large. Taking the image rendering process including face effects as an example, the amount of code is often tens of thousands or even hundreds of thousands of lines. A rendering process includes multiple rendering operations, and each rendering operation corresponds to an image effect. In the rendering process, after each rendering operation is completed, its rendering result is saved to the frame buffer (FBO) unit of the rendering execution hardware, and overwrites the rendering result of the previous rendering operation. After the entire rendering process is completed, the image will be displayed according to the final rendering result. When the rendering result of a complete rendering process including rendering operations of multiple image effects is wrong, it is impossible to intuitively understand which rendering operation has a problem. If the code of the image rendering process is traversed and checked line by line, the workload is huge. In view of this, the embodiment of the present application provides an image processing method, which reads the rendering result from the GPU (Graphics Processing Unit) in time after the target rendering operation is completed, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation is wrong, it will be reflected in the rendering result. According to the rendering result, the abnormality of the target rendering operation can be found, so that the rendering error is located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for the complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved.

[0049] The method provided in the embodiment of the present application can be applied to a virtual machine simulating the operating environment of a mobile terminal, or can be applied to a mobile terminal, and the embodiment of the present application does not limit this. In the image rendering process including multiple rendering operations, such as Figure 1 As shown, the method provided in the embodiment of the present application includes the following operations:

[0050] Step 101: After a target rendering operation is completed, a rendering result of the target rendering operation is written from a frame buffer unit of a GPU into a cache of a main processor.

[0051] The rendering result refers to the texture information obtained by the GPU performing the target rendering operation.

[0052] Different GPUs can be selected in different usage scenarios. As an example but not limitation, the GPU can be a graphics card, or a GPU module in an ASIC (Application Specific Integrated Circuit), etc.

[0053] Different main processors may be selected in different usage scenarios. As an example but not limitation, the main processor may be a CPU (Central Processing Unit), an MPU (Microprocessor Unit), an FPGA (Field Programmable Gate Array), an ASIC, and the like.

[0054] It should be pointed out that in actual applications, the GPU and the main processor may be implemented by the same chip, for example, the functions of the main processor and the GPU may be implemented on an ASIC.

[0055] Step 102: Output the rendering result in the cache of the main processor.

[0056] Optionally, the rendering result is used to perform rendering anomaly detection on the target rendering operation.

[0057] It should be pointed out that in practical applications, the method provided in the embodiment of the present application can also be used for frame capture processing and for other purposes, and the embodiment of the present application does not limit this.

[0058] Wherein, step 102 may include a combination of one or more of the following steps:

[0059] Generate a display image on the main thread of the client according to the rendering result in the cache of the main processor, and display the display image on the display screen;

[0060] Generate a presentation image based on the rendering result in the cache of the main processor on the background thread of the client, and save the presentation image to a specified path of the first non-transient memory;

[0061] Generate a display image on the background thread of the client according to the rendering result in the cache of the main processor, and send the display image to the first receiving end device;

[0062] On the background thread of the client, save the rendering result in the cache of the main processor to a specified path of the second non-transient memory;

[0063] The rendering result in the cache of the main processor is sent to the second receiving end device on the background thread of the client.

[0064] Among them, the size and resolution of the display image displayed on the display screen can be configured as needed, and can be either a large image of the entire display area or a thumbnail of a specified size. As an example and not a limitation, if the rendering operations corresponding to the special effects are all target rendering operations, then the rendering effects of each target rendering operation can be displayed separately after being captured into the cache of the main processor; or after the rendering process is completed, thumbnails of each target rendering operation can be displayed at the same time, and in response to the user's selection of a thumbnail of one of the target rendering operations, a large image of the target rendering operation can be displayed in full screen. Displaying the display image corresponding to the target rendering operation on the display screen can present the rendering results of the target rendering operation to the user in a timely and intuitive manner, so as to facilitate judging whether there is an abnormality in the target rendering operation by observing the display image.

[0065] The size and resolution of the display image stored in the designated path of the first non-transitory memory can be configured as needed. In practical applications, the display image stored in the first non-transitory memory and the display image displayed on the display screen can be the same image, that is, the display image is generated according to the above rendering result, and the display image is displayed on the display screen on the one hand, and the display image is saved to the designated path of the above first non-transitory memory on the other hand.

[0066] Different first non-transitory memories can be selected according to different usage scenarios. Taking the implementation of this method on a mobile terminal as an example, the first non-transitory memory can be the memory of the mobile terminal or a memory card external to the mobile terminal. If the first non-transitory memory is the memory of the mobile terminal, the above-mentioned specified path can be the storage path of the system album of the mobile terminal, that is, the above-mentioned display image is saved to the system album of the mobile terminal, so that the user can retrieve the display image through the system album. Taking the implementation of this method on a virtual machine running on a computer as an example, the first non-transitory memory can be the hard disk of the computer or an external storage device such as a memory card, a USB flash drive, a mobile hard disk, etc. external to the computer.

[0067] There are multiple ways to store the rendering result stored in the designated path of the second non-transitory memory. For example, the rendering result can be saved in a .TXT file, in a word document, or in a system log. In practical applications, the second non-transitory memory can be the same memory as the first non-transitory memory, or can be a different memory.

[0068] By saving display images or rendering results in non-transient memory, they can be used as historical data for later viewing and for viewing in cross-platform scenarios.

[0069] The size and resolution of the display image sent to the first receiving device can be configured as needed. In practical applications, the display image sent to the first receiving device and the stored display image and the display image displayed on the display screen can be the same image, that is, the display image is generated according to the rendering result, and the display image is displayed on the display screen, saved to the specified path of the first non-transient memory, and sent to the first receiving device. The first receiving device can be a mobile terminal, a computer, a server, etc.

[0070] There are many ways to store the rendering result sent to the second receiving end device. For example, the rendering result can be saved in a .TXT file, in a word document, or in a system log. In practical applications, the second receiving end device and the first receiving end device can be the same device or different devices.

[0071] Based on any of the above method embodiments, each rendering operation in the rendering process may be set as a target rendering operation.

[0072] Since reading rendering results from the GPU has a large memory overhead, in order to avoid frequently fetching rendering results, you can select some of the rendering operations as target rendering operations.

[0073] Among them, the target rendering operation can be pre-specified. For example, based on experience, during the rendering process, each special effect rendering operation is often prone to exceptions. In this case, each special effect rendering operation or some specified special effect rendering operations are configured as the target rendering operation in advance. During the rendering process, the target rendering operation is confirmed by reading the configuration information.

[0074] The target rendering operation can also be determined during the rendering process. In one implementation, during the rendering process, the target control of the client human-computer interaction interface is triggered to generate a trigger event, and the target rendering operation is determined based on the target control trigger event. Furthermore, the above-mentioned target rendering operation can be the rendering operation indicated by the target control trigger event, and the above-mentioned target rendering operation can also be the current rendering operation when the target control trigger event occurs. In another implementation, the timing duration of the timer is pre-set, and the timer is started at a specified time (for example, when the rendering process starts). When the timer times out, the current rendering operation is the target rendering operation.

[0075] The following takes the implementation of the above method on a mobile terminal as an example and describes the above method in combination with a specific application scenario.

[0076] The user selects the photo to be rendered through the client software installed on the mobile terminal. The photo is a portrait photo, and three special effects are selected: filter, beauty, and plastic surgery.

[0077] The client software includes a UI (User Interface) module, a VE (Video engine) module, an audio and video codec module, and a rendering (effect) module. The following three threads will be created during the rendering process: a main thread controlled by the UI module, a rendering thread controlled by the VE module, and a background thread controlled by the VE module. The rendering module implements its functions through the rendering thread.

[0078] In one embodiment, it is predetermined that the filter rendering operation, the beauty rendering operation, and the plastic rendering operation are all target rendering operations. After the rendering process starts, on the one hand, the VE module reads the photo selected by the user from the CPU of the mobile terminal through a callback method, converts it into a texture, and passes it to the Effect module. On the other hand, the VE module sends a broadcast message in the rendering thread, which carries the identification information of the filter rendering operation, the beauty rendering operation, and the plastic rendering operation (for example, the special effect name) to request image capture for these three rendering operations. After the effect module receives the texture, on the one hand, it performs filter processing and drawing by calling various OpenGL commands, and binds the texture information (rendering result) obtained by drawing to the FBO of the GPU. On the other hand, it receives the above-mentioned broadcast message, waits for the filter rendering operation to be completed, and then calls read The pixel (read pixel) method reads the texture information in FBO into the CPU cache; the VE module converts the texture information of the filter step into an RGB picture on the background thread and saves it to the system album of the mobile terminal; the effect module uses the texture information saved in FBO as input, continues to call various Opengl commands for beauty processing and drawing, and binds the drawn texture information to the GPU's FBO, and reads the texture information in FBO into the CPU cache by calling the read pixel method; the VE module converts the texture information of the beauty step into an RGB picture on the background thread and saves it to the system album of the mobile terminal; the effect module uses the texture information saved in FBO as input, continues to call various Opengl commands for shaping and drawing, and binds the drawn texture information to the GPU's FBO, and reads the texture information in FBO into the CPU cache by calling the read pixel method The pixel method reads the texture information in the FBO into the CPU cache; the VE module converts the texture information of the shaping step into an RGB image on the background thread and saves it to the system album of the mobile terminal, and saves the texture information (raw data) of the filter step, the beautification step and the shaping step in the mobile terminal's memory in text form; the UI module processes the texture information corresponding to the filter step, the beautification step and the shaping step into thumbnails on the main thread, and displays them together in the display area of ​​the mobile terminal display; if the user selects the thumbnail corresponding to one of the rendering operations, the UI module generates a large image corresponding to the rendering operation on the main thread, and displays the large image in a way that fills the display area.

[0079] In another embodiment, after the rendering process starts, the UI module prompts the current rendering operation on the display area of ​​the mobile terminal display screen through the main thread; the VE module reads the photo selected by the user from the CPU of the mobile terminal through the callback method, converts it into a texture and passes it to the Effect module; after receiving the texture, the effect module performs filter processing and drawing by calling various commands of OpenGL, and binds the texture information obtained by drawing to the FBO of the GPU; during the execution of the filter rendering operation, the user selects this step as the target rendering operation, the UI module detects the user's triggering action through the main thread, and sends an image capture request message to the effect module. The effect module receives the message and waits for the filter rendering operation to be completed by calling read The pixel method reads the texture information in FBO into the CPU cache; the VE module converts the texture information of the filter step into an RGB image on the background thread and saves it to the system album of the mobile terminal; the UI module processes the texture information corresponding to the filter rendering operation into a large image on the main thread, and displays the large image on the mobile terminal display screen in a way that fills the display area; the user confirms that there is no abnormality in the filter rendering operation by observing, and clicks the control on the display screen to indicate to continue the rendering process. The UI module responds to the user's trigger action and notifies the effect module to continue working. The effect module uses the texture information saved in FBO as input and continues Call various commands of OpenGL to perform beauty processing and drawing, and bind the texture information obtained by drawing to the FBO of GPU; the effect module takes the texture information saved in FBO as input, continues to call various commands of OpenGL to perform shaping processing and drawing, and binds the texture information obtained by drawing to the FBO of GPU; during the execution of shaping rendering operation, the user selects this step as the target rendering operation, the UI module detects the user's trigger action through the main thread, sends an image capture request message to the effect module, the effect module receives the message, waits for the shaping rendering operation to be completed, and reads the texture information in FBO into the CPU cache by calling the read pixel method; the VE module converts the texture information of the shaping rendering operation into an RGB picture on the background thread, and saves it to the system album of the mobile terminal; the UI module processes the texture information corresponding to the shaping rendering operation into a large picture on the main thread, and displays the large picture on the mobile terminal display screen in a way of filling the display area; the user determines that the shaping rendering operation is abnormal through observation, and can further view the code corresponding to the shaping rendering operation, or take other means to further locate the abnormal position.

[0080] In order to capture the rendering result of the target rendering operation, an existing frame capture tool can be used. The existing frame capture tools include the frame capture tool provided by Apple and the frame capture tool provided by Google. However, both of these frame capture tools have various problems, which make them unable to be used stably. Therefore, the inventor proposes a new image processing method that can run stably to realize image capture.

[0081] The image processing method can be run on a virtual machine that simulates the operating environment of a mobile terminal, can be applied on a mobile terminal, and can also be run on a computer operating system. Figure 2 As shown, the method provided in the embodiment of the present application includes the following operations:

[0082] Step 201: After the target rendering operation is completed, the rendering result of the target rendering operation is read from the frame buffer unit of the GPU through the rendering thread of the client.

[0083] The description of the GPU may refer to the above embodiments and will not be repeated here.

[0084] The description of the target rendering operation may refer to the above embodiment and will not be repeated here.

[0085] Step 202: Write the rendering result into the cache of the main processor through the rendering thread.

[0086] The description of the main processor may refer to the above embodiment and will not be repeated here.

[0087] The method provided in the embodiment of the present application captures images in the rendering thread of the client, rather than at the operating system level, is not limited by system permissions, and can run normally on different operating systems and different operating system versions.

[0088] Optionally, in the method provided in the embodiment of the present application, the video engine can send an image capture request message to the rendering module through the main thread or rendering thread of the client, and after receiving the image capture request message, the rendering module writes the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread. Accordingly, the target rendering operation is determined according to the image capture request message.

[0089] In actual applications, depending on different business scenarios, the image capture request message can be sent in the main thread of the client or in the rendering thread.

[0090] Since the memory overhead of reading rendering results from the rendering execution hardware is large, in order to avoid frequently capturing rendering results, the rendering effect can be captured only after receiving an image capture request message to reduce memory overhead.

[0091] Based on Figure 1 The same inventive concept as the method shown in the figure, the embodiment of the present application also provides an electronic device, such as Figure 3 As shown, the electronic device includes:

[0092] An image capture module 301 is used to write a rendering result of a target rendering operation from a frame buffer unit of a graphics processor to a cache of a main processor after a target rendering operation is completed, wherein the target rendering operation is a rendering operation in an image rendering process including multiple rendering operations;

[0093] The image output module 302 is used to output the rendering result in the cache of the main processor.

[0094] After the target rendering operation is completed, the electronic device provided by the embodiment of the present application immediately reads the rendering result from the graphics processor and outputs it, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation makes a mistake, it will be reflected in the rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for a complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the electronic device provided by the embodiment of the present application can not only be used for rendering anomaly detection on the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the electronic device provided by the embodiment of the present application is used to perform frame capture processing, so as to capture the target game screen for analysis.

[0095] Optionally, the rendering result is used to perform rendering anomaly detection on the target rendering operation.

[0096] Optionally, the target rendering operation is determined according to a target control triggering event of the client human-computer interaction interface.

[0097] Since the memory overhead of reading rendering results from the graphics processor is large, in order to avoid frequently capturing rendering results, the rendering effect can be captured only when a target space trigger event occurs to reduce memory overhead.

[0098] Based on any of the above electronic device embodiments, optionally, the image output module is used to perform at least one of the following steps:

[0099] Generate a display image on the main thread of the client according to the rendering result in the cache of the main processor, and display the display image on the display screen;

[0100] Generate a presentation image based on the rendering result in the cache of the main processor on the background thread of the client, and save the presentation image to a specified path of the first non-transient memory;

[0101] Generate a display image on the background thread of the client according to the rendering result in the cache of the main processor, and send the display image to the first receiving end device;

[0102] On the background thread of the client, save the rendering result in the cache of the main processor to a specified path of the second non-transient memory;

[0103] The rendering result in the cache of the main processor is sent to the second receiving end device on the background thread of the client.

[0104] Based on any of the above electronic device embodiments, optionally, the image capture module is used to: write the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread of the client.

[0105] The electronic device provided in the embodiment of the present application captures images in the rendering thread of the client, rather than at the operating system level, is not restricted by system permissions, and can run normally on different operating systems and different operating system versions.

[0106] The electronic devices in the above embodiments may include smart phones, PDAs, tablet computers, wearable devices with display screens, vehicle-mounted computers, smart speakers, personal computers, supercomputers, etc. Among them, virtual machines simulating the operating environment of mobile terminals are deployed on personal computers and supercomputers.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the module of the electronic device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0108] Based on the same inventive concept as the method, an embodiment of the present application provides an electronic device, including a processor and a memory;

[0109] The memory is used to store a program for executing any of the above method embodiments;

[0110] The processor is configured to execute the program stored in the memory.

[0111] After the target rendering operation is completed, the electronic device provided by the embodiment of the present application immediately reads the rendering result from the graphics processor and outputs it, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation makes a mistake, it will be reflected in the rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for a complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the electronic device provided by the embodiment of the present application can not only be used for rendering anomaly detection on the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the electronic device provided by the embodiment of the present application is used to perform frame capture processing, so as to capture the target game screen for analysis.

[0112] If the electronic device provided in the embodiment of the present application is a mobile terminal, such as Figure 4 As shown, the mobile terminal provided in the embodiment of the present application includes not only a processor and a memory, but also an input device (such as a touch screen, a camera, a microphone, etc.), an output device (such as a display screen, a speaker, etc.), a communication module, and a power module.

[0113] Among them, the memory, input device, output device, communication module, and power module are connected to the processor through a serial port, bus or USB interface. Among them, for a single-processor mobile terminal, the processor is a CPU (central processing unit); for a dual-processor mobile terminal, the processor includes a main processor and a slave processor, and the main processor executes the application to implement the method provided in the embodiment of the present invention. If communication with the outside world is required, the slave processor controls the communication module to cooperate and implement it; for a terminal device including a GPU (graphics processing unit) and a CPU, the processor refers to the GPU and the CPU, and the method provided in the embodiment of the present application is implemented by the GPU and the CPU in cooperation or by the CPU alone.

[0114] Among them, the memory of the mobile terminal may include but is not limited to: Flash (flash memory), RAM (random access memory), ROM (read-only memory), etc. RAM is mainly used to store programs and data when the terminal device is running. Programs that need to be executed or data that need to be processed must be loaded into RAM first. ROM is mainly used to check the configuration of the terminal device operating system and provide the most basic input and output (I / O) programs. Flash memory is a long-life non-volatile (it can still maintain stored data information in the event of a power outage) memory. Data deletion is not in units of single bytes, but in units of fixed blocks. Since Flash memory can still save data when power is off, it is usually used to save setting information, such as user setting information for the mobile phone.

[0115] It should be noted that for different mobile terminals (single-processor smartphones, dual-processor smartphones, smart wearable devices, tablet computers, etc.), it may include more Figure 4 The terminal device shown may have more or fewer hardware structures, but as long as it includes a memory and a processor and can implement the functions of the above method embodiments, it is within the protection scope of the present disclosure.

[0116] Based on the same inventive concept as the method, an embodiment of the present application provides a computer-readable storage medium storing a program for implementing any of the above method embodiments.

[0117] After the target rendering operation is completed, the program stored in the computer-readable storage medium provided by the embodiment of the present application immediately reads and outputs the rendering result from the graphics processor, so as to perform rendering anomaly detection on the target rendering operation according to the rendering result. If the target rendering operation is wrong, it will be reflected in its rendering result. The anomaly of the target rendering operation can be found according to the rendering result, so that the rendering error can be located in the target rendering operation, and then only the code corresponding to the target rendering operation needs to be troubleshooted. The amount of code for a single rendering operation is much smaller than the amount of code for the complete rendering process, so the purpose of quickly locating the rendering problem code can be achieved. It should be pointed out that the above program can not only be used for rendering anomaly detection of the target rendering operation, but also can be used in other implementation scenarios. For example, in a game scenario, the above program is used for frame capture processing to capture the target game screen for analysis.

[0118] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0119] The embodiments described above are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: In an image rendering process including a plurality of rendering operations, the method comprises: After each target rendering operation in the plurality of rendering operations is completed, writing a rendering result of the target rendering operation from a frame buffer unit of the graphics processor to a cache of the main processor; The rendering result in the cache of the main processor is output, and the rendering result is used to perform rendering anomaly detection on the target rendering operation.

2. The method according to claim 1, characterized in that The target rendering operation is determined according to a target control triggering event of the client human-computer interaction interface.

3. The method according to any one of claims 1 to 2, characterized in that: The step of outputting the rendering result in the cache of the main processor comprises at least one of the following steps: Generate a display image on the main thread of the client according to the rendering result in the cache of the main processor, and display the display image on a display screen; Generate a display image according to the rendering result in the cache of the main processor on the background thread of the client, and save the display image to a specified path of the first non-transitory memory; Generate a display image on a background thread of the client according to the rendering result in the cache of the main processor, and send the display image to a first receiving end device; On the background thread of the client, saving the rendering result in the cache of the main processor to a specified path of the second non-transient memory; The rendering result in the cache of the main processor is sent to the second receiving end device on the background thread of the client.

4. The method according to any one of claims 1 to 2, characterized in that: Writing the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor includes: The rendering result of the target rendering operation is written from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread of the client.

5. An electronic device, characterized in that: include: An image capture module, configured to write a rendering result of each target rendering operation in a plurality of rendering operations from a frame buffer unit of a graphics processor into a cache of a main processor after the target rendering operation is completed; An image output module is used to output the rendering result in the cache of the main processor, and the rendering result is used to perform rendering anomaly detection on the target rendering operation.

6. The electronic device according to claim 5, characterized in that: The image capture module is used to write the rendering result of the target rendering operation from the frame buffer unit of the graphics processor to the cache of the main processor on the rendering thread of the client.

7. An electronic device, characterized in that: including a processor and a memory; The memory is used to store a program for executing the method according to any one of claims 1 to 4; The processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium, characterized in that: A program for executing the method according to any one of claims 1 to 4 is stored.

Images