Image processing method and device, equipment and storage medium

Abstract

The invention discloses an image processing method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of obtaining a service code used for executing an image processing service, wherein the service code comprises at least one adjustable parameter; running the service code through a graphics processing unit (GPU) to obtain running data of the GPU, the running data being used for reflecting the running condition of the GPU; determining an adjustment trend of the adjustable parameters through a decision tree according to the operation data; wherein the decision tree is used for representing the corresponding relation between the adjustment trend of the adjustable parameters and the operation condition of the GPU; according to the adjusting trend of the adjustable parameters, the adjustable parameters are adjusted, adjusted service codes are obtained, and the adjusted service codes are used for executing image processing services. According to the method, the business code adjustment efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] Using business codes to implement business requirements is a very mature technology at present. Generally speaking, different business codes implement different business requirements.

[0003] In the related art, business developers need to adjust business codes so that the adjusted business codes can meet business requirements. In addition to meeting business requirements, the business codes also need to meet hardware requirements, that is, the computer equipment needs to support the operation of the business codes. Business developers need to adjust business codes according to both business requirements and hardware requirements.

[0004] When the above-mentioned business developers adjust the business code, they usually independently determine the adjustment content and direction of the business code based on prior knowledge, so the adjustment efficiency of the business code is low. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, device and storage medium, which can improve the adjustment efficiency of business codes. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, the method comprising:

[0007] Acquire a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter;

[0008] Running the business code through a GPU (Graphics Processing Unit) to obtain operation data of the GPU, where the operation data is used to reflect the operation status of the GPU;

[0009] Determining the adjustment trend of the adjustable parameter according to the operating data through a decision tree; wherein the decision tree is used to characterize the corresponding relationship between the adjustment trend of the adjustable parameter and the operating status of the GPU;

[0010] According to the adjustment trend of the adjustable parameter, the adjustable parameter is adjusted to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0011] According to one aspect of an embodiment of the present application, there is provided an image processing device, the device comprising:

[0012] A code acquisition module, used to acquire a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter;

[0013] A data acquisition module, used to run the business code through a graphics processing unit (GPU) to obtain the operation data of the GPU, where the operation data is used to reflect the operation status of the GPU;

[0014] A trend determination module, configured to determine an adjustment trend of the adjustable parameter according to the operation data through a decision tree; wherein the decision tree is used to characterize a corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU;

[0015] The parameter adjustment module adjusts the adjustable parameter according to the adjustment trend of the adjustable parameter to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0016] According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned image processing method.

[0017] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned image processing method.

[0018] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned image processing method.

[0019] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0020] Compared with the related art, in which business codes are adjusted manually based on prior knowledge, the present application can determine the adjustment trend of adjustable parameters in the business code through a decision tree based on the GPU operation data, and then adjust the business code, thereby improving the adjustment efficiency of the business code.

[0021] In addition, considering that the operation of the business code for executing image processing business is strongly related to the GPU in the computer device, starting from the perspective of the GPU's operating data, the adjustable parameters in the business code for executing image processing business are adjusted according to the operating data and the decision tree, thereby improving the rationality and accuracy of the business code adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of an image processing method provided by an embodiment of the present application;

[0024] Figure 3 is a flowchart of an image processing method provided by an embodiment of the present application;

[0025] Figure 4 is a flowchart of an image processing method provided by another embodiment of the present application;

[0026] Figure 5 is a flowchart of an image processing method provided by another embodiment of the present application;

[0027] Figure 6 is a schematic diagram of a decision tree provided by an embodiment of the present application;

[0028] Figure 7 is a block diagram of an image processing method provided by another embodiment of the present application;

[0029] Figure 8 is a block diagram of an image processing device provided by an embodiment of the present application;

[0030] Fig. 9 is a block diagram of an image processing device provided by another embodiment of the present application;

[0031] Fig.10 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0033] Before introducing the technical solution of the present application, some terms involved in the present application are explained. The following related explanations can be combined arbitrarily with the technical solution of the embodiment of the present application as optional solutions, and they all belong to the protection scope of the embodiment of the present application. The embodiment of the present application includes at least part of the following contents.

[0034] Decision Tree: Based on the known probability of occurrence of various situations, a decision tree is constructed to obtain the probability that the expected value of the net present value is greater than or equal to zero, evaluate the risk of the project, and judge its feasibility. It is a graphical method that intuitively uses probability analysis. Because this decision branch is drawn into a graph that looks like the branches of a tree, it is called a decision tree. In machine learning, a decision tree is a prediction model that represents a mapping relationship between object attributes and object values. Entropy = the degree of disorder of the system. The algorithm ID3, C4.5 and C5.0 are used to generate tree algorithms. This measure is based on the concept of entropy in information theory. A decision tree is a tree structure in which each internal node represents a test on an attribute, each branch represents a test output, and each leaf node represents a category. Classification tree (decision tree) is a very commonly used classification method. It is a supervised learning method. The so-called supervised learning is to give a bunch of samples, each sample has a set of attributes and a category, and these categories are determined in advance. Then, a classifier is obtained through learning, and this classifier can give the correct classification for the newly appeared objects. Such machine learning is called supervised learning. The decision tree in the embodiment of the present application takes the adjustable parameter as the root node, and the increase or decrease of the adjustable parameter as the branch corresponding to the root node, that is, the decision node or child node, and the increase or decrease of the performance indicator caused by the decision on the adjustable parameter as the leaf node of the decision tree.

[0035] ID3 algorithm: a greedy algorithm used to construct decision trees. The ID3 algorithm originated from the concept learning system. It uses the rate of decrease of information entropy as the criterion for selecting test attributes. That is, at each node, the attribute with the highest information gain that has not been used for division is selected as the division criterion, and then this process is continued until the generated decision tree can perfectly classify the training samples. The algorithm is based on information theory and uses information entropy and information gain as the measurement criteria to achieve inductive classification of data. The ID3 algorithm calculates the information gain of each attribute and selects the attribute with the highest gain as the test attribute for a given set. A node is created for the selected test attribute and marked with the attribute of the node. A branch is created for each value of the attribute to divide the sample accordingly.

[0036] GPU: Also known as display core, visual processor, display chip, it is a microprocessor that specializes in image computing on terminal devices such as personal computers, workstations, game consoles, tablet computers, and smart phones. Its purpose is to convert and drive the display information required by the computer system, and provide line scanning signals to the display to control the correct display of the display. It is an important component that connects the display and the personal computer motherboard, and is also one of the important devices for "human-computer dialogue". As an important component of the computer host, the graphics card is responsible for the task of outputting display graphics. For people engaged in professional graphic design, the graphics card is very important.

[0037] Least squares regression nonlinear model: When a model φ(t) is used to describe a series of data in reality, there will always be a certain deviation between the model's prediction results and the actual measurement results. This deviation is called the residual. The purpose of nonlinear least squares is to adjust the parameters of the model to minimize the total residual. In the embodiment of the present application, the least squares regression nonlinear model is used to minimize the modulus of the bottleneck vector.

[0038] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include: a terminal device 10 and a server 20.

[0039] The terminal device 10 includes but is not limited to mobile phones, tablet computers, intelligent voice interaction devices, game consoles, wearable devices, multimedia playback devices, PCs (Personal Computers), vehicle terminals, smart home appliances and other electronic devices. The client of the target application can be installed in the terminal device 10.

[0040] In an embodiment of the present application, the target application may be any application capable of adjusting the service code. In some embodiments, the target application is a developer tool that can adjust the service code, and the service code is adjusted by the developer tool. Of course, in addition to the developer tool, other types of applications may also be used to adjust the service code. For example, other integrated development environments for development or debugging running on a computer device. Of course, the service code involved in the embodiment of the present application is mainly for user image processing services, and the image processing services can be used in any application requiring image processing. Exemplarily, the application requiring image processing is an image application, a game application, a video application, a social application, a browser application, a virtual reality (VR) application, an augmented reality (AR) application, etc., and the embodiment of the present application does not limit this. Of course, for different applications requiring image processing, the image processing services required are different, that is, the service codes corresponding to the image processing services are also different. The embodiment of the present application does not limit this. Optionally, a client of the target application is running in the terminal device 10.

[0041] The server 20 is used to provide background services for the client of the target application in the terminal device 10. For example, the server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but is not limited to these.

[0042] The terminal device 10 and the server 20 can communicate with each other via a network, which can be a wired network or a wireless network.

[0043] The method provided in the embodiment of the present application may be performed by a computer device. The computer device may be any electronic device with data calculation, processing and storage capabilities. For example, the computer device may be Figure 1 The server 20 in may be Figure 1 The terminal device 10 in the example may also be another device other than the terminal device 10 and the server 20.

[0044] In the related art, when adjusting the code, it is generally necessary to rely on performance business developers to manually optimize the rendering performance. The manual tuning process can usually bring relatively good adjustment results, but there are at least two problems. First, the tuning space is large and many adjustments may lead to poor performance of the business code. Second, if the number of tuning times is large, it will bring a large personnel cost.

[0045] Based on the problems caused by the above-mentioned manual tuning, this application adopts a business code adjustment algorithm based on a decision tree, which can reduce manual intervention in the tuning process, improve the efficiency of business code adjustment, and at the same time reduce the cost of business code adjustment.

[0046] In some embodiments, the business code is written by business developers based on business needs, and the written business code is placed on a computer device for running and testing, which can bring about the running results of the software and the running results of the hardware, wherein the running results of the software can be understood as the business results implemented by the code, and the running results of the hardware can be understood as the reaction of the hardware used to run the business code to the business code, that is, the running data of the hardware. The embodiment of the present application takes into account that when executing image business, it is necessary to call the GPU to execute the business code, that is, the business code is strongly related to the GPU, so the running data on the GPU (the running status of the GPU as hardware) is used to indirectly reflect the adjustment trend of the adjustable parameters in the business code, so as to achieve the adjustment of the business code.

[0047] Please refer to Figure 2 , which shows a schematic diagram of an image processing method provided by an embodiment of the present application. Figure 2 As shown, first, a business code including multiple adjustable parameters is sent to a device 200 (such as a GPU) that runs the business code to obtain the operation data of the GPU, which includes the values ​​of multiple performance indicators. After obtaining the values ​​of the performance indicators, the performance indicators that need to be optimized among these performance indicators are determined based on the established function model. At the same time, the optimization trends of these performance indicators that need to be optimized can also be determined. Further, the leaf node corresponding to the optimization trend of the performance indicator that needs to be optimized is found from the established decision tree, and the adjustment trend of the adjustable parameters is deduced, thereby realizing the adjustment of the business code.

[0048] The technical solution provided in the embodiment of the present application establishes a function model based on the experience of running software on hardware. Based on the function model, the performance indicators that need to be optimized in the business code can be inferred from the operation data of the GPU, and further, the adjustment trend of the adjustable parameters can be determined from the established decision tree according to the optimization of the performance indicators. The technical solution provided in the embodiment of the present application can accelerate the automatic adjustment of the business code when the business code needs to be ported to different hardware or needs to be processed with different adjustable parameters, that is, improve the adjustment efficiency of the business code.

[0049] Please refer to Figure 3 , which shows a flow chart of an image processing method provided by an embodiment of the present application. The execution subject of each step of the method can be Figure 1 The terminal device 10 or server 20 in the implementation environment of the solution shown. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as "computer device". The method may include at least one of the following steps (310-340):

[0050] Step 310: Obtain a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter.

[0051] The reason why the embodiment of the present application can associate the operation result of the hardware with the adjustment of the business code is that the embodiment of the present application adjusts the business code based on the operation result of the hardware when the business code related to the image processing business is adjusted. It should be known that GPU, as a graphics processor, is widely used in image processing business. When running the business code related to the image processing business, the operation status of the GPU can reflect the quality of the business code. Exemplarily, when the operation status of the GPU is good, it is considered that the business code is better, and the business code does not need to be further adjusted. When the operation status of the GPU is not good, it is considered that the business code is inferior and the business code needs to be adjusted.

[0052] Of course, the adjustment of the business code in the embodiment of the present application can be adjusted not only according to the operating results of the hardware, but also according to other business needs. The adjustment of this part is not shown in the present application. In addition, the business code can also be adjusted multiple times according to the operating results of the hardware in the present application. For example, each time the business code is adjusted, steps 310 to 340 are executed once. The business code in step 310 can be considered as a business code that has not been adjusted, or it can be considered as a business code at any time during the adjustment process. The present application does not limit this. Of course, the adjustment process of the business code mentioned in the embodiment of the present application can also be placed in the final stage of the overall code adjustment. After the business code is adjusted according to the operating results of the hardware, it is considered that the business code is a better code, and the business code is no longer adjusted further. The adjusted business code can be better used to execute the image processing business.

[0053] Several terms in step 310 are explained below.

[0054] Image processing service: service related to image processing. This application does not limit the specific type of image processing service. Exemplarily, the image processing service is a texture rendering service. When executing the texture rendering service, it is necessary to divide the input image into patches to obtain multiple vertices corresponding to each patch, and determine the texture resource of the point according to one of the vertices and multiple vertices around it. Re-render the image according to the texture resources determined for each vertex to obtain the rendered image. When executing image rendering services like this, it is necessary to call the GPU more frequently, so this type of service is considered to be a service related to image processing in this application. Exemplarily, the image processing service is an artificial intelligence image generation service. When generating artificial intelligence images, it is also closely related to the GPU, so this type of service can also be considered to be a service related to image processing. Exemplarily, the image processing service is an image generation service, that is, when a new image corresponding to the text requirement is generated based on the given image and text requirements, it is also necessary to call the GPU more frequently, so this type of service can also be considered to be a service related to image processing. In the embodiment of this application, the specific type of image processing service is not limited. When the service is related to image processing, it is considered to be the above-mentioned image processing service.

[0055] Business code: code used to execute the above-mentioned image processing business. Exemplarily, the business developer writes the corresponding business code according to the specific business needs or according to the above-mentioned image processing business. The embodiment of the present application does not limit the source of the business code. The business code can be written by the business developer according to the specific image processing business, or it can be the business code directly obtained from the ready-made database, or it can be the code automatically generated based on the model. Of course, no matter what the source of the business code is, the technical solution provided by the embodiment of the present application can be used to adjust the business code, so that the adjusted business code can better match the hardware, so that the final operation result on the hardware is better.

[0056] Adjustable parameters: parameters that can be adjusted in the business code. For a complete business code, there are many parameters, some of which are not adjustable, which can be considered as fixed parameters, and some parameters are adjustable, that is, adjustable parameters. Regardless of whether it is a fixed parameter or an adjustable parameter, the expression in the business code can be a few lines of code (a code segment), or it can be in the form of a numerical value, a vector, etc. In some embodiments, different adjustable parameters correspond to different attributes. Exemplarily, the attribute characterizes at least one of the image rendering method, rendering speed, rendering start position, rendering end position, etc. In some embodiments, the adjustable parameters in the embodiments of the present application are considered to be adjustable parameters that do not affect the image processing performance. That is, when the adjustable parameters in the business code are adjusted, the operating results of the hardware can be improved without affecting the image results, so that the portability of the business code is better.

[0057] Of course, the specific method of obtaining the service code is not limited in the embodiments of the present application. For example, when a service developer adjusts the service code on a computer device, the service code is directly called from the target folder through the integrated development environment on the computer device. Of course, the service developer can also directly enter the service code in the integrated development environment.

[0058] Step 320, the GPU runs the service code to obtain the GPU operation data, and the operation data is used to reflect the operation status of the GPU.

[0059] In some embodiments, a GPU is a type of hardware device. However, the GPU in the embodiments of the present application is not necessarily a GPU on a real computer device, but may also be a GPU on a virtual computer device. In some embodiments, when the GPU in the embodiments of the present application is a GPU on a real computer device, the real computer device may be considered to be a computer device where the above-mentioned client is located, or may be considered to be a computer device other than the computer device where the above-mentioned client is located. Exemplarily, a first computer device is used to run a business code through a GPU to obtain the operation data of the GPU. A second computer device is used to receive the operation data of the GPU sent by the first computer device, and to perform the following step 330. The first computer device and the second computer device may be the same device or different devices. In some embodiments, when the GPU in the embodiments of the present application is a GPU on a virtual computer device, a virtual environment of the virtual computer device is simulated on a computer device, and the business code is run using the virtual environment.

[0060] In some embodiments, step 320 may also be described as running the service code through the GPU on the computer device to obtain the GPU's operating data, and the operating data is used to reflect the operating status of the GPU of the computer device. In other embodiments, step 320 may also be described as running the service code using only the GPU without using other devices to obtain the GPU's operating data, and the operating data is used to reflect the operating status of the GPU.

[0061] The embodiments of the present application do not limit the specific type of operation data, and the operation data is used to reflect the operation status of the GPU. Exemplarily, the operation status of the operation data includes but is not limited to pressure conditions, hardware usage, hardware call frequency, etc. In the embodiments of the present application, there is no need to consider the specific type of operation data. The data generated by the GPU as hardware during operation can be considered as the operation data here, and the operation data reflects the operation status of the GPU.

[0062] Of course, the method for obtaining the operation data is not limited in the embodiments of the present application. Exemplarily, when the first computer device and the second computer device belong to the same device, the operation data is directly obtained from the designated folder storing the operation data. When the first computer device and the second computer device belong to different devices, a communication connection is established between the first computer device and the second computer device, the first computer device sends the operation data to the second computer device, and the second computer device executes the subsequent process based on the operation data.

[0063] Step 330, determining the adjustment trend of the adjustable parameter according to the operating data through a decision tree; wherein the decision tree is used to characterize the corresponding relationship between the adjustment trend of the adjustable parameter and the operating status of the GPU.

[0064] The decision tree in the embodiment of the present application is determined in advance, that is, the corresponding relationship between the adjustment trend of the adjustable parameters and the operating conditions of the GPU is determined in advance in the present application. The corresponding relationship between the adjustment trend of the adjustable parameters and the operating conditions of the GPU here can also be understood as that when the adjustable parameters change, the operating conditions of the GPU will change accordingly. This means that when the operating conditions of the GPU are known, the adjustment trend of the adjustable parameters can be inferred. The specific architecture of the decision tree is not limited in the present application, and the algorithm for constructing the decision tree includes but is not limited to the greedy algorithm.

[0065] In the embodiments of the present application, it is not necessary to use only the decision tree method to characterize the corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU, and other forms can also be used. For example, a corresponding table between the adjustment trend of the adjustable parameter and the operation status of the GPU is established, and the corresponding table characterizes the corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU. For another example, a trend determination model is established, and the trend determination model is a trained machine learning model. Exemplarily, the input of the trend determination model is the operation data of the GPU, and the output is the adjustment trend of the adjustable parameter. The model architecture of the trend determination model is not limited in this application. For example, the trend determination model can be a coding layer, a feature fusion layer, a decoding layer, and a classification layer, wherein the encoding feature corresponding to the operation data is obtained through the coding layer, the fusion feature of the encoded feature is obtained through the feature fusion layer, and finally the decoding feature corresponding to the fusion feature is obtained based on the decoding layer, and finally the adjustment trend of at least one adjustable parameter is obtained through the classification network. The training process of the trend determination model is as follows: determine multiple groups of training samples, and a group of training samples includes sample operation data with corresponding relationships and label adjustment trends of adjustable parameters. The sample operation data is input into the trend determination model to obtain the output adjustment trend of the adjustable parameter. According to the difference between the label adjustment trend of the adjustable parameter and the output adjustment trend of the adjustable parameter, the loss function value is determined, and the trend determination model is trained with the goal of minimizing the loss function value. Of course, no matter which of the above three methods is used, a corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU is provided. That is, step 330 can also be summarized as determining the adjustment trend of the adjustable parameter according to the operation data through the corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU.

[0066] Step 340: Adjust the adjustable parameters according to the adjustment trend of the adjustable parameters to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0067] In some embodiments, the adjustment trend includes but is not limited to upward adjustment or downward adjustment. When the adjustment trend of the adjustable parameter is indicated to be upward adjustment, the adjustable parameter is adjusted upward on the basis of the current adjustable parameter. When the adjustment trend of the adjustable parameter is indicated to be downward adjustment, the adjustable parameter is adjusted downward on the basis of the current adjustable parameter. In some embodiments, the adjustment of the adjustable parameter can be determined by the computer device based on the determined adjustment trend, according to a certain step length, to determine the adjustment result of the adjustable parameter. Exemplarily, when it is determined that the adjustment trend of the adjustable parameter a is upward adjustment, the adjustable parameter a is adjusted according to the set step length to obtain the adjustment result of the adjustable parameter a. Based on the adjustment result, the adjusted business code is determined, and the steps of steps 320 to 340 are continued to be performed on the adjusted business code. When it is determined that the adjustment trend of the adjustable parameter a is still upward adjustment, the adjustment is continued according to the fixed step length on the basis of the last adjustment. Exemplarily, in the embodiment of the present application, each adjustable parameter corresponds to a fixed step length, and the fixed step length is set in advance by the business developer. Exemplarily, different adjustable parameters have different fixed step sizes.

[0068] In some embodiments, the above steps 320 to 340 are continued to be performed for the adjusted business code until the GPU operation data meets the conditions, or the hardware operation results meet the conditions, and the business code is no longer adjusted and is put into use.

[0069] Compared with the related art, the technical solution provided in the embodiment of the present application directly uses manual adjustment of business codes based on prior knowledge. It can determine the adjustment trend of adjustable parameters in the business code through a decision tree based on the operation data of the GPU, and then adjust the business code, which can improve the adjustment efficiency of the business code.

[0070] In addition, considering that the operation of the business code for executing image processing business is strongly related to the GPU in the computer device, starting from the perspective of the GPU's operating data, the adjustable parameters in the business code for executing image processing business are adjusted according to the operating data and the decision tree, thereby improving the rationality and accuracy of the business code adjustment.

[0071] Please refer to Figure 4 , which shows a flow chart of an image processing method provided by another embodiment of the present application. The execution subject of each step of the method may be Figure 1 The terminal device 10 or server 20 in the implementation environment of the solution shown. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method may include at least one of the following steps (410-450):

[0072] Step 410: Obtain a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter.

[0073] Step 420, running the business code through the graphics processor GPU to obtain the GPU operation data, the operation data is used to reflect the operation status of the GPU.

[0074] Step 430: Determine at least one target performance indicator and an expected optimization trend of the target performance indicator from at least one performance indicator according to the operation data, wherein the target performance indicator is a performance indicator that needs to be optimized.

[0075] In some embodiments, the operating data of the GPU includes the value of at least one performance indicator of the GPU, and the performance indicator is an indicator that reflects the operating status of the GPU. In some embodiments, the operating data includes the value of at least one performance indicator. The performance indicator can be considered as an indicator for measuring the performance of the GPU. In some embodiments, a GPU performance counter (GPUCounter) is used to obtain the value of the performance indicator. In some embodiments, a GPU performance counter is used to display different performance indicators on the GPU. Exemplarily, the performance indicator includes but is not limited to parameters such as GPU load, shading, bandwidth, and primitives, which are used to help business developers analyze GPU performance pressure in more detail. In some embodiments, the GPU performance counter corresponds to the performance indicator of the GPU one by one, and the value of the GPU performance counter is equal to the value of the performance indicator of the GPU. In some embodiments, the value of at least one performance indicator of the GPU is obtained through the GPU performance counter.

[0076] In some embodiments, the performance indicator that needs to be optimized is determined according to the value of each performance indicator in the operation data. In some embodiments, different performance indicators correspond to different performance thresholds, and when the value of a performance indicator is greater than the performance threshold, the performance indicator is considered to be a performance indicator that needs to be optimized.

[0077] In some embodiments, step 430 includes step 431 or step 432 (not shown).

[0078] Step 431, determine the model and operating data based on the first indicator, and determine at least one target performance indicator and the expected optimization trend of the target performance indicator from at least one performance indicator. The first indicator determines that the model is a function model, and the function model reflects the functional relationship between the pressure value of the GPU and at least one performance indicator. The functional relationship represents that the pressure value of the GPU changes with the change of at least one performance indicator.

[0079] In some embodiments, the first indicator determination model is determined in advance, and the function model can reflect the functional relationship between the pressure value of the GPU and at least one performance indicator. That is, through the function model, the relationship between the pressure value of the GPU and the change of at least one performance indicator can be known. The pressure value of the GPU here can be understood as the overall performance of the GPU. When the pressure value of the GPU tends to 0, it can be considered that the overall performance of the GPU is better, and when the pressure value of the GPU is larger, it can be considered that the overall performance of the GPU is worse. Therefore, the purpose of establishing the first indicator determination model is to be able to clarify the impact of the performance indicators in the GPU on the performance of the GPU. Therefore, in order to improve the performance of the GPU, it is possible to know the performance indicators that need to be optimized and the expected optimization trend of the indicators.

[0080] The following describes how to determine the first indicator determination model.

[0081] In some embodiments, based on prior knowledge, a functional relationship between the stress value of the GPU and at least one performance indicator is established, and a first indicator determination model is generated based on the functional relationship; or, a test program is run, and adjustable parameters in the test program are adjusted to obtain at least one set of test data, each set of test data including a stress value of the GPU and a value of at least one performance indicator of the GPU that have a corresponding relationship; based on at least one set of test data, a functional relationship between the stress value of the GPU and at least one performance indicator is established, and a first indicator determination model is generated based on the functional relationship.

[0082] There are two ways to construct the first indicator determination model in the embodiment of the present application. One is to establish it based on prior knowledge. The prior knowledge here can be understood as expert knowledge or database in this field, etc. According to these existing knowledge, a functional relationship between the pressure value of the GPU and at least one performance indicator is established. Another is to redefine the functional relationship based on the differences between different computer devices. There are also two possibilities for the computer device here. One is to test on a general computer device to obtain multiple groups of test data, and determine the functional relationship based on the test data. The functional relationship determined by the general computer device test has good versatility, that is, the functional relationship has good portability. Another is to test on a special computer device to obtain multiple groups of test data, and determine the functional relationship based on the test data. The second computer device here is a special computer device, and the special here can be understood as a specially designed computer device. For example, when providing terminal devices such as mobile phones to people with visual impairments, some special designs will be made to the terminal device, including the GPU in the terminal device. The terminal device after special design is different from the general computer device. The functional relationship determined by the test of special computer equipment can meet personalized needs in the subsequent process of determining the indicators that need to be optimized, that is, special equipment is treated specially instead of being generalized, which further improves the accuracy of subsequent code adjustments.

[0083] Of course, the present application does not limit the specific process of obtaining the functional relationship based on the test data. In some embodiments, discrete test data are known, and these data are smoothed by difference and other processing to obtain processed continuous data, and the functional relationship is obtained by fitting the continuous data. In other embodiments, the functional relationship between each performance indicator and the pressure value is preset, and the functional relationship is fine-tuned using the test data so that the fine-tuned functional relationship can meet actual needs. The test program here is also a program or code related to image processing, and the present application does not limit the specific content of the test program.

[0084] In some embodiments, the pressure value of the GPU decreases as the value of at least one performance indicator decreases, and when the pressure value of the GPU is the target pressure value, the value of the performance indicator is the performance bottleneck value corresponding to the performance indicator. In some embodiments, the value of the performance indicator is normalized, and for this functional relationship, when the value of the performance indicator tends to 0, the performance of the GPU is better, and when the value of the performance indicator tends to 1, the performance of the GPU is worse. In other embodiments, for the performance indicators, the above-mentioned functional relationship is established. And, for the functional relationship, when the pressure value is the target pressure value, the value of the performance indicator is the performance bottleneck value corresponding to the performance indicator. The target pressure value here can be considered as the performance that the GPU wants to achieve, and the numerical value of the target pressure value is not fixed and can be set by the developer. In some embodiments, when the value of the performance indicator is greater than the performance bottleneck value, it is considered that the performance indicator needs to be optimized, and when the value of the performance indicator is less than the performance bottleneck value, it is considered that the performance indicator can be not optimized.

[0085] In addition, the pressure value of the GPU in the embodiment of the present application can also be obtained through a specific performance counter, and the pressure value of the GPU is determined by collecting the value of the specific performance counter. In some embodiments, there is a GPU pressure counter, and the value of the GPU pressure counter represents the pressure value of the GPU.

[0086] In some embodiments, step 431 includes steps 431 - a to 431 - b (not shown).

[0087] Step 431 - a, for the i-th performance indicator among the at least one performance indicator, determine the model according to the first indicator, and determine the performance bottleneck value corresponding to the i-th performance indicator, where i is a positive integer.

[0088] In some embodiments, since the first indicator determines the functional relationship between the pressure value and the performance indicator of the GPU, when the pressure value of the GPU reaches the target pressure value, the value of the corresponding performance indicator is the performance bottleneck value of the performance indicator.

[0089] Step 431 - b, when the value of the ith performance indicator is greater than the performance bottleneck value corresponding to the ith performance indicator, determine that the ith performance indicator is the target performance indicator and the expected optimization trend of the target performance indicator is decreasing.

[0090] In some embodiments, when the value of the i-th performance indicator is less than the performance bottleneck value corresponding to the i-th performance indicator, the i-th performance indicator is determined to be the target performance indicator and the expected optimization trend of the target performance indicator is rising.

[0091] Of course, when the value of the i-th performance indicator is greater than the performance bottleneck value corresponding to the i-th performance indicator, the target performance indicator determined is the first performance indicator. When the value of the i-th performance indicator is less than the performance bottleneck value corresponding to the i-th performance indicator, the target performance indicator determined is the second performance indicator. The first performance indicator corresponds to the first priority, the second performance indicator corresponds to the second priority, and the first priority is higher than the second priority. That is, for these performance indicators, when the value of the performance indicator is greater than the performance bottleneck threshold, the performance indicator must be optimized, but this does not mean that the performance indicator whose value is less than the performance bottleneck threshold does not need to be optimized. In some embodiments, by optimizing the second performance indicator, the optimization result of the first performance indicator can be improved sideways. Therefore, the expected optimization trend of the first performance indicator is determined by the first priority, and the expected optimization trend of the second performance indicator is determined by the second priority. For, the number of the first performance indicator and the second performance indicator is not limited in this application. Exemplarily, when executing the subsequent determination of the adjustment trend of the adjustment parameter, it is also determined according to the expected optimization trend of each target performance indicator. Therefore, the target indicators are sorted according to the first priority and the second priority, that is, the first performance indicator is first, and the second performance indicator is later. The technical solution provided by the embodiment of the present application improves the adjustment efficiency of the business code by setting priorities for the target performance indicators, and at the same time speeds up the code adjustment process, making the adjustment mechanism of the business code more reasonable.

[0092] Table 1 below lists the corresponding relationship between performance indicators and their types.

[0093] Table 1 Correspondence between performance indicators and their types

[0094]

[0095]

[0096] In some embodiments, the corresponding relationship between the performance indicators and the types of the indicators is shown in Table 1. Among them, the indicators of the indicator type "ops" are considered to be performance indicators that can be optimized, and the indicators of the indicator type "stress" are considered to be indicators reflecting GPU pressure, which are used to determine the pressure value of the GPU. Among them, the counter that measures the processor pressure is called GPU Counter stress , a counter that measures the number of operations performed on the GPU Counter, called GPUCounter ops In some embodiments, taking global memory as an example: Here, the percentage of global memory peak bandwidth can be included in GPU Counter stress , and the number of global memory transactions is included in the GPU Counter opsIn some embodiments, the operation data in the embodiments of the present application includes first data and second data, wherein the first data refers to the value of the indicator of the above indicator type "ops", that is, the above GPU Counter ops The second data refers to the value of the indicator whose indicator type is "stress", that is, the GPU Counter stress In the above embodiment, when determining the target performance index, it is determined from the index of the index type "ops", that is, the performance index corresponding to the counter of the number of operations executed is optimized. The second data is used to calculate the pressure value of the GPU. The specific calculation method is not limited in this application, and is used to construct the above functional relationship.

[0097] In some embodiments, a bottleneck vector is determined, and the i-th element in the bottleneck vector represents the value of the i-th performance indicator after normalization. When the value of the i-th element in the bottleneck vector is greater than the performance bottleneck value corresponding to the i-th performance indicator, the i-th performance indicator is determined to be the target performance indicator and the expected optimization trend of the target performance indicator is decreasing. In some embodiments, the elements in the bottleneck vector refer to the values ​​obtained after the values ​​of the indicators of the indicator type "ops" are normalized.

[0098] In some embodiments, the bottleneck is represented by a vector B = [b DRAM_read , b DRAM_write , ...], a zero value for an element in the bottleneck vector means that the metric is not stressed, while a value indicates that the component is at its theoretical peak performance. In some embodiments, the elements in the bottleneck vector are divided into several categories, representing stress on various memory types, instructions, and GPU usage. In some embodiments, the memory subsystem bottleneck analysis is calculated in the same manner as global memory, shared memory, and L2 cache.

[0099] In some embodiments, the value of the performance indicator is not completely directly equal to the value of the performance counter, and the value of the performance indicator needs to be determined based on the value of the performance counter. Of course, for different performance indicators, the value of the performance indicator is determined based on the values ​​of different performance counters and different formulas. In some embodiments, the memory utilization reported by the GPU performance counter (DRAM_U) is scaled to <0,1> and weighted by the ratio of read and write transactions. Calculation of global memory read bottleneck: b DRAM_read =DRAM_R / (DRAM_RT+DRAM_WT)·DRAM_U / 10. Calculation of global memory write bottleneck: b DRAM_write=(DRAM_WT_DRAM RT+DRAM_WT)·DRAM_U / 10. For shared memory and L2 cache, the calculation is similar, using SHR_LT, SHR_WT, SHR_U and L2_RT, L2_WT, L2_U respectively. Exemplarily, when measuring the utilization of instructions using special GPU performance counters, when the counters are not very reliable or it is not fully understood what they measure (for example, the matrix transpose example that does not use floating point calculations reports high utilization of FP32 units in some configurations), the utilization is derived from the total number of instructions. First, use the value of the number of instructions (INST_EXE obtained in the API and must be multiplied by 32), corrected by the instruction execution efficiency. The resulting utilization: ins fitted =32·INST_EXE·100 / WARP_E·100 / WARP_NP_E. The lack of parallelism is calculated as: b paral =max(0, (cores·5-threads) / (cores·5)).

[0100] In some embodiments, the modulus of the bottleneck vector is determined; with the goal of minimizing the modulus of the bottleneck vector, at least one target performance indicator and the expected optimization trend of the target performance indicator are adjusted to obtain the adjusted target performance indicator and the expected optimization trend of the adjusted target performance indicator; wherein the expected optimization trend of the adjusted target performance indicator is used to determine the adjustment trend of the adjustable parameter. The modulus of the vector is also called the length or size of the vector. The calculation method of the modulus is described below. For the vector x=(x1, x2, x3, ..., x n ) belongs to the n-dimensional complex vector space, n is a positive integer, then the modulus of the vector x is

[0101] In some embodiments, the modulus of the bottleneck vector is determined according to the value of each element in the bottleneck vector. In the embodiment of the present application, when adjusting each performance index separately, it is also necessary to ensure that the change direction of the modulus of the bottleneck vector is towards 0, that is, the overall pressure of the GPU is reduced or the overall performance of the GPU is improved. In some embodiments, a least squares regression nonlinear model is used to achieve the minimum residual of the modulus of the overall bottleneck vector when adjusting the value of the target performance index. In some embodiments, the relationship between the adjustable parameters and the GPU Counter is modeled by a nonlinear regression model. All or part of the tuning space that has been evaluated and profiled is used-for each tuning configuration (that is, the target performance index and the expected optimization trend) as input, the GPU performance counter will be measured. The adjustment space is divided into subspaces determined by binary adjustment parameter values. For each subspace, data points are selected for training-for each non-binary parameter, two or three values ​​are selected to keep the total number of value combinations relatively low, but to make the sampling of the subspace more uniform. The model of each analysis counter (expected optimization trend) includes the main effects of each non-binary adjustment parameter, interactions (even high-order interactions) and the effects of secondary properties. In some embodiments, the least squares regression method is applied to calculate the coefficients so that the residual of the modulus value of the overall bottleneck vector is minimized. Finally, multiple models (expected optimization trends) are provided for each analysis counter, where the value of the binary adjustment parameter determines the applicability of the model. The optimization direction itself can be calculated using the value of the non-binary adjustment parameter.

[0102] Step 432, determining at least one target performance indicator and an expected optimization trend of the target performance indicator from at least one performance indicator based on the operating data through a second indicator determination model, wherein the second indicator determination model is a machine learning model.

[0103] In the embodiment of the present application, the input of the second indicator determination model is the operation data of the GPU, and the output is at least one target performance indicator and the expected optimization trend of the target performance indicator. The model architecture of the second indicator determination model is not limited in this application. For example, the second indicator determination model can be a coding layer, a feature fusion layer, a decoding layer, and a classification layer, wherein the encoding feature corresponding to the operation data is obtained through the coding layer, the fusion feature of the encoded feature is obtained through the feature fusion layer, and finally the decoding feature corresponding to the fusion feature is obtained based on the decoding layer, and finally the classification network is used to obtain the expected optimization trend of at least one target performance indicator and the target performance indicator. The training process of the second indicator determination model is as follows: determine multiple groups of training samples, a group of training samples includes sample operation data with corresponding relationships and at least one label target performance indicator and the expected optimization trend of the label target performance indicator. Input the sample operation data into the second indicator determination model to obtain at least one target performance indicator and the expected optimization trend of the target performance indicator. According to the difference between the expected optimization trend of at least one target performance indicator and the target performance indicator and the expected optimization trend of at least one label target performance indicator and the label target performance indicator, determine the loss function value, and train the second indicator determination model with the goal of minimizing the loss function value. In the embodiments of the present application, there is no limitation on the specific model architecture and model training process.

[0104] Step 440: Determine the adjustment trend of the adjustable parameters according to the expected optimization trend of the target performance indicator through the decision tree.

[0105] Step 450, adjusting the adjustable parameters according to the adjustment trend of the adjustable parameters to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0106] In the technical solution provided in the embodiment of the present application, the performance index to be optimized is determined based on the operation data. Specifically, a function model or a machine learning model is combined here to realize the deduction of the performance index to be optimized from the operation data, thereby improving the accuracy of the determined target performance index and the expected optimization trend of the target performance index.

[0107] Furthermore, when determining whether a performance indicator needs to be optimized based on its value, the performance indicator that needs to be optimized is directly screened out based on the predetermined performance bottleneck value, thereby improving the efficiency of determining the target performance indicator and the expected optimization trend of the target performance indicator.

[0108] At the same time, in addition to considering the value of a single performance indicator, this application also considers the overall value of the bottleneck vector to avoid the problem of inaccurate optimization caused by local optimality, speed up the adjustment speed, and improve the accuracy of business code adjustment.

[0109] Below through Figure 5 The determination and search process of the decision tree is illustrated by way of example.

[0110] Please refer to Figure 5 , which shows a flow chart of an image processing method provided by another embodiment of the present application. The execution subject of each step of the method may be Figure 1 The terminal device 10 or server 20 in the implementation environment of the solution shown. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as "computer device". The method may include at least one of the following steps (510-570):

[0111] Step 510: Obtain a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter.

[0112] Step 520, running the business code through the graphics processor GPU to obtain the GPU operation data, the operation data is used to reflect the operation status of the GPU.

[0113] Step 530: Determine at least one target performance indicator and an expected optimization trend of the target performance indicator from at least one performance indicator according to the operation data, wherein the target performance indicator is a performance indicator that needs to be optimized.

[0114] Step 540: determine the leaf node corresponding to the expected optimization trend of the target performance indicator from the decision tree.

[0115] In some embodiments, the decision tree has an adjustable parameter as a root node, an adjustment trend of the adjustable parameter as a child node, and a decrease in the performance index as a leaf node. In other embodiments, the decision tree has an adjustable parameter as a root node, an adjustment trend of the adjustable parameter as a child node, and an increase in the performance index as a leaf node. The embodiments of the present application do not limit the specific architecture of the decision tree.

[0116] In some embodiments, each adjustment parameter corresponds to a decision tree. When there are n adjustment parameters, there are n corresponding decision trees, where n is a positive integer.

[0117] In some embodiments, Figure 6 A schematic diagram of a decision tree is shown. Figure 6 In 600, the decision tree has the adjustable parameter as the root node, the adjustment trend of the adjustable parameter as the child node, and the decrease of the performance indicator as the leaf node. Operation 1 can be considered as increasing the adjustable parameter, and operation 2 can be considered as decreasing the adjustable parameter.

[0118] In some embodiments, when the adjustable parameter is known, the decision tree corresponding to the known adjustment parameter is directly found in the adjustment space, and the leaf node corresponding to the expected optimization trend of the target performance indicator is determined from the decision tree. In some embodiments, when it is uncertain which adjustable parameter needs to be adjusted, the leaf nodes of all decision trees are traversed to find the leaf node corresponding to the expected optimization trend of the target performance indicator. For example, Figure 6 As shown, the expected optimization trend of the target performance index is that the performance index 1 decreases, then the traversal Figure 6 The leaf nodes in the decision tree are found, and the first leaf node from left to right is the leaf node corresponding to the expected optimization trend of the target performance indicator.

[0119] In other embodiments, when adjusting the adjustable parameters in the business code, it can be specified whether to adjust them one by one or in a disorderly order. If it is adjusted one by one, it is considered that the adjustable parameters are known. After executing the above steps, the decision tree corresponding to the known adjustment parameters is found, and the leaf node corresponding to the expected optimization trend of the target performance indicator is determined from the decision tree. If it is a disorderly adjustment, after executing the above steps, the leaf nodes of all decision trees are traversed to find the leaf node corresponding to the expected optimization trend of the target performance indicator.

[0120] The following is an exemplary description of the method for determining a decision tree.

[0121] In some embodiments, at least one group of training samples for constructing a decision tree is obtained, each group of training samples includes values ​​of adjustable parameters and the value of at least one performance indicator that have a corresponding relationship, and each adjustment parameter corresponds to a decision tree; based on at least one group of training samples, information entropy corresponding to at least one performance indicator is determined; N performance indicators satisfied by the information entropy are determined from at least one performance indicator, where N is a positive integer; based on the N performance indicators, N leaf nodes are determined, each leaf node corresponds to a decrease in a performance indicator.

[0122] In some embodiments, the information entropy corresponding to each performance indicator is calculated from the training samples according to the number of occurrences of the performance indicator in the training samples, and the specific information entropy calculation process is not described here. In some embodiments, the above information entropy meeting the conditions can be considered as the N performance indicators with the largest information entropy.

[0123] The embodiment of the present application does not limit the method for constructing a decision tree. In addition to the above method, other greedy algorithms can also be used.

[0124] In some embodiments, at least one set of candidate trees may also be generated based on at least one set of training samples. These trees are trained using a randomly selected 50% of the exploration adjustment space, and the rest is used for testing. In some embodiments, the root node of the tree is also changed. The mean absolute error and root mean square error of these trees are calculated, and the tree with the lowest mean absolute error is selected as the final decision tree.

[0125] Step 550, based on the leaf node, determine the path from the leaf node to the root node from the decision tree.

[0126] In some embodiments, Figure 6 As shown, after determining that the first leaf node from left to right is the leaf node corresponding to the expected optimization trend of the determined target performance indicator, find the path from the leaf node to the root node, that is, the path from performance indicator 1 to operation 1 to the adjustable parameter.

[0127] Step 560: Determine the adjustment trend of the adjustable parameter according to the path.

[0128] In some embodiments, Figure 6 As shown, after determining the path from the performance indicator 1 dropping to the operation 1 to the adjustable parameter, it can be known that the adjustable parameter is executed 1, that is, the adjustable parameter is increased.

[0129] Step 570: Adjust the adjustable parameters according to the adjustment trend of the adjustable parameters to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0130] It should be noted that in addition to adjusting the adjustable parameters in the business code, the hardware configuration can also be adjusted in the embodiment of the present application, that is, the combination of software and hardware is achieved.

[0131] In some embodiments, when configuring the hardware, the steps can refer to the above embodiments. When the expected optimization trend of the target performance indicator is obtained, after the adjustment space of the software configuration (the decision tree with the adjustable parameter as the root node) still cannot be adjusted to the GPU operation data that meets the user's expectations, the hardware configuration is further adjusted. The hardware configuration here can be understood as what type of GPU is used, what size of kernel is used, etc.

[0132] In some embodiments, the process of configuring the hardware according to the adjustment space of the hardware configuration can refer to the process of configuring according to the configuration space of the software. The difference is that in the hardware configuration space, the hardware parameters are the root nodes, the adjustment of the hardware parameters is the child node (decision node), and the change trend of the performance indicator is the leaf node. The specific search and adjustment process and the decision tree construction process refer to the above embodiment and will not be repeated here.

[0133] The technical solution provided in the embodiment of the present application introduces a decision tree to simulate the corresponding relationship between adjustable parameters and performance indicators, so that when the expected optimization trend of the performance indicators is known, the adjustment trend of the adjustable parameters can be predicted. Therefore, the embodiment of the present application improves the adjustment efficiency of the business code by introducing a decision tree. In addition, in the embodiment of the present application, when constructing a decision tree, the leaf nodes are determined according to the size of the information entropy of the performance indicators, which improves the accuracy of the decision tree construction and is conducive to the smooth progress of downstream tasks.

[0134] Please refer to Figure 7 , which shows a flow chart of an image processing method provided by another embodiment of the present application. The execution subject of each step of the method may be Figure 1 The terminal device 10 or server 20 in the implementation environment of the solution shown. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced and explained as "computer device".

[0135] In some embodiments, the first part is the process of training the function model. Exemplarily, the function model is a problem assertion model (which can also be understood as the above-mentioned first indicator determination model). This part automatically reverses the shader (shading program), prepares a large amount of rendering graphics data in advance, reversely obtains the rendering program (that is, the business code for executing the image processing business), and then establishes the function model based on the GPU's operating data. The model combines adjustable parameters, GPUCounter relationships, and Mesh and vertex relationships to generate assertions. When the GPU Counter and Mesh and vertex data are all reduced, the performance is improved. Otherwise, the relationship between the problem and the GPU Counter and the adjustable parameters is determined and put into the model. The second part uses a real game program to try to optimize, and combines performance bottleneck analysis to find out the GPU performance bottleneck. Combined with the problem assertion model, a performance score (the value of each performance indicator after normalization) is given and asserted. Next, the configuration is modified according to the decision tree intelligent search. In some embodiments, the configuration modification includes software and hardware, such as the number of GPU cores, data such as texture rendering floating point or transparency that affects the GPU Counter, and after reversing, re-shading and rendering.

[0136] Specifically, when GPU rendering is performed, there is a strong correlation between Mesh grids and vertices. For example, a triangle mesh is a Mesh with three vertices, but there are also vertices that are shared. At this time, a functional relationship is established based on experience: Among them, the complexity of mesh vertex i is V i Indicates that n is the total number of vertices, w j is the jth mesh. w corresponds to the weight, r ijis the Euclidean distance between vertex i and vertex j, and α0 is a constant.

[0137] In the business code that implements this GPU rendering business, gridSpacing is the size of the grid cells, gridSize is the number of grid cells per dimension, and energyGrid contains the output grid. A GPU thread computes one or more grid points, determined by the value of ITERACOUNT, which is the only tuning parameter in this example. For each grid point, the complexity is calculated by the above formula. This code is expected to be compute-bound for two reasons: (i) for each grid's data loaded into registers, multiple grid point values ​​can be calculated, and (ii) all threads in a warp are working simultaneously, so the code can benefit from good cache locality. In some embodiments, a higher ITERACOUNT value improves data locality in registers: once vertex coordinates and meshes are read, they are used multiple times in the loop started in the business code. In addition, this parameter reduces the computation of varyings. However, too high an ITERACOUNT value reduces strong scaling (kernels are executed by a smaller number of threads) and increases register consumption, which can result in lower GPU occupancy or register spilling using slower local memory. Knowing and understanding how tuning parameters relate to GPU performance counters is part of the developer's expertise.

[0138] When analyzing the code, business developers can set the values ​​of tuning parameters by observing the hardware GPU performance counters. In this example, several counters are highly correlated: (1) Floating point unit utilization: If high, ITERACOUNT (number of iterations) should be set to a higher value (to reduce the number of floating point operations by reducing the number of times the derivative rendering is calculated); (2) Texture or L2 cache utilization: If high (and not caused by register overflow), ITERACOUNT should be set to a higher value (to improve cache locality); (3) GPU occupancy: If low, ITERACOUNT should be set to a lower value (to improve strong scaling); (4) Local memory: If high, ITERACOUNT should be set to a lower value in case the utilization of any part of the memory subsystem is also high (to reduce the bandwidth introduced by register overflow). Knowing and understanding how GPU performance counters are related to bottlenecks is also a body of knowledge that developers need to use professionally, and by joining these systems, simulation experts can make performance assertions. In the simple example of image rendering above, only a one-dimensional tuning space is searched. However, the rules mentioned here also apply to multiple tuning parameters: in this case, the resulting kernel runtime and GPU performance counter values ​​are determined by the blended effects of all tuning parameters, but ITERACOUNT still affects the GPU performance counters in the same way. The kernel was tested on a GeForce GTX 1060 with a mesh of 256×256×256 points and 64 meshes. The kernel is supposed to be compute-bound; therefore, high utilization of the floating-point units is expected when tuning. When profiling with ITERACOUNT set to 1, the kernel ran in 10,475 μs. The GPU performance counters pointed out a clear bottleneck in the texture cache (9th usage level out of 10), while floating-point instruction unit load was not high (3rd usage level out of 10). Therefore, the value of ITERACOUNT was increased to 32 to improve register locality. This resulted in a significant performance improvement: the kernel executed in 1,531 μs. The GPU performance counters showed the following: texture cache utilization dropped from 9 to 3, floating-point utilization was 8, and the reported GPU occupancy was 0.6 (out of 1.0). Alternatively, one could try increasing the occupancy to reduce pipeline and memory latency. Setting ITERACOUNT to 8 gives a slightly better runtime of 1,497μs. The GPU occupancy is now 0.98, floating point utilization is 0.9, and texture cache utilization is 9. Texture cache utilization can be reduced by setting ITERACOUNT to 16; however, this change does not result in performance improvements; therefore, one could consider tuning the kernel. Now, consider a change in the input data: using the same GPU, a grid of 25×25×25 points and 4096 grids will be calculated. The previously tuned implementation sets ITERACOUNT to 8, yielding a runtime of 394μs.This implementation has an occupancy of 0.11. Some speedup can be achieved by increasing the occupancy, so values ​​lower than 8 need to be explored for ITERACOUNT. With a setting of 4, the runtime improves to 260μs. With the occupancy increased to 0.17, a bottleneck on the texture cache (utilization level 8) can be seen, indicating that ITERACOUNT should not be reduced further. When attempting to set ITERACOUNT to 2, the runtime increases to 428μs. Despite the occupancy increasing to 0.32, the implementation is limited by the texture cache, which now reports a utilization level of 9. Therefore, the tuning is considered complete and 4 is considered to be the optimal value for ITERACOUNT. Finally, the business code is tuned based on the tuned data.

[0139] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0140] Please refer to Figure 8 , which shows a block diagram of an image processing device provided by an embodiment of the present application. Figure 8 As shown, the device 800 may include: a code acquisition module 810 , a data acquisition module 820 , a trend determination module 830 and a parameter adjustment module 840 .

[0141] The code acquisition module 810 is used to acquire a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter.

[0142] The data acquisition module 820 is used to run the business code through the graphics processing unit GPU to obtain the operation data of the GPU, and the operation data is used to reflect the operation status of the GPU.

[0143] The trend determination module 830 is used to determine the adjustment trend of the adjustable parameter according to the operating data through a decision tree; wherein the decision tree is used to characterize the corresponding relationship between the adjustment trend of the adjustable parameter and the operating status of the GPU.

[0144] The parameter adjustment module 840 adjusts the adjustable parameter according to the adjustment trend of the adjustable parameter to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

[0145] In some embodiments, the operating data of the GPU includes a value of at least one performance indicator of the GPU, where the performance indicator is an indicator reflecting the operating status of the GPU.

[0146] In some embodiments, Fig. 9As shown, the trend determination module 830 includes an indicator determination unit 831 and a trend determination unit 832 .

[0147] The indicator determination unit 831 is used to determine at least one target performance indicator and an expected optimization trend of the target performance indicator from the at least one performance indicator according to the operation data, and the target performance indicator is a performance indicator that needs to be optimized.

[0148] The trend determination unit 832 is used to determine the adjustment trend of the adjustable parameter according to the expected optimization trend of the target performance indicator through the decision tree.

[0149] In some embodiments, the indicator determination unit 831 is used to determine the at least one target performance indicator and the expected optimization trend of the target performance indicator from the at least one performance indicator according to a first indicator determination model and the operating data, wherein the first indicator determination model is a function model, and the function model reflects the functional relationship between the pressure value of the GPU and the at least one performance indicator, and the functional relationship characterizes that the pressure value of the GPU changes with the change of the at least one performance indicator; or, determine the at least one target performance indicator and the expected optimization trend of the target performance indicator from the at least one performance indicator according to the operating data through a second indicator determination model, and the second indicator determination model is a machine learning model.

[0150] In some embodiments, the indicator determination unit 831 is used to determine, for the i-th performance indicator among the at least one performance indicator, a performance bottleneck value corresponding to the i-th performance indicator according to the first indicator determination model, where i is a positive integer; when the value of the i-th performance indicator is greater than the performance bottleneck value corresponding to the i-th performance indicator, determine that the i-th performance indicator is the target performance indicator and the expected optimization trend of the target performance indicator is decreasing.

[0151] In some embodiments, the indicator determination unit 831 is further used to determine a bottleneck vector, wherein the i-th element in the bottleneck vector represents the value of the i-th performance indicator after normalization.

[0152] In some embodiments, the indicator determination unit 831 is also used to determine that the i-th performance indicator is the target performance indicator and the expected optimization trend of the target performance indicator is decreasing when the value of the i-th element in the bottleneck vector is greater than the performance bottleneck value corresponding to the i-th performance indicator.

[0153] In some embodiments, the trend determination unit 832 is also used to determine the modulus of the bottleneck vector; with the goal of minimizing the modulus of the bottleneck vector, the at least one target performance indicator and the expected optimization trend of the target performance indicator are adjusted to obtain an adjusted target performance indicator and the expected optimization trend of the adjusted target performance indicator; wherein the expected optimization trend of the adjusted target performance indicator is used to determine the adjustment trend of the adjustable parameter.

[0154] In some embodiments, the indicator determination unit 831 is further used to establish a functional relationship between the pressure value of the GPU and the at least one performance indicator based on prior knowledge, and generate the first indicator determination model based on the functional relationship; or, run a test program, adjust the adjustable parameters in the test program, and obtain at least one set of test data, each set of test data including a corresponding pressure value of the GPU and a value of at least one performance indicator of the GPU; establish a functional relationship between the pressure value of the GPU and the at least one performance indicator based on the at least one set of test data, and generate the first indicator determination model based on the functional relationship.

[0155] In some embodiments, in the decision tree, the adjustable parameter is a root node, the adjustment trend of the adjustable parameter is a child node, and the decline of the performance indicator is a leaf node.

[0156] In some embodiments, the trend determination unit 832 is used to determine the leaf node corresponding to the expected optimization trend of the target performance indicator from the decision tree; based on the leaf node, determine the path from the leaf node to the root node from the decision tree; and determine the adjustment trend of the adjustable parameter based on the path.

[0157] In some embodiments, the trend determination unit 832 is further used to obtain at least one group of training samples for constructing the decision tree, each group of training samples includes the values ​​of the adjustable parameters and the value of at least one performance indicator that have a corresponding relationship, and each adjustment parameter corresponds to a decision tree; based on the at least one group of training samples, determine the information entropy corresponding to the at least one performance indicator; determine N performance indicators whose information entropy meets the conditions from the at least one performance indicator, where N is a positive integer; based on the N performance indicators, determine N leaf nodes, each leaf node corresponds to a decrease in a performance indicator.

[0158] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0159] Please refer to Fig.10 , which shows a structural block diagram of a computer device 1000 provided in one embodiment of the present application. The computer device 1000 may be any electronic device with data calculation, processing and storage capabilities. The computer device 1000 may be used to implement the above-mentioned image processing method.

[0160] Typically, the computer device 1000 includes a processor 1001 and a memory 1002 .

[0161] Processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. Processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, processor 1001 may be integrated with a GPU, which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, processor 1001 may also include an AI processor for processing computing operations related to machine learning.

[0162] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned image processing method.

[0163] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the computer device 1000, and the computer device 1000 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0164] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned image processing method is implemented. Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random Access Memory), SSD (Solid State Drives) or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0165] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned image processing method.

[0166] It should be noted that the collection and processing of relevant data (including business codes, image processing services, etc.) in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0167] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0168] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter; Running the business code through a graphics processor (GPU) to obtain operation data of the GPU, where the operation data is used to reflect the operation status of the GPU; Determining the adjustment trend of the adjustable parameter according to the operating data through a decision tree; wherein the decision tree is used to characterize the corresponding relationship between the adjustment trend of the adjustable parameter and the operating status of the GPU; According to the adjustment trend of the adjustable parameter, the adjustable parameter is adjusted to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

2. The method according to claim 1, characterized in that The operation data of the GPU includes a value of at least one performance indicator of the GPU, where the performance indicator is an indicator reflecting the operation status of the GPU; Determining the adjustment trend of the adjustable parameter according to the operation data through a decision tree includes: Determining at least one target performance indicator and an expected optimization trend of the target performance indicator from the at least one performance indicator according to the operation data, wherein the target performance indicator is a performance indicator that needs to be optimized; The adjustment trend of the adjustable parameter is determined through the decision tree according to the expected optimization trend of the target performance indicator.

3. The method according to claim 2, characterized in that Determining at least one target performance indicator and an expected optimization trend of the target performance indicator from the at least one performance indicator according to the operating data includes: Determine, according to a first indicator determination model and the operating data, the at least one target performance indicator and an expected optimization trend of the target performance indicator from the at least one performance indicator, wherein the first indicator determination model is a function model, the function model reflects a function relationship between the pressure value of the GPU and the at least one performance indicator, and the function relationship indicates that the pressure value of the GPU changes with the change of the at least one performance indicator; or, The at least one target performance indicator and the expected optimization trend of the target performance indicator are determined from the at least one performance indicator according to the operating data through a second indicator determination model, and the second indicator determination model is a machine learning model.

4. The method according to claim 3, characterized in that The step of determining the model and the operating data according to the first indicator, and determining the at least one target performance indicator and the expected optimization trend of the target performance indicator from the at least one performance indicator, comprises: For an i-th performance indicator among the at least one performance indicator, determining a model according to the first indicator, and determining a performance bottleneck value corresponding to the i-th performance indicator, where i is a positive integer; When the value of the i-th performance indicator is greater than the performance bottleneck value corresponding to the i-th performance indicator, it is determined that the i-th performance indicator is the target performance indicator and the expected optimization trend of the target performance indicator is decreasing.

5. The method according to claim 4, characterized in that The method further comprises: Determine a bottleneck vector, wherein the i-th element in the bottleneck vector represents the value of the i-th performance indicator after normalization; When the value of the i-th performance indicator is greater than the performance bottleneck value corresponding to the i-th performance indicator, determining that the i-th performance indicator is the target performance indicator and the expected optimization trend of the target performance indicator is decreasing includes: When the value of the i-th element in the bottleneck vector is greater than the performance bottleneck value corresponding to the i-th performance indicator, the i-th performance indicator is determined to be the target performance indicator and the expected optimization trend of the target performance indicator is decreasing.

6. The method according to claim 5, characterized in that The method further comprises: determining a modulus value of the bottleneck vector; With the goal of minimizing the modulus of the bottleneck vector, adjusting the at least one target performance indicator and the expected optimization trend of the target performance indicator to obtain an adjusted target performance indicator and the expected optimization trend of the adjusted target performance indicator; The expected optimization trend of the adjusted target performance indicator is used to determine the adjustment trend of the adjustable parameter.

7. The method according to claim 3, characterized in that The method further comprises: According to prior knowledge, a functional relationship between the pressure value of the GPU and the at least one performance indicator is established, and the first indicator determination model is generated based on the functional relationship; or, Run a test program, adjust adjustable parameters in the test program, and obtain at least one set of test data, each set of test data including a pressure value of the GPU and a value of at least one performance indicator of the GPU that have a corresponding relationship; establish a functional relationship between the pressure value of the GPU and the at least one performance indicator based on the at least one set of test data, and generate the first indicator determination model based on the functional relationship.

8. The method according to claim 2, characterized in that: In the decision tree, the adjustable parameter is a root node, the adjustment trend of the adjustable parameter is a child node, and the decrease of the performance indicator is a leaf node; Determining the adjustment trend of the adjustable parameter according to the expected optimization trend of the target performance indicator through the decision tree includes: Determine, from the decision tree, a leaf node corresponding to the expected optimization trend of the target performance indicator; According to the leaf node, determining a path from the leaf node to the root node from the decision tree; An adjustment trend of the adjustable parameter is determined according to the path.

9. The method according to claim 8, characterized in that The method further comprises: Acquire at least one group of training samples for constructing the decision tree, each group of training samples includes values ​​of the adjustable parameters and at least one value of the performance indicator that have a corresponding relationship, and each adjustment parameter corresponds to a decision tree; Determining, according to the at least one group of training samples, information entropy corresponding to the at least one performance indicator; Determine N performance indicators whose information entropy satisfies a condition from the at least one performance indicator, where N is a positive integer; According to the N performance indicators, N leaf nodes are determined, each leaf node corresponding to a decrease in a performance indicator.

10. An image processing device, characterized in that: The device comprises: A code acquisition module, used to acquire a service code for executing an image processing service, wherein the service code includes at least one adjustable parameter; A data acquisition module, used to run the business code through the GPU to obtain the operation data of the GPU, wherein the operation data is used to reflect the operation status of the GPU; A trend determination module, configured to determine an adjustment trend of the adjustable parameter according to the operation data through a decision tree; wherein the decision tree is used to characterize a corresponding relationship between the adjustment trend of the adjustable parameter and the operation status of the GPU; The parameter adjustment module adjusts the adjustable parameter according to the adjustment trend of the adjustable parameter to obtain an adjusted service code, and the adjusted service code is used to execute the image processing service.

11. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the image processing method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program, which is loaded and executed by a processor to implement the image processing method according to any one of claims 1 to 9.

Images