Program processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a program processing method and device, electronic equipment and a readable storage medium, and the method comprises the steps: exporting calling information of calling functions of all specified types in a target program in sequence, and generating a target image; each pixel point in the target image represents a calling function in the target program, and the color of the pixel point represents the function category of the calling function; identifying the arrangement of pixel points in the target image to obtain an identification result; the identification result comprises a target area in the target image and iteration information of an iteration execution structure corresponding to the target area; pixel points in the target area correspond to a repeated call function sequence obtained after iterative execution structure expansion; the iteration information comprises the number of iterations of the iteration execution structure; and generating a code representation of the iterative execution structure according to the identification result. The compiling efficiency and the loop recognition efficiency can be improved, and the efficiency and accuracy of hardware testing based on the target program are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a program processing method, device, electronic device, and readable storage medium. Background Art

[0002] In the field of GPU hardware testing using OpenGL (Open Graphics Library) programs, to accurately evaluate the performance of hardware like GPUs, it's necessary to insert specific test functions into the program to track data. However, compiled executable programs can't be directly modified to insert these test functions. Therefore, it's necessary to reverse engineer the OpenGL program to generate a compilable file (such as a C++ file).

[0003] In the compilable file generated by reverse engineering, test functions can be inserted and then recompiled to generate an executable OpenGL program. However, all loop structures in the compiled file are expanded, converting code snippets that were originally executed repeatedly through loop statements into large amounts of sequentially repeated code. This significantly increases the file size. During the compilation process, the compiler needs to process a large number of lines of code, significantly increasing compilation time and affecting compilation efficiency. Summary of the Invention

[0004] In view of the above problems, an embodiment of the present invention is proposed to provide a program processing method that overcomes the above problems or at least partially solves the above problems, which can shorten the compilation time, improve the compilation efficiency, and improve the loop recognition efficiency, thereby improving the efficiency and accuracy of hardware testing based on the target program.

[0005] Correspondingly, an embodiment of the present invention further provides a program processing device, an electronic device, and a computer program product to ensure the implementation and application of the above method.

[0006] In a first aspect, an embodiment of the present invention discloses a program processing method, the method comprising:

[0007] Exporting call information of all specified types of call functions in the target program in sequence, and encoding the call functions to generate a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function;

[0008] Identifying an arrangement of pixels in the target image to obtain a recognition result; the recognition result includes a target area in the target image and iteration information of an iterative execution structure corresponding to the target area; the pixels in the target area correspond to a repeatedly called function sequence obtained after the iterative execution structure is expanded; the iteration information includes the number of iterations of the iterative execution structure;

[0009] A code representation of the iterative execution structure is generated according to the recognition result.

[0010] In a second aspect, an embodiment of the present invention discloses a program processing device, the device comprising:

[0011] An export generation module is used to sequentially export the call information of all specified types of call functions in the target program through a preset tool, and encode the call functions to generate a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function;

[0012] a model recognition module, configured to input the target image into a pre-trained recognition model to obtain a recognition result; the recognition result includes a target area in the target image and iteration information of an iterative execution structure corresponding to the target area; pixels in the target area correspond to a sequence of repeatedly called functions obtained by expanding the iterative execution structure; the iteration information includes the number of iterations of the iterative execution structure;

[0013] A code conversion module is used to generate a code representation of the iterative execution structure according to the recognition result.

[0014] In a third aspect, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of any of the program processing methods described above.

[0015] In a fourth aspect, an embodiment of the present invention discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it can implement any of the program processing methods described above.

[0016] In a fifth aspect, an embodiment of the present invention discloses a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the program processing methods described above are performed.

[0017] The embodiments of the present invention include the following advantages:

[0018] The embodiment of the present invention uses a preset tool to convert the calling functions appearing in the compilable file of the target program into pixel points on the target image in the order of calling, converts the recognition problem of the iterative execution structure into an image recognition problem, and automatically recognizes the iterative execution structure in the target program using a pre-trained preset recognition model, thereby improving recognition efficiency and improving the efficiency and accuracy of hardware testing based on the target program. Through the embodiment of the present invention, there is no need to rely on manual identification of repeated code and rewriting of the iterative execution structure, which can avoid the cumbersome and inefficient operations brought about by manual methods, and the problem that manual judgment is prone to errors, may miss some repeated code or introduce new errors, thereby affecting the efficiency and accuracy of hardware testing based on the target program.

[0019] In addition, through the embodiments of the present invention, after the target program is reversed to obtain a compilable file, the repeated function call sequence in the compilable file can be replaced with an iterative execution structure, which can greatly reduce the size of the compilable file, shorten the compilation time, and improve the compilation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flowchart of steps of an embodiment of a program processing method of the present invention;

[0021] Figure 2 is a structural block diagram of an embodiment of a program processing device of the present invention;

[0022] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after the association are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0025] Reference Figure 1 , shows a flowchart of a program processing method embodiment of the present invention, the method may include the following steps:

[0026] Step 101: Export call information of all specified types of call functions in a target program in sequence, and encode the call functions to generate a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function;

[0027] Step 102: Identify the arrangement of pixels in the target image to obtain a recognition result; the recognition result includes a target area in the target image and iteration information of an iterative execution structure corresponding to the target area; the pixels in the target area correspond to a repeatedly called function sequence obtained by expanding the iterative execution structure; the iteration information includes the number of iterations of the iterative execution structure;

[0028] Step 103: Generate a code representation of the iterative execution structure according to the recognition result.

[0029] The program processing method provided by the embodiment of the present invention can be used to detect the iterative execution structure in the target program. The embodiment of the present invention does not limit the type of the target program. The target program can be a program written in various programming languages, such as C, C++, Java, Python, JavaScript, etc.; it can be a program for different application fields, including but not limited to operating system kernel programs, graphics rendering programs, program processing and analysis programs, network communication programs, game programs, hardware testing programs, etc.; it can be a program running on different platforms, such as desktop operating systems (Windows, MacOS, Linux, etc.), mobile operating systems (Android, iOS, etc.), embedded systems, and cloud computing platforms. Whether the target program is used for scientific computing, commercial applications, artificial intelligence, or any other field, this program processing method can be applicable. Exemplarily, the target program can be an OpenGL program for GPU hardware testing.

[0030] The iterative execution structure refers to a type of program structure or function execution mode that can decompose a complex task into multiple identical or similar simple operations and achieve the final task goal by repeatedly executing these simple operations.

[0031] For example, the iterative execution structure may include a loop structure, which, when executed, expands into multiple, consecutively repeated function calls, i.e., a repeated function call sequence. Such loop structures include, but are not limited to, for loops, while loops, and do-while loops. By setting a loop condition and a loop body, a loop structure repeatedly executes the code within the loop body when the condition is met, thereby efficiently processing repetitive tasks.

[0032] Example 1: Assume a loop structure as follows:

[0033]

[0034] The number of iterations of this loop structure is 10, and the called function is glDrawElements, that is, the loop structure will repeatedly call the function glDrawElements 10 times.

[0035] After the target program's executable file is reversed to generate a compilable file, the loop structure will be expanded into the following sequence of repeated function calls:

[0036]

[0037] As another example, the iterative execution structure may include a composite function, which, when executed, is decomposed into multiple sub-functions that are repeated continuously, i.e., a sequence of repeatedly called functions. The functionality of a composite function is composed of multiple simple functions, that is, a complex operation is decomposed into multiple repeated simple operations.

[0038] For example, the composite function may include the function glDrawElements. The function glDrawElements can be used to draw primitives based on an index array, and the function glDrawArrays can be used to draw primitives in the order of a vertex array. The function glDrawElements can be used as a composite function, and when executed, it can be split into multiple executions of the sub-function glDrawArrays. For another example, the composite function may include the function numpy.sum, which is used to calculate the sum of all elements in an array, and can be decomposed into an addition operation on each element in the array. It should be noted that the embodiment of the present invention does not limit the type of the composite function.

[0039] For the target program, in the compilable file of the target program generated by reverse deduction, the iterative execution structure will be expanded into a sequence of repeated function calls, resulting in a sharp increase in file size and affecting the efficiency of recompilation.

[0040] To address this issue, an embodiment of the present invention uses a pre-configured tool to convert the calling functions in the target program's compilable file into pixels on a target image in the order they are called. Each pixel in the target image represents a calling function in the target program, and the color of the pixel represents the function category of the calling function.

[0041] The embodiments of the present invention do not limit the type of the preset tool. The preset tool can be a debugging library or tool set provided by various programming languages; a debugging plug-in or function provided by a professional integrated development environment (IDE); an independent third-party debugging and analysis tool; or a customized script or tool developed for a specific programming language or application scenario. The preset tool can be used to track the running process of the target program and sequentially export call information of all specified types of call functions in the target program.

[0042] The calling function of the specified type may be a standard function provided by OpenGL (usually beginning with gl). OpenGL is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics.

[0043] It should be noted that the embodiments of the present invention do not impose any restrictions on the specified type of calling functions. The specified type of calling functions can be various graphics libraries, operating system APIs, third-party development libraries, and functions with specific functions and calling rules in specific domain frameworks.

[0044] In the embodiment of the present invention, the target program is an OpenGL program, and the specified type of calling function is a function starting with gl (hereinafter referred to as gl function) as an example for description. The operation processes of other types of target programs and calling functions are similar and can be referenced to each other.

[0045] This embodiment of the present invention converts all specified types of call functions (e.g., gl functions) in a target program into pixels in a target image. Each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function type of the call function. For example, color 1 represents the call function glClear, color 2 represents the call function glEnable, color 3 represents the call function glDisable, and so on.

[0046] In an optional embodiment of the present invention, each of all the calling functions of the specified type corresponds to respective number information; the method may further include:

[0047] For each calling function of the specified type in the target program, the respective numbering information is multiplied by a preset value in sequence, and then encoded as a hexadecimal number based on the function category corresponding to the calling function as a pixel point in the target image; the first designated bit of the hexadecimal number represents red, the second designated bit represents green, and the third designated bit represents blue; the result obtained by multiplying the numbering information of each calling function by the preset value is less than or equal to the maximum number of colors supported by the color encoding format of the target image.

[0048] For example, OpenGL provides 339 gl functions. These 339 gl functions are numbered in sequence to obtain the number information of each gl function (recorded as index). For example, the indexes of these 339 gl functions are 0 to 338 respectively.

[0049] After exporting the call information of all GL functions called in the target program in sequence using the pre-built tool, assume that the call information of 10 GL functions is exported, assuming that the 10 glDrawElements functions are called as shown in Example 1. In the order of calling, each exported function call is encoded as a hexadecimal number based on its number and function category, and represented as a pixel in the target image.

[0050] Specifically, for a certain derived calling function, after multiplying its number information by a preset value, it is encoded into a hexadecimal number based on the function category corresponding to the calling function and serves as a pixel point in the target image.

[0051] The purpose of multiplying the number information by a preset value is to avoid data redundancy caused by excessive similarity between adjacent pixels. After multiplying by the preset value and then converting to hexadecimal, the differences between pixels are amplified, thereby reducing the similarity between adjacent data and reducing data redundancy.

[0052] The present embodiment does not restrict the value of the preset value, which is used to control the color distribution density. For example, the preset value can be 32, 50, 100, etc., as long as the result of multiplying the number of each calling function by the preset value is less than or equal to the maximum number of colors supported by the color encoding format of the target image.

[0053] The embodiment of the present invention does not limit the color encoding format of the target image. For example, it can be an RGB565 color encoding format. RGB565 is a 16-bit color encoding format that uses 16 binary bits to represent the color of a pixel. These 16 bits are allocated to the three basic color channels: red (R), green (G), and blue (B). For example, if the color of a pixel is (0, 15, 20), it means that the red channel value is 0, the green channel value is 15, and the blue channel value is 20. The color of the pixel is displayed as a dark blue-green.

[0054] In the RGB565 color encoding format, the red channel uses 5 bits, the green channel uses 6 bits, and the blue channel uses 5 bits. Therefore, if the RGB565 color encoding format is used, the first designated bit of the hexadecimal number obtained after encoding (such as the upper 5 bits of the binary number) represents the red channel, the second designated bit (such as the middle 6 bits of the binary number) represents the green channel, and the third designated bit (such as the lower 5 bits of the binary number) represents the blue channel.

[0055] Assume that the exported encoding information for a function call is index = 10 and the preset value is 50. Calculating 10 × 50 = 500 converts this to the following hexadecimal number: 0x01F4 (binary: 0000000111110100). The upper 5 bits (00000) of the hexadecimal number 0x01F4 indicate a red channel value of 0, the middle 6 bits (001111) indicate a green channel value of 15, and the lower 5 bits (10100) indicate a blue channel value of 20. The pixel color corresponding to the hexadecimal number 0x01F4 is (0, 15, 20), and the function category corresponding to these colors is the glDrawElements function.

[0056] The RGB565 color encoding format can represent 65536 colors. Therefore, the result of multiplying the number information of each function call by the preset value should be less than or equal to 65536. For example, when the encoding information ranges from 0 to 338, the maximum allowable preset value is 65535 ÷ 338 ≈ 194.

[0057] It should be noted that the above RGB565 color encoding format is only used as an example. In specific implementations, any color encoding format such as RGB888 (24 bits) and RGB32 (32 bits) may also be used.

[0058] After converting the calling functions appearing in the target program into pixel points on the target image in sequence, the target image is input into a trained preset recognition model to obtain a recognition result; the recognition result includes the target area in the target image and the iteration information of the iterative execution structure corresponding to the target area; the pixel points in the target area correspond to the repeated calling function sequence obtained after the iterative execution structure is expanded; the iteration information includes the number of iterations of the iterative execution structure.

[0059] The embodiment of the present invention pre-trains a preset recognition model. The preset recognition model can identify the target area in the target image, and the target area refers to an area containing a continuous arrangement of pixels with consistent colors. Here, color consistency refers to the same color or the similarity meets a preset threshold. Pixels with consistent colors represent repeated function calls. The continuous repeated arrangement of pixels with consistent colors represents continuous repeated function calls, or is called a repeated function call sequence. The repeated function call sequence is usually the result of the expansion of the iterative execution structure. Therefore, if the repeated function call sequence is identified, it is considered that the iterative execution structure exists in the source code of the target program. That is, the embodiment of the present invention converts the recognition problem of the iterative execution structure into an image recognition problem.

[0060] Taking Example 1 as an example, the call information of all specified types of calling functions (such as all gl functions called) in the target program are exported in sequence through the preset tool. Since the target program will repeatedly call the function glDrawElements 10 times, the call information of 10 calling functions glDrawElements will be exported, and each exported calling function glDrawElements will be encoded and converted into a pixel point in the target image. It should be noted that in a specific implementation, the target program may include one or more iterative execution structures, and each iterative execution structure will be expanded into a corresponding repeated calling function sequence. For each calling function in the repeated calling function sequence obtained after each iterative execution structure is expanded, the preset tool can export its calling information in the order of calling. The calling information includes but is not limited to the function name, parameters, return value and other information of each call.

[0061] In Example 1, each call to the glDrawElements function generates a pixel on the target image. Since these 10 calls to the glDrawElements function are continuous and repeated, these 10 calls to the function correspond to 10 consecutive pixels with the same color on the target image.

[0062] The target image is input into the trained preset recognition model to obtain a recognition result. The recognition result is used to identify the target area composed of the above-mentioned 10 consecutive pixel points, as well as the iteration information of the iterative execution structure corresponding to the target area. In this example, the iterative execution structure corresponding to the target area is a for loop structure. The pixel points in the target area (10 consecutive pixel points) correspond to the repeated function call sequence obtained after the loop structure is expanded, that is, 10 consecutive calls to the function glDrawElements. The iteration information of the iterative execution structure corresponding to the target area includes the number of iterations of the loop structure (such as 10 times).

[0063] The embodiment of the present invention does not limit the representation format of the recognition result. The recognition result can be in the form of text or a mark on the target image. For example, in Example 1, the output recognition result may include: target area coordinates and iteration count. The target area coordinates may be the bounding box coordinates of the target area, such as [x_min, y_min, x_max, y_max].

[0064] Because each pixel in the target image represents a calling function in the target program, and the pixels in the target image are encoded according to the order in which the calling functions are called in the target program, there is a corresponding relationship between the position of the pixel in the target image and the position of the calling function corresponding to the pixel in the target program. Based on this correspondence, the position of the corresponding calling function in the target program can be located based on the position of the pixel.

[0065] In an optional embodiment of the present invention, the method may further include:

[0066] Step S11, obtaining a compilable file of the target program;

[0067] Step S12: determining, in the compilable file, a starting code line and an ending code line of a repeatedly called function sequence according to the coordinates of the target area;

[0068] Step S13: In the compilable file, replace the code sequence from the start code line to the end code line with the code representation of the iterative execution structure.

[0069] According to the recognition results output by the preset recognition model, the coordinates of the target area can be obtained, and then the starting code line and the ending code line of the repeatedly called function sequence can be determined in the compilable file of the target program. For example, the coordinates of the first pixel point in the target area can be obtained according to the coordinates of the target area, and then the position of the corresponding calling function in the compilable file of the target program can be located. This position is the starting code line of the repeatedly called function sequence, such as the code line of the first calling function in 10 consecutive calling functions of glDrawElements. Similarly, the coordinates of the last pixel point in the target area can be obtained according to the coordinates of the target area, and then the position of the corresponding calling function in the compilable file of the target program can be located. This position is the ending code line of the repeatedly called function sequence, such as the code line of the last calling function in 10 consecutive calling functions of glDrawElements.

[0070] Thus, in the target program's compilable file, the code sequence from the start code line to the end code line can be replaced with the code representation of the iterative execution structure (e.g., a for loop structure). That is, the 10 consecutive calls to the glDrawElements function can be restored to the code representation of the for loop structure shown in Example 1. Replacing the repeated function call sequence in the compilable file with the iterative execution structure can significantly reduce the size of the compilable file, shorten compilation time, and improve compilation efficiency.

[0071] Furthermore, the iteration information may also include the function name of the repeatedly called function, and the type of the iterative execution structure (such as a loop structure or a compound function) may be determined based on the function name, thereby replacing the repeatedly called function sequence with a loop structure or a compound function.

[0072] If the target program contains multiple iterative execution structures, each of these structures can be processed according to the above steps. In the resulting compilable file, any sequence of repeatedly called functions can be restored to the corresponding iterative execution structure (loop structure or compound function), reducing the size of the compilable file. Specific test functions can be inserted into this processed compilable file as needed, and then the file can be recompiled to generate the target program's executable file, which can be used for hardware testing.

[0073] The embodiment of the present invention uses a preset tool to convert the calling functions appearing in the compilable file of the target program into pixel points on the target image in the order of calling, converts the recognition problem of the iterative execution structure into an image recognition problem, and automatically recognizes the iterative execution structure in the target program using a pre-trained preset recognition model, thereby improving recognition efficiency and improving the efficiency and accuracy of hardware testing based on the target program. Through the embodiment of the present invention, there is no need to rely on manual identification of repeated code and rewriting of the iterative execution structure, which can avoid the cumbersome and inefficient operations brought about by manual methods, and the problem that manual judgment is prone to errors, may miss some repeated code or introduce new errors, thereby affecting the efficiency and accuracy of hardware testing based on the target program.

[0074] In a specific implementation, the preset recognition model may be a neural network model. The embodiment of the present invention does not limit the network structure of the preset recognition model. The neural network includes but is not limited to at least one or a combination, superposition, or nesting of at least two of the following: CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory), RNN (Simple Recurrent Neural Network), Transformer deformation network, attention neural network, etc.

[0075] In an optional embodiment of the present invention, the method may further include:

[0076] Step S21, generating a plurality of image data by using the preset tool;

[0077] Step S22: annotating the image data with regions and iteration information containing a repeated function call sequence to obtain a training set; the training set includes a plurality of image data and annotated data corresponding to each image data; the annotated data includes region coordinates and iteration information;

[0078] Step S23: Use the training set to train the preset recognition model.

[0079] The embodiment of the present invention does not limit the method of training the preset recognition model. Exemplarily, a number of image data can be generated by the preset tool, and the generated image data can be annotated to construct training data. For example, in the image data in the RGB565 color coding format, an area containing continuous pixels with consistent colors is annotated. The area corresponds to an iterative execution structure in the source code. The original code fragment of the iterative execution structure and the starting code line and the ending code line are recorded to obtain training data, and then a training set is constructed. The training set contains a number of image data and the annotated data corresponding to each image data; each image data in the training set and the annotated data corresponding to the image data are called a piece of training data. The annotated data may include the coordinates of the annotated area and the annotated iteration information. The iteration information includes but is not limited to the number of iterations.

[0080] After the training set is constructed, the preset recognition model can be trained using the training set.

[0081] Exemplarily, the network structure of the preset recognition model can be a network structure improved based on YOLO (object detection framework). The model structure can include a backbone network and a head network. The backbone is used for feature extraction. The head is used to output recognition results, such as [x_min, y_min, x_max, y_max, iterations]. [x_min, y_min, x_max, y_max] represents the coordinates of the recognized area, and iterations represents the iterative information of the recognition.

[0082] Taking this model structure as an example, the model is first configured, such as defining the input size and anchor boxes to adapt to the aspect ratio of the area corresponding to the iterative execution structure. Then, the loss function is defined, such as combining the bounding box regression loss and the iterative information prediction loss to calculate the model loss. Next, the Adam optimizer is used to train on the training set until convergence to obtain the trained preset recognition model. Among them, the bounding box regression loss is used to calculate the difference between the region coordinates (predicted value) predicted by the preset recognition model and the labeled region coordinates (true value). The iterative information prediction loss is used to calculate the difference between the iterative information (predicted value) predicted by the preset recognition model and the labeled iterative information (true value).

[0083] It is understandable that the above model structure and training method are only for illustrative purposes, and the embodiment of the present invention does not limit the model structure and training method of the preset recognition model.

[0084] In an optional embodiment of the present invention, the using the training set to train the preset recognition model may include:

[0085] Step S31: Initialize a preset recognition model, wherein the preset recognition model includes an instance segmentation module, an edge detection module, and a feature extraction module;

[0086] Step S32: Select an image data and corresponding annotation data from the training set;

[0087] Step S33: input the selected image data into the instance segmentation module, and the instance segmentation module divides the continuous pixels in the image data into independent instances according to color to obtain an instance segmentation result;

[0088] Step S34: input the segmentation result into an edge detection module, and use the edge detection module to identify the boundary contour of the area corresponding to each independent instance in the image data to obtain an edge detection result;

[0089] Step S35: inputting the edge detection result into the feature extraction module, and extracting feature parameters within each boundary contour through the feature extraction module; the feature parameters include the number of consecutive pixels within the boundary contour and the color mean of the consecutive pixels within the boundary contour;

[0090] Step S36: outputting a recognition result of the predicted area corresponding to each boundary contour according to the feature parameters;

[0091] Step S37: Calculate the model loss based on the predicted recognition result and the difference between the image data and the corresponding labeled data;

[0092] Step S38: updating the model parameters of the preset recognition model according to the model loss;

[0093] Step S39: Enter the next round of iteration. When the iteration stop condition is met, a preset recognition model that has been trained is obtained.

[0094] After building the training set, you can use it to train a preset recognition model.

[0095] First, a preset recognition model is initialized. The preset recognition model may include an instance segmentation module, an edge detection module, and a feature extraction module. Each module may be composed of one or more network layers.

[0096] The instance segmentation module is used to divide similar pixels in an image (such as continuous pixels with consistent colors) into independent instances, providing a basis for subsequent feature extraction.

[0097] The edge detection module is used to extract boundary contours of independent instances divided by the instance segmentation module.

[0098] The feature extraction module is used to extract feature parameters from the image within the boundary contour extracted by the edge detection module.

[0099] Specifically, an image data and its corresponding annotation data are arbitrarily selected from the training set as current input data. The selected image data is input into the instance segmentation module, and the instance segmentation module divides the continuous pixels in the image data into independent instances according to color, thereby obtaining an instance segmentation result.

[0100] Then, the segmentation result is input into an edge detection module, and the edge detection module identifies the boundary contour of the area corresponding to each independent instance in the image data, such as obtaining the coordinates of each boundary contour, to obtain an edge detection result.

[0101] It should be noted that, in a specific implementation, binarization and morphological processing may be included before edge detection to improve the effect of edge detection. Among them, binarization is to convert a complex grayscale image into a simple binary image, reducing the amount of information in the image and making subsequent processing more efficient. Morphological processing is an operation based on the shape of the image, mainly using structural elements (such as rectangles, circles, etc.) to erode, dilate, open, close, etc. the image, thereby changing the shape and structure of the object in the image, facilitating subsequent edge detection.

[0102] Next, the edge detection results are input into the feature extraction module, which extracts characteristic parameters of the image within each boundary contour. These characteristic parameters include the number of consecutive pixels within the boundary contour and the color mean of these consecutive pixels within the boundary contour. The number of consecutive pixels within the boundary contour represents the number of repeated function calls and can represent the number of iterations. The color mean represents the color of any pixel in the area corresponding to the boundary contour, which represents the function category of the repeated function call.

[0103] Based on the feature parameters extracted by the feature extraction module, a recognition result of the predicted region corresponding to each boundary contour can be output, which includes the predicted region coordinates and prediction iteration information for the input image data.

[0104] The model loss may be calculated based on the predicted recognition result and the difference between the image data and the corresponding labeled data.

[0105] For example, the bounding box regression loss is calculated based on the difference between the predicted region coordinates and the labeled region coordinates; the iteration information prediction loss is calculated based on the difference between the predicted iteration information and the labeled iteration information; the final model loss is calculated comprehensively (such as weighted calculation) based on the bounding box regression loss and the iteration information prediction loss.

[0106] The model parameters of the preset recognition model are updated according to the model loss. For example, the network parameters of each module in the instance segmentation module, the edge detection module, and the feature extraction module can be updated.

[0107] Entering the next round of iteration, reselecting image data and corresponding annotation data from the training set as input data for the next round, repeating the above steps, and obtaining a trained preset recognition model when the iteration stop condition is met.

[0108] It should be noted that the embodiments of the present invention do not limit the method for calculating the bounding box regression loss and the iterative information prediction loss. For example, a common loss function can be used for calculation, such as the cross entropy loss function, the cosine similarity loss function, etc. The iteration stopping condition may include the model loss being less than a preset value, or the number of iterations reaching a preset number.

[0109] In an optional embodiment of the present invention, exporting all call functions of a specified type in the target program in sequence by using a preset tool may include:

[0110] Step S41: Use a preset tool to track the running process of the target program, record the call information of all the specified types of call functions called in the target program in sequence, and save it in a trace file;

[0111] Step S42: using the replay function of the preset tool to sequentially read the call information of the call function recorded in the trace file;

[0112] Step S43: During the replay process, the calling information of the calling functions recorded in the trace file is exported in sequence.

[0113] The embodiments of the present invention do not limit the type of pre-configured tool. For example, the pre-configured tool may be a modified apitrace program, which is a tool for tracing and replaying graphics API calls. The apitrace program has a replay function for replaying the graphics API call sequence during application runtime.

[0114] The modified apitrace program can track the running process of the target program, record the call information of all specified types of calling functions (such as all gl functions called) called in the target program in sequence, and save it in the trace file.

[0115] An embodiment of the present invention further modifies the replay function in the apitrace program so that when the replay function of the apitrace program is used to replay the GL functions called by the target program during execution, the calling information of all GL functions recorded in the trace file is read sequentially, all GL functions called by the target program are automatically exported, and a compilable file is generated by reverse engineering the target program.

[0116] In addition, the embodiment of the present invention further modifies the replay function in the apitrace program. After exporting all calling functions of a specified type in the target program (such as all called gl functions), the exported calling functions are encoded in the calling order to generate a target image.

[0117] It is understandable that the preset tool is not limited to the modification of an existing tool, but may also be a customized script, tool or program developed by the user.

[0118] In summary, the embodiment of the present invention converts the calling functions appearing in the compilable file of the target program into pixel points on the target image in the order of calling through a preset tool, converts the recognition problem of the iterative execution structure into an image recognition problem, and automatically recognizes the iterative execution structure in the target program using a pre-trained preset recognition model, which can improve recognition efficiency and improve the efficiency and accuracy of hardware testing based on the target program. Through the embodiment of the present invention, there is no need to rely on manual recognition of repeated code and rewriting of the iterative execution structure, which can avoid the cumbersome and inefficient operations caused by manual methods, and the problem that manual judgment is prone to errors, may miss some repeated code or introduce new errors, thereby affecting the efficiency and accuracy of hardware testing based on the target program.

[0119] In addition, through the embodiments of the present invention, after the target program is reversed to obtain a compilable file, the repeated function call sequence in the compilable file can be replaced with an iterative execution structure, which can greatly reduce the size of the compilable file, shorten the compilation time, and improve the compilation efficiency.

[0120] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0121] Reference Figure 2 , shows a structural block diagram of an embodiment of a program processing device of the present invention, the device may include:

[0122] The export generation module 201 is used to sequentially export the call information of all specified types of call functions in the target program through a preset tool, and encode the call functions to generate a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function;

[0123] The model recognition module 202 is configured to input the target image into a pre-trained recognition model to obtain a recognition result; the recognition result includes a target region in the target image and iteration information of an iterative execution structure corresponding to the target region; pixels in the target region correspond to a sequence of repeatedly called functions obtained by expanding the iterative execution structure; the iteration information includes the number of iterations of the iterative execution structure;

[0124] The code conversion module 203 is configured to generate a code representation of the iterative execution structure according to the recognition result.

[0125] Optionally, each of all the calling functions of the specified type corresponds to respective number information; and the apparatus further includes:

[0126] A coding conversion module is used to, for each calling function of the specified type in the target program, multiply the respective numbering information by a preset value in sequence, and then encode the result into a hexadecimal number based on the function category corresponding to the calling function as a pixel point in the target image; the first designated bit of the hexadecimal number represents red, the second designated bit represents green, and the third designated bit represents blue; the result obtained by multiplying the numbering information of each calling function by the preset value is less than or equal to the maximum number of colors supported by the color coding format of the target image.

[0127] Optionally, the device further comprises:

[0128] A data generation module, configured to generate a plurality of image data using the preset tool;

[0129] a data annotation module, configured to annotate the image data with regions and iteration information containing a repeated function call sequence to obtain a training set; the training set comprises a plurality of image data and annotated data corresponding to each image data; the annotated data comprises region coordinates and iteration information;

[0130] A model training module is used to train the preset recognition model using the training set.

[0131] Optionally, the model training module includes:

[0132] An initialization submodule, used to initialize a preset recognition model, wherein the preset recognition model includes an instance segmentation module, an edge detection module, and a feature extraction module;

[0133] A data selection submodule, configured to select an image data and corresponding annotation data from the training set;

[0134] A first processing submodule, configured to input the selected image data into the instance segmentation module, and divide the continuous pixels in the image data into independent instances according to color by the instance segmentation module to obtain an instance segmentation result;

[0135] A second processing submodule is configured to input the segmentation result into an edge detection module, and identify the boundary contour of the area corresponding to each independent instance in the image data through the edge detection module to obtain an edge detection result;

[0136] a third processing submodule, configured to input the edge detection result into the feature extraction module, and extract feature parameters within each boundary contour through the feature extraction module; the feature parameters include the number of consecutive pixels within the boundary contour and the color mean of the consecutive pixels within the boundary contour;

[0137] A result output submodule, configured to output a recognition result predicted for the area corresponding to each boundary contour according to the feature parameters;

[0138] a loss calculation submodule, configured to calculate a model loss based on the predicted recognition result and the difference between the image data and the corresponding annotated data;

[0139] A parameter updating submodule, configured to update the model parameters of the preset recognition model according to the model loss;

[0140] The iterative training submodule is used to enter the next round of iteration and obtain the preset recognition model that has been trained when the iteration stop condition is met.

[0141] Optionally, the device further comprises:

[0142] An acquisition module, used for acquiring a compilable file of the target program;

[0143] a positioning module, configured to determine, in the compilable file, a starting code line and an ending code line of a repeatedly called function sequence according to the coordinates of the target area;

[0144] A replacement module is used to replace the code sequence from the start code line to the end code line in the compilable file with the code representation of the iterative execution structure.

[0145] Optionally, the export generation module includes:

[0146] An information recording submodule is used to track the running process of the target program using a preset tool, record the call information of all the specified types of call functions called in the target program in sequence, and save it in a tracking file;

[0147] A replay reading submodule, configured to use the replay function of the preset tool to sequentially read the call information of the call function recorded in the trace file;

[0148] The replay export submodule is used to export the call information of the call function recorded in the tracking file in sequence during the replay process.

[0149] Optionally, the iterative execution structure includes a loop structure, which is expanded into multiple consecutive repeated calling functions during execution to obtain a repeated calling function sequence; or, the iterative execution structure includes a composite function, which is decomposed into multiple consecutive repeated sub-functions during execution to obtain a repeated calling function sequence.

[0150] The embodiment of the present invention uses a preset tool to convert the calling functions appearing in the compilable file of the target program into pixel points on the target image in the order of calling, converts the recognition problem of the iterative execution structure into an image recognition problem, and automatically recognizes the iterative execution structure in the target program using a pre-trained preset recognition model, thereby improving recognition efficiency and improving the efficiency and accuracy of hardware testing based on the target program. Through the embodiment of the present invention, there is no need to rely on manual identification of repeated code and rewriting of the iterative execution structure, which can avoid the cumbersome and inefficient operations brought about by manual methods, and the problem that manual judgment is prone to errors, may miss some repeated code or introduce new errors, thereby affecting the efficiency and accuracy of hardware testing based on the target program.

[0151] In addition, through the embodiments of the present invention, after the target program is reversed to obtain a compilable file, the repeated function call sequence in the compilable file can be replaced with an iterative execution structure, which can greatly reduce the size of the compilable file, shorten the compilation time, and improve the compilation efficiency.

[0152] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0153] Reference Figure 3 , is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3As shown, the electronic device includes: a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the program processing method of the aforementioned embodiment.

[0154] An embodiment of the present invention provides a non-transitory computer-readable storage medium. When instructions in the storage medium are executed by a program or processor of a terminal, the terminal is enabled to perform the steps of the program processing method of the aforementioned embodiment.

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0156] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable program processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable program processing terminal device generate instructions for implementing the process in the flowchart. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable program processing terminal device to operate in a predictable manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable program processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0160] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0161] Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A program processing method, characterized in that: The method comprises: Exporting call information of all specified types of call functions in the target program in sequence, and encoding the call functions to generate a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function; Identifying an arrangement of pixels in the target image to obtain a recognition result; the recognition result includes a target area in the target image and iteration information of an iterative execution structure corresponding to the target area; the pixels in the target area correspond to a repeatedly called function sequence obtained after the iterative execution structure is expanded; the iteration information includes the number of iterations of the iterative execution structure; A code representation of the iterative execution structure is generated according to the recognition result.

2. The method according to claim 1, characterized in that Identifying the arrangement of pixel points includes identifying the continuous and repeated arrangement of pixel points of the same color.

3. The method according to claim 1, characterized in that The iterative execution structure includes a loop structure, which is expanded into multiple consecutive repeated calling functions during execution to obtain a repeated calling function sequence; Alternatively, the iterative execution structure includes a composite function, which is decomposed into a plurality of consecutively repeated sub-functions during execution to obtain a repeated function call sequence.

4. The method according to claim 1, wherein Each of all the calling functions of the specified type corresponds to its own number information; the method further includes: For each calling function of the specified type in the target program, the respective numbering information is multiplied by a preset value in sequence, and then encoded as a hexadecimal number based on the function category corresponding to the calling function as a pixel point in the target image; the first designated bit of the hexadecimal number represents red, the second designated bit represents green, and the third designated bit represents blue; the result obtained by multiplying the numbering information of each calling function by the preset value is less than or equal to the maximum number of colors supported by the color encoding format of the target image.

5. The method according to claim 1, characterized in that The method further comprises: Exporting the call information of all specified types of call functions in the target program in sequence through a preset tool to generate a number of image data; The image data is annotated with regions and iteration information including repeated function calls to obtain a training set; the training set includes a plurality of image data and annotated data corresponding to each image data; the annotated data includes region coordinates and iteration information; The training set is used to train a preset recognition model, and the target image is input into the trained preset recognition model to obtain a recognition result.

6. The method according to claim 5, characterized in that The step of training the preset recognition model using the training set includes: Initializing a preset recognition model, wherein the preset recognition model includes an instance segmentation module, an edge detection module, and a feature extraction module; Selecting an image data and corresponding annotation data from the training set; Inputting the selected image data into the instance segmentation module, and using the instance segmentation module to divide continuous pixels in the image data into independent instances according to color, thereby obtaining an instance segmentation result; Inputting the segmentation result into an edge detection module, and using the edge detection module to identify the boundary contour of the area corresponding to each independent instance in the image data to obtain an edge detection result; Inputting the edge detection result into the feature extraction module, and extracting feature parameters within each boundary contour through the feature extraction module; the feature parameters include the number of continuous pixel points within the boundary contour and the color mean of the continuous pixel points within the boundary contour; Outputting a recognition result of the prediction of the area corresponding to each boundary contour according to the feature parameters; Calculating a model loss based on the predicted recognition result and the difference between the image data and the corresponding labeled data; Updating model parameters of the preset recognition model according to the model loss; Entering the next round of iteration, when the iteration stop condition is met, the trained preset recognition model is obtained.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a compilable file of the target program; Determining, in the compilable file, a starting code line and an ending code line of a repeatedly called function sequence according to the coordinates of the target area; In the compilable file, a code sequence from a start code line to an end code line is replaced with a code representation of the iterative execution structure.

8. The method according to claim 1, characterized in that Identifying the arrangement of pixels in the target image, including: Using a preset tool to track the running process of the target program, sequentially recording the call information of all the specified types of call functions called in the target program, and saving the information in a trace file; Using the replay function of the preset tool, sequentially read the call information of the call function recorded in the trace file; During the replay process, the calling information of the calling functions recorded in the trace file is exported in sequence.

9. A program processing device, characterized in that: The device comprises: An export generation module sequentially exports call information of all specified types of call functions in a target program, encodes the call functions, and generates a target image; each pixel in the target image represents a call function in the target program, and the color of the pixel represents the function category of the call function; a model recognition module that recognizes the arrangement of pixels in the target image and obtains a recognition result; the recognition result includes a target area in the target image and iteration information of an iterative execution structure corresponding to the target area; the pixels in the target area correspond to a repeatedly called function sequence obtained by expanding the iterative execution structure; the iteration information includes the number of iterations of the iterative execution structure; A code conversion module is used to generate a code representation of the iterative execution structure according to the recognition result.

10. An electronic device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the program processing method according to any one of claims 1 to 8.

11. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the program processing method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the program processing method according to any one of claims 1 to 8 are implemented.

Images