Test data generation method, device, storage medium and electronic device

Abstract

The embodiment of the present application discloses a test data generation method, device, storage medium and electronic device, the method comprising: obtaining the source point and convergence point of the data flow of the code data constituting the target operator when executing the computing task, the number of the source point and the number of the convergence point are at least one respectively; extracting the constraint information corresponding to the multiple operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data; obtaining the test data according to the constraint information corresponding to the multiple operator parameters, the test data is used to test whether the target operator has an abnormality when executing the computing task corresponding to the test data. The embodiment of the present application can enhance the accuracy of the generated test data and improve the efficiency of the model test.

Description

Technical Field

[0001] The present application relates to the field of model testing, and in particular to a test data generation method, device, storage medium and electronic device. Background Art

[0002] The machine learning model implemented based on the machine learning framework or machine learning algorithm contains the most basic training and reasoning functions, which are implemented through operators that encapsulate a large number of parameters in the model. In other words, the machine learning model usually includes a large number of operators, and relies on a large number of operator parameters in each operator to control the results generated by the machine learning model.

[0003] Machine learning algorithm models need to be tested before training and use. The current mainstream model testing method is to generate test data through fuzz testing detection technology, and test data is also called test cases. Fuzz testing detection technology can automatically construct random, unexpected, deformed data as test cases, input them into a certain operator of the machine learning model, monitor the anomalies that may occur in the process of executing computing tasks, and realize automated or semi-automated program security testing technology.

[0004] Tools that implement fuzz testing detection technology include AFL, Peach, Honggfuzz, libfuzzer, etc. However, the above-mentioned fuzz testing tools still have the following shortcomings in model detection: (1) Fuzz testing tools such as AFL have strong mutation randomness, making it difficult to generate test data that meets the format for the function interface parameters provided by the model test; (2) Rule-based testing tools such as Peach perform fuzz testing by manually writing test rules, which requires a lot of labor costs.

[0005] In other words, as machine learning models become more complex and more and more machine learning model vulnerabilities are disclosed, current detection tools for machine learning models are unable to generate accurate test data. Summary of the invention

[0006] The embodiments of the present application provide a test data generation method, device, storage medium and electronic device, which can enhance the accuracy of the generated test data and improve the efficiency of model testing. The technical solution is as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for generating test data, the method comprising:

[0008] Obtaining source points and sink points of a data flow of code data constituting a target operator when executing a computing task, wherein the number of the source points and the number of the sink points are at least one;

[0009] Extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data;

[0010] According to the constraint information respectively corresponding to the multiple operator parameters, test data is obtained, and the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data.

[0011] In a second aspect, an embodiment of the present application provides a test data generating device, the device comprising:

[0012] A data acquisition module, used to acquire the source points and the sink points of the data flow of the code data constituting the target operator when executing the computing task, wherein the number of the source points and the number of the sink points are at least one respectively;

[0013] A constraint acquisition module, used for extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data;

[0014] The data generation module is used to obtain test data according to the constraint information respectively corresponding to the multiple operator parameters, and the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data.

[0015] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0016] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0017] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0018] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a schematic diagram of a process of testing a target operator according to test data provided by an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a test data generation method provided in an embodiment of the present application;

[0022] Figure 3 It is a flowchart of a test data generation method provided in an embodiment of the present application;

[0023] Figure 4 It is a flowchart of a test data generation method provided in an embodiment of the present application;

[0024] Figure 5 It is a flowchart of a test data generation method provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of a test data generating device provided in an embodiment of the present application;

[0026] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0029] The present application is described in detail below with reference to specific embodiments.

[0030] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the object features, interactive behavior features and user information involved in this specification are all obtained with full authorization.

[0031] The machine learning model implemented based on the machine learning framework or machine learning algorithm contains the most basic training and reasoning functions, which are implemented through operators that encapsulate a large number of parameters in the model. In other words, the machine learning model usually includes a large number of operators, and relies on a large number of operator parameters in each operator to control the results generated by the machine learning model.

[0032] Machine learning algorithm models need to be tested before training and use. The current mainstream model testing method is to generate test data through fuzz testing detection technology, and test data is also called test cases. Fuzz testing detection technology can automatically construct random, unexpected, deformed data as test cases, input them into a certain operator of the machine learning model, monitor the anomalies that may occur in the process of executing computing tasks, and realize automated or semi-automated program security testing technology.

[0033] In one embodiment, Figure 1 As shown, it is a schematic diagram of a process of testing a target operator according to test data proposed in an embodiment of the present application. The machine learning model 101 includes a plurality of hierarchical structures composed of at least one operator, and the plurality of operators cooperate to perform computing tasks to realize the function of the machine learning model 101. For example, the machine learning model 101 is a Markov chain model or a sequence-to-sequence model or a convolutional neural network model or a Transformer model or other types.

[0034] The Markov chain model is a random process model that describes the transition from one state to another.

[0035] The sequence-to-sequence Seq2Seq model is a special variant of the recurrent neural network model. The traditional recurrent neural network (RNN) model has many limitations. For example, in translation problems, the sequence length of the input characters is indefinite, while the output of the RNN model is fixed (the length is the same as the number of units in the RNN model). However, in many cases, the length of the translated sentence is not one-to-one with the length of the original sentence, so the length of the input and output characters cannot be determined at the beginning. In this case, the sequence-to-sequence Seq2Seq model can be used. The Seq2Seq model uses two recurrent neural network models, one of which is an encoder, which is responsible for processing the sequence of input characters, and the other is called a decoder, which is responsible for processing the results of the encoder to obtain a sequence of output characters.

[0036] The neurons in the convolutional neural network model can well extract the characteristic information of the surrounding neurons within the range of the neurons, and have achieved great success in the field of image processing. With the research and improvement of convolutional neural networks, it has also performed well in other fields such as natural language processing. Figure 1As shown, taking the machine learning model 101 as a convolutional neural network model as an example, there are usually hierarchical structures such as input layer, convolution layer, pooling layer, and fully connected layer, and each hierarchical structure includes at least one operator to complete the calculation task. For example, the convolution layer of the machine learning model 101 includes operators 1011, 1012, and 1013, and the pooling layer includes operators 1014 and 1015. It can be understood that Figure 1 The machine learning model 101 shown is only an example, and the number of operators included in each hierarchical structure is also an example, and this specification does not impose any limitations on this.

[0037] The Transformer model is currently the hottest network model in the field of natural language processing, and has performed well in various tasks. Its joint framework with the Seq2Seq network has greatly improved the performance in various tasks in natural language processing and even in the field of images. Compared with the Seq2Seq network, which still needs to use convolutional neural networks or various recurrent neural networks as the main body of the model, the Transformer model innovatively abandons the inherent mode of the Seq2Seq model and only uses the attention mechanism to build the network model. While reducing the amount of calculation on each operator and solving the disadvantage of low parallel computing efficiency of the traditional Seq2Seq network model, it also guarantees the performance of the Transformer network model in most tasks in the field of natural language processing.

[0038] It can be understood that the machine learning model 101 of the embodiment of the present application is not limited to the above-mentioned model, and other machine learning models that include at least one operator and can perform computing tasks are applicable.

[0039] In machine learning models, different operators accept different input data and implement complex calculation processes. Different operators may include linear operation operators, nonlinear activation operators, pooling operators, loss function operators, optimizer operators, and other types. During the calculation process, each operator may have memory errors such as buffer overflow, repeated memory release, memory leak, integer overflow, etc. At this time, it is necessary to adjust each operator through a large number of test data. Figure 1 As shown, the test data 102 is input into the operator 1013 so that the operator 1013 executes the computing task corresponding to the test data 102 and obtains the test result 103 , and the possible problems in the operator 1013 are discovered in time according to the test result 103 , so as to adjust the operator 1013 .

[0040] However, with the increasing complexity of machine learning models and the disclosure of more and more machine learning model vulnerabilities, the current detection tools for machine learning models cannot generate accurate test data, and the test effect on operators is not ideal. Therefore, the present application embodiment proposes a test data generation method to solve some or all of the above problems.

[0041] In one embodiment, Figure 2 As shown, it is a schematic diagram of a test data generation method proposed in an embodiment of the present application, which can be implemented by a computer program and can be run on a test data generation device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0042] Specifically, the test data generating method includes:

[0043] S101. Obtain the source point and the convergence point of the data flow of the code data constituting the target operator when executing a computing task.

[0044] The execution subject of the embodiment of the present application is a terminal device, which is an electronic device used by a user. The electronic device can be a personal computer, a mobile phone, a tablet computer, etc., which has a certain computing power and runs an application or web page with a prompt word generation function. It can also be a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. It can be understood that the number and type of the above-mentioned terminal devices are only for illustration, and the embodiment of the present application does not impose any restrictions on this.

[0045] The target operator can be understood as one of the multiple operators included in the machine learning model. Figure 1 As shown, the target operator is operator 1013 in the machine learning model 101, and may also be operator 1011 or other operators, which is not limited in the embodiments of the present application.

[0046] The code data that constitutes the target operator can be understood as the function of implementing the target operator to perform computing tasks based on the code data. For example, in the TensorFlow framework, the operator is implemented by extending OpKernel and rewriting the compute method. The multiple operator parameters included in the operator are divided into input type and attr type, where the operator parameter of input type has a tensor with a variable value, while the operator parameter of attr type remains unchanged. It can be understood that the code data that constitutes the target operator can be code data implemented based on any framework, and this application does not impose any restrictions on this.

[0047] like Figure 3 As shown, Figure 3 2 is a flow chart of a test data generation method provided by an embodiment of the present application. The code data constituting the target operator 201 at least includes sub-code data 2011, sub-code data 2012, sub-code data 2013, ..., sub-code data 201N. In other words, a plurality of sub-code data constitute the code data. The sub-code data can be divided according to a preset division rule, for example, the preset division rule is a row, that is, a row of code data is a sub-code data.

[0048] When the target operator performs a computing task, the data flow generated by the calculation starts from at least one source point, further concentrates to at least one convergence point, and finally outputs. For example, the target operator is an operator of the fully-connected layer, which performs a weighted summing computing task. The data flow starts from multiple source points for calculating weights, and multiple weight data are concentrated to the summing convergence point, and finally outputs the summed data. It can be understood that the source point of the target operator can be one or more, the convergence point can be one or more, and the type of the convergence point can be one or more, for example, including a loop convergence point or an ordinary convergence point. In the operator that performs the weighted summing computing task, the convergence point indicating the summation calculation is an ordinary convergence point.

[0049] S102: Extract constraint information corresponding to a plurality of operator parameters included in a target operator according to the sub-code data located at a source point and the sub-code data located at a convergence point in the code data.

[0050] The constraint information of the operator parameters can be understood as the conditions or rules that the operator parameters must satisfy in order for the code data to execute correctly and produce the expected output, that is, the target operator to correctly perform the computing task. The constraint information corresponding to each operator parameter includes the first constraint information related to the operator parameter itself, and also includes the second constraint information related to the dependency relationship between other operator parameters. It is understandable that the operator parameters may include only the first constraint information, or only the second constraint information, or both the first constraint information and the second constraint information, and this application does not impose any restrictions on this.

[0051] Specifically, the first constraint information related to the operator parameter itself includes at least shape constraint, type constraint and value constraint. Among them, shape constraint is used to indicate the shape of the operator parameter, type constraint is used to indicate the shape of the operator parameter, and type constraint is used to indicate the type of the operator parameter. For example, the type constraint of a certain operator parameter is ins type, the value constraint of a certain operator parameter is greater than 0 and less than 1, and the shape of a certain operator parameter is a 3×4 vector. The second constraint information related to the dependency relationship between other operator parameters, for example, the same type as a certain operator parameter, or not less than the type of a certain operator parameter, or the same shape as a certain operator parameter, etc.

[0052] According to the sub-code data at the source point and the sub-code data at the convergence point in the code data, the constraint information corresponding to the multiple operator parameters is extracted. For example, the first constraint information related to the operator parameter itself included in the operator parameter is extracted according to the sub-code data at the source point, and the second constraint information related to the dependency relationship between the operator parameter and other operator parameters is extracted according to the sub-code data at the convergence point. For example, the code data at a certain convergence point indicates that a difference calculation is performed between multiple operator parameters, and the output value of the calculation is greater than 0. Therefore, according to the calculation formula of the above difference calculation, the numerical constraints between the multiple operator parameters can be extracted as the second constraint information of each operator parameter.

[0053] like Figure 3 As shown, based on the sub-code data 2011 at the source point and the sub-code data 2011 at the convergence point, the constraint information corresponding to the multiple operator parameters is extracted. The multiple operator parameters include operator parameters 2021, operator parameters 2022, operator parameters 2023, ..., operator parameters 202M, and the operator parameter 2021 includes constraint information 2031, and the operator parameter 2022 includes constraint information 2032, the operator parameter 2023 includes constraint information 2033, and the operator parameter 202M includes constraint information 203M.

[0054] In the embodiment of the present application, the source points of the data flow are systematically extracted through the structural characteristics of the target operator in the machine learning model, and the user-controllable input is obtained more accurately and comprehensively, which provides a guarantee for the subsequent data flow analysis and avoids the large amount of cost caused by manual analysis. Further, the convergence point of the data flow is obtained through the structural characteristics of the source point and the target operator, and according to the sub-code data corresponding to the source point and the convergence point, comprehensive analysis information is provided for extracting the constraint information included in the operator parameters, so that the test data generated according to the constraint information included in the multiple operator parameters can reach the deep logic of the code data constituting the target operator, cover a wider range of code implementations, and more effectively detect the security of the code data constituting the target operator.

[0055] S103. Obtain test data according to the constraint information corresponding to the plurality of operator parameters.

[0056] The constraint information corresponding to multiple operator parameters is the key to making the code execute correctly and produce the expected output. These constraints play a vital role in ensuring data integrity, maintaining API stability, and preventing errors or exceptions during the execution of the target operator.

[0057] Therefore, when generating test data, that is, test cases, if the input parameters of the test data do not satisfy the semantic rules of the target operator based on the constraint information corresponding to multiple operator parameters, the test data will usually fail in the semantic check and fail in the shallow code of the target operator, and will not be able to complete the task of testing the target operator.

[0058] According to the constraint information corresponding to the multiple operator parameters, test data is obtained. Specifically, the test data satisfies the first constraint information related to the operator parameters themselves among the multiple operator parameters, that is, at least satisfies the shape constraint, type constraint or value constraint of the operator parameters, and also satisfies the second constraint information related to the dependency relationship between each operator parameter and other operator parameters.

[0059] like Figure 3 As shown, according to the constraint information 2031 included in the operator parameter 2021, the constraint information 2032 included in the operator parameter 2022, the constraint information 2033 included in the operator parameter 2023, and the constraint information 203M included in the operator parameter 202M, the test data 204 is obtained. The test data 204 is used to test whether the target operator 204 has an exception when executing the computing task corresponding to the test data 204, such as a buffer overflow, repeated release of memory, memory leak, integer overflow, and other memory error exceptions.

[0060] For example, the type constraint of an operator parameter is ins type, the value constraint of an operator parameter is greater than 0 and less than 1, the shape of an operator parameter is a 3×4 vector, and the operator parameter is of the same type as an operator parameter. Therefore, after determining the type of the operator parameter, the operator parameter with the same type of constraint information as the operator parameter is determined.

[0061] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data.

[0062] In one embodiment, Figure 4 As shown, it is a schematic diagram of a test data generation method proposed in an embodiment of the present application, which can be implemented by a computer program and can be run on a test data generation device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0063] Specifically, the test data generating method includes:

[0064] S201. According to the position of the target operator in the machine learning model, obtain the original code data constituting the target operator.

[0065] Determine the location of the target operator in the machine learning model. For example, the target operator is in the pooling layer, the fully connected layer, or the function activation layer in the machine learning model.

[0066] Further, the original code data constituting the target operator is determined from the large amount of original code data constituting the machine learning model according to the position of the target operator in the machine learning model. For example, the original code data constituting the target operator is determined according to empirical rules or other rules based on the model type to which the machine learning model belongs and the development architecture of the machine learning model.

[0067] In this embodiment, by determining the original code data constituting the target operator from a large amount of code constituting the machine learning model, it is avoided that in the subsequent compilation process of the original code data constituting the target operator, code data belonging to other non-target operators is compiled, and even a complete machine learning model is compiled, which causes performance loss and time waste caused by compiling irrelevant code data.

[0068] S202 : compile the original code data constituting the target operator from the original format into an intermediate representation to obtain the code data constituting the target operator.

[0069] An Intermediate Representation (IR), sometimes called an Intermediate Code (IC), is a data structure of an editor.

[0070] Due to the different development architectures of machine learning models, the original code data that constitutes the target operator can be in various formats. For example, in the TensorFlow framework, the target operator is implemented by extending OpKernel and rewriting the compute method. In other machine learning models, the original code data that constitutes the target operator can also be in JAVA format or C / C++ format.

[0071] This embodiment compiles the original code data constituting the target operator from the original format into an intermediate representation. Specifically, gllvm is used to compile the original code data into a binary form, and finally get-bc is used to extract the LLVM intermediate representation from the original code data in binary form, that is, to obtain the code data of the intermediate representation. gllvm is an important compiler in the R language. The embodiments of the present application may also include other methods of compiling the original code data constituting the target operator from the original format into an intermediate representation to constitute the code data of the target operator, and the above are only examples.

[0072] In this embodiment, by uniformly compiling complex and diverse original code data from the original format into an intermediate representation, the code data is analyzed at the intermediate representation level, without having to process the syntax and semantic details of various high-level languages ​​corresponding to the original code data, thereby improving the efficiency of generating test data.

[0073] S203: Obtain the source point and the convergence point of the data flow of the code data constituting the target operator when executing the computing task.

[0074] Specifically, according to the data flow of the code data constituting the target operator when executing the computing task, the location of the sub-code data in the code data that meets the first preset rule is determined as the source point, and the location of the sub-code data in the code data that meets the second preset rule is determined as the convergence point. In other words, by detecting whether multiple sub-code data in the code data constituting the target operator meet the first preset rule or the second preset rule, the source point and convergence point of the data flow of the target operator when executing the computing task are determined.

[0075] The first preset rule and the second preset rule can be obtained by receiving an input instruction input by a user through an input device of an electronic device that generates test data, or by receiving rule information sent by an electronic device that has a communication connection with the electronic device that generates test data. In other words, the embodiment of the present application does not impose any restrictions on the method of obtaining the first preset rule and the second preset rule.

[0076] In one embodiment, the first preset rule includes sub-code data for characterizing preset instructions, and the preset instructions include at least one of the following instructions: a load instruction, a store instruction, and a pointer getelementptr instruction. Taking the TensorFlow framework as an example, the target operator is implemented by extending OpKernel and rewriting the compute method. The target operator receives an operator parameter of the attr type in the constructor and an operator parameter of the input type in the compute method. Therefore, the return value of the context->GetAttr function and the context->input function is used as the source point of the data flow of the target operator, and the location of the sub-code data corresponding to the load instruction, the store instruction, and the pointer getelementptr instruction is paid attention to.

[0077] The pointer getelementptr instruction includes a vars pointer instruction pointing to an array of tainted operator parameters, which is used to store tainted operator parameters. The length of the array depends on the number of tainted operator parameters recorded when the tainted operator parameters are marked; or includes an exps pointer instruction pointing to an expression type array, which is used to store parameter operators of suspicious statement types; or includes an ops pointer pointing to an operator type array, which is used to store operator parameters of operator types existing in suspicious statements. The pointer getelementptr instruction also includes other types, and the above are only examples.

[0078] In another embodiment, the second preset rule includes subcode data for representing a conditional jump instruction, and target subcode data at a jump destination point for representing an error output instruction or a calculation check instruction. The conditional jump instruction is used to indicate that the data flow jumps to the target subcode data at the destination point.

[0079] Specifically, it is known from experience that most machine learning models use macros in the source code data to evaluate the validity of operator parameters in the target operator. For example, the macro is OP_REQUIRES in the learning model of the TensorFlow framework, the macro is TORCH_CHECK in the machine learning model of the PyTorch framework, and the macro is PADDLE_ENFORCE_EQ in the machine learning model of the Paddle framework.

[0080] Furthermore, for example, the OP_REQUIRES macro is used in a target operator BTCBFS to determine the relationship between the stats_summary and logits_dimension operator parameters. If the test data input to the target operator does not meet the constraints of the above operator parameters, the OP_REQUIRES macro will report an error and terminate the task process of the target operator to perform the calculation task. The second parameter of the OP_REQUIRES macro is an expression, and the third parameter is an error output statement. When the expression is false, the output function is called to print the error statement.

[0081] In fact, after the original code data in the original format is compiled, the sub-code data is represented or characterized as a conditional jump instruction in the intermediate representation IR, and the basic block of the jump target contains an error output function or a check function. Therefore, the second preset rule includes that the sub-code data is used to represent the conditional jump instruction, and the target sub-code data located at the jump destination point is used to represent the error output instruction or the calculation check instruction. When there is sub-code data in the code data constituting the target operator that satisfies the above second preset rule, the location of the sub-code data is the convergence point of the data flow of the target operator.

[0082] In one embodiment, the first preset rule and the second preset rule both include subcode data including taint operator parameters. The taint operator parameters can be understood as one or more set operator parameters. For example, in a target operator that performs a loop calculation task, the taint operator parameters are operator parameters related to the calculation step size. In a target operator that performs weight calculation, the taint operator parameters are operator parameters related to the weight. The taint operator parameters can be set or preset by the user.

[0083] Both the first preset rule and the second preset rule include that the subcode data includes a taint operator parameter. In other words, the subcode data at the source point includes the taint operator parameter, and the subcode data at the convergence point includes the taint operator parameter. For example, the first preset rule includes that the subcode data is used to characterize the preset instruction and the subcode data includes the taint operator parameter, and the second preset rule includes that the subcode data is used to characterize the conditional jump instruction, and the target subcode data at the jump destination point is used to characterize the error output instruction or the calculation check instruction, and the subcode data includes the taint operator parameter.

[0084] In this embodiment, by setting at least one tainted operator parameter among all the operator parameters, more attention is paid to the constraint information corresponding to the tainted operator parameter, so that test data is generated according to the constraint information corresponding to at least one tainted operator parameter, and more attention is paid to information that deserves more attention, thereby avoiding the waste of time and computing resources caused by operations such as extracting constraint information for all operator parameters.

[0085] In one embodiment, the convergence point includes a loop convergence point and a common convergence point. When it is determined that the position of the subcode data satisfying the second preset rule is the external position of the code data, the position of the subcode data satisfying the second preset rule is determined to be the loop convergence point; or, when it is determined that the position of the subcode data satisfying the second preset rule is the internal position of the code data, the position of the subcode data satisfying the second preset rule is determined to be the common convergence point.

[0086] The location of the sub-code data that is located at the external position of the code data and meets the second preset rule is a loop convergence point. For example, the sub-code data corresponding to the convergence point is used to represent the instruction of executing the numerical range in the loop determination array. The convergence point is a loop convergence point. The location of the sub-code data that is located at the internal position of the code data and meets the second preset rule is a common convergence point. The common convergence point is a convergence point that does not belong to any loop body or branch body. For example, the sub-code data corresponding to the convergence point is used to represent the instruction of executing the weight summation. The convergence point is a common convergence point.

[0087] In this embodiment, by distinguishing between cyclic convergence points and common convergence points, the accuracy of obtaining constraint information corresponding to multiple operator parameters is improved.

[0088] S204: Extract constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data.

[0089] See the above S102, which will not be described again here.

[0090] S205. Obtain test data according to the constraint information corresponding to the plurality of operator parameters.

[0091] See the above S103, which will not be described again here.

[0092] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data.

[0093] In one embodiment, Figure 4 As shown, it is a schematic diagram of a test data generation method proposed in an embodiment of the present application, which can be implemented by a computer program and can be run on a test data generation device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0094] Specifically, the test data generating method includes:

[0095] S301. According to the position of the target operator in the machine learning model, the original code data constituting the target operator is obtained.

[0096] See the above S201, which will not be repeated here.

[0097] S302 : compile the original code data constituting the target operator from the original format into an intermediate representation to obtain the code data constituting the target operator.

[0098] See the above S202, which will not be described again here.

[0099] S303: Obtain the source point and the convergence point of the data flow of the code data constituting the target operator when executing the computing task.

[0100] See the above S203, which will not be described again here.

[0101] S304. Extract initial constraint information corresponding to multiple operator parameters included in the target operator based on feature information of the target instruction represented by the sub-code data at the source point in the code data and feature information of the target instruction represented by the sub-code data at the convergence point.

[0102] The characteristic information of the target instruction represented by the sub-code data at the source point in the code data, and the characteristic information of the target instruction represented by the sub-code data at the convergence point, the characteristic information can be understood as the meaning of the target instruction or the calculation content of the instruction data.

[0103] For example, the sub-code data at the source point represents a load instruction. Based on the characteristic information of the load instruction, the first constraint information related to the operator parameter itself included in the loaded operator parameter can be extracted as the initial constraint information, such as the numerical constraint of the operator parameter, etc. For another example, the target instruction represented by the sub-code data at the convergence point is to perform a difference calculation between multiple operator parameters, such as operator parameter A-operator parameter B=0, and the numerical constraint between the above operator parameters can be further extracted as the second constraint information of each operator parameter.

[0104] S305 , simplifying and revising the initial constraint information corresponding to the multiple operator parameters to obtain the constraint information corresponding to the multiple operator parameters.

[0105] According to the characteristic information of the target instruction represented by the sub-code data at the source point in the code data and the characteristic information of the target instruction represented by the sub-code data at the convergence point, the constraint information corresponding to the multiple operator parameters included in the target operator is extracted. However, since the original code data constituting the target operator is compiled from the original format into an intermediate representation, the initial constraint information corresponding to the multiple operator parameters collected is not intuitive, so further processing is required.

[0106] In this embodiment, the initial constraint information corresponding to the multiple operator parameters is simplified and revised to obtain the constraint information corresponding to the multiple operator parameters. The simplification and revision include mapping the multiple initial constraint information at the IR level to constraint information of Python or C / C++ parameter type (or other types of constraint information), and simplifying the constraint information corresponding to the multiple operator parameters. For example, the extracted constraint information includes operator parameter A-operator parameter B=0, so it is simplified to the value of operator parameter A and the value of operator parameter B being the same.

[0107] In this embodiment, by simplifying and revising the initial constraint information corresponding to the multiple operator parameters, the readability of the constraint information corresponding to the multiple operator parameters can be improved, and the efficiency of generating test data can be improved.

[0108] S306 . Generate test data through deep search according to the first constraint information and the second constraint information corresponding to each operator parameter.

[0109] The constraint information corresponding to each operator parameter includes first constraint information related to the operator parameter itself, and second constraint information related to the dependency relationship between other operator parameters. Specifically, the first constraint information related to the operator parameter itself includes at least shape constraints, type constraints, and value constraints. Among them, the shape constraint is used to indicate the shape of the operator parameter, the type constraint is used to indicate the shape of the operator parameter, and the type constraint is used to indicate the type of the operator parameter. The constraint related to the dependency relationship between other operator parameters is the second constraint information. For example, the same type as a certain operator parameter, or not less than the type of a certain operator parameter, or the same shape as a certain operator parameter, etc.

[0110] Generate test data through deep search according to the first constraint information and the second constraint information corresponding to each operator parameter. First, generate each operator parameter according to the first constraint information corresponding to each operator parameter. For example, the type constraint of an operator parameter is ins type, the value constraint of an operator parameter is greater than 0 and less than 1, and the shape of an operator parameter is a 3×4 vector. Furthermore, by continuously covering new operator parameters through deep search, test data that meets all constraints is iteratively generated. For example, the operator parameter is of the same type as a certain operator parameter, so after determining the type of the operator parameter, the operator parameter with the same type of constraint information as the operator parameter is determined.

[0111] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data.

[0112] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0113] See also Figure 6, which shows a schematic diagram of the structure of a test data generating device provided by an exemplary embodiment of the present application. The test data generating device can be implemented as all or part of the device through software, hardware or a combination of both. The test data generating device includes a data acquisition module 401, a constraint acquisition module 402 and a data generation module 403.

[0114] A data acquisition module 401 is used to acquire the source points and the sink points of the data flow of the code data constituting the target operator when executing the computing task, and the number of the source points and the number of the sink points are at least one respectively;

[0115] A constraint acquisition module 402, configured to extract constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data;

[0116] The data generation module 403 is used to obtain test data according to the constraint information respectively corresponding to the multiple operator parameters, and the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data.

[0117] In one embodiment, the test data generating device comprises:

[0118] The code compiling module is used to compile the original code data constituting the target operator from the original format into an intermediate representation (IR) to obtain the code data constituting the target operator.

[0119] In one embodiment, the data acquisition module 401 includes:

[0120] A rule determination unit is used to determine the location of sub-code data in the code data that meets the first preset rule as the source point based on the data flow of the code data constituting the target operator when executing a computing task, and to determine the location of sub-code data in the code data that meets the second preset rule as the convergence point.

[0121] In one embodiment, the second preset rule includes sub-code data for representing a conditional jump instruction, and target sub-code data at a jump destination point for representing an error output instruction or a calculation check instruction.

[0122] In one embodiment, the first preset rule includes sub-code data for representing preset instructions, and the preset instructions include at least one of the following instructions: a load instruction, a store instruction, and a pointer instruction.

[0123] In one embodiment, both the first preset rule and the second preset rule include that the sub-code data includes a taint operator parameter.

[0124] In one embodiment, the test data generating device comprises:

[0125] a first judging unit, configured to determine, when judging that the position of the sub-code data satisfying the second preset rule is an external position of the code data, a position of the sub-code data that is located at the external position of the code data and satisfies the second preset rule as a loop convergence point;

[0126] The second judgment unit is used to determine that the position of the sub-code data that satisfies the second preset rule is an internal position of the code data, and that the position of the sub-code data that is in the internal position of the code data and satisfies the second preset rule is a common convergence point, and the convergence point includes the loop convergence point and the common convergence point.

[0127] In one embodiment, the test data generating device comprises:

[0128] The original acquisition module is used to acquire the original code data constituting the target operator according to the position of the target operator in the machine learning model.

[0129] In one embodiment, the constraint acquisition module 402 includes:

[0130] A constraint extraction unit is used to extract constraint information corresponding to multiple operator parameters included in the target operator based on feature information of the target instruction represented by the sub-code data located at the source point in the code data and feature information of the target instruction represented by the sub-code data located at the convergence point.

[0131] In one embodiment, the constraint extraction unit includes:

[0132] A first extraction unit is used to extract initial constraint information corresponding to a plurality of operator parameters included in the target operator according to feature information of the target instruction represented by the sub-code data located at the source point in the code data and feature information of the target instruction represented by the sub-code data located at the convergence point;

[0133] The second extraction unit is used to simplify and revise the initial constraint information corresponding to the multiple operator parameters to obtain the constraint information corresponding to the multiple operator parameters.

[0134] In one embodiment, the constraint information corresponding to each of the operator parameters includes first constraint information related to the operator parameter itself, and also includes second constraint information related to the dependency relationship between other operator parameters.

[0135] In one embodiment, the data generation module 403 includes:

[0136] The data generating unit is used to generate the test data through deep search according to the first constraint information and the second constraint information corresponding to each operator parameter.

[0137] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data.

[0138] It should be noted that the test data generation device provided in the above embodiment only uses the division of the above functional modules as an example when executing the test data generation method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the test data generation device provided in the above embodiment and the test data generation method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0139] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0140] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1 - Figure 5 The test data generation method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 - Figure 5 The specific description of the illustrated embodiment will not be repeated here.

[0141] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 - Figure 5 The test data generation method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 - Figure 5 The specific description of the illustrated embodiment will not be repeated here.

[0142] See also Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0143] The communication bus 502 is used to realize the connection and communication between these components.

[0144] The user interface 503 may include a display screen (Display) and a camera (Camera), and the optional user interface 503 may also include a standard wired interface and a wireless interface.

[0145] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0146] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts within the entire server 500, and executes various functions and processes data of the server 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.

[0147] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. As Figure 7 As shown, the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a test data generation application.

[0148] exist Figure 7 In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 501 can be used to call the test data generation application stored in the memory 505, and specifically perform the following operations:

[0149] Obtaining source points and sink points of a data flow of code data constituting a target operator when executing a computing task, wherein the number of the source points and the number of the sink points are at least one;

[0150] Extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data;

[0151] According to the constraint information respectively corresponding to the multiple operator parameters, test data is obtained, and the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data.

[0152] In one embodiment, before the processor 505 executes the step of obtaining the code data constituting the target operator and the source and sink points of the data flow when executing the computing task, it further executes:

[0153] The original code data constituting the target operator is compiled from the original format into an intermediate representation (IR) to obtain the code data constituting the target operator.

[0154] In one embodiment, the processor 505 performs the step of obtaining the source point and the convergence point of the data flow of the code data constituting the target operator when executing the computing task, specifically performing:

[0155] According to the data flow of the code data constituting the target operator when executing the computing task, the position of the sub-code data in the code data that meets the first preset rule is determined as the source point, and the position of the sub-code data in the code data that meets the second preset rule is determined as the convergence point.

[0156] In one embodiment, the second preset rule includes sub-code data for representing a conditional jump instruction, and target sub-code data at a jump destination point for representing an error output instruction or a calculation check instruction.

[0157] In one embodiment, the first preset rule includes sub-code data for representing preset instructions, and the preset instructions include at least one of the following instructions: a load instruction, a store instruction, and a pointer instruction.

[0158] In one embodiment, both the first preset rule and the second preset rule include that the sub-code data includes a taint operator parameter.

[0159] In one embodiment, after the processor 505 executes the data flow of the code data constituting the target operator when executing the computing task, determines the location of the sub-code data in the code data that satisfies the first preset rule as the source point, and determines the location of the sub-code data in the code data that satisfies the second preset rule as the convergence point, it further executes:

[0160] In the case where it is determined that the position of the subcode data satisfying the second preset rule is an external position of the code data, determining the position of the subcode data satisfying the second preset rule which is located at the external position of the code data as a loop convergence point;

[0161] Or, when it is determined that the position of the sub-code data that satisfies the second preset rule is the internal position of the code data, the position of the sub-code data that is in the internal position of the code data and satisfies the second preset rule is determined to be a common convergence point, and the convergence point includes the loop convergence point and the common convergence point.

[0162] In one embodiment, before the processor 505 compiles the original code data constituting the target operator from the original format into the intermediate representation to obtain the code data constituting the target operator, it further executes:

[0163] According to the position of the target operator in the machine learning model, the original code data constituting the target operator is obtained.

[0164] In one embodiment, the processor 505 executes the step of extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data, specifically performing:

[0165] Based on the feature information of the target instruction represented by the sub-code data located at the source point in the code data, and the feature information of the target instruction represented by the sub-code data located at the convergence point, constraint information corresponding to multiple operator parameters included in the target operator is extracted.

[0166] In one embodiment, the processor 505 executes the feature information of the target instruction represented by the sub-code data located at the source point in the code data and the feature information of the target instruction represented by the sub-code data located at the convergence point, extracting the constraint information corresponding to the multiple operator parameters included in the target operator, specifically performing:

[0167] Extracting initial constraint information corresponding to a plurality of operator parameters included in the target operator, respectively, according to feature information of the target instruction represented by the sub-code data located at the source point in the code data and feature information of the target instruction represented by the sub-code data located at the convergence point;

[0168] The initial constraint information respectively corresponding to the plurality of operator parameters is simplified and revised to obtain the constraint information respectively corresponding to the plurality of operator parameters.

[0169] In one embodiment, the constraint information corresponding to each of the operator parameters includes first constraint information related to the operator parameter itself, and second constraint information related to the dependency relationship between other operator parameters.

[0170] In one embodiment, the processor 505 executes the constraint information corresponding to the plurality of operator parameters to obtain test data, specifically performing:

[0171] The test data is generated through a deep search according to the first constraint information and the second constraint information corresponding to each operator parameter.

[0172] This application obtains the source point and convergence point of the data flow of the code data of the target operator when executing the computing task, extracts the constraint information corresponding to the multiple operator parameters included in the target operator, and further obtains the test data according to the constraint information corresponding to the multiple operator parameters. In other words, this application can obtain the constraint information of the operator parameters more comprehensively through the sub-code data corresponding to the source point and the convergence point, thereby ensuring the integrity of the test data obtained according to the constraint information of the multiple operator parameters, that is, the test data can meet the verification conditions for the operator parameters, improve the code coverage of the target operator when executing the computing task corresponding to the test data, thereby improving the test efficiency, avoiding the problem of blindly mutating the test data in the traditional fuzzy test to produce a large amount of illegal data, and further, continuously testing the security risks of the target operator in the machine learning model through the test data to ensure the structural security and application security of the machine learning model and the security of user data.

[0173] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0174] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A test data generation method, characterized in that: The method comprises: Obtaining source points and sink points of a data flow of code data constituting a target operator when executing a computing task, wherein when the target operator executes the computing task, the data flow generated by the calculation starts from at least one source point, converges to at least one sink point, and is output, and the number of the source point and the number of the sink point are at least one respectively; Extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data; Obtaining test data according to the constraint information respectively corresponding to the plurality of operator parameters, wherein the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data; The step of extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the sub-code data located at the source point and the sub-code data located at the convergence point in the code data includes: Based on the characteristic information of the target instruction represented by the sub-code data located at the source point in the code data, and the characteristic information of the target instruction represented by the sub-code data located at the convergence point, the constraint information corresponding to the multiple operator parameters included in the target operator is extracted; wherein the characteristic information represents the meaning of the target instruction or instructs the target operator to perform the calculation content of the calculation task.

2. The test data generation method according to claim 1, characterized in that: The step of obtaining the code data constituting the target operator before the source point and the convergence point of the data flow when executing the computing task also includes: The original code data constituting the target operator is compiled from the original format into an intermediate representation (IR) to obtain the code data constituting the target operator.

3. The test data generating method according to claim 2, characterized in that: The step of obtaining the source point and the convergence point of the data flow of the code data constituting the target operator when executing the computing task includes: According to the data flow of the code data constituting the target operator when executing the computing task, the position of the sub-code data in the code data that meets the first preset rule is determined as the source point, and the position of the sub-code data in the code data that meets the second preset rule is determined as the convergence point.

4. The test data generating method according to claim 3, characterized in that: The second preset rule includes that the sub-code data is used to represent the conditional jump instruction, and the target sub-code data located at the jump destination point is used to represent the error output instruction or the calculation check instruction.

5. The test data generating method according to claim 3, characterized in that: The first preset rule includes sub-code data for representing preset instructions, and the preset instructions include at least one of the following instructions: a load instruction, a store instruction, and a pointer instruction.

6. The test data generating method according to claim 3, characterized in that: The first preset rule and the second preset rule both include that the sub-code data includes a stain operator parameter.

7. The test data generating method according to claim 3, characterized in that: After determining, according to the data flow of the code data constituting the target operator when executing the computing task, the location of the sub-code data satisfying the first preset rule in the code data as the source point, and determining the location of the sub-code data satisfying the second preset rule in the code data as the convergence point, the method further includes: In the case where it is determined that the position of the subcode data satisfying the second preset rule is an external position of the code data, determining the position of the subcode data satisfying the second preset rule which is located at the external position of the code data as a loop convergence point; Or, when it is determined that the position of the sub-code data that satisfies the second preset rule is the internal position of the code data, the position of the sub-code data that is in the internal position of the code data and satisfies the second preset rule is determined to be a common convergence point, and the convergence point includes the loop convergence point and the common convergence point.

8. The test data generating method according to claim 2, characterized in that: Before compiling the original code data constituting the target operator from the original format into the intermediate representation to obtain the code data constituting the target operator, the method further includes: According to the position of the target operator in the machine learning model, the original code data constituting the target operator is obtained.

9. The test data generating method according to claim 1, characterized in that: The step of extracting constraint information corresponding to a plurality of operator parameters included in the target operator according to the feature information of the target instruction represented by the sub-code data located at the source point in the code data and the feature information of the target instruction represented by the sub-code data located at the convergence point comprises: Extracting initial constraint information corresponding to a plurality of operator parameters included in the target operator, respectively, according to feature information of the target instruction represented by the sub-code data located at the source point in the code data and feature information of the target instruction represented by the sub-code data located at the convergence point; The initial constraint information respectively corresponding to the plurality of operator parameters is simplified and revised to obtain the constraint information respectively corresponding to the plurality of operator parameters.

10. The test data generating method according to claim 1, characterized in that: The constraint information corresponding to each of the operator parameters includes first constraint information related to the operator parameter itself, and also includes second constraint information related to the dependency relationship between other operator parameters.

11. The test data generating method according to claim 10, characterized in that: The obtaining test data according to the constraint information respectively corresponding to the plurality of operator parameters comprises: The test data is generated through a deep search according to the first constraint information and the second constraint information corresponding to each operator parameter.

12. A test data generating device, characterized in that: The device comprises: A data acquisition module, used to acquire the source points and the sink points of the data flow of the code data constituting the target operator when executing the computing task. When the target operator executes the computing task, the data flow generated by the calculation starts from at least one source point, converges to at least one sink point, and is output. The number of the source point and the number of the sink point are at least one respectively. A constraint acquisition module, configured to extract constraint information corresponding to a plurality of operator parameters included in the target operator, based on the sub-code data located at the source point and the sub-code data located at the convergence point in the code data; A data generation module, used to obtain test data according to the constraint information respectively corresponding to the plurality of operator parameters, wherein the test data is used to test whether an abnormality occurs when the target operator executes the computing task corresponding to the test data; Among them, the constraint acquisition module includes: A constraint extraction unit is used to extract constraint information corresponding to multiple operator parameters included in the target operator based on feature information of the target instruction represented by the sub-code data located at the source point in the code data, and feature information of the target instruction represented by the sub-code data located at the convergence point; wherein the feature information represents the meaning of the target instruction or instructs the target operator to perform the calculation content of the calculation task.

13. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 11.

14. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 11.

Images