A method and device for program synthesis

Abstract

The present application relates to the field of computer technology, and discloses a method and device for program synthesis. The method includes: obtaining and screening TensorFlow mathematical calculation expressions that can be formalized in a satisfiability modulo theory (SMT) solver; encoding the above mathematical expressions in satisfiability modulo theory; encoding methods include broadcasting method, reverse order traversal, multiple traversal, double pointer method, divide and conquer calculation, group processing, and unrolling loop; introducing constraints that satisfy the program synthesis method; the user provides the input and output of the TensorFlow program that meets his expectations, and finally synthesizes a program that meets the input and output constraints through this method. The method has higher synthesis efficiency and stronger scalability, and provides new ideas and methods for the field of program synthesis.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to a method and device for program synthesis. Background Art

[0002] In the field of program synthesis, traditional enumeration methods face many challenges, especially when dealing with complex programs, calling library functions or external modules, and guessing large constants. These challenges limit the effectiveness and scalability of enumeration methods.

[0003] As the length and grammatical complexity of programs increase, traditional enumeration methods need to exhaustively search all possible programs, resulting in an exponential expansion of the search space. This expansion makes enumeration methods impractical when dealing with complex programs because the computing resources and time costs will increase rapidly.

[0004] Traditional enumeration methods are difficult to effectively extend to large-scale programs that need to call library functions or external modules. These methods are usually unable to directly utilize existing library functions or modules, resulting in the need to repeatedly implement the same functions during the synthesis process, increasing the complexity and workload of the synthesis program.

[0005] When it is necessary to guess a large, ungiven constant, traditional enumeration methods are extremely inefficient. Since there is no way to use input data to narrow down the possible range of the constant, enumeration methods need to traverse all possible constant values, which is unrealistic in practical applications.

[0006] To address these problems, an innovative component program synthesis approach is needed.

[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0008] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0009] The embodiments of the present disclosure provide a method and apparatus for program synthesis, which solve the above problems and improve the accuracy and efficiency of program synthesis.

[0010] In some embodiments, the method comprises:

[0011] Obtain and filter TensorFlow mathematical calculation expressions that satisfy the satisfiability modulo theory. The expressions obtained are synthesized based on component programs without loops, and expressions with variable loop times or loop times that have an upper limit that is independent of the input scale are excluded.

[0012] The screened mathematical calculation expressions are encoded modulo the satisfiability theory;

[0013] Introduce constraints that satisfy the program synthesis method;

[0014] Receive the input and output of the TensorFlow program, and use the encoded mathematical calculation expressions and the introduced constraints to synthesize a TensorFlow program that satisfies the input and output constraints.

[0015] Preferably, the screened mathematical calculation expressions are encoded using the satisfiability modulo theory, specifically in the following manner:

[0016] Determine the type of mathematical calculation expression, and encode the addition expression, subtraction expression, multiplication expression, division expression, size relationship judgment expression, maximum value expression, and conditional branch expression by broadcasting method;

[0017] The maximum value subscript expression and the search position expression are traversed and encoded in reverse order;

[0018] Perform multiple traversal encoding on slice expressions, expressions for counting the number of occurrences, expressions for updating content according to subscripts, and rolling expressions;

[0019] Encode the mask expression and the expression for filtering non-zero values ​​using the double pointer method;

[0020] Encode the iterative processing expression by divide-and-conquer calculation;

[0021] The segment-by-segment maximum value expression and the group counting expression are grouped and processed and encoded;

[0022] Unroll the repetition expansion expression into an unrolled loop encoding.

[0023] Preferably, the broadcast method is specifically encoded as follows:

[0024] Get the values ​​of two dimensions from all inputs. If the values ​​of the same dimension of all inputs are equal, the broadcast method encoding condition is met, but the broadcast method encoding is not performed. If the value of the dimension of at least one input in the same dimension is 1, and the values ​​of the dimension of other inputs are not all 1, and the values ​​of the dimensions of all other inputs that are not 1 are equal, then the broadcast method encoding condition is met, and the input with the smaller dimension value is broadcasted. The broadcast method encoding method is as follows: Get the input value of the dimension whose dimension value is 1, copy the value, and fill the value in the direction of the dimension growth until the dimension value is equal to the value of the same dimension of other inputs.

[0025] The program synthesis is terminated for the input that does not meet the encoding conditions of the broadcast method.

[0026] Preferably, the specific method of the reverse traversal encoding is as follows:

[0027] Get the values ​​of two dimensions in the input to be processed, start from the maximum dimension of the input, and traverse in the direction of decreasing values. During the traversal, if there are multiple results that satisfy the expression, use the value of the position with a smaller dimension value to overwrite the value of the position with a larger dimension value.

[0028] Preferably, the multiple traversal coding includes an outer traversal and an inner traversal, and the outer traversal is as follows: obtain a dimension value of the input to be processed, set the loop variable value to 0, start from the dimension value 0, and traverse in the direction of increasing value until it is equal to the dimension value;

[0029] The inner layer traversal method is as follows: obtain another dimension value that needs to be processed, set the loop variable value to 0, start from the dimension value 0, and traverse in the direction of increasing value until it is equal to the dimension value;

[0030] When the dimension value encountered by the inner traversal is equal to the dimension value of the input that needs to be processed, obtain the input value corresponding to the current dimension value of the outer traversal, compare the value of the loop variable in the inner traversal with the dimension value of the input that needs to be processed, if the two are not equal, the value of the dimension position in the output remains unchanged, if the two are equal, process according to the expression semantics.

[0031] Preferably, the double pointer encoding method is as follows:

[0032] Set two variables. The first variable records the location where the result obtained by expression semantic calculation is saved in the output, and the second variable saves the value of the first variable.

[0033] If the value in the input corresponding to the current dimension value does not satisfy the expression semantics, the first variable is set to a value that exceeds the dimension value range, and the second variable is incremented by 1; otherwise, the value of the second variable is assigned to the first variable, and the second variable remains unchanged; the result is stored in the output according to the value of the first variable, and the storage location is the same as the value of the first variable.

[0034] Preferably, the divide-and-conquer computing encoding method is as follows:

[0035] For two dimensions of an input, each dimension is grouped equally;

[0036] Complete semantic calculations that conform to the expression within each group. When the semantics of the expression satisfies the commutative law and the associative law, and the output has only a single value, processing all input contents is equivalent to processing the input contents within the group; the commutative law means that the output result is independent of the position of the input in the expression, and the associative law means that the output result is independent of the order of the input in the expression processing, and the single value is that the values ​​of the two dimensions of the output are both 1;

[0037] The semantic calculation of the expression is completed between groups. When the semantics of the expression satisfies the commutative law and the associative law, and the output has only a single value, processing all input contents is equivalent to processing the input contents between groups.

[0038] Preferably, the group processing encoding method is as follows:

[0039] For the input group information that needs to be processed, record the identifier of the group information, and process the input content according to the expression semantics based on the group information. If the corresponding group information is found according to the group information identifier, process it according to the same group content, otherwise do not process it.

[0040] Preferably, the unfolded loop encoding method is as follows:

[0041] Get the number of times the loop needs to be repeated, and repeatedly write the code that satisfies the semantics of the expression. The code is semantically equivalent to the loop operation, and the number of times the code is written is equal to the number of loops.

[0042] In some embodiments, the device includes: a processor and a memory storing program instructions, and the processor is configured to execute the method for program synthesis when running the program instructions.

[0043] The method and device for program synthesis provided by the embodiments of the present disclosure can achieve the following technical effects:

[0044] This method can obtain and filter TensorFlow mathematical expressions that can be formalized in the Satisfiability Modulo Theory (SMT) solver, and encode these mathematical expressions in SMT theory. By introducing constraints that satisfy the program synthesis method, users only need to provide the input and output of the TensorFlow program that meets their expectations, and finally synthesize a program that meets the input and output constraints.

[0045] The encoding methods include broadcasting method, reverse order traversal, multiple traversal, double pointer method, divide and conquer calculation, group processing, unrolling loop and other strategies. These strategies can efficiently process various TensorFlow mathematical calculation expressions and improve the efficiency and accuracy of program synthesis.

[0046] By decomposing the program into components, the search space is limited. Each time, you only need to search in the combination space of components, rather than all possible programs. This greatly reduces the search space that needs to be traversed and improves synthesis efficiency. The component program synthesis method directly uses components (such as function libraries or operators) as basic units, and naturally supports calling library functions or reusing existing modules. This allows the synthesized program to easily utilize existing library functions and modules, reducing the workload of repeated implementation and improving the reusability and maintainability of the program.

[0047] The component program synthesis method decomposes the program into modular components and uses the combination of these components to generate the required program, effectively solving the challenges faced by traditional enumeration methods when dealing with complex programs, calling library functions or external modules, and guessing large constants. This method has higher synthesis efficiency and stronger scalability, and provides new ideas and methods for the field of program synthesis.

[0048] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0050] Figure 1 is a schematic diagram for program synthesis provided by an embodiment of the present disclosure;

[0051] Figure 2 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0052] Figure 3 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0053] Figure 4 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0054] Figure 5 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0055] Figure 6 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0056] Figure 7 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0057] Figure 8 is a schematic diagram for encoding a mathematical expression using satisfiability modulo theory provided by an embodiment of the present disclosure;

[0058] Fig. 9 It is a schematic diagram of a device for program synthesis provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0060] The terms "first", "second", etc. in the description of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0061] Unless otherwise stated, the term "plurality" means two or more.

[0062] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0063] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0064] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0065] The embodiments of the present disclosure do not limit the execution subject of the method for program synthesis. For example, the method for program synthesis of the embodiments of the present disclosure can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device refers to an electronic device with a wireless connection function. In some embodiments, the terminal device is, for example, a mobile device, a computer, or a vehicle-mounted device built into a hover car, or any combination thereof. Mobile devices may include, for example, mobile phones, smart home devices, wearable devices, smart mobile devices, virtual reality devices, etc., or any combination thereof, wherein wearable devices include, for example: smart watches, smart bracelets, pedometers, etc.

[0066] Combination Figure 1 As shown, the present disclosure provides a method for program synthesis, including:

[0067] S101: Obtain and filter TensorFlow mathematical computation expressions that can be formalized in the satisfiability modulo theory solver.

[0068] The expressions obtained are synthesized based on loop-free component programs, excluding expressions with non-fixed loop times or loop times with an upper limit that is independent of the input size.

[0069] The disclosed examples are based on loop-free component program synthesis, and therefore exclude expressions whose loop times are not fixed or whose upper limit of loop times is independent of the input size.

[0070] S102: Encode the above mathematical expression using satisfiability modulo theory.

[0071] In the disclosed embodiment, the encoding method includes a broadcast method, a reverse order traversal, a multiple traversal, a double pointer method, a divide and conquer calculation, a group processing, and an unrolled loop.

[0072] S103: Introduce constraints that satisfy the program synthesis method.

[0073] S104: The user provides the input and output of the TensorFlow program that meets his or her expectations, and finally a program that meets the input and output constraints is synthesized through this method.

[0074] As a refinement of the above embodiment, the selected mathematical calculation expressions are encoded modulo satisfiability theory in the following specific manner:

[0075] Determine the type of mathematical calculation expression, and perform broadcast method encoding on addition expression, subtraction expression, multiplication expression, division expression, size relationship judgment expression, maximum value expression, and conditional branch expression.

[0076] The maximum value subscript expression and the search position expression are encoded in reverse order traversal.

[0077] Multiple traversal encoding is performed on slice expressions, expressions for counting the number of occurrences, expressions for updating content according to subscripts, and rolling expressions.

[0078] Encode the mask expression and filter non-zero value expression using double pointer method.

[0079] Encode iterative processing expressions using divide-and-conquer computation.

[0080] The segment-by-segment maximum value expression and the group counting expression are grouped and processed and encoded.

[0081] Unroll the repetition expansion expression into an unrolled loop encoding.

[0082] In the embodiments of the present disclosure, for ease of understanding, the above seven encoding methods may be exemplarily described in conjunction with the accompanying drawings respectively.

[0083] Specifically, combined Figure 2 As shown, the encoding process of the broadcast method in the satisfiability modulo theory encoding, that is, step S102, may specifically include steps 21A to 23A:

[0084] Step 21A: Get the values ​​of the two dimensions in all inputs.

[0085] In the embodiment of the present disclosure, each input explicitly stores numerical values ​​of two dimensions, and the numerical values ​​are equal to the actual dimensional values ​​of each input.

[0086] Step 22A: Determine whether the current input meets the broadcast method encoding conditions.

[0087] Specifically, if the values ​​of the same dimension of all inputs are equal, the broadcast method encoding condition is met, but the broadcast method encoding is not performed; if there is an input with a dimension value of 1, and the dimension of the input dimension that is not 1 is equal to the values ​​of the same dimension of other inputs, the broadcast method encoding condition is met; if any of the above conditions is not met, the broadcast method encoding condition is not met.

[0088] Step 23A: Perform broadcast encoding operation on all inputs.

[0089] Specifically, if the broadcast method encoding conditions are not met, the current program synthesis is terminated and the next synthesis is prepared. The specific operations are: all dimensions of all inputs are set to 0. At this time, the solution status of the satisfiability modulo theory solver is not satisfied, so this program synthesis is terminated; if the broadcast method encoding conditions are met, all inputs with dimension values ​​of 1 are found and the values ​​of the inputs under this dimension are saved, and the values ​​are filled in the order of increasing dimensions until the dimension value is equal to the values ​​of the same dimension of other inputs.

[0090] For example: There are three inputs ,enter The value of the first dimension is 1, the value of the second dimension is 3, and the input The values ​​of the first and second dimensions are both 3, input The values ​​of the first and second dimensions of are both 3; then the input The values ​​entered under the first dimension of are filled in the direction of increasing dimension until the value entered The first dimension of is 3.

[0091] Based on the above steps 21A-23A, in the disclosed embodiment, mathematical expressions involving element-by-element operations and supporting automatic expansion of array shapes can be encoded with satisfiability modulo theory, where the situations that support automatic expansion of array shapes include: scalar and array operations, operations between arrays of different shapes. Specifically, it can include: addition expression (tf.add), subtraction expression (tf.subtract), multiplication expression (tf.multiply), division expression (tf.divide), size relationship judgment expression (tf.greater, tf.greater_equal), maximum value expression (tf.maximum, tf.minimum), conditional branch expression (tf.where, with three parameters). Therefore, the disclosed embodiment can synthesize a TensorFlow program that meets the user's expectations, reduce the user's learning difficulty and cost, and make it faster to get started.

[0092] Combination Figure 3 As shown, the encoding process of reverse traversal in satisfiability modulo theory encoding, that is, step S102, can specifically include steps 21B to 23B:

[0093] Step 21B: Obtain the value of the corresponding dimension in the input to be processed.

[0094] In the embodiment of the present disclosure, each input explicitly stores values ​​of two dimensions, records the value of the dimension that needs to be processed later, and performs subsequent operations on this dimension.

[0095] Step 22B: Process the input values ​​of the corresponding dimension values ​​in the direction of decreasing values.

[0096] Specifically, first determine the size of the current dimension value relative to 0. If it is less than 0, terminate the operation. Secondly, set a variable to record the current dimension value, obtain the value entered under the current dimension value, and perform operations according to the expression semantics. Finally, reduce the value of the variable by 1, and repeat the above operations until the operation terminates.

[0097] Step 23B: Terminate the process when the dimension value is less than 0.

[0098] Based on the above steps 21B-23B, in the disclosed embodiment, there are multiple solutions that satisfy the semantics of the expression, but it is necessary to find the first solution, and traversing from back to front can more effectively shorten the search range or avoid unnecessary operations. Mathematical expressions can be encoded using the satisfiability modulo theory, which is particularly suitable for scenarios that need to minimize time complexity or determine the answer as early as possible. Specifically, it can include: finding the maximum value subscript expression (tf.math.argmax), searching for the position expression (tf.searchsorted). In addition, this method can also process other expressions involving recording the subscripts corresponding to tensor values. Therefore, the disclosed embodiment can synthesize a TensorFlow program that meets the user's expectations, reduce the user's learning difficulty and cost, and make it faster to get started.

[0099] Combination Figure 4 As shown, the encoding process of multiple traversals in the satisfiability modulo theory encoding, that is, step S102, includes inner traversal and outer traversal, which may specifically include steps 21C to 23C:

[0100] Step 21C: Select the value of the dimension whose input needs to be processed, starting from 0 and incrementing to this value.

[0101] Step 22C: During the operation of step 21C, each time the value is incremented and before the next increment, the operation of step 21C is repeated.

[0102] Specifically, the two operations are consistent, both of which select the dimension to be processed, obtain the value of the dimension, set the initial value of the loop variable to 0, and increment it to the value in sequence, with a step of 1 each time, until it equals the value of the dimension. The difference lies in the triggering timing, which is after the variable is set in step 21C and before the value is incremented.

[0103] Step 23C: When the value of the variable in step 22C is equal to the expected dimension position (the dimension value of the input dimension needs to be processed), obtain the value of the variable in step 21C corresponding to the input dimension position and process it.

[0104] Specifically, when step 21C and step 22C are repeatedly executed, each time after setting the variable and before incrementing the value, the value of the variable in step 22C is compared with the expected dimension position. If the two are not equal, no processing is performed, which is specifically manifested in that the value of the dimension position in the output remains unchanged; if the two are equal, processing is performed according to the expression semantics.

[0105] Based on the above steps 21C to 23C, in the disclosed embodiment, situations where array contents cannot be directly located and modified by subscripts and need to be processed in stages to ensure the order or integrity of the acquired and modified data can be encoded with satisfiability modulo theory, which can specifically include: slicing expressions (tf.gather, tf.gather_nd), expressions for counting the number of occurrences (tf.math.bincount), expressions for updating content according to subscripts (tf.SparseTensor and tf.sparse.to_dense, tf.tensor_scatter_nd_update), and rolling expressions (tf.roll). This method is applicable to situations where the corresponding values ​​cannot be processed directly according to the semantics of the expression based on the position of the input content. Therefore, the disclosed embodiment can synthesize a TensorFlow program that meets the user's expectations, reduce the difficulty and cost of learning for the user, and make it easier to get started.

[0106] Combination Figure 5 As shown, the encoding process of the double pointer method in the satisfiability modulo theory encoding, that is, step S102, can specifically include steps 21D to 23D:

[0107] Step 21D: Set up two variables to record the location of the processed input values.

[0108] Specifically, two variables are set with initial values ​​of 0. The first variable records the location where the result obtained by semantic calculation of the expression is saved in the output, and the second variable saves the value of the first variable.

[0109] Step 22D: Get the value of the current input dimension, starting from 0 and incrementing to this value.

[0110] Specifically, get the value of the current input dimension, set a variable with an initial value of 0, and increment it by a step of 1 until it equals the value of the current input dimension. This variable records the position of the currently processed input value.

[0111] Step 23D: Process the two variables in step 21D according to the input content and store the result in the output.

[0112] Specifically, the expression semantics is used to determine whether the input at the current position satisfies the condition. If not, the first variable is set to -1 and the second variable is incremented by 1. If satisfied, the value of the second variable is assigned to the first variable and the second variable remains unchanged. The result is then stored in the output according to the value of the first variable, and the storage location is the same as the value of the first variable.

[0113] Based on the above steps 21D-23D, in the embodiment of the present disclosure, when the shape of the output array depends on the input content and needs to be dynamically adjusted, the mathematical expression of the result generation and state tracking can be managed by using a double pointer mechanism for satisfiability modulo theory encoding. This type of method is often used in scenarios where dynamic states need to be maintained, intermediate results need to be stored, or backtracking operations need to be performed. Specifically, it may include: mask expressions (tf.boolean_mask), expressions for filtering non-zero values ​​(tf.where, with only one parameter). This method is suitable for expressions whose output dimensions are not fixed and are related to the input content. Therefore, the embodiment of the present disclosure can synthesize a TensorFlow program that meets the user's expectations, reduce the user's learning difficulty and cost, and make it easier to get started.

[0114] Combination Figure 6 As shown, the encoding process of divide-and-conquer calculation in satisfiability modulo theory encoding, that is, step S102, may specifically include steps 21E to 23E:

[0115] Step 21E: Group the input into equal parts.

[0116] Specifically, first determine the dimension of the input that needs to be processed, obtain the value of the dimension, determine the number of equally divided groups, and divide the input equally along the direction of the dimension.

[0117] Step 22E: Process the input in the group according to the expression semantics.

[0118] When the expression semantics satisfies the commutative and associative laws, and the output has only a single value, processing all inputs is equivalent to processing the inputs in the group. The commutative law means that the output result is independent of the position of the inputs in the expression, and the associative law means that the output result is independent of the order in which the inputs are processed in the expression. A single value means that the values ​​of both dimensions of the output are 1.

[0119] Step 23E: Process inter-group input according to expression semantics.

[0120] Change the content of the input within the processing group to the content of the input between the processing groups, and the rest is the same as step 22E.

[0121] Based on the above steps 21E-23E, in the embodiment of the present disclosure, the output can be a scalar and satisfy the commutative law and the distributive law. The mathematical expression of the calculation can be optimized by grouping the processing steps to perform satisfiability modulo theory encoding. This method is usually used for aggregation operations, simplified calculations of large amounts of data, or step-by-step solutions of complex expressions. Specifically, it can include: iterative processing expressions (tf.math.reduce_max, tf.math.reduce_min, tf.math.reduce_sum, tf.math.reduce_prod, tf.math.reduce_mean, where the parameter axis=None). This method is applicable to situations where the expression semantics satisfy the associative law and the commutative law, and the output has only a single value. Therefore, the embodiment of the present disclosure can synthesize a TensorFlow program that meets the user's expectations, reduce the user's learning difficulty and cost, and make it faster to get started.

[0122] Combination Figure 7 As shown, the encoding process for grouping processing in satisfiability modulo theory encoding, that is, step S102, may specifically include steps 21F to 23F:

[0123] Step 21F: Set a collection to save the grouping information.

[0124] Specifically, if the grouping information is ordered, a variable can be set to record the group number of the currently processed input content; if the grouping information is unordered, a collection can be set to store the group number.

[0125] In the above description, ordered means that the grouping information is all integer type, and the grouping information is arranged in a monotonically non-increasing or monotonically non-decreasing order; a set refers to a combination of two different numbers.

[0126] Step 22F: Based on the grouping information in the set, process the input content according to the expression semantics.

[0127] Specifically, if the grouping information belongs to the ordered state in step 21F, the variable mentioned in the step is used. If the variable does not change, it is in the same group, otherwise it is considered to enter a new group and process it; if the grouping information belongs to the unordered state in step 21F, the set mentioned in the step is used. Before each processing, it is determined whether the current group number exists in the set. If so, the result corresponding to the group number is taken out and processing is continued according to the expression semantics. Otherwise, it is considered to be a new group and the group number is placed in the set.

[0128] Step 23F: Fill the results into the output according to the grouping information.

[0129] Specifically, a variable is set to record the location where the data is stored in the output after each processing. If the group information belongs to the ordered state in step 21F, the variable is incremented by one when a new group number is encountered, otherwise the variable value remains unchanged; if the group information belongs to the unordered state in step 21F, the variable is incremented by one when the current group number exists in the set, otherwise the variable value remains unchanged. Use this variable as the output location to store the processed results

[0130] Based on the above steps 21F to 23F, in the embodiment of the present disclosure, in the scenario where data needs to be grouped based on grouping information, two input arrays, one provides the basis for grouping, and the other provides a mathematical expression of the corresponding value, can be encoded by satisfiability modulo theory, which is often used in data aggregation, group statistics, shard calculations, and other situations where different processing of data needs to be performed based on categories or conditions. Specifically, it can include: segmented acquisition of the maximum value expression within the segment (tf.math.segment_max, the grouping information of this expression is ordered), and the group counting expression (tf.unique_with_counts, the grouping information of this expression is unordered). This method is applicable to the situation where one of the inputs contains grouping information and the other input content is processed according to this information. Therefore, the embodiment of the present disclosure can synthesize a TensorFlow program that meets the user's expectations, reduce the difficulty and cost of learning for the user, and make it faster to get started.

[0131] Combination Figure 8 As shown, the encoding process of expanding the loop in the satisfiability modulo theory encoding, that is, step S102, can specifically include steps 21G to 22G:

[0132] Step 21G: Get the value of the input for the loop.

[0133] Step 22G: Repeat the operation to loop through the content, the number of times equal to the loop value in step 21G.

[0134] Specifically, a variable is set to record the position of the current loop, with an initial value of 0 and a loop step of 1, until it is equal to the value of the relevant loop in step 21G. The input content related to the loop value is obtained according to the expression semantics, and the relevant content is replaced with the loop variable, and the number of replacements is the loop value.

[0135] Based on the above steps 21G-22G, in the disclosed embodiment, the scenario where the number of loops depends on the input content can be encoded by obtaining the loop value and performing a mathematical expression of a finite number of operations on demand, which is suitable for repeated data generation, conditionally dependent loop processing, and finitely repeated operation logic. Specifically, it can include: repeated expansion expression (tf.tile). This method is applicable to the case where the number of iterations is included in the input. Therefore, the disclosed embodiment can synthesize a TensorFlow program that meets the user's expectations, reduce the user's learning difficulty and cost, and make it faster to get started.

[0136] The method for program synthesis provided by the embodiment of the present disclosure is used to obtain and screen TensorFlow mathematical calculation expressions that can be formalized in the satisfiability modulo theory solver; the above mathematical expressions are encoded modulo the satisfiability theory; the encoding methods include broadcasting method, reverse order traversal, multiple traversal, double pointer method, divide and conquer calculation, group processing, and unrolling loop; introduce constraints that satisfy the program synthesis method; then, the user provides the input and output of the TensorFlow program that meets his expectations, and finally synthesizes a program that meets the input and output constraints through this method. This method can help beginners master the development of TensorFlow more quickly and conveniently.

[0137] The following is an example of the program synthesis method using a specific expression:

[0138] Given input 1, 2, 3 and output 7, there are two expressions tf.add and tf.multiply, which are addition and multiplication operations respectively. The following describes the synthesis process of the program tf.add(1, tf.multiply(2,3)):

[0139] Note: Parameter explanation: input is the given program input content; output is the given program output content; param is the content of the expression parameter; return is the result obtained after the expression is calculated; line is the relationship between the above content and components. Specifically, the final synthesis result we get is in the form of:

[0140] a=1 / / ——first line, line=1

[0141] b=2 / / ——The second line, line=2

[0142] c=3 / / ——The third line, line=3

[0143] d=tf.multiply(b,c) / / ——The fourth line, line=4

[0144] e=tf.add(a,d) / / ——The fifth line, line=5

[0145] As mentioned above, each given input and component occupies a line, and line numbers of inputs, outputs, parameters, and components are recorded (repeated ones are not recorded), thereby recording and determining the connections between components.

[0146] 1. Input and output coding: Use variables to record three inputs and outputs, namely input1=1, input2=2, input3=3, output=7.

[0147] 2. Construct logical expressions:

[0148] 2.1. Component internal coding: Write the expression in a form that the SMT solver can recognize. For example, tf.add has two parameters paramadd1 and paramadd2, and the return value is returnadd, so the coding is returnadd=paramadd1+paramadd2. tf.multiply has two parameters parammul1 and parammul2, and the return value is returnmul, so the coding is returnmul=parammul1*parammul2. (For the convenience of description, this use case does not use broadcasting. For broadcasting, see the broadcasting method in the patent application document.)

[0149] 2.2. Coding between components: Determine the variables generated between inferences based on the line number line. For example, in the above example, the two parameters of tf.multiply and the calculated result have 4 lines, and the line number of the second parameter in tf.add has been recorded as 4, so it can be inferred that the second parameter of tf.add is the result of tf.multiply calculation.

[0150] 3. Add constraints:

[0151] 3.1. The size of the line cannot exceed the sum of the number of inputs and components;

[0152] 3.2. The line corresponding to input is in [0, input.len), and the line corresponding to result is in [input.len, input.len+result.len), that is, all inputs must be listed before using component synthesis;

[0153] 3.3. Output cannot be directly obtained from input, and the corresponding lines of the two cannot be equal;

[0154] 3.4. Input cannot be assigned as the value of another input, that is, input can only be assigned by input content;

[0155] 3.5. The value of param can only be the original input or the output of other components. The input content cannot be directly assigned to param;

[0156] 3.6. User input and output constraints, which are described in process 2.

[0157] 3.7. All user-provided inputs must be used. The operator corresponding to the output must be used. If an operator is used, then the parameter of the operator is also used. If the parameter is input, then the input is also used.

[0158] 3.7.1. Since the output line is equal to a result line, it can be inferred that the result is used. Line_output==Line_result_i=>Use_result_i==true.

[0159] 3.7.2、If the result of an operator is used, then the parameters of this operator are also used. Use_result_i==true=>(Use_param_i_0==true,Use_param_i_1==true,...,Use_param_i_j==true).

[0160] 3.7.3. If a parameter is used and is obtained from the results of other operators, then those operators are also valid. (Line_result==Line_params&&Use_params==true)=>Use_result==true.

[0161] 3.7.4. If the number of lines of the parameter is equal to the number of lines of the input, and the parameter is used, then the input is also used. (Line_input==Line_params&&Use_params==true)=>Use_input.

[0162] 4. Description of the synthesis process:

[0163] 4.1. Determine all combinations, including the case where the number of components used is less than the total number of components. For example, in the previous example, given a=1, b=2, c=3, the possible combinations are: tf.add(a,b), tf.add(b,c), tf.add(a,c), tf.multiply(a,b), tf.multiply(b,c), tf.multiply(a,c), tf.add(a,tf.multiply(b,c)), tf.a dd(b,tf.multiply(a,c)), tf.add(c,tf.multiply(a,b)), tf.multiply(a,tf.add(b,c)), tf.multiply(b,tf.add(a,c)), tf.multiply(c,tf.add(a,b)), and the combination of swapping the positions of two parameters in each of the above combinations. All the forms are required as long as one of them satisfies the input-output relationship, otherwise an error is thrown.

[0164] 4.2. Calculate the combination result based on the encoded content and determine whether it is equal to the given output. It has been verified that only tf.add(a, tf.multiply(b, c)) meets the output.

[0165] 4.3. At this time, according to description 3.7, the input, output and expression used are determined. It can be seen that the result calculated by tf.multiply is 6, and the result calculated by tf.add is 7, which is equal to the output. Therefore, the result of tf.add is the final output, and tf.add is the expression used, so the parameters of tf.add are also used. Note that the second parameter in tf.add is the result of tf.multiply calculation, so tf.multiply is also the expression used, and the parameters of tf.multiply are used. Note that the first parameter of tf.add is the first input a, and the first and second parameters of tf.multiply are the second input b and the third input c, so the three inputs are also used. According to the above description, the input, output, parameters (also intermediate results), and the position of components are determined at the same time, so the final result is:

[0166] a=1

[0167] b=2

[0168] c=3

[0169] d = tf.multiply(b,c)

[0170] e = tf.add(a,d).

[0171] Combination Fig. 9 As shown, the embodiment of the present disclosure provides a device 300 for program synthesis, including a processor 304 and a memory 301. Optionally, the device may also include a communication interface 302 and a bus 303. The processor 304, the communication interface 302, and the memory 301 may communicate with each other through the bus 303. The communication interface 302 may be used for information transmission. The processor 304 may call the logic instructions in the memory 301 to execute the method for program synthesis of the above embodiment.

[0172] In addition, the logic instructions in the memory 301 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0173] The memory 301 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor 304 executes the function application and data processing by running the program instructions / modules stored in the memory 301, that is, the method for program synthesis in the above-mentioned embodiments is implemented.

[0174] The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.

[0175] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0176] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0177] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians may clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0178] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically separately, or two or more units may be integrated in one unit.

[0179] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for program synthesis, characterized in that: include: Obtain and filter TensorFlow mathematical calculation expressions that satisfy the satisfiability modulo theory. The expressions obtained are synthesized based on component programs without loops, and expressions with variable loop times or loop times that have an upper limit that is independent of the input scale are excluded. The screened mathematical calculation expressions are encoded modulo the satisfiability theory; Introduce constraints that satisfy the program synthesis method; Receive the input and output of the TensorFlow program, and use the encoded mathematical calculation expressions and the introduced constraints to synthesize a TensorFlow program that satisfies the input and output constraints; The selected mathematical calculation expressions are encoded modulo the satisfiability theory, and the specific method is as follows: Determine the type of mathematical calculation expression, and encode the addition expression tf.add, the subtraction expression tf.subtract, the multiplication expression tf.multiply, the division expression tf.divide, the size relationship judgment expression tf.greater, tf.greater_equal, the maximum value expression tf.maximum, tf.minimum, and the conditional branch expression tf.where with three parameters for broadcast method encoding; The maximum value subscript expression tf.math.argmax and the search position expression tf.searchsorted are traversed in reverse order; Perform multiple traversal encoding on the slice expressions tf.gather, tf.gather_nd, the expression for counting the number of occurrences tf.math.bincount, the expression for updating the content according to the subscript tf.SparseTensor and tf.sparse.to_dense, tf.tensor_scatter_nd_update, and the rolling expression tf.roll; Encode the mask expression tf.boolean_mask and the non-zero value screening expression tf.where using the double pointer method; Divide and conquer the iterative processing expressions tf.math.reduce_max, tf.math.reduce_min, tf.math.reduce_sum, tf.math.reduce_prod, and tf.math.reduce_mean; Encode the segment maximum value expression tf.math.segment_max and the group count expression tf.unique_with_counts for group processing; Unroll the repeated expansion expression tf.tile for loop encoding; The specific encoding method of the broadcast method is as follows: Get the values ​​of two dimensions from all inputs. If the values ​​of the same dimension of all inputs are equal, the broadcast method encoding condition is met, but the broadcast method encoding is not performed. If the value of the dimension of at least one input in the same dimension is 1, and the values ​​of the dimension of other inputs are not all 1, and the values ​​of the dimensions of all other inputs that are not 1 are equal, then the broadcast method encoding condition is met, and the input with the smaller dimension value is broadcasted. The broadcast method encoding method is as follows: Get the input value of the dimension whose dimension value is 1, copy the value, and fill the value in the direction of the dimension growth until the dimension value is equal to the value of the same dimension of other inputs. For input that does not meet the encoding conditions of the broadcast method, terminate the program synthesis; The specific method of reverse traversal encoding is as follows: Get the values ​​of two dimensions in the input to be processed, start from the maximum dimension of the input, and traverse in the direction of decreasing values. During the traversal, if there are multiple results that satisfy the expression, use the value of the position with a smaller dimension value to overwrite the value of the position with a larger dimension value; The multiple traversal coding includes an outer traversal and an inner traversal. The outer traversal is as follows: obtain a dimension value of the input to be processed, set the loop variable value to 0, start from the dimension value 0, and traverse in the direction of increasing value until it is equal to the dimension value; The inner layer traversal method is as follows: obtain another dimension value that needs to be processed, set the loop variable value to 0, start from the dimension value 0, and traverse in the direction of increasing value until it is equal to the dimension value; When the dimension value encountered by the inner traversal is equal to the dimension value of the input to be processed, obtain the input value corresponding to the current dimension value of the outer traversal, compare the value of the loop variable in the inner traversal with the dimension value of the input to be processed, if the two are not equal, the value of the dimension position in the output remains unchanged, if the two are equal, process according to the expression semantics; The double pointer encoding method is as follows: Set two variables. The first variable records the location where the result obtained by expression semantic calculation is saved in the output, and the second variable saves the value of the first variable. If the value in the input corresponding to the current dimension value does not satisfy the expression semantics, the first variable is set to a value that exceeds the dimension value range, and the second variable is incremented by 1; otherwise, the value of the second variable is assigned to the first variable, and the second variable remains unchanged; the result is stored in the output according to the value of the first variable, and the storage location is the same as the value of the first variable; The divide-and-conquer calculation encoding method is as follows: For two dimensions of an input, each dimension is grouped equally; Complete semantic calculations that conform to the expression within each group. When the semantics of the expression satisfies the commutative law and the associative law, and the output has only a single value, processing all input contents is equivalent to processing the input contents within the group; the commutative law means that the output result is independent of the position of the input in the expression, and the associative law means that the output result is independent of the order of the input in the expression processing, and the single value is that the values ​​of the two dimensions of the output are both 1; Complete semantic computations that match expressions between groups. When the semantics of the expression satisfy the commutative and associative laws, and the output has only a single value, processing all inputs is equivalent to processing inputs between groups. The group processing encoding method is as follows: For the input group information that needs to be processed, the identifier of the group information is recorded, and the input content is processed according to the expression semantics based on the group information. If the corresponding group information is found according to the group information identifier, it is processed according to the same group content, otherwise it is not processed; The unfolded loop encoding method is as follows: Get the number of times the loop needs to be repeated, and repeatedly write the code that satisfies the semantics of the expression. The code is semantically equivalent to the loop operation, and the number of times the code is written is equal to the number of loops.

2. A device for program synthesis, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for program synthesis according to claim 1 when running the program instructions.

Images