A model processing method and related equipment

Abstract

Embodiments of the present application disclose a model processing method and related equipment, which are used in the field of model solving. The method can include: obtaining a graph structure corresponding to a first model, randomly partitioning the graph structure to obtain at least two first sets, each first set including at least one first independent variable, updating the value of the first independent variable by using a Levenberg-Marquardt LM algorithm; repeatedly performing the operations of random partitioning and updating the value of the first independent variable until a convergence condition of the first model is met, and obtaining a solution of the plurality of first independent variables in the first model. The solving problem of the first model is divided, and when solving a sub-problem, it is not necessary to occupy too much memory resource; the random partitioning manner is adopted, and the efficiency of the partitioning process is ensured; the solving problem of the first model is divided into the solving problem of a model including fewer independent variables, which is beneficial to perform parallel solving and is beneficial to improve the efficiency of the solving process.

Description

Technical Field

[0001] The present application relates to the field of computer software, and in particular to a model processing method and related equipment. Background Art

[0002] Currently, many problems in computer vision, graphics, or other fields can be solved through modeling. For example, when solving problems in application scenarios such as visual positioning, image fusion, 3D reconstruction, optical flow calculation, or curve fitting, they can be solved by establishing a model and solving the model.

[0003] However, with the development of the times, the number of independent variables in the model continues to increase. In the current solution method, the entire model to be solved is used as input, which easily leads to the problem that memory resources cannot support the solution. Summary of the Invention

[0004] An embodiment of the present application provides a model processing method and related equipment, which divides the solution problem of a first model including a large number of independent variables, so that there is no need to occupy too many memory resources when solving a sub-problem; in addition, the graph structure is divided by random division, and the implementation method is simple, thereby ensuring the efficiency of the division process; in addition, the solution problem of the first model including a large number of independent variables is divided into several solution problems of sub-models including fewer independent variables, which is conducive to executing parallel solutions and improving the efficiency of the solution process.

[0005] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a model processing method that can be used in the field of model solving. The processing device of the model used to execute the method can be a server cluster, an independent server, or a terminal device. The method includes: the processing device of the model obtains a graph structure corresponding to a first model, wherein the first model includes multiple first functions, and there are multiple first independent variables corresponding to the first model. The graph structure includes nodes and edges between the multiple nodes. The nodes in the graph structure correspond to the first independent variables one-to-one, that is, a node in the graph structure points to a first independent variable in the first model; the edges between the multiple nodes in the graph structure are determined according to the multiple first functions; further, if two independent variables are included in the same function, there is a connection relationship between the nodes pointing to the two independent variables; further, if there is a second independent variable and a third independent variable among the multiple first independent variables included in the first model, and the second independent variable and the third independent variable are simultaneously present in the second function and the third function of the first model, then there are two edges between the node pointing to the second independent variable and the node pointing to the third independent variable. The processing device of the model randomly splits the graph structure, that is, splits the multiple first independent variables pointed to by the aforementioned nodes to obtain at least two first sets, each of which includes at least one first independent variable. The processing device of the model uses the Levenberg-Marquardt (LM) algorithm to update the values ​​of the first independent variables in each first set. The processing device of the model repeatedly performs the random splitting operation and the updating operation of the values ​​of the first independent variables in each first set until the convergence condition of the first model is met, thereby obtaining a solution for the multiple first independent variables in the first model.

[0007] In this implementation, after obtaining the graph structure data corresponding to the first model, the graph structure indicated by the graph structure data is randomly divided to obtain at least two first sets, each of which includes at least one first independent variable, that is, all the first independent variables involved in the first model are divided into multiple first sets; then the Levenberg-Marquardt LM algorithm is used to update the value of the first independent variable in each first set, that is, in each iteration process, all the first independent variables involved in the first model are divided into multiple first sets, thereby solving the problem of the first model including a large number of independent variables. The problem is divided. Since all the independent variables in the model to be solved need to be stored in the memory at the same time, and the number of independent variables included in a sub-model is relatively small, when solving the sub-model, memory resources are required to store fewer independent variables, that is, there is no need to occupy too many memory resources when solving a sub-problem; in addition, the graph structure is divided by random division, and the implementation method is simple, thereby ensuring the efficiency of the division process; in addition, the solution problem of the first model including a large number of independent variables is divided into the solution problems of several sub-models including fewer independent variables, which is conducive to executing parallel solutions and improving the efficiency of the solution process.

[0008] In one possible implementation of the first aspect, each edge in a graph structure corresponds to two nodes, each node representing a first independent variable in a first model. A processing device for the model randomly partitions the graph structure to obtain at least two first sets, including: randomly sorting multiple edges in the graph structure to obtain multiple edges arranged in a first order, and obtaining a first edge from the multiple edges in the first order, wherein the first edge is one of the multiple edges in the graph structure. The processing device for the model determines whether the two first independent variables corresponding to the first edge have been paired with other nodes. If neither of the two first independent variables corresponding to the first edge has been paired, the processing device pairs the two first independent variables corresponding to the first edge to obtain a first subset. The processing device for the model performs the aforementioned operation on each edge in the graph structure, thereby obtaining multiple first subsets corresponding to the graph structure. The multiple first subsets corresponding to the graph structure can be stored using a data structure such as a table, a matrix, or a disjoint forest. A merge operation is performed on the multiple first subsets corresponding to the graph structure to obtain at least two first sets.

[0009] In an embodiment of the present application, a specific implementation method for randomly segmenting a graph structure is provided, and two nodes corresponding to the same edge will exist in the same set to improve the correlation between multiple independent variables within the same set, which is conducive to improving the speed of solving the entire first model.

[0010] In a possible implementation of the first aspect, when there is a paired independent variable among the two first independent variables corresponding to the first edge, the processing device of the model can skip the unpaired independent variable among the two first independent variables corresponding to the first edge, and then obtain the next edge from the multiple edges included in the graph structure according to the first order, or directly determine the unpaired independent variable among the two first independent variables corresponding to the first edge as a first subset.

[0011] In a possible implementation of the first aspect, a processing device of a model performs a merging operation on multiple first subsets corresponding to a graph structure, including: the processing device of the model obtains, based on edges between multiple nodes in the graph structure, association information corresponding to the multiple first subsets, wherein the association information indicates the association between each of the multiple first subsets, the association information includes a first association between a second subset and a third subset, the second subset and the third subset being any two different subsets of the multiple first subsets, the first association being determined based on the association between an independent variable in the second subset and an independent variable in the third subset, the more edges between two nodes, the higher the association between the independent variables pointed to by the nodes. The processing device of the model performs a merging operation on the multiple first subsets corresponding to the graph structure based on the association information corresponding to the multiple first subsets, wherein the higher the association between two first subsets, the greater the probability that the two first subsets will be merged.

[0012] In an embodiment of the present application, the higher the correlation between the two first subsets, the higher the probability of merging the two first subsets, that is, it is guaranteed that the independent variables with high correlation are likely to be divided into the same first set, thereby improving the correlation between multiple first independent variables included in the same first set, and reducing the correlation between the first independent variables included in different first sets. Since the main server divides all the first independent variables of the first model into multiple first sets, and solves at least one independent variable included in each first set respectively, the smaller the correlation between the first independent variables included in the two first sets, the smaller the conflict in the process of solving the first independent variables in the two first sets, thereby enabling the solution process of the entire first model to converge more efficiently, and is also conducive to improving the solution accuracy of the independent variables in the entire first model obtained in the end.

[0013] In one possible implementation of the first aspect, the model processing device performs the merging operation in a loop. In one of the multiple loops, after obtaining correlation information indicating the correlation between any two first subsets, the model processing device may select at least one first subset with the highest correlation, or may select at least one first subset with a correlation greater than a first correlation threshold. For any one of the at least one first subsets, the model processing device merges the independent variables included in each first subset in the set of first subsets to obtain a merged first subset. The model processing device determines whether the total number of independent variables included in the merged first subset is greater than or equal to a first threshold. If the total number of independent variables included in the merged first subset is greater than or equal to the first threshold, the merged first subset is determined as a first set and removed from the queue of first subsets. If the total number of independent variables included in the merged first subset is less than the first threshold, the merged first subset remains in the queue of first subsets. After executing the above operations, the processing device of the model enters the next loop, recalculates the association information indicating the association between any two first subsets, and re-executes the merging operation based on the association information.

[0014] In a possible implementation of the first aspect, the value of the first threshold is selected based on the upper limit of the number of independent variables that the memory resources configured by the device used for solving can bear, the number of devices used for solving, or other factors.

[0015] In a possible implementation of the first aspect, after the model processing device performs a merge operation on multiple first subsets corresponding to the graph structure to obtain at least two first sets, the method further includes: the model processing device obtains two first independent variables corresponding to the second edge, determines whether the two first independent variables corresponding to the second edge (i.e., the two first independent variables pointed to by the two nodes connected by the second edge) exist in two different first sets, and if the two first independent variables corresponding to the second edge exist in two different first sets, removes the two first independent variables corresponding to the second edge from the two different first sets, and determines the two first independent variables corresponding to the second edge as two isolated first independent variables, respectively, where the second edge is any one of the multiple edges of the graph structure. The model processing device performs the aforementioned operation on each edge in the graph structure. Correspondingly, the model processing device uses the Levenberg-Marquardt (LM) algorithm to update the values ​​of the first independent variables in each first set, including: the model processing device uses the Levenberg-Marquardt (LM) algorithm to update the values ​​of the first independent variables in each first set and the isolated first independent variables. The processing device of the model repeatedly performs the operation of random segmentation and the operation of updating the value of the first independent variable in each first set, including: the processing device of the model repeatedly performs the operation of random segmentation and the operation of updating the value of the first independent variable in each first set and the isolated first independent variable.

[0016] In the embodiment of the present application, the above-mentioned method can ensure that different first sets do not include independent variables corresponding to the same edge, greatly reducing the correlation between the first independent variables included in different first sets, and completely eliminating the conflict in the process of solving the first independent variables in the two first sets. The smaller the conflict, the more efficient the convergence of the solution process of the entire first model can be, which is also conducive to improving the solution accuracy of the independent variables in the entire first model obtained in the end.

[0017] In one possible implementation of the first aspect, the model processing device employs the Levenberg-Marquardt (LM) algorithm to iteratively update the value of the first independent variable in each first set, including: the model processing device obtains a second model corresponding to the first independent variable in the first set, the second model including at least one second function, and the at least one second function is determined based on multiple first functions included in the first model. Specifically, the first model includes multiple first functions, and the model processing device can determine whether any first independent variable in the first set exists in each first function. If so, the model processing device obtains the first function and, based on the first value of each first independent variable, converts the independent variables in the first function, excluding the independent variables included in the first set, into constants, thereby obtaining a second function. As an example, for example, the first independent variables included in the first target set are x1, x2, and x3. The three first functions involving x1, x2, and x3 in the first model are x1-x2, x2-x3, and x3-x4, respectively. The first value of x4 is 7, and the three second functions corresponding to x1, x2, and x3 are x1-x2, x2-x3, and x3-7. This example is only used to illustrate the difference between the second function and the first function. The processing device of the model obtains the function value of each second function in the at least one second function according to the Levenberg-Marquardt LM algorithm based on the first value of each first independent variable in the first set and at least one second function. The first value is the value of the first independent variable before the update. The first value of the first independent variable can be the initial value of the first independent variable or the value generated by the first independent variable in the previous iteration. The processing device of the model calculates the Jacobian matrix corresponding to the second function according to the Levenberg-Marquardt LM algorithm. The non-zero items in the Jacobian matrix include the derivative of the first scalar with respect to the second independent variable; wherein the first scalar is the sum of the values ​​of all second functions corresponding to the first independent variable in the first set, and the second independent variable is the independent variable constituting at least one second function. Different second independent variables have different positions in the second function, and different first independent variables have different identifiers; in other words, the second independent variable is the result of reorganizing the first independent variable according to the order in which the second function is received. As an example, for example, the second model is minimize x (x3-x7) 2 +(x1-x7) 2 +(x2-x6) 2 +(x3-x5) 2 +(x1-x4) 2 , corresponding to the multiple second functions in the second model, there are 10 second independent variables, namely y 1,1 =x3、y 1,2 =x7、y 2,1 =x1,y 2,2 =x7、y 3,1 =x²,y 3,2=x6,y 4,1 =x3、y 4,2 =x5,y 5,1 = x1 and y 5,2 =x4, where y 1,1 is the independent variable of the first term of the first function of the model shown in formula (3), y 1,2 is the independent variable of the second term of the first function of the model shown in formula (3). It can be seen that the independent variables at different positions in the multiple second functions are all different second independent variables, and there are 7 first independent variables corresponding to the multiple second functions in the second model, namely x1 to x7, where x1 and x2 are different identifiers, x2 and x3 are different identifiers, that is, the identifiers of different first independent variables are different. The example here is only for the convenience of understanding the difference between the first independent variable and the second independent variable. The processing device of the model generates the second value of the first independent variable in the first set according to the Levenberg-Marquardt LM algorithm, based on the first value of each first independent variable in the first set, the function value of the second function and the Jacobian matrix, and the second value is the updated value.

[0018] In the embodiment of the present application, since the AI ​​processor only provides the function of automatically differentiating a scalar with respect to an independent variable, the Jacobian matrix corresponding to the second function is the derivative of the vector function with respect to the first independent variable and therefore cannot be directly calculated using the AI ​​processor. However, in this implementation, the first independent variable is split, so that the non-zero terms in the Jacobian matrix corresponding to the second function can be expressed as the derivatives of the first scalar with respect to the second independent variable. This allows the AI ​​processor to directly update the value of the first independent variable. In other words, the method for calculating the Jacobian matrix provided in this implementation is compatible with the AI ​​processor, which is beneficial to improving the efficiency of the process of updating the value of the first independent variable, thereby improving the efficiency of the entire solution process.

[0019] In one possible implementation of the first aspect, a convergence condition for the first model is that a value of the first model is less than a first threshold, or that a difference between the second value and the first value is less than a second threshold. In the embodiments of the present application, two expressions of the convergence condition for the first model are provided, thereby increasing the implementation flexibility of the present solution.

[0020] In one possible implementation of the first aspect, the first model is a least squares model, and the first model is used in any of the following application scenarios: 3D modeling, image fusion, and mesh deformation for face reconstruction. In the embodiments of the present application, multiple application scenarios of the embodiments of the present application are provided, increasing the implementation flexibility of the solution.

[0021] In a second aspect, an embodiment of the present application provides a model processing method that can be used in the field of model solving. The method may include: a first model processing device obtains a graph structure corresponding to a first model, wherein the first model includes multiple first functions, the first model includes multiple first independent variables, the graph structure includes multiple nodes and edges between the multiple nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the multiple nodes in the graph structure are determined according to the multiple first functions; the first model processing device randomly splits the graph structure to obtain at least two first sets, each first set includes at least one first independent variable, and the at least two first sets are used to instruct the second model processing device to use the Levenberg-Marquardt LM algorithm to update the value of the first independent variable in each first set; the first model processing device receives the updated value of the first independent variable sent by the second model processing device; the first model processing device repeatedly performs the random splitting operation and the operation of receiving the updated value of the first independent variable until the convergence condition of the first model is met, and the solution of the multiple first independent variables in the first model is obtained.

[0022] In a possible implementation of the second aspect, each edge in the graph structure corresponds to two nodes, and one node represents a first independent variable in the first model; the first model processing device randomly divides the graph structure to obtain at least two first sets, including: the first model processing device randomly sorts multiple edges of the graph structure to obtain multiple edges arranged in a first order, and obtains a first edge from the multiple edges in the first order, where the first edge is one of the multiple edges of the graph structure; when the two first independent variables corresponding to the first edge are not paired, the two first independent variables corresponding to the first edge are paired to obtain a first subset; and a merge operation is performed on the multiple first subsets corresponding to the graph structure to obtain at least two first sets.

[0023] In a possible implementation of the second aspect, the first model processing device performs a merging operation on multiple first subsets corresponding to the graph structure, including: the first model processing device obtains association information corresponding to the multiple first subsets based on the edges between the multiple nodes in the graph structure, wherein the association information indicates the association between each first subset in the multiple first subsets, and the association information includes a first association between the second subset and the third subset, the second subset and the third subset are any two different subsets in the multiple first subsets, the first association is determined based on the association between the independent variables in the second subset and the independent variables in the third subset, and the more edges between the two nodes, the higher the association between the independent variables pointed to by the nodes; according to the association information corresponding to the multiple first subsets, the multiple first subsets corresponding to the graph structure are merged, and the higher the association between the two first subsets, the greater the probability of the two first subsets being merged.

[0024] In a possible implementation of the second aspect, after the first model processing device performs a merge operation on multiple first subsets corresponding to the graph structure to obtain at least two first sets, the method also includes: if the two first independent variables corresponding to the second edge exist in two different first sets, the first model processing device removes the two first independent variables corresponding to the second edge from the two different first sets, and determines the two first independent variables corresponding to the second edge as two isolated first independent variables, respectively, where the second edge is any one of the multiple edges of the graph structure, and the at least two first sets and the isolated first independent variables are used to instruct the second model processing device to use the Levenberg-Marquardt LM algorithm to update the values ​​of the first independent variables and the isolated first independent variables in each first set.

[0025] In the second aspect of this application, the constituent modules of the processing device of the first model can also be used to execute the steps performed by the processing device of the model in various possible implementation methods of the first aspect. For details, please refer to the description in the first aspect and the various possible implementation methods in the first aspect, and will not be repeated here.

[0026] In a third aspect, an embodiment of the present application provides a model processing method that can be used in the field of model solving. The method may include: a second model processing device obtains at least one second function corresponding to a first independent variable in the first set, and the at least one second function is determined based on multiple first functions included in the first model; the second model processing device updates the value of the first independent variable of the first independent variable in the first set according to the Levenberg-Marquardt LM algorithm, and sends the updated value of the first independent variable to the first model processing device.

[0027] In a possible design of the third aspect, the second model processing device updates the value of the first independent variable of the first independent variable in the first set according to the Levenberg-Marquardt LM algorithm, including: according to the Levenberg-Marquardt LM algorithm, based on the first value of each first independent variable in the first set and the second function, calculating the function value of the second function, the first value is the value of the first independent variable before the update; according to the Levenberg-Marquardt LM algorithm, calculating the Jacobian matrix corresponding to the second function, the Jacobian matrix including the derivative of the first scalar with respect to the second independent variable, the first scalar is the sum of the values ​​of all second functions corresponding to the first independent variable in the first set, the second independent variable is the independent variable constituting the second function, different second independent variables have different positions in the second function, and different first independent variables have different identifiers; according to the Levenberg-Marquardt LM algorithm, based on the first value of each first independent variable in the first set, the function value of the second function and the Jacobian matrix, generating the second value of the first independent variable in the first set, the second value is the updated value.

[0028] In the third aspect of this application, the constituent modules of the processing device of the second model can also be used to execute the steps performed by the processing device of the model in various possible implementation methods of the first aspect. For details, please refer to the description in the first aspect and the various possible implementation methods in the first aspect, and will not be repeated here.

[0029] In a fourth aspect, an embodiment of the present application provides a model processing device that can be used in the field of model solving. The device may include: an acquisition module for acquiring graph structure data corresponding to a first model, wherein the first model includes multiple first functions, the first model includes multiple first independent variables, the graph structure includes multiple nodes and edges between the multiple nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the multiple nodes in the graph structure are determined according to the multiple first functions; a segmentation module for randomly segmenting the graph structure to obtain at least two first sets, each first set including at least one first independent variable; an update module for updating the values ​​of the first independent variables in each first set using the Levenberg-Marquardt LM algorithm; a loop module for repeatedly performing random segmentation operations and updating the values ​​of the first independent variables in each first set until the convergence conditions of the first model are met and solutions for the multiple first independent variables in the first model are obtained.

[0030] The constituent modules of the model processing device provided in the fourth aspect of this application can also be used to execute the steps performed by the model processing device in various possible implementation methods of the first aspect. For details, please refer to the description in the first aspect and the various possible implementation methods in the first aspect, and will not be repeated here.

[0031] In the fifth aspect, an embodiment of the present application provides a model processing device, including a processor coupled to a memory; the memory is used to store programs; and the processor is used to execute the programs in the memory, so that the execution device executes the steps in each possible implementation method of the first aspect, the second aspect, or the third aspect.

[0032] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the processing method of the model of the first aspect, the second aspect, or the third aspect mentioned above.

[0033] In a seventh aspect, an embodiment of the present application provides a computer program, which, when running on a computer, enables the computer to execute the processing method of the model of the first aspect, the second aspect, or the third aspect mentioned above.

[0034] In an eighth aspect, the present application provides a chip system, which includes a processor for supporting a terminal device or server to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1a A schematic diagram of a system for processing a model provided in an embodiment of the present application;

[0036] Figure 1b Another system schematic diagram of a processing system for a model provided in an embodiment of the present application;

[0037] Figure 2 A flow chart of a method for processing a model provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a graph structure in the model processing method provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of multiple first subsets in the model processing method provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of a process for updating the value of an independent variable using the Levenberg-Marquardt LM algorithm in the processing method of the model provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of generating a Jacobian matrix in the processing method of the model provided in an embodiment of the present application;

[0042] Figure 7 Another flowchart of the method for processing the model provided in the embodiment of the present application;

[0043] Figure 8 A schematic diagram of another flow chart of the method for processing the model provided in the embodiment of the present application;

[0044] Figure 9 A schematic diagram of the structure of a processing device for a model provided in an embodiment of the present application;

[0045] Figure 10 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0046] Figure 11 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0048] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0049] It should be understood that the processing method of the first model provided in the embodiment of the present application can be applied to various application scenarios in which problems are solved by establishing a first model and solving the first model. Among them, the first model can be specifically expressed as a neural network or a function; further, the aforementioned function can be a least squares function or other types of functions; further, the aforementioned function can be a nonlinear function or a linear function, etc. It should be understood that in the embodiment of the present application, only the first model is a nonlinear least squares first model as an example. When the first model is another type of first model, it can be understood by reference. As an example, for example, the embodiment of the present application can be applied to three-dimensional modeling, image fusion, face reconstruction, mesh deformation, image positioning or other application scenarios, etc., and the application scenarios of the embodiment of the present application are not exhaustively listed here.

[0050] In all the above scenarios, there may be a problem that the number of independent variables included in the established model is too large, resulting in the problem that the memory resources cannot support it. In order to solve the above problem, the embodiment of the present application provides a method for model processing. Figure 1a and Figure 1b The processing system of the model provided in the embodiment of the present application is described. Figure 1a and Figure 1b Two system schematic diagrams of the processing system of the model provided in the embodiments of the present application. Figure 1aTaking the processing system of the model as a server cluster as an example, the server cluster includes a main server 101 and a sub-server 102. After obtaining the graph structure data corresponding to the first model, the main server 101 randomly splits the graph structure (that is, randomly splits multiple nodes in the graph structure) to obtain at least two first sets, each of which includes at least one first independent variable, that is, the main server splits the first model into several small second models, each of which includes at least one second function, and the number of independent variables included in the second model is less than the number of independent variables included in the first model. The main server distributes multiple first sets to multiple sub-servers 102, and the multiple sub-servers 102 update the values ​​of the first independent variables in each first set in parallel, and send the updated values ​​to the main server 101 to determine whether the convergence conditions of the first model are met through the main server 101. If not, the main server 101 re-executes the random splitting operation, and the sub-server 102 re-executes the operation of updating the values ​​of the first independent variables in each first set until the convergence conditions of the first model are met and the solution of each first independent variable in the first model is obtained. Although Figure 1a FIG shows a main server 101 and two sub-servers 102, but the embodiment of the present application does not limit the number of main servers 101 and sub-servers 102.

[0051] exist Figure 1b In the embodiment, the processing system of the model is specifically represented by a terminal device 20, which includes a central processing unit (CPU) 201 and an artificial intelligence (AI) processor 202. Specifically, the CPU 201 can execute Figure 1a The steps performed by the main server in the AI ​​processor 202 are performed Figure 1a Steps performed by the Neutron server.

[0052] Because the large first model is divided into several smaller second models, solving a subproblem does not require excessive memory resources. It should be noted that the processing system for this model can also be represented by a single server, with the server performing the random partitioning operation and updating the value of the first independent variable in each first set.

[0053] The following describes the specific implementation process of the model processing method provided in the embodiment of the present application.

[0054] In some embodiments of the present application, the system for model processing is Figure 1a For example, see the system shown in Figure 2 , Figure 2A flowchart of a method for processing a model provided in an embodiment of the present application is provided. The method for processing a model provided in an embodiment of the present application may include:

[0055] 201. The primary server obtains a first model.

[0056] In one embodiment of the present application, a primary server can directly receive a first model sent by a client. The first model is used to solve a problem in a specific application scenario. The first model includes multiple first functions, each of which has multiple first independent variables. Different first functions can have the same first independent variable. The specific types of the first model and the first independent variable should be determined based on the actual application scenario and the problem being solved. For example, in an application scenario where the first model is used for image fusion, for example, fusion of image B into the first region of image A, the goal of the fusion is to ensure that the boundary color of the first region is as close as possible to that of image A, and that the relative values ​​between the interior color of the first region and the perimeter of the first region, and the relative values ​​between image B and the perimeter of the first region, are as consistent as possible. The first independent variables in the first model can include the color values ​​of pixels at the boundary of the first region, the color values ​​of pixels within the first region, or other independent variables. As another example, if the first model is used for face reconstruction, the face itself can be represented by the first model. The first independent variables in the first model can include the coordinates of feature points in the facial image. It should be understood that these examples are provided solely to facilitate understanding of the application scenarios of the present embodiment and are not intended to limit the present solution.

[0057] Specifically, the first model sent by the client can be based on a function template, Lambda expression, or other form. After receiving the first model sent by the client, the main server can describe the first model in a server-readable language, that is, generate multiple first independent variables corresponding to the first model, the variable index corresponding to each independent variable in each first function, and the calculation method of each first function. As an example, the first model is as follows:

[0058]

[0059] Wherein, formula (1) represents the first model, (x1-x2), (x2-x3), (x3-x4), ..., (x7-x8) and (x8-x9) all represent first functions, x1 to x9 represent the nine first independent variables corresponding to the first model, and the server can establish a variable index s corresponding to each item in each first function. i,y , s i,y Represents the index of the jth independent variable in the i-th function in the first model, for example, s 2,2Represents the index of the second independent variable x3 in the second function in the first model, and the i-th first function is any one of the multiple first functions included in the first model. It should be understood that the example in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.

[0060] In another case, the main server can receive the original data corresponding to the problem to be solved sent by the client, and then generate a first model based on the problem to be solved and the original data. As an example, if the client wants to merge picture B into the first area of ​​picture A, the main server can receive a picture fusion request sent by the client, which carries the location information of picture B, picture A and the first area (that is, the original data corresponding to the problem to be solved). After receiving the picture fusion request and the original data, the main server can generate a first model and convert the first model into a language readable by the server. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0061] 202. The main server obtains a graph structure corresponding to the first model.

[0062] In an embodiment of the present application, after obtaining the first model, the main server will generate a graph structure corresponding to the first model. The graph structure data includes node data and edge data. The main server can use an adjacency list to store the graph structure data. Among them, the first model includes multiple first functions, the first model includes multiple first independent variables, the graph structure includes multiple nodes and edges between the multiple nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the multiple nodes in the graph structure are determined according to the multiple first functions; further, if two independent variables are included in the same function, there is a connection relationship between the nodes pointing to the two independent variables; further, if there is a second independent variable and a third independent variable among the multiple first independent variables included in the first model, and the second independent variable and the third independent variable exist in the second function and the third function of the first model at the same time, then there are two edges between the node pointing to the second independent variable and the node pointing to the third independent variable.

[0063] For a more intuitive understanding of this solution, please refer to Figure 3 , Figure 3 A schematic diagram of the graph structure in the model processing method provided in the embodiment of the present application. Figure 3 The graph structure shown in is the graph structure corresponding to formula (1), where Figure 3 The 9 nodes in point to the 9 first independent variables x1,x2,x3,x4,x5,x6,x7,x8,x9 in the first model respectively. Figure 3The eight edges in are e1, e2, e3, e4, e5, e6, e7, and e8. Since x1 and x2 are both included in the first function of formula (1), there is an edge e1 between x1 and x2. x2 and x3 are both included in the second function of formula (1), so there is an edge e2 between x2 and x3. The remaining edges will not be introduced here one by one.

[0064] Generate and Figure 3 The graph data corresponding to the graph structure in includes node data and edge data. The node data V corresponding to the graph structure is {x1, x2, x3, x4, x5, x6, x7, x8, x9}, and the edge data E corresponding to the graph structure is {e1, e2, e3, e4, e5, e6}. Furthermore, e1=<x1,x2> , e2=<x2,x3> , e3=<x3,x4> , e4=<x4,x5> , e5=<x5,x6> , e6=<x6,x7> , e7=<x7,x8> , e8=<x8,x9> The index variable corresponding to each first function of the first model is S={<1,2>,<2,3>,<3,4>,<4,5>,<5,6>,<6,7>,<7,8>,<8,9>}. It should be understood that Figure 3 The examples are only for facilitating understanding of this solution and are not intended to limit this solution.

[0065] 203. The main server randomly partitions the graph structure to obtain at least two first sets.

[0066] In an embodiment of the present application, after generating graph structure data corresponding to the first model, the main server needs to randomly split multiple nodes in the graph structure, that is, to split multiple first independent variables pointed to by the aforementioned nodes to obtain at least two first sets, each first set including at least one first independent variable.

[0067] Specifically, in one implementation, the main server randomly sorts the multiple edges of the graph structure to obtain multiple edges arranged in a first order, and then traverses the multiple edges in sequence according to the first order. For any traversed edge (i.e., the first edge), the main server determines whether the two first independent variables corresponding to the first edge have been paired with other nodes. If the two first independent variables corresponding to the first edge are not paired, the two first independent variables corresponding to the first edge are paired to obtain a first subset. After the server performs the above operation on each edge in the graph structure in the first order, multiple first subsets corresponding to the graph structure can be obtained, wherein the processor can use data structures such as tables, matrices, and disjoint forests to store multiple first subsets corresponding to the graph structure. Then the main server performs a merge operation on the multiple first subsets corresponding to the graph structure to obtain at least two first sets. In an embodiment of the present application, a specific implementation method for randomly segmenting the graph structure is provided, and the two nodes corresponding to the same edge will exist in the same set to improve the correlation between the multiple independent variables within the same set, which is conducive to improving the speed of solving the entire first model.

[0068] Furthermore, in the case that there is a paired independent variable among the two first independent variables corresponding to the first edge, the main server can skip the unpaired independent variable among the two first independent variables corresponding to the first edge, and then obtain the next edge from the multiple edges included in the graph structure according to the first order, or directly determine the unpaired independent variable among the two first independent variables corresponding to the first edge as a first subset.

[0069] To understand the random pairing process of the master server more intuitively, please refer to Figure 4 , Figure 4 A schematic diagram of multiple first subsets in the model processing method provided in an embodiment of the present application, Figure 4 For combination Figure 3 Take the graph structure in as an example. Figure 4 The two first independent variables connected by a solid line in represent two first independent variables that are successfully paired (that is, two first independent variables that form the same first subset). Figure 4 The two first independent variables connected by a dotted line in represent that they do not have a pairing relationship. Specifically, the master server will Figure 3The multiple edges E = {e1, e2, e3, e4, e5, e6, e7, e8} of the graph structure shown are shuffled to become E = {e3, e2, e1, e4, e6, e5, e8, e7}. The main server traverses e3, e2, e1, e4, e6, e5, e8, e7 in sequence (that is, reads each edge in sequence according to the first order after random shuffling). The main server first obtains e3, and obtains the two first independent variables corresponding to e3, respectively x3 and x4. The main server determines that x3 and x4 are not paired, and pairs x3 and x4 to obtain a first subset (corresponding to Figure 4 The solid line is between x3 and x4. Then, the two first independent variables corresponding to e2 are obtained, namely x2 and x3. Since x3 has been paired with x4, the main server skips the first independent variable x2 (corresponding to Figure 4 The main server obtains two first independent variables corresponding to e1, namely x1 and x2. The main server determines that x1 and x2 are not paired, and pairs x1 and x2 to obtain a first subset (corresponding to Figure 4 Similarly, since x4 has been paired, the main server does not pair e4 and skips the first argument x5 (corresponding to Figure 4 (The dashed line between x4 and x5 is shown in the figure). Similarly, for e6, the master server pairs the two first independent variables x6 and x7 corresponding to e6 to obtain a first subset. The pairing process of e5, e8, and e7 can be found in the previous description and will not be repeated here. It should be understood that Figure 4 The examples are only for facilitating understanding of this solution and are not intended to limit this solution.

[0070] A process for a master server to perform a merge operation on multiple first subsets corresponding to a graph structure. In one implementation, the master server obtains association information corresponding to the multiple first subsets based on edges between multiple nodes in the graph structure. The association information indicates the degree of association between each of the multiple first subsets. The master server performs a merge operation on the multiple first subsets corresponding to the graph structure based on the association information corresponding to the multiple first subsets to obtain at least two first sets; the higher the association between the two first subsets, the greater the probability that the two first subsets will be merged. In an embodiment of the present application, after obtaining multiple first subsets, a merging operation is performed using the correlation between the multiple first subsets. The higher the correlation, the higher the correlation between the two first subsets, and the greater the probability of merging the two first subsets. That is, it is ensured that the independent variables with high correlation are likely to be divided into the same first set, thereby improving the correlation between the multiple first independent variables included in the same first set and reducing the correlation between the first independent variables included in different first sets. Since the main server divides all the first independent variables of the first model into multiple first sets and solves at least one independent variable included in each first set respectively, the smaller the correlation between the first independent variables included in the two first sets, the smaller the conflict in the process of solving the first independent variables in the two first sets, thereby enabling the solution process of the entire first model to converge more efficiently, and also helps to improve the solution accuracy of the independent variables in the entire first model obtained in the end.

[0071] More specifically, the association information includes a first association between the second subset and the third subset. The second subset and the third subset are each any two different subsets from the plurality of first subsets. The first association is determined based on the association between the independent variables in the second subset and the independent variables in the third subset. The more edges between two nodes, the higher the association between the independent variables pointed to by the nodes. A process for generating association information for a master server is described. The master server obtains edge data for a graph structure corresponding to a first model. The edge data indicates not only whether any two nodes in the graph structure are connected, but also the number of edges between any two nodes in the graph structure. The master server obtains at least one independent variable included in the second subset and at least one independent variable included in the third subset, and calculates the sum of the associations between each independent variable in the second subset and each independent variable in the third subset, thereby obtaining the association between the second subset and the third subset. The number of edges between two independent variables can be directly determined as the association between the two independent variables, or the number of edges between the two independent variables can be converted to obtain the association between the two independent variables. The main server performs the above operation on any two different first subsets among the multiple first subsets, thereby obtaining the correlation between any two first subsets, that is, obtaining correlation information.

[0072] Combined with the above Figure 3 and Figure 4 For example, the second subset includes x3 and x4, and the third subset includes x1 and x2. Figure 3 As can be seen from the example, if there is an edge between x3 and x2, then the correlation between x3 and x2 is 1; if there is no edge between x3 and x1, then the correlation between x3 and x1 is 0; if there is no edge between x4 and x2, then the correlation between x4 and x2 is 0; if there is no edge between x4 and x1, then the correlation between x4 and x1 is 0, thus the correlation between the second subset and the third subset is 1. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution. Furthermore, the master server can store correlation information using a matrix, table, or other data structure. To more intuitively understand this solution, the following set Table 1 provides an example of the information included in the correlation information.

[0073] Correlation First subset 1 First subset 2 First subset 3 First subset 4 First subset 1 Null 2 1 1 First subset 2 2 Null 3 4 First subset 3 1 3 Null 2 First subset 4 1 4 2 Null

[0074] Table 1

[0075] Among them, as shown in Table 1 above, the correlation between the first set 1 and the first set 2 is 2, the correlation between the first set 1 and the first set 3 is 1, the correlation between the first set 1 and the first set 4 is 1, and so on. The remaining contents in Table 1 will not be repeated here.

[0076] The master server uses correlation information to merge multiple first subsets. The master server performs the merge operation in a loop. In one of the multiple loops, that is, after obtaining correlation information indicating the correlation between any two first subsets, the master server may select at least one first subset with the highest correlation, or may select at least one first subset with a correlation above a first correlation threshold. For any of the at least one first subsets, the master server merges the independent variables included in each first subset within the set to obtain a merged first subset. It should be noted that if a first subset has the same correlation with two different first subsets, a first subset may be randomly selected from the two different first subsets to perform the merge operation. For example, if the correlation between first subset 1 and first subset 2 is 3, and the correlation between first subset 1 and first subset 3 is also 3, a first subset may be randomly selected from first subset 2 and first subset 3 to merge with first subset 1. It should be understood that this example is merely for facilitating understanding of this solution and is not intended to limit this solution.

[0077] The master server may pre-set a first threshold to determine whether the total number of independent variables included in the merged first subset is greater than or equal to the first threshold. If the total number of independent variables included in the merged first subset is greater than or equal to the first threshold, the merged first subset is determined to be a first set and removed from the queue of first subsets. If the total number of independent variables included in the merged first subset is less than the first threshold, the merged first subset remains in the queue of first subsets. After performing the aforementioned operations, the master server enters the next loop, recalculates the correlation information indicating the correlation between any two first subsets, and re-executes the merge operation based on the correlation information.

[0078] The value of the first threshold is selected based on factors including: the upper limit of the number of independent variables that the memory resources configured by the device used for solving can support, the number of devices used for solving, or other factors, which are not exhaustive here. As an example, if the upper limit of the number of independent variables that the memory resources configured by the device used for solving can support is 8 independent variables, then the value of the first threshold is less than or equal to 4; as another example, if there are 3 devices used for solving, then the number of the first set needs to be greater than or equal to 3, thereby affecting the value of the first threshold by affecting the number of the first set, and so on.

[0079] In order to understand this solution more intuitively, an example is given in combination with Table 1 above. Here, the first threshold value is 10, and the first subset 3 includes 4 independent variables and the first subset 4 includes 2 independent variables. After generating the correlation information shown in Table 1, the main server selects a group of first subsets with the highest correlation as the first subset 4 and the first subset 3, and then merges the independent variables included in the first subset 4 and the independent variables included in the first subset 3 to obtain a merged first subset. The main server determines that the number of independent variables included in the merged first subset is less than 10, and then retains the merged first subset in the queue of the first subset, thereby completing a cycle. It should be understood that the example here is only for the convenience of understanding this solution and is not used to limit this solution.

[0080] In another implementation, after obtaining multiple first subsets corresponding to the graph structure, the main server may also perform the merge operation in a loop. In each loop, the main server randomly selects two first subsets from the multiple first subsets, merges the two selected first subsets, and obtains a merged first subset. The main server determines whether the number of independent variables included in the merged first subset is greater than or equal to the first threshold, then determines the merged first subset as a first set, and removes the merged first subset from the queue of the first subset; if the total number of independent variables included in the merged first subset is less than the first threshold, then the merged first subset remains in the queue of the first subset. After performing the above operations, the main server enters the next loop, randomly selects two first subsets from the multiple first subsets again, and re-executes the merge operation.

[0081] It should be noted that the master server may also use other methods to achieve random segmentation of the graph structure, which are not exhaustively listed here.

[0082] 204. The main server determines whether the two first independent variables corresponding to any one of the multiple edges included in the graph structure corresponding to the first model exist in two different first sets. If so, proceed to step 205; if neither exists, proceed to step 206.

[0083] In some embodiments of the present application, after the main server performs a merge operation on the multiple first subsets corresponding to the graph structure to obtain the at least two first sets, it will also traverse each edge in the graph structure corresponding to the first model (that is, sequentially obtain each edge from the multiple edges corresponding to the graph structure) to determine whether the two first independent variables corresponding to each edge in the graph structure exist in two different first sets. Specifically, the second edge is any one of the multiple edges included in the graph structure, the main server obtains the two first independent variables corresponding to the second edge, and determines whether the two first independent variables corresponding to the second edge (that is, the two first independent variables pointed to by the two nodes connected by the second edge) exist in two different first sets. If so, proceed to step 205; if not, proceed to step 206.

[0084] Combine Figure 3 and Figure 4For example, the first set 1 includes x1, x2, x3 and x4, the first set 2 includes x5, x6 and x7, and the first set 3 includes x8 and x9. The main server obtains the two first independent variables x1 and x2 corresponding to the edge e1, and both x1 and x2 exist in the first set 1. The main server then obtains the two first variables x2 and x3 corresponding to the edge e2, and both x2 and x3 exist in the first set 1. The main server then obtains the two first variables x3 and x4 corresponding to the edge e3, and both x3 and x4 exist in the first set 1. The main server then obtains the two first variables x4 and x5 corresponding to the edge e4, and x4 and x5 do not exist in the same first subset, then enters step 205. For the implementation of e5, e6, e7 and e8, please refer to the above description, which will not be repeated here. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0085] 205. The main server removes the two first independent variables corresponding to the second edge from the two different first sets, and determines the two first independent variables corresponding to the second edge as two isolated first independent variables.

[0086] In some embodiments of the present application, when the main server determines that the two first independent variables corresponding to the second edge exist in two different first sets, it removes the two first independent variables corresponding to the second edge from the two different first sets and determines the two first independent variables corresponding to the second edge as two isolated first independent variables.

[0087] Taking the example in step 204 as an example, since the two first variables x4 and x5 corresponding to edge e4 are respectively located in the first set 1 and the first set 2, x4 and x5 are removed from the first set 1 and the first set 2, respectively, that is, the first independent variables included in the first set 1 become x1, x2 and x3, and the first independent variables included in the first set 2 become x6 and x7, and x4 and x5 are respectively determined as two isolated first independent variables. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0088] It should be noted that after the main server determines through step 204 that the two first independent variables corresponding to the second edge exist in two different first sets, it will enter step 205. After executing step 205 and before traversing each edge, it will re-enter step 204 to determine whether the two first independent variables corresponding to the next edge exist in two different first sets until every edge in the graph structure is traversed.

[0089] Furthermore, the master server may pre-set a first threshold for the number of times the judgment is performed, where the value of the first threshold is the total number of graph structures corresponding to the first model. If the master server executes the judgment step of step 204 less than the first threshold, it is deemed that not all edges in the graph structure have been traversed; if the master server executes the judgment step of step 204 equal to the first threshold, it is deemed that all edges in the graph structure have been traversed.

[0090] In the embodiment of the present application, the above-mentioned method can ensure that different first sets do not include independent variables corresponding to the same edge, greatly reducing the correlation between the first independent variables included in different first sets, and completely eliminating the conflict in the process of solving the first independent variables in the two first sets. The smaller the conflict, the more efficient the convergence of the solution process of the entire first model can be, which is also conducive to improving the solution accuracy of the independent variables in the entire first model obtained in the end.

[0091] 206. The main server sends the multiple first sets to different sub-servers respectively.

[0092] In the embodiment of the present application, since steps 204 and 205 are optional steps, if steps 204 and 205 are not performed, the main server can directly proceed to step 206 after performing step 203, and the main server will send the multiple first sets obtained in step 203 to different sub-servers respectively, so that the values ​​of the first independent variables can be updated in parallel through the multiple different sub-servers. If steps 204 and 205 are performed, the main server will return the multiple first sets and at least one isolated first independent variable to different sub-servers respectively, so that the values ​​of the first independent variables can be updated in parallel through the multiple different sub-servers.

[0093] Specifically, in one implementation, the main server may send the first model, the first value of each first independent variable in the first model, and the first set (or isolated first independent variable) to the sub-server, and the sub-server may generate a second model corresponding to the first independent variable (or isolated first independent variable) in the first set based on the first model, the first value of each first independent variable in the first model, and the first set (or isolated first independent variable), wherein the second model includes at least one second function, and the sub-server updates the value of the first independent variable (or isolated first independent variable) in the first set, that is, updates the value of at least one first independent variable included in at least one second function.

[0094] The first value of each first independent variable in the first model may refer to the initial value of each first independent variable, or the value of each first independent variable generated in the previous iteration. Furthermore, the method for determining the initial value of each first independent variable needs to be determined in combination with the actual application scenario. For example, when the first model is applied to an image fusion scenario, the initial value of each first independent variable is the initial color value of the pixel of the image to be fused; as another example, when the first model is applied to a three-dimensional reconstruction application scenario, the initial value of each first independent variable can be calculated using a pre-order algorithm.

[0095] In another implementation, the main server can generate a second model corresponding to the first independent variable in the first set based on the first model, the first value of each first independent variable in the first model and the first set, the second model includes at least one second function, and directly sends the at least one second function corresponding to the first independent variable in the first set to the sub-server.

[0096] Specifically, the step of obtaining at least one second function corresponding to at least one first independent variable included in the target first set is performed. The main server or sub-server obtains at least one first independent variable included in the target first set. The first model includes multiple first functions. The main server can determine whether any first independent variable in the target first set exists in each first function. If not, the main server continues to perform the above operation on the next first function. If it exists, the main server obtains the first function and, based on the first value of each first independent variable, converts the independent variables in the first function other than the independent variables included in the target first set into constants, thereby obtaining a second function. In other words, if a first function contains an independent variable that is not in the target first set, the independent variable not in the target first set is treated as a scalar, that is, the first value corresponding to the independent variable not in the target first set is substituted into the first function to obtain the second function. The main server performs the above operation on each first function, thereby obtaining the second function corresponding to the first independent variable included in the target first set.

[0097] As an example, for example, the first model is the above formula (1), the first independent variables included in the first target set are x1, x2 and x3, and the first values ​​of x1, x2, x3, x4, x5, x6, x7, x8, x9 are 2, 3, 6, 7, 4, 2, 1, 5 and 3 respectively. Then the second model corresponding to x1, x2 and x3 is as follows:

[0098] minimize x (x1-x2) 2 +(x2-x3) 2 +(x3-7) 2 ; (2)

[0099] Wherein, formula (2) represents the second model corresponding to the multiple first independent variables included in the target first set, and the second model includes multiple second functions. The difference between the second model and the first model, as well as the difference between the second function and the first function, can be determined by comparing formula (2) with formula (1), which will not be repeated here.

[0100] 207. The sub-server uses the Levenberg-Marquardt LM algorithm to update the value of the first independent variable to obtain an updated value of the first independent variable.

[0101] In an embodiment of the present application, if a single sub-server is assigned to a first set, the sub-server uses the LM algorithm to update the values ​​of all first independent variables included in the first set to obtain updated values ​​of all first independent variables included in the first set. If a single sub-server is assigned to an isolated first independent variable, the sub-server uses the LM algorithm to update the value of the isolated first independent variable to obtain an updated value of the isolated first independent variable.

[0102] Specifically, the sub-server needs to obtain at least one second function corresponding to all first independent variables (or isolated first independent variables) included in a first set, and use the Levenberg-Maquardt (LM) algorithm to update the values ​​of all first independent variables (or isolated first independent variables) included in the first set to obtain updated values ​​of all first independent variables (or isolated first independent variables) included in the first set. The formula of the LM algorithm is as follows:

[0103] x * =x k -(J T J+λ k I) -1 J T f(x k )

[0104] Among them, x * represents the updated values ​​(i.e., second values) of all first independent variables included in a first set, x k represents the values ​​of all first independent variables in a first set before updating (i.e., the first values), and J represents f(x k ) for x k The Jacobian matrix, J T represents the transpose of J, λ k It is a hyperparameter in each iteration process, and its value can be different in different iteration numbers to control the step size of the iteration. I is the identity matrix, and -1 represents the inverse of the matrix.

[0105] To more intuitively understand the process of updating the first independent variable using the Levenberg-Maquardt (LM) algorithm, see Figure 5 , Figure 5 A flow chart of updating the value of an independent variable using the Levenberg-Marquardt LM algorithm in the model processing method provided in the embodiment of the present application. Specifically, C1, calculating the value of the second model (i.e., multiple second functions) corresponding to at least one first independent variable included in the first set, that is, calculating f(x k ); C2, calculate the Jacobian matrix of the second function with respect to the first independent variable, that is, calculate f(x k ) for x k Jacobian matrix; C3, calculating the difference between the second value and the first value of each first independent variable in the first set by solving a linear equation; C4, generating the second value of each first independent variable in the first set according to the first value of each first independent variable in the first set and the difference between the second value and the first value of each first independent variable in the first set; C5, judging whether the convergence condition of the second model is met, wherein the convergence condition of the second model includes any one of the following: the value of the second model is less than the first threshold value, or the difference between the second value and the first value of the aforementioned first independent variable is less than the second threshold value. If the convergence condition of the second model is met, the second value of each first independent variable in the first set is obtained; if the convergence condition of the second model is not met, re-entering step C1 to perform the next round of updating of the values ​​of the multiple first independent variables included in the second model until the convergence condition of the second model is met.

[0106] Among them, the sub-server can calculate the Jacobian matrix corresponding to the second function in two ways. In one implementation method, the sub-server calculates the derivative of the first scalar with respect to the second independent variable to obtain the non-zero terms in the Jacobian matrix, and then obtains the Jacobian matrix corresponding to the second function.

[0107] Specifically, the sub-server obtains at least one second function corresponding to the first independent variable (or isolated first independent variable) in the first set. The aforementioned at least one second function is determined based on the multiple first functions included in the first model. The specific method of obtaining the at least one second function and the difference between the second function and the first function can be found in the description of step 206, which will not be repeated here. The sub-server calculates the function value of the second function according to the first value and the second function of each first independent variable (or isolated first independent variable) in the first set using the LM algorithm, and obtains the function value of the second function (that is, calculates f(x k ) value), the first value is the value of the first independent variable before it is updated.

[0108] The subserver calculates the Jacobian matrix corresponding to the second function using the LM algorithm. The non-zero entries in the Jacobian matrix include the derivatives of the first scalar with respect to the second independent variable. The first scalar is the sum of the values ​​of all second functions corresponding to the first independent variables in the first set. The second independent variables are the independent variables that constitute at least one second function. Different second independent variables have different positions in the second function, and different first independent variables have different identifiers. In other words, the second independent variables are the result of reorganizing the first independent variables according to the order in which they are received by the second function. As an example, the second model is as follows:

[0109] minimize x (x3-x7) 2 +(x1-x7) 2 +(x2-x6) 2 +(x3-x5) 2 +(x1-x4) 2 ;(3)

[0110] Wherein, formula (3) represents the second model, the index S1 corresponding to the multiple independent variables in the second model is S1 = {<3,7>,<1,7>,<2,6>,<3,5>,<1,4>}, and there are 10 second independent variables corresponding to the multiple second functions in the second model, which are y 1,1 =x3、y 1,2 =x7、y 2,1 =x1,y 2,2 =x7、y 3,1 =x²,y 3,2 =x6,y 4,1 =x3、y 4,2 =x5,y 5,1 = x1 and y 5,2 =x4, where y 1,1 is the independent variable of the first term of the first function of the model shown in formula (3), x 1,2 It is the independent variable of the second term of the first function of the model shown in formula (3). It can be seen that the independent variables at different positions in the multiple second functions are all different second independent variables, and there are 7 first independent variables corresponding to the multiple second functions in the second model, namely x1 to x7, among which x1 and x2 are different identifiers, and x2 and x3 are different identifiers, that is, the identifiers of different first independent variables are different. It should be understood that the example in formula (3) is only for the convenience of understanding the difference between the first independent variable and the second independent variable, and is not used to limit this solution.

[0111] The sub-server calculates the difference between the second value and the first value of each first independent variable in the first set by solving the linear equation based on the LM algorithm, the function value of the second function and the Jacobian matrix, and generates the second value of the first independent variable in the first set based on the difference between the second value and the first value of each first independent variable in the first set and the first value of each first independent variable in the first set, where the second value is the updated value.

[0112] In the embodiment of the present application, since the AI ​​processor only provides the function of automatically differentiating a scalar with respect to an independent variable, the Jacobian matrix corresponding to the second function is the derivative of the vector function with respect to the first independent variable and therefore cannot be directly calculated using the AI ​​processor. However, in this implementation, the first independent variable is split, so that the non-zero terms in the Jacobian matrix corresponding to the second function can be expressed as the derivatives of the first scalar with respect to the second independent variable. This allows the AI ​​processor to directly update the value of the first independent variable. In other words, the method for calculating the Jacobian matrix provided in this implementation is compatible with the AI ​​processor, which is beneficial to improving the efficiency of the process of updating the value of the first independent variable, thereby improving the efficiency of the entire solution process.

[0113] The following is a formula conversion to introduce the derivation principle of how the non-zero items in the Jacobian matrix corresponding to the second function can be converted into the derivative of the first scalar with respect to the second independent variable:

[0114]

[0115] Among them, J r (x) i,si,j represents the value of the i-th row and j-th column (i.e., a non-zero entry) in the Jacobian matrix corresponding to the second function, Represents the derivative of the ith function with respect to the first independent variable of the jth term in the ith function, that is, the original meaning of a value in the Jacobian matrix. represents the derivative of the ith function with respect to the second argument of the jth term in the ith function, represents the derivative of the first scalar with respect to the i-th function, Represents the derivative of the first scalar with respect to the second argument of the j-th term in the i-th function.

[0116] More specifically, regarding the process of generating a Jacobian matrix, when the subserver uses an AI processor to perform this operation, the subserver obtains variable indices corresponding to the second model (i.e., at least one second function), multiple first independent variables, and at least one second function included in the second model. The subserver packages the multiple first independent variables corresponding to the second model (i.e., at least one second function) into a tensor and inputs the variable indices corresponding to the second model (i.e., at least one second function) and the multiple first independent variables into an indexer in the AI ​​processor. The indexer then reindexes the multiple second independent variables in the order in which the first independent variables appear in the multiple second functions to obtain the values ​​of the multiple second independent variables. The subserver then calculates the function value of the at least one second function based on the values ​​of the multiple second independent variables included in the second model and the multiple second independent variables, and accumulates the function values ​​of all the second functions to obtain the value of the first scalar. The AI ​​processor then calculates the derivative of the first scalar with respect to the second independent variable to obtain the values ​​of the non-zero entries in the Jacobian matrix corresponding to the second function. The generated non-zero entries are then placed in the corresponding positions of the Jacobian matrix to obtain the entire Jacobian matrix corresponding to the second function.

[0117] For a more intuitive understanding of this solution, please refer to Figure 6 , Figure 6 A schematic diagram of generating a Jacobian matrix in the processing method of the model provided in the embodiment of the present application. D1. The sub-server indexes the variable corresponding to the second model (ie, at least one second function) (ie Figure 6 index S in the Figure 6 x1 to x7 in the input are input into the indexer in the AI ​​processor, so as to be re-indexed by the indexer according to the variable index corresponding to the second model (i.e., at least one second function), and each second independent variable (i.e., Figure 6 y in 1,1 to y 5,2 ) value. D2. The sub-server calculates each second function (i.e. Figure 6 r1, r2, r shown in i 、r n and r n-1 ), and then calculate the value of the first scalar L (that is, the sum of the function values ​​of all second functions); Figure 6 The second model shown includes a plurality of second functions, r1=y 1,1 -y 1,2 , r2=y 2,1 -y 2,2 , r3=y 3,1 -y 3,2 , r4=y 4,1 -y 4,2 and r5 = y5,1 -y 5,2 , the first scalar L = r1 + r2 + r3 + r4 + r5. D3. The sub-server automatically calculates the derivative of the first scalar L with respect to each second independent variable through the AI ​​processor. Figure 6 Describe, specifically, D4. The sub-server uses the variable index (i.e. Figure 6 The index S in D3 is mapped to the Jacobian matrix corresponding to the second function to obtain the Jacobian matrix corresponding to the second function. i,j (i.e. ) is the value of the i-th row and j-th column in the Jacobian matrix, specifically: and It should be understood that Figure 6 The examples are only for facilitating understanding of this solution and are not intended to limit this solution.

[0118] In another implementation, the sub-server directly calculates the derivative of the second function with respect to the first independent variable to obtain the Jacobian matrix. Step 207 may include: the sub-server obtains at least one second function corresponding to the first independent variable (or isolated first independent variable) in the first set, and according to the LM algorithm, substitutes the first value of each first independent variable (or isolated first independent variable) in the first set into the second function to obtain the function value of the second function (that is, calculate f(x k ) value). The subserver directly calculates the Jacobian matrix corresponding to the second function based on the LM algorithm, that is, directly calculates the derivative of the second function with respect to the first independent variable. Then, based on the LM algorithm, the function value of the second function and the Jacobian matrix are used to solve a linear equation to calculate the difference between the second value and the first value of each first independent variable in the first set. Based on the difference between the second value and the first value of each first independent variable in the first set and the first value of each first independent variable in the first set, a second value of the first independent variable in the first set is generated, where the second value is the updated value.

[0119] 208. The main server receives the updated value of the first independent variable sent by the sub-server.

[0120] In an embodiment of the present application, after each sub-server generates an updated value of the first independent variable (i.e., the second value), it will send the updated value of the first independent variable to the main server. Correspondingly, the main server receives the updated value of the first independent variable sent by each sub-server and summarizes it.

[0121] 209. The main server determines whether the convergence condition of the first model is met. If not, the process re-enters step 203. If so, the process obtains solutions for the multiple first independent variables in the first model.

[0122] In an embodiment of the present application, the main server determines whether the convergence condition of the first model is met based on the updated value of the first independent variable. If not, step 203 is re-entered to repeatedly perform the random segmentation operation and the operation of updating the value of the first independent variable in each first set until the convergence condition of the first model is met; if it is met, the solutions of multiple first independent variables in the first model are obtained, and then the solutions of multiple first independent variables in the first model can be sent to the client.

[0123] The convergence condition of the first model is that the value of the first model is less than a first threshold, or the difference between the second value and the first value is less than a second threshold. In the embodiment of the present application, two forms of expression of the convergence condition of the first model are provided, which improves the implementation flexibility of this solution.

[0124] It should be noted that although Figure 2 In the corresponding embodiments, the processing system of the model is expressed as a server cluster, that is, it includes a main server and multiple sub-servers. However, in actual situations, the processing system of the model can also be expressed as a server. In this case, the execution subject of the above steps executed by the sub-servers must be replaced by the main server, and steps 206 and 208 do not need to appear.

[0125] For a more intuitive understanding of this solution, please refer to Figure 7 , Figure 7 A flow chart of a processing method for a model provided in an embodiment of the present application. Wherein, S1, the main server obtains graph structure data corresponding to the first model. S2, the main server randomly pairs the graph structure data corresponding to the first model to obtain multiple first subsets corresponding to the graph structure. S3, the main server performs a merge operation on the multiple first subsets to obtain at least two first sets and at least two isolated first independent variables. S4, the main server deploys the at least two first sets and at least two isolated first independent variables in different sub-servers respectively. S5, different sub-servers use the Levenberg-Marquardt LM algorithm to update the first value of the first independent variable to obtain the updated value of the first independent variable (that is, the second value). The main server determines whether the convergence condition is met based on the second value of the first independent variable. If the convergence condition is met (that is, Figure 6 If the convergence condition is not met (i.e. Figure 6 If the value of θ is not converged), then step S2 is entered again. It should be understood that Figure 6 The examples are only for facilitating understanding of this solution and are not intended to limit this solution.

[0126] In the embodiment of the present application, after obtaining the graph structure data corresponding to the first model, the graph structure indicated by the graph structure data is randomly divided to obtain at least two first sets, each of which includes at least one first independent variable, that is, all the first independent variables involved in the first model are divided into multiple first sets; then the Levenberg-Marquardt LM algorithm is used to update the value of the first independent variable in each first set, that is, in each iteration process, all the first independent variables involved in the first model are divided into multiple first sets, thereby solving the problem of the first model including a large number of independent variables. The problem is divided. Since all the independent variables in the model to be solved need to be stored in the memory at the same time, and the number of independent variables included in a sub-model is relatively small, when solving the sub-model, memory resources are required to store fewer independent variables, that is, there is no need to occupy too many memory resources when solving a sub-problem; in addition, the graph structure is divided by random division, and the implementation method is simple, thereby ensuring the efficiency of the division process; in addition, the solution problem of the first model including a large number of independent variables is divided into the solution problems of several sub-models including fewer independent variables, which is conducive to executing parallel solutions and improving the efficiency of the solution process.

[0127] In some embodiments of the present application, the system for model processing is Figure 1b For example, refer to the terminal device shown in Figure 8 , Figure 8 A flowchart of a method for processing a model provided in an embodiment of the present application is provided. The method for processing a model provided in an embodiment of the present application may include:

[0128] 801. The processor obtains a first model.

[0129] 802. The processor obtains a graph structure corresponding to the first model.

[0130] 803. The processor randomly partitions the graph structure to obtain at least two first sets.

[0131] 804. The processor determines whether two first independent variables corresponding to any one of the multiple edges included in the graph structure corresponding to the first model exist in two different first sets. If so, the process proceeds to step 805; if neither exists, the process proceeds to step 806.

[0132] 805. The processor removes the two first independent variables corresponding to the second edge from two different first sets, and determines the two first independent variables corresponding to the second edge as two isolated first independent variables.

[0133] In the embodiment of the present application, the specific implementation method of steps 801 to 805 is similar to the specific implementation method of steps 201 to 205, the difference is that the execution body of steps 201 to 205 is the main server, and the execution body of steps 801 to 805 is the processor in the terminal device.

[0134] As an example, for example, the first model can be applied to perform adjustment optimization on three-dimensional modeling. Specifically, for three-dimensional modeling based on multiple pictures, adjustment optimization refers to the optimization of camera parameters and the optimization of the coordinates of feature pixels in the three-dimensional model. Specifically, the process of three-dimensional modeling includes: extracting feature pixels of each picture in multiple pictures, and finding the correspondence between feature pixels in different pictures, calculating the three-dimensional posture of the camera (including the position and orientation information of the camera in space) and the coordinates of the feature pixels in the three-dimensional space according to the two-dimensional coordinates of each feature pixel and the aforementioned correspondence, and then using the processing method of the model provided in the embodiment of the present application to obtain the optimal solution for the camera parameters and the coordinates of the feature pixels in the three-dimensional model. Then the first independent variable includes the three-dimensional posture parameters of the camera and the coordinates of the feature pixels projected into the three-dimensional space. It should be understood that the examples given here are only for the convenience of understanding the application scenarios of this solution and are not used to limit this solution.

[0135] 806. The processor uses the Levenberg-Marquardt LM algorithm to update the value of the first independent variable to obtain an updated value of the first independent variable.

[0136] In the embodiment of the present application, the specific implementation of step 806 is similar to the specific implementation of step 207, except that the execution subject of step 207 is the main server, and the execution subject of step 806 is the processor in the terminal device. It should be noted that step 806 can be executed by the processor through an AI processor, a graphics processing unit (GPU) or other type of processor, etc., and can also be executed in parallel by multiple chips, etc., without limitation here.

[0137] 807. The processor determines whether the convergence condition of the first model is satisfied. If not, the process re-enters step 803. If satisfied, the process obtains solutions of the plurality of first independent variables in the first model.

[0138] In the embodiment of the present application, the specific implementation method of step 807 is similar to the specific implementation method of step 209, the difference is that the execution body of step 209 is the main server, and the execution body of step 807 is the processor in the terminal device.

[0139] In the embodiment of the present application, an implementation scheme is provided when the processing system of the model is represented as a terminal device, which expands the application scenarios of the scheme and improves the implementation flexibility of the scheme.

[0140] In order to more intuitively understand the beneficial effects of this solution, the beneficial effects brought about by the embodiment of the present application are described below in combination with the data in Table 2. Table 2 takes the application scenario of the first model being used for three-dimensional modeling and adjustment optimization as an example.

[0141]

[0142] Table 2

[0143] Among them, Ceres-CG represents the method provided by Google for solving the model, and LadyBug, Dubrovnik, and Venice are three public databases. When the database LadyBug is selected, when the Ceres-CG method is used to solve the model, the memory consumed is 0.52GB, the time used is 46.7 seconds, and the error is 4.15E+05 pixels; when the solution provided in the embodiment of the present application is used to solve the model, the memory consumed is 0.24GB, the time used is 5.87 seconds, and the error is 3.81E+055 pixels. It can be clearly seen from the comparison that this solution not only takes up less memory, takes less time, but also has smaller errors. When the databases Dubrovnik and Venice are selected, the solution provided in the embodiment of the present application also takes up less memory, takes less time, and has smaller errors.

[0144] exist Figure 1a 、 Figure 1b ,as well as Figures 2 to 8 On the basis of the corresponding embodiment, in order to better implement the above solution of the embodiment of the present application, the following also provides related equipment for implementing the above solution. Figure 9 , Figure 9 A structural diagram of a processing device for a model provided in an embodiment of the present application, wherein the processing device 900 of the model includes: an acquisition module 901, a segmentation module 902, an update module 903, and a loop module 904. Among them, the acquisition module 901 is used to obtain a graph structure corresponding to a first model, wherein the first model includes multiple first functions, the first model includes multiple first independent variables, the graph structure includes multiple nodes and edges between the multiple nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the multiple nodes in the graph structure are determined according to the multiple first functions; the segmentation module 902 is used to randomly segment the graph structure to obtain at least two first sets, each first set including at least one first independent variable; the update module 903 is used to update the value of the first independent variable in each first set using the Levenberg-Marquardt LM algorithm; the loop module 904 is used to repeatedly perform the random segmentation operation and the operation of updating the value of the first independent variable in each first set until the convergence condition of the first model is met and the solution of the multiple first independent variables in the first model is obtained.

[0145] In one possible design, each edge in the graph structure corresponds to two nodes, and one node represents a first independent variable in the first model. The segmentation module 902 is specifically used to: randomly sort multiple edges of the graph structure to obtain multiple edges arranged in a first order, and obtain a first edge from the multiple edges in the first order, where the first edge is one of the multiple edges of the graph structure; when the two first independent variables corresponding to the first edge are not paired, pair the two first independent variables corresponding to the first edge to obtain a first subset; and perform a merge operation on the multiple first subsets corresponding to the graph structure to obtain at least two first sets.

[0146] In one possible design, the segmentation module 902 is specifically used to: obtain correlation information corresponding to multiple first subsets based on the edges between multiple nodes in the graph structure, wherein the correlation information indicates the correlation between each first subset in the multiple first subsets, and the correlation information includes the first correlation between the second subset and the third subset, the second subset and the third subset are any two different subsets in the multiple first subsets, and the first correlation is determined based on the correlation between the independent variables in the second subset and the independent variables in the third subset, and the more edges there are between two nodes, the higher the correlation between the independent variables pointed to by the nodes; according to the correlation information corresponding to the multiple first subsets, the multiple first subsets corresponding to the graph structure are merged, and the higher the correlation between the two first subsets, the greater the probability of the two first subsets being merged.

[0147] In one possible design, the segmentation module 902 is further used to, when the two first independent variables corresponding to the second edge exist in two different first sets, remove the two first independent variables corresponding to the second edge from the two different first sets, and determine the two first independent variables corresponding to the second edge as two isolated first independent variables, respectively, and the second edge is any one of the multiple edges of the graph structure; the update module 903 is specifically used to adopt the Levenberg-Marquardt LM algorithm to update the values ​​of the first independent variables in each first set and the isolated first independent variables; the loop module 904 is specifically used to repeatedly perform the random segmentation operation and the operation of updating the values ​​of the first independent variables in each first set and the isolated first independent variables.

[0148] In one possible design, the updating module 903 is specifically used to: obtain at least one second function corresponding to the first independent variable in the first set, where the at least one second function is determined based on multiple first functions included in the first model; calculate the function value of the second function according to the first value of each first independent variable in the first set and the second function according to the Levenberg-Marquardt LM algorithm, where the first value is the value of the first independent variable before the update; calculate the Jacobian matrix corresponding to the second function according to the Levenberg-Marquardt LM algorithm, where the Jacobian matrix includes the derivative of the first scalar with respect to the second independent variable, the first scalar is the sum of the values ​​of all second functions corresponding to the first independent variable in the first set, the second independent variable is the independent variable constituting the at least one second function, different second independent variables have different positions in the second function, and different first independent variables have different identifiers; generate the second value of the first independent variable in the first set according to the Levenberg-Marquardt LM algorithm, based on the first value of each first independent variable in the first set, the function value of the second function, and the Jacobian matrix, where the second value is the updated value.

[0149] In a possible design, a convergence condition of the first model is that a value of the first model is less than a first threshold, or a difference between the second value and the first value is less than a second threshold.

[0150] In one possible design, the first model is a least squares model, and the first model is used in any of the following application scenarios: three-dimensional modeling, image fusion, and face reconstruction mesh deformation.

[0151] It should be noted that the information interaction, execution process and beneficial effects between the modules / units in the processing device 900 of the model are the same as those in the present application. Figure 5 and Figure 6 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0152] Next, we introduce a model processing device provided by an embodiment of the present application. When the model processing device is in the form of a terminal device, please refer to Figure 10 , Figure 10 A schematic diagram of a terminal device provided in an embodiment of the present application, wherein the terminal device 1000 is used to implement Figure 8 The functions of the terminal device in the corresponding embodiment. Specifically, the terminal device 1000 includes: a receiver 1001, a transmitter 1002, a processor 1003 and a memory 1004 (wherein the number of the processor 1003 in the terminal device 1000 can be one or more, Figure 10(taking one processor as an example), the processor 1003 may include an application processor 10031 and a communication processor 10032. In some embodiments of the present application, the receiver 1001, the transmitter 1002, the processor 1003 and the memory 1004 may be connected via a bus or other means.

[0153] The memory 1004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1003. A portion of the memory 1004 may also include non-volatile random access memory (NVRAM). The memory 1004 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0154] Processor 1003 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0155] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1003. Processor 1003 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1003. The above processor 1003 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1003 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1004, and processor 1003 reads the information in memory 1004 and, in conjunction with its hardware, completes the steps of the above method.

[0156] Receiver 1001 can be used to receive input digital or character information and generate signal input related to executing device-related settings and function control. Transmitter 1002 can be used to output digital or character information through the first interface. Transmitter 1002 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack. Transmitter 1002 can also include a display device such as a display screen.

[0157] It should be noted that for the specific implementation of the processing method of the application processor 10031 executing the model and the beneficial effects brought about, reference can be made to Figure 8 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0158] When the model's processing device behaves as a server, see Figure 11 , Figure 11 This is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1100 is used to implement Figures 2 to 7The functions of the main server or sub-server in the corresponding embodiment. Specifically, the server 1100 is implemented by one or more servers. The server 1100 may have relatively large differences due to different configurations or performances. It may include one or more central processing units (CPU) 1122 (for example, one or more processors) and memory 1132, one or more storage media 1130 (for example, one or more mass storage devices) for storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 can be temporary storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1122 can be configured to communicate with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0159] The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0160] It should be noted that for the specific implementation of the processing method of the CPU 1122 executing the model and the beneficial effects brought about, reference can be made to Figures 2 to 7 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0161] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the aforementioned Figures 2 to 8 The illustrated embodiment describes the steps performed by the model processing device in the method.

[0162] The present application also provides a computer-readable storage medium in which a program for signal processing is stored. When the program is run on a computer, the computer executes the above-mentioned Figures 2 to 8 The illustrated embodiment describes the steps performed by the model processing device in the method.

[0163] The processing device, terminal device or server of the model provided in the embodiment of the present application can be specifically a chip, which includes: a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit. The processing unit can execute the computer execution instructions stored in the storage unit to enable the chip in the processing device of the model to execute the above Figures 2 to 8 The processing method of the model described in the illustrated embodiment. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0164] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0165] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0166] Through the description of the above embodiments, it is clear to those skilled in the art that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course it can also be implemented by means of dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits, or dedicated circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, an execution device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0167] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0168] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, an execution device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, execution device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as an execution device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive (SSD)).

Claims

1. A model processing method, characterized in that: The method comprises: The primary server obtains a graph structure corresponding to a first model, wherein the first model includes a plurality of first functions, the first model includes a plurality of first independent variables, the graph structure includes a plurality of nodes and edges between the plurality of nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the plurality of nodes in the graph structure are determined according to the plurality of first functions; The main server randomly divides the graph structure to obtain at least two first sets, each first set including at least one first independent variable; The main server sends the first set to the sub-server, so that the sub-server updates the value of the first independent variable in the first set using the Levenberg-Marquardt LM algorithm to obtain an updated value of the first independent variable; The main server receives the updated value of the first independent variable sent by the sub-server; The main server repeatedly performs the random splitting operation, the sending of the first set to the sub-server, and the receiving of the updated value of the first independent variable sent by the sub-server until the convergence condition of the first model is met, and the solutions of the multiple first independent variables in the first model are obtained.

2. The method according to claim 1, characterized in that Each edge in the graph structure corresponds to two nodes, and one node represents a first independent variable in the first model; The randomly dividing the graph structure to obtain at least two first sets includes: Randomly sorting the multiple edges of the graph structure to obtain multiple edges arranged in a first order, and obtaining a first edge from the multiple edges according to the first order, where the first edge is one of the multiple edges of the graph structure; In a case where the two first independent variables corresponding to the first side are not paired, pairing the two first independent variables corresponding to the first side to obtain a first subset; A merging operation is performed on multiple first subsets corresponding to the graph structure to obtain at least two first sets.

3. The method according to claim 2, characterized in that The merging operation on the plurality of first subsets corresponding to the graph structure includes: Obtaining, based on edges between a plurality of nodes in the graph structure, association information corresponding to the plurality of first subsets, wherein the association information indicates an association between respective first subsets in the plurality of first subsets, and the association information includes a first association between a second subset and a third subset, wherein the second subset and the third subset are any two different subsets in the plurality of first subsets, and the first association is determined based on an association between independent variables in the second subset and independent variables in the third subset, wherein the more edges between two nodes, the higher the association between the independent variables pointed to by the nodes; A merging operation is performed on the multiple first subsets corresponding to the graph structure according to the association information corresponding to the multiple first subsets. The higher the association between two first subsets, the greater the probability of merging the two first subsets.

4. The method according to claim 2, characterized in that After performing the merging operation on the multiple first subsets corresponding to the graph structure to obtain the at least two first sets, the method further includes: if the two first independent variables corresponding to the second edge exist in two different first sets, removing the two first independent variables corresponding to the second edge from the two different first sets, and determining the two first independent variables corresponding to the second edge as two isolated first independent variables, respectively, where the second edge is any one of the multiple edges of the graph structure; The updating of the value of the first independent variable in the first set by using the Levenberg-Marquardt LM algorithm includes: The Levenberg-Marquardt (LM) algorithm is used to update the values ​​of the first independent variable in the first set and the isolated first independent variable.

5. The method according to any one of claims 1 to 4, characterized in that The updating of the value of the first independent variable in the first set by using the Levenberg-Marquardt LM algorithm includes: Obtaining a second function corresponding to a first independent variable in the first set, where the second function is determined based on a plurality of first functions included in the first model; Calculating, according to the Levenberg-Marquardt LM algorithm, a function value of the second function based on the first value of each first independent variable in the first set and the second function, where the first value is the value of the first independent variable before the update; Calculate, according to the Levenberg-Marquardt (LM) algorithm, a Jacobian matrix corresponding to the second function, where the Jacobian matrix includes a derivative of a first scalar with respect to a second independent variable, the first scalar being the sum of values ​​of all second functions corresponding to the first independent variables in the first set, the second independent variables being independent variables constituting the second function, different second independent variables having different positions in the second function, and different first independent variables having different identities; According to the Levenberg-Marquardt LM algorithm, the second value of the first independent variable in the first set is generated based on the first value of each first independent variable in the first set, the function value of the second function and the Jacobian matrix, and the second value is the updated value.

6. The method according to claim 5, characterized in that The convergence condition of the first model is that the value of the first model is less than a first threshold, or the difference between the second value and the first value is less than a second threshold.

7. The method according to any one of claims 1 to 4, characterized in that The first model is a least squares model, and the first model is used in any of the following application scenarios: three-dimensional modeling, image fusion, and face reconstruction mesh deformation.

8. A model processing device, characterized in that: The device is applied to a main server, and includes: an acquisition module, configured to acquire a graph structure corresponding to a first model, wherein the first model includes a plurality of first functions, the first model includes a plurality of first independent variables, the graph structure includes a plurality of nodes and edges between the plurality of nodes, the nodes in the graph structure correspond to the first independent variables, and the edges between the plurality of nodes in the graph structure are determined according to the plurality of first functions; a segmentation module, configured to randomly segment the graph structure to obtain at least two first sets, each first set including at least one first independent variable; an updating module, configured to send the first set to a sub-server, so that the sub-server updates the value of the first independent variable in the first set using a Levenberg-Marquardt (LM) algorithm to obtain an updated value of the first independent variable; The updating module is further configured to receive the updated value of the first independent variable sent by the sub-server; A loop module is used to repeatedly perform the random segmentation operation, the sending of the first set to the sub-server, and the receiving of the updated value of the first independent variable sent by the sub-server until the convergence condition of the first model is met and solutions for the multiple first independent variables in the first model are obtained.

9. The device according to claim 8, characterized in that Each edge in the graph structure corresponds to two nodes, one node represents a first independent variable in the first model, and the segmentation module is specifically configured to: Randomly sorting the multiple edges of the graph structure to obtain multiple edges arranged in a first order, and obtaining a first edge from the multiple edges according to the first order, where the first edge is one of the multiple edges of the graph structure; In a case where the two first independent variables corresponding to the first side are not paired, pairing the two first independent variables corresponding to the first side to obtain a first subset; A merging operation is performed on multiple first subsets corresponding to the graph structure to obtain at least two first sets.

10. The device according to claim 9, characterized in that The segmentation module is specifically used to: Obtaining, based on edges between a plurality of nodes in the graph structure, association information corresponding to the plurality of first subsets, wherein the association information indicates an association between respective first subsets in the plurality of first subsets, and the association information includes a first association between a second subset and a third subset, wherein the second subset and the third subset are any two different subsets in the plurality of first subsets, and the first association is determined based on an association between independent variables in the second subset and independent variables in the third subset, wherein the more edges between two nodes, the higher the association between the independent variables pointed to by the nodes; A merging operation is performed on the multiple first subsets corresponding to the graph structure according to the association information corresponding to the multiple first subsets. The higher the association between two first subsets, the greater the probability of merging the two first subsets.

11. The device according to claim 9, characterized in that The segmentation module is further configured to, when two first independent variables corresponding to a second edge exist in two different first sets, remove the two first independent variables corresponding to the second edge from the two different first sets, and determine the two first independent variables corresponding to the second edge as two isolated first independent variables, respectively. The second edge is any one of the multiple edges of the graph structure. The update module is specifically configured to send the first set and the isolated first independent variable to the sub-server, so that the sub-server updates the values ​​of the first independent variable in the first set and the isolated first independent variable using the Levenberg-Marquardt LM algorithm.

12. The device according to any one of claims 8 to 11, characterized in that The updating of the value of the first independent variable in the first set by using the Levenberg-Marquardt LM algorithm includes: Obtaining a second function corresponding to a first independent variable in the first set, where the second function is determined based on a plurality of first functions included in the first model; Calculating, according to the Levenberg-Marquardt LM algorithm, a function value of the second function based on the first value of each first independent variable in the first set and the second function, where the first value is the value of the first independent variable before the update; Calculate, according to the Levenberg-Marquardt (LM) algorithm, a Jacobian matrix corresponding to the second function, where the Jacobian matrix includes a derivative of a first scalar with respect to a second independent variable, the first scalar being the sum of values ​​of all second functions corresponding to the first independent variables in the first set, the second independent variables being independent variables constituting the at least one second function, different second independent variables having different positions in the second function, and different first independent variables having different identities; According to the Levenberg-Marquardt LM algorithm, the second value of the first independent variable in the first set is generated based on the first value of each first independent variable in the first set, the function value of the second function and the Jacobian matrix, and the second value is the updated value.

13. The device according to claim 12, characterized in that The convergence condition of the first model is that the value of the first model is less than a first threshold, or the difference between the second value and the first value is less than a second threshold.

14. The device according to any one of claims 8 to 11, characterized in that The first model is a least squares model, and the first model is used in any of the following application scenarios: three-dimensional modeling, image fusion, and face reconstruction mesh deformation.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that The invention comprises a program, which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

17. An execution device, characterized in that: comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory, so that the execution device executes the method according to any one of claims 1 to 7. 18 . A chip, wherein the chip is configured in an execution device, so that the execution device implements the method according to claim 1 .

Images