A data processing method and apparatus

Abstract

The application provides a data processing method and device, which can be used for solving large-scale linear programming problems, and introduces a parallel mechanism by data splitting to improve the efficiency of solving problems. The method comprises the following steps: obtaining business data for defining business characteristics, wherein the business data comprises constant data for solving an objective function; obtaining at least one constraint term, wherein the constraint term comprises a constraint relationship between a decision variable in the objective function and the constant data; splitting the constant data to obtain a plurality of data blocks, each data block corresponding to at least one sub-constraint term; then distributing the plurality of data blocks to a plurality of computing nodes for parallel processing, so that the plurality of computing nodes generate a plurality of constraint blocks in parallel according to the plurality of data blocks and the corresponding sub-constraint terms to obtain a constraint matrix; and then determining the value of the decision variable according to the constraint matrix to achieve the solution target of the business problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of business planning, and particularly relates to a data processing method and device. BACKGROUND

[0002] A planning problem is a process of modeling a complex real problem and then solving it. Its application in the industry is extremely common, such as warehouse inventory, supply and demand simulation, multi-factory production scheduling, vehicle scheduling, etc. can use a planning model to obtain an optimal solution and select an appropriate scheme to achieve the demand. An algebraic modeling system (AMS) is a system used by the industry to solve problems using a planning method. An algebraic modeling language (AML) is a high-level computer language used to describe and solve large-scale and complex planning problems. AML can be used to convert requirements into AMS, so as to realize the optimal solution through AMS and select an appropriate scheme to achieve the demand.

[0003] However, with the sharp increase in complexity and scale of actual problem business scenarios, some planning problems in the industry become very large and complex, and higher requirements are put forward for the performance of AMS, such as the need to adapt to large-scale, short time and rapid change, etc. Therefore, how to efficiently solve the problem becomes a problem to be solved. SUMMARY

[0004] The present application provides a data processing method and device, which can be used to solve large-scale linear programming problems, introduce a parallel mechanism by data splitting, and improve the efficiency of solving problems.

[0005] In a first aspect, the present application provides a data processing method, comprising: first, obtaining service data for defining service features, the service data including constant data, the service features being features related to a service problem and being used to represent the service problem, such as the type of data related to the service problem, the type of data related to a decision variable, etc., the service data being used to solve a target function, the target function indicating a solution target of the service problem, the target function including a decision variable related to the service features; then, obtaining at least one constraint term, the constraint term including a constraint relationship between the decision variable and the constant data; splitting the constant data to obtain a plurality of data blocks, each data block corresponding to at least one sub-constraint term, the at least one constraint term including the at least one sub-constraint term, the at least one sub-constraint term including a constraint relationship between the split constant data and the decision variable; then distributing the plurality of data blocks to a plurality of computing nodes for parallel processing, so that the plurality of computing nodes generate a plurality of constraint blocks in parallel according to the plurality of data blocks and the sub-constraint term corresponding to each data block, the plurality of constraint blocks forming a constraint matrix; and then determining the value of the decision variable according to the constraint matrix to achieve the solution target of the service problem.

[0006] Therefore, in the embodiments of the present application, the constant data provided by the service side can be split to obtain a plurality of data blocks, and the constraint term is also divided into a plurality of sub-constraint terms and distributed to a plurality of computing nodes, so that the plurality of computing nodes can process the plurality of data blocks in parallel, thereby quickly obtaining the constraint matrix relied on by the service problem, and efficiently obtaining the solution target of the service problem, thereby improving the efficiency of solving the service problem. Moreover, for some multi-core or multi-thread devices or multiple devices, such as server clusters, the method provided by the present application can be used to solve the service problem, which can fully utilize the hardware devices and improve the effective utilization rate of the hardware devices and the working efficiency of the devices. For example, the method provided by the present application can be executed by a server, and the plurality of computing nodes can include a plurality of servers, computers, mobile phones and other electronic devices with data processing capabilities, or a plurality of processors in an electronic device, or a plurality of processes in at least one processor, or a plurality of threads possessed by at least one processor, thereby having the ability to process data in parallel. Through the method provided by the present application, the plurality of data blocks can be processed in parallel, thereby efficiently obtaining the constraint matrix.

[0007] In a possible implementation, before splitting the input data into a plurality of data blocks, the method can further include: converting the input data into a sparse tensor including a plurality of constants, and splitting the sparse tensor to obtain the plurality of data blocks.

[0008] Therefore, in the embodiments of the present application, the data blocks can be fused with the corresponding sub-constraint terms in parallel by multiple computing nodes, so that multiple constraint blocks can be efficiently obtained, the efficiency of composing the constraint matrix is improved, and the efficiency of solving the business problem is improved. Specifically, the input data can be converted into a sparse tensor, and the sparse tensor can be segmented. The sparsity of the data can be fully utilized to construct the model of the subsequent business problem, so that the complexity of construction is independent of the dimension and summation range of the constraint expression. In particular, for large-scale solving problems, the solving time is positively correlated with the size of the constant, so that efficient solving is achieved.

[0009] In a possible implementation, the constraint term can be converted into the form of an expression graph, such as associating each variable and / or the variable and a specific term in the constraint term, so that the association between each variable and / or the variable and the specific term in the constraint term can be more clearly expressed, and the constraint term can be more conveniently divided into multiple sub-constraint terms.

[0010] In a possible implementation, the steps performed by any one of the computing nodes can include: taking any one data block (denoted as a first data block) and the corresponding sub-constraint term (denoted as a target sub-constraint term) as an example, the first computing node substitutes the first data block into the target sub-constraint term to obtain a sub-constraint expression, then filters out a non-trivial term in the sub-constraint expression, the non-trivial term includes at least one variable in the decision variable, and analyzes (such as extracting the coefficient corresponding to each variable) the non-trivial term to obtain the corresponding constraint block. Therefore, in the embodiments of the present application, the multiple computing nodes can respectively traverse the sub-constraint expression, efficiently filter out the non-trivial term in the sub-constraint expression, and quickly find the variable in the sub-constraint term.

[0011] In a possible implementation, the solving target of the business problem obtained according to the constraint matrix can include: dividing the constraint matrix in the time dimension to obtain multiple sub-problems; and then solving the multiple sub-problems respectively to obtain the value of the decision variable, and the value of the decision variable constitutes the solving target of the business problem.

[0012] In the embodiments of the present application, the business problem can be decomposed into multiple sub-problems in multiple time periods by dividing the constraint matrix in the time dimension, and then each sub-problem is solved. The solution of the multiple sub-problems can constitute the solution of the business problem. Therefore, the business problem is segmented, and then each sub-problem is solved, so that the workload of solving is reduced, and the large-scale problem does not need to be solved at the same time, and the solving efficiency is improved.

[0013] In a possible implementation, the solving the plurality of sub-problems respectively can include: aggregating the constraint matrix, and solving the aggregated matrix to obtain a guide solution; and solving the plurality of sub-problems by taking the guide solution as a constraint condition to obtain values of at least one of the decision variables included in the plurality of sub-problems.

[0014] The application provides an aggregation mechanism, generates a small-scale linear programming master problem, solves the small-scale master problem, and uses the solution to guide the solving of each sub-problem, and the solution of the sub-problem forms a constraint. In the implementation of the application, the model data of the linear programming is aggregated, and the scale of the master problem is reduced. Meanwhile, the solving process of the sub-problems is guided by adding the solution of the master problem, so that the solution of the sub-problems is closer to the solution of the master problem, and the problems of being unable to capture long-period correlations, being easily trapped in local optimum, and / or being easily infeasible solution are avoided, and the effectiveness of the final solution is improved.

[0015] In a possible implementation, the solving the plurality of sub-problems by taking the guide solution as a constraint condition can include: solving the plurality of sub-problems in a preset order by taking the guide solution as a constraint condition to obtain values of at least one of the decision variables included in the plurality of sub-problems, wherein at least one sub-problem arranged after the first sub-problem in the preset order is aggregated to obtain an additional sub-problem, the first sub-problem is any one of the plurality of sub-problems, the first sub-problem and the additional sub-problem are fused to obtain a fused sub-problem, and the fused sub-problem is solved to obtain values of at least one of the decision variables included in one of the plurality of sub-problems.

[0016] In the implementation of the application, when the business problem is solved in a local unit, the future information of the business problem is fused, and the solution of the aggregated master problem is taken as a constraint to solve each sub-problem, and the effectiveness of the final solution is improved. It is equivalent to fusing the future information of each sub-problem and taking the solution of the master problem as a constraint, so that the loss of optimality of the solution of the business problem is minimized.

[0017] In a possible implementation, the dividing the constraint matrix to obtain the plurality of sub-problems can include: obtaining user input data; determining a decomposition period according to the user input data; and decomposing the constraint matrix according to the decomposition period to obtain the plurality of sub-problems.

[0018] Therefore, in the implementation of the application, the decomposition period can be determined according to user demand, and user experience is improved.

[0019] Optionally, the user input data can be included in the business data or can be independent data relative to the business data. For example, the data processing method provided by the present application can be deployed in a server, and the user can send the user input data, such as the user's requirements for efficiency or optimality of the solution, to the server through a client.

[0020] In a possible implementation, the solving efficiency and the decomposition period have a negative correlation relationship, that is, the smaller the decomposition period or the aggregation period, the higher the solving efficiency, and the higher the efficiency requirement, the shorter the unit time length of division (i.e., the time length corresponding to each sub-problem); and the longer the decomposition period, the more data on which the solving is based, and the higher the effectiveness of the solution. Therefore, a suitable decomposition period or aggregation period can be determined according to the requirements for efficiency and optimality, so that the constraint matrix is decomposed according to the decomposition period, or the sub-problems are aggregated according to the aggregation period, and the like, to facilitate solving.

[0021] In a second aspect, the present application provides a method for generating a production plan, comprising: first, obtaining business data for defining business characteristics, the business data including constant data, the business data being used to solve a target function, the target function indicating a solving target of a business problem, the solving target indicating solving a production plan for producing a product, the constant data including a production target of the product to be produced and parameters related to producing the product, the production target including a quantity of the product to be produced, and the target function including decision variables related to the business characteristics; obtaining at least one constraint item, the constraint item including a constraint relationship between the decision variables and the constant data; dividing the constant data to obtain a plurality of data blocks, each data block corresponding to at least one sub-constraint item, the at least one sub-constraint item including a constraint relationship between the divided constant data and the decision variables; and distributing the plurality of data blocks to a plurality of computing nodes, so that the plurality of nodes generate a plurality of constraint blocks in parallel according to the plurality of data blocks and the sub-constraint items corresponding to each data block, the plurality of constraint blocks forming a constraint matrix; and determining values of the decision variables according to the constraint matrix to achieve the solving target of the business problem, the solving target indicating the production plan for producing the product.

[0022] Therefore, in the embodiments of the present application, the business side can provide data representing features of the business problem, such as production targets of products that need to arrange production plans and parameters related to production, and then the AMS can segment the data provided by the business side to obtain a plurality of data blocks, and correspondingly divide the constraint term into a plurality of sub-constraint terms, so that the plurality of data blocks can be processed in parallel, thereby quickly obtaining the constraint matrix on which the business problem depends, and efficiently obtaining the solution target of the business problem, thereby improving the efficiency of solving the business problem. Moreover, for some multi-core or multi-thread devices or multiple devices, such as server clusters, the method provided by the present application can be used to solve the business problem, which can fully utilize the hardware devices and improve the effective utilization rate of the hardware devices and the working efficiency of the devices. For example, the method provided by the present application can be executed by a server, and the plurality of computing nodes can include a plurality of servers, or a plurality of processors in a server, or a plurality of processes in one processor in a server, or a plurality of threads of at least one processor, thereby having the ability to process data in parallel. Through the method provided by the present application, the plurality of data blocks can be processed in parallel, thereby efficiently obtaining the constraint matrix.

[0023] In a possible implementation, the fusion of each data block in the plurality of data blocks and the corresponding sub-constraint term can specifically include that the input data can be converted into a sparse tensor including a plurality of constants, and the input data is segmented to obtain the plurality of data blocks.

[0024] Therefore, in the embodiments of the present application, the data blocks can be substituted into the corresponding sub-constraint terms in parallel, thereby obtaining the constraint blocks and improving the efficiency of composing the constraint matrix, thereby improving the efficiency of solving the business problem. Specifically, the input data can be converted into a sparse tensor, and the sparse tensor can be segmented, which can fully utilize the sparsity of the data to construct the subsequent model of the business problem, so that the complexity of construction is not dependent on the dimension of the constraint and the summation range of the expression, especially for large-scale solving problems, so that the solving time is positively correlated with the size of the constant, thereby realizing efficient solving.

[0025] In a possible implementation, the constraint term can be converted into the form of an expression graph, such as associating each variable and / or the variable with a specific term in the constraint term, thereby more clearly expressing the association relationship between each variable and / or the variable and the specific term in the constraint term, and more conveniently dividing the constraint term into a plurality of sub-constraint terms.

[0026] In a possible implementation, the step performed by any one of the plurality of computing nodes (hereinafter referred to as a first computing node) can include: fusing, by the first computing node, a first data block (any one of the plurality of data blocks) and a target sub-constraint term corresponding to the first data block to obtain a sub-constraint expression; screening out non-trivial terms in the sub-constraint expression, the non-trivial terms in the sub-constraint expression including at least one variable in the aforementioned decision variables; and performing analysis on the at least one variable (such as extracting a corresponding coefficient of each variable) to obtain a corresponding constraint block. Therefore, in the implementation of the present application, the first computing node is any one of the plurality of computing nodes, and the first computing node can efficiently screen out non-trivial terms in the sub-constraint expression by traversing the sub-constraint expression, thereby quickly finding variables in the sub-constraint term.

[0027] In a possible implementation, the solving target of the business problem is obtained according to the constraint matrix, including: dividing the constraint matrix in a time dimension to obtain a plurality of sub-problems, the business problem indicating a production plan in a preset time period in the time dimension to be solved, and each of the plurality of sub-problems indicating a production plan for producing a product in a corresponding time period to be solved; and solving the plurality of sub-problems respectively to obtain values of decision variables, the values of the decision variables constituting the solving target of the business problem.

[0028] Therefore, in the implementation of the present application, the production plan to be solved can be divided into a plurality of stages in the time dimension, and then the production plan of each stage is solved, thereby reducing the scale of each solving, improving the solving efficiency, quickly obtaining the production plan of each stage, and improving the overall solving efficiency.

[0029] In a possible implementation, the solving the plurality of sub-problems respectively includes: aggregating the constraint matrix, and solving the aggregated matrix to obtain a guide solution; and solving the plurality of sub-problems by taking the guide solution as a constraint condition to obtain values of at least one variable in the decision variables included in the plurality of sub-problems.

[0030] The application provides a data aggregation mechanism to generate a small-scale linear programming master problem and solve the small-scale master problem, and use the solution to guide the solving of each sub-problem, and the solution of the sub-problem forms a constraint. In the embodiments of the application, the model data of the linear programming is aggregated to reduce the scale of the master problem. Meanwhile, the solution of the master problem is used to guide the solving process of the sub-problem, so that the solution of the sub-problem is closer to the solution of the master problem, and the problems of being unable to capture long-period correlations, being easily trapped in a local optimum, and / or being easily trapped in an infeasible solution are avoided, and the effectiveness of the final solution is improved. In a possible implementation, the solving of the multiple sub-problems with the guided solution as a constraint can include: solving the multiple sub-problems in a preset order with the guided solution as a constraint to obtain a value of at least one variable in a decision variable included in the multiple sub-problems, wherein at least one sub-problem arranged after the first sub-problem in the preset order is aggregated to obtain an additional sub-problem, the first sub-problem is any one of the multiple sub-problems, the first sub-problem and the additional sub-problem are fused to obtain a fused sub-problem, and the fused sub-problem is solved to obtain the value of the at least one variable in the decision variable included in one of the multiple sub-problems.

[0031] In the embodiments of the application, when the business problem is solved in a local unit, the future information of the business problem is fused, and the solution of the aggregated master problem is also used as a constraint to solve each sub-problem, and the effectiveness of the final solution is improved. The future information of each sub-problem is fused, and the solution of the master problem is used as a constraint, so that the loss of the optimality of the solution of the business problem is minimized.

[0032] In a possible implementation, the constraint matrix is divided in a time dimension to obtain multiple sub-problems, including: obtaining user input data; determining a decomposition period according to the user input data, the length of the decomposition period being less than a preset length; and decomposing the constraint matrix according to the decomposition period to obtain the multiple sub-problems.

[0033] Therefore, in the embodiments of the application, the decomposition period can be determined according to the user demand, so that the optimality or solving efficiency of the final solution meets the user demand, and the user experience is improved.

[0034] Optionally, the user input data can include business data or be independent data relative to the business data. For example, the data processing method provided by the application can be deployed in a server, and the user can send the user input data, such as a requirement of the user for efficiency or optimality of the solution, to the server through a client.

[0035] In a possible implementation, the solving efficiency and the decomposition period have a negative correlation, the smaller the decomposition period or the aggregation period, the higher the solving efficiency, and the higher the efficiency requirement in dividing the sub-problems, the shorter the unit time length of the division (i.e., the time length corresponding to each sub-problem); and the longer the decomposition period, the more data on which the solving is based, and the higher the validity of the solution. Therefore, a suitable decomposition period or aggregation period can be determined according to the efficiency and optimality requirements, so as to decompose the constraint matrix according to the decomposition period, or aggregate the sub-problems according to the aggregation period, and the like, to facilitate solving.

[0036] In a third aspect, an embodiment of the present application provides a data processing apparatus having a function of implementing the data processing method in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0037] In a fourth aspect, an embodiment of the present application provides a production plan generation apparatus having a function of implementing the method for generating a production plan in the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0038] In a fifth aspect, an embodiment of the present application provides a data processing apparatus including a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program code in the memory to execute functions related to processing in the data processing method in any one of the first aspect. Optionally, the data processing apparatus can be a chip.

[0039] Optionally, the number of the processors can be one or more, and the one or more processors have multiple threads or multiple processes. After the data block is divided in the first aspect, the obtained multiple data can be allocated to the multiple threads or the multiple processes, so that the multiple threads or the multiple processes can process in parallel, quickly obtain multiple constraint blocks, and efficiently obtain a constraint matrix, thereby constructing a planning model of a business problem.

[0040] In a sixth aspect, an embodiment of the present application provides a production plan generation apparatus including a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program code in the memory to execute functions related to processing in the first method in any one of the second aspect. Optionally, the production plan generation apparatus can be a chip.

[0041] Optionally, the number of the processors can be one or more, and the one or more processors have multiple threads or multiple processes; after the data block is split in the foregoing first aspect, the obtained multiple data can be distributed to the multiple threads or the multiple processes, so that the multiple threads or the multiple processes can be processed in parallel, multiple constraint blocks can be quickly obtained, a constraint matrix can be efficiently obtained, and a planning model of a business problem can be constructed.

[0042] In a seventh aspect, an embodiment of the present application provides a digital processing chip or a chip, which comprises a processing unit and a communication interface; the processing unit acquires program instructions through the communication interface; the program instructions are executed by the processing unit; and the processing unit is used for executing the processing-related functions in any of the optional implementation manners of the first aspect or the second aspect.

[0043] In an eighth aspect, an embodiment of the present application provides a computer readable storage medium, which comprises instructions; when the instructions are run on a computer, the computer executes the method in any of the optional implementation manners of the first aspect or the second aspect.

[0044] In a ninth aspect, an embodiment of the present application provides a computer program product comprising computer programs / instructions; when the computer programs / instructions are executed by a processor, the processor executes the method in any of the optional implementation manners of the first aspect or the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A system architecture schematic diagram is provided for the present application;

[0046] Figure 2 Another system architecture schematic diagram is provided for the present application;

[0047] Figure 3 A data processing method flowchart is provided for the present application;

[0048] Figure 4 Another data processing method flowchart is provided for the present application;

[0049] Figure 5 An expression graph schematic diagram is provided for the present application;

[0050] Figure 6 Another data processing method flowchart is provided for the present application;

[0051] Figure 7 A hardware resource allocation manner schematic diagram is provided for the present application;

[0052] Figure 8 A traversal and allocation flowchart schematic diagram is provided for the present application;

[0053] Figure 9 A forward rolling time domain algorithm solving method provided by the present application is shown in the figure;

[0054] Figure 10 A rolling solution method of aggregating future information provided by the present application is shown in the figure;

[0055] Figure 11 A guided rolling time domain solving method provided by the present application is shown in the figure;

[0056] Figure 12 A fusion algorithm solving method provided by the present application is shown in the figure;

[0057] Figure 13 A flowchart of a method for generating a production plan provided by the present application is shown in the figure;

[0058] Figure 14 An application scenario of a planning problem provided by the present application is shown in the figure;

[0059] Figure 15 A solving effect of a data processing method provided by the present application is shown in the figure;

[0060] Figure 16 A structure diagram of a data processing device provided by the present application is shown in the figure;

[0061] Figure 17 A structure diagram of a production plan generation device provided by the present application is shown in the figure;

[0062] Figure 18 A structure diagram of another data processing device provided by the present application is shown in the figure;

[0063] Figure 19 A structure diagram of another production plan generation device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0065] Planning problem is a process of modeling complex real-world problem and then solving it. It is widely used in industry, such as warehouse inventory, supply and demand simulation, multi-factory scheduling, vehicle scheduling, etc. can use planning model to obtain the optimal solution, select the appropriate scheme to achieve the demand. Algebraic modeling system AMS is a system used by industry to solve problems by using planning method. Algebraic modeling language AML is a high-level computer language used to describe and solve large-scale and complex planning problems. The demand can be converted into AMS through AML, so as to realize the optimal solution through AMS, and select the appropriate scheme to achieve the demand.

[0066] First, the AMS system architecture applied by the data processing method provided in the present application is exemplarily described. AMS can be deployed in a server to provide solving service for users, such as the user can input the demand and provide data, so as to solve the data input by the user through AMS to obtain the solution meeting the user's demand, or the server can generate data according to the actual business problem and solve based on the data to obtain the solution that can solve the actual business problem.

[0067] Exemplarily, the architecture applied by AMS can be as shown in Figure 1 , wherein the business side and AMS can be deployed in the same device or in different devices. Generally, the server or other device carrying AMS can have one or more central processing units (CPU), support multi-process or support multi-thread, etc., or can carry AMS through server cluster, so as to realize parallel processing of data. The business side can be used to communicate with AMS through the interface connected with AMS, and transmit data to AMS. For example, the business side can be an application program or a webpage deployed on the client, and the user can input data in the client, and transmit the data input by the user to AMS through the interface connected with AMS of the business side for subsequent processing.

[0068] Business problem

[0069] For example, the business side and AMS can be deployed in the same server or client device, and AMS can be downloaded and installed on the device where the business side is located in the form of software. The data and demand obtained by the business side or generated according to the actual scene can be input to AMS, for example, the data and demand located in the local device or other device can be opened when the software carrying the function of AMS is running. Generally, the data and demand input by the business side to AMS can indicate the business problem.

[0070] For another example, as shown in Figure 2The AMS can be deployed on a server, the business side can be a client program deployed on a client device, and the user can access the AMS system on the server through the client program. The server and the client program are connected through a wired or wireless network. The user can also access the services provided by the AMS on the server through the web page API.

[0071] It should be noted that the data processing method and the production plan generation method provided in the present application can be deployed in the AMS, which can be deployed in a server or a server cluster. The computing nodes mentioned in the present application can be nodes with computing capabilities, such as servers, computers, mobile phones, and other electronic devices with data processing capabilities. They can also be processes or threads in electronic devices with data processing capabilities. One server, one processor, one process, or one thread is equivalent to one computing node, and the following will not be described in detail.

[0072] In the processing of the AMS, the data input by the business side can be used as model data, and the demand can be used as model expression. Then, based on the model expression and the model data, a planning model is constructed. Then, the business problem is solved by the to-be-solved model, and the obtained solution is fed back to the business side as processed data. The business side can perform the next operation according to the received data.

[0073] For example, in the scenario of vehicle scheduling, the vehicle scheduling platform can be deployed in a server, and the business side of the vehicle scheduling platform can be deployed in the server or a client. The business side can generate scheduling requirements for vehicles and input the route requirements, driving time requirements, and other conditions for each vehicle to the AMS. Then, the AMS uses the data input by the business side as model data and model expression, and converts it into a planning model through AML. Then, the scheduling scheme of each vehicle is output through the planning model and fed back to the business side. This is equivalent to inputting a large number of constraint items and data in a linear programming problem by the business side, and solving the business problem based on the data input by the business side.

[0074] For another example, when planning the production plan of a factory, i.e., factory scheduling, the server can be used for planning, and the business side can be deployed in the server or a client. Here, the business side is deployed in the client as an example. The user can provide the number of production lines of the factory, the production capacity of each production line, or the materials required for production in the client. The client sends this information to the AMS. Then, the AMS uses the data input by the business side as model data and model expression, and converts it into a planning model through AML. Then, the production arrangement of the factory, such as the product type and product quantity produced by each production line per day, is planned through the planning model and fed back to the client.

[0075] For example, the process of solving by AMS can specifically include:

[0076] Construction: that is, converting the business-side input problem into a computer-recognizable planning model. Based on AML, the planning model is compiled with actual data into a computer-recognizable model language, which is a bridge between the definition of the planning model and the computer's understanding of the problem.

[0077] Solving: numerical calculation is performed on the input planning model to obtain an optimal solution.

[0078] Analysis: the solution obtained by solving is restored into an actual decision scheme that can be understood by the business side, and the feasibility and robustness of the solution are analyzed.

[0079] Optimization: based on different requirements (such as optimality requirements, efficiency requirements, etc.), optimization algorithm deployment is performed to improve the performance of AMS.

[0080] In recent years, with the expansion of industrial problems and the rapid increase in the complexity and scale of actual problem business scenarios, some industrial planning problems have become very large and complex, and algebraic modeling systems have been greatly hindered, such as low modeling efficiency caused by large-scale sparse problems, inability to cope with dramatic changes in problem size, large differences in solving time on problems of different sizes, and low quality of decomposition algorithm solutions. For example, the planning problem has the following difficulties: large scale, that is, the scale of variables and constraints often exceeds the order of magnitude of millions, and even reaches the order of magnitude of ten million; short time, that is, often needs to output a solution in minutes; fast change, not only the input data changes fast, but also the iteration frequency of the model is high.

[0081] Under this background, AMS needs to be able to support the rapid conversion from industrial data to planning models, while having a clear, flexible, and high-maintenance modeling application programming interface (API), and being able to stably output solutions in terms of efficiency and results on problems of different sizes. Among them, the two key steps in AMS are construction and solving, that is, the process of constructing a planning model and solving based on the constructed planning model, which is also the main dependent step of the performance of AMS.

[0082] The process of constructing a planning model, that is, converting the requirements transmitted by the business side into a planning model, can be understood as describing the requirements through the planning model, so that the server can determine the problem to be solved through the planning model. However, current AMS is usually single-threaded, and generally, the implementation hardware environment of AMS is usually multi-core and low frequency, so the existing AMS may not perfectly match the hardware facilities, not only the efficiency will be reduced, but also the waste of computing resources will be caused.

[0083] The process of solving, i.e. solving the planning model constructed to obtain the solution of the business problem. Since the problem to be solved usually has a sequence (such as time sequence) structure, the commonly used acceleration method is the decomposition algorithm of rolling horizon (RH), i.e. the sequence problem is divided and solved in sequence to accelerate the solving, but because the long-period correlation information cannot be captured, it is easy to fall into the local optimum and the global poor error.

[0084] Therefore, the present application provides a data processing method, which can be used to solve large-scale linear programming problems, such as warehouse order, supply and demand simulation, multi-factory scheduling, vehicle scheduling and the like, introduces the parallel mechanism of one or more threads of a processor and fully utilizes the sparsity of the model, so that the time required for constructing the planning model is only related to the constant scale, and can be greatly accelerated in the hardware environment of multi-core low main frequency, improve the solving efficiency, thereby efficiently completing the planning of warehouse order, supply and demand simulation, multi-factory scheduling, vehicle scheduling and the like. And, fully utilize the global information of the input data, such as using the solution of the main problem to guide the solving process of the sub-problem, improve the solution quality of the decomposition algorithm. The specific steps of AMS provided by the present application are described in detail below.

[0085] Referring to Figure 3 The flowchart of the data processing method provided by the present application is as follows.

[0086] 301, obtaining input data, the input data including business data and at least one constraint term.

[0087] Among them, the business data includes constant data.

[0088] The business data can be used to represent the business characteristics related to the business problem, such as the production quantity of each type of product in the scheduling problem, etc. Different business problems may have different business characteristics. The constant data included in the business data can include part of the parameters in the objective function of the business problem, or all or part of the parameters in the constraint term, etc.

[0089] The at least one constraint term includes the constraint relationship between the decision variable and the constant data, i.e. the at least one constraint term includes the constraint relationship formed between the decision variable and the constant data. Generally, the input data can include data related to the business problem, and the constant data and the constraint term can form the constraint condition of the business problem to be solved, and the decision variable involved in the business problem to be solved, and the value of the decision variable obtained by subsequent solving can constitute the solving target.

[0090] The input data can include data inputted by the business side to the AMS, and the input data can include constant data, and the business data can be used to define the business characteristics, i.e., the data included in the business data can be used to represent the business characteristics of the business problem. It can be understood that each business problem itself has characteristics that are distinguished from other business problems, such as the type of data involved, the amount of data, or the type of target to be solved, etc.

[0091] For example, if the business problem is to solve the scheduling of a factory, the constant data included in the business data can include the number of production lines of the factory, the type of product produced by each production line, the production capacity of each production line, or the cost of producing a product by each production line, etc. information representing business characteristics.

[0092] For another example, if the business problem indicates to solve the logistics planning, the constant data in the business data can include a plurality of nodes, nodes that each line must pass through, or specific information of each node, etc. information that can ensure the business characteristics.

[0093] The business data can be used to solve the objective function, i.e., at least the solution target of the business problem, and the objective function includes decision variables related to business characteristics, i.e., variables to be solved. For example, the objective function can represent the minimum production capacity or the minimum production capacity of the production target of the factory, and the objective function can include variables related to the production capacity of the factory, such as the production capacity of each production line or the production time of each production line, etc.

[0094] When the business side and the AMS are deployed in the same server, the input data can be data generated by the server according to the actual business needs, and when the business side and the AMS are deployed in different devices, such as the AMS is deployed in the server and the business side is deployed in the client, the business data can be data generated by the client according to the actual needs or data inputted by the user in the client, and then transmitted to the server through wired or wireless network, so that the server receives the business data.

[0095] For example, in the scenario of factory production scheduling, the constant data in the input data can include production cost, transportation cost, procurement cost, factory production capacity upper limit or article processing structure, and production target to be achieved or cost to be spent by the factory, and the input data can also include constraint expression. The constant data and constraint expression are fused to form a constraint for the business problem. For example, the constant data can include that there are two existing factories, each with a processing capacity of 1000 units / day, and that one desktop computer requires one display, and the target to be achieved (i.e., constant) can include 2000 desktop computers, 1000 displays, and 800 laptops, etc. The 1000 units / day, 2000 desktop computers, 1000 displays, and 800 laptops can be understood as constants, and the expression of the constraint item can be represented as M computers require N displays. The constraint item and the provided data are combined to indicate the problem to be solved. The business problem can be understood as the problem to be considered when making production plans, which can include the processing sequence, processing amount, processing time, and processing location of desktop computers, displays, and laptops, i.e., decision variables.

[0096] 302、cutting the constant data in the business data to obtain a plurality of data blocks, each data block corresponding to at least one sub-constraint item.

[0097] Among them, the constant data can be cut according to certain rules to obtain a plurality of data blocks, and correspondingly, the constraint item is divided into a plurality of sub-constraint items (for the sake of distinction, referred to as sub-constraint items), each data block and one or more sub-constraint items correspond.

[0098] When cutting data, the constraint item can be cut first, and then the constant data is cut according to the cut sub-constraint item; or the data block can be cut first, and then the constraint item is cut according to the cut data block to obtain a sub-constraint item corresponding to each data block.

[0099] For example, if the constant includes 2000 desktop computers, 1000 displays, and 800 laptops, the data can be divided into 2000 desktop computers, 1000 displays, or 800 laptops, respectively. The constraint can be understood as a constraint on producing the aforementioned 2000 desktop computers, 1000 displays, and 800 laptops, such as producing each computer requiring 1 display, producing each laptop requiring 1 display, which is equivalent to including 2800 constraints of the constraint. Correspondingly, after the data is divided, the constraint is also divided into multiple parts with the division of the data block, and each data block corresponds to one or more sub-constraints. For example, a sub-constraint can be expressed as: producing a computer requires 1 display, producing a laptop requires 1 display, etc.

[0100] Generally, the constraint can be expressed by an expression, and the business data can be expressed by a constant. Of course, the constraint and the business data can also be expressed by other means, such as a matrix, a textual description, etc. For ease of understanding, the business data is taken as a constant and the constraint is taken as an expression in the following embodiments of the present application for example. For example, the constant data can include constants corresponding to requirements or targets, such as the value of the production target, and can also include weighting parameters or parameters corresponding to requirements or targets, such as processing period or processing cost, etc.

[0101] In actual scenarios, the scale is often large, such as tens of thousands of items, hundreds of factories, and often needs to make production scheduling decisions for several months. Just looking at the variables that need to be decided in the problem, there can be as many as 10 10 However, the actual transportation network and processing network are highly sparse, such as items can only be transported between designated factories and within a specified time period, and the raw materials required for the production of items only account for a small part of all codes. The sparsity of these structures is ultimately reflected in the data. Therefore, in the embodiments of the present application, the constant data can be divided, so that the model for solving the problem is only related to the scale of the constant, improving the efficiency of solving the problem. The following will be introduced by taking the division of data from the time dimension as an example.

[0102] In addition, in addition to dividing the constant data from the time dimension, it can also be divided from the types of various data included in the constant data, or according to a predetermined size, etc., such as dividing the same type of data into the same data block. The following embodiments of the present application will be described by taking the division of constant data from the time dimension as an example, and the following will not be described in detail.

[0103] 303、allocating the plurality of data blocks to a plurality of computing nodes, so that the plurality of computing nodes respectively generate a plurality of constraint blocks according to each data block in the plurality of data blocks and the corresponding sub-constraint term in parallel, the plurality of constraint blocks constituting a constraint matrix.

[0104] After the constant data is split, a plurality of data blocks can be obtained. Then the plurality of data blocks and the corresponding sub-constraint terms are allocated to a plurality of computing nodes, so that the plurality of nodes generate a plurality of constraint blocks based on the plurality of data blocks and the corresponding sub-constraint terms. That is, the plurality of computing nodes respectively fuse each data block and the corresponding sub-constraint term to obtain a plurality of fused sub-constraint terms (referred to as constraint blocks for the sake of distinction), and the plurality of constraint blocks can constitute a constraint matrix. The constraint matrix can be understood as a matrix capable of representing a problem to be solved.

[0105] The plurality of data blocks are allocated to the plurality of computing nodes for processing, and the plurality of computing nodes can perform parallel processing on the plurality of data blocks and the corresponding sub-constraint terms, thereby improving the efficiency of obtaining the constraint matrix. A computing node can be a server, a personal computer (PC), a processor, a process, or a thread, etc. having processing capability. The specific application scenario can be adjusted, and the present application does not limit it.

[0106] More specifically, the allocation to the plurality of computing nodes can include a plurality of ways. For the sake of understanding, some possible allocation ways are exemplarily introduced as follows.

[0107] 1. Multi-thread execution

[0108] The data blocks and the corresponding sub-constraint terms can be allocated to a plurality of different threads for execution. The plurality of threads are different threads or different threads under the same process. The constant data in the data blocks is fused into the corresponding sub-constraint terms by the plurality of threads in parallel, to obtain the constraint blocks.

[0109] Specifically, after the plurality of data blocks are obtained by splitting, the API of the operating system can be called to realize the allocation of the data blocks, for example, the process scheduling function of the operating system is used to allocate the processing resources. The plurality of data blocks and the corresponding sub-constraint terms can be randomly allocated to the plurality of threads, or can be evenly allocated to the plurality of threads, or according to the quantity relationship between the data blocks and the threads, the busy degree of the threads, etc. information, to determine how to allocate the plurality of data blocks and the corresponding sub-constraint terms to the plurality of threads, and to allocate each data block and the corresponding sub-constraint term to the plurality of threads by calling the API.

[0110] 2. Multi-process execution

[0111] Wherein, the plurality of data blocks and corresponding sub-constraint terms can be allocated to a plurality of different processes for parallel processing, that is, the data blocks and corresponding sub-constraint terms are fused. Specifically, each different data block can be allocated to a different process for processing, or a plurality of data blocks can be allocated to the same process, that is, one process can process a plurality of data blocks and corresponding sub-constraint terms, which is not limited in the present application.

[0112] Further, each process can further allocate the allocated data blocks to different threads under the process for processing, which can be adjusted according to actual application, and the specific manner of allocating a plurality of data blocks to a plurality of processes is not limited herein.

[0113] For example, the software carrying the AMS function provided in the present application can call the API of the operating system to allocate the plurality of data blocks obtained by segmentation to other processes or threads under other processes. The allocation manner can be random allocation, average allocation, or allocation according to the load of each process or the state of each process, and the specific allocation manner can be determined by the process scheduling function of the operating system, which is not limited herein.

[0114] 3. Multi-CPU execution

[0115] Wherein, in a device with multiple CPUs, a plurality of data blocks and corresponding sub-constraint terms can be allocated to different CPUs for processing, and the specific allocation manner can include but is not limited to random allocation, allocation based on the load of each CPU, or allocation based on the computing power of each CPU, etc.

[0116] For example, the aforementioned data segmentation step can be executed by one CPU, and after the CPU segments a plurality of data blocks, the plurality of data blocks can be allocated to other CPUs for processing through a communication interface or bus between the CPUs. For example, more data blocks can be allocated to a CPU with lower load, and fewer data blocks can be allocated to a CPU with higher load; or more data blocks can be allocated to a CPU with higher computing power, and fewer data blocks can be allocated to a CPU with lower computing power, so as to balance the data processing amount of each CPU and improve the efficiency of processing the plurality of data blocks.

[0117] 4. Multi-device execution

[0118] Wherein, the AMS can be deployed in a server cluster, and the servers in the server cluster can be connected through wired or wireless interfaces. One of the servers can be designated as a server that controls or manages other servers, or each server in the server cluster can control or manage other servers. One of the servers can segment the business data, and the plurality of data blocks and corresponding sub-constraint terms obtained by segmentation can be allocated to different servers in the cluster for processing.

[0119] Specifically, the aforementioned data splitting step can be performed by one of the servers, and after obtaining the plurality of data blocks, the plurality of data blocks can be sent to other servers by the server or a server with a management function in the cluster for processing. For example, the load balancing server in the cluster can allocate the amount of data blocks processed by each server according to the load of each server in the cluster, and the load balancing server can allocate the data blocks and the corresponding sub-constraint terms to each server through the interface connected to each server. For example, less data blocks can be allocated to servers with heavier loads, and more data blocks can be allocated to servers with lighter loads, etc. Therefore, the data blocks can be allocated to each device according to the load of each server, so that the amount of data processed by each server and the load are matched, and the resource utilization of each server is improved.

[0120] In addition, other distributed systems can also be used to implement parallel processing of the plurality of data blocks to speed up the processing, which is not limited here.

[0121] Therefore, in the embodiments of the present application, the constant data can be split into a plurality of data blocks, and the first constraint can also be divided into a plurality of sub-constraint terms, so that each data block and the corresponding sub-constraint term can be processed in parallel by a plurality of computing nodes, and the constraint matrix can be efficiently obtained. And the hardware resources can be fully used, and the utilization rate of hardware resources is improved.

[0122] Optionally, the constant data group can be converted into a sparse tensor, and then the sparse tensor can be split to obtain a plurality of data blocks. Then, the constants in the plurality of data blocks can be substituted into the corresponding sub-constraint terms to obtain a plurality of constraint blocks. For ease of distinction, one of the sub-constraint terms (referred to as a target sub-constraint term) is allocated to a first computing node (any one of the plurality of nodes) for processing as an example. For example, taking any one of the plurality of data blocks as an example, the first computing node substitutes the constants in the first data block into the corresponding sub-constraint term to obtain a constraint block.

[0123] Further, the first computing node can substitute the first data block into the corresponding sub-constraint term to obtain a sub-constraint expression carrying constants, and then the first computing node can traverse the sub-constraint expression to filter out at least one variable included in at least one non-trivial term; and the at least one variable is analyzed, such as filtering out the coefficients of each variable or directly filtering out the non-trivial term, to obtain the constraint block.

[0124] More specifically, in the process of traversing the sub-constraint expression and screening at least one variable in the sub-constraint expression, each term in the sub-constraint expression can be traversed and screened to obtain non-trivial terms, i.e. terms with variables. In addition, in the process of analyzing the at least one variable, coefficients associated with the variable can be screened from the non-trivial terms, and the coefficients of a non-trivial term in the sub-constraint expression can be used as a row or a column in the matrix, and the coefficients of non-trivial terms in multiple sub-constraint terms can form a constraint matrix.

[0125] For example, taking the production capacity of a factory as an example, constant data can be divided into multiple data blocks from the dimension of data types, and the constraint term can be represented as constraint c 1 ,…,c N , and the data block can be represented as constant C. The constraint term can be divided into multiple sub-constraint terms from the dimension of constraint categories, such as c 1 , which can represent the maximum production capacity of a factory in a day, c 2 , which can represent the maximum cost of producing products in a day of a factory, and so on, c 1 can be divided into a sub-constraint term, c 2 can be divided into a sub-constraint term, and so on, or a plurality of constraints associated with a data block can be divided into a sub-constraint term. For example, production targets are divided according to product categories, data related to the production of computers and related data are taken as a data block, and data related to the production of displays and related data are taken as a data block, then a plurality of constraints related to the production of computers can be taken as sub-constraint terms related to the computer data block, and a plurality of constraints related to the production of displays can be taken as sub-constraint terms related to the display data block.

[0126] In a possible implementation, a constraint c i may be further divided into a plurality of sub-constraint terms. For example, when dividing data blocks from the dimension of time, the constraint c i may also be divided from the dimension of time. For example, a constraint term of one item of inventory can be represented as: c inv :inv t,p,i =inv t-1,p,i +ib t,p,i -ob t,p,i , which means that the inventory of the day = the inventory of the previous day + the amount of inventory + the amount of inventory, i: all products, p: all factories, t: all time. After dividing the production target according to the number of products produced in each stage, the corresponding constraint can also be further divided according to each stage, for example

[0127] The constant related to a certain C is substituted into the corresponding sub-constraint term, and then the non-trivial terms are screened out from the sub-constraint term after the substitution of the constant, and then the coefficients of the non-trivial terms in a sub-constraint term are selected as a row or a column of the constraint matrix A, so as to obtain the constraint matrix A. For example, the constraint matrix A can be expressed as:

[0128]

[0129] For example, each row or each multiple of rows represents the coefficients in a sub-constraint term. For example, the first row can represent the coefficients of c 1 after the substitution of the constant in a time period, the second row represents the coefficients of c 1 after the substitution of the constant in another time period, the first column to the third column in the first row and the second row can represent the coefficients of the non-trivial terms in the first term of c 1 , the fourth column to the fifth column in the first row and the second row represents the coefficients of the non-trivial terms in the second term of c 1 , the third row to the fifth row represents the coefficients of c 2 after the substitution of the constant, and so on. For example, the first row can represent the maximum production of a certain factory on the first day, the second row represents the maximum production of a certain factory on the second day, and each column represents the corresponding coefficient of a variable (such as the amount of a certain material required for producing a product, the time required for producing a product, etc.). For example, the value of the ith row and the jth column represents the coefficient of the jth variable in the ith constraint.

[0130] It can be understood that the objective function corresponding to the business problem can include one or more variables, and the constraint matrix combined with the one or more variables included in the objective function corresponding to the business problem can constitute a planning model of the business problem. For example, it is expressed as Ax = b, b is a constant, and x is a variable in the objective function corresponding to the business problem, so as to realize the solution of the business problem based on the constraint matrix. For example, in the factory scheduling scenario, the variables can include the type, quantity or production sequence of the products produced by each machine in the factory every day. As shown in Table 1, the variables in the factory scheduling scenario can be expressed as:

[0131] Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 …… Day N Production line 1 x 1,1 ]]> x 1,2 ]]> x 1,3 ]]> x 1,4 ]]> x 1,5 ]]> x 1,6 ]]> …… x 1,N ]]> … …… …… …… …… …… …… …… …… Production line M x M,1 ]]> x M,2 ]]> x M,3 ]]> x M,4 ]]> x M,5 ]]> x M,6 ]]> …… x M,N ]]>

[0132] Table 1

[0133] x 1,1 represents the production of the first day in the production line 1, x 1,2 represents the production of the second day in the production line 1, and so on.

[0134] 304、According to the constraint matrix, the value of the decision variable is determined to realize the solution target of the business problem.

[0135] The constraint matrix can be used to represent constraints for solving the business problem. After obtaining the constraint matrix, the business problem can be solved under the constraints of the constraint matrix to obtain a solution target. In different businesses, the business meaning of the solution target can be different. For example, in production scheduling, the solution target can be a production plan that minimizes production cost and production time. In traffic, the solution target can be a traffic light display scheme that minimizes waiting time for vehicles and pedestrians in each direction at an intersection. The specific embodiments are not limited here.

[0136] Therefore, in the embodiments of the present application, the constant data provided by the business side can be divided to obtain a plurality of data blocks, and the constraint items can also be divided into a plurality of sub-constraint items, so that the plurality of data blocks can be processed in parallel, thereby quickly obtaining the constraint matrix on which the business problem depends, and efficiently obtaining the solution target of the business problem, thereby improving the efficiency of solving the business problem. Moreover, for some multi-core or multi-thread devices, the method provided by the present application can be used to solve the business problem, which can fully utilize the hardware device and improve the effective utilization rate of the hardware device and the working efficiency of the device.

[0137] The specific solving method by using the constraint matrix can include various methods. If the business problem can be represented by a simple linear relationship, the linear relationship can be directly solved to obtain the solution target of the business problem. The specific solving calculation method can be adjusted according to the actual application scenario. The following exemplary embodiments are described by way of example with some specific solving methods.

[0138] Optionally, the specific solving method can include: dividing the constraint matrix to obtain a plurality of sub-problems; solving the plurality of sub-problems respectively to obtain a plurality of sub-analysis results, each sub-analysis result can include the value of one or more variables in the decision variable, and the plurality of sub-analysis results constitute the analysis result of the business problem.

[0139] First, the constraint matrix can be divided, which can be understood as dividing the business problem into a plurality of sub-problems. For example, the variable to be solved is x, and the constraint matrix can be represented as A, Ax≤b, b is a constant, A is divided into a plurality of parts, and one or more parts can form an expression of a sub-problem in combination with the variable. Thus, each sub-problem can be solved based on the expression of each sub-problem to obtain the solution of each sub-problem, and the solutions of all sub-problems constitute the solution of the business problem.

[0140] More specifically, the way of dividing the constraint matrix can include various manners, for example, the values in the constraint matrix can be selected from the constraint terms, each column or each row can represent the coefficient values corresponding to a certain variable, the constraint matrix can be divided in the way of row or column, so as to divide each type of variable into a sub-problem, or it can also be divided according to the time dimension, for example, the variables in the constraint matrix can be arranged in time sequence, the constraint matrix can be divided into the variables corresponding to multiple time periods according to the time dimension, and the variables in each time period can be understood as a sub-problem.

[0141] Further, after the multiple sub-problems are divided, the way of solving the business problem can also include various manners, including but not limited to one or more of the following manners:

[0142] 1. Directly solving each sub-problem to obtain the solution of each sub-problem, wherein the solution of each sub-problem includes the value of one or more variables in the decision variable, and then fusing the solution of each sub-problem to obtain the solution of the business problem.

[0143] 2. Solving each sub-problem according to a preset order, aggregating all unsolved sub-problems after each sub-problem to obtain an additional sub-problem during the process of solving each sub-problem, then appending the additional sub-problem to the current sub-problem to obtain a fused sub-problem, then solving the fused sub-problem and taking the obtained solution as the solution of the current sub-problem, and after obtaining the solution of all sub-problems (i.e. the value of one or more variables in the decision variable included in the sub-problems), fusing the solution of all sub-problems to obtain the solution of the business problem.

[0144] 3. Aggregating the constraint matrix and solving based on the aggregated matrix to obtain a guide solution; then solving multiple sub-problems with the guide solution as a constraint condition, so that the solution of each sub-problem is closer to the guide solution, to obtain the value of at least one variable in the decision variable included in the multiple sub-problems.

[0145] 4. Solving each sub-problem in combination with the above manners 2 and 3. That is, solving multiple sub-problems according to a preset order with the guide solution as a constraint condition, to obtain the value of at least one variable in the decision variable included in the multiple sub-problems, wherein at least one sub-problem arranged after the first sub-problem in the preset order is aggregated to obtain an additional sub-problem, the first sub-problem is any one of the multiple sub-problems, the first sub-problem and the additional sub-problem are fused to obtain a fused sub-problem, and the fused sub-problem is solved to obtain the value of at least one variable in the decision variable included in one of the multiple sub-problems.

[0146] Therefore, in the embodiments of the present application, the constraint matrix can be divided into multiple parts, thereby obtaining multiple sub-problems, and each sub-problem is solved, thereby improving the solving efficiency of the business problem and quickly obtaining the solution of the business problem.

[0147] Optionally, after the business problem is solved to obtain the analysis result, the analysis result can be verified to verify the effectiveness of the analysis result, and when the effectiveness is lower than a certain value, the solving can be performed again or the business side is prompted to generate data again and solve based on the newly generated data to obtain a usable solution.

[0148] The foregoing introduces the flow of the data processing method provided by the present application. Next, the flow of the data processing method provided by the present application is further illustrated in combination with a specific application scenario. Referring to Figure 4 , another flowchart of the data processing method provided by the present application.

[0149] Among them, the data processing method provided by the present application can be divided into three parts, such as Figure 4 shown in the data preprocessing 410, parallel modeling 420 and solving 430, which is equivalent to the present application proposes a parallel three-stage construction planning model, the foregoing steps are introduced in detail.

[0150] First, the input data includes constant data and input expressions (i.e. constraint terms).

[0151] For example, if the production plan in a certain period of time needs to be arranged, the constant data can include the production quantity of some products in a certain period of time, and the input expression can include the production capacity of the equipment in the factory in a certain period of time, such as the number of equipment, the type of product produced by each equipment, the time spent by each equipment to produce a product or the relationship between each other, and the corresponding expression. The specific parameters involved in the expression, such as the number of equipment, the type of product produced by each equipment, the time spent by each equipment to produce a product, etc. can be carried in the constant data.

[0152] After obtaining the constant data and the input expression, the data preprocessing 410 step can be performed.

[0153] Specifically, the process of data preprocessing 410 can specifically include: converting the constant data into a sparse tensor, or converting the input expression into an expression graph, etc. The expression graph can represent the association relationship between each variable in the constraint term and / or between the variable and the term.

[0154] For example, the objective function (i.e. the model corresponding to the business problem) can be expressed as: min∑ (i,j)∈E x i,j , x i,jrepresents production, the objective function is to minimize production. Constant data can be represented as: V = {1, 2, 3, 4}, E = {{1, 2}, {2, 4}, {1, 3}, {3, 4}}, S = [1, 0, 0, -1}, and the corresponding sparse tensor can be represented as: c = [0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0] T .

[0155] For example, the constraint term can include multiple terms, respectively represented as c 1 ,…,c N , and the expression graph after data preprocessing can be as shown in Figure 5 , that is, after each variable in the constraint term is independently represented, it is associated with other variables in the same term in the constraint term to form a graph form, so that the relationship between variables and between variables and terms is clearer. Among them, Figure 5 , the Inv: inventory variable; Inv(t, p, i): the end-of-period inventory of item i in factory p at time t; x: production variable; LT(p, i): processing period of item i in factory p; Inbound: inbound quantity (will increase inventory) (because of the processing period, the item produced at (t-LT(p, i)) will be completed at t, resulting in an increase in inventory), so the inbound at t includes x(t-LT(p, i)); Outbound: outbound quantity (will decrease inventory), when the parent item code of the item is processed in the BOM, it will consume itself, so when I' is the parent item of item i, the start of I' at t is as the outbound of i, that is, for example, if the screen is i, the mobile phone is i', and processing the mobile phone requires the use of the screen; Z: demand delivery variable; Negate, sum, shift: calculation symbols, respectively subtraction, summation, and subscript shift, such as Figure 5 , shift(t, -1) represents inv(t, p, i) in the t dimension minus 1, that is, inv(t, p, t-1); Left_expr: left expression of constraint, RHS: right side constant term of constraint.

[0156] For example, one of the constraint terms can be represented as: The inbound represents the amount of entering the warehouse, that is, the sum of the production amount x, wherein t represents time, p represents factory, i represents all products, t, p, i are global subscripts, t2 is a local subscript, and C is a constant. The constant usually includes sparse data in a multi-dimensional space, which is exemplarily represented as a set of four tuples (t, p, i, t2) in this embodiment. The foregoing sparse tensor is substituted into the constraint term c to obtain the specific constraint condition. Therefore, in this embodiment, the sparsity of the constant can be fully utilized for subsequent processing, and the constraint condition can be quickly obtained, thereby improving the solving efficiency.

[0157] In addition, the data preprocessing process can also include other steps, such as desensitization processing, noise reduction processing, etc. on the data. The specific application scenarios can be adjusted, and the application only exemplarily illustrates the conversion of the sparse tensor and the expression graph as an example, and is not limited.

[0158] After obtaining the sparse tensor and the expression graph through the data preprocessing 410, the data blocks are segmented based on the sparse tensor, and the sparse tensor is divided into multiple data blocks. Correspondingly, the constraint terms related to each data block in the expression graph are also divided into corresponding sub-constraint terms, that is, each data block is allocated a constraint term, and one data block corresponds to one or more sub-constraint terms. In addition, each data block can also be allocated a corresponding CPU, process or thread computing node, such as one CPU, one process or one thread processing one or more data blocks.

[0159] By substituting the constant in each data block into the sub-constraint term through the computing node, the constraint block (i.e., the constraint block) can be obtained. Multiple constraint blocks obtained by substituting multiple data blocks into the sub-constraint term can constitute a constraint matrix.

[0160] The process of substituting the constant in each data block into the corresponding sub-constraint term by multiple computing nodes can be performed in parallel. For example, one computing node can include one CPU, such as a first CPU that can process a first data block and a corresponding sub-constraint term, and a second CPU that can process a second data block and a corresponding sub-constraint term. Thus, multiple data blocks can be quickly substituted into the corresponding sub-constraint terms, and the efficiency of obtaining the constraint matrix can be improved.

[0161] After obtaining the constraint matrix, the solving 430 step can be performed.

[0162] In the embodiment of the application, the rolling time domain can be used for solving. The constraint matrix is divided into multiple parts from the time dimension, which is equivalent to dividing the business problem into multiple sub-problems from the time dimension, so as to efficiently obtain an effective solution.

[0163] The output requirement can be input by a user or generated by the server according to an actual scenario. Specifically, the output requirement can include an efficiency requirement or an optimality requirement of the solution. The efficiency requirement refers to a requirement range of the efficiency of the solution, and the optimality requirement refers to a requirement of the effectiveness of the solution to the business problem.

[0164] Generally, when the sub-problems are divided, the higher the efficiency requirement is, the shorter the unit time length (i.e., the time length corresponding to each sub-problem) of the division is, that is, the smaller the decomposition period or the aggregation period is, and the higher the solving efficiency is, thereby improving the solving efficiency. The longer the decomposition period is, the more data the solution is based on, and the higher the effectiveness of the solution is. Therefore, a suitable decomposition period or aggregation period can be determined according to the requirements of the efficiency and the optimality, so as to decompose the constraint matrix according to the decomposition period or aggregate the sub-problems according to the aggregation period, and the like, to facilitate the solving.

[0165] After the decomposition period is determined, the business problem can be decomposed. For example, the constraint matrix generally includes a coefficient value of one or more variables corresponding to a time length. After the decomposition period is determined, the constraint matrix can be decomposed in the time dimension according to the decomposition period, and then the expression of the sub-problem can be formed using the decomposed part, thereby facilitating the solving of the sub-problem to obtain the solution of each sub-problem.

[0166] Therefore, in the embodiments of the present application, the data can be decomposed to obtain a plurality of data blocks, and the constraint terms are also divided into a plurality of constraint sub-terms accordingly. Each data block corresponds to one or more constraint sub-terms, so that the constraint conditions are decomposed into a plurality of terms. The server can process each data block and constraint term in parallel to obtain the corresponding constraint block, realize the conversion from the data block to the constraint block, improve the efficiency of obtaining the constraint matrix, and further improve the efficiency of solving the business problem to be solved.

[0167] For the convenience of understanding, the detailed processes of the foregoing steps 420 and 430 are described in more detail below.

[0168] I. Parallel modeling

[0169] The detailed process of the foregoing step 420 can be referred to Figure 6 as described below.

[0170] The step 420 can be specifically divided into three parts, including segmentation, traversal and assignment. The segmentation is to divide the constant data into multiple data blocks, then the data in the data blocks is traversed in parallel, the data included in the data blocks is mapped with the variables in the sub-constraint items, and the mapping relationship between the data blocks and the variables in the sub-constraint items is determined. Then, based on the mapping relationship, the data in the data blocks is substituted into the corresponding constraint items, that is, the data in the data blocks is assigned to the corresponding sub-constraint items by the assigner, to obtain the constraint blocks.

[0171] Firstly, the constant data is divided into multiple data blocks. The specific classification manner can be classified according to the data size, data type, data quantity, preset period or related variables in the constraint items. For example, every 1000 data is divided into a data block, or every type of data is divided into a data block, or the data in each preset period is divided into a data block, or the data with the same type of variables included in each item in the constraint items in the constant data is divided into a data block, etc.

[0172] Correspondingly, after the data blocks are segmented, the constraint items are also divided into multiple sub-constraint items. Each sub-constraint item corresponds to one or more data blocks. Then the constraint blocks and the corresponding data blocks can be assigned to the corresponding CPU or thread, etc. In this application, the assignment to the corresponding CPU is exemplarily illustrated.

[0173] For example, as shown in Figure 7 The constraint item of the inventory can be expressed as: c inv :inv t,p,i =inv t-1,p,i +ib t,p,i -ob t,p,i , that is, the inventory of the day = the inventory of the previous day + the quantity of the warehouse + the quantity of the warehouse - the quantity of the warehouse, i: all products, p: all factories, t: all time. The inventory is divided into multiple sub-constraint items according to a certain period, such as Then each sub-constraint item, such as is assigned to CPU1, and is assigned to CPU2, and is assigned to CPU3, etc.

[0174] Specifically, for example, the expression graph of the constraint item can be expressed as When the constant data is segmented, the variables related to the constant data involved in the constraint items in the expression graph can be classified, such as being divided into multiple parts according to the global subscript, that is, multiple sub-constraint items, and then the constant data is divided into multiple data blocks according to the classification in the constraint items, and the data corresponding to each classification in the constant data is divided into a data block, thereby obtaining multiple data blocks.

[0175] The data in the data block is mapped to the variables in the sub-constraint item by the iterator. After the traversal is completed, the data in the data block is substituted into the corresponding sub-constraint item by the assigner, thereby obtaining the corresponding constraint block. The segmentation mechanism provided in the application guarantees the equivalence of data segmentation and constraint segmentation, so that the data and the constraints can be kept in synchronization, thereby improving the efficiency and feasibility of parallel processing.

[0176] After the constant data is segmented to obtain a plurality of data blocks and the relationship between the data blocks and the sub-constraint items is traversed, one or more data blocks and the corresponding sub-constraint items can be assigned to the corresponding CPU or thread, and then the plurality of data blocks are processed in parallel. The processing process of each CPU or thread on one or more data blocks can be regarded as a pass, and one of the passes is exemplarily introduced below.

[0177] For example, the specific steps of traversal and assignment in a pass can be as shown in Figure 8 .

[0178] The input of the pass is one or more data blocks and the corresponding one or more sub-constraint items. The iterator is used to traverse the sub-constraint items and find the mapping relationship between each data in the data block and the variable in the sub-constraint item.

[0179] Firstly, the iterator includes two layers, i.e., the outer iterator and the inner iterator as shown in Figure 8 . The outer iterator traverses the abstract expression in the sub-constraint item first. The inner iterator is used to traverse the constant in the sub-constraint item and record the corresponding relationship between the identifier of each constant and the value. The identifier can be the global subscript or the assigned subscript of the constant. In the following embodiment, the identifier is exemplarily taken as the global subscript. When a variable is traversed, the global subscript of the constant and the corresponding value are attached to the current variable.

[0180] The assigner is used to extract the variable generated by the iterator and the corresponding global subscript, and assign the variable to each sub-constraint item according to the value of the global subscript, so as to obtain the constraint block. Therefore, the application provides a double-layer iteration mechanism, which constructs the constraint item item by item, thereby obtaining the constraint block after the constant is substituted.

[0181] That is, the mapping relationship between the data in the data block and each variable in the sub-constraint item is found by the inner iterator and the outer iterator, and then the data in the data block is assigned to the sub-constraint item by the assigner, so as to obtain the constraint block.

[0182] For example, the constraint item can be represented as: c inv :inv t,p,i =inv t-1,p,i +ibt,p,i -ob t,p,i , one of the sub-constraint items ib t,p,i For example, the outer iterator traverses the items included in the expression itself, and then the inner iterator further traverses, i.e. traverses the constants in each item (specifically, the constants in the summation symbol), such as traverses the constant values in it, such as: establishes the mapping relationship between the variables and the constraints in it, such as:

[0183] x 0,0,1 →inv 1,0,1 ;

[0184] x 1,1,0 →inv 2,1,0

[0185] x 2,1,0 →inv 2,1,0

[0186] Then the variables in the data block are assigned to the corresponding sub-constraint items, and the constraint block is obtained, such as:

[0187] inv 1,0,1 =x 0,0,1 +…

[0188] inv 2,1,0 =x 1,1,0 +x 2,1,0 +…

[0189] After obtaining multiple constraint blocks, the multiple constraint blocks can be combined. For example, constraint block 1 contains two constraints z1=x1+y1, z2=x2+y2, and constraint block 2 contains two constraints z3=x3+y3, z4=x4+y4, and the combined result contains four constraints z1=x1+y1, z2=x2+y2, z3=x3+y3, z4=x4+y4. Thus, the complete constraint matrix is output, which can constitute the expression representing the business problem.

[0190] Therefore, in the embodiments of the present application, the data blocks and the sub-constraint items can be converted into constraint blocks in a parallel manner, and the constraint matrix used for solving the business problem can be obtained by merging multiple constraint blocks, thereby improving the efficiency of obtaining the constraint matrix. Especially in the scenario of processing a large amount of data, the efficiency of obtaining the constraint matrix can be greatly improved by parallel processing, and the CPU utilization of the device can be improved and resource waste can be reduced. Moreover, the present application iterates the abstract expression through the outer layer and iterates the constant data in a sub-constraint item through the inner layer, thereby ensuring that the expression item of the constraint block is non-trivial (i.e., the coefficient of the variable is non-zero), and screening out the effective constraint block. In addition, the context information is recorded when the expression item is constructed, and the constraint subscript to which the expression belongs is inferred and marked. After all the items are constructed, the constant is allocated to the corresponding constraint item through the marked constraint subscript, thereby obtaining the constraint block and reducing the error rate. In the actual application scenario, for the super-large-scale planning model in the industry, the constraint subscript space is extremely large, often reaching the order of billions; the expression form is given by the user arbitrarily, and there are difficulties such as nested summation symbol and large summation range. Therefore, in the embodiments of the present application, when constructing the model of the solving problem, the sparsity of the data is fully utilized, and the model can be constructed by only iterating the data once (instead of iterating the entire constraint subscript space), thereby improving the construction efficiency and supporting parallelization to adapt to the industrial hardware environment and improve the resource utilization.

[0191] II. Solving

[0192] After obtaining the constraint matrix, the constraint matrix can be divided, the business problem can be divided into multiple sub-problems through the division of the constraint matrix, and then the sub-problems can be solved respectively, thereby reducing the complexity of the solving and improving the solving efficiency.

[0193] The solving process in the data processing method provided by the present application can be applied to a scenario in which a linear programming problem can be divided into multiple stages for solving in the time dimension. For example, the data processing method provided by the present application can be applied to solving the production arrangement of a factory in a month, and the production arrangement in a month can be divided into three stages of production plan, i.e., 10 days for each stage, and the production plan of each 10 days is solved.

[0194] The difference between the result obtained by dividing the constraint matrix and the constraint block described above is that the division granularity or the division dimension is different. For example, the division granularity of the constraint matrix is greater than the size of the constraint block, and therefore the constraint matrix can be divided according to different dimensions or different granularities after the constraint matrix is obtained, so as to facilitate subsequent solving.

[0195] Specifically, when the constraint matrix is divided, it can be divided in the time dimension, for example, the constraint matrix corresponding to a certain time period [t1, t2] is divided into two sub-problems in the time periods [t1, t3] and [t3, t2], and then the sub-problems are solved respectively. When the constraint matrix is divided in the time dimension, the decomposition period can be determined according to user requirements, and the decomposition period is usually negatively correlated with the solving efficiency. For example, if the user requirement is to improve the efficiency, a smaller period can be determined to improve the solving efficiency, and if the user requirement is to improve the optimality of the solution, the decomposition period can be reduced so that more information in the vicinity can be combined when solving each sub-problem, thereby improving the effectiveness of the obtained solution. For example, if the user requires a 50% improvement in solving efficiency, a decomposition period of 5 minutes can be determined to improve the solving efficiency.

[0196] The way of solving multiple sub-problems can also include multiple ways, which will be described below by way of example.

[0197] Solution mode one, rolling horizon decomposition algorithm

[0198] After obtaining multiple sub-problems, each sub-problem can be directly solved to obtain the solution of each sub-problem, that is, the solution of one or more variables in the decision variables included in each sub-problem. By fusing the solution of each sub-problem, the solution of the business problem can be obtained.

[0199] Generally, most industrial problems often have a certain sequence structure, for example, manufacturing requires assembly steps to be performed in sequence, and logistics transportation requires a certain time sequence to be arranged, so the decision-making process (i.e., the solving process of the business problem) is usually in order, such as operation sequence or time sequence. For large-scale mixed integer linear programming problems with sequence structure, direct solving is often too low in efficiency, so a rolling horizon decomposition algorithm can be used to solve the problem to quickly obtain the final solution.

[0200] Specifically, the problem can be divided into a series of sub-problems, each sub-problem only solves the variables included in a part of the sequence, and the remaining part of the sequence that is not solved is aggregated. According to the aggregation method, it can be divided into forward rolling horizon algorithm, backward rolling horizon algorithm, etc.

[0201] For example, the solution mode of the forward rolling horizon algorithm can refer to Figure 9The problem in period T is divided into four subproblems, that is, the constraint matrix is ​​divided into multiple parts. For example, if the variable to be solved is x, the constraint matrix can be represented as A, Ax ≤ b, where b is a constant. Dividing A into multiple parts forms the expressions for the subproblems. The variables included in each subproblem are solved separately to obtain the solution for each subproblem. Then, the solutions of each subproblem are combined to obtain the final solution. The fusion method can be weighted fusion or direct superposition, which can be adjusted according to the actual application scenario.

[0202] Therefore, in this embodiment, the business problem can be solved by the rolling time domain method, which divides the constraint matrix on which the business problem depends into multiple sub-problems, and solves each sub-problem separately, thereby improving the efficiency of solving the business problem and efficiently obtaining the solution target.

[0203] Solution Method 2: Future Aggregation Rolling Horizon (FARH)

[0204] Similar to the first method described above, the constraint matrix is ​​divided into multiple parts to obtain multiple sub-problems. The difference lies in that, when solving each sub-problem, other sub-problems can be aggregated to obtain aggregated information. Then, each sub-problem and the aggregated information are fused, and the fused data is solved to obtain the solution to each sub-problem. Finally, the solutions to all sub-problems are fused to obtain the final solution to the business problem. The methods of aggregating sub-problems or aggregating the business problem mentioned in this application can include summation, weighted summation, or weighted averaging. For example, the production demand for multiple days following a certain sub-problem can be summed to obtain future information. Another example is the weighted averaging of the production cycles of each factory.

[0205] For example, such as Figure 10 As shown, the problem in period T is divided into four subproblems, which is equivalent to dividing the constraint matrix into multiple parts. When solving the sequence block of the first subproblem, the unsolved sequence blocks are aggregated into one period and combined with the first sequence block for solution. When solving the second subproblem, the solution of the first sequence block is fixed first, and the unsolved sequence blocks are aggregated. Then, the second sequence block and the aggregated sequence blocks are solved together. This process continues until the last subproblem. In the process of solving the last subproblem, the solutions of the previously solved subproblems are fixed, and the last sequence block is solved.

[0206] For example, a problem spanning a period of time, such as period T, is divided into N sub-problems. The question then becomes: which period t is being solved? p ~t q When dealing with sub-problems of time intervals, t q+1 ~tN All data at time t are aggregated into one period, which is then appended to the original subproblem to solve for t. p ~t q The sub-problems at each time step are solved until all sub-problems are solved. The solutions to all sub-problems are then merged, such as by weighted fusion or superposition, to obtain the solution to the business problem.

[0207] Therefore, in this embodiment, after dividing the business problem, other sub-problems can be aggregated. This allows for the solution of each sub-problem to be combined with global information, resulting in a more accurate solution for each sub-problem. Essentially, a guiding mechanism is provided, introducing future information into the rolling solution process, thus leading to a more accurate final solution.

[0208] Solution Method 3: Incorporating the Guided Rolling Horizon (GRH) Algorithm from the Main Problem

[0209] Similar to the first method described above, the constraint matrix is ​​divided into multiple parts to obtain multiple subproblems. The difference is that the problem to be solved is also aggregated before solving. That is, the constraint matrix is ​​aggregated, and then the solution is solved based on the aggregated matrix. The aggregated matrix is ​​equivalent to a compressed constraint matrix, that is, the aggregated matrix is ​​smaller than the constraint matrix before splitting. Then, the solution obtained based on the aggregated matrix (i.e., the guiding solution) is used as a constraint to solve each subproblem, so that the solutions of multiple subproblems, after being merged, are close to the guiding solution. For example, the solutions of multiple subproblems, after being merged, are within a certain range near the guiding solution, or the solutions of multiple subproblems, after being superimposed, are within a certain range near the guiding solution, etc.

[0210] For example, such as Figure 11 As shown, the problem in period T is divided into 4 sub-problems. If the business problem is in period T, each period h is aggregated into a smaller problem, resulting in a total of... This is a minor issue. The subproblems in each period are grouped together to form a smaller main problem, meaning the size of the main problem is smaller than the size of the business problem, where h is a positive integer greater than 1. The solution to this smaller main problem is then obtained; that is, the size of the main problem is smaller than the size of the business problem, or in other words, the main problem is a compressed version of the business problem. Subsequently, similar to the RH algorithm, the subproblems in each period h are solved sequentially. The difference is that the solution to the main problem is used to "guide" the solution process of the subproblems, essentially adding the solution to the smaller main problem as a constraint to ensure that the solutions to the subproblems are as close as possible to the solution to the main problem.

[0211] Therefore, in this approach, by incorporating the solution to the main problem to guide the solution process of the subproblems, the solutions to the subproblems become closer to the solution to the main problem, thus improving the effectiveness of the final solution. Furthermore, compared to methods one and two, method three avoids the problem of easily getting trapped in local optima and / or encountering infeasible solutions due to the inability to capture long-term correlations. By using the solution to the main problem to guide the solution of local problems, the effectiveness of the final solution is improved. This is equivalent to providing an aggregation mechanism that generates a smaller-scale linear programming main problem, solves this smaller-scale main problem, and uses this solution to guide the differentiation of each subproblem, constraining the solutions to the subproblems. This keeps the solutions to the subproblems close to the main problem during the rolling solution process, aggregating the linear programming model data, reducing the size of the main problem while maintaining global characteristics, and making the solutions to the subproblems more accurate.

[0212] Solution Method 4: Combining FARH and GRH

[0213] Method four can be understood as a fusion of methods two and three. When solving each subproblem, aggregated information from other subproblems is obtained. This aggregated information is then fused with each subproblem, and the fused data is used to solve for the solution to each subproblem. Simultaneously, while solving the subproblems, the solution to be solved is also aggregated, i.e., the constraint matrix is ​​aggregated. The solution is then based on the aggregated matrix. For clarity, the solution obtained from the aggregated matrix is ​​called the guiding solution. This guiding solution is then used as a constraint to solve each subproblem that has incorporated the aggregated information. This ensures that the fused solutions to multiple subproblems are close to the guiding solution, improving the effectiveness of the final solution. Essentially, by adding additional constraints and objectives when solving subproblems, the sum of the solutions to the subproblems along the time dimension is constrained in an adjustable manner to be closer to the solution to the main problem.

[0214] For example, such as Figure 12 As shown, business issues are aggregated from period T into The main problem of period T is reduced in size, and then the solution to the even smaller main problem is obtained. The problem of period T is divided into 4 subproblems, that is, the constraint matrix is ​​divided into multiple parts. When solving the sequence block of the first subproblem, the unsolved sequence blocks of the following periods are aggregated into one period and merged with the first sequence block for solution. Then, using the solution of the smaller main problem as constraints, the subproblems that aggregate the unsolved sequences are solved respectively, so that the solutions of the subproblems are closer to the solutions of the smaller main problem.

[0215] Therefore, in this mode, the FARH and the GRH are fused, so that when the business problem is solved in a local unit, the future information of the business problem is fused, and the solution of the aggregated main problem is also added as a constraint to solve each sub-problem, so as to improve the effectiveness of the final solution. It is equivalent to using the future information of each sub-problem and the information of the main problem at the same time, so as to minimize the loss of optimality of the solution.

[0216] The foregoing introduces in detail the flow of the data processing method provided by the application. Next, the flow of the method for generating a production plan provided by the application is introduced. Figure 13

[0217] It should be understood that the method for generating a production plan provided by the application and the foregoing data processing method are related. The foregoing business problem is materialized into a production scheduling problem, and the constant data is materialized into the number of products to be produced and the parameters related to the production of the products, which are introduced in detail below.

[0218] 1301. Obtain input data, which is used to indicate that the production plan is solved.

[0219] The input data can include the foregoing business data and at least one constraint item.

[0220] The step is similar to the foregoing step 301, and the difference is that the business problem corresponding to the business data is to indicate that the production plan of the production product is solved, and the constant data included in the business data includes the production target of the product to be produced in a period of time and / or the related parameters of the production product, etc. The production target, such as the minimum number or the maximum number of the product to be produced, etc. The production target can be represented by an objective function, which has a decision variable to be solved. The value of the decision variable obtained after solving can constitute the production plan.

[0221] For example, the constant data can include the target number of products to be produced in March, and the number of factories, the number of production lines of each factory, or the information of materials, cost or cycle required for producing one product, etc.

[0222] 1302. The constant data in the business data is segmented to obtain a plurality of data blocks.

[0223] The constant data in the business data, such as the production target or the related parameters of the production product, etc. is cut into pieces, and is cut according to the data type or from the time dimension, etc. to obtain a plurality of data blocks.

[0224] ​For example, the constant data can include a target quantity of products to be produced in March, and information of the number of factories, the number of production lines of each factory, or materials, cost or cycle required for producing one product, etc. The information can be divided according to data types to obtain multiple data blocks.

[0225] 1303, performing parallel processing on each data block in the multiple data blocks and the corresponding sub-constraint item respectively to obtain multiple constraint blocks.

[0226] Then, the multiple data blocks and the corresponding sub-constraint items are fused by the multiple computing nodes in parallel to obtain multiple constraint blocks.

[0227] 1304, obtaining a production plan according to the constraint matrix.

[0228] The multiple constraint blocks obtained in the foregoing step 1303 can constitute a constraint matrix, and then a production plan can be solved based on the constraint matrix, and a more optimal production plan can be efficiently obtained.

[0229] Among them, steps 1302-1304 are similar to the foregoing steps 302-304, and the application will not repeat the similar parts, and some differences will be introduced below.

[0230] Among them, when the constraint matrix is divided in the time dimension to obtain multiple sub-problems, the business problem indicates a production plan in a preset time period in the time dimension, and each sub-problem indicates a production plan for producing products in a corresponding time period. For example, the business problem is to solve a production plan of a factory in the future one month, which can be divided into 5 sub-problems, i.e. a production plan for every 6 days. Then, the production plan can be solved according to the foregoing Figures 4-12 The solving mode shown is used to solve the business problem.

[0231] Therefore, in the embodiments of the present application, the business side can provide production targets related to products that need to arrange production plans and parameters related to production, etc., and then the data provided by the business side can be divided to obtain a plurality of data blocks, and the constraint items are also divided into a plurality of sub-constraint items, so that the plurality of data blocks can be processed in parallel, thereby quickly obtaining the constraint matrix relied on by the business problem, and efficiently obtaining the solution target of the business problem, thereby improving the efficiency of solving the business problem. And for some multi-core or multi-thread devices or multiple devices, such as server clusters, the method provided by the present application is used to solve the business problem, which can fully utilize the hardware devices and improve the effective utilization rate of the hardware devices and the working efficiency of the devices. For example, the method provided by the present application can be executed by a server, and the plurality of computing nodes can include a plurality of servers, or a plurality of processors in a server, or a plurality of processes in a processor in a server, or a plurality of threads of at least one processor, thereby having the ability to process data in parallel. Through the method provided by the present application, a plurality of data blocks can be processed in parallel to efficiently obtain the constraint matrix.

[0232] To further facilitate understanding, the application scenarios of the data processing method and the method for generating a production plan provided by the present application are exemplarily introduced as follows.

[0233] For example, the data processing method and the method for generating a production plan provided by the present application can be applied to supply-demand simulation. For a supply chain, the differences in objects, time granularity, the number of considered constraints, or customer-oriented differences, etc. may all lead to different supply-demand matching problems. Therefore, there are various supply-demand matching problems in different domains or even in each domain. It is a difficult problem to quickly respond and make reasonable decisions under variable conditions, and there may also be characteristics such as a large number of product categories, large demand volume, and complex production relationships. Therefore, the process needs a more efficient and more accurate decision-making method to support.

[0234] For example, the data processing method provided by the present application can be applied to a plurality of problems that can be solved by linear programming, such as production planning problems in a supply chain and constraint routing problems in a network. The form of linear programming can be expressed as:

[0235] Objective function: min c T x

[0236] Constraint item: s.t. Ax ≥ b; x ≥ 0.

[0237] Wherein, x is a variable, i.e. the quantity that needs to be decided in the business problem; A, b, and c are usually known constants, wherein A is a coefficient matrix, b is a resource limit, and c is a target coefficient.

[0238] The following exemplary application scenarios are exemplarily illustrated.

[0239] 1. Production planning

[0240] Production planning in the supply chain is a special supply-demand matching problem, which aims to decide how to optimally use raw materials to meet customer demand, and the decision-making process involves production, transportation, procurement, etc., and is limited by factory production capacity upper limit, article processing structure, etc. For example, the production target can include 2000 desktop computers, 1000 displays, and 800 laptops; there are two factories, and the processing capacity of each factory is 1000 units / day, and it is known that processing one desktop computer requires one display. When considering production planning, decisions need to be made on the processing sequence, processing amount, processing time, processing location, etc. of desktop computers, displays, and laptops.

[0241] To solve this kind of problem, linear programming can be used to model and solve the problem, and the scale of the business problem in the actual scene is large, so the modeling process needs to be accelerated. In addition, because this kind of problem generally has "inventory update equation" type constraints in the time dimension, and the RH algorithm decomposes the model from the time dimension, it can be adapted to production planning problems. For example, the following model is a planning model for production planning problems, which can be expressed as:

[0242] Objective function:

[0243] Constraint term: s.t.s t +x t =d t +s t (t=1,…n)

[0244] s0=s n =0,

[0245] x t ≤M t y t ,(t=1,…,n),

[0246]

[0247]

[0248] y∈{0,1} n .

[0249] Where t represents the time dimension, and the production amount of n periods needs to be decided in the problem; p t q t h t d tFor constants in the problem, x t ,y t ,s t are variables in the problem, where x t represents the production quantity at time t, s t represents the end-of-period inventory at time t, and y t represents whether an item is produced at time t.

[0250] The first constraint in the constraint term is the "inventory update equation", which represents that the remaining inventory at the end of time t is equal to the remaining inventory at the end of time t-1, plus the amount that leads to an increase in inventory at time t, minus the amount that leads to a decrease in inventory at time t.

[0251] For example, the solution obtained by fitting the production planning problem planning model to the instance and solving the solution can be as shown in Table 1, which includes the processing sequence, processing quantity, processing time, and processing location of the desktop computers, displays, and laptops, such as producing 1000 displays in two factories on the first day, then using the 2000 displays produced on the first day in one of the factories to produce desktop computers on the second and third days, and the other factory producing 1000 displays and 800 laptops on the second and third days, respectively. Figure 14

[0252] Therefore, in the embodiments of the present application, the constant data can be divided, and then the data blocks and the corresponding sub-constraint terms can be processed in parallel, so as to quickly obtain the constraint matrix, and then solve the problem, thereby improving the efficiency of obtaining the constraint matrix. Especially for large-scale linear programming problems, the parallel processing method can greatly improve the solving efficiency, and fully utilize the hardware resources of the device, thereby reducing the waste of resources.

[0253] 2. Logistics planning

[0254] The scheme provided in the present application can be applied to various problems that can be solved using linear programming, such as network flow problems in the logistics field. Here, the minimum-cost flow problem in network flow is exemplarily described.

[0255] The minimum-cost flow problem aims to find an optimal path in a flow network to send a specified amount of supply from a specified starting point to a specified ending point. A path can have multiple points, and there is a cost between two points, so the optimal path generally refers to the path with the minimum total transportation cost.

[0256] ​The planning model can be described as follows: Assume there is a flow network G = (V, E), where V represents all transport nodes, s ∈ V is the starting point, t ∈ V is the ending point, and E represents all edges, i.e., the lines connecting the nodes. Define a(u, v) and f(u, v) to represent the unit transport cost and transport volume from node u to node v, respectively. When the demand for transport volume is d, the planning model for the problem is as follows:

[0257] The objective function is expressed as: (e.g., minimizing total transportation cost) ∑ (u,v)∈E a(u,v).f(u,v)

[0258] The constraint term is expressed as: f(u,v)≤c(u,v)

[0259] f(u,v)=-f(v,u)

[0260]

[0261]

[0262] The first constraint is the "upper limit constraint on transport flow", which means that there is an upper limit to the transport volume between two points; the third constraint is the "flow conservation constraint"; the fourth constraint is used to indicate that the inflow at the starting point and the outflow at the key point must be equal to the demand d.

[0263] The method provided in this application allows for the rapid fusion of constant data and constraint expressions using multiple computing nodes in parallel to obtain a constraint matrix. The optimal transportation route can then be obtained based on this constraint matrix.

[0264] To facilitate understanding of the effects of the data processing method provided in this application on the modeling section, the modeling efficiency of the data processing method provided in this application is compared with some commonly used modeling methods, such as... Figure 15 As shown, the performance of the modeling part of the data processing method provided in this application in both single-threaded and 64-threaded scenarios is compared with the modeling efficiency of Gurobi Python API and ZIMPL. Figure 15 As shown. By Figure 15 It can be seen that, compared with the modeling efficiency of Gurobi Python API and ZIMPL, the modeling efficiency of the single-threaded version of this application can be accelerated by one order of magnitude. Due to the support for parallelism, the modeling efficiency of the multi-threaded version is further improved, with an acceleration rate of 50 to 80 times, and the construction of a million-scale planning model can be completed within 10 seconds.

[0265] In the solution section, the solution effects of the various solution methods provided above are compared, as shown in Table 2:

[0266]

[0267] Table 2

[0268] Wherein, the target value means the value of the target function after being weighted, and the value of the target function is usually the smaller the better, and the solution directly solved is the most accurate, and the algorithm consumed by the direct solution is also larger, wherein the value in the () in the target value represents the optimality loss of the solution, and the smaller the optimality loss is, the better it is when the efficiency is the same; the demand satisfaction rate represents the on-time completion degree of the production target.

[0269] Obviously, from Table 2, it can be seen that the master-slave guiding idea provided by the present application can significantly improve the optimality of the original rolling time domain algorithm. The commonly used full-scale direct solution needs 52 minutes, and several different rolling time domain algorithms need 1-4 minutes, wherein the optimality loss of G-FARH (i.e. the fourth solving mode described above) is the smallest, only 0.36%. In practical application, the algorithm can be selected according to different needs of efficiency and optimality, such as adjusting the segmentation period or the aggregation period, etc.

[0270] The foregoing has introduced the flow of the data processing method provided by the present application in detail, and the following introduces the data processing device provided by the present application based on the flow of the data processing method, which can be used to execute the method steps in the foregoing Figures 3-13 .

[0271] Referring to Figure 16 , a structural schematic diagram of a data processing device provided by the present application.

[0272] The data processing device comprises:

[0273] The transceiver module 1601 is configured to obtain service data for defining service characteristics, wherein the service data comprises constant data, and the service data is used to solve a target function, the target function indicates a solution target of a service problem, and the target function comprises a decision variable related to the service characteristics;

[0274] The transceiver module 1601 is further configured to obtain at least one constraint term, wherein the constraint term comprises a constraint relationship between the decision variable and the constant data;

[0275] The segmentation module 1602 is configured to segment the constant data to obtain a plurality of data blocks, wherein each data block corresponds to at least one sub-constraint term, and the at least one sub-constraint term comprises a constraint relationship between the segmented constant data and the decision variable;

[0276] The distribution module 1603 is configured to distribute the plurality of data blocks to a plurality of computing nodes, so that the plurality of computing nodes generate a plurality of constraint blocks in parallel according to the plurality of data blocks and the corresponding sub-constraint terms, and the data in the plurality of constraint blocks form a constraint matrix.

[0277] The solving module 1604 is configured to determine values of the decision variables according to the constraint matrix to achieve a solution target of the business problem.

[0278] In a possible implementation, the assigning module 1603 is specifically configured to assign the first data block to the first computing node, so that the first computing node performs the following steps: substituting the first data block into a target sub-constraint term to obtain a sub-constraint expression, the first data block being any one of the plurality of data blocks, the target sub-constraint term being a sub-constraint term corresponding to the first data block among the plurality of sub-constraint terms; screening out non-trivial terms in the sub-constraint expression, the non-trivial terms in the sub-constraint expression including at least one of the decision variables; and analyzing the non-trivial terms to obtain a constraint block.

[0279] In a possible implementation, the solving module 1604 is specifically configured to: divide the constraint matrix in a time dimension to obtain a plurality of sub-problems; and solve the plurality of sub-problems respectively to obtain the values of the decision variables, the values of the decision variables constituting the solution target of the business problem.

[0280] In a possible implementation, the solving module 1604 is specifically configured to: aggregate the constraint matrix, and solve the aggregated matrix to obtain a guide solution; and solve the plurality of sub-problems by taking the guide solution as a constraint condition to obtain the values of the decision variables in the plurality of sub-problems.

[0281] In a possible implementation, the solving module 1604 is specifically configured to solve the plurality of sub-problems in a preset order by taking the guide solution as a constraint condition to obtain the values of at least one of the decision variables included in the plurality of sub-problems, wherein at least one sub-problem arranged after a first sub-problem in the preset order is aggregated to obtain an additional sub-problem, the first sub-problem being any one of the plurality of sub-problems, and the first sub-problem and the additional sub-problem are fused to obtain a fused sub-problem, and the fused sub-problem is solved to obtain the values of at least one of the decision variables included in one of the plurality of sub-problems.

[0282] In a possible implementation, the solving module 1604 is specifically configured to: obtain user input data; determine a decomposition period according to the user input data; and decompose the constraint matrix according to the decomposition period to obtain the plurality of sub-problems.

[0283] With reference to Figure 17 The application further provides a production plan generation device, which comprises:

[0284] The transceiver module 1701 is configured to obtain service data used to define service characteristics, the service data including constant data, the service data being used to solve a target function, the target function indicating a solution target of a service problem, the solution target indicating solving a production plan of a production product, the constant data including a production target of the product to be produced and parameters related to the production product, the production target including a quantity of the production product, and the target function including decision variables related to the service characteristics;

[0285] The transceiver module 1701 is further configured to obtain at least one constraint term, the constraint term including a constraint relationship between the decision variables and the constant data;

[0286] The cutting module 1702 is configured to cut the constant data to obtain a plurality of data blocks, each data block corresponding to at least one sub-constraint term, the at least one sub-constraint term being obtained by cutting the at least one constraint term, and the at least one sub-constraint term including a constraint relationship between the cut constant data and the decision variables;

[0287] The distribution module 1703 is configured to distribute the plurality of data blocks to a plurality of computing nodes, so that the plurality of nodes generate a plurality of constraint blocks in parallel according to the plurality of data blocks and corresponding sub-constraint terms, the plurality of constraint blocks forming a constraint matrix, and the plurality of constraint blocks forming constraints when solving the service problem;

[0288] The solving module 1704 is configured to determine values of the decision variables according to the constraint matrix to achieve the solution target of the service problem, the solution target indicating a production plan of the product.

[0289] In a possible implementation, the distribution module 1703 is specifically configured to distribute a first data block to a first computing node, so that the first computing node performs the following steps: substituting the first data block into a target sub-constraint term to obtain a sub-constraint expression, the first data block being any one of the plurality of data blocks, and the target sub-constraint term being a sub-constraint term corresponding to the first data block among the plurality of sub-constraint terms; traversing the sub-constraint expression to filter out at least one variable, a non-trivial term in the sub-constraint expression including at least one variable in the decision variables, and the first computing node being one of the plurality of computing nodes; and analyzing the at least one variable to obtain a constraint block.

[0290] In a possible implementation, the solving module 1704 is specifically configured to: divide the constraint matrix in the time dimension to obtain a plurality of sub-problems, the business problem indicates solving the production plan in a preset time period in the time dimension, and each of the plurality of sub-problems indicates solving the production plan for producing products in a corresponding time period; and solve the plurality of sub-problems respectively to obtain values of at least one of the decision variables included in the plurality of sub-problems, the values of the at least one of the decision variables included in the plurality of sub-problems constitute a solving target of the business problem.

[0291] In a possible implementation, the solving module 1704 is specifically configured to: aggregate the constraint matrix, and solve the aggregated matrix to obtain a guide solution; and solve the plurality of sub-problems by taking the guide solution as a constraint condition to obtain values of at least one of the decision variables included in the plurality of sub-problems.

[0292] In a possible implementation, the solving module 1704 is specifically configured to: solve the plurality of sub-problems in a preset order by taking the guide solution as a constraint condition to obtain values of at least one of the decision variables included in the plurality of sub-problems, wherein at least one sub-problem arranged after a first sub-problem in the preset order is aggregated to obtain an additional sub-problem, the first sub-problem is any one of the plurality of sub-problems, the first sub-problem and the additional sub-problem are fused to obtain a fused sub-problem, and the fused sub-problem is solved to obtain the values of the at least one of the decision variables included in one of the plurality of sub-problems.

[0293] In a possible implementation, the solving module 1704 is specifically configured to: obtain user constant data; determine a decomposition period according to the user constant data, the decomposition period being shorter than the preset time period; and decompose the constraint matrix according to the decomposition period to obtain the plurality of sub-problems.

[0294] The embodiments of the present application also provide a data processing apparatus, please refer to Figure 18 , Figure 18 A structural schematic diagram of the data processing apparatus provided by the embodiments of the present application. The data processing apparatus 1800 can be deployed with Figure 16 The data processing apparatus described in the corresponding embodiments is used to realize Figures 3 to 14 The functions of the data processing apparatus in the corresponding embodiments. Specifically, the data processing apparatus 1800 can have great differences due to different configurations or performances, and can include one or more central processing units CPU 1822 (for example, one or more processors) and a memory 1832, one or more storage media 1830 (for example, one or more mass storage devices) storing application programs 1842 or data 1844.

[0295] The memory 1832 and storage medium 1830 can be temporary or persistent storage. In one embodiment, the memory 1832 is random access memory (RAM), which can directly exchange data with the central processing unit 1822. It is used to load data 1844 and application programs 1842 and / or operating system 1841 for direct execution and use by the central processing unit 1822, and typically serves as a temporary data storage medium for the operating system or other running programs. The program stored in the storage medium 1830 may include one or more modules (…). Figure 18 (not shown in the image), each module may include a series of instruction operations on the data processing device.

[0296] Furthermore, the central processing unit 1822 can be configured to communicate with the storage medium 1830 and execute a series of instructions stored in the storage medium 1830 on the data processing device 1800. In one possible implementation, the storage medium 1830 stores the aforementioned... Figures 3-13 The program instructions and data corresponding to the method steps shown in any of the embodiments.

[0297] The data processing device 1800 may also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input / output interfaces 1858, and / or one or more operating systems 1841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0298] In one possible implementation, the central processing unit 1822 is used to perform the aforementioned... Figures 3-13 The method steps performed by the data processing apparatus shown in any of the embodiments. The number of central processing units can be one or more; when the number of central processing units is multiple, as described above... Figure 5 As shown, after the data is divided into multiple data blocks, these multiple processors can process the multiple data blocks in parallel, thereby improving the solution efficiency.

[0299] This application embodiment also provides a data processing device, which may also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit can obtain program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform the aforementioned... Figures 3-12 The method steps performed by the data processing apparatus shown in any of the embodiments.

[0300] The embodiments of the present application also provide a digital processing chip. The digital processing chip integrates a circuit for implementing the processor 1801 or the functions of the processor 1801 and one or more interfaces. When the digital processing chip integrates a memory, the digital processing chip can complete the method steps of any one or more of the preceding embodiments. When the digital processing chip does not integrate a memory, the digital processing chip can be connected with an external memory through a communication interface. The digital processing chip implements the actions performed by the data processing apparatus in the embodiments of the present application according to program codes stored in the external memory.

[0301] In more detail, when the data processing apparatus provided by the embodiments of the present application is a chip, the chip specifically includes a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin, a circuit or the like. The processing unit can execute computer execution instructions stored in the storage unit, so that the chip in the server executes the actions described above. Figures 3-13 The data processing method described in the embodiments.

[0302] The embodiments of the present application also provide a production plan generation apparatus. Please refer to Figure 19 , Figure 19 A structural diagram of the production plan generation apparatus provided by the embodiments of the present application. The production plan generation apparatus 1900 can be deployed with Figure 16 The production plan generation apparatus described in the corresponding embodiments, for implementing Figures 13 to 14 The functions of the production plan generation apparatus in the corresponding embodiments. Specifically, the production plan generation apparatus 1900 can have great differences due to different configurations or performances, and can include one or more central processing units CPU 1922 (for example, one or more processors) and a memory 1932, one or more storage media 1930 (for example, one or more mass storage devices) for storing application programs 1942 or data 1944.

[0303] The memory 1932 and the storage medium 1930 can be temporary storage or persistent storage. In one embodiment, the memory 1932 is a random access memory RAM, which can directly exchange data with the central processing unit 1922, and is used to load data 1944 and application programs 1942 and / or an operating system 1941 for the central processing unit 1922 to directly run and use, and is usually used as a temporary data storage medium for the operating system or other programs being run. The programs stored in the storage medium 1930 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the production plan generation apparatus. Figure 19

[0304] ​Further, the central processing unit 1922 can be configured to communicate with the storage medium 1930 to execute a series of instructions in the storage medium 1930 on the production plan generation apparatus 1900. In one possible implementation, the storage medium 1930 stores the program instructions and data corresponding to the method steps shown in any of the preceding embodiments. Figures 13-13

[0305] The production plan generation apparatus 1900 can further include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input / output interfaces 1958, and / or one or more operating systems 1941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0306] In one possible implementation, the central processing unit 1922 is configured to execute the method steps performed by the production plan generation apparatus shown in any of the preceding embodiments. The number of central processing units can be one or more. When the number of central processing units is more than one, the multiple central processing units can be configured to perform parallel processing on the multiple data blocks after the data is divided into the multiple data blocks, thereby improving the solving efficiency. Figures 3-13 Figure 5

[0307] Embodiments of the present application also provide a production plan generation apparatus, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit can obtain program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is configured to execute the method steps performed by the production plan generation apparatus shown in any of the preceding embodiments. Figures 13-14

[0308] Embodiments of the present application also provide a digital processing chip. The digital processing chip integrates a circuit for implementing the processor 1901 or the functions of the processor 1901 and one or more interfaces. When the digital processing chip integrates a memory, the digital processing chip can complete the method steps of any one or more of the preceding embodiments. When the digital processing chip does not integrate a memory, the digital processing chip can be connected to an external memory through the communication interface. The digital processing chip implements the actions of the production plan generation apparatus shown in the preceding embodiments according to the program code stored in the external memory.

[0309] ​​​​In more detail, when the production plan generation apparatus provided by the embodiments of the present application is a chip, the chip specifically comprises a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin, a circuit, or the like. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the server executes the above-mentioned Figures 13-14 The method for generating a production plan described in the embodiments shown in the drawings.

[0310] Optionally, the storage unit mentioned above can be a storage unit in the chip, such as a register, a cache, or the like. The storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), or the like.

[0311] Specifically, the processing unit or the processor mentioned above can be a central processing unit, a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or any conventional processor, or the like.

[0312] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above-mentioned Figures 3-14 method by a program.

[0313] The embodiments of the present application also provide a computer readable storage medium, which stores a program, and when the program runs on a computer, the computer executes the steps in the method described in the embodiments shown in the above-mentioned Figures 3-14

[0314] The embodiments of the present application also provide a computer program product, and when the computer program product runs on a computer, the computer executes the steps executed by the data processing apparatus or the production plan generation apparatus in the method described in the embodiments shown in the above-mentioned Figures 3-14

[0315] ​​It should be noted that the above-described apparatus embodiments are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiments provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0316] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc., including a plurality of instructions for making a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0317] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part.

[0318] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0319] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and above-mentioned drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so construed can be interchanged, such that the embodiments described herein can be carried out in other than the order discussed herein without departing from the scope of the application. Further, the terms "comprise" and "include" and variations thereof as used in enlisting the features, steps or elements of the processes, methods, systems, products or apparatus of the present application are intended to be open-ended, and to permit the inclusion of additional features, steps or elements not expressly mentioned, but which are inherent to such processes, methods, systems, products or apparatus.

[0320] Finally, it should be noted that the above only is a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A data processing method, characterized in that, An application is made in the algebraic modeling system AMS, which is used to transform user-provided data based on the algebraic modeling language AML and user input requirements into AMS for solving, including: Acquire business data for defining business characteristics. The business data is input by the user and includes constant data. The business data is used to solve an objective function. The objective function indicates the solution objective of the business problem and includes decision variables related to the business characteristics. Obtain at least one constraint term, the constraint term including the constraint relationship between the decision variable and the constant data; The constant data is divided into multiple data blocks, each data block corresponding to at least one sub-constraint item. The at least one sub-constraint item is obtained by dividing the at least one constraint item, and the at least one sub-constraint item includes the constraint relationship between the divided constant data and the decision variable. The multiple data blocks are allocated to multiple computing nodes, so that the multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and the sub-constraints corresponding to each data block. The multiple constraint blocks form a constraint matrix. The objective function and the constraint matrix form a planning model for the business problem. Each computing node is a server, a processor, a process, or a thread. When the multiple computing nodes are multiple servers, the multiple data blocks are allocated to each server so that the amount of data processed by each server matches the load. When the multiple computing nodes are multiple processes or multiple threads, the software carrying the AMS function calls the API to allocate the multiple data blocks to the multiple processes or multiple threads. The values ​​of the decision variables are determined based on the constraint matrix to achieve the goal of solving the business problem.

2. The method according to claim 1, characterized in that, The multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and the sub-constraint terms corresponding to each data block, including: The first data block is allocated to the first compute node, causing the first compute node to perform the following steps: The first data block is fused with the target sub-constraint term to obtain a sub-constraint expression. The first data block is any one of the plurality of data blocks, the first computing node is one of the plurality of computing nodes, and the target sub-constraint term is a sub-constraint term that corresponds to the first data block among the plurality of sub-constraint terms. Filter out the non-trivial terms in the sub-constraint expressions, where the non-trivial terms include at least one of the decision variables; The non-trivial terms are parsed to obtain the corresponding constraint blocks.

3. The method according to claim 1 or 2, characterized in that, The step of determining the values ​​of the decision variables based on the constraint matrix to achieve the solution objective of the business problem includes: The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems; The values ​​of the decision variables are obtained by solving the multiple sub-problems respectively, and the values ​​of the decision variables constitute the objective of solving the business problem.

4. The method according to claim 3, characterized in that, Solving the plurality of sub-problems respectively includes: The constraint matrix is ​​aggregated, and the aggregated matrix is ​​solved to obtain the guided solution; The guided solution is used as a constraint to solve the plurality of subproblems, so as to obtain the value of at least one of the decision variables included in the plurality of subproblems.

5. The method according to claim 4, characterized in that, The process of solving the plurality of subproblems using the guided solution as a constraint includes: Using the guiding solution as a constraint, the plurality of sub-problems are solved in a preset order to obtain the value of at least one of the decision variables included in the plurality of sub-problems. Among them, at least one sub-problem arranged in the preset order after the first sub-problem is aggregated to obtain an additional sub-problem. The first sub-problem is any one of the plurality of sub-problems. The first sub-problem and the additional sub-problem are merged to obtain a fused sub-problem. The fused sub-problem is solved to obtain the value of at least one of the decision variables included in one of the plurality of sub-problems.

6. The method according to any one of claims 3-5, characterized in that, The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems, including: Obtain user input data; The decomposition cycle is determined based on the user input data; The constraint matrix is ​​decomposed according to the decomposition period to obtain the plurality of subproblems.

7. A method for generating a production plan, characterized in that, An application is made in the algebraic modeling system AMS, which is used to transform user-provided data based on the algebraic modeling language AML and user input requirements into AMS for solving, including: Obtain business data for defining business characteristics. The business data is input by the user and includes constant data. The business data is used to solve an objective function. The objective function indicates the goal of solving the business problem. The goal indicates the solution of the production plan for the manufactured products. The constant data includes the production target of the products to be produced and parameters related to the production of the products. The production target includes the quantity of the products to be produced. The objective function includes decision variables related to the business characteristics. Obtain at least one constraint term, the constraint term including the constraint relationship between the decision variable and the constant data; The constant data is divided into multiple data blocks, each data block corresponding to at least one sub-constraint item. The at least one sub-constraint item is obtained by dividing the at least one constraint item, and the at least one sub-constraint item includes the constraint relationship between the divided constant data and the decision variable. The multiple data blocks are allocated to multiple computing nodes, so that the multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and the sub-constraints corresponding to each data block. The multiple constraint blocks form a constraint matrix. The objective function and the constraint matrix form a planning model for the business problem. Each computing node is a server, a processor, a process, or a thread. When the multiple computing nodes are multiple servers, the multiple data blocks are allocated to each server so that the amount of data processed by each server matches the load. When the multiple computing nodes are multiple processes or multiple threads, the software carrying the AMS function calls the API to allocate the multiple data blocks to the multiple processes or multiple threads. The values ​​of the decision variables are determined based on the constraint matrix to achieve the solution objective of the business problem, wherein the solution objective represents the production plan for producing the product.

8. The method according to claim 7, characterized in that, The multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and the sub-constraint terms corresponding to each data block, including: The first computing node merges the first data block with the target sub-constraint term to obtain a sub-constraint expression. The first data block is any one of the plurality of data blocks, the first computing node is one of the plurality of computing nodes, and the target sub-constraint term is a sub-constraint term that corresponds to the first data block among the plurality of sub-constraint terms. Filter out the non-trivial terms in the sub-constraint expressions, wherein the non-trivial terms include at least one of the decision variables; The non-trivial terms are parsed to obtain the corresponding constraint blocks.

9. The method according to claim 7 or 8, characterized in that, The step of determining the values ​​of the decision variables based on the constraint matrix to achieve the goal of solving the business problem includes: The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems. The business problem indicates the solution of the production plan within a preset time period along the time dimension. Each of the multiple sub-problems indicates the solution of the production plan for producing the product within the corresponding time period. The values ​​of the decision variables are obtained by solving the multiple sub-problems respectively, and the values ​​of the decision variables constitute the objective of solving the business problem.

10. The method according to claim 9, characterized in that, Solving the plurality of sub-problems respectively includes: The constraint matrix is ​​aggregated, and the aggregated matrix is ​​solved to obtain the guided solution; The guided solution is used as a constraint to solve the plurality of subproblems, so as to obtain the values ​​of the ratio decision variables in the plurality of subproblems.

11. The method according to claim 10, characterized in that, The process of solving the plurality of subproblems using the guided solution as a constraint includes: Using the guiding solution as a constraint, the plurality of subproblems are solved in a preset order to obtain the value of at least one of the decision variables included in the plurality of subproblems. Among them, at least one subproblem arranged in the preset order after the first subproblem is aggregated to obtain an additional subproblem. The first subproblem is any one of the plurality of subproblems. The first subproblem and the additional subproblem are merged to obtain a fused subproblem. The fused subproblem is solved to obtain the value of at least one of the decision variables included in one of the plurality of subproblems.

12. The method according to any one of claims 9-11, characterized in that, The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems, including: Obtain user input data; The decomposition cycle is determined based on the user input data; The constraint matrix is ​​decomposed according to the decomposition period to obtain the plurality of subproblems.

13. A data processing apparatus, characterized in that, An application is made in the algebraic modeling system AMS, which is used to transform user-provided data based on the algebraic modeling language AML and user input requirements into AMS for solving, including: The transceiver module is used to acquire business data for defining business characteristics. The business data is input by the user and includes constant data. The business data is used to solve an objective function. The objective function indicates the solution objective of the business problem and includes decision variables related to the business characteristics. Obtain at least one constraint term, the at least one constraint term including the constraint relationship between the decision variables and the constant data in the business problem; The segmentation module is used to segment the constant data to obtain multiple data blocks, each data block corresponding to at least one sub-constraint item, the at least one sub-constraint item being obtained by segmenting the at least one constraint item, and the at least one sub-constraint item including the constraint relationship between the segmented constant data and the decision variable; The allocation module is used to allocate the multiple data blocks to multiple computing nodes, so that the multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and the sub-constraints corresponding to each data block. The multiple constraint blocks form a constraint matrix, and the objective function and the constraint matrix form a planning model for the business problem. Each computing node is a server, a processor, a process, or a thread. When the multiple computing nodes are multiple servers, the multiple data blocks are allocated to each server to match the amount of data processed and the load of each server. When the multiple computing nodes are multiple processes or multiple threads, the software carrying the AMS function calls the API to allocate the multiple data blocks to the multiple processes or multiple threads. The solution module is used to determine the values ​​of the decision variables based on the constraint matrix in order to achieve the solution objective of the business problem.

14. The apparatus according to claim 13, characterized in that, The allocation module is specifically used to allocate the first data block to the first computing node, so that the first computing node performs the following steps: The first data block is fused with the target sub-constraint term to obtain a sub-constraint expression. The first data block is any one of the plurality of data blocks, and the target sub-constraint term is a sub-constraint term that corresponds to the first data block among the plurality of sub-constraint terms. Filter out the non-trivial terms in the sub-constraint expressions, wherein the non-trivial terms include at least one of the decision variables; The non-trivial terms are parsed to obtain the corresponding constraint blocks.

15. The apparatus according to claim 13 or 14, characterized in that, The solution module is specifically used for: The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems; The values ​​of the decision variables are obtained by solving the multiple sub-problems respectively, and the values ​​of the decision variables constitute the objective of solving the business problem.

16. The apparatus according to claim 15, characterized in that, The solution module is specifically used for: The constraint matrix is ​​aggregated, and the aggregated matrix is ​​solved to obtain the guided solution; The guided solution is used as a constraint to solve the plurality of subproblems, so as to obtain the value of at least one of the decision variables included in the plurality of subproblems.

17. The apparatus according to claim 16, characterized in that, The solution module is specifically used to solve the plurality of subproblems in a preset order, using the guiding solution as a constraint, to obtain the value of at least one of the decision variables included in the plurality of subproblems. Specifically, at least one subproblem arranged in the preset order after the first subproblem is aggregated to obtain an additional subproblem. The first subproblem is any one of the plurality of subproblems. The first subproblem and the additional subproblem are merged to obtain a fused subproblem. The fused subproblem is solved to obtain the value of at least one of the decision variables included in one of the plurality of subproblems.

18. The apparatus according to any one of claims 15-17, characterized in that, The solution module is specifically used for: Obtain user input data; The decomposition cycle is determined based on the user input data; The constraint matrix is ​​decomposed according to the decomposition period to obtain the plurality of subproblems.

19. A production plan generation device, characterized in that, An application is made in the algebraic modeling system AMS, which is used to transform user-provided data based on the algebraic modeling language AML and user input requirements into AMS for solving, including: The transceiver module is used to acquire business data for defining business characteristics. The business data is input by the user and includes constant data. The business data is used to solve an objective function. The objective function indicates the goal of solving the business problem. The goal indicates the solution of the production plan for the manufactured products. The constant data includes the production target of the products to be produced and parameters related to the production of the products. The production target includes the quantity of the products to be produced. The objective function includes decision variables related to the business characteristics of the business problem. The transceiver module is further configured to acquire at least one constraint term, the constraint term including the constraint relationship between the decision variable and the constant data; The segmentation module is used to segment the constant data to obtain multiple data blocks, each data block corresponding to at least one sub-constraint item, the at least one sub-constraint item being obtained by segmenting the at least one constraint item, and the at least one sub-constraint item including the constraint relationship between the segmented constant data and the decision variable; The allocation module is used to allocate the multiple data blocks to multiple computing nodes, so that the multiple computing nodes generate multiple constraint blocks in parallel based on the multiple data blocks and corresponding sub-constraint items. The multiple constraint blocks form a constraint matrix, and the objective function and the constraint matrix form a planning model for the business problem. Each computing node is a server, a processor, a process, or a thread. When the multiple computing nodes are multiple servers, the multiple data blocks are allocated to each server so that the amount of data processed by each server matches the load. When the multiple computing nodes are multiple processes or multiple threads, the software carrying the AMS function calls the API to allocate the multiple data blocks to the multiple processes or multiple threads. The solution module is used to determine the values ​​of the decision variables based on the constraint matrix to achieve the solution objective of the business problem, wherein the solution objective represents the production plan for producing the product.

20. The apparatus according to claim 19, characterized in that, The allocation module is specifically used to allocate the first data block to the first computing node so that the first computing node performs the following steps: merging the first data block with the target sub-constraint item to obtain a sub-constraint expression, wherein the first data block is any one of the plurality of data blocks, the target sub-constraint item is a sub-constraint item corresponding to the first data block among the plurality of sub-constraint items, and the first computing node is one of the plurality of computing nodes; Filter out the non-trivial terms in the sub-constraint expressions, wherein the non-trivial terms include at least one of the decision variables; The nontrivial terms are analyzed to obtain the corresponding constraint blocks.

21. The apparatus according to claim 19 or 20, characterized in that, The solution module is specifically used for: The constraint matrix is ​​divided along the time dimension to obtain multiple sub-problems. The business problem indicates the solution of the production plan within a preset time period along the time dimension. Each of the multiple sub-problems indicates the solution of the production plan for producing the product within the corresponding time period. The values ​​of the decision variables are obtained by solving the multiple sub-problems respectively, and the values ​​of the decision variables constitute the objective of solving the business problem.

22. The apparatus according to claim 21, characterized in that, The solution module is specifically used for: Obtain user input data; The decomposition cycle is determined based on the user input data; The constraint matrix is ​​decomposed according to the decomposition period to obtain the plurality of subproblems.

23. A data processing apparatus, characterized in that, The method includes one or more processors coupled to a memory storing a program that, when executed by the one or more processors, implements the method of any one of claims 1 to 6.

24. A production plan generation device, characterized in that, The method includes one or more processors coupled to a memory storing a program that, when executed by the one or more processors, implements the method of any one of claims 7 to 12.

25. A computer-readable storage medium, characterized in that, The program, when executed by the processing unit, performs the method as described in any one of claims 1 to 12.

26. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 12.

Images