Automatic parallel strategy search method based on polyhedral model modeling and related equipment

Abstract

The application discloses an automatic parallel strategy search method based on a polyhedral model and related equipment, and the method comprises the following steps: according to the intelligence center resources of the intelligence network of each two clients, taking the optimal parallel strategy as a constraint condition, respectively distributing the coding and decoding architecture models on the two clients; controlling the two clients to utilize all the distributed coding and decoding architecture models to cooperatively train and aggregate the intelligence center resources, and calculating the gradient value of the polyhedral model. According to the intelligence center resources of the intelligence network of each two clients, the respective coding and decoding architecture models are differentially distributed, and then the distributed coding and decoding architecture models are utilized to cooperatively train and aggregate the intelligence center resources, so as to calculate the gradient value, thereby realizing the distribution according to different computing power distribution, data distribution and different business scenarios, meeting various business scenarios, and realizing the large-scale expansion of large model training in the intelligence network.

Description

Technical Field

[0001] This invention relates to the field of intelligent computing network technology, and in particular to an automatic parallel strategy search method and related equipment based on polyhedral model modeling. Background Technology

[0002] Since 2017, cities such as Shenzhen, Wuhan, Zhuhai, and Xi'an have built and put into operation intelligent computing centers primarily using domestically produced chips. Cities like Chengdu, Nanjing, and Xuchang are currently under construction, while intelligent computing centers in Beijing, Shanghai, Hangzhou, Guangzhou, Dalian, Qingdao, Taiyuan, and Nanning are being planned. These intelligent computing centers mainly utilize domestically produced CPUs and AI accelerators. Currently, my country has established an independent and controllable cloud computing and big data technology system, standards, and solutions.

[0003] However, when performing collaborative training based on intelligent computing networks, the commonly used PS architecture generates a large amount of communication data during model parameter fusion, which can cause bottlenecks. Furthermore, training may fail when a single device cannot handle the model parameters. Therefore, how to perform efficient collaborative computing based on the environment provided by intelligent computing networks is an urgent problem to be solved.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The main objective of this invention is to provide an automatic parallel strategy search method and related equipment based on polyhedral model modeling, aiming to solve the problem of low efficiency in intelligent computing networks for collaborative computing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] An automatic parallel strategy search method based on polyhedral model modeling, the automatic parallel strategy search method based on polyhedral model model modeling includes:

[0008] Based on the intelligent computing center resources of each pair of clients' intelligent computing networks, and using the optimal parallel strategy as a constraint, the encoding and decoding architecture models on the two clients are allocated respectively.

[0009] The two clients are controlled to use all allocated encoding and decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, and calculate the gradient value of the polyhedral model.

[0010] In the automatic parallel strategy search method based on polyhedral model modeling, the allocation of encoding / decoding architecture models on the two clients respectively, based on the intelligent computing center resources of the intelligent computing network for each pair of clients and using the optimal parallel strategy as a constraint, specifically includes:

[0011] When one intelligent computing network has high computing power but no local data, while another intelligent computing network has low computing power but has the first type of local data, the encoding and decoding architecture model is assigned to the intelligent computing network with high computing power.

[0012] When one intelligent computing network has high computing power and possesses the second type of local data, while another intelligent computing network has low computing power and possesses the first type of local data, the encoding and decoding architecture model is assigned to the intelligent computing network with high computing power.

[0013] When the computing power of the intelligent computing network and the type of local data of the two clients are the same, and the distribution of the local data of the two clients is different, then the encoding and decoding architecture model will not be allocated.

[0014] When the computing power of the intelligent computing networks of two clients is the same, and the distribution of the local data of the two clients is the same, then the encoding and decoding architecture model will not be allocated.

[0015] When the computing power of the intelligent computing networks of the two clients, the type of the local data, and the distribution of the local data of the two clients are all the same, the encoding and decoding architecture model is evenly distributed to the two intelligent computing networks.

[0016] In the automatic parallel strategy search method based on polyhedral model modeling, controlling the two clients to use all allocated encoding / decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, and calculating the gradient value of the polyhedral model, specifically includes:

[0017] The first client controls the first client to perform the first forward computation operation on the pre-stored local data using the allocated encoding and decoding architecture model, and after obtaining the first forward output result, it sends it to the second client.

[0018] The second client is controlled to perform aggregation and a second forward calculation operation on the first forward output result using the allocated encoding and decoding architecture model, and then send the second forward output result to the first client.

[0019] The first client is controlled to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and the obtained first gradient value is sent to the second client.

[0020] After controlling the second client to perform aggregation and second backward gradient calculation operations on the first gradient value, the obtained second gradient value is sent to the first client.

[0021] The first client is controlled to perform a third inverse gradient calculation on the second gradient value to obtain the third gradient value.

[0022] In the automatic parallel strategy search method based on polyhedral model modeling, the control of the first client utilizes the allocated encoding / decoding architecture model to perform the first forward computation operation on the pre-stored local data, obtains the first forward output result, and then sends it to the second client, specifically including:

[0023] The first client is controlled to perform the first forward computation operation on the local data using the embedded layer in the allocated codec architecture model, to obtain the first forward output result, and then send the first forward output result to the second client.

[0024] In the automatic parallel strategy search method based on polyhedral model modeling, controlling the second client to perform aggregation and a second forward calculation operation on the first forward output result using the allocated encoding / decoding architecture model, and then sending the second forward output result to the first client, specifically includes:

[0025] The second client is controlled to perform an aggregation operation on the first forward output result to obtain the first aggregation result;

[0026] The second client is controlled to use the multi-head attention mechanism layer in the allocated codec architecture model to perform the second forward computation operation on the first aggregation result, so as to obtain the second forward output result;

[0027] The second client is controlled to send the second forward output result to the first client.

[0028] In the automatic parallel strategy search method based on polyhedral model modeling, controlling the first client to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and sending the obtained first gradient value to the second client, specifically includes:

[0029] The first client is controlled to use the head layer of the allocated codec architecture model to perform the third forward calculation operation on the second forward output result to obtain the third forward output result;

[0030] The first client is controlled to perform the first backward gradient calculation operation on the third forward output result, and after obtaining the first gradient value, it is sent to the second client.

[0031] In the automatic parallel strategy search method based on polyhedral model modeling, the step of controlling the second client to perform aggregation and a second backward gradient calculation operation on the first gradient value, and then sending the obtained second gradient value to the first client, specifically includes:

[0032] The second client is controlled to perform the aggregation operation on the first gradient value to obtain the second aggregation result;

[0033] The second client is controlled to use the encoding and decoding layer in the allocated encoding and decoding architecture model to perform the second reverse calculation operation on the second aggregation result, and after obtaining the second gradient value, it is sent to the first client.

[0034] In the automatic parallel strategy search method based on polyhedral model modeling, controlling the first client to perform a third back gradient calculation operation on the second gradient value to obtain the third gradient value specifically includes:

[0035] The first client is controlled to use the embedded layer in the allocated codec architecture model to perform the third inverse gradient calculation operation on the second gradient value to obtain the third gradient value.

[0036] In the automatic parallel strategy search method based on polyhedral model modeling, the intelligent computing center resources include: the computing power of the intelligent computing network and the local data; the first type of local data is domain data; the second type of local data is general data.

[0037] In the automatic parallel strategy search method based on polyhedral model modeling, the first forward output result is the embedded feature value; the second forward output result is the hidden layer information; and the third forward output result is the probability of the output character.

[0038] An automatic parallel strategy search system based on polyhedral model modeling, the automatic parallel strategy search system based on polyhedral model model modeling includes:

[0039] The model allocation module is used to allocate the encoding and decoding architecture model on each of the two clients based on the intelligent computing center resources of the intelligent computing network and with the optimal parallel strategy as a constraint.

[0040] The gradient value calculation module is used to control the two clients to perform collaborative training and aggregation operations on the intelligent computing center resources using all allocated encoding and decoding architecture models, and to calculate the gradient value of the polyhedron model.

[0041] A controller includes: a memory, a processor, a display, and an automatic parallel strategy search program based on polyhedral modeling, stored in the memory and executable on the processor. When executed by the processor, the automatic parallel strategy search program based on polyhedral modeling implements the steps of the automatic parallel strategy search method based on polyhedral modeling as described above.

[0042] A computer-readable storage medium stores an automatic parallel strategy search program based on polyhedral model modeling, wherein when executed by a processor, the automatic parallel strategy search program based on polyhedral model modeling implements the steps of the automatic parallel strategy search method based on polyhedral model modeling as described above.

[0043] Compared to existing technologies, this invention provides an automatic parallel strategy search method and related equipment based on polyhedral model modeling. The method includes: allocating encoding / decoding architecture models on two clients respectively, using the optimal parallel strategy as a constraint, based on the intelligent computing center resources of each pair of clients' intelligent computing networks; controlling the two clients to perform collaborative training and aggregation operations on the intelligent computing center resources using all allocated encoding / decoding architecture models to calculate the gradient value of the polyhedral model. By first allocating different encoding / decoding architecture models to each pair of clients based on the intelligent computing center resources of each pair of clients' intelligent computing networks, and then using the allocated encoding / decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources to calculate the gradient value, this method achieves allocation based on different computing power distributions, data distributions, and different business scenarios, thus satisfying various business scenarios and enabling large-scale expansion of large model training in intelligent computing networks. Attached Figure Description

[0044] Figure 1 A flowchart illustrating a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention;

[0045] Figure 2 The timing diagram of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention when the encoding and decoding architecture model is not allocated;

[0046] Figure 3 The flowchart of step S100 is provided in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0047] Figure 4 The distribution graph of the encoding / decoding architecture model in a scene is shown in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0048] Figure 5 The distribution diagram of the encoding / decoding architecture model in scenario two is shown in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0049] Figure 6 The distribution diagram of the encoding / decoding architecture model in scenario three is shown in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0050] Figure 7 The distribution diagram of the encoding / decoding architecture model in scenario five is shown in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0051] Figure 8 The flowchart of step S200 is provided in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0052] Figure 9 A schematic diagram illustrating the training process between the server and N clients in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0053] Figure 10 The flowchart of step S220 is provided in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0054] Figure 11 The flowchart of step S230 is provided in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0055] Figure 12 The flowchart of step S240 is provided in a preferred embodiment of the automatic parallel strategy search method based on polyhedral model modeling provided by the present invention.

[0056] Figure 13 A schematic diagram of the principle of the automatic parallel strategy search system based on polyhedral model modeling provided by the present invention;

[0057] Figure 14 This is a schematic diagram illustrating the different allocation of the transformer architecture model between two clients for different scenarios in a preferred embodiment of the automatic parallel strategy search system based on polyhedral model modeling provided by the present invention.

[0058] Figure 15 This is a schematic diagram of the operating environment of a preferred embodiment of the controller provided by the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0061] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0062] To facilitate understanding of the embodiments of this application, the design intentions involved in the embodiments of the present invention will be introduced first.

[0063] When performing collaborative training based on intelligent computing networks, the commonly used PS architecture generates a large amount of communication data during model parameter fusion, which can cause bottlenecks. Furthermore, training may fail when a single device cannot accommodate the model parameters.

[0064] To address the aforementioned problems in the prior art, this invention provides an automatic parallel strategy search method and related equipment based on polyhedral model modeling. In this invention, the intelligent computing center resources of each pair of clients are allocated according to the optimal parallel strategy as a constraint. The encoding / decoding architecture models on each of the two clients are then allocated. The two clients are then controlled to utilize all allocated encoding / decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, calculating the gradient value of the polyhedral model. This not only enables different allocation methods based on different computing power distributions, data distributions, and business scenario requirements, but also effectively improves the collaborative training performance of the intelligent computing network under various business scenarios.

[0065] This invention employs a cross-domain pipeline parallelism to allocate model resources and utilizes different fusion strategies. The ultimate goal is to unify the input and output representations, converting all information into tokens (characters), which are then transformed into feature vectors for representation. During training, text information from different tasks is replaced with token sequences to form batched input. This batch input then enters the backbone network (multi-head attention mechanism) to obtain output values. Finally, the output values ​​and label values ​​are used to calculate the loss. Furthermore, to address the challenge of distinguishing between multiple tasks, many template prompts can be pre-written. The input text is then rewritten according to these prompts, significantly standardizing the input space and making the differences between different tasks more apparent, thus exhibiting a certain degree of transferability.

[0066] Furthermore, this invention supports three fusion strategies: First, each client encodes and infers the original data using its current embedding and head layers, sharing information through the embedding and head layers to enhance the model. Second, the embedding layers of each client are fused on an average basis. Third, the embedding and head layers of each client are fused on an average basis.

[0067] The following describes the design scheme of the automatic parallel strategy search method based on polyhedral model modeling through specific exemplary embodiments. It should be noted that the following embodiments are only used to explain the technical solution of the invention and are not intended to limit it specifically:

[0068] Please see Figure 1 This invention provides an automatic parallel strategy search method based on polyhedral model modeling, the automatic parallel strategy search method based on polyhedral model model modeling includes:

[0069] S100. Based on the intelligent computing center resources of the intelligent computing network for each pair of clients, and using the optimal parallel strategy as a constraint, allocate the encoding / decoding architecture model on each of the two clients respectively. The intelligent computing center resources include: the computing power of the intelligent computing network and the local data.

[0070] Specifically, existing collaborative computing based on intelligent computing networks relies on a central server to control the training process and accept model parameters from all clients, without allocating encoding / decoding architecture models to the clients. Figure 2 The diagram shown is the timing diagram of the entire model without allocation, which can easily lead to single points of failure or congestion.

[0071] Therefore, this embodiment proposes a cross-domain pipeline parallel allocation method for models between two clients: first, the respective codec architecture models on the clients are allocated according to the situation, mainly by dividing the multi-head attention mechanism layer (Backbone) in the codec architecture model's transformer layer to different clients in different scenarios, specifically:

[0072] First, based on the computing center resources of the intelligent computing network for each pair of clients (the computing power of the intelligent computing network and the local data, i.e., computing, storage, communication resources, etc.), the optimal parallel strategy (including hybrid parallelism such as data parallelism, model parallelism, and pipeline parallelism) is used as a constraint. At the same time, the allocation of layers does not exceed the device memory limit, and the workload avoids computing bottlenecks as constraints. Different allocation methods are used for the respective encoding and decoding architecture models (mainly the multi-head attention mechanism layer (Backbone), and also the data loading layer, embedding layer (used to map sentences to a representation vector), query layer (used to predict the position of the next Query Q) and head layer) of each client.

[0073] This invention proposes a method for allocating computing power, data, and different business scenarios to meet the needs of various business scenarios under intelligent computing network collaborative computing. By coordinating the computing power of the intelligent computing network, it enables the large-scale expansion of large model training in the intelligent computing network, effectively improving the performance of the intelligent computing network collaborative computing.

[0074] Furthermore, please refer to Figure 3 S100, based on the intelligent computing center resources of the intelligent computing network for each pair of clients, and using the optimal parallel strategy as a constraint, allocates the encoding / decoding architecture model on each of the two clients respectively, specifically including:

[0075] S110. When one intelligent computing network has high computing power and no local data, while another intelligent computing network has low computing power and has the first type of local data, the encoding and decoding architecture model is allocated to the intelligent computing network with high computing power.

[0076] S120. When one intelligent computing network has high computing power and possesses the second type of local data, while another intelligent computing network has low computing power and possesses the first type of local data, the encoding and decoding architecture model is allocated to the intelligent computing network with high computing power.

[0077] S130. When the computing power of the intelligent computing network of two clients and the type of the local data are the same, and the distribution of the local data of the two clients is different, then the encoding and decoding architecture model is not allocated.

[0078] S140. When the computing power of the intelligent computing networks of two clients is the same, and the distribution of the local data of the two clients is the same, then the encoding and decoding architecture model is not allocated.

[0079] S150. When the computing power of the intelligent computing networks of the two clients, the type of the local data, and the distribution of the local data of the two clients are all the same, the encoding and decoding architecture model is evenly distributed to the two intelligent computing networks.

[0080] The first type of local data is domain data; the second type of local data is general data.

[0081] Specifically, when assigning different codec architecture models, the allocation method varies depending on the scenario objective. Common scenarios suitable for collaborative training are shown in Table 1:

[0082]

[0083] Table 1

[0084] like Figure 4 As shown, in Scenario 1, when one intelligent computing network has high computing power but no local data, while another intelligent computing network has low computing power but possesses the first type of local data (domain data), the encoding / decoding architecture model is assigned to the intelligent computing network with high computing power. That is, when the intelligent computing network of the first client B1 has low computing power but possesses domain data, while the intelligent computing network of the second client B2 has high computing power but no local data, the multi-head attention mechanism layer (Backbone) is assigned to the intelligent computing network of the first client B1.

[0085] Because the backbone portion accounts for over 70% of the total computation time, while the remaining Embedding ( Figure 4 In this scenario, the computational share of the Embed, Query, and Head layers is less than 30%. Meanwhile, Participant 1 (first client B1) possesses domain data but has limited computing power, while Participant 2 (second client B2) possesses general-purpose data and has significant computing power. Therefore, in this scenario, the allocation is based on placing the Backbone layer on the side with greater cluster computing power (Participant 2), and both parties' data can remain entirely local. Thus, through the support of the high-computing-power side of the intelligent computing network, the performance of the domain data owner's large model capabilities in domain tasks is enhanced.

[0086] like Figure 5As shown, in Scenario 2, when one intelligent computing network has high computing power and possesses the second type of local data (general data), while another intelligent computing network has low computing power and possesses the first type of local data, the encoding / decoding architecture model is assigned to the intelligent computing network with high computing power. That is, when the intelligent computing network of the first client B1 has low computing power and possesses the domain data, while the intelligent computing network of the second client B2 has high computing power and possesses general data, the multi-head attention mechanism layer (Backbone) is assigned to the intelligent computing network of the second client B2.

[0087] Similarly, because the Backbone layer accounts for over 70% of the total computation time, while the remaining embedding, query, and head layers account for less than 30%, and participant 1 possesses domain data but has limited computing power, while participant 2 possesses general-purpose data but has significant computing power, the allocation in scenario two is based on placing the Backbone layer on the side with greater cluster computing power (participant 2), while the data for both parties can remain entirely local. Thus, with the support of the high-computing-power side of the intelligent computing network, a higher-performance large model can be obtained by fusing domain data and general-purpose data.

[0088] like Figure 6 As shown, in scenario three, when the computing power of the intelligent computing networks of two clients and the type of local data are the same, but the distribution of the local data is different, the encoding / decoding architecture model is not allocated. That is, when the computing power of the intelligent computing networks of the first client B1 and the second client B2 is the same, and the type of local data they possess is also the same (i.e., both are general data or domain data), but the distribution of their respective local data is different, meaning the computing power of the two parties is balanced, and both parties possess the same type of data but with different data distributions, cross-center data parallelism is adopted, and model parameters are periodically synchronized. The encoding / decoding architecture model is not allocated, thus maintaining... Figure 2 The model does not assign a method.

[0089] Therefore, by leveraging the collaborative computing power of intelligent computing networks, large-scale model training can be scaled up within these networks. Figure 6 In this context, the training of an entire layer of the model through several iterations is called a "step." Therefore, each iteration of the complete model (which the user can set) constitutes a "step." Figure 2 One of the "Train steps" is called Fusion.

[0090] Similarly, Figure 6As shown, in scenario four, when the computing power of the intelligent computing networks of the two clients is the same, and the distribution of their local data is the same, the encoding / decoding architecture model is not allocated. That is, when the computing power of the intelligent computing networks of the first client B1 and the second client B2 is the same, and the distribution of their respective local data is also the same (i.e., the computing power is balanced, both parties have data and it is distributed similarly), cross-center data parallelism is used, and model parameters are periodically synchronized; the encoding / decoding architecture model is not allocated. Therefore, through the collaboration of intelligent computing network computing power, large-scale model training can be achieved within the intelligent computing network.

[0091] like Figure 7 As shown, in Scenario 5, when the computing power of the intelligent computing networks of the two clients, the type of the local data, and the distribution of the local data of the two clients are all the same, the encoding / decoding architecture model is evenly distributed among the two intelligent computing networks. That is, when the computing power of the intelligent computing networks of the first client B1 and the second client B2 is the same, and the type of local data they each possess is also the same, but the distribution of the local data they each possess is different, that is, when the computing power of the two parties is balanced, both parties possess the same type of data, and the data distribution is the same, the multi-head attention mechanism layer (Backbone) is evenly distributed among the intelligent computing networks of the first client B1 and the second client B2.

[0092] For further information, please refer to [link / reference]. Figure 1 S200 controls the two clients to use all allocated encoding and decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, and calculate the gradient value of the polyhedron model.

[0093] Specifically, after allocating the encoding / decoding architecture model for the client, the following describes the collaborative training and aggregation process between the two clients as the execution entities:

[0094] The two clients utilize all allocated codec architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources in order to calculate the gradient value of the polyhedral model. The polyhedral model is composed of all allocated codec architecture models.

[0095] By using the two clients to perform efficient collaborative training with the allocated codec architecture model, the gradient values ​​can be calculated quickly, which not only effectively reduces training time but also avoids congestion problems.

[0096] Furthermore, please refer to Figure 8S200 controls the two clients to use all allocated codec architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, and calculate the gradient value of the polyhedral model, specifically including:

[0097] S210. Control the first client B1 to use the allocated encoding and decoding architecture model to perform the first forward calculation operation on the pre-stored local data, and after obtaining the first forward output result, send it to the second client B2.

[0098] S220. Control the second client B2 to use the allocated encoding and decoding architecture model to perform aggregation operation and second forward calculation operation on the first forward output result, and after obtaining the second forward output result, send it to the first client B1.

[0099] S230. Control the first client B1 to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result in sequence, and send the obtained first gradient value to the second client B2.

[0100] S240. Control the second client B2 to perform aggregation operation and second reverse gradient calculation operation on the first gradient value in sequence, and then send the obtained second gradient value to the first client B1.

[0101] S250: Control the first client B1 to perform a third inverse gradient calculation operation on the second gradient value to obtain the third gradient value.

[0102] The first forward output is the embedded feature value; the second forward output is the hidden layer information.

[0103] For details, please continue reading Figure 6 ,for Figure 6 Each "step" in the process has two scenarios: When the embedding layer and the head layer in the encoding / decoding architecture model are evenly distributed (i.e., when the computing power of the intelligent computing networks of the two clients and the type of local data are the same, but the distribution of the local data is different), and when the computing power of the intelligent computing networks of the two clients is the same and the distribution of their respective local data is the same, the collaborative training process between the two intelligent computing networks is as follows:

[0104] First, the first client B1 uses the embedding layer (Emd / Embedding) in the allocated codec architecture model to perform a first forward computation operation on the pre-stored local data (characters in NLP (Natural Language Processing, an important direction in computer science and artificial intelligence)) to obtain a first forward output result (embedded feature value), which is then sent to the second client B2. The second client B2 then uses the multi-head attention mechanism layer (Backbone) in the allocated codec architecture model to perform an aggregation operation and a second forward computation operation on the first forward output result to obtain a second forward output result (hidden layer information), which is then sent to the first client B1.

[0105] Then, the first client B1 uses the head layer of the allocated codec architecture model to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and sends the obtained first gradient value to the second client B2. The second client B2 uses the transformer layer of the allocated codec architecture model to perform an aggregation operation and a second backward gradient calculation operation on the first gradient value, and sends the obtained second gradient value to the first client B1. Finally, the first client B1 uses the embedding layer of the allocated codec architecture model to perform the third backward gradient calculation operation on the second gradient value to obtain the third gradient value. The above steps are repeated until a preset number of iterations is reached, thereby completing the collaborative training between the first client B1 and the second client B2.

[0106] Please continue reading. Figure 4 In the second embodiment of this invention, in a scenario where one intelligent computing network has high computing power but no local data, while another intelligent computing network has low computing power but possesses domain data, the collaborative training process between the two intelligent computing networks is as follows:

[0107] First, the first client B1 utilizes the embedded layer in the allocated codec architecture model ( Figure 4 The first Emd / Embed / Embedding on participant 1 performs the first forward computation operation on the domain data, and after obtaining the first forward output result (embedded feature value), it is sent to the second client B2; the second client B2 then utilizes the multi-head attention mechanism layer in the allocated encoding / decoding architecture model ( Figure 4The first Backbone on participant 2 performs a second forward calculation on the first forward output result to obtain the second forward output result (hidden layer information), and then sends it to the first client B1.

[0108] Then, the first client B1 utilizes the query layer in the allocated codec architecture model ( Figure 4 The first query on participant 1) and the head layer ( Figure 4 The first head on participant 1 performs a third forward computation operation on the second forward output result to obtain the third forward output result (the probability of output characters), and utilizes the head layer of the allocated encoding / decoding architecture model ( Figure 4 The second head on participant 1) and the query layer ( Figure 4 The second Query on participant 1 performs the first backward gradient calculation operation on the third forward output result and sends the obtained first gradient value to the second client B2.

[0109] Secondly, the second client B2 utilizes the multi-head attention mechanism layer in the allocated codec architecture model ( Figure 4 After the second Backbone on participant 2 performs a second backward gradient calculation on the first gradient value, it sends the obtained second gradient value to the first client B1. Finally, the first client B1 uses the embedded layer in the allocated codec architecture model ( Figure 4 The second Embed on participant 1 performs a third backward gradient calculation on the second gradient value to obtain the third gradient value. The above steps are repeated until the preset number of iterations is reached, thereby completing the collaborative training between the first client B1 and the second client B2.

[0110] Please continue reading. Figure 5 In the third embodiment of the present invention, the collaborative training process between the two intelligent computing networks, where one network has high computing power and possesses second-type local data (general data), while the other network has low computing power and possesses first-type local data (domain data), is as follows:

[0111] First, the first client B1 utilizes the embedding layer in the allocated codec architecture model ( Figure 5 The first Embed on B1 performs the first forward computation operation on the domain data, obtaining the first forward output result (embedded feature value, i.e.) Figure 5 After being embedded (B1), it is sent to the second client B2; at the same time, the second client B2 utilizes the embedding layer in the allocated codec architecture model ( Figure 5 The first Embed on B2 performs a second forward computation on the general data to obtain a second forward output (embedded feature value). Then, the second client B2 merges the first and second forward outputs and uses the multi-head attention mechanism layer in the allocated encoder-decoder architecture model (…). Figure 5 The first backbone on B2 undergoes a third forward computation operation to obtain the third forward output (hidden layer information, i.e.) Figure 5 The second client B2 then transmits the third forward output result to the first client B1.

[0112] Secondly, the first client B1 and the second client B2 simultaneously utilize the query layer in the allocated encoding / decoding architecture model ( Figure 5 The first query on B1 and B2) and the head layer ( Figure 5 After performing a fourth forward calculation operation on the third forward output result (the first head on B1 and B2 respectively), the fourth forward calculation result is obtained (the probability of output characters, i.e.) Figure 5 (L1+L2). Then, the first client B1 and the second client B2 simultaneously utilize the head layer (L1+L2) of the allocated codec architecture model. Figure 5 The second head on B1 and B2) and the query layer ( Figure 5 The second Query on B1 and B2 respectively performs the first backward gradient calculation operation on the third forward output result to obtain the first gradient value.

[0113] Furthermore, the second client B2 utilizes the multi-head attention mechanism layer in the allocated encoding / decoding architecture model ( Figure 5 The second backpropagation gradient calculation is performed on the second backbone on B2 to obtain the second gradient value. Figure 5 The activation value (B1+B2) is then transmitted to the first client B1. Both the first client B1 and the second client B2 simultaneously utilize the embedded layer in the allocated codec architecture model. Figure 5 After performing a third backpropagation gradient calculation operation on the second gradient value on B2 and the second Embed on B2 respectively, the third gradient value is obtained. Finally, the above steps are repeated until the preset number of iterations is reached, thereby completing the collaborative training between the first client B1 and the second client B2.

[0114] Please continue reading. Figure 7In the fifth embodiment of this invention, when the computing power of the intelligent computing networks of the two clients, the type of the local data, and the distribution of their respective local data are all the same, the collaborative training process between the two intelligent computing networks is as follows:

[0115] First, the first client B1 utilizes the embedded layer in the allocated codec architecture model ( Figure 7 The first Embed on B1) and the multi-head attention mechanism layer ( Figure 7 The first Backbone1 on B1 performs the first forward computation operation on the pre-stored local data, obtaining the first forward output result (embedded feature value, i.e.) Figure 7 In the Feat(B)), the first forward output result and the data index ( Figure 7 The data index is transmitted together to the second client B2.

[0116] Then, the second client B2 utilizes the multi-head attention mechanism layer in the allocated codec architecture model ( Figure 7 The first Backbone2 on B2), query layer ( Figure 7 The first query on B2) and the head layer ( Figure 7 The first head on B2 performs a second forward computation on the first forward output result to obtain the second forward output result (the probability of output characters). Next, the second client B2 utilizes the head layer of the allocated encoding / decoding architecture model (…). Figure 7 The second head on B2), query layer ( Figure 7 The second query on B2) and the multi-head attention mechanism layer ( Figure 7 The second Backbone2 on B2 performs the first backward gradient calculation on the second forward output result to obtain the first gradient value. Figure 7 The Activation(B) in the middle is transmitted to the first client B1.

[0117] Furthermore, the first client B1 utilizes the multi-head attention mechanism layer in the allocated encoding / decoding architecture model ( Figure 7 The second Backbone1 on B1 and the embedding layer ( Figure 7 The second Embed on B1 performs a second back gradient calculation on the second gradient value to obtain the second gradient value. The above steps are repeated until the preset number of iterations is reached, thereby completing the collaborative training between the first client B1 and the second client B2.

[0118] In this invention, the training can be collaborative training between the first client B1 and the second client B2, or collaborative training between the server and N clients (Client1-N). The training process is as follows: Figure 9 As shown.

[0119] Furthermore, in step S210, controlling the first client B1 to perform a first forward computation operation on the pre-stored local data using the allocated encoding / decoding architecture model, and then sending the first forward output result to the second client B2, specifically includes:

[0120] S211. Control the first client B1 to perform the first forward calculation operation on the local data using the embedded layer in the allocated codec architecture model, obtain the first forward output result, and send the first forward output result to the second client B2.

[0121] Specifically, when co-training between the two clients begins, the first client B1 first uses the embedding layer (Emd / Embed / Embedding) in the allocated codec architecture model to perform the first forward computation operation on the local data (characters in NLP) to obtain the first forward output result (embedded feature value).

[0122] Furthermore, please refer to Figure 10 S220, controlling the second client B2 to perform aggregation and a second forward calculation operation on the first forward output result using the allocated encoding / decoding architecture model, and then sending the second forward output result to the first client B1, specifically including:

[0123] S221. Control the second client B2 to perform an aggregation operation on the first forward output result to obtain the first aggregation result;

[0124] S222. Control the second client B2 to use the multi-head attention mechanism layer in the allocated encoding and decoding architecture model to perform the second forward calculation operation on the first aggregation result to obtain the second forward output result;

[0125] S223. Control the second client B2 to send the second forward output result to the first client B1.

[0126] Specifically, after obtaining the first forward output result, the second client B2 first performs an aggregation operation on the first forward output result to obtain a first aggregation result, and then uses the multi-head attention mechanism layer (Backbone) in the allocated codec architecture model to perform the second forward calculation operation on the first aggregation result to obtain the second forward output result (hidden layer information). Then, the second client B2 sends the second forward output result back to the first client B1.

[0127] Furthermore, please refer to Figure 11 S230, controlling the first client B1 to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and sending the obtained first gradient value to the second client B2, specifically including:

[0128] S231. Control the first client B1 to use the head layer of the allocated encoding and decoding architecture model to perform the third forward calculation operation on the second forward output result to obtain the third forward output result.

[0129] S232. Control the first client B1 to perform the first backward gradient calculation operation on the third forward output result, and after obtaining the first gradient value, send it to the second client B2.

[0130] Specifically, after obtaining the second forward output result, the first client B1 uses the head layer of the allocated encoding / decoding architecture model to perform the third forward calculation operation on the second forward output result to obtain the third forward output result (the probability of a character). Then, the first client B1 performs the first backward gradient calculation operation on the third forward output result to obtain the first gradient value, and sends it to the second client B2.

[0131] Furthermore, please refer to Figure 12 S240, controlling the second client B2 to perform aggregation and a second backward gradient calculation operation on the first gradient value, and then sending the obtained second gradient value to the first client B1, specifically including:

[0132] S241. Control the second client B2 to perform the aggregation operation on the first gradient value to obtain the second aggregation result;

[0133] S242. Control the second client B2 to use the encoding and decoding layer in the allocated encoding and decoding architecture model to perform the second reverse calculation operation on the second aggregation result, obtain the second gradient value, and then send it to the first client B1.

[0134] Specifically, after receiving the first gradient value, the second client B2 first performs the aggregation operation on the first gradient value to obtain the second aggregation result. Then, the second client B2 uses the transformer layer in the allocated codec architecture model to perform the second reverse calculation operation on the second aggregation result to obtain the second gradient value. Next, the second client B2 sends the second gradient value to the first client B1.

[0135] Furthermore, in step S250, the first client B1 is controlled to perform a third backward gradient calculation on the second gradient value to obtain the third gradient value, specifically including:

[0136] S251. Control the first client B1 to use the embedded layer in the allocated codec architecture model to perform the third inverse gradient calculation operation on the second gradient value to obtain the third gradient value.

[0137] Specifically, after the first client B1 receives the second gradient value, the first client B1 uses the embedded layer in the allocated codec architecture model to perform the third backward gradient calculation operation on the second gradient value to obtain the third gradient value, thereby completing the entire collaborative training process.

[0138] Further, please refer to Figure 13 The present invention also provides an automatic parallel strategy search system based on polyhedral model modeling, the automatic parallel strategy search system based on polyhedral model model modeling comprising:

[0139] Model allocation module 1 is used to allocate the encoding and decoding architecture model on each of the two clients based on the intelligent computing center resources of the intelligent computing network for each client, with the optimal parallel strategy as a constraint.

[0140] The gradient value calculation module 2 is used to control the two clients to perform collaborative training and aggregation operations on the intelligent computing center resources using all allocated encoding and decoding architecture models, and to calculate the gradient value of the polyhedron model.

[0141] Specifically, the automatic parallel strategy search method based on polyhedral modeling described in this invention is applied to the automatic parallel strategy search system based on polyhedral modeling. The automatic parallel strategy search method based on polyhedral modeling is implemented as follows:

[0142] This invention provides a method for allocating the respective codec architecture models on the client side according to the situation. Specifically, it assigns the multi-head attention mechanism layer (Backbone) within the codec architecture model's transformer layer to different clients in different scenarios.

[0143] First, based on the computing center resources of the intelligent computing network for each pair of clients (the computing power of the intelligent computing network and the local data, i.e., computing, storage, communication resources, etc.), the optimal parallel strategy (including hybrid parallelism such as data parallelism, model parallelism, and pipeline parallelism) is used as a constraint. At the same time, the allocation of layers does not exceed the device memory limit, and the workload avoids computing bottlenecks as constraints. Different allocation methods are used for the respective encoding and decoding architecture models (mainly the multi-head attention mechanism layer (Backbone), and also the data loading layer, embedding layer, query layer, and head layer)).

[0144] The diagram illustrates how the transformer architecture model is allocated differently between the two clients (Client1 and Client2) for different scenarios, as shown below. Figure 14 As shown, Figure 14 The entire model is divided into three parts: Part 1: embedding (Emd) layer; Part 2: backbone layer; Part 3: head layer. During forward propagation, the forward output of the embedding layer is transmitted to the backbone layer in Part 3, and the forward output of the backbone layer is transmitted to the head layer in Part 1 to calculate the logits (loss value). During backward propagation, the gradient information calculated from the logits in Part 1 is transmitted to the backbone layer, which then calculates the gradient and transmits it back to the embedding layer, thus completing the entire backward propagation process. Figure 14 In this code, Token embedding is a vector encoding; Position embedding is a positional encoding; Sentence embedding is the output of the self-attention module; and Gradient is the gradient value.

[0145] Then, the allocated codec architecture model is used to perform collaborative training and aggregation operations on the intelligent computing center resources in order to calculate the gradient value. By using the allocated codec architecture model for efficient collaborative training, the gradient value can be calculated quickly, which not only effectively reduces training time but also avoids congestion problems.

[0146] This invention employs different allocation methods and communication frequency requirements for different computing power distributions, data distributions, and business scenario needs, enabling it to meet various business scenarios. Through the collaboration of computing power in the intelligent computing network, it solves the problem of improving model performance through collaborative training when data does not leave the local area or computing power is insufficient. It realizes the large-scale expansion of large model training in the intelligent computing network, enriches the scenarios of cross-domain collaborative training, and accelerates the development of balanced computing power.

[0147] Furthermore, multiple tasks can be accomplished using a single neural network with equal weights, and for transformer (encoder / decoder) architectures, a larger number of parameters may yield better results. In the implementation, the model is divided into multiple parts, and point-to-point communication modules are implemented for transmission between different parts. This reduces unnecessary transmissions. Parameters requiring remote transmission are marked "remote," while parameters that can be obtained locally are marked "local."

[0148] Furthermore, such as Figure 15 As shown, based on the above-mentioned automatic parallel strategy search method based on polyhedral model modeling, the present invention also provides a controller, which includes a processor 10, a memory 20 (and a display 30). Figure 15 Only a portion of the controller components are shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented instead.

[0149] In some embodiments, the memory 20 may be an internal storage unit of the controller, such as the controller's hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the controller, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the controller. Further, the memory 20 may include both internal and external storage units of the controller. The memory 20 is used to store application software and various types of data installed on the controller, such as the program code installed on the controller. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an automatic parallel strategy search program 40 based on polyhedral model modeling, which can be executed by the processor 10 to implement the automatic parallel strategy search method based on polyhedral model modeling in this application.

[0150] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the automatic parallel strategy search method based on polyhedral model modeling.

[0151] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the controller and to display a visual user interface. The components 10-30 of the controller communicate with each other via a system bus.

[0152] In one embodiment, the steps implemented by the processor 10 when executing the automatic parallel strategy search program 40 based on polyhedral modeling in the memory 20 can be referred to the corresponding description in the above-described automatic parallel strategy search method based on polyhedral modeling, and will not be repeated here.

[0153] Furthermore, the present invention also provides a computer-readable storage medium storing an automatic parallel strategy search program based on polyhedral model modeling. When the automatic parallel strategy search program based on polyhedral model modeling is executed by a processor, it implements the steps of the automatic parallel strategy search method based on polyhedral model modeling as described above. Since the steps of the automatic parallel strategy search method based on polyhedral model modeling have been described in detail above, they will not be repeated here.

[0154] In summary, this invention provides an automatic parallel strategy search method and related equipment based on polyhedral model modeling. The method includes: allocating encoding / decoding architecture models on two clients respectively, using the optimal parallel strategy as a constraint, based on the intelligent computing center resources of each pair of clients' intelligent computing networks; controlling the two clients to perform collaborative training and aggregation operations on the intelligent computing center resources using all allocated encoding / decoding architecture models to calculate the gradient value of the polyhedral model. By first allocating different encoding / decoding architecture models according to the intelligent computing center resources of each pair of clients' intelligent computing networks, and then using the allocated encoding / decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources to calculate the gradient value, this method achieves allocation based on different computing power distributions, data distributions, and different business scenarios, thus satisfying various business scenarios and enabling large-scale expansion of large model training in intelligent computing networks.

[0155] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. An automatic parallel strategy search method based on polyhedral model modeling, characterized in that, The automatic parallel strategy search method based on polyhedral modeling includes: Based on the intelligent computing center resources of each pair of clients' intelligent computing networks, and using the optimal parallel strategy as a constraint, the encoding and decoding architecture models on the two clients are allocated respectively. The two clients are controlled to use all allocated encoding and decoding architecture models to perform collaborative training and aggregation operations on the intelligent computing center resources, and calculate the gradient value of the polyhedron model. The control mechanism, which involves the two clients utilizing all allocated encoding / decoding architecture models, performs collaborative training and aggregation operations on the intelligent computing center resources to calculate the gradient values ​​of the polyhedral model. Specifically, this includes: The first client controls the first client to perform the first forward computation operation on the pre-stored local data using the allocated encoding and decoding architecture model, and after obtaining the first forward output result, it sends it to the second client. The second client is controlled to perform aggregation and a second forward calculation operation on the first forward output result using the allocated encoding and decoding architecture model, and then send the second forward output result to the first client. The first client is controlled to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and the obtained first gradient value is sent to the second client. After controlling the second client to perform aggregation and second backward gradient calculation operations on the first gradient value, the obtained second gradient value is sent to the first client. The first client is controlled to perform a third inverse gradient calculation on the second gradient value to obtain the third gradient value.

2. The automatic parallel strategy search method based on polyhedral model modeling according to claim 1, characterized in that, The allocation of encoding / decoding architecture models on each of the two clients, based on the intelligent computing center resources of the intelligent computing network and using the optimal parallel strategy as a constraint, specifically includes: When one intelligent computing network has high computing power but no local data, while another intelligent computing network has low computing power but has the first type of local data, the encoding and decoding architecture model is assigned to the intelligent computing network with high computing power. When one intelligent computing network has high computing power and possesses the second type of local data, while another intelligent computing network has low computing power and possesses the first type of local data, the encoding and decoding architecture model is assigned to the intelligent computing network with high computing power. When the computing power of the intelligent computing network and the type of local data of the two clients are the same, and the distribution of the local data of the two clients is different, then the encoding and decoding architecture model will not be allocated. When the computing power of the intelligent computing networks of two clients is the same, and the distribution of the local data of the two clients is the same, then the encoding and decoding architecture model will not be allocated. When the computing power of the intelligent computing networks of the two clients, the type of the local data, and the distribution of the local data of the two clients are all the same, the encoding and decoding architecture model is evenly distributed to the two intelligent computing networks.

3. The automatic parallel strategy search method based on polyhedral modeling according to claim 1, characterized in that, The first client, using the allocated encoding / decoding architecture model, performs a first forward computation on the pre-stored local data, obtains a first forward output result, and sends it to the second client. Specifically, this includes: The first client is controlled to perform the first forward computation operation on the local data using the embedded layer in the allocated codec architecture model, to obtain the first forward output result, and then send the first forward output result to the second client.

4. The automatic parallel strategy search method based on polyhedral model modeling according to claim 1, characterized in that, The control of the second client to perform aggregation and a second forward computation operation on the first forward output result using the allocated encoding / decoding architecture model, and then send the second forward output result to the first client, specifically includes: The second client is controlled to perform an aggregation operation on the first forward output result to obtain the first aggregation result; The second client is controlled to use the multi-head attention mechanism layer in the allocated codec architecture model to perform the second forward computation operation on the first aggregation result, so as to obtain the second forward output result; The second client is controlled to send the second forward output result to the first client.

5. The automatic parallel strategy search method based on polyhedral modeling according to claim 1, characterized in that, The process of controlling the first client to perform a third forward calculation operation and a first backward gradient calculation operation on the second forward output result, and then sending the obtained first gradient value to the second client, specifically includes: The first client is controlled to use the head layer of the allocated codec architecture model to perform the third forward calculation operation on the second forward output result to obtain the third forward output result; The first client is controlled to perform the first backward gradient calculation operation on the third forward output result, and after obtaining the first gradient value, it is sent to the second client.

6. The automatic parallel strategy search method based on polyhedral model modeling according to claim 1, characterized in that, The step of controlling the second client to perform aggregation and a second backward gradient calculation operation on the first gradient value, and then sending the obtained second gradient value to the first client, specifically includes: The second client is controlled to perform the aggregation operation on the first gradient value to obtain the second aggregation result; The second client is controlled to use the codec layer in the allocated codec architecture model to perform the second inverse gradient calculation operation on the second aggregation result, and after obtaining the second gradient value, it is sent to the first client.

7. The automatic parallel strategy search method based on polyhedral model modeling according to claim 1, characterized in that, The step of controlling the first client to perform a third back gradient calculation operation on the second gradient value to obtain the third gradient value specifically includes: The first client is controlled to use the embedded layer in the allocated codec architecture model to perform the third inverse gradient calculation operation on the second gradient value to obtain the third gradient value.

8. The automatic parallel strategy search method based on polyhedral model modeling according to claim 2, characterized in that, The intelligent computing center resources include: the computing power of the intelligent computing network and the local data; the first type of local data is domain data; the second type of local data is general data.

9. The automatic parallel strategy search method based on polyhedral modeling according to claim 5, characterized in that, The first forward output is the embedded feature value; the second forward output is the hidden layer information; and the third forward output is the probability of the output character.

10. An automatic parallel strategy search system based on polyhedral model modeling, characterized in that, The automatic parallel strategy search system based on polyhedral model modeling is used to implement the automatic parallel strategy search method based on polyhedral model modeling as described in any one of claims 1-9, including: The model allocation module is used to allocate the encoding and decoding architecture model on each of the two clients based on the intelligent computing center resources of the intelligent computing network and with the optimal parallel strategy as a constraint. The gradient value calculation module is used to control the two clients to perform collaborative training and aggregation operations on the intelligent computing center resources using all allocated encoding and decoding architecture models, and to calculate the gradient value of the polyhedron model.

11. A controller, characterized in that, The controller includes: a memory, a processor, a display, and an automatic parallel strategy search program based on polyhedral modeling, stored in the memory and executable on the processor. When the automatic parallel strategy search program based on polyhedral modeling is executed by the processor, it implements the steps of the automatic parallel strategy search method based on polyhedral modeling as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an automatic parallel strategy search program based on polyhedral model modeling, which, when executed by a processor, implements the steps of the automatic parallel strategy search method based on polyhedral model modeling as described in any one of claims 1-9.

Images