Data processing method and related equipment

Abstract

A data processing method and related equipment can be used in the field of large models in the field of artificial intelligence, and the method comprises the following steps: obtaining first data; according to the first parameter and / or the second parameter, the value of r is determined, r is used for indicating that the first data is segmented into r sub-data, and the r sub-data is processed by a neural network layer in a machine learning model, the processing process of the first data is divided into a sending task of r sub-data, a processing task of the r sub-data and a sending task of a processing result of the r sub-data, and the time occupied by the tasks can be overlapped; the first parameter indicates the network condition of the communication network used for sending the sub-data, the second parameter indicates the load condition of the communication node used for processing the sub-data, and the value of r is determined according to the network condition and / or the load condition of the communication node, so that the value of r conforming to the actual condition can be obtained, and the user experience is improved. Therefore, the time consumed by one-time data processing process of the whole first data is shortened.

Description

A data processing method and related equipment Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art

[0002] With the development of artificial intelligence (AI) technology, trillion-level large models have been launched. The entire trillion-level large model can include at least one neural network module. The first neural network module in at least one neural network module can be deployed on multiple communication nodes. When any communication node on the multiple communication nodes (hereinafter referred to as the "first communication node" for the convenience of description) processes the first data through the neural network layer in the aforementioned first neural network module, the first communication node is required to first send the first data to other communication nodes among the multiple communication nodes. After the other communication nodes process the first data to obtain processing results, the other communication nodes send the processing results to the first communication node.

[0003] At present, the above three steps are in a serial relationship, that is, the first communication node sends the entire first data to other communication nodes, the other communication nodes process the entire first data to obtain the processing results of the entire first data, and the other communication nodes send the processing results of the entire first data to the first communication node; it takes a long time to perform data processing on the entire first data through the first neural network module.

[0004] Summary of the Invention

[0005] An embodiment of the present application provides a data processing method and related equipment, which divides the entire first data into r sub-data. The time occupied by the task of sending the r sub-data and the time occupied by the task of sending the processing results of the r sub-data can overlap with the time occupied by the task of processing the r sub-data, which is conducive to shortening the time consumed by a data processing process of the first data; the value of r is determined based on the network situation and / or the data processing load of the communication node, which is conducive to obtaining an r value that conforms to the actual situation, so as to further shorten the time consumed by a data processing process of the entire first data.

[0006] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a data processing method that can be used in the large model field of the artificial intelligence field. Multiple neural network layers in a first neural network module included in a first machine learning model can be deployed on multiple communication nodes included in the same communication node cluster; the method includes: a first communication node among the aforementioned multiple communication nodes obtains first data; the first communication node determines the value of r based on the first parameter and / or the second parameter, r is used to indicate that the first data is divided into r sub-data, and the r sub-data are respectively processed by the neural network layer in the machine learning model, r is an integer greater than or equal to 1, that is, the processing task of the first data is divided into the sending task of r sub-data, the processing task of r sub-data, and the sending task of the processing results of r sub-data.

[0008] The first parameter indicates the network status of the communication network used to send the sub-data, that is, the network status of the communication network that performs the task of sending each sub-data and the task of sending the processing result of each sub-data; the aforementioned communication network can be the network used for communication between different communication nodes in the communication node cluster used to deploy the first neural network module. The second parameter indicates the data processing load of the communication node used to process the sub-data, reflecting the length of time consumed by the communication node processing the sub-data to perform the processing task of each sub-data.

[0009] In this implementation, the entire first data is divided into r sub-data, and the r sub-data are processed by the neural network layer in the machine learning model respectively, that is, the processing process of the entire first data is divided into the sending task of r sub-data, the processing task of r sub-data and the sending task of the processing results of r sub-data. The time occupied by the sending task of r sub-data and the time occupied by the sending task of the processing results of r sub-data can overlap with the time occupied by the processing task of r sub-data, which is conducive to shortening the time consumed by a data processing process of the first data; in addition, the first parameter indicates the network condition of the communication network used to send the sub-data, and the second parameter indicates the load condition of the communication node used to process the sub-data. The value of r is determined according to the network condition and / or the load condition of the data processing of the communication node, which is conducive to obtaining an r value that conforms to the actual situation, so as to further shorten the time consumed by a data processing process of the entire first data.

[0010] In one possible implementation, the first parameter includes a first startup duration and a first duration. The first startup duration indicates the duration between determining a data transmission task and executing the data transmission task. The first startup duration can also be understood as the communication startup duration. The "data transmission task" can be a task of sending any one of r sub-data, or a task of sending the processing result of any one of the sub-data. The first duration represents the duration required to send a first data amount of data. The first data amount can also be understood as a unit data amount, which is used to indicate the size of the data. For example, the first data amount can be 1MB (megabyte), 512KB (kilobyte), or other sizes.

[0011] The first parameter includes a second startup duration and a second duration. The second startup duration indicates the duration between determining a data processing task and executing the data processing task. The second startup duration can also be understood as a processing startup duration. The data processing task represents processing data through the neural network layer in the first neural network module included in the first machine learning model. The "data processing task" represents a task of processing any one of r sub-data through the neural network layer in the first neural network module included in the first machine learning model. The second duration represents the time required to process a second amount of data through the neural network layer in the first machine learning model. The concept of the second data amount is similar to that of the first data amount. The values ​​of the second data amount and the first data amount may be the same or different.

[0012] In this implementation, the value of r is determined based on the first startup duration, the first duration, the second startup duration and the second duration. Relying on the first startup duration, the first duration, the second startup duration and the second duration can more accurately reflect the time consumed by each stage in the one-time data processing process of the entire first data, which is conducive to obtaining the optimal r to further shorten the time consumed by the one-time data processing process of the entire first data.

[0013] In one possible implementation, the first communication node determines a value of r based on a first parameter and / or a second parameter, including: the first communication node solves an objective function based on the first parameter and / or the second parameter to determine the value of r, wherein the objective function represents the total time consumed to obtain the processing results of r sub-data. That is, the objective function can be obtained by modeling the total time consumed by executing the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data. The solution goal of the objective function can be to minimize the value of the objective function. The constraints of the objective function are related to the time occupied by the task of sending the sub-data and the time occupied by the task of processing the sub-data.

[0014] In this implementation, the objective function represents the total time consumed to obtain the processing results of r sub-data, and the constraints of the objective function are related to the time occupied by the sub-data sending task and the time occupied by the sub-data processing task. That is, the objective function can accurately reflect the total time consumed to process the r sub-data, and convert the determination process of r into the solution process of the objective function, thereby improving the precision of the determination process of r and facilitating obtaining the optimal value of r, so as to further shorten the total time consumed by the entire first data processing process.

[0015] In one possible implementation, the objective function includes one or more of a first objective function, a second objective function, or a third objective function; wherein, the constraint of the first objective function includes that the time occupied by a sub-data sending task is greater than or equal to the time occupied by a sub-data processing task; the constraint of the second objective function includes that the time occupied by a sub-data sending task is less than the time occupied by a sub-data processing task, and the second time is not a subset of the first time, the first time includes the time occupied by r sub-data processing tasks, the second time includes the time occupied by r sub-data sending tasks and the time occupied by the sending task of r sub-data processing results; the constraint of the third objective function includes that the time occupied by a sub-data sending task is less than the time occupied by a sub-data processing task, and the second time is a subset of the first time.

[0016] In this implementation, the total time consumed to obtain the processing results of r sub-data is represented by the first objective function, the second objective function and the third objective function. The constraints of the first objective function, the second objective function and the third objective function are all different, that is, the process of determining the value of r is divided into three situations, which are described by the first objective function and the constraints of the first objective function, the second objective function and the constraints of the second objective function, and the third objective function and the constraints of the third objective function respectively. Determining the value of r at a finer level is conducive to obtaining a better solution for r, thereby helping to further shorten the time consumed by the entire first data processing process.

[0017] In one possible implementation, the first communication node solves the objective function based on the first parameter and / or the second parameter to determine the value of r, which may include: after the first communication node determines the first startup duration, the first duration, the second duration, the second duration and the data volume of the first data, it can determine the first ratio between the total data volume of the r data packets corresponding to the r sub-data and the first data volume, and the second ratio between the data volume of the first data and the second data volume based on the data volume of the first data, and then solve the first objective function, the second objective function and / or the third objective function based on the first startup duration, the first duration, the second duration, the second duration, the first ratio and the second ratio to determine the value of r.

[0018] In one possible implementation, the first communication node solves the first objective function, the second objective function, and / or the third objective function based on the first startup duration, the first duration, the second duration, the second duration, the first ratio, and the second ratio to determine the value of r. This may include: the first communication node substitutes the first startup duration, the first duration, the second duration, the second duration, the first ratio, and the second ratio into the first objective function and the constraints of the first objective function, into the second objective function and the constraints of the second objective function, and into the third objective function and the constraints of the third objective function, and solves the first objective function, the second objective function, and the third objective function respectively, so as to obtain the r value corresponding to the first objective function, the r value corresponding to the second objective function, and the r value corresponding to the third objective function, respectively. The first communication node may obtain the r value corresponding to the objective function with the smallest value among the first objective function, the second objective function, and the third objective function, and determine the obtained r value as the value of r.

[0019] In one possible implementation, the first communication node solves the first objective function, the second objective function and / or the third objective function based on the first startup duration, the first duration, the second duration, the second duration, the first ratio and the second ratio to determine the value of r, which may include: the first communication node may select an objective function that matches the aforementioned "first startup duration, first duration, second duration and second duration" from the first objective function, the second objective function and the third objective function based on the first startup duration, the first duration, the second duration and the second duration, and bring the first startup duration, the first duration, the second duration, the second duration, the first ratio and the second ratio into the aforementioned selected objective function and the constraints of the objective function, solve the aforementioned selected objective function to obtain a value of r.

[0020] In one possible implementation, the first machine learning model includes at least one neural network module, and the multiple neural network layers included in the first neural network module in the at least one neural network module are deployed in the same communication node cluster, and the first parameters and second parameters of the multiple communication nodes included in the communication node cluster are the same.

[0021] In this implementation, since the communication loads of different communication nodes in the communication node cluster used to deploy the first neural network module are likely to be similar, it is in line with actual communication conditions for different communication nodes to deploy the same first parameters, and the adoption of the aforementioned scheme is conducive to reducing the difficulty of the process of "determining the first parameters deployed on multiple communication nodes"; the number of neural network layers deployed by different communication nodes in the communication node cluster used to deploy the first neural network module is generally the same, and the data processing loads on different communication nodes are also likely to be similar, and it is in line with actual conditions for different communication nodes to deploy the same second parameters, and the adoption of the aforementioned scheme is conducive to reducing the difficulty of the process of "determining the second parameters deployed on multiple communication nodes"; in addition, since the first parameters and second parameters on different communication nodes will be used in the process of determining the value of r, different communication nodes deploying the same first parameters and second parameters is also conducive to reducing the complexity of the process of determining the value of r.

[0022] In one possible implementation, the first duration represents the duration required to send a first amount of data using an all-to-all communication method. Exemplarily, the all-to-all communication method may include: when a first communication node in a communication node cluster sends data to a second communication node, the first communication node may call an interface of an all-to-all communication function to participate in queuing, and when it is the turn to execute the data sending task of the first communication node, each communication node in the communication node cluster sends a data packet to other communication nodes in the communication node cluster, that is, a total of multiple data packets are sent between the multiple communication nodes included in the communication node cluster, and the aforementioned multiple data packets include multiple first data packets carrying valid data, and the first data packet refers to a data packet sent by the first communication node to the second communication node. Optionally, the aforementioned multiple data packets may also include a second data packet, and the second data packet refers to a data packet other than the first data packet in the aforementioned multiple data packets, and the second data packet may carry invalid data.

[0023] In this implementation, a specific communication method is provided for communication between different communication nodes in a communication node cluster, thereby improving the degree of integration of this solution with practical applications.

[0024] In one possible implementation, the first machine learning model is a mixture of experts (MoE) model, the first neural network module in the first machine learning model can be expressed as an MoE module in the MoE model, each MoE module can include a gate neural network layer and multiple expert neural network layers, and the second duration represents the duration required for each expert neural network layer in the MoE module included in the MoE model to process the second amount of data. In this implementation, the first machine learning model is determined to be a MoE model, which provides a specific application scenario of the method provided in this application, improves the degree of integration of this solution with the specific application scenario, and the MoE model has a high degree of matching with the method provided in this application, which is conducive to greatly shortening the time consumed by the MoE model in the data processing process.

[0025] In one possible implementation, the first parameter is obtained by statistically analyzing multiple data transmission tasks executed using an all-to-all communication method, where some of the multiple data transmission tasks have different data volumes. In this implementation, different communication nodes all communicate using the all-to-all communication method, further increasing the probability that the first parameters corresponding to different communication nodes are consistent. The first parameter obtained by statistically analyzing multiple data transmission tasks executed using the all-to-all communication method is more consistent with the actual communication situation in the communication node cluster, which helps ensure that the determination process of r is consistent with the actual communication situation.

[0026] In one possible implementation, the second parameter is obtained by statistically analyzing multiple data processing tasks performed by the neural network layer in the first machine learning model, where the multiple data processing tasks involve processing data of varying amounts. This implementation provides a method for determining the second parameter, which reduces the difficulty of implementing the method provided herein. Furthermore, the second parameter is obtained by statistically analyzing multiple data processing tasks, where the multiple data processing tasks involve processing data of varying amounts. This facilitates compatibility of the determined second parameter with data processing tasks of varying amounts, and facilitates obtaining the optimal value of r for data processing tasks of varying amounts.

[0027] In one possible implementation, a first parameter deployed in a plurality of communication nodes included in a communication node cluster for deploying a first neural network module is adjustable; illustratively, a first startup duration, a first duration, a second startup duration, and a second duration deployed in a plurality of communication nodes included in a communication node cluster for deploying a first neural network module are adjustable.

[0028] In one possible implementation, the r sub-data are respectively processed by the neural network layer in the first machine learning model, including: in the inference phase of the first machine learning model, the r sub-data are respectively processed by the neural network layer in the first machine learning model; or, in the training phase of the first machine learning model, the r sub-data are respectively processed by the neural network layer in the first machine learning model.

[0029] In this implementation, the data processing method provided in this application can be used in both the inference stage and the training stage of the first machine learning model, which expands the application scenarios of this solution, shortens the time taken for data processing in the inference stage and the training stage of the first machine learning model, and is conducive to improving the efficiency of data processing in the inference stage and the training stage of the first machine learning model.

[0030] In a second aspect, an embodiment of the present application provides a data processing device that can be used in the large model field of artificial intelligence. The aforementioned data processing device includes: an acquisition module for acquiring first data; a determination module for determining the value of r based on a first parameter and / or a second parameter, where r is used to indicate that the first data is divided into r sub-data, and the r sub-data are respectively processed by the neural network layer in the machine learning model, r is an integer greater than or equal to 1, the first parameter indicates the network condition of the communication network used to send the sub-data, and the second parameter indicates the load condition of the communication node used to process the sub-data.

[0031] In the second aspect of the present application, the data processing device can also be used to execute the steps performed by the first communication node in the first aspect and various possible implementation methods of the first aspect. The specific implementation methods, meanings of terms and beneficial effects brought about by the steps in each possible implementation method of the second aspect can all be referred to the first aspect and will not be repeated here.

[0032] In a third aspect, an embodiment of the present application provides a communication node, which includes a processor, and the processor is used to execute the data processing method of the first aspect above.

[0033] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; and the processor is used to execute the programs in the memory, so that the electronic device executes the data processing method of the first aspect mentioned above.

[0034] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data processing method described in the first aspect above.

[0035] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program is run on a computer, it enables the computer to execute the data processing method described in the first aspect above.

[0036] In a seventh aspect, the present application provides a chip system, which includes a processor for supporting a communication node or electronic device to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of the structure of an artificial intelligence main framework provided in an embodiment of the present application;

[0038] FIG2 is a schematic diagram of deploying a first neural network module in a first machine learning model on a communication node cluster according to an embodiment of the present application;

[0039] FIG3 is a system architecture diagram of a data processing system provided in an embodiment of the present application;

[0040] FIG4 is a schematic diagram showing three embodiments of the present invention showing the total time taken for a data processing process of the first data using the first neural network module;

[0041] FIG5 is a flow chart of a data processing method provided in an embodiment of the present application;

[0042] FIG6 is a schematic diagram of the relationship between the first machine learning model and the first neural network module provided in an embodiment of the present application;

[0043] FIG7 is a schematic diagram of deploying a MoE module in a MoE model on a communication node cluster according to an embodiment of the present application;

[0044] FIG8 is a schematic diagram of an all-to-all communication mode adopted between different communication nodes in a communication node cluster provided in an embodiment of the present application;

[0045] 9 is a schematic diagram illustrating the relationship between the time taken by the task of sending r sub-data and the task of sending the processing results of r sub-data and the time taken by the task of processing r sub-data according to an embodiment of the present application;

[0046] 10 is another schematic diagram of the relationship between the time taken by the task of sending r sub-data and the task of sending the processing results of r sub-data and the time taken by the task of processing r sub-data provided by an embodiment of the present application;

[0047] 11 is another schematic diagram of the relationship between the time taken by the task of sending r sub-data and the task of sending the processing results of r sub-data and the time taken by the task of processing r sub-data provided by an embodiment of the present application;

[0048] FIG12 is a flow chart of a data processing method provided in an embodiment of the present application;

[0049] FIG13 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0050] FIG14 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0051] FIG15 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] First, let's describe the overall workflow of an AI system. See Figure 1, which shows a schematic diagram of the AI ​​framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0055] (1) Infrastructure

[0056] The infrastructure provides computing power for AI systems, enabling communication with the outside world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by smart chips, which can specifically adopt hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs); the basic platform includes related platform guarantees and support such as distributed computing frameworks and networks, and can include cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to obtain data, which is then provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0057] (2) Data

[0058] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0059] (3) Data processing

[0060] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0061] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0062] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0063] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0064] (4) General ability

[0065] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0066] (5) Smart products and industry applications

[0067] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart manufacturing, smart transportation, smart homes, smart medical care, smart security, autonomous driving, smart cities, etc.

[0068] This application can be applied to various fields in the field of artificial intelligence. Optionally, it can be applied to scenarios where a large first machine learning model is used for data processing. For example, the aforementioned first machine learning model can be a trillion-level machine learning model, etc., which is not limited here. Exemplarily, the aforementioned first machine learning model can be applied to fields that require machine learning models to implement, such as natural language synthesis tasks related to speech semantics, visual perception tasks, and audio and video processing tasks. The following examples illustrate multiple application scenarios of this application.

[0069] Application Scenario 1: Natural Language Processing (NLP)

[0070] Natural language processing is the processing of human language. It's the process of systematically analyzing, understanding, and extracting information from text data using a machine learning model. Machine learning models can be used to perform natural language processing tasks in areas like smart terminals and smart homes. By using these machine learning models, we can manage very large amounts of text data, perform a large number of automated tasks, and solve a wide variety of problems, such as automatic summarization, machine translation (MT), named entity recognition (NER), relation extraction (RE), information extraction (IE), sentiment analysis, speech recognition, question answering, and topic segmentation.

[0071] For example, natural language processing tasks can be divided into the following categories.

[0072] Sequence labeling: Each word in a sentence requires the model to give a classification category based on the context, such as Chinese word segmentation, part-of-speech tagging, named entity recognition, and semantic role labeling.

[0073] Classification task: The entire sentence outputs a classification value, such as text classification.

[0074] Sentence Relationship Inference: Given two sentences, determine whether they have a certain nominal relationship. Examples include question-answering systems, semantic rewriting, and natural language inference.

[0075] Generative tasks: Output a piece of text and generate another piece of text. Examples include machine translation, text summarization, poetry and sentence composition, and picture description.

[0076] Application Scenario 2: Object Detection

[0077] For example, in the field of autonomous driving, an autonomous vehicle may use sensors to collect point cloud data corresponding to its surroundings. Based on the collected point cloud data, the autonomous vehicle may perform object detection using a first machine learning model to obtain a prediction result corresponding to the point cloud data. The prediction result indicates the location of at least one object in the surroundings of the autonomous vehicle. The autonomous vehicle may then plan its driving path based on the prediction result. The aforementioned vehicles may include cars, trucks, motorcycles, buses, boats, airplanes, helicopters, recreational vehicles, amusement park vehicles, construction equipment, trams, golf carts, or trains, and are not specifically limited in the present embodiments.

[0078] For example, in the field of intelligent monitoring, many cameras will be installed in public places and traffic roads. After a few intelligent cameras collect image information of the surrounding environment, they can use the first machine learning model to perform target detection tasks on the collected images, etc. There may be a need to use the first machine learning model to perform target detection on images in many application fields, which will not be listed one by one here.

[0079] Application Scenario 3: Semantic Segmentation of Images

[0080] Semantic segmentation refers to the classification of all pixels in an image using a first machine learning model. The quantization method of the model provided in this application can be used to compress the aforementioned first machine learning model. In fields such as smart terminals and smart healthcare, there may be a need to use the first machine learning model to perform semantic segmentation on images.

[0081] It should be noted that the data processing method provided in this application can also be applied to other scenarios in which data processing is performed using the first machine learning model. The above examples of various application scenarios of this application are only for the convenience of understanding this solution and are not used to limit this solution.

[0082] The first machine learning model in this application may include one or more first neural network modules, each of which includes multiple neural network layers. The multiple neural network layers included in each first neural network module are deployed in the same communication node cluster, that is, the multiple neural network layers included in the first neural network module are deployed in multiple communication nodes included in the same communication node cluster. Optionally, the first machine learning model may also include a second neural network module, and the neural network layers included in the second neural network module are all deployed on a communication node.

[0083] Each communication node may include one or more processors. For example, a communication node may be expressed as an NPU, CPU, GPU, network card, or other product forms; alternatively, each communication node may be expressed as a virtualized device deployed on a physical device, such as a virtual machine (VM), etc. The product form of each communication node can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application.

[0084] Multiple communication nodes used to deploy the first machine learning model can be included in the same electronic device. For example, the aforementioned electronic device can be expressed as a server, or as a server cluster composed of multiple servers, etc. The specific product form can be determined in combination with the actual application scenario, and is not limited in the embodiments of this application.

[0085] In order to understand the present solution more intuitively, please refer to Figure 2, which is a schematic diagram of a first neural network module in a first machine learning model deployed on a communication node cluster provided in an embodiment of the present application. In Figure 2, a communication node cluster including three communication nodes is taken as an example. Neural network layer 1, neural network layer 2 and neural network layer 3 are deployed on communication node 1, neural network layer 1, neural network layer 4 and neural network layer 5 are deployed on communication node 2, and neural network layer 1, neural network layer 6 and neural network layer 7 are deployed on communication node 3. All the neural network layers deployed on communication node 1, communication node 2 and communication node 3 constitute a first neural network module in the first machine learning model; it should be noted that a first neural network module may include more or fewer neural network layers, and the neural network layers included in a first neural network module may be deployed in more or fewer communication nodes. The example in Figure 2 is only for the convenience of understanding the present solution and is not used to limit the present solution.

[0086] When it is necessary to process the first data through a first neural network module in the first machine learning model, since the multiple neural network layers included in the first neural network module are deployed on multiple communication nodes included in a communication node cluster, after the first data arrives at the first communication node among the aforementioned multiple communication nodes, the neural network layer that processes the first data may be deployed on other communication nodes other than the first communication node (hereinafter referred to as "second communication node" for the convenience of description); the currently adopted method is that the first communication node first sends the entire first data to the second communication node; the first data is processed by the second communication node to obtain the processing result of the first data; the second communication node then sends the processing result of the first data to the first communication node, and the time consumed for data processing of the entire first data by the first neural network module is relatively long.

[0087] Before introducing the data processing method provided by the present application, please refer to Figure 3. Figure 3 is a system architecture diagram of the data processing system provided by an embodiment of the present application. In Figure 3, the data processing system 300 includes a training device 310, a database 320, an execution device 330, a data storage system 340 and a client device 350.

[0088] Among them, the database 320 stores a training data set, the training device 310 generates a first machine learning model / rule 301, and uses the training data set to iteratively train the first machine learning model / rule 301 to obtain the trained first machine learning model / rule 301.

[0089] The trained first machine learning model / rule 301 obtained by the training device 310 can be applied to the execution device 330, wherein the execution device 330 can call data, code, etc. in the data storage system 340, and can also store data, instructions, etc. in the data storage system 340. The data storage system 340 can be located in the execution device 330, or the data storage system 340 can be an external memory relative to the execution device 330.

[0090] In some embodiments of the present application, please refer to Figure 3, the execution device 330 and the client device 350 can be independent of each other, and the execution device 330 is configured with an input / output (I / O) interface to interact with the client device 350 for data. The "user" can input the data to be processed through the client device 350, and the client device 350 sends the data to be processed to the execution device 330 through the I / O interface. After the execution device 330 generates prediction information corresponding to the data to be processed through the first machine learning model / rule 301, the aforementioned prediction information can be returned to the client device 350 through the I / O interface and provided to the user.

[0091] It is worth noting that FIG3 is only a schematic diagram of the architecture of two data processing systems provided by an embodiment of the present invention, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation.

[0092] For example, since the first machine learning model / rule 301 is relatively large, each first neural network module in the first machine learning model / rule 301 is deployed on multiple communication nodes, so the training device 310 and the execution device 330 can both include multiple communication nodes. In both the inference stage and the training stage of the first machine learning model / rule 301, there is a problem that the time consumed by the first neural network module in the first machine learning model / rule 301 to process the first data is too long.

[0093] In order to shorten the time consumed by the first neural network module in the first machine learning model / rule 301 to perform data processing on the first data, the present application provides a data processing method. When the first data needs to be processed by the first neural network module in the first machine learning model / rule 301, the first data can be divided into r sub-data, and the neural network layer in the first neural network module is used to process the r sub-data respectively to obtain the processing results of the r sub-data, that is, the processing results of the entire first data are obtained. Since there is an overlap between the first time and the second time, the first time includes the time occupied by the task of sending the r sub-data and the time occupied by the task of sending the processing results of the r sub-data, and the second time includes the time occupied by the processing task of the r sub-data, which is conducive to shortening the time occupied by the data processing process of the first data.

[0094] To understand the present solution more intuitively, please refer to FIG4 . FIG4 is three schematic diagrams of the total time taken by a data processing process of the first data using the first neural network module according to an embodiment of the present application. The first row in FIG4 shows a situation where the first data is not segmented. As shown in FIG4 , there is no overlap between the time taken by the task of sending the entire first data, the time taken by the task of processing the entire first data, and the time taken by the task of sending the processing result of the entire first data. The second and third rows in FIG4 show a situation where the first data is segmented into two sub-data. The second row in FIG4 shows the time taken by the task of sending the two sub-data and the time taken by the task of sending the processing result of the two sub-data (i.e., an example of the first time). The third row in FIG4 shows the time taken by the task of processing the two sub-data (i.e., an example of the second time). There is overlap between the first time and the second time shown in the second and third rows in FIG4 .

[0095] The fourth and fifth rows in FIG4 illustrate the situation where the first data is divided into four sub-data. The fourth row in FIG4 illustrates the time taken by the task of sending the four sub-data and the time taken by the task of sending the processing results of the four sub-data (i.e., an example of the first time). The third row in FIG4 illustrates the time taken by the task of processing the four sub-data (i.e., an example of the second time). There is an overlap between the first and second times shown in the fourth and fifth rows in FIG4. Among the three situations shown in FIG4, the situation where "the first data is not divided" takes the longest total time, and the situation where "the first data is divided into two sub-data" takes the shortest total time. It should be understood that the example in FIG4 is only for the convenience of understanding the concept that "dividing the first data is beneficial to shortening the time taken by the first data processing process" and is not used to limit this solution.

[0096] In order to shorten the time consumed by the processing of the first data as much as possible, the data processing method provided in the present application is used to determine the value of r, that is, to determine how many sub-data to divide the first data into. Since the first neural network module in the first machine learning model / rule 301 processes the first data in both the reasoning stage and the training stage of the first machine learning model / rule 301, the data processing method provided in the present application can be used in both the reasoning stage and the training stage of the first machine learning model / rule 301. The reasoning stage and the training stage of the first machine learning model / rule 301 are described below respectively.

[0097] 1. Reasoning Stage

[0098] In the present embodiment, the inference phase refers to the process by which the execution device 330 generates the processing results of the second data using the first machine learning model / rule 301 that has been trained. The meaning of "second data" will be explained in the subsequent description. Please refer to Figure 5, which is a flow chart of a data processing method provided in the present embodiment. The data processing method provided in the present embodiment may include:

[0099] 501. Obtain first data.

[0100] In an embodiment of the present application, when the execution device needs to process the second data through the first machine learning model, the execution device can input the second data into the first machine learning model. In the process of processing the second data using the first machine learning model, one of the multiple communication nodes deployed with the first neural network module (hereinafter referred to as the "first communication node" for convenience of description) can obtain the first data processed by the first neural network module.

[0101] The second data may be of any data type. For example, the data type of the second data may be text, image, audio, video or other data types, etc., which are not limited here.

[0102] The second data refers to the data input into the first machine learning model, and the first data refers to the data that needs to be processed by a first neural network module in the first machine learning model. Since the first machine learning model may include one or more first neural network modules; optionally, the first machine learning model may also include at least one second neural network module, the second neural network module includes at least one neural network layer, and all neural network layers included in the second neural network module can be deployed on a single communication node, the second data and the first data may be the same or different.

[0103] For example, in one case, if the first neural network module used to process the first data is the first neural network module in the first machine learning model, the first data and the second data may be the same data; in another case, if the first neural network module used to process the first data is not the first neural network module in the first machine learning model, the first data may be obtained by processing the second data using some neural network modules in the first machine learning model, that is, the first data is an intermediate result generated in the process of processing the second data using the first machine learning model. Specifically, the relationship between the first data and the second data needs to be determined in combination with factors such as the position of the first neural network module used to process the first data in the first machine learning model and the architecture of the first machine learning model, which is not limited here.

[0104] Exemplarily, the first data may be specifically represented as a tensor.

[0105] The first machine learning model can be expressed as various types of large models. Optionally, the first machine learning model can be expressed as a mixture of experts (MoE) model, and the first neural network module in the first machine learning model can be expressed as an MoE module in the MoE model, and each MoE module (i.e., an example of the first neural network module) can include a gate neural network layer and multiple expert neural network layers.

[0106] For example, when the MoE module is deployed across multiple communication nodes in a communication node cluster, each communication node may be deployed with a gate neural network layer and at least one expert neural network layer; the gate neural network layers deployed on different communication nodes may be the same, and the number of expert neural network layers deployed on different communication nodes may be the same or different. When first data arrives at the first communication node, the gate neural network layer in the MoE module is used to determine which expert neural network layer or layers in the MoE module are to process the first data, and the expert neural network layer in the MoE module is used to actually process the first data.

[0107] In order to understand this solution more intuitively, please refer to Figures 6 and 7. Figure 6 is a schematic diagram of the relationship between the first machine learning model and the first neural network module provided in an embodiment of the present application, and Figure 7 is a schematic diagram of deploying a MoE module in the MoE model on a communication node cluster provided in an embodiment of the present application. First refer to Figure 6. In Figure 6, the MoE model (that is, an example of the first machine learning model) not only includes a MoE module (that is, an example of the first neural network module), but also includes two neural network modules based on the attention mechanism and other types of neural network modules, etc., which are not exhaustively listed in the embodiments of the present application. Among them, the MoE module includes a gate neural network layer and N expert neural network layers. The multiple expert neural network layers included in a MoE module can be deployed on multiple communication nodes. The specific deployment will be introduced through Figure 7, and will not be described here.

[0108] As shown in Figure 6, the second data is input into the first machine learning model, and the first data is obtained after the second data is processed by the first neural network module based on the attention mechanism; the first data is processed by the expert neural network layer 2 and the expert neural network layer 5 in the MoE module to obtain the processing result of the first data; the processing result of the first data is further processed by the second neural network based on the attention mechanism to obtain the intermediate result generated by the second neural network based on the attention mechanism; and then the intermediate result generated by the second neural network based on the attention mechanism is continued to be processed by other neural network modules in the first machine learning model, and so on, and finally the processing result of the second data output by the first machine learning model can be obtained; it should be understood that the example in Figure 6 is only for the convenience of understanding this solution and is not used to limit this solution.

[0109] Referring to Figure 7 again, Figure 7 takes the processing of text data through the MoE module as an example. A gate neural network layer and two expert neural network layers are deployed on each communication node in the communication node cluster. The expert neural network layers deployed on different communication nodes are used to process different types of words. For example, as shown in Figure 7, the two expert neural network layers deployed on communication node 1 (i.e., expert1 and expert2 in Figure 7) are used to process nouns and verbs, respectively. The two expert neural network layers deployed on communication node 2 (i.e., expert3 and expert4 in Figure 7) are used to process prepositions and adjectives, respectively. The two expert neural network layers deployed on communication node 3 (i.e., expert5 and expert6 in Figure 7) are used to process adverbs and articles, respectively. It should be understood that the example in Figure 7 is only for the convenience of understanding this solution and is not used to limit this solution.

[0110] 502. Acquire a first parameter, where the first parameter indicates a network condition of a communication network used to send sub-data.

[0111] In an embodiment of the present application, step 502 is an optional step. After obtaining the first data, the first communication node can obtain a first parameter in order to determine the value of r. The first parameter indicates the network status of the communication network used to send each sub-data in the r sub-data, that is, the network status of the communication network that performs the sending task of each sub-data and the sending task of the processing result of each sub-data; the aforementioned communication network can be a network used for communication between different communication nodes in the communication node cluster used to deploy the first neural network module.

[0112] Optionally, the first parameter may include a first startup duration and a first duration, wherein the first communication node in the communication node cluster used to deploy the first neural network module may have a first startup duration between determining the data sending task and starting to execute the data sending task. The first startup duration can also be understood as the communication startup duration; the first duration represents the time required to send the first amount of data.

[0113] The aforementioned "data transmission task" may be a task for transmitting any one of the r sub-data, or a task for transmitting the processing result of any one of the sub-data. For example, since the "r sub-data transmission task" and the "r sub-data processing result transmission task" are performed within the same communication node set, the communication environment in which the sub-data is transmitted and the communication environment in which the sub-data is transmitted can be considered the same. Therefore, the communication startup duration for transmitting the sub-data and the communication startup duration for transmitting the processing result of the sub-data can be considered the same, and the duration for transmitting the sub-data of the first data volume and the duration for transmitting the sub-data of the first data volume can be considered the same.

[0114] The first data amount can also be understood as a unit data amount, which is used to indicate the size of the data. For example, the first data amount can be 1MB (megabyte), 512KB (kilobyte), or other sizes. The examples here are only for the convenience of understanding the meaning of the "first data amount" and are not used to limit this solution.

[0115] Optionally, an all-to-all communication method may be used between different communication nodes in the communication node cluster used to deploy the first neural network module, and the first duration may represent the duration required to send the first amount of data using the all-to-all communication method.

[0116] Exemplarily, the use of an all-to-all communication method may include: when the first communication node in a communication node cluster sends data to the second communication node, the first communication node may call the interface of the all-to-all communication function to participate in the queue, and when it is the turn to execute the data sending task of the first communication node, each communication node in the communication node cluster sends a data packet to other communication nodes in the communication node cluster, that is, a total of multiple data packets are sent between the multiple communication nodes included in the communication node cluster, and the aforementioned multiple data packets include multiple first data packets carrying valid data, and the first data packet refers to a data packet sent by the first communication node to the second communication node. Optionally, the aforementioned multiple data packets may also include a second data packet, and the second data packet refers to a data packet other than the first data packet in the aforementioned multiple data packets, and the second data packet may carry invalid data.

[0117] To understand this solution more intuitively, please refer to Figure 8, which is a schematic diagram of the all-to-all communication method adopted between different communication nodes in the communication node cluster provided in an embodiment of the present application. As shown in Figure 8, the six expert neural network layers included in the first neural network module are deployed on three communication nodes. Taking communication node 1 as the first communication node, and the gate neural network layer in communication node 1 determining that the first data needs to be processed by expert 3 as an example, that is, communication node 2 is the second communication node, communication node 1 uses the all-to-all communication method to send each sub-data to communication node 2, which may include: communication node 1 sends a data packet to communication node 2 and communication node 3 respectively, communication node 2 sends a data packet to communication node 1 and communication node 3 respectively, and communication node 3 sends a data packet to communication node 1 and communication node 2 respectively. Among them, the data packet sent by communication node 1 to communication node 2 carries sub-data, and the data packet sent by communication node 1 to communication node 3 carries invalid data; the data packet sent by communication node 2 to communication node 1 and communication node 3 carries invalid data; the data packet sent by communication node 3 to communication node 1 and communication node 2 carries invalid data; it should be understood that the example in Figure 8 is only for the convenience of understanding this solution and is not used to limit this solution.

[0118] In an embodiment of the present application, a specific communication method is provided for communication between different communication nodes in a communication node cluster, thereby improving the degree of integration of the present solution with practical applications.

[0119] Exemplarily, the communication node cluster used to deploy the first neural network module includes multiple communication nodes, and the multiple communication nodes may be pre-deployed with first parameters. Optionally, the first parameters deployed in the multiple communication nodes included in the communication node cluster are all the same. For example, the first startup duration and the first duration deployed in the multiple communication nodes included in the communication node cluster are all the same.

[0120] In an embodiment of the present application, since the communication loads of different communication nodes in the communication node cluster used to deploy the first neural network module are likely to be similar, deploying the same first parameters on different communication nodes is in line with actual communication conditions. The adoption of the aforementioned scheme is conducive to reducing the difficulty of the process of "determining the first parameters deployed on multiple communication nodes." In addition, since the first parameters on different communication nodes will be used in the process of determining the value of r, deploying the same first parameters on different communication nodes is also conducive to reducing the complexity of the process of determining the value of r.

[0121] An embodiment of the present application also provides a method for obtaining the first parameter. The first parameter is obtained after statistics are collected on multiple data sending tasks performed by any one or more communication nodes in the aforementioned communication node cluster. There are data sending tasks with different data amounts among the multiple data sending tasks, that is, there are at least two data sending tasks among the multiple data sending tasks, and the size of the data to be sent is different.

[0122] For example, the above-mentioned multiple data sending tasks may include executing 15 data sending tasks, and the 15 data sending tasks include sending 2MB data packets 5 times, 3MB data packets 5 times, 5MB data packets 2 times, 8MB data packets 3 times and 10MB data packets 5 times. After counting the duration of the above-mentioned 15 data sending tasks, the first parameter is obtained. It should be noted that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0123] Exemplarily, in the statistical process of the first parameter, the above-mentioned multiple data sending tasks can be executed by the same communication node (hereinafter referred to as the "third communication node" for the convenience of description) in the communication node cluster used to deploy the first neural network module, and the third communication node determines the first parameter based on the total duration of each data sending task in the multiple data sending tasks.

[0124] Alternatively, the above-mentioned multiple data sending tasks can also be executed by different communication nodes in the aforementioned communication node cluster, and the total duration of executing each data sending task will be summarized on the same third communication node, and the third communication node will determine the first parameter based on the total duration of each data sending task in the multiple data sending tasks.

[0125] Here, taking the first parameter including the first startup duration and the first duration as an example, in this application, the total duration of executing a data transmission task by any communication node in the communication node cluster used to deploy the first neural network module can be modeled as the following formula: a2a (y) = α a +β a ×y;(1)

[0126] Among them, t a2a (y) represents the total time it takes for any communication node in the above communication node cluster to execute a data sending task, α a represents the duration between determining the aforementioned data sending task and starting to execute the data sending task (i.e., the first startup duration), β a represents the time required to send the first amount of data (i.e., the first time), and y represents the amount of data to be sent, which is the product of y and the first amount of data. After obtaining the total time consumed by each data sending task in the above multiple data sending tasks, α can be fitted. a and βa The value of , that is, the value of the first startup duration and the first duration are obtained. It should be understood that the example in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.

[0127] Optionally, if an all-to-all communication mode is adopted between different communication nodes in the communication node cluster used to deploy the first neural network module, the first parameter is obtained by counting multiple data sending tasks performed by any one or more communication nodes in the aforementioned communication node cluster using the all-to-all communication mode, and there are data sending tasks with different data amounts among the multiple data sending tasks.

[0128] Exemplarily, the first startup duration and the first duration are obtained by collecting statistics on multiple data sending tasks executed by any one or more communication nodes in the aforementioned communication node cluster using an all-to-all communication mode.

[0129] In the embodiment of the present application, different communication nodes adopt an all-to-all communication mode when communicating, which further improves the probability that the first parameters corresponding to different communication nodes are consistent. The first parameter obtained after statistics of multiple data sending tasks performed using the all-to-all communication mode is more consistent with the actual communication situation in the communication node cluster, which is conducive to making the determination process of r consistent with the actual communication situation.

[0130] Optionally, the first parameters deployed in the above-mentioned multiple communication nodes are adjustable; illustratively, the first startup duration and the first duration deployed in the above-mentioned multiple communication nodes are adjustable; that is, during the operation of the above-mentioned multiple communication nodes, at least one communication node in the communication node cluster used to deploy the first neural network module can re-execute multiple data sending tasks, and perform statistics on the aforementioned re-executed multiple data sending tasks to obtain updated parameters; and redeploy the updated first parameters on the multiple communication nodes included in the aforementioned communication node cluster.

[0131] For example, when the first neural network module is initially deployed on the communication node cluster, the first parameter may be statistically obtained; after the first neural network module has been running for a first duration, the updated first parameter may be statistically obtained again. For another example, the updated first parameter may be re-obtained every preset duration. For another example, when at least one communication node in the communication node cluster used to deploy the first neural network module changes, the updated first parameter may be re-obtained, and so on. The updated first parameter may also be re-obtained in other situations. The specific details may be flexibly determined in combination with actual application scenarios and are not limited in the embodiments of the present application.

[0132] 503. Obtain a second parameter, where the second parameter indicates a load condition of a communication node for processing sub-data.

[0133] In an embodiment of the present application, step 503 is an optional step. After obtaining the first data, the first communication node can obtain a second parameter in order to determine the value of r. The second parameter indicates the data processing load of the communication node used to process the sub-data, so as to reflect the time consumed by the communication node for processing the sub-data in executing the processing task of each sub-data.

[0134] Optionally, the second parameter may include a second startup duration and a second duration, the second startup duration indicating the duration between determining the data processing task and executing the data processing task, the data processing task representing processing of data through the neural network layer in the first machine learning model, and the second duration representing the duration required to process the second amount of data through the neural network layer in the first machine learning model.

[0135] In an embodiment of the present application, the first communication node in the communication node cluster used to deploy the first neural network module may also have a second startup duration between determining the data processing task to be performed by the neural network layer in the first neural network module and starting to execute the data processing task. The second startup duration can also be understood as the processing startup duration; then, after obtaining the first data, the first communication node can also obtain the second startup duration and the second duration in order to determine the value of r.

[0136] Among them, the above-mentioned "data processing task" represents the task of processing any one of the r sub-data through the neural network layer in the first neural network module included in the first machine learning model.

[0137] The second duration represents the duration required for the neural network layer in the first neural network module included in the first machine learning model to process the second data volume. The concept of "second data volume" is similar to that of "first data volume" and can also be understood as a unit data volume used to indicate the size of the data. Please refer to the above description for understanding, and will not be repeated here. The value of the second data volume can be the same as or different from the value of the first data volume, and this is not limited in the embodiments of the present application.

[0138] Optionally, if the first machine learning model is expressed as an MoE model and the first neural network module is an MoE module, then the "data processing task" represents the task of processing any one of the r sub-data through the expert neural network layer in the MoE module; the second duration represents the duration required for the expert neural network layer in the MoE module included in the MoE model to process the second amount of data. In the embodiment of the present application, the first machine learning model is determined to be an MoE model, which provides a specific application scenario of the method provided by the present application, improves the degree of integration of the present solution with the specific application scenario, and the MoE model has a high degree of matching with the method provided by the present application, which is conducive to greatly shortening the time consumed by the MoE model in the data processing process.

[0139] Exemplarily, the communication node cluster used to deploy the first neural network module includes multiple communication nodes, and the second parameters may be pre-deployed on the multiple communication nodes. Optionally, the second parameters deployed in the multiple communication nodes included in the communication node cluster are the same. For example, the second startup duration and the second duration deployed in the multiple communication nodes included in the communication node cluster are the same.

[0140] In an embodiment of the present application, since the number of neural network layers deployed on different communication nodes in the communication node cluster used to deploy the first neural network module is generally the same, and the data processing load on different communication nodes is also likely to be similar, it is in line with the actual situation for different communication nodes to deploy the same second parameter. The adoption of the aforementioned scheme is conducive to reducing the difficulty of the process of "determining the second parameter deployed on multiple communication nodes"; since the second parameters on different communication nodes will be used in the process of determining the value of r, it is also conducive to reducing the complexity of the process of determining the value of r.

[0141] An embodiment of the present application also provides a method for obtaining a second parameter. The second parameter is obtained by statistically analyzing multiple data processing tasks performed by the neural network layer in the first neural network module included in the first machine learning model. The multiple data processing tasks involve processing data of different amounts, that is, the size of the data to be processed in at least two data processing tasks is different among the multiple data processing tasks.

[0142] Exemplarily, the above-mentioned multiple data processing tasks may include executing 20 data processing tasks, and the 20 data processing tasks include processing 1.5MB of data 2 times, 2MB of data 3 times, 3MB of data 5 times, 4MB of data 5 times, 7MB of data 3 times, and 9MB of data 2 times. After statistics are taken on the duration of the above-mentioned 20 data processing tasks, the second startup duration and the second duration are obtained. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0143] Optionally, among the multiple data processing tasks mentioned above, there are at least two data processing tasks in which the size of the data to be processed is the same. The multiple time lengths obtained when processing data of the same data size can be averaged to obtain an average value, and statistics can be performed based on multiple average values ​​corresponding to data of different data sizes to obtain a second parameter.

[0144] For example, the multiple data processing tasks may include executing 20 data processing tasks, wherein the 20 data processing tasks include processing 1.5MB of data 2 times, 2MB of data 3 times, 3MB of data 5 times, 4MB of data 5 times, 7MB of data 5 times, and 1.5MB of data 2 times. The data is processed 3 times and 9MB of data is processed 2 times. When the duration of the aforementioned 20 data processing tasks is counted, the 2 durations of processing 1.5MB of data twice can be averaged to obtain duration 1, the 3 durations of processing 2MB of data 3 times can be averaged to obtain duration 2, the 5 durations of processing 3MB of data 5 times can be averaged to obtain duration 3, the 5 durations of processing 4MB of data 5 times can be averaged to obtain duration 4, the 3 durations of processing 7MB of data 3 times can be averaged to obtain duration 5, and the 2 durations of processing 9MB of data 2 times can be averaged to obtain duration 6. Then, according to the duration 1 corresponding to processing 1.5MB of data, the duration 2 corresponding to processing 2MB of data, the duration 3 corresponding to processing 3MB of data, the duration 4 corresponding to processing 4MB of data, the duration 5 corresponding to processing 7MB of data, and the duration 6 corresponding to processing 9MB of data, a second parameter is generated. It should be noted that the examples here are only for the convenience of understanding the method of obtaining the "second parameter" and are not used to limit this solution.

[0145] Exemplarily, in the statistical process of the second parameter, the above-mentioned multiple data processing tasks can be executed by the same communication node (hereinafter referred to as the "third communication node" for the convenience of description) in the communication node cluster used to deploy the first neural network module, and the third communication node determines the second parameter based on the total duration of each data processing task in the multiple data processing tasks. In the embodiment of the present application, since the data processing load on different communication nodes in the communication node cluster where the first neural network module is deployed is likely to be similar, it is in line with the actual situation to obtain the second parameter after statistically analyzing the multiple data processing tasks executed on the same communication node, and executing the above-mentioned multiple data processing tasks on the same communication node avoids data communication between different communication nodes, which is conducive to shortening the time consumed by the process of "determining the second parameter".

[0146] Alternatively, the above-mentioned multiple data processing tasks can also be executed by different communication nodes in the aforementioned communication node cluster, and the total execution time of each data processing task will be aggregated to the same third communication node, and the third communication node will determine the second parameter based on the total execution time of each data processing task in the multiple data processing tasks.

[0147] Here, taking the second parameter including the second startup duration and the second duration as an example, in this application, the total duration of executing a data processing task by any communication node in the communication node cluster used to deploy the first neural network module can be modeled as the following formula: e (x) = α e +β e ×x; (2)

[0148] Among them, t e (x) represents the total time it takes for any communication node in the communication node cluster to execute a data processing task, α e represents the duration between determining the aforementioned data processing task and starting to execute the data processing task (i.e., the second startup duration), β e represents the time required to process the second amount of data (i.e., the second time), x represents the amount of data to be processed, which is the product of x and the second amount of data. After obtaining the total time consumed by each data processing task in the above multiple data processing tasks, α can be fitted. e and β e The value of the second startup duration and the second duration are obtained. It should be understood that the example in formula (2) is only for the convenience of understanding this solution and is not used to limit this solution.

[0149] In an embodiment of the present application, a scheme for determining the second parameter is provided, which reduces the difficulty of implementing the method provided in the present application, and the second parameter is obtained after statistics are performed on multiple data processing tasks. In multiple data processing tasks, data of different data amounts are processed, which is conducive to making the determined second parameter compatible with data processing tasks of various data amounts, and is conducive to achieving the optimal value of r under data processing tasks of various data amounts.

[0150] Optionally, the second parameter deployed in the above-mentioned multiple communication nodes is adjustable; illustratively, the second startup duration and the second duration deployed in the above-mentioned multiple communication nodes are adjustable, that is, during the operation of the above-mentioned multiple communication nodes, at least one communication node in the communication node cluster for deploying the second neural network module can re-execute multiple data processing tasks, and perform statistics on the aforementioned re-executed multiple data processing tasks to obtain an updated second startup duration and an updated second duration; the updated second startup duration and the updated second duration are redeployed on the multiple communication nodes included in the above-mentioned communication node cluster.

[0151] It should be noted that the "second parameter" can be updated together with the "first parameter". The update scenario of the "first parameter" has been described in step 503. The update scenario of the "second parameter" can be understood by reference and will not be repeated here.

[0152] 504. Determine a value of r according to the first parameter and / or the second parameter.

[0153] In an embodiment of the present application, the first communication node can determine the value of r based on the first parameter and / or the second parameter, as well as the data volume of the first data, so that the first data is divided into r sub-data; that is, the process of processing the entire first data through the first neural network module may include: the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data.

[0154] Exemplarily, the entire first data processing process may be represented as follows: as well as Among them, D1, D2, ..., D r Represents r sub-data sending tasks, E1, E2, ..., E r Represents the processing tasks of r sub-data, C1, C2, ..., C r The task of sending the processing results of r sub-data sets is described. It should be noted that since the "sending task of r sub-data sets" and the "sending task of the processing results of r sub-data sets" are performed within the same communication node set, there is no overlap between the "time occupied by the sending task of r sub-data sets" and the "time occupied by the sending task of the processing results of r sub-data sets." Optionally, the present application stipulates that the sending task of the processing results of r sub-data sets must be started after the sending task of r sub-data sets is completed.

[0155] Exemplarily, the first communication node can solve the objective function based on the first parameter and / or the second parameter to determine the value of r. The objective function represents the total time consumed to obtain the processing results of the r sub-data, that is, the objective function can be obtained by modeling the total time consumed by executing the sending task of r sub-data, the processing task of r sub-data, and the sending task of the processing results of r sub-data. The solution goal of the objective function can be to minimize the value of the objective function. The constraints of the objective function are related to the time occupied by the sending task of the sub-data and the time occupied by the processing task of the sub-data.

[0156] In an embodiment of the present application, the objective function represents the total time consumed to obtain the processing results of r sub-data, and the constraints of the objective function are related to the time occupied by the sub-data sending task and the time occupied by the sub-data processing task. That is, the objective function can accurately reflect the total time consumed for the processing results of r sub-data, and convert the determination process of r into the solution process of the objective function, thereby improving the precision of the determination process of r, which is conducive to obtaining the optimal value of r, so as to further shorten the total time consumed by the entire processing process of the first data.

[0157] Specifically, when the first parameter includes the first startup time and the first time, and the second parameter includes the second startup time and the second time, optionally, step 504 may include: the first communication node models the total time consumed by executing the sending task of r sub-data, the processing task of r sub-data, and the sending task of the processing results of r sub-data based on the first startup time, the first time, the second startup time, the second time and the data volume of the first data, and takes minimizing the total time consumed by the sending task of r sub-data, the processing task of r sub-data, and the sending task of the processing results of r sub-data as the solution goal, and obtains the optimal solution of r.

[0158] To further understand this solution, the following is an example of a formula used to model the total time consumed by the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data:

[0159] D1,D2,…,D r Sending task D of any sub-data i The execution time required is E1,E2,…,E r The processing task E of any sub-data i The execution time required is C1,C2,…,C r Sending task C of processing result of any sub-data i The execution time required is and Respectively represent D i ,E i and C i The time when execution started, and It can be expressed as the following formula:

[0160] Referring to the above formulas (3) to (6), we can see that if the start time of the first sub-data sending task (i.e., the first data sending task) D1 is regarded as 0, then when 2≤i≤r, the sending task D1 of any sub-data i The start execution time is the start execution time of the previous sub-data sending task Add the execution time of the previous sub-data sending task Since the second communication node can process the i-th sub-data using the neural network layer deployed thereon only after the i-th sub-data is sent to the second communication node and the second communication node completes processing of the i-1-th sub-data, when i=1, the start time of the first sub-data processing task (i.e., the first data processing task) E1 is That is, the start time of the first sub-data sending task D1 plus the execution time of the first sub-data sending task. When 2≤i≤r, the processing task E of any sub-data i The start execution time is the later of the two times, and the above two times include the end time of the processing task of the i-1th sub-data (that is, the end time of the i-1th sub-data in the formula ) and the end time of the sending task of the i-th sub-data (that is, the end time of the ).

[0161] Since the task of sending the processing result of the first sub-data needs to be started after the task of sending the r-th sub-data is completed, and the second communication node needs to start the task of sending the processing result of the i-th sub-data after the task of processing the i-th sub-data is completed, when i=1, the start time of the task C1 of sending the processing result of the first sub-data is the end time of the task of sending the r-th sub-data (that is, the end time of the task C2 in the formula). ) and the end time of the processing task of the first sub-data (that is, the end time of the processing task of the first sub-data in the formula ) At the later of these two times, when 2≤i≤r, the task C of sending the processing result of any sub-data i The start execution time is the end time of the processing task of the i-th sub-data (that is, the ) and the end time of the task of sending the processing result of the i-1th sub-data The later of these two times.

[0162] t moe represents the total time consumed by executing the tasks of sending r sub-data, processing r sub-data, and sending the processing results of r sub-data. When the value of r is 1, it means that the first data is not split, then t moe Equal to the execution time of the entire first data sending task (that is, td ), the execution time of the entire first data processing task (that is, the t e ) and the execution time of the task of sending the processing result of the entire first data (that is, t in the formula c ). When r≥2, it means that the first data is segmented. If the start time of the first sub-data sending task (that is, the first data sending task) D1 is regarded as 0, t moe Equal to the end time of the task of sending the processing result of the rth sub-data (that is, the end time of the task in the formula ).

[0163] The above formula (1) is the execution time of the sub-data sending task And the execution time of the task of sending the processing results of the sub-data For modeling, and It can be further expressed by the above formula (1); the above formula (2) is the execution time of the sub-data processing task For modeling, It can be further expressed by the above formula (2).

[0164] Because the multiple neural network layers included in the first neural network module are known, the relationship between the data volume of the processing result of each sub-data and the data volume of each sub-data can be known in advance. For example, the data volume of the processing result of each sub-data is a preset multiple of the data volume of each sub-data. That is, the data volume of the processing result of the first data can be obtained based on the data volume of the first data, and then the data volume of the processing result of each sub-data in the r sub-data can be obtained. Exemplarily, the value of the preset multiple can be 0.7, 0.8, 1, 1.2, 3, or other values, etc., and should be determined in combination with the actual application scenario and is not limited here.

[0165] Optionally, if the first neural network module adopts an MoE module, the value of the preset multiple can be 1, that is, the data amount of each sub-data and the data amount of the processing result of each sub-data are considered to be the same.

[0166] During the transmission of each of the r sub-data, each sub-data needs to be encapsulated into a data packet. Specifically, the data packet encapsulated from each sub-data is larger than the sub-data by a preset data amount, which represents the size of the data packet header. Furthermore, based on the data amount of each sub-data, the data amount of the data transmission task corresponding to each sub-data can be obtained. Correspondingly, during the transmission of the processing result of each sub-data, the processing result of each sub-data needs to be encapsulated into a data packet. Specifically, the data packet encapsulated from each sub-data is larger than the processing result of each sub-data by a preset data amount, which represents the size of the data packet header. Furthermore, based on the data amount of the processing result of each sub-data, the data amount of the data transmission task corresponding to the processing result of each sub-data can be obtained.

[0167] Therefore, after determining the first startup duration α a , first duration β a , Second startup duration α e , the second duration β e And the amount of data of the first data, it can be t moe The minimum is taken as the solution goal to determine the optimal solution of r. It should be noted that the examples in formulas (3) to (6) are only for the convenience of understanding this solution and proving the feasibility of this solution, and are not used to limit this solution.

[0168] In an embodiment of the present application, the first communication node can perform step 504 in various ways. In one implementation, the first communication node can obtain an objective function that models the total duration of the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data, and minimize the total duration of the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data as the optimization goal of the aforementioned objective function, and solve for r. It should be noted that the dependencies between the multiple items included in the aforementioned objective function can be understood by referring to the aforementioned formulas (1) to (6).

[0169] In another implementation, when the data volume of the sub-data and the data volume of the processing result of the sub-data are considered to be the same, the solution process of r can be divided into three situations. In one situation, the time occupied by the sending task of a sub-data (that is, the time occupied by the sending task of the processing result of a sub-data) is greater than or equal to the time occupied by the processing task of a sub-data.

[0170] For a more intuitive understanding of this solution, please refer to Figure 9, which is a schematic diagram of the relationship between the time taken by the sending task of r sub-data and the sending task of the processing results of r sub-data and the time taken by the processing task of r sub-data, provided in an embodiment of the present application. In Figure 9, taking the value of r as 4 as an example, D1, D2, D3, and D4 represent the time taken by the sending task of the four sub-data, E1, E2, E3, and E4 represent the time taken by the processing task of the four sub-data, and C1, C2, C3, and C4 represent the time taken by the sending task of the processing results of the four sub-data. The time taken by the sending task of each sub-data is greater than the time taken by the processing task of each sub-data. As shown in Figure 9, let i be any value from 1 to 4. The time taken by the sending task of the i+1th sub-data can completely cover the time taken by the processing task of the ith sub-data, and the sending task of the processing result of the 1st sub-data can completely cover the time taken by the processing task of the 4th sub-data. It should be understood that the example in Figure 9 is only for the convenience of understanding this solution and is not used to limit this solution.

[0171] In another situation, the time occupied by the task of sending a sub-data (that is, the time occupied by the task of sending the processing result of a sub-data) is less than the time occupied by the task of processing a sub-data, and the second time is not a subset of the first time, that is, the second time cannot completely cover the first time; wherein, the first time includes the time occupied by the task of sending r sub-data and the time occupied by the task of sending the processing result of r sub-data, and the second time includes the time occupied by the task of processing r sub-data.

[0172] For a more intuitive understanding of this solution, please refer to Figure 10. Figure 10 is another schematic diagram illustrating the relationship between the time taken by the task of sending r sub-data and the task of sending the processing results of r sub-data, and the time taken by the task of processing r sub-data, provided in an embodiment of the present application. In Figure 10, taking the value of r as 4 as an example, the time taken by the task of sending each sub-data is less than the time taken by the task of processing each sub-data, there is overlap between the second time and the first time, and the second time cannot completely cover the first time. It should be understood that the example in Figure 10 is only for facilitating understanding of this solution and is not intended to limit this solution.

[0173] In another case, the time occupied by a sub-data sending task is less than the time occupied by a sub-data processing task, and the second time is a subset of the first time, that is, the second time can completely cover the first time.

[0174] For a more intuitive understanding of this solution, please refer to Figure 11. Figure 11 is another schematic diagram illustrating the relationship between the time taken to send tasks for r sub-data and the time taken to send tasks for processing the results of r sub-data, as provided in an embodiment of the present application. In Figure 11, taking the value of r as 4 as an example, the time taken to send each sub-data task is less than the time taken to process each sub-data task, and the second time completely covers the first time. It should be understood that the example in Figure 11 is merely for facilitating understanding of this solution and is not intended to limit this solution.

[0175] Correspondingly, the first communication node can deploy a first objective function, a second objective function, and a third objective function corresponding to the above three scenarios, respectively. The first objective function, the second objective function, and the third objective function all represent the total time consumed to obtain the processing results of r sub-data. That is, the first objective function, the second objective function, and the third objective function all represent the total time consumed to transmit the r sub-data, process the r sub-data, and transmit the processing results of the r sub-data.

[0176] The difference is that the constraint condition of the first objective function includes that the time taken by the task of sending a sub-data is greater than or equal to the time taken by the task of processing a sub-data. The constraint condition of the second objective function includes that the time taken by the task of sending a sub-data is less than the time taken by the task of processing a sub-data, and the second time is not a subset of the first time. The constraint condition of the third objective function includes that the time taken by the task of sending a sub-data is less than the time taken by the task of processing a sub-data, and the second time is a subset of the first time.

[0177] To further understand the present solution, the following discloses an example of the formulas for the first objective function and the constraints of the first objective function, the second objective function and the constraints of the second objective function, and the third objective function and the constraints of the third objective function. It should be noted that the formulas disclosed below are derived formulas.

[0178] The first objective function and the constraints of the first objective function are as follows: Minimize: f1(r) = 2α a r+2n d β a ;(7)

[0179] Among them, f1(r) represents the first objective function. Under the situation corresponding to the first objective function, the total duration of the task of sending r sub-data, the task of processing r sub-data, and the task of sending the processing results of r sub-data is the duration occupied by the task of sending r sub-data and the task of sending the processing results of r sub-data (which can be understood in conjunction with Figure 9), which is equivalent to twice the duration occupied by the task of sending r sub-data. The optimization goal of f1(r) is to take the minimum value of f1(r). Indicates that the time taken by a sub-data sending task is greater than or equal to the time taken by a sub-data processing task, n d represents the ratio between the total data volume of the r data packets corresponding to the r sub-data and the first data volume (hereinafter referred to as the "first ratio" for the convenience of description), which can also be understood as the ratio between the total data volume of the r data packets corresponding to the processing results of the r sub-data and the first data volume; n e represents the ratio between the amount of the first data and the amount of the second data (hereinafter referred to as the “second ratio” for the convenience of description), r represents the first data being divided into r sub-data, α a Represents the first startup duration, β a Represents the first duration, α e Represents the second startup duration, β e Represents the second duration. It should be understood that the examples in formulas (7) and (8) are only for facilitating the understanding of this solution and proving the feasibility of this solution, and are not used to limit this solution.

[0180] The second objective function and the constraints of the second objective function are as follows: r≥2; (12)

[0181] Wherein, f2(r) represents the second objective function. Under the situation corresponding to the second objective function, the total duration of the tasks of sending r sub-data, processing r sub-data, and sending the processing results of r sub-data is the duration occupied by the tasks of sending one sub-data and sending the processing results of one sub-data plus the duration occupied by the tasks of processing r sub-data (which can be understood in conjunction with FIG10 ), which is equivalent to the duration occupied by the tasks of sending two sub-data (that is, ) plus the time taken by the processing task of r sub-data (that is, α e r+β e n e ), the optimization goal of f2(r) is to take the minimum value of f2(r). The time taken by the sending task of a sub-data is less than the time taken by the processing task of a sub-data. It means that the second time is not a subset of the first time. It should be noted that the meaning of each letter in formula (9) to formula (12) can be understood by referring to the above description of formula (1) to formula (8), which will not be repeated here.

[0182] The third objective function and the constraints of the third objective function are as follows: Minimize: f3(r) = 2α a r+2n d β a ; (13) r≥2; (16)

[0183] Where f3(r) represents the third objective function. The meaning of the third objective function (which can be understood in conjunction with Figure 11) is similar to that of the first objective function. Please refer to the above description for understanding. The meaning of formula (14) is the same as that of formula (10). Please refer to the above description for understanding. It represents that the second time is a subset of the first time. It should be noted that the meaning of each letter in formula (13) to formula (16) can be understood by referring to the above description of formula (1) to formula (12), and will not be repeated here.

[0184] Exemplarily, step 504 may include: after the first communication node determines the first startup duration, the first duration, the second duration, the second duration and the data volume of the first data, it can determine the first ratio between the total data volume of the r data packets corresponding to the r sub-data and the first data volume, as well as the second ratio between the data volume of the first data and the second data volume based on the data volume of the first data, and then solve the first objective function, the second objective function and / or the third objective function based on the first startup duration, the first duration, the second duration, the second duration, the first ratio and the second ratio to determine the value of r.

[0185] Specifically, in one implementation, the first communication node may incorporate the first startup duration, the first duration, the second duration, the second duration, the first ratio, and the second ratio into the first objective function and the constraints of the first objective function, into the second objective function and the constraints of the second objective function, and into the third objective function and the constraints of the third objective function, and solve the first objective function, the second objective function, and the third objective function, respectively, to obtain the r value corresponding to the first objective function, the r value corresponding to the second objective function, and the r value corresponding to the third objective function, respectively. The first communication node may obtain the r value corresponding to the objective function with the smallest value among the first objective function, the second objective function, and the third objective function, and determine the aforementioned obtained r value as the value of r.

[0186] In another implementation, the first communication node can select an objective function that matches the aforementioned "first startup duration, first duration, second duration and second duration" from the first objective function, the second objective function and the third objective function based on the first startup duration, the first duration, the second duration and the second duration, and bring the first startup duration, the first duration, the second duration, the second duration, the first ratio and the second ratio into the aforementioned selected objective function and the constraints of the objective function, solve the aforementioned selected objective function, and obtain a value of r.

[0187] In an embodiment of the present application, the entire first data is divided into r sub-data, and the r sub-data are processed by the neural network layer in the machine learning model respectively, that is, the processing process of the entire first data is divided into the sending task of r sub-data, the processing task of r sub-data, and the sending task of the processing results of r sub-data. The time occupied by the sending task of r sub-data and the time occupied by the sending task of the processing results of r sub-data can overlap with the time occupied by the processing task of r sub-data, which is conducive to shortening the time consumed by a data processing process of the first data; in addition, the first parameter indicates the network condition of the communication network used to send the sub-data, and the second parameter indicates the load condition of the communication node used to process the sub-data. The value of r is determined based on the network condition and / or the load condition of the data processing of the communication node, which is conducive to obtaining an r value that conforms to the actual situation, so as to further shorten the time consumed by a data processing process of the entire first data.

[0188] The value of r is determined based on the first startup time, the first time, the second startup time and the second time. Relying on the first startup time, the first time, the second startup time and the second time can more accurately reflect the time consumed by each stage in the one-time data processing process of the entire first data, which is conducive to obtaining the optimal r to further shorten the time consumed by the one-time data processing process of the entire first data.

[0189] The total time consumed to obtain the processing results of r sub-data is represented by the first objective function, the second objective function and the third objective function. The constraints of the first objective function, the second objective function and the third objective function are all different, that is, the process of determining the value of r is divided into three situations, which are described by the first objective function and the constraints of the first objective function, the second objective function and the constraints of the second objective function, and the third objective function and the constraints of the third objective function respectively. Determining the value of r at a finer level is conducive to obtaining a better solution for r, thereby helping to further shorten the time consumed by the entire first data processing process.

[0190] 505. Obtain a processing result of the first data.

[0191] In an embodiment of the present application, after determining the value of r, the first communication node can divide the first data into r sub-data, and the first communication node can send r sub-data to each second communication node (that is, execute the sending task of r sub-data), and each second communication node processes each sub-data in the r sub-data (that is, executes the processing task of r sub-data), and each second communication node also sends the processing result of each sub-data in the r sub-data to the first communication node (that is, executes the sending task of the processing result of r sub-data). The relationship between the time occupied by the sending task of r sub-data, the time occupied by the processing task of r sub-data, and the time occupied by the sending task of the processing result of r sub-data has been shown in the description of the above steps and will not be repeated here.

[0192] Optionally, when the first communication node sends sub-data to the second communication node, an all-to-all communication mode may be adopted. Correspondingly, when the second communication node sends the processing result of the sub-data to the first communication node, an all-to-all communication mode may be adopted.

[0193] Optionally, if the first neural network module is an MoE module, after the first communication node obtains the first data, the gate neural network layer in the MoE module determines which neural network layer deployed on which second communication node or nodes will process the first data, that is, each of the r sub-data needs to be sent to each second communication node determined by the gate neural network layer.

[0194] After obtaining the processing results of r sub-data, the first communication node can fuse the processing results of the r sub-data to obtain the processing result of the first data. The processing result of the first data is used to obtain the processing result of the second data, and the processing result of the second data is the processing result obtained after the second data is processed by the entire first machine learning model.

[0195] In an embodiment of the present application, the first machine learning model may include one or more first neural network modules. Optionally, the first machine learning model may also include a second neural network module. In one case, if the processing result of the first data obtained in step 505 is the processing result generated by the last neural network module in the first machine learning model, the processing result of the first data may also be determined as the processing result of the second data.

[0196] In another case, after obtaining the processing result of the first data, the first communication node can send the processing result of the first data to the communication node of other neural network modules deployed with the first machine learning model, so as to continue to process the processing result of the first data through other neural network modules of the first machine learning model, and thus obtain the processing result of the second data output by the entire first machine learning model.

[0197] It should be noted that if the first machine learning model includes multiple first neural network modules, at least one of the multiple first neural network modules can use the method shown in steps 501 to 505 to process the data. The specific settings can be flexibly made based on actual conditions and are not limited here.

[0198] 2. Training Phase

[0199] In the embodiment of the present application, the training phase describes the process of how the training device 310 uses the training data in the database 220 to train the first machine learning model / rule 301. Specifically, please refer to Figure 12, which is a flow chart of a data processing method provided in the embodiment of the present application. The data processing method provided in the embodiment of the present application may include:

[0200] 1201. Obtain first data, wherein the first data is determined based on second data, the second data is data input into a first machine learning model, and the first data is data that needs to be processed by a first neural network module in the first machine learning model.

[0201] 1202. Obtain a first parameter, where the first parameter indicates a network condition of a communication network used to send sub-data.

[0202] 1203. Obtain a second parameter, where the second parameter indicates a load condition of a communication node for processing sub-data.

[0203] 1204. Determine a value of r according to the first parameter and / or the second parameter.

[0204] 1205. Obtain a processing result of the first data, where the processing result of the first data is used to obtain a processing result of the second data.

[0205] In an embodiment of the present application, during the process of the training device training the first machine learning model, the training device can obtain a training data set, and the training data set includes multiple training data. One training data may include second data and expected information corresponding to the second data. The expected information corresponding to the second data can also be understood as the correct information corresponding to the second data. The expected information corresponding to the second data represents the information that the first machine learning model is expected to output after processing the second data using the first machine learning model.

[0206] During the training phase of the first machine learning model, the "second data" can be understood as training data used to train the first machine learning model.

[0207] During the process of training the first machine learning model by the training device, the first neural network module in the first machine learning model is also deployed to a communication node cluster. When the first data reaches the first neural network module in the first machine learning model, the first data can be obtained by a certain communication node in the training device (for ease of description, perhaps referred to as the "first communication node"), and the first communication node in the training device can use steps 1201 to 1205 to obtain the processing result of the first data. The specific implementation of steps 1201 to 1205 can refer to the description of the specific implementation of steps 501 to 505 in the embodiment corresponding to Figure 5. The meaning of the terms in steps 1201 to 1205 can refer to the description in the embodiment corresponding to Figure 5, and they are not repeated here.

[0208] 1206. Train the first machine learning model based on the processing result of the second data, the expected information corresponding to the second data, and the first loss function, where the first loss function indicates the similarity between the processing result of the second data and the expected information.

[0209] In an embodiment of the present application, the purpose of training the first machine learning model using the first loss function includes improving the similarity between the processing results of the second data and the expected information. In other words, the purpose of training the first machine learning model using the first loss function includes making the processing results of the second data close to the expected information of the second data.

[0210] After obtaining the processing result of the second data, a communication node in the training device can generate a function value of the first loss function based on the processing result of the second data and the expected information corresponding to the second data, perform a gradient derivative on the function value of the first loss function, and use a backpropagation algorithm to update the parameters of the first machine learning model to complete a training of the first machine learning model. The training device repeats steps 1201 to 1205 until a convergence condition is met, thereby obtaining a trained first machine learning model.

[0211] The convergence condition may be that the convergence condition of the first loss function is satisfied, or that the number of executions of steps 1201 to 1205 reaches a preset number.

[0212] In the process of updating the parameters of the first machine learning model using the backpropagation algorithm, since the first machine learning model includes one or more first neural network modules, the multiple neural network layers included in each first neural network module are deployed on multiple communication nodes included in the same communication node cluster. In the process of updating the parameters of the first neural network module in the first machine learning model using the backpropagation algorithm, the method shown in steps 501 to 505 will also be used, with the difference that the second data is replaced by "the function value of the first loss function", and the first data is replaced by "the value when the gradient derivative of the function value of the first loss function reaches the first neural network module using the backpropagation algorithm". That is, after the gradient derivative of the function value of the first loss function is performed, it will be backpropagated in the first machine learning model to continuously update the parameters of the first machine learning model, and the first data will be replaced by "the value when it reaches the first neural network module during the aforementioned backpropagation process".

[0213] In an embodiment of the present application, the data processing method provided in the present application can be used in both the inference stage and the training stage of the first machine learning model, which expands the application scenarios of the present solution, shortens the time taken for data processing in the inference stage and the training stage of the first machine learning model, and is conducive to improving the efficiency of data processing in the inference stage and the training stage of the first machine learning model.

[0214] On the basis of the embodiments corresponding to Figures 1 to 12, in order to better implement the above-mentioned scheme of the embodiment of the present application, the following also provides related equipment for implementing the above-mentioned scheme. Please refer to Figure 13 in detail. Figure 13 is a structural diagram of the execution device provided by the embodiment of the present application. The data processing device 1300 includes: an acquisition module 1301 for acquiring first data; a determination module 1302 for determining the value of r based on the first parameter and / or the second parameter, r is used to indicate that the first data is divided into r sub-data, and the r sub-data are respectively processed by the neural network layer in the machine learning model, r is an integer greater than or equal to 1, the first parameter indicates the network status of the communication network used to send the sub-data, and the second parameter indicates the load status of the communication node used to process the sub-data.

[0215] In one possible design, the first parameter includes a first startup duration and a first duration corresponding to the communication network, the first startup duration indicates the duration between determining a data sending task and executing the data sending task, and the first duration indicates the duration required to send a first amount of data; the second parameter includes a second startup duration and a second duration, the second startup duration indicates the duration between determining a data processing task and executing the data processing task, the data processing task represents processing data through a neural network layer in a machine learning model, and the second duration indicates the duration required to process a second amount of data through the neural network layer in the machine learning model.

[0216] In one possible design, the determination module 1302 is specifically used to solve the objective function based on the first parameter and / or the second parameter to determine the value of r, where the objective function represents the total time consumed to obtain the processing results of r sub-data, and the constraints of the objective function are related to the time occupied by the sub-data sending task and the time occupied by the sub-data processing task.

[0217] In one possible design, the objective function includes one or more of a first objective function, a second objective function, or a third objective function; wherein, the constraint conditions of the first objective function include that the time occupied by a sub-data sending task is greater than or equal to the time occupied by a sub-data processing task; the constraint conditions of the second objective function include that the time occupied by a sub-data sending task is less than the time occupied by a sub-data processing task, and the first time is not a subset of the second time, the first time includes the time occupied by the processing tasks of r sub-data, and the second time includes the time occupied by the sending tasks of r sub-data and the time occupied by the sending tasks of the processing results of r sub-data; the constraint conditions of the third objective function include that the time occupied by the sending task of a sub-data is less than the time occupied by the processing task of a sub-data, and the first time is a subset of the second time.

[0218] In one possible design, the machine learning model includes at least one neural network module, and the multiple neural network layers included in the first neural network module in the at least one neural network module are deployed in the same communication node cluster, and the first parameters and second parameters of the multiple communication nodes included in the communication node cluster are the same.

[0219] In one possible design, the first duration represents the duration required to send the first amount of data using an all-to-all communication method.

[0220] In one possible design, the machine learning model is a hybrid expert MoE model, and the second duration represents the duration required to process the second amount of data using an expert neural network layer in the MoE model.

[0221] In a possible design, the first parameter is obtained by counting multiple data sending tasks executed using an all-to-all communication method, and the multiple data sending tasks include data sending tasks with different data amounts.

[0222] In one possible design, the second parameter is obtained by statistically analyzing multiple data processing tasks performed by a neural network layer in a machine learning model, where different amounts of data are processed in the multiple data processing tasks.

[0223] In one possible design, the r sub-data are respectively processed by the neural network layer in the machine learning model, including: in the inference phase of the machine learning model, the r sub-data are respectively processed by the neural network layer in the machine learning model; or, in the training phase of the machine learning model, the r sub-data are respectively processed by the neural network layer in the machine learning model.

[0224] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 1300 are based on the same concept as the various method embodiments corresponding to Figures 2 to 12 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0225] An embodiment of the present application further provides a communication node, comprising a processor, the processor being configured to execute the data processing method executed by the first communication node in the embodiments corresponding to Figures 2 to 12. The specific manner in which the processor executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 2 to 12 of the present application, and the technical effects achieved are the same as those of the various method embodiments corresponding to Figures 2 to 12 of the present application. For details, please refer to the description of the method embodiments shown above in the present application, and will not be repeated here.

[0226] The embodiment of the present application also provides an electronic device, which may include one or more communication nodes. For example, the communication node for deploying the first neural network module may be deployed in the aforementioned electronic device. The electronic device may be represented as a server, or as a server cluster composed of multiple servers. Please refer to Figure 14, which is a structural diagram of a communication node provided in an embodiment of the present application. Specifically, the electronic device 1400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1422 (for example, one or more processors) and memory 1432, one or more application programs 1442 or data 1444 storage media 1430 (for example, one or more mass storage devices). Among them, the memory 1432 and the storage medium 1430 may be temporary storage or persistent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 1422 may be configured to communicate with the storage medium 1430 and execute a series of instruction operations in the storage medium 1430 on the electronic device 1400 .

[0227] The electronic device 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0228] In the embodiment of the present application, the central processing unit 1422 is configured to execute the data processing method executed by the first communication node in the embodiments corresponding to Figures 2 to 12. Specifically, the specific manner in which the central processing unit 1422 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 2 to 12 in the present application, and the technical effects brought about are the same as the various method embodiments corresponding to Figures 2 to 12 in the present application. For specific details, please refer to the description of the method embodiments shown above in the present application, and will not be repeated here.

[0229] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for signal processing. When the program is run on a computer, the computer executes the steps performed by the first communication node in the method described in the embodiments shown in Figures 2 to 12 above.

[0230] An embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the steps performed by the first communication node in the method described in the embodiments shown in Figures 2 to 12 above.

[0231] The communication node or data processing device provided in the embodiments of the present application may specifically be a chip, which includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip performs the data processing method described in the embodiments shown in Figures 2 to 12 above. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0232] Specifically, see Figure 15 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 150. NPU 150 is mounted on the host CPU (host CPU) as a coprocessor and is assigned tasks by the host CPU. The core of the NPU is arithmetic circuit 1503, which is controlled by controller 1504 to extract matrix data from memory and perform multiplication operations.

[0233] In some implementations, arithmetic circuit 1503 includes multiple processing units (PEs). In some implementations, arithmetic circuit 1503 is a two-dimensional systolic array. Arithmetic circuit 1503 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, arithmetic circuit 1503 is a general-purpose matrix processor.

[0234] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1502 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1501 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1508.

[0235] Unified memory 1506 is used to store input and output data. Weight data is directly transferred to weight memory 1502 through the Direct Memory Access Controller (DMAC) 1505. Input data is also transferred to unified memory 1506 through the DMAC.

[0236] BIU stands for Bus Interface Unit 1510 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1509 .

[0237] The bus interface unit 1510 (BIU) is used for the instruction fetch memory 1509 to obtain instructions from the external memory, and is also used for the storage unit access controller 1505 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0238] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1506 or transfer weight data to the weight memory 1502 or transfer input data to the input memory 1501.

[0239] The vector calculation unit 1507 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0240] In some implementations, the vector calculation unit 1507 can store the processed output vector to the unified memory 1506. For example, the vector calculation unit 1507 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1503, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1507 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1503, such as for use in subsequent layers in a neural network.

[0241] An instruction fetch buffer 1509 connected to the controller 1504 is used to store instructions used by the controller 1504;

[0242] Unified memory 1506, input memory 1501, weight memory 1502, and instruction fetch memory 1509 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0243] In which, the operations of each layer in the first machine learning model shown in the above embodiments can be performed by the operation circuit 1503 or the vector calculation unit 1507.

[0244] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above method.

[0245] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0246] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0247] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0248] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A data processing method, characterized in that: The method comprises: Acquire first data; According to the first parameter and / or the second parameter, the value of r is determined, where r is used to indicate that the first data is divided into r sub-data, and the r sub-data are respectively processed by the neural network layer in the machine learning model, and r is an integer greater than or equal to 1. The first parameter indicates the network condition of the communication network used to send the sub-data, and the second parameter indicates the load condition of the communication node used to process the sub-data.

2. The method according to claim 1, characterized in that The first parameter includes a first startup duration and a first duration corresponding to the communication network, the first startup duration indicating the duration between determining a data sending task and executing the data sending task, and the first duration indicating the duration required to send data of a first data volume; The second parameter includes a second startup duration and a second duration, the second startup duration indicating the duration between determining a data processing task and executing the data processing task, the data processing task represents processing data through the neural network layer in the machine learning model, and the second duration indicates the duration required to process a second amount of data through the neural network layer in the machine learning model.

3. The method according to claim 1 or 2, characterized in that: Determining the value of r according to the first parameter and / or the second parameter includes: According to the first parameter and / or the second parameter, the objective function is solved to determine the value of r, wherein the objective function represents the total time consumed to obtain the processing results of the r sub-data, and the constraints of the objective function are related to the time occupied by the task of sending the sub-data and the time occupied by the task of processing the sub-data.

4. The method according to claim 3, characterized in that The objective function includes one or more of a first objective function, a second objective function or a third objective function; The constraint condition of the first objective function includes that the time taken by a task of sending the sub-data is greater than or equal to the time taken by a task of processing the sub-data; The constraint condition of the second objective function includes that the time occupied by a task of sending the sub-data is less than the time occupied by a task of processing the sub-data, and the first time is not a subset of the second time, the first time includes the time occupied by the tasks of processing the r sub-data, and the second time includes the time occupied by the tasks of sending the r sub-data and the time occupied by the tasks of sending the processing results of the r sub-data; The constraint conditions of the third objective function include that the time occupied by a task of sending the sub-data is less than the time occupied by a task of processing the sub-data, and the first time is a subset of the second time.

5. The method according to claim 1 or 2, characterized in that: The machine learning model includes at least one neural network module, and the multiple neural network layers included in the first neural network module of the at least one neural network module are deployed in the same communication node cluster, and the first parameter and the second parameter in the multiple communication nodes included in the communication node cluster are the same.

6. The method according to claim 2, characterized in that The first duration represents the duration required to send the first data volume of data using an all-to-all communication method.

7. The method according to claim 6, characterized in that The machine learning model is a hybrid expert MoE model, and the second duration represents the duration required for processing the second data volume using an expert neural network layer in the MoE model.

8. The method according to claim 1 or 2, characterized in that: The first parameter is obtained by counting multiple data sending tasks executed in an all-to-all communication mode, wherein the multiple data sending tasks include data sending tasks with different data amounts.

9. The method according to claim 1 or 2, characterized in that: The second parameter is obtained by statistically analyzing multiple data processing tasks performed by the neural network layer in the machine learning model, wherein the multiple data processing tasks involve processing data of different amounts.

10. The method according to claim 1 or 2, characterized in that: The r sub-data are processed by the neural network layer in the machine learning model respectively, including: During the inference phase of the machine learning model, the r sub-data are respectively processed by the neural network layers in the machine learning model; or, during the training phase of the machine learning model, the r sub-data are respectively processed by the neural network layers in the machine learning model.

11. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring first data; A determination module is used to determine the value of r based on a first parameter and / or a second parameter, wherein r is used to indicate that the first data is divided into r sub-data, and the r sub-data are respectively processed by a neural network layer in a machine learning model, and r is an integer greater than or equal to 1. The first parameter indicates the network condition of the communication network used to send the sub-data, and the second parameter indicates the load condition of the communication node used to process the sub-data.

12. The device according to claim 11, characterized in that The first parameter includes a first startup duration and a first duration corresponding to the communication network, the first startup duration indicating the duration between determining a data sending task and executing the data sending task, and the first duration indicating the duration required to send data of a first data volume; The second parameter includes a second startup duration and a second duration, the second startup duration indicating the duration between determining a data processing task and executing the data processing task, the data processing task represents processing data through the neural network layer in the machine learning model, and the second duration indicates the duration required to process a second amount of data through the neural network layer in the machine learning model.

13. The device according to claim 11 or 12, characterized in that The determination module is specifically used to solve the objective function according to the first parameter and / or the second parameter to determine the value of r, wherein the objective function represents the total time consumed to obtain the processing results of the r sub-data, and the constraints of the objective function are related to the time occupied by the task of sending the sub-data and the time occupied by the task of processing the sub-data.

14. The device according to claim 13, characterized in that The objective function includes one or more of a first objective function, a second objective function or a third objective function; The constraint condition of the first objective function includes that the time taken by a task of sending the sub-data is greater than or equal to the time taken by a task of processing the sub-data; The constraint conditions of the second objective function include that the time taken by a task of sending the sub-data is less than the time taken by a task of processing the sub-data, and the first time is not a subset of the second time, and the first time includes The first time includes the time occupied by the task of processing the r sub-data, and the second time includes the time occupied by the task of sending the r sub-data and the time occupied by the task of sending the processing results of the r sub-data; The constraint conditions of the third objective function include that the time occupied by a task of sending the sub-data is less than the time occupied by a task of processing the sub-data, and the first time is a subset of the second time.

15. The device according to claim 11 or 12, characterized in that The machine learning model includes at least one neural network module, and the multiple neural network layers included in the first neural network module of the at least one neural network module are deployed in the same communication node cluster, and the first parameter and the second parameter in the multiple communication nodes included in the communication node cluster are the same.

16. The device according to claim 12, characterized in that The first duration represents the duration required to send the first data volume of data using an all-to-all communication method.

17. The device according to claim 16, characterized in that The machine learning model is a hybrid expert MoE model, and the second time length represents the time length required to process the second data volume using an expert neural network layer in the MoE model.

18. The device according to claim 11 or 12, characterized in that The first parameter is obtained by counting multiple data sending tasks executed in an all-to-all communication mode, wherein the multiple data sending tasks include data sending tasks with different data amounts.

19. The device according to claim 11 or 12, characterized in that The second parameter is obtained by statistically analyzing multiple data processing tasks performed by the neural network layer in the machine learning model, wherein the multiple data processing tasks involve processing data of different amounts.

20. The device according to claim 11 or 12, characterized in that The r sub-data are processed by the neural network layer in the machine learning model respectively, including: During the inference phase of the machine learning model, the r sub-data are respectively processed by the neural network layers in the machine learning model; or, during the training phase of the machine learning model, the r sub-data are respectively processed by the neural network layers in the machine learning model.

21. A communication node, characterized in that: Including processor, The processor is configured to execute the method according to any one of claims 1 to 10.

22. An electronic device, characterized in that: comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the method according to any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

Images