Data processing method and related apparatus
Patent Information
- Application Number
- PCT/CN2026/076866
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-21
- Filing Date
- 2026-02-03
- Publication Date
- 2026-08-27
Smart Images
Figure CN2026076866_27082026_PF_FP_ABST
Abstract
Description
A data processing method and related apparatus
[0001] This application claims priority to Chinese Patent Application No. CN202510201529.8, filed on February 21, 2025, entitled "A Data Processing Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of artificial intelligence, and more particularly to a data processing method and related apparatus. Background Technology
[0003] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.
[0004] Training large models typically involves a combination of parallel strategies, such as tensor parallel (TP), data parallel (DP), and pipeline parallel (PP). Different model structures and parameters have varying requirements for parallel strategies, and each strategy has its specific applicable scenarios and performance optimization methods. For example, module A of a model might be better suited for training using tensor parallelism, while module B might be better suited for training using data parallelism. Therefore, configuring appropriate parallel strategies for the model to be trained is a crucial problem that needs to be solved. Summary of the Invention
[0005] In a first aspect, embodiments of this application provide a data processing method, including:
[0006] Receive model configuration information input by the user;
[0007] A population is generated based on the input configuration information. This population is a randomly generated initial population and includes multiple parallel policy sets, each of which is used to train the model.
[0008] Based on the fitness of each parallel policy set in the multiple parallel policy sets, the multiple parallel policy sets are divided into multiple clusters. The fitness is used to indicate the training time of the model based on the parallel policy sets. Each cluster includes at least one parallel policy set.
[0009] The population is updated through crossover operations until the updated population meets the target conditions. The target parallel policy set is determined from the updated population that meets the target conditions. The target parallel policy set is the parallel policy set with the shortest training time in the updated population. Crossover operations include intra-cluster crossover and inter-cluster crossover. Intra-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from the same cluster and combined to generate a new parallel policy set. Inter-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from different clusters and combined to generate a new parallel policy set.
[0010] After determining the target parallel strategy set, the target parallel strategy set is fed back to the user.
[0011] In this application, the optimal solution is sought in the search space of parallel policy sets by utilizing the heuristic search of a genetic algorithm. Multiple parallel policy sets in the initially generated population can be considered as individuals in the genetic algorithm. When updating the population based on the crossover operation, the parallel policy set selected by the crossover operation is the parent individual in the iterative update of the genetic algorithm, and the new parallel policy set generated by combining them through the crossover operation is the offspring individual. Different parameter values of the generated offspring are inherited from different parent individuals.
[0012] For example, "Child Individual 1" can be generated by cross-processing "Parent Individual 1" and "Parent Individual 2", where the values of "Parent Individual 1" are (4, 2, 3, 5, 7) and the values of "Parent Individual 2" are (3, 4, 4, 5, 6). The parameters of the two parent individuals are randomly selected; for example, the first two parameters of "Child Individual 1" can be from "Parent Individual 1", while the last three parameters can be from "Parent Individual 2", resulting in the value set of "Child Individual 1" as (4, 2, 4, 5, 6).
[0013] Using the above method, the population is stratified into different clusters based on the different fitness levels of parallel strategy sets. The crossover operation in the genetic algorithm is then used to obtain factors for updating the population from each cluster. Intra-cluster crossover, especially crossover of high-fitness, superior individuals, maintains the genes of excellent individuals and preserves high-quality individuals. Inter-cluster crossover, by combining individuals within different fitness ranges, explores more possibilities for gene combinations, increases population diversity, and avoids missing potential high-quality solutions.
[0014] In one possible implementation, the multiple parallel policy sets are divided into multiple clusters based on the fitness of each parallel policy set, including:
[0015] Multiple fitness ranges are determined based on the fitness of multiple parallel policy sets, and the multiple fitness ranges do not overlap;
[0016] Multiple parallel policy sets are divided into multiple clusters based on multiple fitness ranges, wherein each cluster contains at least one parallel policy set belonging to the same fitness range.
[0017] In one possible implementation, the method further includes the following steps during population updates via crossover:
[0018] Multiple fitness ranges are updated based on the fitness of the parallel policy set in the updated population, and multiple clusters are re-divided using the updated fitness ranges.
[0019] During the iterative update process, if the fitness distribution of individuals changes significantly, the stratification strategy needs to be adjusted in a timely manner. Specifically, the fitness threshold for distinguishing clusters should be redefined. For example, when the training time for a new generation of individuals is significantly shortened, leading to an increase in the overall fitness level, the stratification should be adjusted promptly to ensure it matches the current data distribution.
[0020] In one possible implementation, during the population update process via crossover, the computational resource threshold of the updated population is less than that of the population before the update. The computational resource threshold is used to indicate the maximum amount of computational resources that any set of parallel policies in the population is allowed to use.
[0021] In this embodiment, a computational resource threshold is established based on the amount of computational resources allocated by the system for training the model. In the initial stages of population iteration, to improve solution diversity and avoid premature convergence to local optima, the basic computational resource threshold can be relaxed. During population iteration, this threshold is gradually constrained to ensure that the final result meets the predetermined requirements.
[0022] Using the above method, the constraints are appropriately relaxed in the early stages of population iteration to reduce the generation of infeasible solutions and the omission of potential high-quality solutions caused by prematurely strict restrictions. As the iteration process progresses into the later stages, convergence is performed on the high-quality individuals generated later, ensuring that exploration resources are efficiently allocated at different stages of the algorithm and achieving a balance between global and local search.
[0023] In one possible implementation, a set of parallel strategies for the objective is obtained from the updated population that satisfies the objective conditions, including:
[0024] Obtain the target parallel policy set from the candidate set. The candidate set includes the parallel policy set with the shortest training time in each updated population that meets the target condition. The target parallel policy set is the parallel policy set with the shortest training time in the candidate set.
[0025] Using the above method, unnecessary storage and computational overhead can be reduced, and the set of parallel strategies with the highest fitness can be determined more directly and efficiently.
[0026] In one possible implementation, the updated population satisfies the following target conditions:
[0027] The updated population corresponds to an update count greater than or equal to the first threshold; or,
[0028] The highest fitness of the updated population decreases by a factor greater than or equal to the second threshold compared to the highest fitness of the population before the update. The highest fitness is the maximum fitness of the parallel policy set in the population.
[0029] In this embodiment, if the highest fitness of the updated population is higher than that of the population before the update, it indicates that the update is positive and the iteration process can continue. If the highest fitness of the updated population is lower than that of the population before the update, but the decrease does not exceed the second threshold, it can be considered a normal fluctuation, and the iteration process can continue. When the decrease is greater than or equal to the second threshold, it indicates that the update is ineffective, and therefore the update is stopped.
[0030] In one possible implementation, the model's configuration information includes one or more of the following:
[0031] The model's network structure;
[0032] Model training configuration information;
[0033] Parallel strategy configuration information for the model;
[0034] The model's collective communication algorithm; or,
[0035] The training communication bandwidth of the model.
[0036] In one possible implementation, the set of parallel strategies includes one or more of the following parallel strategies:
[0037] Tensor parallelism;
[0038] Data parallelism;
[0039] Parallel production lines;
[0040] Experts in parallel; or,
[0041] Sequence parallelism.
[0042] In one possible implementation, the method further includes: obtaining a performance report of the model training process;
[0043] Based on the performance report, first optimization information is generated, which includes code optimization schemes for the model and / or optimization schemes for the target parallel strategy set.
[0044] In one possible implementation, a performance report of the model training process is obtained, including:
[0045] Based on the historical performance collection code of the knowledge base and the performance collection configuration information of the model, the first information is generated through the large language model LLM;
[0046] Based on the first information and the historical performance data collection code of the knowledge base, the second information is generated through LLM;
[0047] Based on the second information and the model's performance acquisition configuration information, the third information is generated through LLM;
[0048] Based on third-party information and configuration recommendations for performance acquisition tools, target performance acquisition code is generated using LLM.
[0049] By configuring the target performance acquisition code, a performance report of the model training process can be obtained.
[0050] In this embodiment, by using multi-step reflective correction performance acquisition configuration code, the accuracy and quality of the generated configuration can be improved, and the personalized needs of users can be taken into account, thus optimizing the output results of LLM.
[0051] In one possible implementation, based on the performance report, first optimization information is generated, including:
[0052] Based on the performance report, the first code snippet was identified as the code snippet in the model that caused the performance loss.
[0053] The first representation is obtained, which is generated based on the fusion network fusing multiple modal representations of the first code segment;
[0054] Obtain the second code segment with the highest similarity to the first representation from the knowledge base;
[0055] Based on the second code snippet, a code optimization scheme is generated from the large code model.
[0056] In one possible implementation, the modal representations include one or more of the following:
[0057] The semantic representation of the first code snippet;
[0058] Structural representation of the first code snippet;
[0059] The execution characteristics of the first code snippet;
[0060] The contextual representation of the first code snippet; or,
[0061] The first code snippet represents the type of performance bottleneck.
[0062] In one possible implementation, the fusion network includes a projection layer module and an embedding layer module, which are used to map multiple modal representations of the first code segment into vector representations of the same dimension.
[0063] In this embodiment, multi-embedded retrieval is performed by fusing multiple embedding representations of the code, which enhances the model's understanding of the code context and intent, thereby improving the accuracy of code case retrieval.
[0064] Secondly, embodiments of this application provide a data processing apparatus, including:
[0065] The receiving module is used to receive the model configuration information input by the user;
[0066] The generation module is used to generate a population based on the configuration information. The population includes multiple parallel policy sets, which are used to train the model.
[0067] The hierarchical module is used to divide multiple parallel policy sets into multiple clusters based on the fitness of each parallel policy set. The fitness is used to indicate the training time of the model based on the parallel policy sets. Each cluster includes at least one parallel policy set.
[0068] The processing module is used to update the population through crossover operations until the updated population meets the target conditions. It determines the target parallel policy set from the updated population that meets the target conditions. The target parallel policy set is the parallel policy set with the shortest training time in the updated population. The crossover operation includes intra-cluster crossover and inter-cluster crossover. Intra-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from the same cluster and combined to generate a new parallel policy set. Inter-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from different clusters and combined to generate a new parallel policy set.
[0069] The feedback module is used to provide users with the target parallel strategy set.
[0070] In one possible implementation, the layered modules are specifically used for:
[0071] Multiple fitness ranges are determined based on the fitness of multiple parallel policy sets, and the multiple fitness ranges do not overlap;
[0072] Multiple parallel policy sets are divided into multiple clusters based on multiple fitness ranges, wherein each cluster contains at least one parallel policy set belonging to the same fitness range.
[0073] In one possible implementation, during the population update process via crossover, the processing module is also used to:
[0074] Multiple fitness ranges are updated based on the fitness of the parallel policy set in the updated population, and multiple clusters are re-divided using the updated fitness ranges.
[0075] In one possible implementation, during the population update process via crossover, the computational resource threshold of the updated population is less than that of the population before the update. The computational resource threshold is used to indicate the maximum amount of computational resources that any set of parallel policies in the population is allowed to use.
[0076] In one possible implementation, the processing module is specifically used for:
[0077] Obtain the target parallel policy set from the candidate set. The candidate set includes the parallel policy set with the shortest training time in each updated population that meets the target condition. The target parallel policy set is the parallel policy set with the shortest training time in the candidate set.
[0078] In one possible implementation, the updated population satisfies the following target conditions:
[0079] The updated population corresponds to an update count greater than or equal to the first threshold; or,
[0080] The highest fitness of the updated population decreases by a factor greater than or equal to the second threshold compared to the highest fitness of the population before the update. The highest fitness is the maximum fitness of the parallel policy set in the population.
[0081] In one possible implementation, the model's configuration information includes one or more of the following:
[0082] The model's network structure;
[0083] Model training configuration information;
[0084] Parallel strategy configuration information for the model;
[0085] The model's collective communication algorithm; or,
[0086] The training communication bandwidth of the model.
[0087] In one possible implementation, the set of parallel strategies includes one or more of the following parallel strategies:
[0088] Tensor parallelism;
[0089] Data parallelism;
[0090] Parallel production lines;
[0091] Experts in parallel; or,
[0092] Sequence parallelism.
[0093] Thirdly, embodiments of this application provide a computing device, including: a processor, a memory, and a transceiver. The memory stores computer programs or computer instructions, and the processor is used to call and run the computer programs or computer instructions stored in the memory, causing the processor to perform processing operations as described in the first aspect and any implementation thereof. The transceiver is used to transmit and receive signals, such as implementing the receiving and sending operations as described in the first aspect and any implementation thereof.
[0094] Fourthly, embodiments of this application provide a computing device cluster, including at least one computing device, each computing device including a processor and a memory, the memory storing computer programs or computer instructions, the processor being used to call and run the computer programs or computer instructions stored in the memory, so that the computing device cluster performs the first aspect and any of its optional methods described above.
[0095] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the first aspect and any of its optional methods.
[0096] Sixthly, embodiments of this application provide a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the methods described in the first aspect and any of its alternatives.
[0097] The technical effects of the second, third, fourth, fifth, and sixth aspects of this application can be understood in conjunction with the technical effects of the first aspect and any implementation thereof. Attached Figure Description
[0098] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0099] Figure 1 is a schematic diagram of a structural framework for artificial intelligence.
[0100] Figure 2 is a schematic diagram of an application architecture provided in an embodiment of this application;
[0101] Figure 3 is a schematic diagram of another application architecture provided in an embodiment of this application;
[0102] Figure 4 is a schematic diagram of another application architecture provided in an embodiment of this application;
[0103] Figure 5 is a flowchart illustrating a data processing method provided in an embodiment of this application.
[0104] Figure 6 is another flowchart illustrating the data processing method provided in an embodiment of this application;
[0105] Figure 7 is another flowchart illustrating the data processing method provided in an embodiment of this application;
[0106] Figure 8 is another flowchart illustrating the data processing method provided in an embodiment of this application;
[0107] Figure 9 is a schematic diagram of the code snippet that causes performance loss in this application;
[0108] Figure 10 is a schematic diagram of the fusion network provided in an embodiment of this application;
[0109] Figure 11 is a schematic diagram of an optimization system provided in an embodiment of this application;
[0110] Figure 12 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;
[0111] Figure 13 is a schematic diagram of a computing device provided in an embodiment of this application;
[0112] Figure 14 is a schematic diagram of a computing device cluster provided in an embodiment of this application;
[0113] Figure 15 is another structural schematic diagram of the computing device cluster provided in an embodiment of this application. Detailed Implementation
[0114] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0115] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0116] First, the overall workflow of an artificial intelligence system is described, as shown in Figure 1. Figure 1 is a structural diagram of the main framework of artificial intelligence. The framework is then elaborated on from two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed by technology) to the industrial ecosystem of the system.
[0117] (1) Infrastructure
[0118] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips (hardware acceleration chips such as CPUs, NPUs, GPUs, ASICs, and FPGAs); and the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.
[0119] (2) Data
[0120] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.
[0121] (3) Data processing
[0122] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.
[0123] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.
[0124] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.
[0125] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.
[0126] (4) General ability
[0127] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.
[0128] (5) Smart Products and Industry Applications
[0129] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent terminals, intelligent transportation, intelligent healthcare, autonomous driving, smart cities, etc.
[0130] This application can be applied, but is not limited to, to the field of performance analysis in the field of artificial intelligence. Specifically, it can be applied to the performance analysis and optimization of neural network training in the field of neural networks. The following will introduce several application scenarios that have been implemented in products.
[0131] To better understand the solutions of the embodiments of this application, the possible application scenarios of the embodiments of this application will be briefly introduced below.
[0132] The product form of this application embodiment can be a performance analysis application. Performance analysis applications can run on terminal devices or cloud-based servers.
[0133] In one possible implementation, referring to Figure 2, a performance analysis application can perform performance analysis tasks and obtain processing results.
[0134] For example, performance analysis can be performed on the initial parallel strategy set based on the configuration information of the model to be trained, such as network structure, training configuration, parallel strategy configuration, ensemble communication algorithm, and bandwidth. Based on the analysis results, a genetic algorithm is used to iteratively optimize the parallel strategy set to generate a set that matches the model configuration and meets its computational resource requirements.
[0135] For example, performance analysis can also be performed through the performance report of the model training process to identify code snippets that cause performance loss and provide corresponding optimization suggestions.
[0136] In one possible implementation, a user can open a performance analysis application installed on a terminal device and input configuration information of the model to be trained or data such as performance reports of the training process. The performance analysis application can use the model trained by the method provided in the embodiments of this application, or process the input data by the method provided in the embodiments of this application, and present the processing results to the user (the presentation method may include, but is not limited to, displaying, playing, saving, uploading to the cloud, etc.).
[0137] In one possible implementation, a user can open a performance analysis application installed on a terminal device and input configuration information of the model to be trained or data such as performance reports during the training process. The performance analysis application can send the input data to a cloud-based server. The cloud-based server processes the input data using the model trained by the method provided in this application embodiment and sends the processing results back to the terminal device. The terminal device can then present the processing results to the user (the presentation method may include, but is not limited to, displaying, playing, saving, or uploading to the cloud).
[0138] Referring to Figure 2, which is a schematic diagram of the functional architecture of the performance analysis application in this embodiment of the application:
[0139] In one possible implementation, as shown in Figure 2, the performance analysis application 202 can receive input parameters 201 (e.g., user information and other data) and generate a processing result 203. The performance analysis application 202 can execute on at least one computer system (for example) and includes computer code that, when executed by one or more computers, causes the computers to execute a model trained using the methods provided in the embodiments of this application.
[0140] Please refer to Figure 3, which is a schematic diagram of the entity architecture of a performance analysis application running in an embodiment of this application. Figure 3 shows a schematic diagram of a system architecture. The system may include a terminal 300 and a server 400. The server 400 may include one or more servers (Figure 3 illustrates this with one server as an example), and the server 400 can provide performance analysis functions for one or more terminals.
[0141] The terminal 300 may have a performance analysis application installed or a webpage related to performance analysis functions open. The application and webpage can provide an interface. The terminal 300 can receive relevant parameters input by the user on the performance analysis function interface and send the parameters to the server 400. The server 400 can obtain the processing result based on the received parameters and return the processing result to the terminal 300.
[0142] It should be understood that in some optional implementations, the terminal 300 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.
[0143] The terminal 300 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.
[0144] In one possible implementation, the server can provide performance analysis services to the client side through an application programming interface (API).
[0145] In this process, the terminal device can send relevant parameters (such as configuration information of the model to be trained) to the server through the API provided by the cloud. The server can obtain the processing results based on the received parameters and return the processing results to the terminal.
[0146] The description of the terminal and server can be found in the above embodiments, and will not be repeated here.
[0147] Figure 4 illustrates the process of using a cloud service with performance analysis capabilities provided by a cloud platform.
[0148] 1. Activate and purchase performance analysis services.
[0149] 2. Users can download the software development kit (SDK) corresponding to the performance analysis service. Cloud platforms usually provide multiple development versions of the SDK for users to choose from according to their development environment needs, such as JAVA version SDK, Python version SDK, PHP version SDK, Android version SDK, etc.
[0150] 3. After downloading the corresponding version of the SDK to their local machine according to their needs, users can import the SDK project into their local development environment, configure and debug it in the local development environment, and develop other functions in the local development environment to form an application that integrates performance analysis capabilities.
[0151] 4. When an application needs to perform performance analysis, it can trigger an API call for the performance analysis function. When the application triggers the performance analysis function, it sends an API request to the running instance of the performance analysis service in the cloud environment. The API request carries an image, which is then processed by the running instance in the cloud environment to obtain the processing result.
[0152] 5. The cloud environment returns the processing results to the application, thus completing a performance analysis function call.
[0153] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts such as neural networks involved in the embodiments of this application will be introduced below.
[0154] (1) Neural Network
[0155] A neural network can be composed of neural units, which can be operational units that take xs and an intercept of 1 as inputs, and whose output can be:
[0156] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer, and the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.
[0157] (2) Parallel Strategy Set
[0158] A set of parallel strategies refers to a combination of multiple parallel strategies selected during model training. Based on the values of different computing resources (such as graphics processing units (GPUs), neural processing units (GPUs), and tensor processing units (TPUs), the selected parallel strategies are ensured to be within a valid range. Parallel strategies, on the other hand, refer to the methods for allocating and coordinating computational tasks among multiple computing resources (such as GPUs, NPUs, and TPUs). These include:
[0159] ① Tensor parallel (TP) strategy:
[0160] Tensor parallelism involves splitting a tensor operation (such as matrix multiplication) along a specific dimension and distributing it across multiple devices for simultaneous execution. In this mode, parameters within the same layer are divided into smaller parts and distributed across different GPUs. This is primarily useful for situations where the number of parameters is so large that a single GPU cannot store the entire model. Alternatively, it can be used to alleviate memory pressure by maintaining higher numerical accuracy without mixed-precision training.
[0161] ② Data parallel (DP) strategy
[0162] The entire dataset is divided into multiple subsets, and each subset is assigned to different computing devices (such as GPUs) for parallel processing. In each iteration, a copy of the model performs forward and backward propagation computations on different devices using different data subsets. All devices independently compute the gradients for their assigned batch of data, and then aggregate these gradients through an "all-reduce" operation to update the model parameters. This approach is primarily designed for computationally intensive tasks with a relatively small number of model parameters.
[0163] ③ Pipeline parallel (PP) strategy
[0164] The neural network model is divided into several stages, each of which can be executed on a different GPU computing device. For tasks involving long input sequences, such as text generation or translation in natural language processing, pipeline parallelism can reduce the amount of GPU memory required for each stage by dividing the model into multiple stages.
[0165] ④ Expert parallel (EP) strategy
[0166] In the Mixture of Experts (MoE) model, expert parallelism is a method of assigning different experts (i.e., different parts of the model) to different devices for parallel computation.
[0167] ⑤ Sequence parallel (SP) strategy
[0168] Sequence parallelism is a parallel computing approach that partitions input sequences into columns, further improving computational efficiency compared to tensor parallelism. In sequence parallelism, weights are partitioned during computation and then computed on the same computing device as other tensor parallelism methods. For tasks involving long input sequences, such as text generation or translation in natural language processing, sequence parallelism can significantly improve computational efficiency.
[0169] (3) Performance Acquisition
[0170] Performance acquisition refers to configuring corresponding performance acquisition code according to the user's performance acquisition requirements and using performance acquisition tools to collect performance reports (profiling) during the model training process. Information from profiling helps to understand the model's training efficiency, resource utilization, and potential performance bottlenecks. This profiling information includes computation time (e.g., total training time, iteration time), resource utilization (e.g., CPU, GPU, memory, VRAM), throughput, latency, and accuracy.
[0171] (4) Multimodal embedding
[0172] Multimodal embedding refers to transforming the semantic information, structural features, runtime behavior, and other features of a code fragment into an embedded representation in the same high-dimensional space, enabling the system to comprehensively understand the meaning of the code at different levels and improve retrieval performance.
[0173] (5) Context awareness
[0174] The system can understand the context of code snippets (such as function call relationships, variable dependencies, comments, etc.) and the types of performance problems, thereby more accurately understanding the purpose, intent, and optimization direction of the code.
[0175] (6) Genetic Algorithm (GA)
[0176] Genetic algorithms are heuristic search algorithms that simulate biological evolution. They optimize solutions to problems by mimicking biological mechanisms such as natural selection, heredity, crossover (or hybridization), and mutation. The core principles of genetic algorithms include encoding, initial population generation, fitness evaluation (defining a fitness function based on the algorithm's objective to evaluate the performance of each individual), selection (selecting individuals for reproduction based on fitness, with higher-fitness individuals having a higher probability of being selected), crossover (selected individuals generate new offspring through crossover operations, simulating gene recombination), mutation (introducing random mutations into individuals), and the formation of the next generation population.
[0177] Different model architectures and parameter configurations have varying requirements for parallel processing strategies. Each parallel strategy is suitable for specific scenarios and has corresponding performance improvement methods. When formulating a parallel strategy, both model construction and computational resources need to be comprehensively considered. In existing technologies, manually selecting and configuring parallel strategies often requires extensive experimentation and adjustments, resulting in low efficiency.
[0178] Furthermore, existing technologies propose using genetic algorithms to gradually approach the optimal solution within the search space of a parallel strategy set. However, since the values of the parallel strategies are influenced by the specific model architecture and parameter configuration, genetic algorithms typically focus on combining individuals with high fitness to generate new individuals, which can easily lead to getting trapped in local optima.
[0179] To address the aforementioned problems, this application provides a data processing method. As shown in Figure 5, the model training method provided in this application includes the following steps 501-505.
[0180] 501. Receive the model configuration information input by the user.
[0181] The model's configuration information includes the model's network structure, training configuration information, parallel strategy configuration information, ensemble communication algorithm, and training communication bandwidth. The training communication bandwidth refers to the communication bandwidth of the AI cluster that trains the model.
[0182] For example, a user inputs the model's configuration information through an interactive interface and triggers a request for a set of parallel recommendation strategies.
[0183] 502. Generate a population based on the configuration information. The population includes multiple parallel policy sets, which are used to train the model.
[0184] Specifically, based on the model's configuration information, an initial population is randomly generated. Each individual in the population consists of multiple parallel policies, meaning each individual is a set of parallel policies.
[0185] Model training is constrained by the system's computational resources. To ensure the effective implementation of parallel strategies, the selected parallel strategy values must be factors of the number of GPUs, NPUs, and / or TPUs in the system. By factoring the number of GPUs, the parallel strategy values become GPU factor values, ensuring that the computational resources required by the combination of these strategies do not exceed the total amount of computational resources configured in the system. Specifically, the computational resources required for the set of parallel strategies are the product of the values of each parallel strategy in the set (i.e., the required computational resources).
[0186] For example, if the first constraint is that the number of GPUs does not exceed 30, and the configured parallel strategy set includes TP, DP, and PP strategies, and since the number of GPUs required to execute the parallel strategy is greater than 1, then the selectable values for these parallel strategies are factors of 30, including (2, 3, 5, 6, 10, 15). The value combinations of each parallel strategy in the parallel strategy set can be (2, 3, 6) or (3, 3, 3), etc., ensuring that the product of all strategy values does not exceed the total number of 30 GPUs provided by the system.
[0187] When generating the initial population, a first constraint is first determined based on the maximum computing resources allocated by the system. This first constraint indicates the maximum value of the computing resources required for the parallel policy set. For example, the computing resources indicated by the first constraint may be the computing resources allocated by the system for the model to be trained.
[0188] In one possible implementation, to mitigate the risk of infeasible solutions or omission of potential high-quality solutions due to strict resource constraints, relaxed constraints are used when generating the parallel policy set in the initial stage. For example, if the aforementioned first constraint specifies that the total number of GPUs required is N, then in the initial stage, this constraint is relaxed to allow the use of up to 2N GPUs, thereby generating a parallel policy set based on more relaxed restrictions.
[0189] The following section will use the number of GPUs as an example to illustrate the concept of computing resources.
[0190] For example, the generated individuals are defined as {Tensor parallelism TP, Data parallelism DP, Pipeline parallelism PP, Expert parallelism EP, Sequence parallelism SP}, and the randomly generated population is shown in Table 1 below:
[0191] Table 1
[0192] Individuals 1 through 5 are five randomly generated sets of parallel strategies. Taking individual 1 as an example, the parallel strategy set of individual 1 indicates that the model uses 4, 2, 3, 5, and 7 GPUs respectively during training for tensor parallelism, data parallelism, pipeline parallelism, expert parallelism, and sequence parallelism.
[0193] After generating the initial population, simulations are performed on each parallel policy set using the model's configuration information to determine the single-step time for each policy set. Since the optimization objective is to reduce model training time, the fitness of a parallel policy is defined as the reciprocal of its single-step time. A shorter single-step time results in higher fitness.
[0194] For example, the fitness and time taken per step for each individual satisfy the following formula:
[0195] Fitness(Individual)=1 / Titer=1 / (Tcompute+Tcomm+Tbubble)
[0196] Where Tcompute represents the computation time for model training, Tcomm represents the communication time for data synchronization and exchange between computing resources during parallel training, and Tbubble represents the idle waiting time of computing resources during the training process.
[0197] In one possible implementation, the user can configure the values of some parallel strategies. For example, the user can explicitly set the values of TP and DP to 2 and 3, respectively. Based on these user configuration requirements, the values of PP, EP, and SP are randomly assigned to construct the initial population. Furthermore, during subsequent population updates and iterations, the user-defined values of TP and DP remain constant, allowing the values of other parallel strategies to continue to be optimized and adjusted within permissible limits.
[0198] 503. Based on the fitness of each parallel strategy set in the multiple parallel strategy sets, divide the multiple parallel strategy sets into multiple clusters.
[0199] Specifically, fitness is used to indicate the training time of the model based on a set of parallel policies, with each cluster including at least one set of parallel policies.
[0200] In this embodiment, multiple fitness ranges (or intervals) are obtained by integrating and analyzing the fitness of each parallel policy set. The fitness corresponds to the training time (single-step time) of the model training of the parallel policy set. The population is stratified according to these multiple fitness ranges, dividing the multiple parallel policy sets in the population into multiple clusters. The parallel policies contained in each cluster belong to the same fitness range.
[0201] Taking Table 1 above as an example, individuals are sorted in descending order of fitness value. The sorted order is as follows: individual 2 (fitness 0.056), individual 1 (fitness 0.05), individual 5 (fitness 0.048), individual 3 (fitness 0.045), and individual 4 (fitness 0.038).
[0202] Then, the sorted individuals are stratified. First, multiple fitness ranges are determined based on each fitness level to divide the clusters.
[0203] For example, individuals in the population can be divided in half or into two clusters according to a preset ratio: a "high-quality layer" and a "low-quality layer." Wherein:
[0204] The fitness range of the "high-quality layer" is [0.05-0.065), and the individuals in the cluster include "individual 2" and "individual 1".
[0205] The fitness range of the "inferior layer" is [0.035-0.05), and the individuals within the cluster include "individual 5", "individual 3" and "individual 4".
[0206] For example, individuals in the population can also be divided into three clusters: a "high-quality layer", a "medium-quality layer", and a "low-quality layer".
[0207] in:
[0208] The fitness range of the "high-quality layer" is [0.054-0.06), and the individuals within the cluster include "individual 2".
[0209] The fitness range of the "intermediate layer" is [0.046-0.054), and the individuals within the cluster include "individual 1" and "individual 5".
[0210] The fitness range of the "inferior layer" is [0.038-0.046), and the individuals within the cluster include "individual 3" and "individual 4".
[0211] Because the requirements of model structure and parameters for parallel strategies are uncertain, and different parallel strategies have different requirements for network architecture, the impact of changes in the value of the parallel strategy on fitness in an individual often exhibits a non-linear characteristic. Combining the principles of genetic algorithms, the advantage of stratifying individuals in the population according to their fitness range is that: individuals with higher fitness can be combined to retain superior gene combinations; individuals with higher fitness can also be combined with individuals with lower fitness to achieve diversity exploration and avoid falling into the trap of local optima.
[0212] It is understood that the number of clusters and the corresponding fitness range mentioned above are merely examples, and this application does not limit the specific number of clusters or the corresponding fitness range.
[0213] 504. Update the population through crossover operations until the updated population satisfies the target condition, and determine the target parallel policy set from the updated population that satisfies the target condition.
[0214] The target parallel policy set is the set of parallel policies with the shortest training time in the updated population, and the crossover operation includes intra-cluster crossover and inter-cluster crossover.
[0215] Intra-cluster crossover refers to selecting multiple individuals as parent individuals within the same cluster, combining some parallel strategy parameters of each parent individual, and generating new child individuals. For example, in the "Medium Layer", selection operations can be performed to obtain "Individual 1" and "Individual 5" for intra-cluster crossover to generate new child individuals.
[0216] Inter-cluster crossover refers to selecting multiple individuals from different clusters as parent individuals, combining some parallel strategy parameters of each parent individual, and generating new child individuals. For example, a selection operation is performed between the "high-quality layer" and the "medium layer" to obtain "individual 2" and "individual 5" for inter-cluster crossover to generate new child individuals.
[0217] Specifically, the target conditions to be met include the following two situations:
[0218] First, the number of iterations has reached or exceeded the preset maximum number of iterations (first threshold). For example, if the preset maximum number of iterations is set to 10, then when the population is updated to the 10th iteration, further updates will stop.
[0219] Second, the decrease in the highest fitness of the updated population relative to the highest fitness of the population before the update is greater than or equal to a second threshold. The highest fitness refers to the maximum fitness of all individuals in the population (i.e., the set of parallel policies). The second threshold is a preset threshold value; when the decrease is greater than or equal to the second threshold, it indicates that the update was ineffective, and therefore the update is stopped.
[0220] It should be noted that if the highest fitness of the updated population is higher than that of the population before the update, then the update is considered positive and the iteration process can continue. If the highest fitness of the updated population is lower than that of the population before the update, but the decrease does not exceed the second threshold, then it can be considered a normal fluctuation, and the iteration process can continue. This application does not impose any restrictions on the specific value of the second threshold.
[0221] 505. Provide feedback to the user on the target parallel strategy set.
[0222] For example, after determining the optimal set of parallel strategies for model training, the set of parallel strategies is displayed on the display interface.
[0223] In this application, the population is stratified by different fitness levels among individuals. Individuals with similar fitness levels are grouped into the same cluster. Offspring generated by intra-cluster crossover of individuals within the same fitness range, especially by combining individuals from the superior layer, can maintain the superior genes of individuals and retain high-quality individuals. Furthermore, inter-cluster crossover of individuals within different fitness ranges can explore more possibilities for gene combinations, increase population diversity, and avoid missing potential high-quality solutions.
[0224] The specific process for updating the population in this application is described below with reference to Figure 6. Figure 6 is a schematic diagram of the data processing method in this application, including:
[0225] 601. Generate an initial population based on the relaxed constraints.
[0226] The constraint is the maximum amount of computing resources that any set of parallel strategies in the population is allowed to use, i.e., the computing resource threshold.
[0227] The system allocates computational resources for training the model based on a computational resource threshold, assuming a total of N GPUs. In the early stages of population iteration, to improve solution diversity and avoid premature convergence to local optima, the basic computational resource threshold can be relaxed; for example, it can be increased to twice the total number of GPUs, i.e., 2N.
[0228] Specifically, each policy in the parallel policy set should meet the following conditions:
[0229] TP*SP*DP*PP<=2N
[0230] Among them, TP, SP, DP and PP represent the value results of different types of parallel strategies.
[0231] Understandably, the initial intention of relaxing constraints is to reduce the omission of infeasible solutions or potentially high-quality solutions caused by strict constraints in the initial iterations. The specific extent of relaxation and the subsequent strategy of gradually tightening constraints can be flexibly adjusted according to actual needs. The relaxation can be 2 times, 3 times, or 1.5 times, etc., and this application does not limit this.
[0232] 602. Calculate fitness.
[0233] The fitness of each individual in the population (i.e., each parallel policy set) is calculated, and the fitness of each parallel policy set is obtained by calculating the single-step time corresponding to the parallel policy set.
[0234] In this embodiment, the specific calculation method of fitness is similar to the fitness calculation in step 501 of the embodiment shown in Figure 5 above, and will not be repeated here.
[0235] 603. Divide the clusters according to fitness.
[0236] In this embodiment, the clustering process adopts the fitness partitioning method of parallel strategy set. The specific processing process is similar to step 502 in the embodiment shown in Figure 5 above, and will not be repeated here.
[0237] In one possible implementation, if the fitness distribution of individuals changes significantly during iterative updates, the hierarchical strategy is adjusted accordingly, i.e., the fitness range used to distinguish clusters is reset. For example, when the single-step time corresponding to the new generation of individuals is greatly reduced, leading to an increase in the overall fitness level, the hierarchical division will be adjusted to adapt to the current data distribution.
[0238] For example, the fitness range before the update is assumed to be 0.03 to 0.06, and fitness intervals for the "high-quality layer" ([0.05, 0.06)), "medium-quality layer" ([0.04, 0.05)), and "low-quality layer" ([0.03, 0.04)) are set accordingly. However, as the single-step time of the updated individuals decreases, the fitness distribution changes, widening to 0.04 to 0.07. Accordingly, the fitness intervals of each cluster are also adjusted: the "high-quality layer" is increased to [0.06, 0.07), the "medium-quality layer" becomes [0.05, 0.06), and the "low-quality layer" is increased to [0.04, 0.05). This dynamic adjustment ensures that the stratification strategy is always synchronized with the latest data state.
[0239] 604. Generate offspring individuals through selection and crossover operations.
[0240] Specifically, from each cluster, a selection operation is performed to select some individuals as parents, and then intra-cluster crossover and inter-cluster crossover are performed to generate new offspring individuals.
[0241] For example, if "Individual 1" (4, 2, 3, 5, 7) and "Individual 5" (3, 4, 4, 5, 6) from the previous example are selected as the two parent entities for the crossover operation, the generated offspring will inherit some parameter values from "Individual 1" and some from "Individual 5". Specifically, the first two parameter values of the offspring can be set to come from "Individual 1", and the last three parameter values can come from "Individual 5". Therefore, the value of "Individual 1-1" generated by this crossover operation is (4, 2, 4, 5, 6).
[0242] In this application, the offspring individuals generated by the crossover operation also need to be determined whether they meet the aforementioned constraints, namely, the computational resources required for the newly generated parallel strategy set cannot exceed the computational resource threshold indicated by the constraints. After filtering out offspring individuals that do not meet the constraints, the remaining offspring individuals replace the individuals of the old generation, and the population is updated.
[0243] 605. Determine whether to terminate population updates.
[0244] If the updated population satisfies the target conditions shown in the aforementioned embodiment in Figure 5, the iteration is terminated, and step 608 is executed to obtain the set of parallel strategies with the highest fitness from the updated population.
[0245] If the updated population does not meet the target conditions shown in the aforementioned embodiment in Figure 5, then continue the iteration operation and proceed to step 606.
[0246] 606. Tighten constraints.
[0247] As the number of iterations increases, the constraint is gradually tightened until it reverts to the system-specified threshold, i.e., the aforementioned N GPUs. Specific methods for tightening the constraint include:
[0248] I. Continued tightening
[0249] After each iteration, the constraints are tightened until they reach a threshold. For example, if the system allocates 100 GPUs and the initial constraint is 200 GPUs, the constraints can be gradually tightened according to a preset ratio, such as decreasing by 10% each round, until the constraints meet the threshold.
[0250] II. Tightening in stages
[0251] Depending on the iteration rounds, different tightening ratios are used. For example, the first three rounds reduce by 20% each, and the last four rounds reduce by 10% each, until the constraint condition meets the threshold value.
[0252] III. Tightening the intervals
[0253] Instead of tightening the constraints in every iteration, you can tighten them once every few iterations until the constraints meet the threshold value.
[0254] IV. Adaptation Changes Drive Tightening
[0255] When the fitness distribution changes, the constraints are adjusted according to a preset magnitude or proportion. For example, when the fitness distribution is detected to be stabilizing or reaching a certain preset condition, the constraints are tightened by a certain proportion until the constraints meet the threshold value.
[0256] In this embodiment, the constraints are appropriately relaxed in the early stages of population iteration to reduce the generation of infeasible solutions and the omission of potential high-quality solutions caused by prematurely imposing strict restrictions. As the iteration process progresses into the later stages, convergence is performed on the high-quality individuals generated later, ensuring that exploration resources are efficiently allocated at different stages of the algorithm and achieving a balance between global and local search.
[0257] 607. Generate a new population.
[0258] Specifically, after generating offspring through crossover, mutation is typically performed on these offspring. Mutation involves randomly selecting one or more parameters of an individual and altering them to simulate genetic variation, thereby increasing population diversity. This randomness ensures that the algorithm can explore new regions of the solution space during the search process.
[0259] For example, suppose the population in the i-th generation includes an "individual A", and the values of the parallel strategies (TP, DP, PP, EP, SP) in "individual A" are (2, 4, 5, 6, 2). Then, a mutation operation is performed on "individual A", randomly selecting one parameter to mutate, for example, changing the value of the third strategy PP to 3, generating a mutated individual "individual a" (2, 4, 3, 6, 2); or, the parameters are adjusted proportionally, for example, multiplying the value of TP in "individual A" by 2 and dividing the value of DP by 2, obtaining a mutated individual "individual a" (4, 2, 3, 6, 2).
[0260] Then, the fitness of the mutated individual "individual a" is evaluated. If its fitness does not decrease significantly and the computational resources required by "individual a" meet the constraints of the i-th round, then the updated population retains "individual a".
[0261] 608. Output the individual with the highest fitness.
[0262] After terminating the iteration, the set of parallel strategies with the highest fitness (i.e. the shortest training time) is output. This set of parallel strategies is the optimal combination of parallel strategies that satisfies the constraints.
[0263] In one possible implementation, the set of parallel strategies generated in each iteration and its corresponding fitness value are recorded, and the set with the highest fitness is selected as the target set of parallel strategies.
[0264] In one possible implementation, after each update, the set of parallel policies with the highest fitness in the current population is recorded in the candidate set. After the iteration stops, the set of parallel policies with the highest fitness is selected from the candidate set as the target set of parallel policies. Since this scheme only focuses on the optimal solution, this approach can reduce unnecessary storage and computational overhead, and more directly and efficiently determine the set of parallel policies with the highest fitness.
[0265] After selecting the optimal set of target parallel strategies, the target parallel strategies are output to the user, for example, the value results of each parallel strategy in the target parallel strategy set are displayed on the output interface.
[0266] After the user obtains the target parallel policy set, the method also includes: training a model based on the target parallel policy.
[0267] In this embodiment, an incremental constraint strengthening mechanism is adopted, which gradually guides the search process to converge to the feasible region under relatively relaxed conditions. This can avoid generating infeasible solutions or skipping some potential good solutions by directly applying strict resource constraints.
[0268] To optimize the training performance of large models, performance profiling reports can be obtained through performance acquisition tools. However, the use of performance acquisition and analysis tools requires professional knowledge, and the analysis process is tedious and complex, relying on expert experience.
[0269] Based on this, this application also provides a data processing method for performance analysis optimization. Please refer to Figure 7, which is a flowchart illustrating a data processing method provided in an embodiment of this application. The method includes:
[0270] 701. Obtain the performance report of the first model training process.
[0271] Specifically, systems used for training models typically come pre-loaded with performance acquisition tools (such as Perfetto, TensorFlow, Ascend Insight, PyTorch Profiler, etc.). By configuring the performance acquisition code, you can obtain performance reports of the first model during the training process.
[0272] Performance acquisition code is used to instruct performance acquisition tools to set acquisition targets, configuration parameters, etc. These configurations typically include the acquisition time interval, the performance metrics to be detected (such as CPU utilization, memory usage, GPU usage, etc.), and the data storage location and format.
[0273] The output performance report typically includes: basic information such as the purpose and time of data collection, as well as performance metrics such as CPU usage, memory consumption, GPU usage, and network bandwidth during model training.
[0274] In one possible implementation, the configuration performance acquisition code is configured after multi-step reflective correction of the initial configuration result, and this initial configuration result is obtained through retrieval augmented generation (RAG) technology of the second model. This second model can be a Large Language Model (LLM).
[0275] Specifically, by inputting historical performance acquisition code from the knowledge base and the performance acquisition configuration information of the first model into the LLM, an initial configuration result is generated. The model's performance acquisition configuration information includes the user's personalized requirements. The aforementioned multi-step reflection includes:
[0276] I. Reflecting on whether the output results of LLM are illusory.
[0277] ① Determine whether the output results conform to the specifications and standards of historical performance collection codes in the knowledge base.
[0278] ② Determine whether the output results meet the user's needs, that is, determine whether the output results match the model's performance acquisition configuration information.
[0279] II. Reflecting on whether we can optimize details based on configuration recommendations.
[0280] Explore whether performance can be improved by recommending specific configuration optimization details. Configuration recommendations refer to configuration recommendations for performance acquisition tools, such as suggesting that both configuration A and configuration B of the performance acquisition tool be started at the same time to accelerate running efficiency.
[0281] In this embodiment, by using multi-step reflective correction performance acquisition configuration code, the accuracy and quality of the generated configuration can be improved, and the personalized needs of users can be taken into account, thus optimizing the output results of LLM.
[0282] In one possible implementation, prior to step 701, the method further includes:
[0283] 700. Train the first model based on the objective parallel policy set.
[0284] In this embodiment, the target parallel strategy set can be the optimal parallel strategy set obtained through the embodiment shown in Figure 5 above, or it can be a parallel strategy set configured by the user. No specific restrictions are imposed here.
[0285] 702. Based on the performance report, generate the first optimization information.
[0286] Analyzing performance reports can identify the primary code snippet causing performance loss in the model, or pinpoint bottlenecks that occur during training based on the target parallel strategy set. Correspondingly, the initial optimization information includes code optimization schemes for the model and / or optimization schemes for the target parallel strategy set.
[0287] Please refer to Figure 8, which illustrates the implementation of the code optimization scheme for the generated model. It includes:
[0288] 801. Based on the performance report, identify the first code snippet, which is the code snippet in the model that causes performance loss.
[0289] For example, by analyzing the response time in the performance report, if the response time of a certain code segment during execution exceeds a preset threshold, that code segment is considered a factor affecting the overall system performance. The first code segment determined based on the performance report is shown in Figure 9, which is a schematic diagram of the first code segment.
[0290] 802. Integrate multiple modal representations of the first code segment.
[0291] In this embodiment, the multiple model representations of the first code segment include semantic representation, structural representation, runtime feature representation, contextual representation, and performance bottleneck type representation.
[0292] Please refer to Figure 10, which is a schematic diagram of obtaining a unified representation by fusing multiple modal representations of the first code segment through a fusion network. Referring to the first code segment shown in Figure 9, the representations of each modality of the first code segment are described below:
[0293] I. Semantic Representation
[0294] By utilizing the embedding layer of the large code model, we generate a d-dimensional semantic embedding for the code snippet. For example, the semantic representation output of the first code snippet might be a vector of the form [0.12, -0.34, ..., 0.56], where each value represents a feature of the code snippet in a certain semantic dimension.
[0295] II. Structural Characterization
[0296] An abstract syntax tree (AST) is generated by parsing the code, and then processed using a graph neural network (GNN) to obtain a dg-dimensional structural embedding. The AST may contain nodes such as ForLoop, RangeCall, and LengthFunction. After GNN processing, the structural embedding is mapped to a d-dimensional vector, for example, [0.23, 0.45, ..., -0.12]. This mapping process is implemented through a code structure projection layer.
[0297] III. Characterization of Motion Features (or Dynamic Characterization)
[0298] A dp-dimensional dynamic feature embedding is generated using a profiling encoder. Data is transmitted 10 times with an average latency of 5ms and a data size of 100MB. The dynamic feature embedding is defined as [10, 5, 100, ...]. The dynamic feature embedding is mapped to a d-dimensional vector through a dynamic feature projection layer.
[0299] IV. Contextual Representation
[0300] By combining the contextual information of the code snippet (such as other code and comments in the same file), a d-dimensional context-aware embedding is generated using the embedding layer of the large code model. This embedding can reflect the position and role of the code snippet in a larger code environment, such as [0.2, -0.3, ..., 0.1].
[0301] V. Characterization of Performance Bottleneck Types
[0302] For specific performance bottleneck tasks (such as data transfer blocking between CPU and GPU), relevant features (such as blocking time and number of blocking events) are defined, and a d-dimensional task-specific embedding is generated through the embedding layer of the large code model. For example, for data transfer blocking tasks, the embedding may be of the form [50, 3, ...].
[0303] Then, the above 5 d-dimensional embeddings are processed by a transformer (from a pre-trained large language model (LLM)) to integrate information from different modalities, outputting 5 d-dimensional embeddings. The 5 d-dimensional embeddings are then fused by a fusion network to form the final multimodal embedding, outputting a dm-dimensional unified code representation embedding.
[0304] For example, suppose we have the aforementioned five features, such as code structure and code context information. After processing in the previous steps, we obtain [l, d], [m, d], [n, d], [i, d], and [j, d], where l, m, n, i, and j are the number of tokens for the five features, and each token has a dimension of d. Concatenating these tokens yields [l + m + n + i + j, d], with a total token count c = l + m + n + i + j. Inputting a [c, d]-sized token into a transformer layer outputs a [c, d]-sized token, which is then input into a fusion network. The operation in the fusion network can be considered as multiplying the input [c, d] by [1, c] to output a [1, d]-sized token.
[0305] 803. Obtain the second code snippet with the highest similarity.
[0306] In the multimodal embedding space, the similarity between the code unified representation embedding of the code to be analyzed and historical cases is calculated, and the case with the highest similarity is returned, which is the second code fragment.
[0307] 804. Code optimization scheme for generating the first model
[0308] Specifically, based on the second code snippet, a code optimization scheme for the first model is generated through a large code model.
[0309] For example, in the model's code, function A performs an all-reduce operation on each input parameter, leading to frequent network communication and low resource utilization. An optimized solution could be to add a bucket to function A to temporarily store input parameters. When the number of parameters in the bucket reaches a certain threshold or a certain latency condition is met, then an all-reduce operation is performed on these temporarily stored input parameters.
[0310] In this embodiment, multi-embedded retrieval is performed by fusing multiple embedding representations of the code, which enhances the model's understanding of the code context and intent, thereby improving the accuracy of code case retrieval.
[0311] For the optimization of the target parallel strategy set, for example, the original parallel strategy selected 16 GPUs for tensor parallelism (TP). However, performance report analysis revealed a significant difference in the weights of odd-numbered and even-numbered GPUs when processing tensor parallelism, leading to an unbalanced load. This imbalance caused some GPUs to experience latency while processing tasks, thus affecting the overall parallel efficiency. Therefore, the final optimization solution for the parallel strategy could be to change the TP value to 8 GPUs and reconfigure the parallel environment.
[0312] The data processing method provided in this application is applied to an optimization system for model training. Please refer to Figure 11, which is a schematic diagram of the optimization system. The optimization system comprises three modules: a parallel strategy recommendation module, a performance acquisition and configuration module, and a performance analysis and optimization module. The three modules of the optimization system are described below with reference to Figure 11:
[0313] I. Parallel Strategy Recommendation Module:
[0314] The parallel strategy recommendation module is primarily used to output the optimal combination of parallel strategies for the model to be trained. Specifically, it acquires user-input information such as the large model network structure, training configuration, parallel strategy configuration, ensemble communication algorithm, and bandwidth; models and evaluates parallel efficiency; and uses the data processing method provided in this application's embodiments to perform parallel strategy training using a hierarchical selection and progressive constraint reinforcement genetic strategy, outputting the optimal set of parallel strategies and corresponding analysis results. This includes multiple steps such as card factorization, multi-constraint modeling, and heuristic search.
[0315] For example, users can flexibly select and configure some of the parallel strategy parameters. For the unconfigured parameters, the system will recommend the optimal parallel strategy value while meeting the user's specific needs or constraints. Users can also interact with the system multiple times based on the recommended parallel strategy to adjust the parameter values and complete analysis tasks using different parallel strategies. Intermediate results of the genetic strategy are cached. During multiple interactions, based on the user's parallel strategy constraints, the system retrieves strategy combinations that meet the constraints from the cache and uses the best result as the new initial population, instead of starting with a random population, thus improving search efficiency.
[0316] As shown in Figure 11, after obtaining the optimal set of parallel strategies, users can train models based on this set of parallel strategies, including communication mode modeling and multi-level communication modeling, and analyze the parallel communication time consumption during the model training process to generate corresponding performance reports (profiling) for user reference.
[0317] II. Performance Acquisition Configuration Module:
[0318] The performance acquisition configuration module is used to generate performance acquisition configuration code. The system pre-configures common performance acquisition tools, including Perfetto, TensorFlow, Ascend Insight, and PyTorch Profiler. After receiving the user's performance acquisition request, the system searches the knowledge base and generates performance acquisition configuration code based on the search results of the large language model. In this application, the performance acquisition configuration code generated by the large model is reviewed and corrected through multiple steps, combining the knowledge base, the user's performance acquisition requirements, and the recommended configurations of the selected performance acquisition tools. The multi-step review process is as follows: first, reviewing whether it conforms to the knowledge base; second, reviewing whether it meets the user's needs; and third, reviewing to supplement details and improve best practices. The system outputs the corresponding performance acquisition configuration code and instructions, guiding users to add the acquisition and profiling configuration code to their source code.
[0319] In addition, users can further interact and request modifications to the profiling configuration.
[0320] III. Performance Analysis and Optimization Module:
[0321] The performance analysis and optimization module generates corresponding optimization suggestions by analyzing performance reports, including code optimization for training models and parallel strategy optimization. Specifically, it acquires performance data and training code, analyzes standardized performance data, identifies the TOP-K performance-affecting code locations, retrieves similar cases from the historical experience library based on multimodal embedding and context-aware mechanisms, and adds the problematic code and similar cases to a prompt to analyze the impact points, optimization schemes, and performance gains after optimization for the large-scale code model.
[0322] Optionally, users can interact with the system-called code model multiple times to further confirm issues and choose optimization measures independently. To protect users' critical product code assets from leakage, the system-called code model can also be configured as a locally instantiated code model.
[0323] The methods provided in the embodiments of this application have been described in detail above. Next, the device for performing the above methods provided in the embodiments of this application will be described.
[0324] Please refer to Figure 12, which is a schematic diagram of the structure of a model training device 1200 provided in an embodiment of this application. As shown in Figure 12, the device includes:
[0325] The receiving module 1201 is used to receive the configuration information of the model input by the user;
[0326] The generation module 1202 is used to generate a population based on the configuration information. The population includes multiple parallel policy sets, which are used to train the model.
[0327] The hierarchical module 1203 is used to divide multiple parallel policy sets into multiple clusters based on the fitness of each parallel policy set in multiple parallel policy sets. The fitness is used to indicate the training time of the model based on the parallel policy sets. Each cluster includes at least one parallel policy set.
[0328] The processing module 1204 is used to update the population through crossover operations until the updated population meets the target conditions. The target parallel strategy set is determined from the updated population that meets the target conditions. The target parallel strategy set is the parallel strategy set with the shortest training time in the updated population. The crossover operation includes intra-cluster crossover and inter-cluster crossover. Intra-cluster crossover indicates that some parameters of each parallel strategy set in multiple parallel strategy sets are selected from the same cluster and combined to generate a new parallel strategy set. Inter-cluster crossover indicates that some parameters of each parallel strategy set in multiple parallel strategy sets are selected from different clusters and combined to generate a new parallel strategy set.
[0329] Feedback module 1205 is used to provide the user with a set of target parallel strategies.
[0330] In one possible implementation, the layered module 1203 is specifically used for:
[0331] Multiple fitness ranges are determined based on the fitness of multiple parallel policy sets, and the multiple fitness ranges do not overlap;
[0332] Multiple parallel policy sets are divided into multiple clusters based on multiple fitness ranges, wherein each cluster contains at least one parallel policy set belonging to the same fitness range.
[0333] In one possible implementation, during the population update process via crossover, processing module 1204 is further configured to:
[0334] Multiple fitness ranges are updated based on the fitness of the parallel policy set in the updated population, and multiple clusters are re-divided using the updated fitness ranges.
[0335] In one possible implementation, during the population update process via crossover, the computational resource threshold of the updated population is less than that of the population before the update. The computational resource threshold is used to indicate the maximum amount of computational resources that any set of parallel policies in the population is allowed to use.
[0336] In one possible implementation, the processing module 1204 is specifically used for:
[0337] Obtain the target parallel policy set from the candidate set. The candidate set includes the parallel policy set with the shortest training time in each updated population that meets the target condition. The target parallel policy set is the parallel policy set with the shortest training time in the candidate set.
[0338] In one possible implementation, the updated population satisfies the following target conditions:
[0339] The updated population corresponds to an update count greater than or equal to the first threshold; or,
[0340] The highest fitness of the updated population decreases by a factor greater than or equal to the second threshold compared to the highest fitness of the population before the update. The highest fitness is the maximum fitness of the parallel policy set in the population.
[0341] In one possible implementation, the model's configuration information includes one or more of the following:
[0342] The model's network structure;
[0343] Model training configuration information;
[0344] Parallel strategy configuration information for the model;
[0345] The model's collective communication algorithm; or,
[0346] The training communication bandwidth of the model.
[0347] In one possible implementation, the set of parallel strategies includes one or more of the following parallel strategies:
[0348] Tensor parallelism;
[0349] Data parallelism;
[0350] Parallel production lines;
[0351] Experts in parallel; or,
[0352] Sequence parallelism.
[0353] This application also provides a computing device 100. As shown in FIG13, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 100.
[0354] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 13, but this does not imply that there is only one bus or one type of bus. Bus 102 can include pathways for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).
[0355] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0356] Memory 106 may include volatile memory, such as random access memory (RAM). Processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).
[0357] The memory 106 stores executable program code, which the processor 104 executes to implement the functions of the aforementioned receiving model, generation module, layering module, processing module, and feedback module, thereby realizing the data processing method. In other words, the memory 106 stores instructions for executing the data processing method.
[0358] Alternatively, the memory 106 stores executable code, which the processor 104 executes to implement the functions of the aforementioned data processing device, thereby implementing the data processing method. That is, the memory 106 stores instructions for executing the data processing method.
[0359] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.
[0360] It should be understood that the computing device 100 provided according to this application may correspond to the data processing device 1200 in this application, and may correspond to the corresponding subject in executing the method shown in FIG5 according to the embodiment of this application. The above and other operations and / or functions of each module of the computing device 100 are respectively for implementing the corresponding processes of each method in FIG5 to FIG10. For the sake of brevity, they will not be described in detail here.
[0361] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.
[0362] As shown in Figure 14, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for executing data processing methods.
[0363] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing data processing methods. In other words, a combination of one or more computing devices 100 can jointly execute instructions for executing data processing methods.
[0364] It should be noted that the memory 106 in different computing devices 100 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the data processing device. That is, the instructions stored in the memory 106 of different computing devices 100 can implement the functions of one or more of the receiving model, generation module, layering module, processing module, and feedback module.
[0365] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 15 illustrates one possible implementation. As shown in Figure 15, two computing devices 100A and 100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 106 in computing device 100A stores instructions for performing the functions of the receiving module and the generating module. Simultaneously, the memory 106 in computing device 100B stores instructions for performing the functions of the hierarchical module, the processing module, and the feedback module.
[0366] It should be understood that the functions of computing device 100A shown in Figure 15 can also be performed by multiple computing devices 100. Similarly, the functions of computing device 100B can also be performed by multiple computing devices 100.
[0367] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a data processing method.
[0368] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a data processing method.
[0369] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0370] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.
Claims
1. A data processing method, characterized in that, include: Receive model configuration information input by the user; A population is generated based on the configuration information, and the population includes multiple parallel policy sets, which are used to train the model. Based on the fitness of each of the multiple parallel policy sets, the multiple parallel policy sets are divided into multiple clusters, wherein the fitness is used to indicate the training time of the model based on the parallel policy sets, and each cluster includes at least one of the parallel policy sets. The population is updated through crossover operations until the updated population meets the target conditions. A target parallel strategy set is determined from the updated population that meets the target conditions. The target parallel strategy set is the parallel strategy set with the shortest training time in the updated population. The crossover operations include intra-cluster crossover and inter-cluster crossover. Intra-cluster crossover indicates that some parameters of each parallel strategy set in multiple parallel strategy sets are selected from the same cluster and combined to generate a new parallel strategy set. Inter-cluster crossover indicates that some parameters of each parallel strategy set in multiple parallel strategy sets are selected from different clusters and combined to generate a new parallel strategy set. The target parallel strategy set is fed back to the user.
2. The method according to claim 1, characterized in that, The step of dividing the multiple parallel strategy sets into multiple clusters based on the fitness of each parallel strategy set in the multiple parallel strategy sets includes: Multiple fitness ranges are determined based on the fitness of the multiple parallel policy sets, and the multiple fitness ranges do not overlap. The multiple parallel policy sets are divided into multiple clusters based on the multiple fitness ranges, wherein at least one parallel policy set contained in each cluster belongs to the same fitness range.
3. The method according to claim 2, characterized in that, In updating the population through the crossover operation, the method further includes: The fitness ranges are updated based on the fitness of the parallel policy set in the updated population, and the multiple clusters are re-divided using the updated fitness ranges.
4. The method according to any one of claims 1-3, characterized in that, During the process of updating the population through the crossover operation, the computational resource threshold of the updated population is less than the computational resource threshold of the population before the update, the computational resource threshold being used to indicate the maximum amount of computational resources allowed to be used by any set of parallel strategies in the population.
5. The method according to any one of claims 1-4, characterized in that, The step of obtaining the target parallel policy set from the updated population that satisfies the target condition includes: Obtain a target parallel strategy set from the candidate set, the candidate set including the parallel strategy set with the shortest training time in each updated population that satisfies the target condition, and the target parallel strategy set is the parallel strategy set with the shortest training time in the candidate set.
6. The method according to any one of claims 1-5, characterized in that, The updated population satisfies the following target conditions: The updated population corresponds to an update count greater than or equal to the first threshold; or, The highest fitness of the updated population decreases by a factor greater than or equal to a second threshold relative to the highest fitness of the population before the update, where the highest fitness is the maximum fitness of the parallel policy set in the population.
7. The method according to any one of claims 1-6, characterized in that, The configuration information of the model includes one or more of the following: The network structure of the model; The training configuration information of the model; The parallel strategy configuration information of the model; The model's set communication algorithm; or, The training communication bandwidth of the model.
8. The method according to any one of claims 1-7, characterized in that, The set of parallel strategies includes one or more of the following parallel strategies: Tensor parallelism; Data parallelism; Parallel production lines; Experts in parallel; or, Sequence parallelism.
9. A data processing apparatus, characterized in that, include: The receiving module is used to receive the model configuration information input by the user; A generation module is used to generate a population based on the configuration information. The population includes multiple parallel policy sets, which are used to train the model. A hierarchical module is used to divide the multiple parallel policy sets into multiple clusters based on the fitness of each parallel policy set in the multiple parallel policy sets. The fitness is used to indicate the training time of the model based on the parallel policy sets. Each cluster includes at least one of the parallel policy sets. The processing module is used to update the population through crossover operations until the updated population meets the target conditions, and to determine the target parallel policy set from the updated population that meets the target conditions. The target parallel policy set is the parallel policy set with the shortest training time in the updated population. The crossover operations include intra-cluster crossover and inter-cluster crossover. Intra-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from the same cluster and combined to generate a new parallel policy set. Inter-cluster crossover indicates that some parameters of each parallel policy set in multiple parallel policy sets are selected from different clusters and combined to generate a new parallel policy set. The feedback module is used to provide feedback on the target parallel strategy set to the user.
10. The apparatus according to claim 9, characterized in that, The hierarchical module is specifically used for: Multiple fitness ranges are determined based on the fitness of the multiple parallel policy sets, and the multiple fitness ranges do not overlap. The multiple parallel policy sets are divided into multiple clusters based on the multiple fitness ranges, wherein at least one parallel policy set contained in each cluster belongs to the same fitness range.
11. The apparatus according to claim 10, characterized in that, During the process of updating the population through the crossover operation, the processing module is further configured to: The fitness ranges are updated based on the fitness of the parallel policy set in the updated population, and the multiple clusters are re-divided using the updated fitness ranges.
12. The apparatus according to any one of claims 9-11, characterized in that, During the process of updating the population through the crossover operation, the computational resource threshold of the updated population is less than the computational resource threshold of the population before the update, the computational resource threshold being used to indicate the maximum amount of computational resources allowed to be used by any set of parallel strategies in the population.
13. The apparatus according to any one of claims 9-12, characterized in that, The processing module is specifically used for: Obtain a target parallel strategy set from the candidate set, the candidate set including the parallel strategy set with the shortest training time in each updated population that satisfies the target condition, and the target parallel strategy set is the parallel strategy set with the shortest training time in the candidate set.
14. The apparatus according to any one of claims 9-13, characterized in that, The updated population satisfies the following target conditions: The updated population corresponds to an update count greater than or equal to the first threshold; or, The highest fitness of the updated population decreases by a factor greater than or equal to a second threshold relative to the highest fitness of the population before the update, where the highest fitness is the maximum fitness of the parallel policy set in the population.
15. The apparatus according to any one of claims 9-14, characterized in that, The configuration information of the model includes one or more of the following: The network structure of the model; The training configuration information of the model; The parallel strategy configuration information of the model; The model's set communication algorithm; or, The training communication bandwidth of the model.
16. The apparatus according to any one of claims 9-15, characterized in that, The set of parallel strategies includes one or more of the following parallel strategies: Tensor parallelism; Data parallelism; Parallel production lines; Experts in parallel; or, Sequence parallelism.
17. A computing device, characterized in that, Including processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1 to 8.
18. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 8.
19. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1 to 8.
20. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 8.