Model processing method and device, equipment, storage medium and computer program product

Abstract

The invention provides a model processing method and device, equipment, a storage medium and a computer program product. The method comprises the steps that the nth module is determined from a to-be-migrated model, the nth module comprises at least one operator, and n is a positive integer increasing in sequence; adding the nth module into the model deployed in the first hardware acceleration equipment to obtain a first model, and adding the nth module into the model deployed in the second hardware acceleration equipment to obtain a second model; performing model training on the first model to obtain a first model training result, and performing model training on the second model to obtain a second model training result; determining a first precision difference between the first model and the second model based on a difference between the first model training result and the second model training result; and when the first precision difference is greater than a first preset threshold value, determining a target operator with abnormal precision from at least one operator contained in the nth module.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model processing method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] Artificial Intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. Deep learning, a core research area of ​​machine learning based on artificial neural networks, focuses on large-scale parallel computing on hardware-accelerated devices to achieve efficient training and inference. To fully utilize hardware performance and maintain technological advancement, mature neural network models often need to be migrated from one hardware acceleration device to another. However, due to inherent differences in the underlying implementation, computational logic, and precision control of different hardware acceleration devices, inconsistent training results often occur after model migration. In complex neural network models, these small computational errors caused by hardware differences accumulate and amplify with each training iteration, making troubleshooting extremely difficult. Related technologies generally face the challenge of low efficiency in locating accuracy anomalies. Summary of the Invention

[0003] This application provides a model processing method, apparatus, device, storage medium, and computer program product that can achieve efficient operator-level precision anomaly localization.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a model processing method, the method comprising: The nth module is determined from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increases sequentially. The nth module is added to the model deployed on the first hardware acceleration device to obtain the first model, and the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model; The first model is trained to obtain the first model training result, and the second model is trained to obtain the second model training result; Based on the difference between the training results of the first model and the training results of the second model, a first accuracy difference between the first model and the second model is determined. When the first precision difference is greater than the first preset threshold, the target operator with precision abnormality is determined from at least one operator included in the nth module.

[0005] This application provides a model processing apparatus, the apparatus comprising: The model decomposition module is used to determine the nth module from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increases sequentially. The model deployment module is used to add the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and to add the nth module to the model deployed on the second hardware acceleration device to obtain the second model; The model training module is used to train the first model to obtain the first model training result, and to train the second model to obtain the second model training result. The difference determination module is used to determine a first accuracy difference between the first model and the second model based on the difference between the training results of the first model and the training results of the second model. The difference determination module is further configured to determine, when the first precision difference is greater than a first preset threshold, a target operator with an abnormal precision from at least one operator included in the nth module.

[0006] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the model processing method provided in the embodiments of this application.

[0007] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the model processing method provided in this application when executed by a processor.

[0008] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the model processing method provided in this application.

[0009] The embodiments of this application have the following beneficial effects: By incrementally identifying the nth module from the model to be transferred and simultaneously adding it to two models deployed on the first and second hardware acceleration devices respectively, a first model and a second model for parallel training are constructed. This module-by-module addition and dual-path parallel architecture enables fine-grained controllability and observability of the model transfer process. Subsequently, the two synchronously constructed models are trained separately, and the first precision difference between the first and second models is precisely quantified based on the difference between their training results. The first precision difference directly reflects the degree of precision deviation introduced by hardware differences. When the first precision difference is detected to exceed a first preset threshold, the target operator causing the precision anomaly is accurately located from at least one operator contained in the current nth module. This achieves a progressive investigation and location from macroscopic precision differences to microscopic abnormal operators. This mechanism not only makes the originally black-box model transfer process transparent, allowing precision problems to be captured in time at the initial stage of module addition, but more importantly, by finely screening the operators within the module, the fault is directly located at the specific operator level, thereby achieving efficient location of precision anomaly problems at the operator level. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the architecture of the model processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of an electronic device for model processing provided in an embodiment of this application; Figure 3 This is a first flowchart illustrating the model processing method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the second process of the model processing method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the third process of the model processing method provided in the embodiments of this application; Figure 6 This is a schematic diagram of the fourth process of the model processing method provided in the embodiments of this application; Figure 7 This is a schematic diagram of the fifth process of the model processing method provided in the embodiments of this application; Figure 8 This is a first schematic diagram of the model processing method provided in the embodiments of this application; Figure 9 This is a schematic diagram of the sixth process of the model processing method provided in the embodiments of this application.

[0011] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0015] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0016] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0017] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0018] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0019] 1) Hardware Acceleration Devices: Hardware acceleration devices refer to computer systems that integrate dedicated hardware units to improve the execution efficiency of specific computing tasks. They typically include at least one hardware accelerator, a general-purpose processor, storage media, and input / output modules. Hardware acceleration devices offload computationally intensive tasks (such as matrix operations in neural networks) to the hardware accelerator for execution, while utilizing the general-purpose processor to handle control logic and non-intensive computing tasks. This allows them to achieve computing performance far exceeding that of relying solely on a general-purpose processor. In the field of artificial intelligence, hardware acceleration devices are a key infrastructure supporting the efficient training and inference of large-scale neural network models. Hardware acceleration devices from different manufacturers or models may differ in architecture and implementation.

[0020] 2) Hardware accelerators: Hardware accelerators refer to hardware components in hardware acceleration devices that are specifically designed to perform specific types of computing tasks to improve processing speed and energy efficiency. Examples include graphics processing units (GPUs), tensor processing units (TPUs), or field-programmable gate arrays (FPGAs). The design focus of hardware accelerators is to efficiently execute operations that frequently occur in neural network operations, such as matrix multiplication and convolution, through massively parallel processing architectures. Due to their specialized design, hardware accelerators significantly outperform general-purpose processors when processing these specific tasks.

[0021] 3) General-purpose processor: A general-purpose processor refers to the central processing unit (CPU) in a hardware acceleration device that is used to perform general-purpose computing tasks and control logic. Its design goal is to efficiently process various types of instructions and data, and to have the ability to handle complex control flow, operating system management and serial tasks. In the training and inference process of neural network models, the general-purpose processor is usually responsible for data preprocessing, task scheduling, communication with external devices and execution of some computing logic that cannot or is not suitable to run on hardware accelerators. Since the industry has formed a relatively unified standard on the architecture and instruction set of general-purpose processors, the computing logic and accuracy of the general-purpose processors deployed on different hardware acceleration devices are usually highly consistent.

[0022] 4) Model Training: In the field of deep learning, model training refers to the process of iteratively adjusting the internal parameters of a neural network model by inputting a large amount of labeled training data and using specific algorithms (e.g., backpropagation and gradient descent). The goal of this process is to enable the model to learn potential patterns or rules in the data, so that it can make accurate predictions or classifications when receiving new and unseen data. Model training is usually a computationally intensive process involving a large number of matrix and tensor operations, so it is usually performed on hardware acceleration devices to shorten the training cycle. The quality of the training results is evaluated by the loss function, and the ultimate goal of model training is to minimize the value of the loss function.

[0023] 5) Operator: An operator is the smallest functional unit in deep learning frameworks and neural network models used to describe a basic computational operation. It can be regarded as the basic building block of complex neural network computation graphs. Each operator performs a specific mathematical or logical operation, such as convolution, activation, pooling, or matrix multiplication. A complete neural network model is composed of many operators connected according to a specific topology. The specific implementation logic and computational accuracy of the operator directly affect the final output of the model. On different hardware accelerators, the underlying implementation of the same operator may differ, which may become a potential source of accuracy problems during model transfer.

[0024] This application provides a model processing method, apparatus, device, computer-readable storage medium, and computer program product that can achieve efficient operator-level precision anomaly localization.

[0025] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the model processing system provided in this application embodiment. In the model processing system 10 provided in this application embodiment, in order to support a model processing application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0026] Terminal 400 can be used to obtain model processing requests carrying the model to be migrated.

[0027] In some embodiments, a model processing plugin may be embedded in the client running in the terminal 400 to implement the model processing method locally on the client. For example, the terminal 400 calls the model processing plugin to implement the model processing method, which involves determining the nth module from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increments sequentially; adding the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and adding the nth module to the model deployed on the second hardware acceleration device to obtain the second model; training the first model to obtain the first model training result, and training the second model to obtain the second model training result; determining the first precision difference between the first model and the second model based on the difference between the first model training result and the second model training result; and when the first precision difference is greater than a first preset threshold, identifying the target operator with an abnormal precision from the at least one operator contained in the nth module.

[0028] In some embodiments, after the terminal 400 obtains a model processing request carrying the model to be migrated, it calls the model processing interface of the server 200 (which can be provided as a cloud service, i.e., a model processing service). The server 200 implements a model processing method through a model processing plugin, determining the nth module from the model to be migrated, wherein the nth module contains at least one operator, and n is a positive integer that increments sequentially; adding the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and adding the nth module to the model deployed on the second hardware acceleration device to obtain the second model; training the first model to obtain the first model training result, and training the second model to obtain the second model training result; determining the first precision difference between the first model and the second model based on the difference between the first model training result and the second model training result; when the first precision difference is greater than a first preset threshold, determining the target operator with an abnormal precision from the at least one operator contained in the nth module, and returning the target operator to the terminal 400.

[0029] The model processing method of this application embodiment can be applied to various application scenarios in various fields, as detailed below.

[0030] For example, in the field of autonomous driving, when migrating a fully validated vehicle perception model from a dedicated server in the R&D stage to the embedded computing unit of a mass-produced vehicle, the accuracy of the model output often drifts due to differences in the hardware architecture of the two. For instance, there may be slight deviations in the recognition coordinates of obstacles. These deviations can be amplified at high speeds, posing a serious safety hazard. To address the aforementioned problems in related technologies, in this embodiment, the R&D engineer's terminal workstation (i.e., terminal 400) uploads the model to be migrated to a cloud verification server (i.e., server 200). The cloud verification server determines the nth module from the model to be migrated, and then adds the nth module to the model deployed on the first hardware acceleration device to obtain the first model. The nth module is then added to the model deployed on the second hardware acceleration device. After obtaining the second model, the cloud verification server trains the first model to obtain the first model training result, and then trains the second model to obtain the second model training result. Based on the difference between the first and second model training results, the cloud verification server determines the first precision difference between the first and second models. When the first precision difference is greater than a preset safety threshold (i.e., the first preset threshold), the cloud verification server identifies the target operator with precision anomalies from at least one operator contained in the nth module and returns a detailed diagnostic report containing the location of the target operator to the terminal workstation. In this process, by performing incremental verification module by module and accurately locating problems before model deployment, clear modification guidelines are provided to terminal engineers, significantly improving the efficiency of model migration and the final operational security.

[0031] For example, in the field of intelligent medical image analysis, a certified AI model for assisted diagnosis faces the risk of inconsistent diagnostic results when deployed on newly purchased imaging equipment in hospitals due to differences between the hardware accelerators built into the new equipment and the original development environment. This directly affects the accuracy of diagnosis and patient safety. To solve this technical problem, in this embodiment, the terminal computer (i.e., terminal 400) of the medical device manufacturer or the hospital's information department uploads the certified AI model to be migrated to a remote compliance test server (i.e., server 200). The remote compliance test server determines the nth module from the model to be migrated. Subsequently, the remote compliance test server adds the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and adds the nth module to the model deployed on the second hardware acceleration device. In the model, a second model is obtained. Then, the remote compliance testing server trains the first model to obtain the training result of the first model, and trains the second model to obtain the training result of the second model. Then, based on the difference between the training results of the first model and the training results of the second model, the remote compliance testing server determines the first precision difference between the first model and the second model. When the first precision difference is greater than the first preset threshold of compliance requirements, the remote compliance testing server determines the target operator with precision abnormality from at least one operator contained in the nth module. Finally, a non-compliance report and the problem operator location information are generated and sent back to the terminal computer. This realizes the automation and high-precision verification of the model migration process, ensures the consistency of the functional performance of the medical AI model on different hardware, and guarantees the reliability of medical diagnosis.

[0032] In some embodiments, server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0033] Terminal 400 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0034] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device for model processing provided in an embodiment of this application. Figure 2The electronic device 500 shown can be Figure 1 The terminal 400 or server 200, and the electronic device 500 include: at least one processor 510, memory 550, and at least one network interface 520. The various components in server 200 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0035] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0036] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls; In some embodiments, when the model processing task is completed independently by the terminal 400, the server 200 provided in this application embodiment does not include the user interface 530.

[0037] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0038] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0039] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0040] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 553 is used to enable the presentation of information (e.g., user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with user interface 530. In some embodiments, when the model processing task is completed independently by the terminal 400, the server 200 provided in this application embodiment may not include the presentation module 553.

[0041] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532; in some embodiments, when the model processing task is performed independently by the terminal 400, the server 200 provided in this application embodiment may not include the presentation module 553.

[0042] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A model processing device 555 stored in memory 550 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a model decomposition module 5551, a model deployment module 5552, a model training module 5553, and a difference determination module 5554. These modules are logically linked and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0043] As mentioned above, the electronic device implementing the model processing method of this application embodiment can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below. See [link to relevant documentation]. Figure 3 , Figure 3 This is a first flowchart illustrating the model processing method provided in the embodiments of this application, which will be combined with... Figure 3 The steps shown are explained.

[0044] In step 101, the nth module is determined from the model to be transferred.

[0045] The nth module contains at least one operator, where n is a positive integer that increments sequentially.

[0046] Here, the nth module refers to the smallest logical computation unit in the hidden layer of the model to be transferred that has a specific function and can be executed independently. The nth module can be a module composed of a single operator or a module encapsulated by multiple operators with computational dependencies. The hidden layer of the model to be transferred refers to the computational layer other than the output layer and the input layer, which is used to learn the intrinsic representation of the data. The input layer refers to the network structure that serves as the entry point of the model computation graph, responsible for receiving external raw data and converting it into the multidimensional tensor format required by subsequent computational layers. The output layer refers to the network structure located at the end of the model computation graph, responsible for converting the high-dimensional feature representation output by the last hidden layer into the final result that meets the specific task requirements (e.g., class probability, regression value, or generated sequence).

[0047] As an example, the nth module can be determined as follows: the model to be transferred is split into modules to obtain multiple modules, the detection priority of each module is determined, the detection priority is a quantitative indicator used to determine the detection order of modules, the multiple modules are sorted in descending order based on the detection priority of each module to obtain multiple sorted modules, and the nth module is determined from the multiple sorted modules.

[0048] In some embodiments, see Figure 4 , Figure 4 This is a schematic diagram of the second process of the model processing method provided in the embodiments of this application. Figure 3 Step 101 shown can be implemented through steps 1011 to 1013, as explained in detail below.

[0049] In step 1011, the model to be migrated is split into modules to obtain multiple modules.

[0050] As an example, a computation graph of the model to be migrated is constructed, and based on the computation graph of the model to be migrated, the model to be migrated is modularized to obtain multiple modules. Alternatively, based on the predefined functional components or architectural layers of the model to be migrated, a set of operators with specific functions can be identified as a module. Alternatively, a combination of operators specified by the user can be received through a human-computer interaction interface, and each received combination of operators can be identified as a module. No restrictions are imposed here.

[0051] In some embodiments, see Figure 5 , Figure 5 This is a schematic diagram of the third process of the model processing method provided in the embodiments of this application. Figure 4 Step 1011 shown can be implemented through steps 10111 to 10112, which are explained in detail below.

[0052] In step 10111, the computational graph of the model to be transferred is constructed.

[0053] In this computation graph, the nodes are the operators in the model to be transferred, and the edges are the data flow dependencies between the operators. The data flow dependency means that the output tensor of one computation unit (operator or module) is the input tensor of another computation unit. The computation graph of the model to be transferred can be constructed by parsing the static structure description file of the model to be transferred or by dynamically tracing the execution process of the model.

[0054] As an example, when constructing the computation graph of the model to be transferred by parsing the static structure description file of the model to be transferred, the static structure description file of the model to be transferred (e.g., an ONNX format model file) is obtained. The static structure description file of the model to be transferred defines multiple operators (NodeProto) and the input tensor and output tensor of each operator. Based on the type of each operator (e.g., unique identifiers of Conv, ReLU, and MatMul), attributes (e.g., kernel size and stride), unique identifiers of the input tensor and output tensor, the corresponding nodes are constructed. Based on the unique identifiers of the input tensor and output tensor of each operator, the data flow dependencies between the nodes corresponding to multiple operators are determined. For example, if the node corresponding to operator A is node A and the node corresponding to operator B is node B, and the identifier of the output tensor of operator A matches the identifier of the input tensor of operator B, then a directed edge from node A to node B is constructed in the computation graph.

[0055] As an example, when constructing the computation graph of the model to be transferred by dynamically tracking the execution process of the model, the training samples are forward-propagated based on the model to be transferred. Each operator called during the forward propagation process of the model to be transferred is recorded, and a node corresponding to each operator is created. Whenever the output tensor of one operator is used as the input tensor of another operator, an edge is established between the two nodes corresponding to the two operators. This continues until the complete forward propagation process of the model to be transferred for the training sample is completed. Thus, the computation graph of the model to be transferred is obtained.

[0056] In step 10112, the computation graph is divided into multiple subgraphs based on preset operator fusion rules, and multiple modules are determined based on the multiple subgraphs.

[0057] Among them, the operator fusion rule refers to the set of operator combination patterns defined based on the logical execution dependency between operators. Logical execution dependency means that multiple operators must be executed as a whole in terms of function. The operator combination pattern refers to the topological structure of multiple operators that must be called together.

[0058] As an example, based on the multiple operator combination patterns contained in the operator fusion rule, pattern recognition is performed on the computation graph. When a local structure in the computation graph is identified as matching any operator combination pattern in the operator fusion rule, the local structure is determined as a subgraph, and the operators contained in the subgraph constitute a module.

[0059] For example, when dealing with a Transformer model that includes a multi-head self-attention mechanism, an operator combination pattern can be defined as: "treating the three linear transformation operators (MatMul) used to compute the query (Q), key (K), and value (V), the subsequent scaled dot product attention operator (ScaledDotProductAttention), and the final output linear transformation operator (MatMul) as an indivisible whole." The mathematical principle behind this is that the core computational formula of the self-attention mechanism can be represented as... In this formula, Q, K, and V are strongly coupled, and the input of the softmax operation depends on the dot product of Q and K. The absence of any part will result in an incomplete calculation logic of the entire attention score, thus making it impossible to perform effective accuracy verification or transfer as an independent module. Therefore, after applying this operator combination mode, all operator nodes that constitute a complete attention head in the computation graph will be automatically delineated into a subgraph.

[0060] In this embodiment, a computation graph is constructed with operators in the model to be migrated as nodes and data flow dependencies as edges, thereby realizing a graphical analysis of the model structure. Based on this, the computation graph is divided into multiple subgraphs according to preset operator fusion rules, so that each module after division corresponds to a subgraph in the computation graph. This division method aggregates operators with close data dependencies in the same subgraph, ensuring the integrity of data interaction between operators within the module. At the same time, the preset rules realize the standardization and automation of the division, so that the modular decomposition of the model can accurately reflect its original computation logic and data flow structure.

[0061] In some embodiments, "dividing the computation graph into multiple subgraphs and determining multiple modules based on the multiple subgraphs" can be achieved through the following process: Independent execution verification is performed on the operators corresponding to each node in the computation graph. When the operator corresponding to a node passes the independent execution verification, the operator corresponding to the node constitutes a module. When the operator corresponding to a node fails the independent execution verification, the neighboring nodes of that node are determined from the computation graph. An initial module is formed by the operator corresponding to that node and the operators corresponding to the neighboring nodes. The initial module is then subjected to independent execution verification. If the initial module passes the independent execution verification, it constitutes a module; otherwise, it is used as a separate module. Based on this initial module, the operators corresponding to its neighboring nodes are iteratively incorporated into the initial module. The independent execution verification is repeated for each new initial module formed after merging until the new initial module passes the independent execution verification. Finally, the initial module that passes the independent execution verification is determined as a complete module. The independent execution verification can provide the operator or initial module to be verified with a simulated input tensor that conforms to its input port specifications in an isolated sandbox execution environment, and determine whether it can successfully execute and produce a valid output tensor without generating runtime errors (such as segmentation fault, null pointer reference, illegal memory access, etc.) caused by missing dependencies or incomplete logic.

[0062] For example, when processing the computation graph of a Long Short-Term Memory (LSTM) network unit, the computation graph contains information for calculating the forget gate (…). ), Input gate ( ), output gate ( ) and candidate cell states ( Multiple MatMul (matrix multiplication) and Add (addition) operators, and operators for updating cell states ( ) and hidden state ( First, a MatMul operator (denoted as ) is chosen for computing the forget gate, along with operators such as Multiply (element-wise multiplication) and Tanh (hyperbolic tangent). Perform independent execution verification on the operator. The mathematical formula can be represented as For operators Perform independent execution verification, due to the operator The input tensor depends on the hidden state of the previous time step. However, in an isolated sandbox environment, this hidden state input cannot be provided, causing independent verification to fail due to the operator. The operator failed the independent execution verification. The predecessor neighbor node was determined from the computation graph. Operators corresponding to its predecessor and neighbor nodes The components are merged into an initial module, and then independently executed and verified again. This is because the execution of this initial module still requires external access to the hidden state from the previous time step. As input, it still cannot be executed independently in the sandbox environment, causing the verification to fail again. Therefore, based on this initial module, we continue to iteratively merge other neighbor operators that have logical execution dependencies, such as those with... and The related gate calculations and state update operators are performed until the final initial module can be executed without errors. Finally, through independent execution verification, the final initial module is determined to be a module.

[0063] In step 1012, for each module, an accuracy anomaly prediction is performed to obtain the accuracy anomaly level of the module. Based on the accuracy anomaly level and the number of operators contained in the module, the detection priority of the module is evaluated to obtain the detection priority of the module.

[0064] Among them, accuracy anomaly prediction refers to the degree of risk that a module’s output tensor will deviate from its output tensor in the source device after it is migrated from the source device (first hardware acceleration device) to the target device (second hardware acceleration device). Accuracy anomaly level is a quantitative representation of this risk level, and detection priority is a sorting index used to guide the migration verification order.

[0065] As an example, for each module, the following processing is performed: the module's accuracy anomaly can be predicted using a pre-trained accuracy risk prediction model to obtain the module's accuracy anomaly level, or the module's accuracy anomaly can be predicted using a rule-based static analysis system to obtain the module's accuracy anomaly level; no limitation is made here.

[0066] After determining the accuracy anomaly level of the module, the first sum between the number of operators contained in the module and a preset first value is calculated, and a logarithmic operation is performed on the first sum to obtain the logarithmic result corresponding to the number of operators contained in the module. Using formula (1.1), based on the first weight corresponding to the accuracy anomaly level and the second weight corresponding to the number of operators contained in the module, a weighted sum is performed on the accuracy anomaly level and the logarithmic result to obtain the detection priority of the module. Formula (1.1) is as follows: (1.1) in, It is the detection priority of the nth module. It is the accuracy anomaly level of the nth module. It is the number of operators contained in the nth module. The preset first value, As the preset first weight, The second preset weight, along with the first and second weights, is used to adjust the relative importance of accuracy risk and module complexity (the number of operators included). For example, if more emphasis is placed on quickly locating high-risk problems, the weight can be set to... Using logarithmic functions This is to smooth out the impact of the number of operators and avoid the disproportionate amplification of priority due to the excessive size of the module.

[0067] In some embodiments, "predicting accuracy anomalies in a module using a pre-trained accuracy risk prediction model to obtain the module's accuracy anomaly level" can be achieved through the following process: obtaining the module's static information; and based on the pre-trained accuracy risk prediction model, predicting accuracy anomalies in the module's static information to obtain the module's accuracy anomaly level.

[0068] The static information of a module describes its computational complexity and numerical stability. This information includes the types and number of operators in the module, the data type used by each operator, and whether it contains numerically sensitive operators. Numerically sensitive operators refer to those operators that are more prone to problems such as gradient vanishing, gradient exploding, significant rounding error accumulation, or numerical overflow in finite-precision floating-point representations (e.g., FP16, BF16). Examples include exponential functions (Exp), logarithmic functions (Log), power functions (Pow), and operators such as Softmax and LayerNorm used for normalization. The accuracy risk prediction model can be a gradient boosting decision tree model or a small neural network, without limitation. The accuracy risk prediction model is obtained through supervised learning training based on a large amount of data collected from historical model transfer projects.

[0069] For example, if module A contains 5 MatMul operators with FP16 precision and 1 Softmax operator, and module B contains 10 ReLU operators with FP32 precision, and the static information of module A and module B is input into the precision risk prediction model in sequence, the model will output a precision anomaly level of 0.92 (range 0 to 1) for module A because FP16 matrix multiplication has a large risk of accumulated intermediate results error, and the Softmax operator involves exponential operations and is sensitive to numerical precision. The ReLU operator is simple to calculate, and FP32 has high precision and good numerical stability; therefore, the model will output a precision anomaly level of 0.15 for module B.

[0070] In some embodiments, "predicting accuracy anomalies in a module using a static analysis system to obtain the module's accuracy anomaly level" can be achieved through the following process: predicting accuracy anomalies in a module using the expert knowledge base built into the static analysis system to obtain the module's accuracy anomaly level.

[0071] As an example, the rule-based static analysis system has a built-in expert knowledge base that stores the historical accuracy anomaly levels of different operators. The historical accuracy anomaly level corresponding to each operator included in the module is determined from the expert knowledge base, and the average of the historical accuracy anomaly levels of at least one operator included in the module is determined as the accuracy anomaly level of the module.

[0072] In step 1013, based on the detection priority of each module, multiple modules are sorted to obtain multiple sorted modules, and the nth module is determined from the multiple sorted modules.

[0073] As an example, based on the detection priority of each module, multiple modules are sorted in descending order to obtain multiple modules in descending order. Based on the data flow dependencies between multiple modules, the multiple modules in descending order are adjusted to obtain multiple modules in sorted order. The nth module is determined from the multiple modules in sorted order. In the multiple modules in sorted order, the predecessor module of any module in the computation graph is located before any module. The predecessor module of any module refers to the module in the computation graph whose output tensor directly or indirectly affects the input tensor of any module.

[0074] In this embodiment, based on the modularization of the model to be migrated into multiple modules, an accuracy anomaly prediction is performed for each module to determine its accuracy anomaly level. The detection priority is evaluated in combination with the number of operators contained in the module. This process comprehensively considers the possibility of accuracy anomalies in the module and the complexity of its internal structure, so that the evaluated detection priority can effectively reflect the degree to which the module needs to be paid attention during the migration process. Subsequently, the multiple modules are sorted based on the detection priority of each module, and the nth module is determined from the sorting results. The modules can be hierarchically distinguished according to their own attribute differences, so as to orderly locate the modules that need to be processed in the subsequent migration process.

[0075] In some embodiments, Figure 3 Step 101 shown can be implemented through the following process: splitting the model to be migrated into modules to obtain multiple modules; performing performance analysis on the computational resource consumption of each module on the source device to obtain at least one performance index for each module; sorting the multiple modules in ascending order based on at least one performance index for each module to obtain multiple sorted modules, and determining the nth module from the multiple sorted modules.

[0076] In this context, the source device refers to the device that serves as the starting point for migration in the model migration task. The source device is used to provide performance benchmarks and accuracy references.

[0077] For example, deploy the model to be migrated on the source device, perform forward propagation on each module in the model to be migrated, and find that the computation time of module A is 10ms, the computation time of module B is 50ms, and the computation time of module C is 5ms. Based on the computation time (i.e., performance metric), sort all modules in descending order of computation time to obtain multiple sorted modules [B, A, C]. At this time, if we want to determine the first module (i.e., n=1), we choose the module with the longest computation time, B. If we want to determine the second module (i.e., n=2), we choose module A.

[0078] In some embodiments, Figure 3 Step 101 shown can be implemented through the following process: splitting the model to be transferred into modules to obtain multiple modules; constructing a visual model structure diagram of the model to be transferred based on the multiple modules; and in response to receiving a selection operation for any selectable module in the visual model structure diagram, determining any module as the nth module.

[0079] For example, when the nth module is determined through human-computer interaction, the model to be migrated is split into multiple modules. Then, based on the multiple modules, a visual model structure diagram of the model to be migrated is rendered on the graphical user interface (GUI). In the visual model structure diagram, the modules that have been deployed on the first hardware acceleration device and the second hardware acceleration device are in an unselectable state, while the modules that have not been deployed on the first hardware acceleration device and the second hardware acceleration device are in a selectable state. Test engineers can select the nth module from the multiple selectable modules by clicking, selecting with a box, or checking the module list.

[0080] In some embodiments, during execution Figure 3 Before step 101 shown, the following processes may also be performed: obtaining a preset deep learning framework; deploying the deep learning framework, the input layer and the output layer of the model to be transferred in a first hardware acceleration device to obtain a first initial model; deploying the deep learning framework, the input layer and the output layer of the model to be transferred in a second hardware acceleration device to obtain a second initial model; when the target operator is not determined from the first initial model and the second initial model, step 101 is performed.

[0081] As an example, the deep learning framework, the input layer and the output layer of the model to be transferred are deployed in the first general-purpose processor of the first hardware acceleration device to obtain the first initial model, and the deep learning framework, the input layer and the output layer of the model to be transferred are deployed in the second general-purpose processor of the second hardware acceleration device to obtain the second initial model. From the multiple operators contained in the first initial model and the multiple operators contained in the second initial model, the target operator with an accuracy abnormality is searched. When no target operator is determined from the first initial model and the second initial model, step 101 is executed. The specific method for finding the target operator with an accuracy abnormality can be found in the method of "determining the target operator with an accuracy abnormality from at least one operator contained in the nth module" shown below when the deployment method is general-purpose processor deployment, which will not be elaborated here.

[0082] In step 102, the nth module is added to the model deployed on the first hardware acceleration device to obtain the first model, and the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model.

[0083] As an example, based on the number of operators contained in the nth module, the deployment method of the nth module is determined. Based on the deployment method, the nth module is added to the model deployed on the first hardware acceleration device to obtain the first model. Based on the same deployment method, the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model.

[0084] In some embodiments, see Figure 6 , Figure 6 This is a schematic diagram of the fourth process of the model processing method provided in the embodiments of this application. Figure 3 Step 102 shown can be implemented through steps 1021 to 1023, which are explained in detail below.

[0085] In step 1021, the deployment method of the nth module is determined.

[0086] The deployment method refers to the strategy for allocating a computing execution backend to the nth module. Deployment methods include hardware accelerator deployment and general-purpose processor deployment.

[0087] As an example, when the number of operators contained in the nth module is one, the deployment of the hardware accelerator is determined as the deployment method of the nth module. When the number of operators contained in the nth module is greater than one, the deployment method of the nth module can be either hardware accelerator deployment or general processor deployment.

[0088] In step 1022, based on the deployment method, the nth module is added to the model deployed on the first hardware acceleration device to obtain the first model.

[0089] The first hardware acceleration device includes a first general-purpose processor and a first hardware accelerator. The first general-purpose processor is a central processing unit responsible for performing general computing tasks, operating systems, and control logic. The first hardware accelerator is a high-performance coprocessor specifically designed for parallel computing, such as a graphics processing unit (GPU), a tensor processing unit (TPU), or a neural network processing unit (NPU).

[0090] In some embodiments, Figure 6 Step 1022 shown can be implemented through the following process: when the deployment method is hardware accelerator deployment, the nth module is deployed to the first hardware accelerator of the first hardware acceleration device; based on the nth module deployed in the first hardware accelerator and the model deployed in the first hardware acceleration device, a first model is formed.

[0091] The model deployed in the first hardware acceleration device includes the input layer, output layer, and the first n-1 modules of the model to be migrated.

[0092] As an example, the execution engine of the deep learning framework deployed in the first hardware acceleration device allocates storage space of the corresponding size in the high-speed video memory (e.g., HBM) of the first hardware accelerator for the parameter tensors (e.g., weights, biases) and intermediate result tensors contained in the nth module based on the computation graph of the nth module. Then, it stores the parameter tensors and intermediate result tensors contained in the nth module in the allocated storage space and binds the computation logic code of the nth module with the kernel function provided by the first hardware accelerator to realize the deployment of the nth module.

[0093] After deploying the nth module to the first hardware accelerator of the first hardware acceleration device, based on the computation graph of the model to be transferred, the neighbor operator of the nth module is determined from the model deployed in the first hardware acceleration device, and the nth module is connected to its neighbor operator via data flow to integrate the nth module into the model deployed in the first hardware acceleration device, thereby obtaining the first model. In this embodiment, the specific implementation of the data flow connection is not limited. For example, the specific implementation of the data flow connection may be to directly modify the adjacency list corresponding to the computation graph of the model deployed in the first hardware acceleration device, or to modify the node attributes in the computation graph of the model deployed in the first hardware acceleration device, or to rewrite the forward function call logic of the deep learning framework deployed in the first hardware acceleration device, etc.

[0094] For example, when the specific implementation of the data flow connection involves modifying node attributes in the computation graph, the model to be migrated can be a BERT model. The model deployed on the first hardware acceleration device includes the BERT model's input layer, output layer, embedding layer (first module), and first Transformer encoder layer (second module). The third module can be the second Transformer encoder layer in the BERT model. Based on the computation graph of the BERT model, the neighbor operator of the second Transformer encoder layer is determined from the model deployed on the first hardware acceleration device. This neighbor operator is the operator LayerNorm in the first Transformer encoder layer. The output of the operator LayerNorm is a tensor named h1_output, which is stored at memory address 0x1A00 of the first hardware accelerator. The second Transformer encoder layer is then... The encoder layer is deployed in the first hardware accelerator. The input tensor h2_input of the second Transformer encoder layer is allocated at memory address 0x2B00. In the computation graph maintained by the deep learning framework, the metadata object representing the tensor h1_output is found, and this metadata object is pointed to the memory address 0x2B00 corresponding to the input tensor of the second Transformer encoder layer. Thus, the second Transformer encoder layer is integrated into the model deployed in the first hardware accelerator, resulting in the first model. Therefore, when the first model is forward propagated, after the computation flow completes the computation of the first Transformer encoder layer and generates h1_output, it will directly pass h1_output to the second Transformer encoder layer as its input, thus seamlessly integrating the second Transformer encoder layer into the execution chain.

[0095] In this embodiment, by deploying the nth module one by one during the model migration process, the module is dynamically combined with the model already deployed in the first hardware acceleration device to progressively build the first model. Since only one nth module to be verified is introduced at a time and immediately integrated with the previously verified model part, when subsequent accuracy comparisons show differences, the root cause of potential accuracy problems can be directly attributed to the newly integrated nth module. This processing method decomposes the complex full-model accuracy problem into a series of sub-problems with clear boundaries for individual modules, making the accuracy problem localization process more accurate and efficient when migrating models in a heterogeneous hardware environment, improving the accuracy of problem localization and the efficiency of the overall migration process.

[0096] In some embodiments, Figure 6The step 1022 shown can be implemented by the following process: when the deployment method is general-purpose processor deployment, the nth module is deployed to the first general-purpose processor of the first hardware acceleration device; based on the nth module deployed in the first general-purpose processor and the model deployed in the first hardware acceleration device, a first model is formed.

[0097] The model deployed in the first hardware acceleration device includes the input layer, output layer, and the first n-1 modules of the model to be migrated.

[0098] As an example, the execution engine of the deep learning framework deployed in the first hardware acceleration device marks the computational task of the nth module as to be executed on the first general-purpose processor based on the computation graph of the nth module. The execution engine requests and allocates storage space of the corresponding size in the main system memory directly connected to the first general-purpose processor for the parameter tensors and intermediate result tensors contained in the nth module. Then, it loads the parameter tensors contained in the nth module into the main memory space allocated to it, and binds the computational logic code of the nth module with functions in a computational library (e.g., Intel MKL, OpenBLAS) optimized for general-purpose processors to realize the deployment of the nth module.

[0099] After deploying the nth module to the first general-purpose processor of the first hardware acceleration device, based on the computation graph of the model to be migrated, the neighbor operator of the nth module is determined from the model deployed in the first hardware acceleration device, and the nth module is connected to the neighbor operator via data flow to integrate the nth module into the model deployed in the first hardware acceleration device to obtain the first model. In this embodiment, the specific implementation of the data flow connection is not limited. If the neighbor operator of the nth module is located on the first hardware accelerator, the data flow connection also includes inserting a cross-device memory transfer node in the computation graph of the model deployed in the first hardware acceleration device.

[0100] In this embodiment, a technical solution is provided in which the nth module is deployed to the first general-purpose processor of the first hardware acceleration device when the deployment method is general-purpose processor deployment, and a first model is formed based on this and the existing model. This achieves flexible scheduling and utilization of heterogeneous computing resources. When the nth module cannot be executed on the hardware accelerator, this solution can dynamically allocate its computing tasks to the general-purpose processor. By constructing a new model containing the module, the continuity and integrity of the entire model migration and verification process are ensured. This method of co-deploying the hardware accelerator and the general-purpose processor provides an effective path for processing modules with special operator dependencies or requiring fine-grained analysis, thereby improving the applicability and debugging flexibility of the model migration method for different types of neural network models.

[0101] In step 1023, based on the deployment method, the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model.

[0102] The second hardware acceleration device includes a second general-purpose processor and a second hardware accelerator.

[0103] As an example, when the deployment method is hardware accelerator deployment, the nth module is deployed to the second hardware accelerator of the second hardware accelerator device; based on the nth module deployed in the second hardware accelerator device and the model deployed in the second hardware accelerator device, a second model is constructed; when the deployment method is general-purpose processor deployment, the nth module is deployed to the second general-purpose processor of the second hardware accelerator device; based on the nth module deployed in the second general-purpose processor device and the model deployed in the second hardware accelerator device, a second model is constructed. The method of adding the nth module to the model deployed in the second hardware accelerator device can be found in step 1022 shown above, and will not be repeated here.

[0104] In this embodiment, by determining the deployment method of the nth module before adding it to build the first and second models, and then performing subsequent addition operations based on this deployment method, precise control over the allocation of module-level computing resources during model migration is achieved. By introducing a step to determine the deployment method, the computing tasks of the module can be autonomously allocated to hardware accelerators or general-purpose processors according to the characteristics or migration strategy of the module. This addition mechanism based on a preset deployment method enables the model to be incrementally built on heterogeneous hardware devices with a unified and clear computing path, ensuring the consistency of the first and second models in module deployment strategies. This provides a solid foundation for subsequent fair and effective accuracy difference comparisons and improves the reliability of locating the accuracy problem in the entire model migration process.

[0105] In step 103, the first model is trained to obtain the first model training result, and the second model is trained to obtain the second model training result.

[0106] Model training refers to the iterative process of adjusting the model's internal parameters by inputting one or more batches of training data into the model and adjusting them based on the difference between the model's output and the true labels. The model training result is a set of quantitative indicators that reflect the model's performance and state, generated during the model training process.

[0107] As an example, the training dataset is obtained. Multiple input data points and their corresponding ground truth labels for the first training batch are extracted from the training dataset. Based on these input data points and ground truth labels, the first model is trained to obtain the first model after its first training iteration. Similarly, based on these input data points and ground truth labels, the second model is trained to obtain the second model after its first training iteration. Then, multiple input data points and their corresponding ground truth labels for the kth training batch are extracted from the training dataset. Based on these input data points and ground truth labels, the first model after its (k-1)th training iteration is trained to obtain the first model after its kth training iteration. Finally, based on these input data points and ground truth labels, the second model after its (k-1)th training iteration is trained to obtain the second model after its kth training iteration. Here, k is a sequentially increasing positive integer, 1 < k ≤ K, and K is the preset number of training iterations.

[0108] After obtaining the first model and the second model after the Kth training iteration, the first loss value corresponding to each of the K training batches of the first model, as well as the input tensor and output tensor of each operator in the first model during the forward propagation of the K training batches, are determined as the training result of the first model. Similarly, the second loss value corresponding to each of the K training batches of the second model, as well as the input tensor and output tensor of each operator in the second model during the forward propagation of the K training batches, are determined as the training result of the second model.

[0109] In some embodiments, "training the first model based on multiple input data of the first training batch and the real label corresponding to each input data to obtain the first model after the first training is completed" can be achieved through the following process: based on the first model, perform forward propagation on each input data to obtain the predicted data output by the first model; determine the initial loss value based on the difference between the real label corresponding to each input data and its corresponding predicted data; determine the average of the initial loss values ​​corresponding to multiple input data as the first loss value; and train the first model based on the first loss value to obtain the first model after the first training is completed.

[0110] In step 104, a first accuracy difference between the first model and the second model is determined based on the difference between the training results of the first model and the training results of the second model.

[0111] The first precision difference is a numerical metric used to quantify the degree of inconsistency between the training results of the first model and the training results of the second model.

[0112] As an example, the difference between the K first loss values ​​contained in the training result of the first model and the K second loss values ​​contained in the training result of the second model is determined, and the difference value is determined as the first accuracy difference between the first model and the second model. The difference value can be the average absolute error between the K first loss values ​​and the K second loss values, the average relative error between the K first loss values ​​and the K second loss values, or the maximum absolute error between the K first loss values ​​and the K second loss values, which is not limited here.

[0113] In step 105, when the first precision difference is greater than the first preset threshold, the target operator with precision abnormality is determined from at least one operator contained in the nth module.

[0114] In some embodiments, when the deployment method is a general-purpose processor deployment, Figure 3 The step 105 shown, "determining the target operator with an accuracy anomaly from at least one operator contained in the nth module," can be achieved by performing the following processing on the mth operator contained in the nth module: transferring the mth operator deployed in the first general-purpose processor of the first hardware acceleration device to the first hardware accelerator of the first hardware acceleration device to obtain the third model; transferring the mth operator deployed in the second general-purpose processor of the second hardware acceleration device to the second hardware accelerator of the second hardware acceleration device to obtain the fourth model; training the third model to obtain the third model training result, and training the fourth model to obtain the fourth model training result; determining the second accuracy difference between the third model and the fourth model based on the difference between the third model training result and the fourth model training result; when the second accuracy difference is greater than the second preset threshold, the mth operator is determined as the target operator.

[0115] Where m is a positive integer that increases sequentially, m≤M, and M is the number of operators contained in the nth module.

[0116] As an example, when the deployment method is general-purpose processor deployment, the m-th operator deployed in the first general-purpose processor of the first hardware acceleration device is migrated to the first hardware accelerator of the first hardware acceleration device. Based on the remaining operators of the n-th module deployed in the first general-purpose processor, the first m operators of the n-th module deployed in the first hardware accelerator, and the model deployed in the first hardware acceleration device, a third model is constructed. The m-th operator deployed in the second general-purpose processor of the second hardware acceleration device is migrated to the second hardware accelerator of the second hardware acceleration device. Based on the remaining operators of the n-th module deployed in the second general-purpose processor, the first m operators of the n-th module deployed in the second hardware accelerator, and the model deployed in the second hardware acceleration device, a fourth model is constructed. The model is trained to obtain the training result of the third model, and then trained to obtain the training result of the fourth model. The second precision difference between the third model and the fourth model is determined. When the second precision difference is greater than the second preset threshold, the m-th operator is determined as the target operator, and the iteration process is terminated. The target operator is reported to the developers for further manual analysis or repair. The specific implementation of deploying the m-th operator to the first hardware accelerator and the second hardware accelerator can be found in step 1022 above. The specific implementation of training the third model and the fourth model can be found in step 103 above. The specific implementation of determining the second precision difference between the third model and the fourth model can be found in step 104 above. These details will not be elaborated here.

[0117] For example, the nth module contains three operators [Op_A, Op_B, Op_C], and a second preset threshold. Set as In the first iteration (m=1), on the first hardware acceleration device, the first operator Op_A is migrated from the first general-purpose processor to the first hardware accelerator. At this point, the execution path of the third model becomes: Op_A is executed on the first hardware accelerator, and then Op_B and Op_C are executed on the first general-purpose processor. On the second hardware acceleration device, the first operator Op_A is migrated from the second general-purpose processor to the second hardware accelerator. At this point, the execution path of the fourth model becomes: Op_A is executed on the second hardware accelerator, and then Op_B and Op_C are executed on the second general-purpose processor. After synchronous training of the third and fourth models, the second precision difference between them is calculated as follows: Since the difference in second precision did not exceed the second preset threshold The implementations of Op_A on the two hardware accelerators are determined to be compatible, and the next iteration continues. In the second iteration (m=2), on the first hardware accelerator, Op_A remains on the first hardware accelerator, and the second operator Op_B is migrated from the first general-purpose processor to the first hardware accelerator. At this time, the execution path of the updated third model becomes: Op_A and Op_B are executed on the first hardware accelerator, and then Op_C is executed on the first general-purpose processor. On the second hardware accelerator, Op_A remains on the second hardware accelerator, and the second operator Op_B is migrated from the second general-purpose processor to the second hardware accelerator. At this time, the execution path of the updated fourth model becomes: Op_A and Op_B are executed on the second hardware accelerator, and then Op_C is executed on the second general-purpose processor. After synchronously training these two latest third and fourth models, the difference in second precision between them is calculated. Because the difference in second precision exceeds the second preset threshold Therefore, it was determined that the operator Op_B, which had just been synchronized and migrated, was the target operator causing the accuracy problem. This indicated that the specific implementation of operator Op_B on the first and second hardware accelerators had numerical calculation deviations that exceeded the allowable range. At this point, the iteration process was immediately terminated, and an accuracy anomaly report was generated. The accuracy anomaly report pointed out that in the nth module, there was a significant accuracy deviation in the implementation of operator Op_B on the two hardware acceleration devices, and requested the developers to conduct in-depth manual analysis.

[0118] In this embodiment, a migration verification scheme for operators within a general-purpose processor deployment module is provided, enabling refined localization of the root cause of accuracy anomalies. This scheme iteratively migrates the m-th operator from the first general-purpose processor (serving as a baseline) to the first hardware accelerator, and simultaneously performs the same operation on the second hardware acceleration device, constructing a third model and a fourth model. By comparing the training results of these two new models and calculating the second accuracy difference, the numerical deviation introduced solely by the hardware accelerator implementation of the m-th operator can be isolated. When this deviation exceeds a second preset threshold, the m-th operator can be identified as the target operator. This differential testing method, which introduces variables one by one and verifies them in real time, provides a systematic technical path for accurately locating and attributing the source of accuracy problems within complex modules, improving the accuracy and efficiency of problem localization.

[0119] In some embodiments, Figure 3The step 105 shown, "determining the target operator with an accuracy anomaly from at least one operator contained in the nth module", can be achieved by performing the following processing on the mth operator contained in the nth module: determining the historical input data and historical output data of the mth operator from the training results of the first model; processing the historical input data through the mth operator deployed on the second hardware acceleration device to obtain the output data; when the difference between the historical output data and the output data is greater than the difference threshold, the mth operator is determined as the target operator.

[0120] Where m is a positive integer that increases sequentially, m≤M, and M is the number of operators contained in the nth module.

[0121] As an example, when the deployment method is general-purpose processor deployment or hardware accelerator deployment, the target training batch is determined from the K training batches used to train the first and second models. During the forward propagation process of the target training batch determined from the training results of the first model, the historical input data and historical output data of the m-th operator of the n-th module are processed by the m-th operator deployed on the second hardware acceleration device to obtain the output data. When the difference between the historical output data and the output data is greater than the preset difference threshold, the m-th operator is determined as the target operator, and the iteration process is terminated. The target operator is reported to the developers for further manual analysis or repair. The difference between the historical output data and the output data can be the element-wise maximum absolute error, the relative L2 norm difference, etc., which are not limited here.

[0122] In this embodiment, a scheme for independently verifying operators based on historical data is provided, achieving high-precision localization of operators with abnormal accuracy within a module. First, the historical input and output data of the m-th operator are determined from the training results of the first model as a benchmark. Then, the historical input data is processed by the same operator deployed on the second hardware acceleration device to obtain new output data. By providing identical inputs to the operators on both devices, the interference of accumulated errors from the upstream module is effectively isolated. Finally, based on the comparison of the difference between the historical output data and the newly generated output data, the inherent computational differences of the m-th operator on the two hardware devices can be directly reflected, thereby accurately identifying the operators with differences greater than a threshold as target operators, improving the accuracy and efficiency of problem localization.

[0123] In some embodiments, Figure 3The step 105 shown, "Identifying the target operator with an accuracy anomaly from at least one operator contained in the nth module," can be achieved through the following processing: Identifying the target training batch from the K training batches used to train the first and second models; during the i-th iteration, determining the operator sequence, uniformly dividing the operator sequence into a preorder sequence and a postorder sequence, identifying the first operator in the postorder sequence as the operator to be verified, determining the first input data and first output data of the operator to be verified during the forward propagation process of the target training batch from the first model training results, and determining the forward propagation process of the target training batch from the second model training results. In the process, the second input data of the operator to be verified is processed by the operator deployed on the second hardware acceleration device to obtain the second output data. When the difference between the first input data and the second input data is greater than a preset input data difference threshold, the operator sequence is updated to the preceding sequence. When the difference between the first input data and the second input data is less than or equal to the input data difference threshold, the first input data is processed to obtain the second output data. When the difference between the first output data and the second output data is greater than the difference threshold, the operator to be verified is determined as the target operator. When the difference between the first output data and the second output data is less than or equal to the difference threshold, the operator sequence is updated to the following sequence.

[0124] Where i is an increasing positive integer, and the operator sequence in the first iteration contains M operators.

[0125] For example, the nth module contains an ordered sequence of 8 operators [Op_1, Op_2, ..., Op_8], with a preset input data difference threshold. for Difference threshold for In the first iteration, the operator sequence [Op_1, Op_2, ..., Op_8] is divided into a preceding sequence [Op_1, ..., Op_4] and a following sequence [Op_5, ..., Op_8]. Op_5 is identified as the operator to be verified. If the difference between the first and second input data of the operator to be verified, Op_5, is greater than the input data difference threshold, it indicates that the target operator causing the accuracy difference is located in the preceding sequence. Therefore, the operator sequence is updated to [Op_1, ..., Op_4]. In the second iteration, the operator sequence [Op_1, ..., Op_4] is divided into a preceding sequence [Op_1, ..., Op_8]. Given the sequence [Op_1, Op_2] and the subsequent sequence [Op_3, Op_4], Op_3 is identified as the operator to be verified. If the difference between the first and second input data of the operator to be verified, Op_3, is less than the input data difference threshold, it indicates that its upstream operator has no significant problem. The first input data of Op_3 is processed on the second hardware acceleration device to obtain the second output data. The second output data is compared with the first output data, and it is found that the difference between the two is greater than the difference threshold. This indicates that Op_3 itself introduces an unacceptable precision deviation. Therefore, Op_3 is identified as the target operator, and the search process terminates.

[0126] In some embodiments, "determining the target training batch from the K training batches used to train the first model and the second model" can be achieved through the following process: determining the difference between the kth first loss value contained in the training result of the first model and the kth second loss value contained in the training result of the second model as the loss difference value corresponding to the kth training batch; and determining the target training batch from the K training batches based on the loss difference value corresponding to each training batch.

[0127] As an example, the difference between the kth first loss value contained in the training result of the first model and the kth second loss value contained in the training result of the second model is determined as the loss difference corresponding to the kth training batch. The difference between the kth first loss value and the kth second loss value can be an absolute loss difference or a relative loss difference, which is not limited here.

[0128] The training batch corresponding to the largest loss difference among the K loss differences can be determined as the target training batch; alternatively, the K training batches can be traversed in ascending order, and when the loss difference corresponding to any training batch is greater than a preset loss difference threshold, the traversal of training batches can be stopped, and any training batch can be determined as the target training batch; or, any training batch and a preset number of training batches after any training batch can be determined as candidate training batches, and the candidate training batch with the largest loss difference can be determined as the target training batch.

[0129] For example, after training the model with K=100 batches, a loss difference sequence containing 100 elements was calculated. And the preset loss difference threshold With a preset quantity of 10, the maximum difference criterion is used. By performing the argmax operation on the entire loss difference sequence, the 10th difference is found. Loss difference corresponding to each training batch The 87th training batch is the largest among all 100 loss differences; therefore, it is directly selected as the target training batch. In another scenario, a criterion combining the first threshold exceedance with the maximum value of subsequent windows is adopted. Start by traversing the loss difference sequence in order, and continue until the nth iteration. During each training batch, the difference in loss was observed. This is the first one with a loss difference threshold. If the value is found, stop iterating and determine the 50th training batch (i.e., any training batch) and the following 10 training batches (i.e., training batches 51 to 60) as candidate training batches. Then, search for the maximum loss difference among the 11 candidate training batches. If the maximum loss difference is found, the maximum loss difference is determined. Loss difference corresponding to each candidate training batch If the maximum value is found, then the 58th training batch will be designated as the target training batch.

[0130] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0131] In the field of neural network training, efficient computation typically relies on hardware acceleration devices, including hardware accelerators, general-purpose processors, and storage media. However, due to inherent differences in the underlying implementations of different types of hardware accelerators, the deep learning frameworks built upon them exhibit variations in implementation logic and computational accuracy. In contrast, deep learning framework implementations on general-purpose processors generally maintain a high degree of consistency. In practical applications, for example, to verify whether a target graphics processing unit (GPU) can achieve the performance of a reference GPU, it is often necessary to migrate the neural network model from one hardware acceleration device to another for training. During this process, the output training evaluation metrics, such as model loss, may differ. When this difference exceeds a predetermined range, the training results of the neural network are usually considered inconsistent, thus requiring the identification of the root cause of the discrepancy. The challenge of related technologies is that neural network training involves multiple iterative calculations. In the early iterations of training, the accuracy of the output of each module may meet the requirements, but the small calculation errors generated will continue to accumulate. This may lead to a situation in subsequent iterations where even if the input data of a certain module meets the accuracy requirements, its output results may not meet the accuracy requirements. This makes the process of locating the real module and specific cause of the difference in training results complex and time-consuming.

[0132] See Figure 7 , Figure 7 This is a schematic diagram of the fifth process of the model processing method provided in the embodiments of this application. The following will be combined with... Figure 7 The model processing method provided in the embodiments of this application will be described.

[0133] Step 401: Deploy the deep learning framework, including modules and model single operators, to meet the accuracy requirements on the first and second hardware acceleration devices.

[0134] The same deep learning framework is deployed on the first hardware acceleration device and the second hardware acceleration device, respectively. Some modules of the deep learning framework are deployed on the general-purpose processor of the hardware acceleration device, and some modules are deployed on the hardware accelerator of the hardware acceleration device.

[0135] See Figure 8 , Figure 8This is a first schematic diagram of the model processing method provided in this application embodiment. The hardware acceleration device 501 includes a general-purpose processor 502, a hardware accelerator 503, and an input / output module 504. The general-purpose processor 502, the hardware accelerator 503, and the input / output module 504 are all communicatively connected to a peripheral component interconnect standard switch (PCIe switch). The general-purpose processor 502 and the hardware accelerator 503 respectively realize bidirectional data interaction and instruction transmission through the PCIe standard switch. At the same time, the input / output module 504 provides an external data communication interface through the PCIe standard switch, thereby constructing a hardware collaborative processing topology with the PCIe standard switch as the core of the communication bus.

[0136] When a neural network model is transferred from a first hardware acceleration device to a second hardware acceleration device for training, it is first necessary to ensure that the two hardware acceleration devices support the same deep learning framework. The implementation of various modules in the deep learning framework may differ on different devices. Therefore, it is necessary to ensure that the output data accuracy meets the set accuracy requirements when the input data of various modules in the deep learning framework on the second hardware acceleration device is consistent with the corresponding modules on the first hardware acceleration device.

[0137] Step 402: Set the same training parameters on the first hardware acceleration device and the second hardware acceleration device.

[0138] Step 403: Decompose the neural network model to be transferred at the module level and operator level to obtain multiple operators to be added and multiple modules to be added.

[0139] Each module to be added contains multiple non-removable operators.

[0140] Step 404A: Connect each operator to be added sequentially to the first hardware acceleration device and the second hardware acceleration device.

[0141] The training process of the neural network model to be transferred consists of various computational operators, communication operators, and training modules in the deep learning framework. Therefore, the operators included in the training can be gradually added. Specifically, the operators to be added are determined from the neural network model, and the operators to be added are added to the hardware accelerators (e.g., GPUs) of the first and second hardware acceleration devices, respectively, so as to deploy a neural network model with a small parameter scale that can be trained normally on the first and second hardware acceleration devices.

[0142] Step 404B: Connect each module to be added sequentially to the general-purpose processor of the first hardware acceleration device and the second hardware acceleration device, and gradually migrate them to the hardware accelerators of the first hardware acceleration device and the second hardware acceleration device.

[0143] Some operators are dependent on each other and cannot be separated and added individually to the two hardware acceleration devices. Therefore, dependent operators are treated as modules and deployed and invoked on a general-purpose processor. This allows for the identification of operators within the module that significantly impact the accuracy of the training results. Specifically, dependent operators are deployed as modules to the general-purpose processors of the two hardware acceleration devices (the first and second hardware acceleration devices). (The module implementation logic and accuracy on the general-purpose processor are completely identical, so it can be assumed that the modules on the general-purpose processor will not increase the difference in training result accuracy between the two hardware acceleration devices.) All operators in the general-purpose processors of the two hardware acceleration devices are migrated one by one to their corresponding hardware accelerators for processing. After each migration, the accuracy difference between the training results of the small models deployed on the two hardware acceleration devices is assessed. If the accuracy difference does not meet the set threshold, the currently migrated operator is identified as having an accuracy anomaly, and operator migration is stopped. Simultaneously, the addition of new operators or modules to the first and second hardware acceleration devices is also halted.

[0144] Step 405: Compare the accuracy difference between the training results of the small model (a neural network model composed of added operators and modules) on the first hardware acceleration device and the training results of the small model (a neural network model composed of added operators and modules) on the second hardware acceleration device.

[0145] After each operator or module is added to the first hardware acceleration device and the second hardware acceleration device, the small model on the first hardware acceleration device and the small model on the second hardware acceleration device are trained respectively. Then, the accuracy difference between the training results of the small model on the first hardware acceleration device and the training results of the small model on the second hardware acceleration device is compared.

[0146] If the accuracy difference meets the set threshold requirement, return to step 404A or step 404B to continue adding new operators and modules to the deployed small model.

[0147] If the accuracy difference does not meet the set threshold requirements, stop adding new operators or modules to the first and second hardware acceleration devices.

[0148] Step 406: After all the modules and operators to be added are connected to the first hardware acceleration device and the second hardware acceleration device, the complete model deployed in the first hardware acceleration device and the second hardware acceleration device outputs data with precision aligned on the first hardware acceleration device and the second hardware acceleration device.

[0149] See Figure 9 , Figure 9 This is a schematic diagram of the sixth process of the model processing method provided in the embodiments of this application. The following will be combined with... Figure 9The model processing method provided in the embodiments of this application will be further explained.

[0150] First, step 601 is executed, deploying a deep learning framework that meets the accuracy requirements for each module and model single operator on the first and second hardware acceleration devices. Then, step 602 is executed, setting the same training parameters on the first and second hardware acceleration devices. Next, step 603 is executed, decomposing the neural network model to be transferred at the module level or operator level to obtain the operator or module to be added. It is then determined whether the module to be added is an operator or a module. If the operator to be added is an operator, steps 604A to 606A are executed; if the module to be added is a module, steps 604B to 609B are executed.

[0151] When the operator to be added is executed, firstly, step 604A is executed to connect the operator to be added to the hardware accelerators of the first and second hardware acceleration devices. Then, step 605A is executed to train the small model on the first and second hardware acceleration devices. Then, step 606A is executed to compare the accuracy difference between the training results of the small model on the first and second hardware acceleration devices and determine whether the accuracy difference is less than a preset threshold. If the accuracy difference is greater than or equal to the preset threshold (No), the operator accuracy problem is analyzed. If the accuracy difference is less than the preset threshold (Yes), it is determined whether there are any unconnected operators. If there are no unconnected operators (No), the operator migration is completed. If there are any unconnected operators (Yes), the process returns to step 603.

[0152] If the module to be added is executed, firstly, step 604B is performed to connect the module to be added to the general-purpose processor of the first and second hardware acceleration devices. Then, step 605B is performed to train the small models on the first and second hardware acceleration devices. Next, step 606B is performed to compare the accuracy difference between the training results of the small models on the first and second hardware acceleration devices and determine whether the accuracy difference is less than a preset threshold. If the accuracy difference is greater than or equal to the preset threshold (No), the module access method is analyzed. If the accuracy difference is less than the preset threshold (Yes), step 607B is performed to transfer the operators of the module to be added from the general-purpose processor of the first and second hardware acceleration devices to the hardware accelerators of the first and second hardware acceleration devices. Subsequently, step 608B is executed to train the small model on the first hardware acceleration device and the small model on the second hardware acceleration device. Then, step 609B is executed to compare the accuracy difference between the training results of the small model on the first hardware acceleration device and the training results of the small model on the second hardware acceleration device, and to determine whether the accuracy difference is less than a preset threshold. If the accuracy difference is greater than or equal to the preset threshold (No), the module accuracy problem is analyzed. If the accuracy difference is less than the preset threshold (Yes), it is determined whether there are any operators in the module to be added that are not connected to the hardware accelerator. If there are any operators that are not connected to the hardware accelerator (Yes), the process returns to step 607B. If there are no operators that are not connected to the hardware accelerator (No), it is determined whether there are any modules that are not connected. If there are any modules that are not connected (Yes), the process returns to step 603. If there are no modules that are not connected (No), the module migration is complete.

[0153] The model processing method provided in this application embodiment can also sequentially connect each module to be added to the hardware accelerators of the first and second hardware acceleration devices. If the module to be added has an accuracy anomaly, further problem localization can be performed on the module with the accuracy anomaly. Specifically, the iteration record of model training is traced back to find the target training iteration step where the loss does not meet the preset requirements for the first time. For each operator to be verified contained in the module with the accuracy anomaly, the following processing is performed: In the target training iteration step, the input data and output data of the operator to be verified in the first hardware acceleration device are captured and recorded as input 1 and output 1. Similarly, the input data and output data of the operator to be verified in the second hardware acceleration device are captured and recorded as input 2 and output 2. Input 2 is used as input on the first hardware acceleration device to perform calculations and obtain output 3. The accuracy of output 2 and output 3 are compared. If the accuracy does not meet the requirements of a single operator, it means that the processing of the data set by the operator to be verified is not up to standard, that is, the operator to be verified is an operator with an accuracy anomaly and manual analysis of the operator to be verified is required.

[0154] For example, the neural network model to be transferred could be a 0.1B GPT-MoE model. The first hardware acceleration device uses GPU1 as the hardware accelerator and a CPU as the general-purpose processor. The second hardware acceleration device uses GPU2 as the hardware accelerator and a CPU as the general-purpose processor. The deep learning framework is PyTorch 2.8 Megatron-LM core_r 0.9.0, with 20 training iterations and a learning rate of 0.0001. Both GPU1 and GPU2 have corresponding deep learning software stacks. All torch operators and modules in the deep learning framework used in the GPT-MoE model meet the computational accuracy requirements on both GPU1 and GPU2. The GPT-MoE model consists of an embedding layer, a transformer layer, and an output layer. The transformer layer contains six operators: linear_qkv, rope, attention, router, and token. permutation, mlp_groupgemm, tokenunpermutation: The deep learning framework includes a data preprocessing module for training, a model parameter processing and update module, a loss calculation module, and a training organization module. Since the modules in the deep learning framework are interdependent and cannot be added one by one, this example focuses on connecting some operators in the GPT-MoE model to GPU1 and GPU2 one by one.

[0155] First, the data preprocessing module, model parameter processing and update module, loss calculation module, and training organization module of the deep learning framework are deployed to two hardware acceleration devices. The data preprocessing module is deployed to GPU1 and GPU2, while the other modules are deployed to a general-purpose processor.

[0156] Then, the embedding and output layers of the GPT-MoE model are connected to the general-purpose processors (CPUs) of two hardware acceleration devices. The training results of the two hardware acceleration devices are compared. If there is no difference in accuracy, multiple modules of the deep learning framework on the CPU are connected to GPU1 and GPU2 in sequence. After each module is connected, the training results of the two hardware acceleration devices are compared. If the accuracy requirements are not met, it is considered that the module most recently connected to the hardware accelerator has a problem.

[0157] Next, after all the deep learning frameworks are connected to the hardware accelerators and the training results meet the accuracy requirements, the model embedding layer and output layer of the GPT-MoE model are gradually connected to the hardware accelerators of the two hardware accelerators. The training results of the two hardware accelerators are compared. If there is a difference in accuracy, the corresponding problem operator can be located.

[0158] Finally, after connecting the embedding layer and output layer to the hardware accelerators and ensuring the training accuracy meets the requirements, the three operators in the transformer layer—linear_qkv, rope, and attention—are individually connected to the hardware accelerators of the two hardware accelerators. The accuracy of the training results is then compared and identified. The router, token permutation, mlp_groupgemm, and tokenunpermutation operators in the transformer layer cannot be individually connected to the GPT-MoE model; therefore, they need to be integrated as a whole. These four operators are first placed on the CPUs of the two hardware accelerators, and then connected to their respective hardware accelerators one by one. The accuracy of the training results is then compared. If the requirements are not met, the problematic operator can be identified for individual accuracy analysis. If all requirements are met, the entire GPT-MoE model training has been successfully transferred from GPU1 to GPU2.

[0159] The following description continues to illustrate the exemplary structure of the model processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the model processing device 555 of the memory 550 may include: The model decomposition module 5551 is used to determine the nth module from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increases sequentially.

[0160] The model deployment module 5552 is used to add the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and to add the nth module to the model deployed on the second hardware acceleration device to obtain the second model.

[0161] The model training module 5553 is used to train the first model to obtain the first model training result, and to train the second model to obtain the second model training result.

[0162] The difference determination module 5554 is used to determine a first accuracy difference between the first model and the second model based on the difference between the training results of the first model and the training results of the second model.

[0163] The difference determination module 5554 is further configured to determine, when the first precision difference is greater than the first preset threshold, a target operator with an abnormal precision from at least one operator included in the nth module.

[0164] In some embodiments, the model deployment module 5552 is further configured to determine the deployment method of the nth module; based on the deployment method, add the nth module to the model deployed on the first hardware acceleration device to obtain the first model; based on the deployment method, add the nth module to the model deployed on the second hardware acceleration device to obtain the second model.

[0165] In some embodiments, the model deployment module 5552 is further configured to deploy the nth module to the first hardware accelerator of the first hardware acceleration device when the deployment method is hardware accelerator deployment; and to construct the first model based on the nth module deployed in the first hardware accelerator and the model deployed in the first hardware acceleration device.

[0166] In some embodiments, the model deployment module 5552 is further configured to deploy the nth module to the first general-purpose processor of the first hardware acceleration device when the deployment method is general-purpose processor deployment; and to construct the first model based on the nth module deployed in the first general-purpose processor and the model deployed in the first hardware acceleration device.

[0167] In some embodiments, when the deployment method is a general-purpose processor deployment, the difference determination module 5554 is further configured to perform the following processing on the m-th operator contained in the n-th module: migrate the m-th operator deployed in the first general-purpose processor of the first hardware acceleration device to the first hardware accelerator of the first hardware acceleration device to obtain a third model, where m is a positive integer that increments sequentially, m≤M, and M is the number of operators contained in the n-th module; migrate the m-th operator deployed in the second general-purpose processor of the second hardware acceleration device to the second hardware accelerator of the second hardware acceleration device to obtain a fourth model; perform model training on the third model to obtain a third model training result, and perform model training on the fourth model to obtain a fourth model training result; determine a second precision difference between the third model and the fourth model based on the difference between the third model training result and the fourth model training result; when the second precision difference is greater than a second preset threshold, determine the m-th operator as the target operator.

[0168] In some embodiments, the difference determination module 5554 is further configured to perform the following processing on the m-th operator included in the n-th module: determine the historical input data and historical output data of the m-th operator from the first model training results, where m is a positive integer that increments sequentially, m≤M, and M is the number of operators included in the n-th module; process the historical input data using the m-th operator deployed on the second hardware acceleration device to obtain output data; and determine the m-th operator as the target operator when the difference between the historical output data and the output data is greater than a difference threshold.

[0169] In some embodiments, the model decomposition module 5551 is used to decompose the model to be transferred into multiple modules; for each module, to perform accuracy anomaly prediction on the module to obtain the accuracy anomaly level of the module, and to evaluate the detection priority of the module based on the accuracy anomaly level and the number of operators contained in the module to obtain the detection priority of the module; to sort the multiple modules based on the detection priority of each module to obtain the sorted multiple modules; and to determine the nth module from the sorted multiple modules.

[0170] In some embodiments, the model decomposition module 5551 is used to construct a computation graph of the model to be migrated, wherein the nodes in the computation graph are operators in the model to be migrated, and the edges in the computation graph are data flow dependencies between the operators; the computation graph is divided based on a preset operator fusion rule to obtain multiple subgraphs, and the multiple modules are determined based on the multiple subgraphs.

[0171] This application provides a computer program product, which includes computer-executable instructions or a computer program. An electronic device's processor reads the computer-executable instructions or computer program from a computer-readable storage medium, and executes the computer-executable instructions or computer program, causing the electronic device to perform the model processing method described above in this application.

[0172] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the model processing method provided in this application.

[0173] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0174] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0175] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0176] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0177] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A model processing method, characterized in that, The method includes: The nth module is determined from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increases sequentially. The nth module is added to the model deployed on the first hardware acceleration device to obtain the first model, and the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model; The first model is trained to obtain the first model training result, and the second model is trained to obtain the second model training result; Based on the difference between the training results of the first model and the training results of the second model, a first accuracy difference between the first model and the second model is determined. When the first precision difference is greater than the first preset threshold, the target operator with precision abnormality is determined from at least one operator included in the nth module.

2. The method according to claim 1, characterized in that, The step of adding the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and adding the nth module to the model deployed on the second hardware acceleration device to obtain the second model, includes: Determine the deployment method of the nth module; Based on the deployment method, the nth module is added to the model deployed on the first hardware acceleration device to obtain the first model; Based on the deployment method, the nth module is added to the model deployed on the second hardware acceleration device to obtain the second model.

3. The method according to claim 2, characterized in that, The step of adding the nth module to the model deployed on the first hardware acceleration device based on the deployment method to obtain the first model includes: When the deployment method is hardware accelerator deployment, the nth module is deployed to the first hardware accelerator of the first hardware acceleration device; The first model is constituted based on the nth module deployed in the first hardware accelerator and the model deployed in the first hardware acceleration device.

4. The method according to claim 2, characterized in that, The step of adding the nth module to the model deployed on the first hardware acceleration device based on the deployment method to obtain the first model includes: When the deployment method is general-purpose processor deployment, the nth module is deployed to the first general-purpose processor of the first hardware acceleration device; The first model is constituted based on the nth module deployed in the first general-purpose processor and the model deployed in the first hardware acceleration device.

5. The method according to claim 4, characterized in that, When the deployment method is general-purpose processor deployment, the step of determining the target operator with an accuracy abnormality from at least one operator included in the nth module includes: For the m-th operator contained in the n-th module, the following processing is performed: The m-th operator deployed in the first general-purpose processor of the first hardware acceleration device is migrated to the first hardware accelerator of the first hardware acceleration device to obtain the third model, where m is a positive integer that increases sequentially, m≤M, and M is the number of operators contained in the n-th module; The m-th operator deployed in the second general-purpose processor of the second hardware acceleration device is migrated to the second hardware accelerator of the second hardware acceleration device to obtain the fourth model; The third model is trained to obtain the training result of the third model, and the fourth model is trained to obtain the training result of the fourth model; Based on the difference between the training results of the third model and the training results of the fourth model, a second accuracy difference between the third model and the fourth model is determined; When the second precision difference is greater than the second preset threshold, the m-th operator is determined as the target operator.

6. The method according to claim 1, characterized in that, The step of determining the target operator with an accuracy anomaly from at least one operator included in the nth module includes: For the m-th operator contained in the n-th module, the following processing is performed: The historical input data and historical output data of the m-th operator are determined from the training results of the first model, where m is a positive integer that increases sequentially, m≤M, and M is the number of operators contained in the n-th module; The historical input data is processed by the m-th operator deployed on the second hardware acceleration device to obtain output data; When the difference between the historical output data and the output data is greater than the difference threshold, the m-th operator is determined as the target operator.

7. The method according to claim 1, characterized in that, The process of determining the nth module from the model to be transferred includes: The model to be transferred is split into multiple modules; For each module, an accuracy anomaly prediction is performed to obtain the accuracy anomaly level of the module. Based on the accuracy anomaly level and the number of operators contained in the module, the detection priority of the module is evaluated to obtain the detection priority of the module. Based on the detection priority of each module, the multiple modules are sorted to obtain the sorted multiple modules, and the nth module is determined from the sorted multiple modules.

8. The method according to claim 7, characterized in that, The model to be transferred is modularized into multiple modules, including: Construct a computation graph of the model to be migrated, wherein the nodes in the computation graph are operators in the model to be migrated, and the edges in the computation graph are data flow dependencies between the operators; Based on preset operator fusion rules, the computation graph is divided into multiple subgraphs, and based on the multiple subgraphs, the multiple modules are determined.

9. A model processing device, characterized in that, The device includes: The model decomposition module is used to determine the nth module from the model to be transferred, wherein the nth module contains at least one operator, and n is a positive integer that increases sequentially. The model deployment module is used to add the nth module to the model deployed on the first hardware acceleration device to obtain the first model, and to add the nth module to the model deployed on the second hardware acceleration device to obtain the second model; The model training module is used to train the first model to obtain the first model training result, and to train the second model to obtain the second model training result. The difference determination module is used to determine a first accuracy difference between the first model and the second model based on the difference between the training results of the first model and the training results of the second model. The difference determination module is further configured to determine, when the first precision difference is greater than a first preset threshold, a target operator with an abnormal precision from at least one operator included in the nth module.

10. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the model processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the model processing method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the model processing method according to any one of claims 1 to 8 is implemented.

Images