Method and device for operator migration between hardware platforms, and migration model construction method

Abstract

The present application discloses a method and device for operator migration between hardware platforms, and a method for constructing a migration model. Among them, the migration method includes: obtaining the operator code and function label of the operator to be migrated; when the existing operator information of the target hardware platform does not contain the function label, obtaining the first information of the target hardware platform and the second information of the source hardware platform; training the large language model based on the first information and the second information, and using the trained large language model as the operator automatic migration development model; generating an initial operator that matches the accuracy requirements of the target hardware platform based on the operator automatic migration development model and operator code; analyzing the initial operator to obtain the operator accuracy difference; using the initial operator corresponding to the verified operator accuracy difference as the target operator; and storing the target operator to the corresponding level of the target hardware platform. This method can realize the automatic and seamless migration of model operators on different hardware platforms, and achieve efficient and seamless adaptation with different platforms.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for operator migration between hardware platforms, and a migration model construction method. Background Art

[0002] With the rapid development of deep learning and artificial intelligence models in different fields, the demand for compatibility between multiple hardware platforms (such as GPU, NPU, XPU, etc.) is increasing. However, the diversity of heterogeneous hardware architectures makes cross-platform migration of models more difficult. Currently, the migration of deep learning models on different platforms usually requires manual adjustment of special operators in the model, which requires a lot of manual intervention; existing automated migration solutions mostly rely on general operator libraries, cannot fully handle custom operators, and do not have the ability to seamlessly adapt across platforms. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a method and device for operator migration between hardware platforms, and a migration model construction method, which can solve the problems of architectural incompatibility between different heterogeneous hardware, poor adaptability caused by a large amount of manual intervention required for operator migration of the model, and low migration efficiency.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for operator migration between hardware platforms, including:

[0005] Obtaining an operator code and a function label of an operator to be migrated; the operator to be migrated is an operator to be migrated from a source hardware platform to a target hardware platform;

[0006] Obtain information about existing operators on the target hardware platform;

[0007] When the existing operator information does not include the function tag, obtaining first information of the target hardware platform and second information of the source hardware platform; the first information and the second information are both information related to generating an operator;

[0008] Training a large language model based on the first information and the second information, and using the trained large language model as an operator to automatically migrate a development model;

[0009] Analyzing the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirement of the target hardware platform;

[0010] Analyzing the initial operator to obtain operator precision difference;

[0011] Verifying the operator precision difference, and if the verification passes, determining the initial operator as the target operator;

[0012] The target operator is stored at a corresponding level of the target hardware platform.

[0013] Optionally, analyzing the operator code by using the operator automatic migration development model to generate an initial operator matching the accuracy requirement of the target hardware platform includes:

[0014] Inputting the operator code into the operator automatic migration development model to obtain the initial calculation order and initial calculation logic of the operator to be migrated;

[0015] The operator automatic migration development model generates an initial operator that matches the accuracy requirement of the target hardware platform according to the initial calculation sequence and the initial calculation logic.

[0016] Optionally, analyzing the initial operator to obtain the operator precision difference includes:

[0017] Executing the initial operator on the target hardware platform to obtain a first output;

[0018] Executing the initial operator on the source hardware platform to obtain a second output;

[0019] The first output and the second output are calculated using a mean square error to obtain an operator precision difference.

[0020] Optionally, the verifying the operator precision difference and, if the verification passes, determining the initial operator as the target operator includes:

[0021] Determine whether the operator precision difference is less than a preset threshold, if so, determine that the verification is passed, and determine that the initial operator is the target operator;

[0022] If not, it is determined that the verification fails, and a target accuracy is determined based on the accuracy requirement, and the level of the target accuracy is higher than the accuracy requirement;

[0023] A new initial operator is regenerated according to the target accuracy, and the new initial operator that has passed the verification is used as the target operator.

[0024] Optionally, if the number of times the new initial operator is regenerated is greater than a preset number threshold, the method further includes:

[0025] Selecting a target modification strategy from a plurality of candidate preset modification strategies, and executing the target modification strategy;

[0026] The multiple candidate preset modification strategies include a calculation logic modification strategy, a calculation order modification strategy and an operator replacement strategy.

[0027] Optionally, the selecting a target modification strategy from a plurality of alternative preset modification strategies includes:

[0028] Traverse multiple candidate preset modification strategies to obtain the operator precision difference corresponding to each candidate preset modification strategy after execution;

[0029] Determine whether the number of operator precision differences less than the preset threshold is greater than 1;

[0030] If not, selecting the alternative preset modification strategy corresponding to the operator precision difference less than the preset threshold as the target modification strategy;

[0031] If so, the operator delays corresponding to the candidate preset modification strategies corresponding to all operator precision differences smaller than the preset threshold are obtained, and the candidate preset modification strategy corresponding to the minimum operator delay is selected as the target modification strategy.

[0032] Optionally, obtaining the operator precision difference corresponding to each candidate preset modification strategy after execution includes:

[0033] When executing the computing logic modification strategy, modifying the initial computing logic based on the computing logic modification strategy to obtain a target computing logic;

[0034] Generate a first operator that matches the accuracy requirement of the target hardware platform according to the target calculation logic, the initial calculation sequence, and the operator automatic migration development model;

[0035] The first operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the calculation logic modification strategy.

[0036] Optionally, obtaining the operator precision difference corresponding to each candidate preset modification strategy after execution includes:

[0037] When executing the calculation order modification strategy, modifying the initial calculation order based on the calculation order modification strategy to obtain a target calculation order;

[0038] Generate a second operator that matches the accuracy requirement of the target hardware platform according to the target calculation sequence, the initial calculation logic, and the operator automatic migration development model;

[0039] The second operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the calculation order modification strategy.

[0040] Optionally, obtaining the operator precision difference corresponding to each candidate preset modification strategy after execution includes:

[0041] When executing the operator replacement strategy, other operators having the same initial calculation logic, initial calculation order, and existing operator information are used as replacement operators;

[0042] The replacement operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the operator replacement strategy.

[0043] Optionally, if the operator precision differences corresponding to each strategy obtained after execution are not less than a preset threshold, the calculation logic and the calculation order are modified based on the calculation logic modification strategy and the calculation order modification strategy.

[0044] Optionally, determining whether the existing operator information includes the function tag includes:

[0045] Obtaining a target function label corresponding to the existing operator information according to a preset operator label mapping library;

[0046] Determining whether the target function tag contains the function tag;

[0047] The method for migrating operators between hardware platforms also includes: if the target function label includes the function label, obtaining the operator name of the function label in the target hardware platform through a preset operator label mapping library, and calling the operator corresponding to the operator name.

[0048] Optionally, the first information includes a first hardware architecture, a first computing capability, a first memory configuration, a first hardware operator interface, a target platform operator code and its corresponding function label;

[0049] The second information includes a second hardware architecture, a second computing capability, a second memory configuration, a second hardware operator interface, a source platform operator code and a corresponding function tag thereof.

[0050] Optionally, the storing the target operator to the corresponding level of the target hardware platform includes: based on the underlying information of the target hardware platform and the calling frequency of the operator to be migrated, storing the corresponding target operator to the corresponding level of the target hardware platform.

[0051] In a second aspect, the embodiment of the present disclosure further provides a device for migrating operators between hardware platforms, specifically comprising:

[0052] The to-be-migrated operator information acquisition module is used to acquire the function label and operator code of the to-be-migrated operator; the to-be-migrated operator is an operator to be migrated from the source hardware platform to the target hardware platform;

[0053] The acquisition module is used to obtain the existing operator information of the target hardware platform;

[0054] A hardware perception module, used for obtaining first information of a target hardware platform and second information of a source hardware platform when the existing operator information does not contain the function tag; the first information and the second information are both information related to the generated operator;

[0055] A training module, used for training a large language model based on the first information and the second information, and automatically migrating the trained large language model as an operator to develop a model;

[0056] An operator generation module is used to analyze the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirement of the target hardware platform; analyze the initial operator to obtain the operator accuracy difference; verify the operator accuracy difference, and if the verification passes, determine that the initial operator is the target operator;

[0057] The storage module is used to store the target operator to the corresponding level of the target hardware platform.

[0058] In a third aspect, the present application discloses a migration model construction method, which specifically includes the following schemes:

[0059] Obtain the model structure and operators to be migrated;

[0060] In response to the model migration instruction, based on the method for migrating operators between hardware platforms, or starting the device for migrating operators between hardware platforms, the operator to be migrated is migrated to the target hardware platform;

[0061] Encapsulating the operator migrated to the target hardware platform and the model structure to obtain a migration model;

[0062] The migration model has a unified calling interface, and the unified calling interface includes execution commands corresponding to different platforms.

[0063] In a fourth aspect, the present application discloses a method for unified deployment of multiple hardware platforms, which specifically includes the following solutions:

[0064] Obtaining model information of the task instruction; the model information includes model structure and model operator;

[0065] Obtain all unadapted hardware platforms in the user platform, and migrate the model operators to all unadapted hardware platforms based on any one of the above methods for migrating operators between hardware platforms, or start the above device for migrating operators between hardware platforms;

[0066] Based on the model operator, obtain operator types of all operators;

[0067] When determining whether the operator type is computationally complex, if so, assigning the corresponding operator to the hardware platform with the highest computing performance in the user platform;

[0068] If not, the corresponding operator is assigned to the hardware platform with the least execution process in the user platform.

[0069] In a fifth aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:

[0070] The computer device comprises:

[0071] at least one processor; and,

[0072] a memory communicatively connected to the at least one processor; wherein,

[0073] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned methods for migrating operators between hardware platforms or the migration model building methods, or the multi-hardware platform unified deployment methods.

[0074] In the sixth aspect, the embodiments of the present disclosure also provide a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned methods for migrating operators between hardware platforms or the migration model building methods, or the above-mentioned methods for unified deployment of multiple hardware platforms.

[0075] In a seventh aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0076] The operator migration method between hardware platforms disclosed in the present application trains a large language model by combining first information of a target hardware platform and second information of a source hardware platform. During the training process, the characteristics of different hardware platforms can be considered, which can help the model reduce its dependence on specific hardware and improve the adaptability and stability of the model on different hardware platforms. By automatically generating an initial operator that matches the accuracy requirements of the target hardware platform and automatically verifying the initial operator through the trained large language model, the compatibility and performance of the determined target operator on the target hardware platform can be ensured, the requirements of the target hardware platform can be met, and seamless adaptation across platforms can be achieved, while shielding the underlying hardware differences.

[0077] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 A flowchart of a method for operator migration between hardware platforms provided in an embodiment of the present disclosure.

[0080] Figure 2 A schematic diagram of the operator mapping relationship framework between hardware platforms provided in an embodiment of the present disclosure.

[0081] Figure 3 A flowchart of a method for generating an initial operator provided in an embodiment of the present disclosure.

[0082] Figure 4 A flowchart of a method for obtaining operator precision difference provided in an embodiment of the present disclosure is provided.

[0083] Figure 5 A flowchart of a method for determining a target operator provided in an embodiment of the present disclosure.

[0084] Figure 6 A flowchart of a method for selecting a target modification strategy from a plurality of alternative preset modification strategies provided in an embodiment of the present disclosure.

[0085] Figure 7 A flowchart of a method for obtaining operator precision differences of a computing logic modification strategy provided in an embodiment of the present disclosure.

[0086] Figure 8 A flowchart of a method for obtaining operator precision differences of a calculation order modification strategy provided in an embodiment of the present disclosure.

[0087] Fig. 9 A flowchart of a method for obtaining operator precision differences of an operator replacement strategy provided in an embodiment of the present disclosure.

[0088] Fig.10 A flowchart of a method for storing target operators to corresponding levels of a target hardware platform provided in an embodiment of the present disclosure.

[0089] Fig.11 A flowchart of a migration model construction method provided in an embodiment of the present disclosure.

[0090] Fig.12 A flowchart of a method for unified deployment of multiple hardware platforms provided in an embodiment of the present disclosure.

[0091] Fig.13 A schematic diagram of the process of an apparatus for migrating operators between hardware platforms provided in an embodiment of the present disclosure.

[0092] Fig.14 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0093] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0094] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0095] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0096] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0097] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0098] Reference Figure 1 , the present application discloses a method for operator migration between hardware platforms, comprising:

[0099] S100, obtaining the operator code and function label of the operator to be migrated; the operator to be migrated is the operator to be migrated from the source hardware platform to the target hardware platform.

[0100] specifically refer to Figure 2 , through the API or code library of the source hardware platform, the operator code (i.e., the underlying implementation code) of the operator to be migrated can be obtained; then, by parsing the operator code of the operator to be migrated in the source hardware platform, the name of the operator to be migrated can be extracted.

[0101] For example, when the source hardware platform is chip A, the operator to be migrated in chip A corresponds to the hardware a operator library. According to the hardware a operator library, the name of the operator to be migrated can be obtained as: "mlu_layernorm_kernel: layernorm"; then, through the preset operator label mapping library, it can be determined that the function label corresponding to the name of the operator to be migrated is "layernorm", and the mathematical function it implements is layer normalization.

[0102] The preset operator label mapping library stores mapping relationship information between operator names in different hardware platforms and unified function labels. For example, the function label "layernorm" stored in the preset operator label mapping library has an operator name with a mapping relationship of "cude_layernorm_kernel:layernorm" in chip B, and the operator is stored in the hardware b operator library on chip B. In chip C, the operator name with a mapping relationship is "xpu_layernorm_kernel:layernorm", and the operator is stored in the hardware c operator library on chip C.

[0103] Although the mathematical functions implemented by the same function label are the same, the operator names on different hardware platforms are different due to the different underlying information of the hardware platforms.

[0104] In this embodiment, the hardware d operator library of chip D and the hardware e operator library of chip E do not include operators that implement the same mathematical function.

[0105] S200, obtaining information about existing operators of the target hardware platform.

[0106] The existing operator information (i.e., the existing operator information on the target hardware platform) includes the target platform operator code and its corresponding function label.

[0107] Specifically, you can obtain a list of all supported operators and their related information (such as operator name, parameters, accuracy requirements, etc.) through the API or documentation of the target hardware platform; fully understand the support status of the target hardware platform to avoid unsupported operators or functions during the migration process.

[0108] S300: When the existing operator information does not include the function tag, first information of the target hardware platform and second information of the source hardware platform are obtained.

[0109] The first information and the second information are both information related to the generation operator.

[0110] Specifically, the method for determining whether the function label is included in the existing operator information includes: determining the operator name in the source hardware platform according to the existing operator information; determining the function label corresponding to the operator name in the target hardware platform based on the preset operator label mapping library, which can be recorded as the target function label, and then determining whether the target function label includes the function label of the operator to be migrated. For example, when the source hardware platform is chip A and the target hardware platform is chip D, the function label of the operator to be migrated in the hardware a operator library (the operator name is: "mlu_layernorm_kernel:layernorm") is "layernorm", and the function label of the hardware d operator library determined based on the preset operator label mapping library does not include the function label "layernorm". At this time, perform the next operation.

[0111] Specifically, the first information includes first underlying information and a first basic operator data set. The first underlying information includes a first hardware architecture (such as GPU, NPU, XPU or specific accelerator), a first computing capability (floating-point computing capability, etc.), a first memory configuration (total memory and video memory bandwidth), and a first hardware operator interface; the first basic operator data set includes a first platform operator code and its corresponding function tag information.

[0112] The second information includes second underlying information and a second basic operator data set. The second underlying information includes a second hardware architecture (such as GPU, NPU, XPU or specific accelerator), a second computing capability (floating-point computing capability, etc.), a second memory configuration (total memory and video memory bandwidth), and a second hardware operator interface. The second basic operator data set includes a second platform operator code and its corresponding function tag information.

[0113] S400: Train a large language model based on the first information and the second information, and use the trained large language model as an operator to automatically migrate the development model.

[0114] By combining the first information of the target hardware platform and the second information of the source hardware platform to train the large language model (LLM), the characteristics of different hardware platforms can be considered during the training process, which can help the model reduce its dependence on specific hardware and improve the adaptability and stability of the model on different hardware platforms.

[0115] S500, analyzes the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirements of the target hardware platform.

[0116] Specifically, the initial operators automatically generated using a fine-tuned large language model (LLM) can automatically match the accuracy requirements of the target hardware platform, effectively avoiding operator incompatibility problems caused by hardware platform differences, while reducing manual intervention and improving the intelligence and efficiency of model migration.

[0117] S600, analyzing the initial operator to obtain operator precision difference.

[0118] By analyzing the operator precision difference, the migration effect of the generated initial operator can be automatically and accurately evaluated.

[0119] S700, verify the operator precision difference, and if the verification passes, determine the initial operator as the target operator.

[0120] Through verification, we ensure that the automatically generated operators meet the requirements, that is, the operators that pass the verification can run stably on the target hardware platform, which can improve the reliability of the system.

[0121] S800, storing the target operator to the corresponding level of the target hardware platform.

[0122] Specifically, the storage location of the target operator can be determined according to the architecture of the target hardware platform, such as the standard library, custom library, etc. of the hardware platform, and the target operator code can be stored in the corresponding location to ensure that it can be directly called on the target hardware platform, simplifying the development process and improving the resource utilization efficiency of the hardware platform.

[0123] The method for migrating operators between hardware platforms disclosed in the present application trains a large language model by combining first information of a target hardware platform and second information of a source hardware platform. During the training process, the characteristics of different hardware platforms can be considered, which can help the model reduce its dependence on specific hardware and improve the adaptability and stability of the model on different hardware platforms. The trained large language model is used to automatically generate initial operators that match the accuracy requirements of the target hardware platform and automatically verify the initial operators, thereby realizing intelligent identification and automatic adaptation of hardware differences, ensuring the compatibility and performance of a determined target operator on the target hardware platform, meeting the requirements of the target hardware platform, and realizing automatic, efficient, and seamless adaptation across platforms, while shielding the underlying hardware differences.

[0124] The operator migration method between hardware platforms disclosed in this application can also process custom operators, solving the limitations of existing automation solutions in processing custom operators. At the same time, automation and efficient migration reduce labor costs and time costs, and improve development efficiency. The solution can adapt to the needs of different hardware platforms, has good versatility and scalability, and helps enterprises and developers to quickly complete heterogeneous hardware migration of models, promote the application and development of deep learning technology in a diverse heterogeneous hardware ecosystem, and has broad application prospects.

[0125] The process of judging whether the existing operator information contains a function tag includes:

[0126] Obtain the target function label corresponding to the existing operator information according to the preset operator label mapping library;

[0127] Determine whether the target function tag contains the function tag.

[0128] If the existing operator information contains a function label, the operator name of the function label in the target hardware platform is obtained through the preset operator label mapping library, and the operator corresponding to the operator name is called, that is, the target hardware platform has the implementation of the operator to be migrated.

[0129] Specifically, although the name of the operator to be migrated is different from the operator name in the target hardware platform, if the mathematical function corresponding to the operator to be migrated exists in the target hardware platform, when executing the task on the target hardware platform, the operator corresponding to the same function label on the target hardware platform can be directly called, effectively reducing the amount of code and improving development efficiency. By presetting the operator label mapping library, the same operator function can be flexibly mapped on different hardware platforms, which enables the model to run on a variety of hardware without the need to develop operators separately for different hardware corresponding to the same function label.

[0130] Specifically, in the preset operator label mapping library, the function label of the operator to be migrated can be queried to obtain its corresponding operator name on the target hardware platform; wherein, the preset operator label mapping library can be generated through historical data, expert experience or automated tools to ensure the accurate correspondence between the function label and the operator name. Through the preset operator label mapping library, the corresponding operator on the target hardware platform can be quickly found, avoiding the process of generating or adjusting the operator from scratch, that is, the operator migration can be completed quickly and efficiently when the existing operator information contains the function label.

[0131] For S500, refer to Figure 3 , the generation method of the initial operator includes:

[0132] S510: Input the operator code into the operator automatic migration development model to obtain the initial calculation order and initial calculation logic of the operator to be migrated.

[0133] By extracting the initial calculation order and logic of the operator, we can deeply understand the calculation basis of the operator, which helps to ensure that the core functions and logic of the original operator are not lost during the migration process. After clarifying the initial calculation order and logic of the operator, we can perform subsequent migration steps more specifically, which can reduce unnecessary calculations and resource consumption, thereby improving migration efficiency.

[0134] S520, the operator automatic migration development model generates an initial operator that matches the accuracy requirement of the target hardware platform according to the initial calculation sequence and initial calculation logic.

[0135] Through the operator automatic migration development model, the initial operator that matches the precision requirements of the target hardware platform can be automatically generated, avoiding the tedious process of manual code writing and improving development efficiency. The generated initial operator can accurately match the precision requirements of the target hardware platform, ensuring that the performance and precision of the operator on the target platform meet expectations, which helps to improve the operation effect of the operator and reduce performance problems caused by precision mismatch.

[0136] The use of the operator automatic migration development model makes the operator migration process more intelligent and efficient. The model can be optimized according to the characteristics of different hardware platforms to generate operator codes that are more suitable for the platform. At the same time, the automatic generation process reduces the possibility of human errors and ensures the functional and logical correctness of the generated initial operators, laying a good foundation for subsequent verification and optimization.

[0137] For S600, refer to Figure 4 , the method for obtaining the operator precision difference includes:

[0138] S610, executing the initial operator on the target hardware platform to obtain a first output; executing the initial operator on the source hardware platform to obtain a second output.

[0139] The first output includes the first platform operator output and the first platform operator delay, and the second output includes the second platform operator output and the second platform operator delay.

[0140] By executing the initial operator on the target hardware platform and the source hardware platform respectively, real performance data can be obtained, including operator output and latency; the process of executing the initial operator can capture the characteristic differences between the target hardware platform and the source hardware platform, thus providing a basis for subsequent optimization and adjustment.

[0141] S620, using mean square error to calculate the first output and the second output to obtain an operator precision difference.

[0142] .

[0143] in, The first output The value of the position, The second output The value of the position, is the number of elements.

[0144] The operator precision difference is obtained by the method disclosed in this step, which can accurately quantify the difference between the first output and the second output, that is, capture the subtle precision difference, which makes the calculation result of the precision difference more detailed and reliable, and helps to discover potential performance problems.

[0145] For S700, refer to Figure 5 , the method for determining the target operator includes

[0146] Determine whether the operator precision difference is less than a preset threshold. If so, the verification is considered to be successful, and the initial operator is determined to be the target operator.

[0147] If not, it is determined that the verification fails, and the target accuracy is determined based on the accuracy requirement, and the level of the target accuracy is higher than the accuracy requirement; a new initial operator is regenerated according to the target accuracy, and the new initial operator that passes the verification is used as the target operator.

[0148] Among them, the preset threshold is preferably 0.00001.

[0149] In this embodiment, by setting a preset threshold, the operator quality can be effectively controlled. Only when the operator precision difference is within an acceptable range is the initial operator accepted as the target operator to ensure that the performance of the operator on the target hardware platform meets expectations. Automatically judging whether the precision difference is less than the preset threshold reduces the need for human intervention and improves the efficiency and consistency of decision-making, which helps to quickly screen out operators that meet the requirements and avoid the tedious process of manual verification.

[0150] When the accuracy difference is not less than the preset threshold, the target accuracy can be determined according to the accuracy requirements, and a new initial operator can be regenerated. This provides flexibility for operator optimization, allowing developers to adjust the accuracy of the operator according to actual needs. By setting the target accuracy level higher than the accuracy requirement, the operator can be regenerated under a higher standard, ensuring that the final generated operator not only meets the basic accuracy requirements, but also provides higher performance.

[0151] The process of regenerating a new initial operator and verifying it is an iterative optimization process. By repeating this process, the performance of the operator can be continuously optimized, gradually approaching or even exceeding the accuracy requirements initially set.

[0152] Through multiple iterations and verifications, the target operator finally generated can meet or even exceed the initial accuracy requirements, provide higher computing accuracy and performance guarantees, and the reliability of operators that have been repeatedly verified and optimized has been significantly improved. This helps reduce the possibility of performance problems and errors in actual applications and ensures the stable operation of the system.

[0153] Furthermore, the method for determining the number of times a new initial operator is regenerated includes: when the number of times a new initial operator is regenerated is greater than a preset number threshold, selecting a target modification strategy from a plurality of candidate preset modification strategies, and executing the target modification strategy.

[0154] Among them, the multiple candidate preset modification strategies include a calculation logic modification strategy, a calculation order modification strategy and an operator replacement strategy.

[0155] Among them, the preset number threshold is preferably 5.

[0156] By setting a preset number threshold (such as 5 times), an upper limit is set for the iteration process, which helps to avoid infinite loops and waste of resources, and ensures that the optimization process finds a balance between high efficiency and controllability; when the number of times the initial operator is regenerated exceeds the preset number threshold, it automatically enters the alternative strategy selection stage, reducing the possibility of continuing meaningless iterations, which can effectively improve the efficiency of decision-making and avoid wasting time on inefficient attempts.

[0157] Multiple alternative preset modification strategies (such as calculation logic modification strategy, calculation order modification strategy, and operator replacement strategy) provide diverse optimization paths. When the accuracy requirements cannot be met after multiple iterations, diverse strategies can provide new ideas and solutions.

[0158] Different modification strategies can be used to deal with different types of precision problems. By selecting the target modification strategy, we can flexibly respond to various complex optimization requirements and provide targeted solutions. Diversified strategy selection increases the possibility of successful optimization. By trying different strategies, we can find more effective optimization methods and improve the final performance of the operator.

[0159] By executing the target modification strategy, the probability of successful optimization can be increased. Compared with a single iterative generation, diversified strategy selection and execution can better cope with complex optimization problems and ensure that the operator finally generated meets the accuracy requirements. In the case of limited resources, this strategy selection can better utilize resources and achieve more efficient optimization.

[0160] Reference Figure 6 , a method for selecting a target modification strategy from a plurality of alternative preset modification strategies, comprising:

[0161] Traverse multiple candidate preset modification strategies to obtain the operator precision difference corresponding to each candidate preset modification strategy after execution;

[0162] Determine whether the precision differences of all operators are less than the preset threshold. If not, use the calculation logic modification strategy and the calculation order modification strategy as the target modification strategy;

[0163] If so, determine whether the number of operator precision differences less than the preset threshold is greater than 1, and if not, select the alternative preset modification strategy corresponding to the operator precision difference less than the preset threshold as the target modification strategy;

[0164] If yes (that is, the number of operator precision differences less than the preset threshold is greater than 1), obtain the operator delays corresponding to the alternative preset modification strategies corresponding to all operator precision differences less than the preset threshold, and select the alternative preset modification strategy corresponding to the minimum operator delay as the target modification strategy.

[0165] In this embodiment, if the operator precision difference corresponding to each strategy obtained after execution is not less than the preset threshold, the initial calculation logic and initial calculation order are modified simultaneously based on the calculation logic modification strategy and the calculation order modification strategy to ensure the effectiveness of the modification and avoid unnecessary waste of steps.

[0166] If the operator precision difference corresponding to each strategy obtained after execution is less than a preset threshold, a strategy is selected from multiple candidate preset modification strategies as the target modification strategy.

[0167] By traversing multiple alternative preset modification strategies, the impact of each strategy on the operator precision difference can be comprehensively evaluated; setting the threshold of operator precision difference can effectively control the operator quality. Only when the precision difference of all strategies is less than 1, further selection is made to ensure that the final selected target strategy can meet the basic precision requirements.

[0168] By setting a unified accuracy difference threshold, the selection process is simplified. If the accuracy difference of all strategies is less than 1, it means that these strategies have reached an acceptable level of accuracy and can be further selected based on other criteria (such as latency).

[0169] When the accuracy difference of all strategies is no less than 1, the calculation logic modification strategy and the calculation order modification strategy are preferred. This shows that these strategies have a higher priority in dealing with accuracy issues and help to quickly find solutions to the problem. The modification of calculation logic and calculation order usually has a direct impact on accuracy issues. Selecting these two strategies as target modification strategies can more accurately solve accuracy issues and improve the performance of operators.

[0170] By judging whether the number of operator precision differences that are less than the preset threshold is greater than 1, the accuracy performance of multiple strategies can be evaluated, which helps to discover the advantages and disadvantages of multiple strategies in terms of accuracy and provide a basis for subsequent selection; if the precision differences of multiple strategies are all less than the preset threshold, it means that these strategies have reached an acceptable level in terms of accuracy. In this case, a strategy that further optimizes the latency can be selected to improve the overall performance of the operator.

[0171] Selecting the strategy corresponding to the smallest operator latency can optimize the latency performance of the operator, help improve the operating efficiency of the operator, reduce resource consumption, and improve overall performance; further optimizing latency while ensuring accuracy can achieve a balance between accuracy and performance, help improve the overall performance of the operator, and provide a better user experience; selecting the strategy with the smallest latency can quickly find the strategy with the best performance, reducing the complexity and uncertainty in the selection process.

[0172] Among them, when the number of operator precision differences less than the preset threshold is greater than 1, the number of operator precision differences less than the preset threshold may be 3 or 4. Therefore, the operator delays corresponding to the alternative preset modification strategies corresponding to all operator precision differences less than the preset threshold are obtained, and the alternative preset modification strategy corresponding to the smallest operator delay is selected as the target modification strategy.

[0173] Reference Figure 7 , a method for obtaining the operator precision difference of the calculation logic modification strategy includes:

[0174] A110, modify the initial calculation logic based on the calculation logic modification strategy to obtain the target calculation logic.

[0175] By modifying the initial calculation logic, the calculation logic of the operator can be optimized in a targeted manner, specific accuracy problems can be solved, and the performance of the operator can be improved; according to different optimization requirements, the calculation logic can be flexibly adjusted and diversified optimization paths can be provided to cope with complex optimization problems and increase the possibility of successful optimization.

[0176] A120, based on the target computing logic, initial computing order, and operator automatic migration development model, generates the first operator that matches the accuracy requirements of the target hardware platform.

[0177] Specifically, the target calculation logic and the initial calculation order are used as inputs of the operator automatic migration development model, and the first operator matching the accuracy requirement of the target hardware platform is generated through the operator automatic migration development model.

[0178] Through the operator automatic migration development model, the first operator generated can match the accuracy requirements of the target hardware platform, which helps to ensure that the performance of the operator on the target hardware platform meets expectations and improves the final accuracy performance; the operator automatic migration development model can automatically generate operators, reducing the complexity and time cost of manual development, effectively improving development efficiency, and ensuring that operators can be quickly applied to actual scenarios.

[0179] A130, executing the first operator on the target hardware platform and the source hardware platform respectively, calculating the execution results of the two platforms by using mean square error, and obtaining the operator precision difference of the calculation logic modification strategy.

[0180] Executing the first operator on the target hardware platform and the source hardware platform respectively and calculating the mean square error can compare the operator accuracy on the two platforms, which helps to evaluate the performance of the operator on different hardware platforms and ensure that the performance of the operator on the target hardware platform meets expectations; through the mean square error calculation, the impact of the calculation logic modification strategy on the operator accuracy can be quantified, which provides an objective evaluation standard and facilitates subsequent decision-making and optimization; executing the operator on the two platforms and calculating the accuracy difference can verify the performance of the operator, help discover potential problems and deficiencies, and further improve the performance of the operator.

[0181] Reference Figure 8 , a method for obtaining the operator precision difference of the calculation order modification strategy includes:

[0182] A210, modifying the initial calculation order based on the calculation order modification strategy to obtain the target calculation order.

[0183] By modifying the initial calculation order, the calculation process of the operator can be optimized, unnecessary calculation steps can be reduced, and calculation efficiency can be improved; targeted optimization can be carried out for specific calculation order problems, which will help solve accuracy problems and performance bottlenecks; according to different optimization requirements, the calculation order can be flexibly adjusted to provide a variety of optimization paths and increase the possibility of successful optimization.

[0184] A220, based on the target calculation sequence, initial calculation logic, and operator automatic migration development model, generates a second operator that matches the accuracy requirements of the target hardware platform.

[0185] Through the operator automatic migration development model, the generated second operator can match the accuracy requirements of the target hardware platform, which helps to ensure that the performance of the operator on the target hardware platform meets expectations and improves the final accuracy performance; the operator automatic migration development model can automatically generate operators, reducing the complexity and time cost of manual development, which improves development efficiency and ensures that operators can be quickly applied to actual scenarios; the automatic migration development model can ensure the consistency of the generated operators in calculation logic and calculation order, reduce human errors, and improve the reliability of operators.

[0186] A230, execute the second operator on the target hardware platform and the source hardware platform respectively, calculate the execution results of the two platforms using the mean square error, and obtain the operator precision difference of the calculation order modification strategy.

[0187] By executing the second operator on the target hardware platform and the source hardware platform respectively and calculating the mean square error, the operator accuracy on the two platforms can be compared, which helps to evaluate the performance of the operator on different hardware platforms and ensure that the performance of the operator on the target hardware platform meets expectations; through the mean square error calculation, the impact of the calculation order modification strategy on the operator accuracy can be quantified, which provides an objective evaluation standard to facilitate subsequent decision-making and optimization; executing the operator on the two platforms and calculating the accuracy difference can verify the performance of the operator, help to discover potential problems and deficiencies, and further improve the performance of the operator.

[0188] Reference Fig. 9 , the method for obtaining the operator precision difference of the operator replacement strategy includes:

[0189] A310, other operators with the same initial calculation logic, initial calculation order, and existing operator information are used as replacement operators.

[0190] Selecting alternative operators with the same calculation logic and calculation order can simplify the replacement process. Because these alternative operators are similar to the original operators in function, the complexity and risk of replacement are reduced; alternative operators with the same calculation logic and calculation order are usually more compatible with existing systems and hardware platforms, which helps to ensure that the replaced operators can be seamlessly integrated into the existing system; selecting alternative operators with existing operator information can improve the reusability of operators. These alternative operators have been verified in other projects or scenarios, reducing the cost of redevelopment and testing.

[0191] A320, executes the replacement operator on the target hardware platform and the source hardware platform respectively, and uses the mean square error to calculate the execution results of the two platforms to obtain the operator precision difference of the operator replacement strategy.

[0192] By executing the replacement operator on the target hardware platform and the source hardware platform respectively and calculating the mean square error, the operator accuracy on the two platforms can be compared, which helps to evaluate the performance of the replacement operator on different hardware platforms and ensure that the performance of the replaced operator on the target hardware platform meets expectations; through the calculation of the mean square error, the impact of the operator replacement strategy on the operator accuracy can be quantified, which provides an objective evaluation standard to facilitate subsequent decision-making and optimization; by executing the replacement operator on the two platforms and calculating the accuracy difference, the performance of the replacement operator can be verified, which helps to discover potential problems and deficiencies and further improve the performance of the replacement operator; by executing the replacement operator on different platforms, the reliability and stability of the replacement operator can be verified to ensure that the replacement operator performs consistently in different environments.

[0193] It should be noted that there is no order restriction for obtaining the operator precision difference of the calculation logic modification strategy, obtaining the operator precision difference of the calculation order modification strategy, and obtaining the operator precision difference of the operator replacement strategy. Any of the three strategies can be obtained first. The step description in this embodiment is only for the convenience of discussion and does not limit the scope of protection of this application.

[0194] Reference Fig.10 , a method for storing a target operator to a corresponding level of a target hardware platform, comprising:

[0195] B110, based on the first hardware architecture, the first computing capability, the first memory configuration, the first hardware operator interface, and the calling frequency of the operator to be migrated, obtain the memory management policy of the target operator.

[0196] Specifically, first, it is necessary to analyze the architecture type of the target hardware, such as CPU, GPU, ASIC, etc. Different hardware architectures have different memory access modes and computing characteristics; evaluate the computing power of the hardware, such as floating-point computing power (FLOPS), memory bandwidth (GB / s), etc. Hardware with high computing power is usually more suitable for processing high-frequency operators; check the memory configuration of the hardware, including memory size, memory type (such as DDR4, HBM, etc.), memory bandwidth and memory latency. Different memory configurations will affect the storage and access strategies of operators; identify the operator interfaces supported by the hardware, such as CUDA, OpenCL, TVM, etc.; different interfaces may have different optimization strategies for memory management; count the call frequency of the operators to be migrated in the model, and high-frequency operators usually need to be optimized first to reduce access latency. Based on the above analysis, a comprehensive evaluation is performed to generate the memory management strategy of the target operator.

[0197] B120, stores the corresponding target operator to the corresponding level of the target hardware platform according to the memory management policy.

[0198] Specifically, determine the type and hierarchical structure of the target hardware platform, select the most suitable memory area (such as high-bandwidth memory, low-latency memory, etc.) according to the memory management policy, and store the target operator in the selected memory area.

[0199] In this embodiment, the hardware storage is in a pyramid shape, where operators with lower calling frequencies are on worse physical hardware, and operators with higher calling frequencies are on better physical hardware.

[0200] Specifically, the memory management of operators can be automatically optimized in combination with hardware-specific memory configurations (such as shared memory, L2 cache, etc.) to avoid redundant data transmission. For operators that are frequently called, their data is cached at an appropriate level, such as the GPU's L2 cache, and on-chip memory is used to store weight data on the NPU to reduce dependence on main memory.

[0201] Furthermore, when the call frequency of the operator to be migrated is greater than the first preset call threshold, that is, the operator to be migrated is a high-frequency call operator, the determined architecture is: adopting distributed storage and local caching strategies, requiring higher computing power, and storing in high-bandwidth memory to maximize data transmission speed and reduce computing waiting time. Specifically, it can be stored in the CPU first-level cache, such as the shared memory of the GPU or the HBM of the TPU, and the configured interface is PCIe or NVMe.

[0202] When the call frequency of the operator to be migrated is not greater than the first preset call threshold and greater than the second preset call threshold, that is, the operator to be migrated is a medium-frequency call operator, a hybrid storage strategy can be adopted, combining high-speed memory and medium-speed storage devices, and storing them in the CPU secondary cache. The configured interface can be SATA, SAS, etc.

[0203] When the call frequency of the operator to be migrated is not greater than the second preset call threshold, that is, the operator to be migrated is a low-frequency call operator, it can mainly rely on HDD as a storage device, and can choose low computing power, low-cost memory (such as DDR4 or DDR5), and low-cost interface (such as SATA).

[0204] Reference Fig.11 The second aspect of the present application discloses a migration model construction method, comprising:

[0205] B10, obtain the model structure and operators to be migrated.

[0206] Obtaining the model structure and operators to be migrated can fully grasp the information of the model and operators, which helps to better understand the overall architecture and components of the model, and provide detailed information support for migration and optimization; clarifying the operators and model structure to be migrated, ensuring that the migration process has clear goals and directions, and reducing uncertainties and risks in the migration process.

[0207] B20, in response to the model migration instruction, based on the operator migration method between hardware platforms, or starting the operator migration device between hardware platforms, migrate the operator to be migrated to the target hardware platform.

[0208] By using the operator migration method between hardware platforms or starting the operator migration device between hardware platforms, the automatic migration of operators can be achieved, which effectively reduces the complexity and time cost of manual operations and improves the migration efficiency; the operators to be migrated are migrated to the target hardware platform to ensure the compatibility and performance of the operators on the target hardware platform, which helps to solve the problem of cross-platform migration and improve the applicability of the model on different hardware platforms; according to different migration requirements, the migration method and device can be flexibly selected to provide a variety of migration paths and increase the possibility of successful migration.

[0209] B30, encapsulate the operator and model structure migrated to the target hardware platform to obtain the migration model;

[0210] The migration model has a unified calling interface, which includes execution commands corresponding to different platforms.

[0211] The operators and model structures migrated to the target hardware platform are encapsulated to achieve modular management, which helps to simplify the model structure and improve the maintainability and scalability of the model; the migration model has a unified calling interface, including execution commands corresponding to different platforms, which simplifies the complexity of model calling, ensures that the model is called in a consistent manner on different platforms, and improves the efficiency of development and deployment; through the unified calling interface, the compatibility and consistency of the migration model on different hardware platforms are ensured, which helps to achieve cross-platform deployment of the model and improve the applicability of the model; the unified calling interface simplifies the model calling process and reduces the workload of developers. Developers do not need to write different calling codes for different platforms, reducing development difficulty and maintenance costs.

[0212] Reference Fig.12 The third aspect of the present application discloses a method for unified deployment of multiple hardware platforms, including:

[0213] C10, obtains the model information of the task instruction; the model information includes the model structure and the model operator.

[0214] Obtaining the model structure and model operators can fully grasp the information of the model, which helps to better understand the overall architecture and components of the model, and provide detailed information support for subsequent migration and allocation; clarifying the model structure and operators ensures that the deployment process has clear goals and directions, reducing uncertainties and risks in the deployment process.

[0215] C20, obtain all unadapted hardware platforms in the user platform, and migrate the model operators to all unadapted hardware platforms based on the operator migration method between hardware platforms, or start the operator migration device between hardware platforms.

[0216] By using the operator migration method between hardware platforms or starting the operator migration device between hardware platforms, the automatic migration of operators can be achieved, which can effectively reduce the complexity and time cost of manual operations and improve the migration efficiency; the model operators can be migrated to all unadapted hardware platforms to ensure the compatibility and performance of the operators on the target hardware platforms, which helps to solve the problem of cross-platform migration and improve the applicability of the model on different hardware platforms; according to different deployment requirements, the migration methods and devices can be flexibly selected to provide diversified migration paths and increase the possibility of successful migration.

[0217] C30, based on the model operator, obtains the operator types of all operators.

[0218] By obtaining the operator types of all operators and implementing classified management of operators, it is helpful to carry out targeted deployment and optimization according to the characteristics of the operators and improve the overall deployment efficiency. Clarifying the operator types facilitates targeted allocation and deployment according to the characteristics and requirements of the operators and optimizes resource utilization.

[0219] C40, when judging whether the operator type belongs to the computationally complex type, if so, the corresponding operator is allocated to the hardware platform with the highest computing performance in the user platform;

[0220] C50, if not, the corresponding operator is allocated to the hardware platform with the least execution process in the user platform.

[0221] Assigning computationally complex operators to the hardware platform with the highest computing performance can fully utilize the computing resources of the hardware platform, improve computing efficiency, and help ensure the efficient execution of computationally complex operators on high-performance hardware platforms. By assigning computationally complex operators to high-performance hardware platforms, the overall performance and response speed of the model can be significantly improved to meet the needs of high-performance computing. Assigning non-computationally complex operators to the hardware platform with the least execution processes can achieve load balancing, help avoid overloading of certain hardware platforms, and ensure that the resources of each hardware platform are reasonably utilized. By assigning non-computationally complex operators to hardware platforms with idle resources, energy consumption can be reduced and the operating costs of the hardware platform can be reduced.

[0222] Specifically, several operators in the model operators can be sorted according to the calling frequency of each operator to obtain the sorting of operator calling frequency from high to low. The operator type can be judged starting from the operator with the highest calling frequency, and corresponding allocation can be made according to the judgment result.

[0223] The unified deployment method for multiple hardware platforms disclosed in this application includes the following two scenarios: Scenario 1: the model already exists on any hardware platform of the user platform, and during training, the participation of other hardware platforms in the user platform is required; Scenario 2: the model has not been adapted to the current user platform, and currently it is necessary to call several hardware platforms in the current user platform to perform model training tasks.

[0224] For scenario one, because the model already exists on any hardware platform of the user platform, that is, the model is certain, therefore, the model information of the task instruction can be obtained, wherein the model information includes the model structure and the model operator; then all the unadapted hardware platforms in the user platform are obtained, and based on the operator migration method between hardware platforms, or, the operator migration device between hardware platforms is started, the model operators are migrated to all the unadapted hardware platforms, and the model operators in the determined model are seamlessly adapted on the user platform, that is, all hardware platforms can be called.

[0225] When executing a specific task, the operator types of all operators in the model operator are obtained, and each operator is assigned to the corresponding hardware platform according to its operator type, and the task can be executed in the corresponding hardware platform.

[0226] For scenario 2, because the model does not exist on any hardware platform of the user platform, that is, the model is uncertain, therefore, the model and model information of the task instruction can be determined first; the model information includes the model structure and model operators; then based on the operator migration method between hardware platforms, or starting the operator migration device between hardware platforms, the model operator is seamlessly adapted to all hardware platforms, and the model operator in the determined model is seamlessly migrated on the user platform, that is, all hardware platforms can be called. Similarly, when executing a specific task, the operator types of all operators in the model operator are obtained, and each operator is assigned to the corresponding hardware platform according to its operator type, and the task can be executed on the corresponding hardware platform.

[0227] The third aspect of the present application discloses a unified deployment method for multiple hardware platforms, which can automatically optimize the parallel computing mode and maximize the computing resource utilization of multiple heterogeneous chips by analyzing the data dependencies between operators. For computing or memory intensive operators, the system decomposes and distributes the workload to different computing units of the hardware (such as the SM unit of the GPU or the NPU core).

[0228] Reference Fig.13 In a fourth aspect, the present application discloses a device for migrating operators between hardware platforms, including:

[0229] The to-be-migrated operator information acquisition module 11 is used to acquire the function label and operator code of the to-be-migrated operator; the to-be-migrated operator is an operator to be migrated from the source hardware platform to the target hardware platform;

[0230] The acquisition module 12 is used to obtain the existing operator information of the target hardware platform;

[0231] The hardware perception module 13 is used to obtain the first information of the target hardware platform and the second information of the source hardware platform when the existing operator information does not contain the function label; the first information and the second information are both information related to the generated operator;

[0232] A training module 14, used to train the large language model based on the first information and the second information, and automatically migrate the trained large language model as an operator to develop a model;

[0233] The operator generation module 15 is used to analyze the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirement of the target hardware platform; analyze the initial operator to obtain the operator accuracy difference; verify the operator accuracy difference, and if the verification passes, determine that the initial operator is the target operator;

[0234] The storage module 16 is used to store the target operator to the corresponding level of the target hardware platform.

[0235] It should be noted that the solutions in the method for migrating operators between hardware platforms disclosed in the first aspect of this application are all applicable to the acquisition of the specific solutions in the device for migrating operators between hardware platforms disclosed in the fourth aspect of this application, so they will not be repeated here.

[0236] Furthermore, the operator migration method between hardware platforms disclosed in the first aspect of the present application can be encapsulated in a tool chain, which will automatically generate configuration files or scripts adapted to each hardware platform, automatically set environmental parameters, and support unified deployment and centralized management of multiple platforms; specifically, by encapsulating hardware details and customized operators through the tool chain, the migrated model can be called using a unified interface on the new hardware platform, thereby achieving unawareness of the underlying hardware differences at the application layer, and the migrated model runs in a unified manner on the target hardware, achieving truly seamless switching.

[0237] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0238] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the operator migration method between hardware platforms or the migration model construction method of each embodiment of the present disclosure, or the unified deployment method of multiple hardware platforms.

[0239] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0240] like Fig.14 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Fig.14 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0241] like Fig.14 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0242] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Fig.14 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0243] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the operator migration method between hardware platforms of an embodiment of the present disclosure or the migration model construction method, or the multi-hardware platform unified deployment method are executed.

[0244] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0245] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by the processor, all or part of the steps of the operator migration method between hardware platforms or the migration model construction method or the multi-hardware platform unified deployment method of the aforementioned embodiments of the present disclosure are executed.

[0246] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0247] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0248] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0249] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0250] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0251] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0252] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0253] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0254] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for operator migration between hardware platforms, characterized in that: include: Get the operator code and function label of the operator to be migrated; The operator to be migrated is an operator to be migrated from a source hardware platform to a target hardware platform; Obtain information about existing operators on the target hardware platform; When the existing operator information does not include the function tag, obtaining first information of the target hardware platform and second information of the source hardware platform; the first information and the second information are both information related to generating an operator; Training a large language model based on the first information and the second information, and using the trained large language model as an operator to automatically migrate a development model; Analyzing the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirement of the target hardware platform; Analyzing the initial operator to obtain operator precision difference; Verifying the operator precision difference, and if the verification passes, determining the initial operator as the target operator; Storing the target operator to a corresponding level of the target hardware platform; The analyzing the initial operator to obtain the operator precision difference includes: executing the initial operator on the target hardware platform to obtain a first output; executing the initial operator on the source hardware platform to obtain a second output; and calculating the first output and the second output using a mean square error to obtain the operator precision difference.

2. The method for operator migration between hardware platforms according to claim 1, characterized in that: The analyzing the operator code by the operator automatic migration development model to generate an initial operator matching the accuracy requirement of the target hardware platform includes: Inputting the operator code into the operator automatic migration development model to obtain the initial calculation order and initial calculation logic of the operator to be migrated; The operator automatic migration development model generates an initial operator that matches the accuracy requirement of the target hardware platform according to the initial calculation sequence and the initial calculation logic.

3. The method for operator migration between hardware platforms according to claim 2, characterized in that: The verifying the operator precision difference, and if the verification passes, determining the initial operator as the target operator, includes: Determine whether the operator precision difference is less than a preset threshold, if so, determine that the verification is passed, and determine that the initial operator is the target operator; If not, it is determined that the verification fails, and a target accuracy is determined based on the accuracy requirement, and the level of the target accuracy is higher than the accuracy requirement; A new initial operator is regenerated according to the target accuracy, and the new initial operator that has passed the verification is used as the target operator.

4. The method for operator migration between hardware platforms according to claim 3, characterized in that: If the number of times the new initial operator is regenerated is greater than a preset number threshold, the method further includes: Selecting a target modification strategy from a plurality of candidate preset modification strategies, and executing the target modification strategy; The multiple candidate preset modification strategies include a calculation logic modification strategy, a calculation order modification strategy and an operator replacement strategy.

5. The method for operator migration between hardware platforms according to claim 4, characterized in that: The step of selecting a target modification strategy from a plurality of alternative preset modification strategies includes: Traverse multiple candidate preset modification strategies to obtain the operator precision difference corresponding to each candidate preset modification strategy after execution; Determine whether the number of operator precision differences less than the preset threshold is greater than 1; If not, selecting the alternative preset modification strategy corresponding to the operator precision difference less than the preset threshold as the target modification strategy; If so, the operator delays corresponding to the candidate preset modification strategies corresponding to all operator precision differences smaller than the preset threshold are obtained, and the candidate preset modification strategy corresponding to the minimum operator delay is selected as the target modification strategy.

6. The method for operator migration between hardware platforms according to claim 5, characterized in that: The obtaining of the operator precision difference corresponding to each candidate preset modification strategy after execution includes: When executing the computing logic modification strategy, modifying the initial computing logic based on the computing logic modification strategy to obtain a target computing logic; Generate a first operator that matches the accuracy requirement of the target hardware platform according to the target calculation logic, the initial calculation sequence, and the operator automatic migration development model; The first operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the calculation logic modification strategy.

7. The method for operator migration between hardware platforms according to claim 5, characterized in that: The obtaining of the operator precision difference corresponding to each candidate preset modification strategy after execution includes: When executing the calculation order modification strategy, modifying the initial calculation order based on the calculation order modification strategy to obtain a target calculation order; Generate a second operator that matches the accuracy requirement of the target hardware platform according to the target calculation sequence, the initial calculation logic, and the operator automatic migration development model; The second operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the calculation order modification strategy.

8. The method for operator migration between hardware platforms according to claim 5, characterized in that: The obtaining of the operator precision difference corresponding to each candidate preset modification strategy after execution includes: When executing the operator replacement strategy, other operators having the same initial calculation logic, initial calculation order, and existing operator information are used as replacement operators; The replacement operator is executed on the target hardware platform and the source hardware platform respectively, and the execution results of the two platforms are calculated using mean square error to obtain the operator precision difference of the operator replacement strategy.

9. The method for operator migration between hardware platforms according to claim 5, characterized in that: If the operator precision differences corresponding to each strategy obtained after execution are not less than a preset threshold, the initial calculation logic and the initial calculation order are modified based on the calculation logic modification strategy and the calculation order modification strategy.

10. The method for operator migration between hardware platforms according to claim 1, characterized in that: Determining whether the existing operator information contains the function tag includes: Obtaining a target function label corresponding to the existing operator information according to a preset operator label mapping library; Determining whether the target function tag contains the function tag; The method for migrating operators between hardware platforms also includes: if the target function label includes the function label, obtaining the operator name of the function label in the target hardware platform through a preset operator label mapping library, and calling the operator corresponding to the operator name.

11. The method for operator migration between hardware platforms according to claim 1, characterized in that: The first information includes a first hardware architecture, a first computing capability, a first memory configuration, a first hardware operator interface, a target platform operator code and its corresponding function label; The second information includes a second hardware architecture, a second computing capability, a second memory configuration, a second hardware operator interface, a source platform operator code and a corresponding function tag thereof.

12. The method for operator migration between hardware platforms according to claim 11, characterized in that: The storing of the target operator to the corresponding level of the target hardware platform includes: based on the underlying information of the target hardware platform and the calling frequency of the operator to be migrated, storing the corresponding target operator to the corresponding level of the target hardware platform.

13. A device for migrating operators between hardware platforms, characterized in that: include: The module for obtaining information of operators to be migrated is used to obtain function labels and operator codes of operators to be migrated; The operator to be migrated is an operator to be migrated from a source hardware platform to a target hardware platform; The acquisition module is used to obtain the existing operator information of the target hardware platform; A hardware perception module, used for obtaining first information of a target hardware platform and second information of a source hardware platform when the existing operator information does not contain the function tag; the first information and the second information are both information related to the generated operator; A training module, used for training a large language model based on the first information and the second information, and automatically migrating the trained large language model as an operator to develop a model; An operator generation module is used to analyze the operator code through the operator automatic migration development model to generate an initial operator that matches the accuracy requirement of the target hardware platform; analyze the initial operator to obtain the operator accuracy difference; The operator precision difference is verified, and if the verification passes, the initial operator is determined to be the target operator; the analyzing the initial operator to obtain the operator precision difference includes: executing the initial operator on the target hardware platform to obtain a first output; executing the initial operator on the source hardware platform to obtain a second output; calculating the first output and the second output using a mean square error to obtain the operator precision difference; A storage module is used to store the target operator to the corresponding level of the target hardware platform.

14. A migration model construction method, characterized in that: include: Obtain the model structure and operators to be migrated; In response to the model migration instruction, based on the method for migrating operators between hardware platforms according to any one of claims 1 to 12, or starting the device for migrating operators between hardware platforms according to claim 13, the operator to be migrated is migrated to the target hardware platform; Encapsulating the operator migrated to the target hardware platform and the model structure to obtain a migration model; The migration model has a unified calling interface, and the unified calling interface includes execution commands corresponding to different platforms.

15. A method for unified deployment of multiple hardware platforms, characterized in that: include: Obtaining model information of the task instruction; the model information includes model structure and model operator; Obtain all unadapted hardware platforms in the user platform, and migrate the model operators to all unadapted hardware platforms based on the method for migrating operators between hardware platforms described in any one of claims 1 to 12, or start the device for migrating operators between hardware platforms described in claim 13; Based on the model operator, obtain operator types of all operators; When determining whether the operator type is computationally complex, if so, assigning the corresponding operator to the hardware platform with the highest computing performance in the user platform; If not, the corresponding operator is assigned to the hardware platform with the least execution process in the user platform.

16. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the operator migration method between hardware platforms described in any one of claims 1-12 or the migration model construction method described in claim 14, or the unified deployment method for multiple hardware platforms described in claim 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the operator migration method between hardware platforms described in any one of claims 1 to 12, the migration model construction method described in claim 14, or the unified deployment method for multiple hardware platforms described in claim 15.

18. A computer program product comprising computer instructions, characterized in that When the computer instruction is executed by the processor, the steps of the claim are implemented based on the method for migrating operators between hardware platforms described in any one of claims 1 to 12, the method for building a migration model described in claim 14, or the method for unified deployment of multiple hardware platforms described in claim 15.

Images