Lightweight method and device for large model facing industrial scene and edge inference acceleration

Abstract

The present disclosure relates to the technical field of big data, and specifically discloses a method and device for lightweight and edge inference acceleration of a large model for an industrial scene. The method comprises: obtaining an original large model, performing lightweight processing on the original large model to obtain an intermediate large model; taking the original large model as a teacher model, performing knowledge distillation training on the intermediate large model to obtain a target large model; wherein the knowledge distillation training comprises migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model; determining a deployment strategy of the target large model according to the hardware type of a target edge device; and deploying the target large model to the target edge device according to the deployment strategy. The method can effectively reduce the calculation complexity and storage requirement of the model while retaining high inference accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, in particular to a large model lightweight and edge inference acceleration method and device for industrial scenarios, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] With the rapid development of industrial intelligence, the application of large model technology based on deep learning in the fields of industrial visual detection and equipment fault diagnosis is increasingly widespread. In traditional technology, a pre-trained large model is usually directly deployed on an industrial computing device for running, or a general model compression method is used to simply prune the model before deployment. However, the existing industrial large model deployment method faces a serious performance and efficiency imbalance problem: when the large model is deployed to a resource-limited edge device, due to the high model calculation complexity and large memory occupation, the inference speed cannot meet the real-time requirements of industry. SUMMARY

[0003] Therefore, it is necessary to provide a large model lightweight and edge inference acceleration method and device for industrial scenarios, computer equipment, a computer readable storage medium and a computer program product in view of the above technical problems.

[0004] In a first aspect, the present application provides a large model lightweight and edge inference acceleration method for industrial scenarios. The method comprises:

[0005] obtaining an original large model, wherein the original large model is obtained by pre-training on an industrial dataset;

[0006] performing lightweight processing on the original large model to obtain an intermediate large model;

[0007] performing knowledge distillation training on the intermediate large model by taking the original large model as a teacher model to obtain a target large model, wherein the knowledge distillation training comprises migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model;

[0008] determining a deployment strategy of the target large model according to the hardware type of a target edge device, wherein the deployment strategy comprises at least one of an operator fusion strategy, a memory access optimization strategy and a customized intellectual property core;

[0009] deploying the target large model to the target edge device according to the deployment strategy, wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis and parameter optimization.

[0010] In one embodiment, the lightweight processing of the original large model to obtain an intermediate large model comprises:

[0011] performing a sensitivity analysis on the original large model to determine an influence degree of model parameters and structural components of different layers of the original large model on model prediction performance, and determining a non-sensitive part of the original large model according to the influence degree;

[0012] performing low-precision quantization on a layer corresponding to the non-sensitive part to obtain an intermediate sub-model; wherein the low-precision quantization comprises converting model parameters of the original large model from high-bit-width floating-point number representation to low-bit-width integer or low-bit-width floating-point number representation;

[0013] performing structural pruning on the intermediate sub-model to obtain the intermediate large model; wherein the structural pruning is used to remove structural components identified as redundant.

[0014] In one of the embodiments, the determining of the deployment strategy of the target large model according to the hardware type of the target edge device comprises:

[0015] in the case where the target edge device is a graphics processing unit, merging a plurality of adjacent neural network layers into one composite kernel by using an inference optimization tool;

[0016] in the case where the target edge device is a field programmable gate array, mapping a plurality of operators in a computation graph to the same customized intellectual property core.

[0017] In one of the embodiments, the performing of the knowledge distillation training on the intermediate large model to obtain the target large model comprises:

[0018] constructing a distillation training framework to align the output layer soft label of the teacher model with the output layer prediction of the intermediate large model;

[0019] establishing a feature mapping between the intermediate layers of the teacher model and the intermediate large model, determining a target difference by using a preset feature algorithm, and migrating intermediate representation knowledge by using the target difference; wherein the feature algorithm comprises at least one of the following: minimum feature map, attention matrix;

[0020] using a weighted loss function to jointly optimize a first loss generated by the soft label alignment and a second loss generated by the intermediate representation migration to obtain the target large model.

[0021] In one of the embodiments, the method further comprises:

[0022] obtaining initial industrial data;

[0023] simulating industrial noise to perform noise processing on the initial industrial data to obtain target industrial data;

[0024] Fine-tune the target large model using target industrial data and adjust the weight of the loss function.

[0025] In one of the embodiments, the deploying the target large model to the target edge device comprises:

[0026] Encrypting and integrity signing the target large model with a customized inference engine to obtain a deployment package;

[0027] Distributing the deployment package to the target edge device through a secure communication link and performing security verification and unpacking;

[0028] Loading and starting the inference engine on the target edge device; wherein the inference engine is used to monitor computing resources and adjust model calculation frequency according to load conditions.

[0029] In a second aspect, the present application also provides a large model lightening and edge inference acceleration device for industrial scenarios. The device comprises:

[0030] A model acquisition module for acquiring an original large model, wherein the original large model is obtained by pre-training with an industrial data set;

[0031] A lightening module for lightening the original large model to obtain an intermediate large model;

[0032] A knowledge distillation module for distilling knowledge from the original large model as a teacher model to the intermediate large model to obtain a target large model; wherein the knowledge distillation training comprises migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model;

[0033] A strategy determination module for determining a deployment strategy of the target large model according to the hardware type of the target edge device; wherein the deployment strategy comprises at least one of an operator fusion strategy, a memory access optimization strategy and a customized intellectual property core;

[0034] The strategy determination module is also used to deploy the target large model to the target edge device according to the deployment strategy; wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis and parameter optimization.

[0035] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the large model lightening and edge inference acceleration method for industrial scenarios as any one of the embodiments of the present application when executing the computer program.

[0036] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium has stored thereon a computer program, and the computer program, when executed by a processor, implements the method for large model light-weighting and edge inference acceleration for industrial scenarios according to any one of the embodiments of the present disclosure.

[0037] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and the computer program, when executed by a processor, implements the method for large model light-weighting and edge inference acceleration for industrial scenarios according to any one of the embodiments of the present disclosure.

[0038] The method, device, computer device, storage medium and computer program product for large model light-weighting and edge inference acceleration for industrial scenarios described above can effectively reduce the complexity of the model while retaining its core capabilities by obtaining an original large model and performing light-weighting processing on it. During the knowledge distillation training process, the migration of the soft label distribution of the teacher model and the attention feature data not only improves the performance of the intermediate large model but also ensures its efficiency in specific tasks. The hardware type of the target edge device determines the specific implementation of the deployment strategy, for example, the operator fusion strategy can reduce computational redundancy, the memory access optimization strategy can improve running efficiency, and the application of customized intellectual property cores further enhances the adaptability of the model. Finally, after deploying the target large model to the edge device, its actual application capability in the industrial scenario is fully utilized, providing reliable support for quality detection, fault diagnosis and parameter optimization tasks. The advantage of this method is that it can balance model performance and hardware limitations to achieve a more efficient edge computing solution. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 An application environment diagram for the method for large model light-weighting and edge inference acceleration for industrial scenarios in an embodiment;

[0040] Figure 2 A flowchart for the method for large model light-weighting and edge inference acceleration for industrial scenarios in an embodiment;

[0041] Figure 3 A structural block diagram of the device for large model light-weighting and edge inference acceleration for industrial scenarios in an embodiment;

[0042] Figure 4 An internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0043] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0044] The large model lightweight and edge inference acceleration method for industrial scenarios provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The server 104 is responsible for obtaining a pre-trained original large model, performing lightweight processing to obtain an intermediate large model, and then migrating the output layer soft label and intermediate layer attention feature of the teacher model (original large model) to the student model (intermediate large model) through knowledge distillation to obtain an optimized target large model. The server 104 formulates a deployment strategy including operator fusion, memory optimization, etc. according to the hardware characteristics of the terminal edge device, and finally deploys the target large model to the terminal 102 device for intelligent tasks such as industrial quality detection and fault diagnosis. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0045] In one embodiment, as shown in Figure 2 A large model lightweight and edge inference acceleration method for industrial scenarios is provided, including the following steps:

[0046] Step S200, obtaining an original large model, wherein the original large model is obtained by pre-training on an industrial dataset.

[0047] The obtaining of the original large model can include extracting sample data related to the industrial field from a public dataset and pre-training through a deep learning framework. These datasets can include device operation logs, sensor collected data, and historical maintenance records, etc. to ensure that the model has a broad knowledge base. In addition, distributed computing resources can be used in the pre-training stage of the original large model to accelerate the training process and improve the model performance.

[0048] In an exemplary embodiment, the pre-training process of the original large model can also incorporate transfer learning techniques, using models trained in other related fields as a starting point, and further fine-tuning to adapt to the needs of specific industrial scenarios. This method can effectively reduce the dependence on large-scale labeled data, while improving the generalization ability of the model. In addition, an adversarial sample generation mechanism can be introduced during training to enhance the model's resistance to noise and abnormal data.

[0049] Step S202, the original large model is subjected to lightweight processing to obtain an intermediate large model.

[0050] Among them, lightweight processing can be used to reduce the computational complexity and storage requirements of the model, while trying to maintain its prediction performance. Specifically, sensitivity analysis can be used to identify parts of the original large model that have less impact on prediction performance, and optimize them. For example, low-precision quantization can be implemented for non-sensitive layers first, converting high-precision floating-point representations to low-precision integer or floating-point representations, thereby reducing memory usage and computational overhead. In addition, structured pruning techniques can further remove redundant neural network components, such as unnecessary neurons or channels, to simplify the model structure. These operations not only improve the running efficiency of the model in resource-constrained environments, but also lay the foundation for its subsequent deployment to edge devices.

[0051] In an exemplary embodiment, the specific implementation of lightweight processing can be customized and adjusted in combination with hardware characteristics. For example, in optimization for specific edge devices, the quantization bit width and pruning ratio can be dynamically adjusted according to the computing power and storage limitations of the target hardware. In addition, a verification mechanism can be introduced after each operation to quickly evaluate the intermediate large model using a small-scale data set, and adjust the optimization strategy based on the evaluation results. This method ensures that key features and representation capabilities are preserved during model compression, providing a reliable input basis for subsequent knowledge distillation training. At the same time, lightweight processing can also incorporate automatic machine learning techniques, using search algorithms to automatically explore the best compression parameter combination, further improving optimization efficiency.

[0052] Step S204, using the original large model as a teacher model, the intermediate large model is subjected to knowledge distillation training to obtain a target large model; wherein the knowledge distillation training includes migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model.

[0053] Among them, knowledge distillation training can be used to effectively transfer the rich knowledge contained in the teacher model to the intermediate large model, so as to reduce the complexity of the model while retaining its performance as much as possible. Specifically, the migration of soft label distribution can help the intermediate large model learn the fine-grained probability distribution of the teacher model for sample classification, rather than just the final hard label result. This training method can significantly improve the generalization ability of the intermediate large model when facing ambiguous or uncertain data. At the same time, the migration of intermediate layer attention feature data further enhances the ability of the intermediate large model to capture key features, making it more robust when dealing with complex industrial tasks.

[0054] In an exemplary embodiment, knowledge distillation training can be implemented by constructing a multi-stage optimization framework. An alignment mechanism for the output layer can be established between the teacher model and the intermediate large model, and the smoothness of the soft label can be adjusted by setting appropriate temperature parameters, so as to control the strength of knowledge transfer. Feature mapping algorithms, such as methods based on minimum feature maps or attention matrices, are introduced between the intermediate layers to measure the representation differences between them, and these differences are gradually reduced through backpropagation. In addition, the design of the weighted loss function can balance the relationship between soft label alignment and intermediate representation migration, ensuring that both are optimized in the training process. The effectiveness of knowledge distillation training is highly dependent on the quality of the teacher model and the initial state of the intermediate large model. Therefore, in practical applications, the performance of the target large model can be gradually improved through multiple iterations of distillation. For example, after the first round of distillation is completed, the generated target large model can be used as a new teacher model to perform secondary distillation on another lighter intermediate model, thereby further compressing the model size and optimizing its efficiency. This method is particularly suitable for resource-constrained edge device scenarios, and can maximize the reduction of computational and storage overheads while ensuring performance.

[0055] Step S206, according to the hardware type of the target edge device, determine the deployment strategy of the target large model; wherein the deployment strategy includes at least one of operator fusion strategy, memory access optimization strategy and customized intellectual property core.

[0056] Among them, corresponding optimization schemes can be formulated for different hardware characteristics. For example, in the environment of a graphics processor, a plurality of adjacent neural network layers are merged into a composite kernel through an operator fusion strategy, which can significantly reduce redundant operations in the calculation process, thereby improving the execution efficiency. In the field programmable gate array scenario, a plurality of operators in the calculation graph can be mapped to the same hardware module through the design of a customized intellectual property core, so as to realize higher calculation density and lower power consumption. In addition, the memory access optimization strategy can effectively alleviate the data transmission bottleneck problem. Especially when processing large-scale industrial data, by adjusting the memory access mode, the cache miss rate can be reduced and the data throughput can be improved. The comprehensive application of these strategies not only meets the hardware limitations of the target edge device, but also maximizes the running efficiency on the premise of ensuring the model accuracy, providing strong support for real-time inference in actual industrial scenarios.

[0057] In an exemplary embodiment, the target large model can be packaged and optimized in combination with an automated tool chain. For example, in the process of generating a customized intellectual property core, a high-level synthesis tool is used to automatically convert an algorithm described in a high-level language into a hardware description language, and through iterative optimization, it is ensured that it meets the performance and power consumption constraints. At the same time, according to the characteristics of different hardware platforms, a dynamic scheduling mechanism can be introduced to adjust the operator execution order and resource allocation strategy according to the device load at runtime, thereby realizing more efficient utilization of computing resources. In addition, the deployment strategy can also be pre-verified through a simulation environment to ensure its stability and reliability in actual application. This method not only shortens the period from model development to hardware deployment, but also significantly reduces the cost of manual intervention.

[0058] Step S208, according to the deployment strategy, deploying the target large model to the target edge device; wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis and parameter optimization.

[0059] Among them, in the process of deploying the target large model to the target edge device, the model can be subjected to necessary security processing. Through encryption algorithm, the weight and structure information of the model are protected, to prevent being maliciously stolen or tampered with during transmission. At the same time, a unique digital fingerprint is generated by using an integrity signature technology, to ensure that the model can be verified for its integrity and legal source through a verification mechanism after being issued to the edge device.

[0060] In an exemplary embodiment, in terms of the establishment of a secure communication link, a public key infrastructure-based protocol can be utilized to ensure the security of data transmission. By assigning each edge device a unique identity and combining a dynamic key agreement mechanism, end-to-end encrypted communication is achieved. This mechanism not only resists man-in-the-middle attacks, but also provides efficient key management capabilities when multiple devices are deployed concurrently. Upon receiving a deployment package, an edge device can automatically trigger a security verification process. This process can include digital signature verification of the deployment package, hash verification of the model file, and compatibility check of the running environment. Only when all verification steps pass, the device will perform unpacking operation and load the model into memory. In addition, a real-time monitoring mechanism can be designed to detect abnormal behavior during model running and respond in a timely manner. The inference engine, as the core running environment of the target large model, has intelligent resource scheduling capabilities. It can monitor key indicators such as CPU occupancy, memory usage, and temperature in real time, and dynamically adjust the calculation frequency of the model according to the current load. For example, when device resources are tight, the inference engine can reduce the inference accuracy of the model or reduce unnecessary calculation operations to ensure the stable operation of the system. At the same time, the inference engine also supports multi-task concurrent processing, and can flexibly allocate computing resources according to the needs of different application scenarios.

[0061] In the above method of lightweight large model for industrial scenarios and edge inference acceleration, by obtaining the original large model and performing lightweight processing, the complexity of the model can be effectively reduced while retaining its core capabilities. During the knowledge distillation training process, the transfer of soft label distribution of the teacher model and attention feature data not only improves the performance of the intermediate large model, but also ensures its efficiency in specific tasks. The hardware type of the target edge device determines the specific implementation of the deployment strategy, for example, the operator fusion strategy can reduce computational redundancy, the memory access optimization strategy can improve running efficiency, and the application of customized intellectual property cores further enhances the adaptability of the model. Finally, after deploying the target large model to the edge device, its actual application capability in the industrial scenario is fully utilized, providing reliable support for quality detection, fault diagnosis, and parameter optimization tasks. The advantage of this method is that it can balance model performance and hardware limitations to achieve a more efficient edge computing solution.

[0062] In one embodiment, the lightweight processing of the original large model to obtain an intermediate large model comprises:

[0063] Performing sensitivity analysis on the original large model to determine the influence of model parameters and structural components of different layers of the original large model on model prediction performance, and determining the non-sensitive part of the original large model according to the influence.

[0064] The non-sensitive parts are quantized at low precision to obtain an intermediate sub-model; wherein the low-precision quantization includes converting the model parameters of the original large model from high-bit-width floating-point representation to low-bit-width integer or low-bit-width floating-point representation.

[0065] The intermediate sub-model is structurally pruned to obtain the intermediate large model; wherein the structural pruning is used to remove structural components identified as redundant.

[0066] In the sensitivity analysis, various methods can be used to evaluate the importance of each part of the model. For example, by calculating the contribution of each neural network layer to the final prediction result, those layers or parameters with less impact on performance are identified. These non-sensitive parts usually include redundant neurons, channels or other structural components, which have relatively limited role in the model. Based on the analysis results, these non-sensitive parts can be preferentially selected for optimization processing, so as to reduce the impact on the prediction performance while reducing the complexity of the model. Low-precision quantization is one of the key steps in lightweight processing, which is to convert high-bit-width floating-point representation to low-bit-width integer or floating-point representation. This conversion not only can significantly reduce the storage requirements of the model, but also can reduce the resource consumption in the calculation process. For example, using 8-bit integers instead of 32-bit floating-point numbers can greatly improve the calculation efficiency on some hardware platforms, while maintaining high prediction accuracy. In order to ensure that the quantized model still has good performance, a calibration mechanism can be introduced in the quantization process, which uses a small-scale validation dataset to quickly test the quantized model, and adjusts the quantization strategy according to the test results.

[0067] In an exemplary embodiment, structural pruning can further simplify the structure of the model, reduce the number of parameters and computational overhead of the model by removing neural network components identified as redundant. Unlike unstructured pruning, structural pruning removes entire neurons or channels in a more regular manner, making the optimized model easier to adapt to different hardware platforms. In actual operation, the pruning intensity can be controlled by setting the pruning ratio or threshold, and combined with the validation mechanism to ensure that the performance of the pruned model does not decrease significantly. In addition, the pruning process can be combined with the quantization operation to form a multi-stage optimization process, thereby achieving more efficient model compression effect.

[0068] In this embodiment, by performing low-precision quantization and structured pruning on the non-sensitive parts of the original large model, the computational complexity and storage requirements of the model are effectively reduced while preserving the key prediction performance. This method not only improves the applicability of the model in resource-constrained environments, but also lays the foundation for subsequent knowledge distillation training. Through sensitivity analysis, the parts of the model that have less impact on performance can be accurately located, thereby achieving lightweight processing while ensuring model functionality. In addition, the combination of low-precision quantization and structured pruning further optimizes the running efficiency of the model, making it more suitable for deployment on edge devices. This processing method can significantly reduce the consumption of computing resources without significantly reducing the accuracy of the model, providing possibilities for real-time applications in industrial scenarios.

[0069] In one embodiment, the determination of the deployment strategy of the target large model according to the hardware type of the target edge device includes:

[0070] In the case where the target edge device is a graphics processing unit, multiple adjacent neural network layers are merged into a composite kernel using an inference optimization tool.

[0071] In the case where the target edge device is a field programmable gate array, multiple operators in the computation graph are mapped to the same customized intellectual property core.

[0072] In the context of a graphics processing unit, the operation of merging adjacent neural network layers can significantly improve computational efficiency by reducing kernel launch frequency and memory access frequency. This operator fusion strategy not only reduces redundant overhead in the computation process, but also fully utilizes the parallel computing capabilities of the graphics processing unit, thereby achieving higher throughput. In the context of a field programmable gate array, the design of a customized intellectual property core places more emphasis on efficient use of hardware resources. By mapping multiple operators to the same hardware module, data transmission delay between different modules can be reduced, and circuit layout can be optimized to increase computational density. In addition, the functional configuration of the intellectual property core can be flexibly adjusted according to the requirements of specific tasks, making it better adapt to the target application scenario. The combination of these two strategies can effectively address the limitations of different hardware platforms while ensuring efficient operation of the target large model on edge devices.

[0073] In an exemplary embodiment, the details of the deployment strategy can be further optimized in combination with hardware characteristics. For example, in a graphics processor environment, in addition to operator fusion, optimization of memory access patterns can also be introduced to reduce cache miss rates by reorganizing data storage structures. This optimization approach can significantly improve data throughput, especially when processing large-scale industrial data. In the field programmable gate array scenario, the design of customized intellectual property cores can be combined with dynamic reconfiguration technology to adjust the functions of hardware modules in real time according to task requirements. This approach not only improves resource utilization, but also enables more flexible scheduling when multiple tasks are executed concurrently. In addition, to address the power consumption limitations of edge devices, energy consumption modeling tools can be introduced during the design phase to evaluate the impact of different deployment strategies on power consumption and select the optimal solution to meet the energy efficiency requirements of the device. The comprehensive application of these methods can further improve the adaptability and efficiency of the target large model in actual industrial scenarios, while ensuring that its stability and reliability meet the expected standards.

[0074] In this embodiment, by taking appropriate optimization measures for different hardware types, the running efficiency of the target large model on the target edge device can be significantly improved. For example, in a graphics processor environment, by merging adjacent neural network layers, the overhead of data transmission is reduced, thereby improving the utilization of computing resources; in the field programmable gate array scenario, the application of customized intellectual property cores enables multiple operators in the computation graph to efficiently share hardware resources, further reducing latency and improving throughput. These strategies not only adapt to the characteristics of different hardware architectures, but also provide flexible and efficient solutions for subsequent actual deployment. In addition, through the fine design of the deployment strategy, the energy consumption and hardware cost can be minimized while ensuring model performance, providing strong support for real-time and reliability requirements in industrial application scenarios.

[0075] In one embodiment, the knowledge distillation training of the intermediate large model to obtain a target large model comprises:

[0076] A distillation training framework is constructed to align the output layer soft labels of the teacher model with the output layer predictions of the intermediate large model.

[0077] A feature mapping is established between the intermediate layers of the teacher model and the intermediate large model, and a target difference is determined by a pre-set feature algorithm; and intermediate representation knowledge is transferred using the target difference; wherein the feature algorithm includes at least one of the following: minimum feature map, attention matrix.

[0078] A weighted loss function is used to jointly optimize a first loss generated by soft label alignment and a second loss generated by intermediate representation migration to obtain a target large model.

[0079] In constructing the distillation training framework, the use of soft labels can help the intermediate large model better learn the prediction distribution of the teacher model. The probability distribution output by the teacher model can be used as a supervision signal, and the intermediate large model can not only mimic its classification results, but also capture more rich semantic information. This way of knowledge transfer is especially suitable for complex task scenarios, such as industrial quality detection or equipment fault diagnosis, where the teacher model often has stronger generalization ability and higher precision. In addition, the introduction of feature mapping can further enhance the transfer effect of intermediate representation knowledge. Through feature algorithms such as minimum feature map or attention matrix, the difference between the teacher model and the intermediate large model at the intermediate layer can be accurately measured, and the training process can be adjusted based on this to make the intermediate large model gradually approach the performance of the teacher model. In actual operation, one of the key steps of knowledge distillation training can include the design of a weighted loss function. By jointly optimizing the first loss generated by the alignment of soft labels and the second loss generated by the transfer of intermediate representations, effective knowledge transfer can be achieved at different levels. Specifically, the first loss can ensure the consistency of the output distribution of the intermediate large model with the teacher model, while the second loss can guide the intermediate large model to learn more refined pattern expressions in the feature space. The weight coefficient can be dynamically adjusted according to task requirements. For example, in the early stage, a higher weight can be given to the first loss to quickly align the output distribution, and in the later stage, the proportion of the second loss can be gradually increased to strengthen the learning effect of the intermediate representation.

[0080] In an exemplary embodiment, the process of knowledge distillation can also be further optimized in combination with an adaptive mechanism. For example, by monitoring the trend of loss changes during training, the weight parameters in the weighted loss function can be dynamically adjusted to improve the stability and convergence speed of training. In addition, a multi-scale feature mapping strategy can be designed to enable the intermediate large model to learn the knowledge of the teacher model at multiple abstraction levels. This method not only improves the performance of the target large model, but also enhances its applicability in diverse industrial scenarios, laying a solid foundation for subsequent deployment.

[0081] In this embodiment, through the fine design of the alignment and transfer process, the target large model is ensured to approach the performance of the original large model while significantly reducing the computational complexity and storage requirements. In addition, by dynamically adjusting the weight coefficients in the weighted loss function, the optimization effect between soft label alignment and intermediate representation transfer can be further balanced, thereby improving the generalization ability of the target large model. This method also introduces an adaptive learning rate strategy, which automatically adjusts the learning rate during training based on the model convergence situation to speed up the training process and avoid falling into a local optimal solution.

[0082] In one embodiment, the method further comprises:

[0083] Obtaining initial industrial data.

[0084] Simulate industrial noise to process the initial industrial data to obtain target industrial data.

[0085] Fine-tune the target large model using the target industrial data and adjust the weight of the loss function.

[0086] After obtaining the initial industrial data, the step of simulating industrial noise can be implemented in various ways. For example, based on common interference sources in the actual industrial environment, such as electromagnetic interference, sensor drift or signal transmission loss, corresponding noise models can be generated. These noise models can be superimposed on the initial industrial data in the form of additive noise, multiplicative noise or mixed form, thereby generating target industrial data that is closer to the real scene. Through the use of target industrial data, the target large model can be fine-tuned under more challenging conditions, further improving its robustness and adaptability. In addition, during the fine-tuning process, the adjustment strategy of the loss function weight can be customized according to specific task requirements. For example, in the quality detection task, higher weights can be given to abnormal samples to enhance the model's ability to identify key issues; while in the fault diagnosis scenario, the weight distribution between different categories can be dynamically balanced to ensure that the model can maintain high accuracy when dealing with unbalanced data. This fine-tuning method not only enhances the practical application ability of the target large model, but also provides strong protection for its deployment in complex industrial scenarios.

[0087] In an exemplary embodiment, the performance of the target large model in specific industrial tasks can be further optimized by combining transfer learning techniques. For example, during the fine-tuning process, the knowledge of a pre-trained model can be introduced by sharing part of the network structure or initializing parameters, thereby accelerating the convergence speed of the target large model. This method is particularly suitable for scenarios with limited data, which can reduce training time while improving model performance. In addition, to address the problem of uneven data distribution in industrial scenarios, domain adaptation methods can be used to align the feature distributions of the source and target domains, reducing the risk of performance degradation due to data differences. This method not only enhances the generalization ability of the target large model, but also makes it better adapt to different types of industrial application requirements. In actual operation, a multi-stage fine-tuning strategy can also be designed to gradually improve the performance of the target large model. For example, first use large-scale general-purpose data for preliminary fine-tuning to improve the basic ability of the model; then use noise-processed target industrial data for fine-tuning to further strengthen its performance in specific tasks. This phased training method can effectively balance the learning efficiency and accuracy of the model, while avoiding the instability problem caused by directly using complex data. In addition, combined with online learning mechanisms, the model's parameters can be continuously updated after deployment, allowing it to dynamically adapt to changes in the industrial environment, thereby maintaining long-term high performance levels.

[0088] In this embodiment, by simulating industrial noise and customizing fine-tuning strategies, the robustness and accuracy of the target large model in complex industrial scenarios have been significantly improved. At the same time, combined with transfer learning and domain adaptation technology, the model's adaptability to diversified tasks is further enhanced. The comprehensive application of these methods provides more comprehensive support for the actual deployment of the target large model, ensuring its stable operation in real environments and meeting various industrial needs.

[0089] In one embodiment, the deployment of the target large model to the target edge device includes:

[0090] Encrypting and integrity signing the target large model and the customized inference engine to obtain a deployment package.

[0091] Distribute the deployment package to the target edge device through a secure communication link and perform security verification and unpacking.

[0092] Load and start the inference engine on the target edge device; wherein the inference engine is used to monitor computing resources and adjust model computing frequency according to load conditions.

[0093] In the process of encrypting and integrity signing the target large model and the customized inference engine, encryption algorithms can be used to ensure the security of the deployment package. This encryption method not only prevents the model from being maliciously tampered with during transmission, but also effectively protects intellectual property rights and prevents unauthorized use. At the same time, the integrity signature mechanism can generate a unique digital fingerprint of the contents of the deployment package, further ensuring the integrity and consistency of the data. Once the deployment package is distributed to the target edge device through a secure communication link, the system will automatically perform a strict security verification process. This includes verifying the validity of the signature and checking whether the file has been damaged or tampered with during transmission. Only when all verification steps are successfully completed will the target edge device unpack the deployment package, ensuring that the loaded model and inference engine are trustworthy and complete.

[0094] In an exemplary embodiment, after the inference engine is loaded and started on the target edge device, the inference engine can monitor the device's computing resource usage in real time. For example, it can dynamically adjust the model's computation frequency according to the current CPU or GPU load, memory occupancy, and energy consumption level. This adaptive mechanism enables the target large model to maintain efficient operation in resource-constrained environments. In addition, the inference engine also supports multi-task concurrent processing, allocating computing resources through intelligent scheduling algorithms to maximize the overall performance of the device. The inference engine can also implement specific acceleration strategies in combination with hardware characteristics. For example, in a graphics processor environment, the inference engine can fully utilize its parallel computing capabilities through batch processing techniques; while in a field programmable gate array scenario, it can reduce latency and increase throughput through hardware-level optimization. The introduction of these functions provides strong technical support for the stable operation of the target large model in actual industrial scenarios.

[0095] In this embodiment, by encrypting and integrity signing the target large model, tampering or leakage of the model during transmission can be effectively prevented, thereby improving the security of deployment. At the same time, the use of secure communication links further ensures the confidentiality and integrity of the deployment package during transmission. After unpacking on the target edge device, the start of the inference engine enables comprehensive monitoring of the model's running environment, and the computation frequency can be dynamically adjusted according to the actual load to balance performance and energy consumption. In addition, with the support of customized inference engines, the target large model can exhibit better adaptation capability in different hardware environments, meeting the diverse application requirements in industrial scenarios. This deployment method not only improves the running efficiency of the model, but also significantly enhances the reliability and security of the system.

[0096] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0097] Based on the same inventive concept, the embodiments of the present application also provide an industrial-scene-oriented large model lightweight and edge inference acceleration device for implementing the industrial-scene-oriented large model lightweight and edge inference acceleration method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more industrial-scene-oriented large model lightweight and edge inference acceleration device embodiments provided below can be referred to the limitations of the industrial-scene-oriented large model lightweight and edge inference acceleration method described above, which will not be repeated here.

[0098] In one embodiment, as shown in Figure 3 An industrial-scene-oriented large model lightweight and edge inference acceleration device is provided, comprising: a model acquisition module 301, a lightweight module 303, a knowledge distillation module 305, and a strategy determination module 307, wherein:

[0099] The model acquisition module is configured to acquire an original large model, wherein the original large model is obtained by pre-training on an industrial dataset;

[0100] The lightweight module is configured to perform lightweight processing on the original large model to obtain an intermediate large model;

[0101] The knowledge distillation module is configured to perform knowledge distillation training on the intermediate large model by taking the original large model as a teacher model to obtain a target large model, wherein the knowledge distillation training includes migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model;

[0102] The strategy determination module is configured to determine a deployment strategy of the target large model according to the hardware type of a target edge device, wherein the deployment strategy includes at least one of an operator fusion strategy, a memory access optimization strategy, and a customized intellectual property core;

[0103] The strategy determination module is further configured to deploy the target large model to the target edge device according to the deployment strategy, wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis, and parameter optimization.

[0104] Each module in the above industrial-scene-oriented large model lightweight and edge inference acceleration device can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0105] In one embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown inFigure 4 The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals, and the wireless mode can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize an industrial scene-oriented large model lightweight and edge inference acceleration method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0106] Those skilled in the art can understand that, Figure 4 The skilled in the art can understand that,

[0107] In one embodiment, an industrial scene-oriented large model lightweight and edge inference acceleration system is provided, which specifically includes five core modules of model lightweight processing, knowledge distillation optimization, hardware acceleration adaptation, industrial scene fine-tuning and edge deployment running. The system can adopt a "cloud-edge" collaborative architecture, in which model compression, distillation and preliminary optimization are completed in the cloud, and the lightweight model is deployed to the industrial edge device through a secure channel. The edge end integrates a customized inference engine to support efficient execution of inference tasks in a resource-constrained environment. The overall flow of the system is driven by a collaborative optimization controller to dynamically match the model compression strategy, distillation target and hardware characteristics, ensuring that the end-to-end lightweight deployment of large models is completed under the premise of meeting the industrial real-time, precision and power consumption constraints.

[0108] Model lightweight processing mechanism: The system adopts a lightweight method combining mixed precision quantization and structured pruning. First, sensitivity analysis is performed on each layer of the large model to identify weights and channels that have less impact on accuracy. Based on the analysis results, INT8 or FP16 low-precision representation is used for non-sensitive layers, and FP32 precision is retained for key layers to achieve mixed precision quantization. At the same time, structured pruning is performed to remove redundant convolution channels or attention heads, significantly reducing model parameter quantity and computational load. This process is completed in the cloud, and a preliminary lightweight model with reduced parameters and improved computational efficiency is output.

[0109] Knowledge distillation optimization mechanism: The system constructs a teacher-student collaborative training framework, with the original large model as the teacher model and the lightweight model as the student model. A multi-level knowledge transfer strategy is designed: transferring soft label distribution (logits distillation) at the output layer, transferring feature maps or attention matrices (feature distillation) at the intermediate layer, and balancing the distillation effects of different tasks (such as classification and detection) through a weighted loss function. The distillation process incorporates industrial data priors to enhance the robustness of the student model to noise, small samples, and class imbalance, ensuring that the lightweight model maintains low computational overhead while inheriting the inference accuracy of the large model as much as possible.

[0110] Hardware acceleration adaptation mechanism: The system customizes inference acceleration schemes based on the hardware architecture (such as GPU, FPGA, NPU) of the target edge device. For GPU devices, CUDA kernel scheduling and memory access patterns are optimized, and operator fusion and inter-layer optimization are implemented using TensorRT or Torch-TensorRT. For FPGA devices, custom IP cores are generated to support sparse computation and low-bit operation, fully leveraging their parallel processing advantages. The system has a built-in hardware characteristics database that automatically matches the optimal quantization format (such as INT4, Binary) and computation graph optimization strategy, enabling deep collaboration between algorithms and hardware to maximize inference throughput and reduce latency.

[0111] Industrial scene fine-tuning and optimization mechanism: After lightweight and distillation, the system fine-tunes the model using real industrial scene data. A small sample adaptive training strategy is used in combination with data augmentation (such as simulated light changes and sensor noise) to improve model generalization. For specific industrial tasks (such as surface defect detection and equipment vibration analysis), the loss function and evaluation metrics are adjusted to ensure high accuracy and low false positive rate in actual production line environments. The fine-tuning process supports incremental learning, allowing dynamic model updates as production lines change.

[0112] Edge deployment and running mechanism: After the lightweight model is encrypted and packaged, it is deployed to the industrial edge device through a secure communication protocol (such as MQTT over TLS). The edge end runs a customized inference engine, supporting the whole process of model loading, input preprocessing, efficient inference, and result output. The system monitors device resources (CPU, memory, power consumption) and inference performance (delay, throughput), supports dynamic frequency reduction or model switching strategies to cope with sudden loads. Inference results can be uploaded to the industrial cloud platform in real time for data analysis, visualization, and closed-loop control, forming an intelligent closed loop of "perception-decision-execution".

[0113] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards.

[0114] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0115] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0116] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for lightweighting and edge inference acceleration of large models for industrial scenarios, characterized in that, The method comprises the following steps: obtaining an original large model, wherein the original large model is obtained through pre-training on an industrial dataset; performing sensitivity analysis on the original large model to determine the influence of model parameters and structural components of different layers of the original large model on model prediction performance, and determining a non-sensitive part of the original large model according to the influence; performing low-precision quantization on the layer corresponding to the non-sensitive part to obtain an intermediate sub-model; wherein the low-precision quantization comprises converting the model parameters of the original large model from high-bit-width floating-point number representation to low-bit-width integer or low-bit-width floating-point number representation; performing structured pruning on the intermediate sub-model to obtain an intermediate large model; wherein the structured pruning is used to remove structural components identified as redundant; performing knowledge distillation training on the intermediate large model using the original large model as a teacher model to obtain a target large model; wherein the knowledge distillation training comprises migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model; determining a deployment strategy of the target large model according to the hardware type of a target edge device; wherein the deployment strategy comprises at least one of an operator fusion strategy, a memory access optimization strategy, and a customized intellectual property core; deploying the target large model to the target edge device according to the deployment strategy; wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis, and parameter optimization.

2. The method of claim 1, wherein, The method further comprises: obtaining an initial industrial dataset; simulating industrial noise to perform noise processing on the initial industrial dataset to obtain a target industrial dataset; 3. The method of claim 1, wherein, fine-tuning the target large model using the target industrial dataset and adjusting the weight of the loss function. The method further comprises: encrypting and integrity signing the target large model and a customized inference engine to obtain a deployment package; downloading the deployment package to the target edge device through a secure communication link and performing security verification and unpacking.

4. The method of claim 1, wherein, ​ ​ ​ ​ 5. The method of claim 1, wherein, ​ ​ ​ Load and start the inference engine on the target edge device; wherein the inference engine is used to monitor computing resources and adjust model calculation frequency according to load conditions.

6. An apparatus for large model lightweight and edge inference acceleration for industrial scenarios, characterized in that, The device comprises: A model acquisition module is configured to acquire an original large model, wherein the original large model is obtained through pre-training on an industrial dataset. A light-weight module is configured to perform sensitivity analysis on the original large model to determine the influence of model parameters and structural components of different layers of the original large model on model prediction performance, and determine a non-sensitive part of the original large model according to the influence; perform low-precision quantization on the layers corresponding to the non-sensitive part to obtain an intermediate sub-model; wherein the low-precision quantization includes converting the model parameters of the original large model from high-bit-width floating-point number representation to low-bit-width integer or low-bit-width floating-point number representation; perform structured pruning on the intermediate sub-model to obtain an intermediate large model; wherein the structured pruning is used to remove structural components identified as redundant. A knowledge distillation module is configured to perform knowledge distillation training on the intermediate large model by taking the original large model as a teacher model to obtain a target large model; wherein the knowledge distillation training includes migrating the soft label distribution of the output layer and the attention feature data of the intermediate layer of the teacher model to the intermediate large model. A strategy determination module is configured to determine a deployment strategy of the target large model according to the hardware type of a target edge device; wherein the deployment strategy includes at least one of an operator fusion strategy, a memory access optimization strategy, and a customized intellectual property core. The strategy determination module is further configured to deploy the target large model to the target edge device according to the deployment strategy; wherein the target large model is used for at least one of industrial quality detection, equipment fault diagnosis, and parameter optimization. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

Images