Self-training large model neural network inference calculation method and system based on NPU acceleration

Abstract

The application discloses a self-training large model neural network inference calculation method and system based on NPU acceleration, relates to the technical field of artificial intelligence inference calculation, and specifically comprises the following steps: feature extraction and label generation, NPU preprocessing data is adapted, features are extracted, and pseudo labels are generated; confidence checking, double-threshold comparison rules are used for checking; precision switching and collaborative inference, precision is switched according to confidence, and heterogeneous inference is triggered when the value is low; result output and feedback packaging, output result and feedback information are packaged; model optimization, model parameters are adaptively optimized based on feedback; through combination of dynamic precision adjustment and heterogeneous collaborative processing technology, the NPU is used to switch inference precision by real-time checking based on double-threshold comparison, a reasoning-feedback-optimization closed loop system is constructed, and the improvement of calculation efficiency and inference accuracy and self-adaptive optimization of the model are realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence inference computing technology, specifically to a method and system for inference computing of self-trained large-scale neural networks based on NPU acceleration. Background Technology

[0002] With the rapid development of artificial intelligence technology, self-trained large models have demonstrated powerful generalization capabilities in fields such as image recognition and natural language processing. However, their inference computation process places increasingly higher demands on hardware resources. Traditional CPU or GPU-based inference architectures, limited by computational efficiency and energy efficiency, struggle to meet the real-time requirements of resource-constrained scenarios such as edge devices. The NPU, as dedicated acceleration hardware, provides hardware-level support for high-precision inference through optimized computing unit and dataflow design. However, the inference precision mode of existing NPUs is typically fixed and cannot be dynamically adjusted based on input data, leading to insufficient precision or computational redundancy in complex scenarios. Furthermore, pseudo-label generation strategies lack closed-loop optimization mechanisms, making models prone to performance degradation due to the accumulation of noisy data over long-term operation.

[0003] In traditional technologies, NPU inference systems often employ static precision configuration, requiring pre-set fixed thresholds to switch between high and low precision modes. However, a single threshold cannot adapt to dynamic changes in the confidence level of input data, leading to wasted computing resources in high-confidence scenarios or insufficient precision in low-confidence scenarios. Simultaneously, pseudo-label generation relies on initial model parameters and lacks real-time feedback optimization, making the model prone to getting trapped in local optima when data distribution shifts or noise interference occurs. Some solutions attempt to introduce CPU-NPU collaborative inference, but lack standardized processes for data verification and feature completion, resulting in inefficient scheduling of heterogeneous computing resources. This invention addresses these technical bottlenecks through dynamic precision adjustment and closed-loop optimization mechanisms. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method and system for self-training large-scale neural network inference computation based on NPU acceleration. By combining dynamic precision adjustment and heterogeneous collaborative processing technology, the NPU is used to verify in real time based on dual threshold comparison and automatically switch inference precision modes, which significantly improves the computational efficiency and accuracy of neural network inference. At the same time, the constructed inference-feedback-optimization closed-loop system effectively enhances the adaptability and stability of the model, ensures stable operation in resource-constrained environments, and realizes data-driven adaptive optimization.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a self-trained large-scale neural network inference computation method based on NPU acceleration, the specific steps of which are as follows:

[0006] S1, Feature Extraction and Label Generation: Perform preprocessing operations on the input data to adapt to NPU inference, input the preprocessed data into the self-trained large model, extract data features through the self-trained large model, generate pseudo-labels based on the extracted features, and calculate the confidence level corresponding to the pseudo-label.

[0007] S2, Confidence verification: The calculated confidence data is transmitted to the NPU. The NPU's built-in dedicated verification unit verifies the confidence in real time using a preset dual-threshold comparison rule. Based on the verification result, the corresponding precision control signal is output.

[0008] S3, Precision Switching and Collaborative Inference: The NPU receives a precision control signal and dynamically switches the inference precision mode according to the signal. When the confidence level is lower than the preset second threshold, heterogeneous collaborative inference between the CPU and the NPU is triggered.

[0009] S4, Result Output and Feedback Packaging: The NPU outputs the inference results and simultaneously collects three types of data: confidence data, precision mode identifier, and inference error. These are then packaged into feedback information according to a preset format and transmitted to the system's designated storage area.

[0010] S5, Model Optimization: Based on feedback information within the storage area, adaptive optimization logic is used to adjust the parameters of the self-trained model, optimize the pseudo-label generation strategy, and form a closed-loop mechanism of inference-feedback-optimization.

[0011] Furthermore, in step S1, the confidence level is calculated using Softmax numerical stabilization logic. The specific process is as follows: obtain the feature vector output from the last layer of the self-trained large model, calculate the maximum value of the feature vector, subtract the maximum value from each element in the feature vector, perform Softmax normalization on the processed feature vector to obtain a probability vector, and take the maximum value in the probability vector as the confidence level corresponding to the pseudo-label.

[0012] Furthermore, in step S2, the dual thresholds in the dual threshold comparison rule include a first threshold and a second threshold. The values ​​of both the first threshold and the second threshold are in the range of 0 to 1, and the first threshold is greater than the second threshold. The dual thresholds are preset and configured through a software interface. The dedicated verification unit of the NPU reads the configured dual threshold parameters through the AXI bus, and the parameters take effect immediately after being read.

[0013] Furthermore, in step S2, the confidence data is transmitted using PCIe bus high-speed transmission technology. The transmission process follows streaming transmission logic, and the integrity of the transmission is verified by adding a check code to the transmitted data. The check code is transmitted synchronously with the confidence data.

[0014] Furthermore, in step S3, the precision switching adopts a confidence-precision mapping rule, specifically: when the confidence level is higher than the first threshold, the NPU enables low-precision inference mode and performs quantization processing on the weights of the self-trained model; when the confidence level is between the first and second thresholds, the NPU enables mixed-precision inference mode and dynamically allocates the proportion of high- and low-precision computing cores according to the confidence level range; when the confidence level is lower than the second threshold, the NPU enables high-precision inference mode and triggers CPU-NPU heterogeneous collaborative inference.

[0015] Furthermore, in step S3, the specific process of heterogeneous collaborative inference is as follows: the CPU identifies and deletes invalid pseudo-labels through anomaly detection rules, supplements the missing feature dimensions corresponding to the pseudo-labels using feature completion logic, and optimizes the probability distribution of the pseudo-labels through probability adjustment rules; the CPU sends the optimized pseudo-label data back to the NPU, and the NPU performs inference calculations in full-precision mode after receiving the data.

[0016] Furthermore, in step S5, the adaptive optimization logic includes a regularization coefficient adjustment strategy, specifically: calculating the batch confidence mean, and adjusting the regularization coefficient of the pseudo-label generator in the self-trained model based on the mean. The batch confidence mean and the regularization coefficient are negatively correlated, that is, the lower the batch confidence mean, the larger the regularization coefficient.

[0017] Furthermore, in step S5, the model parameter adjustment adopts incremental optimization technology, specifically: only the network layer parameters directly related to pseudo-label generation in the self-trained model are updated, without updating the full weights of the model; the parameter update is fine-tuned through the Adam optimizer, balancing the model optimization effect and parameter update efficiency during the adjustment process.

[0018] On the other hand, a self-trained large-scale neural network inference computing system based on NPU acceleration, the system includes:

[0019] The self-training module is used to perform preprocessing operations on the input data, extract data features, and generate pseudo-labels and corresponding confidence scores.

[0020] The NPU verification unit, integrated inside the NPU, is used to receive confidence data, perform real-time verification through dual threshold comparison rules, and output precision control signals.

[0021] The NPU computing unit is used to receive precision control signals and dynamically switch inference precision modes, perform inference calculations, and trigger heterogeneous collaborative inference when the confidence level is lower than the second threshold.

[0022] The heterogeneous collaboration module is used to schedule the CPU to delete invalid pseudo-labels, supplement feature dimensions, and optimize probability distribution, and then send the optimized pseudo-label data back to the NPU computing unit.

[0023] The feedback packaging module is used to collect confidence data, accuracy mode identifiers, and inference errors, package them into feedback information according to a preset format, and transmit them to a designated storage area.

[0024] The feedback optimization module is used to read feedback information from the storage area, adjust the parameters of the self-training module using adaptive optimization logic, and optimize the pseudo-label generation strategy.

[0025] Compared with existing technologies, this NPU-accelerated self-trained large-scale neural network inference computation method and system has the following advantages:

[0026] I. This invention proposes a self-trained large-scale neural network inference computation method based on NPU acceleration, cleverly combining dynamic precision adjustment and heterogeneous collaborative processing techniques. In practical applications, the NPU verifies confidence data in real time according to a dual-threshold comparison rule and automatically switches to an appropriate inference precision mode, significantly improving computational efficiency while ensuring the accuracy of inference results. The introduction of high-speed PCIe bus transmission and streaming verification logic further ensures the integrity and timeliness of data transmission, providing strong support for the stable operation of the model in resource-constrained environments.

[0027] Second, this invention significantly enhances the adaptability and stability of self-trained large models by constructing a closed-loop optimization system of inference-feedback-optimization. During inference, the heterogeneous collaborative mechanism of CPU and NPU effectively filters out invalid pseudo-labels and optimizes data quality through feature completion and probability adjustment; the feedback optimization module intelligently adjusts the regularization coefficient based on the batch confidence mean and accurately updates model parameters by combining incremental optimization techniques, avoiding unnecessary computational overhead; this data-driven adaptive optimization strategy enables the model to continuously adapt to complex and ever-changing input environments.

[0028] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0030] Figure 1 The flowchart shows the inference computation method for a self-trained large-scale neural network based on NPU acceleration.

[0031] Figure 2 This is a framework diagram of a self-trained large-scale neural network inference computation method based on NPU acceleration;

[0032] Figure 3 This is a flowchart of a self-trained large-scale neural network inference computing system based on NPU acceleration. Detailed Implementation

[0033] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0034] Example 1:

[0035] This embodiment is applied to an intelligent security monitoring system for urban roads. It performs real-time classification and identification of three types of targets in the monitoring video stream: pedestrians, vehicles, and background. It adopts the self-trained large-model neural network inference calculation method and system based on NPU acceleration described in this invention to achieve collaborative inference with high accuracy and low latency.

[0036] like Figure 1 As shown, the first embodiment of the present invention provides a self-training large model neural network inference computation method based on NPU acceleration. The method includes steps S1 to S5, and the specific implementation process is as follows:

[0037] Step S1, Feature Extraction and Label Generation: The system receives a 1080P video stream from a high-definition camera at the intersection and extracts image data at 50ms intervals per frame as input data. Preprocessing operations adapted for NPU inference are performed on the input images. First, the image is scaled to 640×480 pixels, and RGB channel normalization is used to map pixel values ​​to the 0-1 range. Then, image enhancement algorithms are used to improve image contrast in backlit and shadow scenes. Finally, the processed image data is converted into a tensor format supported by the NPU. The preprocessed tensor data is input into a self-trained large model. This model is based on an improved ResNet50 network structure and extracts edge features, texture features, and semantic features of the image step by step through convolutional and pooling layers. Shallow networks extract basic features such as target contours, while deep networks aggregate advanced features such as target shape and motion trends. Pseudo-labels are generated based on the extracted features. The pseudo-label categories include pedestrians, vehicles, and background. Softmax numerical stability logic is used to calculate the confidence level: the 3D feature vector output from the last fully connected layer of the self-trained large model is obtained. Assuming the feature vector corresponding to a certain frame image is [3.2, 1.5, 0.8], the maximum value of this vector, 3.2, is calculated. Each element in the feature vector is subtracted from 3.2 to obtain [-0.0, -1.7, -2.4]. Softmax normalization is then performed on the processed vector to obtain the probability vector [0.89, 0.08, 0.03]. The maximum value of this probability vector, 0.89, is taken as the confidence level of the pseudo-label corresponding to the pedestrian in that frame image.

[0038] Step S2, Confidence Verification: The calculated confidence data is streamed to the NPU via the PCIe bus. During transmission, a 32-bit CRC checksum is added to the confidence data of each frame. The checksum is transmitted synchronously with the confidence data to ensure data transmission integrity. The dedicated verification unit built into the NPU reads the dual threshold parameters pre-configured via the software interface through the AXI bus. The first threshold is set to 0.85, and the second threshold is set to 0.6. The parameters take effect immediately after being read. The dedicated verification unit performs real-time verification on the received confidence data, comparing the confidence of each frame with the dual thresholds. Based on the comparison result, a corresponding precision control signal is generated. This signal contains a confidence interval identifier and a corresponding inference precision instruction.

[0039] Step S3, Precision Switching and Collaborative Inference: After receiving the precision control signal, the NPU dynamically switches the inference precision mode according to the confidence level range. For image data with a confidence level of 0.89, since it is higher than the first threshold of 0.85, the NPU enables low-precision inference mode, using INT8 quantization to quantize the weights of the self-trained model, converting the 32-bit floating-point weights into 8-bit integer data to reduce computational complexity. When a frame image has incomplete feature extraction due to occlusion, and the generated pseudo-label vehicle has a confidence level of 0.72, which is between the first threshold of 0.85 and the second threshold of 0.6, the NPU enables hybrid precision inference mode, allocating 60% of the high-precision computing cores and 40% of the low-precision computing cores according to the confidence level range. The high-precision cores process key features such as vehicle outlines and license plate areas, while the low-precision cores process secondary features such as background textures. When an image frame suffers from insufficient clarity due to heavy fog, and the confidence score of the generated pseudo-label background is 0.52, which is lower than the second threshold of 0.6, the NPU activates high-precision inference mode and triggers CPU-NPU heterogeneous collaborative inference. During heterogeneous collaborative inference, the CPU identifies and deletes invalid pseudo-labels with a confidence score below 0.6 in the image frame using anomaly detection rules. It then uses feature completion logic to supplement the missing edge contour feature dimensions in the image due to fog obstruction. Through probability adjustment rules, the probability distribution corresponding to the remaining valid features is optimized to [0.12, 0.15, 0.73]. Subsequently, the CPU sends the optimized pseudo-label data back to the NPU, which then performs inference calculations in FP32 full-precision mode after receiving the data.

[0040] Step S4, Result Output and Feedback Packaging: After the NPU completes the inference calculation, it outputs the classification result for each frame of image. The classification result is transmitted in real time to the display terminal of the security monitoring center via the network interface. Simultaneously, the feedback packaging module collects three types of data: confidence scores for each frame of image (e.g., 0.89, 0.72, 0.52), precision mode identifiers (01 for low precision mode, 10 for mixed precision mode, and 11 for high precision mode), and inference error data calculated by comparing with manually labeled results (2.3% for low precision mode, 1.8% for mixed precision mode, and 0.9% for high precision mode). These three types of data are packaged into a binary data stream according to a fixed format of "confidence score - precision identifier - inference error," and transmitted via the internal bus to the system-designated SSD storage area. The storage address is indexed and managed by timestamp.

[0041] Step S5, Model Optimization: The feedback optimization module periodically reads feedback information from the SSD storage area and calculates the batch confidence average for every 1000 frames. Assuming the batch confidence average of 1000 frames is 0.78, according to the regularization coefficient adjustment strategy in the adaptive optimization logic, the batch confidence average is negatively correlated with the regularization coefficient. Therefore, the regularization coefficient of the pseudo-label generator in the self-trained model is adjusted from the initial 0.01 to 0.008. Model parameter adjustment uses incremental optimization techniques, updating only the parameters of the last two fully connected layers directly related to pseudo-label generation in the self-trained model, without performing a full model weight update. Parameter updates are fine-tuned using the Adam optimizer with a learning rate of 0.0005. This balances model optimization effectiveness with parameter update efficiency, improving the accuracy of pseudo-label generation while avoiding the waste of computational resources caused by a full update. Through the aforementioned closed-loop mechanism of inference-feedback-optimization, after the model ran continuously for 72 hours, the average confidence level of pseudo-label generation increased from the initial 0.75 to 0.83, and the average inference error decreased by 1.2%.

[0042] like Figure 3 As shown in this embodiment, each step of the above inference calculation process corresponds to a system module: the self-training module performs feature extraction and label generation in step S1, the NPU verification unit performs confidence verification in step S2, the NPU computing unit performs core inference in step S3, the heterogeneous collaboration module schedules the CPU to complete the collaborative optimization in step S3, the feedback packaging module performs result output and feedback packaging in step S4, and the feedback optimization module performs model optimization in step S5.

[0043] In summary, in intelligent security monitoring image classification scenarios, the method of this invention first performs preprocessing operations on the monitoring images using an adapted NPU, extracts image features through a self-trained model and generates pseudo-labels and confidence scores, then transmits the confidence score data via the PCIe bus, and the NPU's dedicated verification unit performs verification according to a dual-threshold rule, thereby dynamically switching the inference accuracy mode. For different confidence levels, low-precision and mixed-precision inference are enabled, or CPU-NPU heterogeneous collaborative inference is triggered to optimize the pseudo-labels. Subsequently, key data is packaged and stored through feedback, and model parameters are adjusted and incrementally updated using adaptive optimization logic, effectively balancing the accuracy and latency of image classification, fully meeting the dual requirements of real-time recognition and accurate classification for urban road security monitoring.

[0044] Example 2:

[0045] This embodiment is applied to a user review sentiment analysis system on a large e-commerce platform. It classifies product review texts on the platform into three sentiment categories: positive, negative, and neutral. By using the method and system of this invention, the throughput of review processing can be improved while ensuring the accuracy of the analysis.

[0046] As Figure 2 shown, the neural network inference calculation method of the self-training large model based on NPU acceleration provided by the second embodiment of the present invention includes steps S1 to S5, and the specific implementation process is as follows:

[0047] Step S1, feature extraction and label generation: The system receives the user review text data transmitted from the back-end of the e-commerce platform, with a daily processing volume of about 500,000 pieces, and the length of the review text ranges from 10 to 200 words. Perform preprocessing operations adapted to NPU inference on the input text: First, use a dictionary-based word segmentation algorithm to split the Chinese review into a sequence of words, remove stop words such as "de", "le", "ma" and special symbols, convert the words into 128-dimensional word vectors through the Word2Vec model, and then use sequence padding and truncation operations to unify the length of the word vector sequences of all reviews to 50, forming a standardized text feature tensor. Input the preprocessed feature tensor into the self-training large model, which is constructed based on the LSTM network structure, captures the context semantic association of the text through the bidirectional LSTM layer, strengthens the feature weights of key sentiment words through the attention mechanism, and then extracts the sentiment features of the text. Generate pseudo labels based on the extracted sentiment features. The pseudo label categories are divided into three categories: positive, negative, and neutral. At the same time, use the Softmax numerical stability logic to calculate the confidence: Obtain the three-dimensional feature vector output by the last layer of the self-training model. Assume that the feature vector corresponding to a certain review is [2.8, 0.6, 1.2]. Calculate the maximum value 2.8 of this vector, subtract 2.8 from each element in the feature vector to get [0.0, -2.2, -1.6], perform Softmax normalization operation on the processed vector to obtain the probability vector [0.82, 0.05, 0.13], and take the maximum value 0.82 in this probability vector as the confidence corresponding to the positive side of the pseudo label of this review.

[0048] Step S2, confidence verification: Transmit the confidence data of each review to the NPU in a streaming manner through the PCIe bus. Add a 16-bit parity check code to each confidence data during the transmission process, and the check code is transmitted synchronously with the confidence data to ensure that the data is not lost or tampered with during the transmission process. The dedicated verification unit built in the NPU reads the dual-threshold parameters configured in advance through the software interface through the AXI bus, where the first threshold is set to 0.8 and the second threshold is set to 0.55, and the parameters take effect immediately after being read. The dedicated verification unit performs real-time verification on the received confidence data, compares the confidence of each review with the dual thresholds one by one, and generates an accuracy control signal containing confidence interval information and inference accuracy instructions according to the comparison results.

[0049] Step S3, Precision Switching and Collaborative Inference: After receiving the precision control signal, the NPU dynamically switches the inference precision mode according to the confidence-precision mapping rule: For comment data with a confidence of 0.82, since it is higher than the first threshold of 0.8, the NPU enables low-precision inference mode, using INT16 quantization to quantize the weights of the self-trained model, reducing computational resource consumption. When the confidence of a pseudo-label corresponding to neutrality generated due to semantic ambiguity in a comment is 0.68, which is between the first threshold of 0.8 and the second threshold of 0.55, the NPU enables hybrid precision inference mode, allocating 70% of high-precision computing cores and 30% of low-precision computing cores according to the confidence interval. High-precision cores process lexical features with unclear sentiment in the comment, while low-precision cores process commonly used lexical features with clear semantics. When a comment contains dialect words, resulting in insufficient feature extraction, and the confidence of the generated pseudo-label corresponding to negativeness is 0.50, which is lower than the second threshold of 0.55, the NPU enables high-precision inference mode and triggers CPU-NPU heterogeneous collaborative inference. During heterogeneous collaborative inference, the CPU identifies and deletes invalid pseudo-labels with a confidence level below 0.55 using anomaly detection rules. It then uses feature completion logic to supplement the semantic feature dimensions corresponding to dialect words. Through probability adjustment rules, it corrects the probability distribution corresponding to the optimized sentiment features to [0.10, 0.78, 0.12]. Subsequently, the CPU sends the optimized pseudo-label data back to the NPU. After receiving the data, the NPU performs inference calculations in FP32 full-precision mode.

[0050] Step S4, Result Output and Feedback Packaging: After the NPU completes the inference calculation, it outputs the sentiment classification result for each comment. The classification result is synchronized to the product review management backend of the e-commerce platform, providing merchants with user sentiment analysis data. Simultaneously, the feedback packaging module collects three types of data: confidence scores for each comment (e.g., 0.82, 0.68, 0.50), precision mode identifiers (001 for low precision mode, 010 for mixed precision mode, and 100 for high precision mode), and inference error data calculated by comparing with manually labeled results (3.1% for low precision mode, 2.2% for mixed precision mode, and 1.0% for high precision mode). These three types of data are packaged into structured data blocks according to a preset format of "Comment ID - Confidence Score - Precision Identifier - Inference Error," and transmitted to the system's designated distributed storage area via the internal storage bus, using time-sharding for data storage management.

[0051] Step S5, Model Optimization: The feedback optimization module reads feedback information from the distributed storage area and calculates the batch average confidence score for every 5000 comments. Assuming the average confidence score of a batch of 5000 comments is 0.71, the regularization coefficient of the pseudo-label generator in the self-trained model is adjusted from the initial 0.02 to 0.015 according to the regularization coefficient adjustment strategy in the adaptive optimization logic. Model parameter adjustment uses incremental optimization techniques, updating only the LSTM output layer and fully connected layer parameters directly related to pseudo-label generation in the self-trained model, without updating the entire model weights. Parameter updates are fine-tuned using the Adam optimizer with a learning rate of 0.0008. This balances model optimization effectiveness with parameter update efficiency, avoiding overfitting caused by excessive fine-tuning. Through a closed-loop mechanism of inference-feedback-optimization, after running continuously for 30 days, the average confidence level of pseudo-label generation increased from the initial 0.69 to 0.78, the average inference error decreased by 1.5%, and the comment processing throughput increased from 80,000 comments per hour to 120,000 comments per hour.

[0052] like Figure 3 As shown in this embodiment, each step of the above inference calculation process corresponds to a system module: the self-training module performs feature extraction and label generation in step S1, the NPU verification unit performs confidence verification in step S2, the NPU computing unit performs core inference in step S3, the heterogeneous collaboration module schedules the CPU to complete the collaborative optimization in step S3, the feedback packaging module performs result output and feedback packaging in step S4, and the feedback optimization module performs model optimization in step S5.

[0053] In summary, in the scenario of sentiment analysis for user comments on e-commerce platforms, the method of this invention first performs preprocessing on the comment text, such as word segmentation and word vector conversion, to adapt to the NPU. It then extracts sentiment features and calculates confidence scores using a self-trained model, followed by NPU validation units and dual-threshold rules to trigger corresponding precision inference. For situations involving semantic ambiguity or dialectal vocabulary, it optimizes pseudo-labels through CPU-NPU heterogeneous collaboration. After the feedback data is packaged and stored, the model parameters are adjusted according to adaptive optimization logic. Ultimately, while maintaining the accuracy of sentiment analysis, it significantly improves comment processing efficiency, meeting the actual needs of large e-commerce platforms for rapid and accurate analysis of massive amounts of user comments.

[0054] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for inference computation of self-trained large-scale neural networks based on NPU acceleration, characterized in that, The specific steps of this method are as follows: S1, Feature Extraction and Label Generation: Perform preprocessing operations on the input data to adapt to NPU inference, input the preprocessed data into the self-trained large model, extract data features through the self-trained large model, generate pseudo-labels based on the extracted features, and calculate the confidence level corresponding to the pseudo-label. S2, Confidence verification: The calculated confidence data is transmitted to the NPU. The NPU's built-in dedicated verification unit verifies the confidence in real time using a preset dual-threshold comparison rule. Based on the verification result, the corresponding precision control signal is output. S3, Precision Switching and Collaborative Inference: The NPU receives a precision control signal and dynamically switches the inference precision mode according to the signal. When the confidence level is lower than the preset second threshold, heterogeneous collaborative inference between the CPU and the NPU is triggered. S4, Result Output and Feedback Packaging: The NPU outputs the inference results and simultaneously collects three types of data: confidence data, precision mode identifier, and inference error. These are then packaged into feedback information according to a preset format and transmitted to the system's designated storage area. S5, Model Optimization: Based on feedback information within the storage area, adaptive optimization logic is used to adjust the parameters of the self-trained model, optimize the pseudo-label generation strategy, and form a closed-loop mechanism of inference-feedback-optimization.

2. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S1, the confidence level is calculated using Softmax numerical stabilization logic. The specific process is as follows: obtain the feature vector output from the last layer of the self-trained large model, calculate the maximum value of the feature vector, subtract the maximum value from each element in the feature vector, perform Softmax normalization on the processed feature vector to obtain the probability vector, and take the maximum value in the probability vector as the confidence level corresponding to the pseudo-label.

3. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S2, the dual thresholds in the dual threshold comparison rule include a first threshold and a second threshold. The values ​​of the first threshold and the second threshold are both in the range of 0 to 1, and the first threshold is greater than the second threshold. The dual thresholds are preset and configured through a software interface. The dedicated verification unit of the NPU reads the configured dual threshold parameters through the AXI bus, and the parameters take effect immediately after being read.

4. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S2, the confidence data is transmitted using PCIe bus high-speed transmission technology. The transmission process follows streaming transmission logic. Transmission integrity is verified by adding a check code to the transmitted data. The check code is transmitted synchronously with the confidence data.

5. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S3, the precision switching adopts a confidence-precision mapping rule, specifically: when the confidence level is higher than the first threshold, the NPU enables low-precision inference mode and performs quantization processing on the weights of the self-trained model; when the confidence level is between the first and second thresholds, the NPU enables mixed-precision inference mode and dynamically allocates the proportion of high- and low-precision computing cores according to the confidence level range; when the confidence level is lower than the second threshold, the NPU enables high-precision inference mode and triggers CPU-NPU heterogeneous collaborative inference.

6. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S3, the specific process of heterogeneous collaborative inference is as follows: the CPU identifies and deletes invalid pseudo-labels through anomaly detection rules, supplements the missing feature dimensions corresponding to the pseudo-labels using feature completion logic, and optimizes the probability distribution of the pseudo-labels through probability adjustment rules; the CPU sends the optimized pseudo-label data back to the NPU, and the NPU performs inference calculations in full-precision mode after receiving the data.

7. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S5, the adaptive optimization logic includes a regularization coefficient adjustment strategy, specifically: calculating the batch confidence mean, and adjusting the regularization coefficient of the pseudo-label generator in the self-trained model based on the mean. The batch confidence mean and the regularization coefficient are negatively correlated, that is, the lower the batch confidence mean, the larger the regularization coefficient.

8. The method for inference computation of a self-trained large-scale neural network based on NPU acceleration according to claim 1, characterized in that, In step S5, the model parameter adjustment adopts incremental optimization technology, specifically: only the network layer parameters directly related to pseudo-label generation in the self-trained model are updated, and the full weight of the model is not updated; the parameter update is fine-tuned through the Adam optimizer, and the model optimization effect and parameter update efficiency are balanced during the adjustment process.

9. A self-trained large-scale neural network inference computing system based on NPU acceleration, the system being applicable to the self-trained large-scale neural network inference computing method based on NPU acceleration as described in any one of claims 1-8, characterized in that, The system includes: The self-training module is used to perform preprocessing operations on the input data, extract data features, and generate pseudo-labels and corresponding confidence scores. The NPU verification unit, integrated inside the NPU, is used to receive confidence data, perform real-time verification through dual threshold comparison rules, and output precision control signals. The NPU computing unit is used to receive precision control signals and dynamically switch inference precision modes, perform inference calculations, and trigger heterogeneous collaborative inference when the confidence level is lower than the second threshold. The heterogeneous collaboration module is used to schedule the CPU to delete invalid pseudo-labels, supplement feature dimensions, and optimize probability distribution, and then send the optimized pseudo-label data back to the NPU computing unit. The feedback packaging module is used to collect confidence data, accuracy mode identifiers, and inference errors, package them into feedback information according to a preset format, and transmit them to a designated storage area. The feedback optimization module is used to read feedback information from the storage area, adjust the parameters of the self-training module using adaptive optimization logic, and optimize the pseudo-label generation strategy.

Images