Target prediction method and device based on hybrid precision quantization, equipment and medium

Abstract

The application relates to the technical field of model deployment, and discloses a target prediction method and device based on mixed precision quantization, equipment and a medium, which comprises the following steps: analyzing an initial neural network model to generate network layer structure data; determining a set of network layers to be quantized, obtaining running overhead data and quantization sensitivity data; generating weight quantization correlation data and a quantization candidate model; generating basic loss data and error constraint data based on a training sample set, updating trainable model parameters, and generating a quantization training model; generating a compressed neural network model, and outputting a target prediction result. The application can be applied to business scenarios such as financial technology and medical health, quantization configuration is participated by the running overhead data and the quantization sensitivity data, and parameter updating is jointly constrained by the basic loss data and the error constraint data, so that the deployment availability of the compressed model and the target prediction stability are improved.

Description

Technical Field

[0001] This invention relates to the field of model deployment technology, and in particular to a target prediction method, apparatus, device and medium based on hybrid precision quantization. Background Technology

[0002] As target prediction models gradually extend from the server side to edge environments such as mobile terminals, in-vehicle devices, and edge computing devices, the core problem faced by existing deep learning models when deployed on edge is that it is difficult to simultaneously meet deployment resource constraints and prediction reliability requirements after model compression. Existing models typically rely on large parameter scales and complex network layer structures, and direct operation is limited by storage space, inference overhead, and power consumption. On the other hand, uniform quantization compression methods usually use the same bit width for different network layers, failing to fully distinguish the sensitivity of different network layers to quantization changes. This can easily lead to the loss of feature representation in key layers, resulting in a reduced model size but insufficient stability of prediction results.

[0003] In the fintech business, scenarios such as insurance claims assessment, auto insurance inspection, claims review, and damage amount verification require rapid image recognition, damage area identification, and damage severity prediction on mobile terminals or in-vehicle devices. Existing damage assessment models, if directly deployed to edge devices, are easily limited by device computing power, storage capacity, and real-time response requirements. Using a uniform low-bit quantization method for compression can weaken the expression of vehicle damage boundaries, component areas, and damage severity features, causing deviations in damage assessment predictions under complex image conditions, making it difficult to balance edge deployment constraints and claims prediction reliability.

[0004] In the healthcare sector, there is also a demand for edge-side target prediction in scenarios such as initial medical image screening, mobile diagnosis and treatment, health monitoring, and chronic disease follow-up. Existing medical image recognition or health risk prediction models are also limited by storage resources and inference overhead when running on the edge. When using a unified quantization compression method, the network layers containing lesion edges, abnormal areas, and fine-grained physiological features may suffer feature loss due to their high sensitivity to quantization, making it difficult for the compressed model to maintain stable predictive capabilities in the edge environment. Summary of the Invention

[0005] The main objective of this invention is to provide a target prediction method, apparatus, device, and storage medium based on mixed-precision quantization, aiming to solve the technical problem that existing unified quantization model compression methods do not distinguish the quantization sensitivity of different network layers, resulting in insufficient prediction reliability of the compressed model under resource-constrained deployment conditions on the edge.

[0006] To achieve the above objectives, the present invention provides a target prediction method based on mixed-precision quantization, comprising: Obtain the initial neural network model, training sample set, and model deployment constraint data; perform hierarchical structure analysis on the initial neural network model to generate network layer structure data. Based on the network layer structure data, determine the set of network layers to be quantized, and obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized; Based on the model deployment constraint data, the runtime overhead data, and the quantization sensitivity data, the set of network layers to be quantized is subjected to mixed precision quantization processing to generate weighted quantization correlation data and quantization candidate models. Forward propagation is performed based on the training sample set and the quantized candidate model to generate prediction result data. Basic loss data is generated based on the prediction result data and the sample labeling data in the training sample set. Error constraint data is generated based on the weight quantization correlation data. Based on the basic loss data and the error constraint data, overall loss data is generated, and based on the overall loss data, the trainable model parameters of the quantization candidate model are updated to generate a quantization training model. Compressed analysis data is generated based on the initial neural network model, the weighted quantization correlation data, and the quantization training model. A compressed neural network model is then generated based on the quantization training model, provided that the compressed analysis data satisfies the model deployment constraints. Obtain the target input data, input the target input data into the compressed neural network model for prediction, and generate the target prediction result.

[0007] Furthermore, to achieve the above objectives, the present invention provides a target prediction device based on hybrid precision quantization, comprising: The hierarchical structure parsing module is used to obtain the initial neural network model, training sample set and model deployment constraint data, and to perform hierarchical structure parsing on the initial neural network model to generate network layer structure data. The quantization layer analysis module is used to determine the set of network layers to be quantized based on the network layer structure data, and to obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized. The mixed-precision quantization module is used to perform mixed-precision quantization on the set of network layers to be quantized based on the model deployment constraint data, the runtime overhead data and the quantization sensitivity data, and generate weighted quantization correlation data and quantization candidate models. The training loss generation module is used to perform forward propagation based on the training sample set and the quantized candidate model to generate prediction result data, generate basic loss data based on the prediction result data and the sample labeling data in the training sample set, and generate error constraint data based on the weight quantization correlation data. The quantization training update module is used to generate overall loss data based on the basic loss data and the error constraint data, and update the trainable model parameters of the quantization candidate model based on the overall loss data to generate a quantization training model. The compressed model generation module is used to generate compressed analysis data based on the initial neural network model, the weighted quantization correlation data, and the quantization training model, and to generate a compressed neural network model based on the quantization training model under the condition that the compressed analysis data satisfies the model deployment constraint data. The target prediction module is used to acquire target input data, input the target input data into the compressed neural network model for prediction, and generate target prediction results.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a target prediction program based on mixed precision quantization stored in the memory and executable on the processor, wherein when the target prediction program based on mixed precision quantization is executed by the processor, it implements the steps of the target prediction method based on mixed precision quantization as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a target prediction program based on mixed-precision quantization, wherein the target prediction program based on mixed-precision quantization, when executed by a processor, implements the steps of the target prediction method based on mixed-precision quantization as described above.

[0010] Beneficial Effects: This invention relates to the field of model deployment technology and discloses a target prediction method, apparatus, device, and medium based on mixed-precision quantization. The method includes: parsing an initial neural network model to generate network layer structure data; determining the set of network layers to be quantized and acquiring runtime overhead data and quantization sensitivity data; generating weighted quantization correlation data and quantization candidate models based on model deployment constraint data, runtime overhead data, and quantization sensitivity data; generating basic loss data and error constraint data based on a training sample set, and updating trainable model parameters to generate a quantized training model; generating a compressed neural network model, and generating target prediction results based on the compressed neural network model. This invention can be applied to business scenarios such as fintech and healthcare. By using runtime overhead data and quantization sensitivity data in mixed-precision quantization, different network layers form corresponding quantization configurations; by using basic loss data and error constraint data to jointly constrain parameter updates, the model can take into account both prediction bias and quantization bias, thereby improving the deployment availability and target prediction stability of the compressed model. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a target prediction method based on hybrid precision quantization in one embodiment of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the target prediction method based on hybrid precision quantization of the present invention. Figure 3 for Figure 2 A detailed flowchart of step S10; Figure 4 for Figure 2 A detailed flowchart of step S20; Figure 5 for Figure 2 A detailed flowchart of step S30; Figure 6 for Figure 2 A detailed flowchart of step S40; Figure 7 for Figure 2 A detailed flowchart of step S50; Figure 8 for Figure 2 A detailed flowchart of step S60; Figure 9 This is a schematic diagram of the functional modules of a preferred embodiment of the target prediction device based on hybrid precision quantization of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The target prediction method based on mixed-precision quantization provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can parse the initial neural network model from the client to generate network layer structure data; determine the set of network layers to be quantized and obtain runtime overhead data and quantization sensitivity data; generate weighted quantization correlation data and quantization candidate models based on model deployment constraint data, runtime overhead data, and quantization sensitivity data; generate basic loss data and error constraint data based on the training sample set, update the trainable model parameters, and generate a quantization training model; generate a compressed neural network model, and generate target prediction results based on the compressed neural network model. This invention can be applied to business scenarios such as fintech and healthcare. By using runtime overhead data and quantization sensitivity data to participate in mixed precision quantization, different network layers form corresponding quantization configurations; by using basic loss data and error constraint data to jointly constrain parameter updates, the model can take into account both prediction bias and quantization bias, thereby improving the deployment availability and target prediction stability of the compressed model. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the target prediction method based on mixed-precision quantization provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the target prediction method based on hybrid precision quantization proposed in this invention includes the following steps: S10, Obtain the initial neural network model, training sample set and model deployment constraint data, perform hierarchical structure parsing on the initial neural network model, and generate network layer structure data; In this embodiment, the initial neural network model provides the model structure and parameter source. The structure file records the data transfer relationship between the convolutional layer, normalization layer, activation layer, pooling layer, attention layer, feature fusion layer and output layer, and the parameter file records the weight data, bias data and parameter storage location of each network layer.

[0016] The training sample set provides the source of input format and annotation structure, which may include sample input data, sample annotation data, sample identifiers, and sample format information. The model deployment constraint data provides the edge-side operating boundaries, which may include storage boundaries, inference overhead boundaries, power consumption boundaries, supported operator types, and supported quantization bit width ranges.

[0017] Hierarchical structure parsing is used to break down the initial neural network model into identifiable network layer objects. During parsing, the layer arrangement, layer type, input / output tensors, parameter index positions, and inter-layer data transfer directions are read. The input layer boundaries are verified using the training sample set, and the operational constraints corresponding to the network layers are recorded using model deployment constraint data, generating network layer structure data.

[0018] With the initial neural network model structure fixed, the structure file is read using a static structure parsing method. The network layer sequence is extracted according to the network node registration order. The weight data and bias data are located according to the parameter index position. Then, the storage boundary, inference overhead boundary, and quantization bit width support range are written into the corresponding network layer record.

[0019] When the initial neural network model contains branching structures, residual connections, or multiple output structures, a sample-driven parsing method is adopted. The sample input data from the training sample set is input into the initial neural network model, and the input tensor shape, output tensor shape, branch merging position, and output node position of each network layer are recorded. Then, the data is matched with the network layer definition to generate network layer structure data with the actual data transmission direction.

[0020] When there are differences in the operating constraints on the device side, the model deployment constraint data is split into multiple sets of deployment constraint reference information according to the device type. The mobile terminal side focuses on storage boundary and power consumption boundary, the vehicle device side focuses on inference overhead boundary and image size boundary, and the health monitoring device side focuses on low power consumption boundary and timing input window boundary.

[0021] In the fintech business, the insurance claims platform uses the vehicle damage image recognition model as the initial neural network model. It organizes the vehicle insurance inspection images, damage area annotations, and damage assessment review annotations into a training sample set. It organizes the storage boundaries, inference overhead boundaries, and supported quantization bit widths of the mobile inspection terminal into model deployment constraint data. After hierarchical structure parsing, the generated network layer structure data records the data transmission relationship between the vehicle image input layer, the local texture extraction layer, the damage area fusion layer, and the damage assessment result output layer.

[0022] In the field of healthcare, the health management platform uses the abnormal state recognition model as the initial neural network model, organizes body surface images, physiological time-series signals and health status annotations into a training sample set, and organizes the low power consumption boundary, input window boundary and inference overhead boundary of mobile health devices into model deployment constraint data. After hierarchical structure parsing, the generated network layer structure data records the data transmission relationship between the time-series coding layer, image feature extraction layer, feature fusion layer and risk output layer.

[0023] This embodiment uses the initial neural network model, training sample set, and model deployment constraint data together for hierarchical structure analysis. The network layer structure data can simultaneously express the hierarchical relationship of the model, the parameter storage state, the input data form, and the edge operation boundary, so that the subsequent network layer selection and quantification analysis have a unified data source, reducing the compression object deviation caused by unclear model structure, inconsistent input form, or missing deployment constraints.

[0024] S20, determine the set of network layers to be quantized based on the network layer structure data, and obtain the running overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized; In this embodiment, the network layer structure data records the layer type, parameter size, input / output tensor form, data transmission direction, and parameter storage location. The set of network layers to be quantized is obtained by filtering based on layer type and parameter size. The filtering objects include network layers containing weight data or bias data, and network layers that did not participate in output generation are excluded based on the data transmission direction.

[0025] The overhead data is used to express the resource consumption of each network layer to be quantized. It can be obtained from the parameter size, input and output tensor size, layer type and data transfer number, including parameter access volume, feature transfer volume and layer operation volume, and is bound to the network layer identifier to be quantized.

[0026] Quantization sensitivity data is used to express the degree of influence of each network layer to be quantized on the output variation under different quantization bit widths. During generation, calibration sample data is input into the initial neural network model to obtain baseline output data. Then, candidate bit width probe quantization is performed on a single network layer to be quantized, while keeping the parameter states of other network layers unchanged. The quantization sensitivity data is obtained based on the offset between the probe output data and the baseline output data.

[0027] When the network structure is clear and the input size is fixed, a structure-based selection method is used to determine the set of network layers to be quantized. Convolutional layers, fully connected layers, attention projection layers, and feature fusion layers are selected based on layer type. Network layers with too few parameters are removed based on parameter size. The set of network layers to be quantized is obtained by combining the data propagation direction. The runtime data is generated based on parameter size, input and output tensor size, and layer type.

[0028] In cases where the network has branches, residual connections, or multiple output heads, a calibration sample tracking method is adopted. The calibration sample data is input into the initial neural network model to record whether each network layer participates in output generation, whether the output tensor is consumed by subsequent network layers, and the branch merging position. Then, the set of network layers to be quantized is determined, and runtime overhead data is generated based on the actual tensor flow records and parameter access records.

[0029] In the fintech business, the insurance claims platform analyzes the network layer structure data of the vehicle damage recognition model, selects the set of network layers to be quantized to participate in vehicle body area recognition, damage texture extraction and damage assessment result output, generates runtime overhead data based on the input size of the survey image, the scale of network layer parameters and end-side inference records, and obtains quantization sensitivity data through the output offset under different candidate quantization bit widths.

[0030] In the field of healthcare, the health management platform analyzes the network layer structure data of the abnormal state recognition model, selects the set of network layers to be quantized to participate in the extraction of body surface image features, physiological time sequence encoding and risk output, generates runtime overhead data based on input window size, parameter access volume and feature transfer volume, and obtains quantization sensitivity data by calibrating the output offset of the sample before and after detection quantization.

[0031] This embodiment determines the set of network layers to be quantized based on network layer structure data and obtains the corresponding runtime overhead data and quantization sensitivity data. This allows the quantization object to be limited to network layers that participate in the generation of effective output and contain quantizable parameters. This enables each network layer to be quantized to have both resource usage description and quantization impact description, thereby reducing data interference caused by invalid network layers participating in quantization and improving the accuracy of network layer selection and bit width allocation in subsequent mixed precision quantization.

[0032] S30, based on the model deployment constraint data, the runtime overhead data and the quantization sensitivity data, perform mixed precision quantization processing on the set of network layers to be quantized to generate weighted quantization correlation data and quantization candidate models; In this embodiment, model deployment constraint data is used to define edge-side resource boundaries, including storage boundaries, inference overhead boundaries, power consumption boundaries, and supported bit width ranges. Runtime overhead data is used to represent the resource consumption differences of each network layer to be quantized, and quantization sensitivity data is used to represent the degree of impact of quantization on the output of each network layer.

[0033] Mixed-precision quantization assigns different quantization bit widths to different network layers in the set of network layers to be quantized, based on model deployment constraint data, runtime overhead data, and quantization sensitivity data. Network layers with higher runtime overhead and lower quantization sensitivity are assigned lower bit widths, while network layers with higher quantization sensitivity retain higher bit widths.

[0034] The weight quantization association data record records the correspondence between network layer identifiers, quantization bit widths, quantization mapping parameters, and quantization weight data. The quantization candidate model is obtained by replacing the weights in the set of network layers to be quantized with the quantized weight data; network layers not included in the set of network layers to be quantized retain their original parameter states.

[0035] When edge storage resources are limited, a constraint-first allocation method is adopted. The quantization bit width of each network layer is determined based on the operating overhead data and quantization sensitivity data, and the overall storage usage is verified based on the model deployment constraint data.

[0036] In situations where real-time response requirements are high on the edge, an overhead constraint allocation method is adopted. For network layers that contribute significantly to inference overhead, the bit width is reduced first, while for network layers with large output offsets, the bit width reduction is limited, thereby generating weighted quantized correlation data and quantized candidate models.

[0037] In the fintech business, the insurance claims platform uses the storage boundary and inference overhead boundary of the mobile survey terminal as model deployment constraint data for the vehicle damage recognition model. Based on the operating overhead data and quantization sensitivity data of the vehicle body area recognition layer, damage texture extraction layer and damage assessment output layer, different quantization bit widths are allocated to generate weighted quantization correlation data and quantization candidate models.

[0038] In the field of healthcare, the health management platform uses the power consumption boundary and inference overhead boundary of mobile health devices as model deployment constraints for abnormal state identification models. Based on the operating overhead data of the image feature extraction layer, physiological time series encoding layer and risk output layer, as well as the quantization sensitivity data, quantization weight data is generated to obtain quantization candidate models.

[0039] This embodiment defines the overall resource boundary through model deployment constraint data, distinguishes the resource consumption of network layers through runtime overhead data, and distinguishes the output impact of network layers on quantization changes through quantization sensitivity data, so that different network layers can obtain differentiated quantization bit widths; weighted quantization correlation data records the correspondence between bit width, mapping parameters and quantization weights, so that the quantization candidate model can achieve a more accurate balance between compression scale and output stability.

[0040] S40, perform forward propagation based on the training sample set and the quantized candidate model to generate prediction result data, generate basic loss data based on the prediction result data and the sample labeling data in the training sample set, and generate error constraint data based on the weight quantization correlation data; In this embodiment, the training sample set provides sample input data and sample annotation data. After the sample input data is input into the quantization candidate model, the quantization weight data participates in the forward propagation operation to generate prediction result data. The prediction result data is aligned with the sample annotation data according to the sample identifier or output field, and basic loss data is generated based on the difference between the predicted value and the labeled value.

[0041] The weight quantization associated data record includes the identifier of the network layer to be quantized, the original weight data, the quantized weight data, and the quantization bit width data. A correspondence is established between the original weight data and the quantized weight data according to the identifier of the network layer to be quantized. Error constraint data is generated through a weight offset metric to express the parameter offset caused by the quantization transformation.

[0042] In the case of classification prediction, the sample input data is input into the quantization candidate model to obtain classification probability data; after the classification probability data is aligned with the category label, basic loss data is generated, and error constraint data is generated based on the difference between the original weight data and the quantized weight data.

[0043] In the case of regression prediction, the continuous predicted values ​​output by the quantization candidate model are aligned with the labeled values ​​in the sample labeled data to obtain the basic loss data; then, error constraint data is generated by combining the network layer identifier to be quantized and the quantization bit width data.

[0044] In the fintech business, the insurance claims platform uses vehicle insurance inspection images as sample input data, damage location and damage assessment review labels as sample annotation data, and after the quantified candidate model generates damage prediction result data, it aligns with the sample annotation data to generate basic loss data, and combines weighted quantified correlation data to generate error constraint data.

[0045] In the field of healthcare, health management platforms use body surface images or physiological time-series signals as sample input data and abnormal states or risk levels as sample annotation data. After quantifying candidate models to generate health status prediction results, they align with the sample annotation data to generate basic loss data, and generate error constraint data based on weighted quantified correlation data.

[0046] This embodiment generates prediction result data by quantizing candidate models, and generates basic loss data based on the prediction result data and sample labeling data, which can express the model output bias; it generates error constraint data by quantizing weighted correlation data, which can express the parameter offset caused by weight quantization, so that subsequent model updates are constrained by both output bias and quantization bias, thereby improving the prediction stability in quantization training.

[0047] S50, Generate overall loss data based on the basic loss data and the error constraint data, and update the trainable model parameters of the quantization candidate model based on the overall loss data to generate a quantization training model; In this embodiment, the basic loss data represents the output deviation between the prediction result and the sample label, while the error constraint data represents the parameter offset caused by weight quantization. The numerical ranges of the two types of loss data may be different. Scale alignment can be performed first, and then basic loss weight data and error constraint weight data can be generated separately, so that the output deviation and quantization offset can jointly participate in the model parameter update.

[0048] The overall loss data is obtained by fusing the weighted base loss data and the weighted error constraint data. The fusion method can be weighted summation, segmented weighting, or dynamic weight adjustment, so that the quantization candidate model retains both the prediction task constraint and the quantization error constraint during training.

[0049] Trainable model parameters include the weight parameters, bias parameters, normalization parameters, or output layer parameters that are allowed to be updated in the quantization candidate model. Backpropagation is performed based on the overall loss data to generate parameter update data, which is then used to update the trainable model parameters, resulting in the quantized trained model. The quantized trained model retains the quantization structure of the quantization candidate model while ensuring that the trainable model parameters are adjusted after applying loss constraints.

[0050] When there is a significant difference between the base loss data and the error constraint data, a scaling normalization method is used to generate the overall loss data. The base loss data and the error constraint data are transformed to a unified numerical range, and then fused according to preset weights to obtain the overall loss data. The trainable model parameters in the quantization candidate model are then updated through backpropagation.

[0051] In situations where the impact of quantization offset needs to be controlled, a dynamic weighting approach is used to generate overall loss data. In the early stages of training, the weight of the basic loss data is increased to allow the model output to quickly approach the sample labels; in the later stages of training, the weight of the error constraint data is increased to maintain a more stable correspondence between the updated parameters and the quantization weights, ultimately resulting in a quantized training model.

[0052] In the field of fintech business, the insurance claims platform generates basic loss data from the vehicle damage prediction deviation and generates error constraint data from the offset between the quantized weights and the original weights in the vehicle damage identification model. After the two types of loss data are merged, the trainable model parameters of the quantized candidate model are updated, so that the generated quantized training model reduces the output offset caused by quantization in the end-side damage assessment prediction.

[0053] In the field of healthcare, the health management platform generates basic loss data from the abnormal state prediction deviation, generates error constraint data from the weight offset of each network layer to be quantized in the abnormal state identification model, and then updates the trainable model parameters based on the overall loss data, so that the generated quantized training model can maintain the stability of health state prediction in low-power devices.

[0054] This embodiment generates overall loss data by combining basic loss data and error constraint data, which allows model parameter updates to be constrained by both prediction output bias and weight quantization offset. Updating the trainable model parameters of the quantization candidate model using overall loss data can reduce the fluctuation of the quantized model output, enabling the generated quantized training model to balance prediction accuracy and quantization stability.

[0055] S60, based on the initial neural network model, the weight quantization correlation data and the quantization training model, compressed analysis data is generated, and under the condition that the compressed analysis data satisfies the model deployment constraint data, a compressed neural network model is generated based on the quantization training model; In this embodiment, the initial neural network model provides a pre-compression structure and parameter reference, the quantization training model provides the post-training quantization parameter state, and the weight quantization correlation data provides the relationship between network layer identifiers, quantization bit widths, quantization mapping parameters, and quantization weights. The compression analysis data may include parameter storage changes, quantization bit width distributions, quantization weight write states, and inference overhead changes, used to determine whether the quantization training model meets the edge deployment requirements.

[0056] When generating compressed analysis data, the parameter storage data in the initial neural network model and the quantization training model are compared to statistically analyze the changes in parameter usage of the network layers to be quantized. Then, based on the weighted quantization correlation data, the quantization bit width, quantization mapping parameters, and quantization weight data are verified to obtain the stored compressed data, inference overhead analysis data, and quantization integrity analysis data. After the compressed analysis data meets the model deployment constraints, the training auxiliary state is removed, while the inference structure, quantization weight data, and quantization mapping parameters are retained to generate the compressed neural network model.

[0057] When storage capacity is limited on the edge, compressed analysis data is generated by comparing parameter usage. The parameter storage of the same network layer in the initial neural network model and the quantized training model is compared, and the overall storage usage is calculated in conjunction with the quantization bit width. When the overall storage usage does not exceed the storage boundary in the model deployment constraint data, the compressed neural network model is derived.

[0058] When the inference overhead is limited on the edge side, a network layer overhead aggregation method is adopted. Inference overhead analysis data is generated based on the input and output tensor shape, quantization bit width, and parameter access volume of each network layer in the quantization training model. When the inference overhead analysis data meets the inference overhead boundary in the model deployment constraint data, the inference structure and quantization parameters are retained, and the training state data is removed.

[0059] In cases where the integrity of the quantization results needs to be verified, the identification of the network layer to be quantized, the quantization mapping parameters, and the quantization weight data are checked against the weighted quantization correlation data to see if they have been written into the quantization training model. When the verification results meet the model structure boundaries and bit width support range, a compressed neural network model is generated.

[0060] In the fintech business, the insurance claims platform compares the pre-compression parameter state of the vehicle damage recognition model with the quantized training model, and combines the weighted quantized correlation data to statistically analyze the quantization bit width, parameter usage, and inference overhead of the vehicle body area recognition layer, damage texture extraction layer, and damage assessment output layer; when the compressed analysis data meets the storage boundary and inference overhead boundary of the mobile survey terminal, a compressed neural network model for end-side damage assessment prediction is generated.

[0061] In the field of healthcare, the health management platform compares the initial parameter state of the abnormal state identification model with the quantized training model, and verifies the quantized weights of the image feature extraction layer, physiological time series encoding layer and risk output layer with the weighted quantized correlation data. When the compressed analysis data meets the low power consumption boundary and inference overhead boundary of the mobile health device, a compressed neural network model for edge health state prediction is generated.

[0062] This embodiment generates compressed analysis data through an initial neural network model, weight quantization correlation data, and quantization training model. This data can reflect changes in parameter usage, quantization weight writing status, and inference overhead before and after compression. By matching the compressed analysis data with model deployment constraint data, a compressed neural network model is generated. This can reduce model outputs that do not meet the edge resource boundaries and improve the structural integrity and resource matching degree of the compressed model before deployment.

[0063] S70, acquire target input data, input the target input data into the compressed neural network model for prediction, and generate target prediction results.

[0064] In this embodiment, the target input data is the data to be predicted that enters the compressed neural network model during the deployment and operation phase. It can be image data, time-series signals, structured fields, or multimodal combined data. Before input, the target input data is subjected to size adjustment, channel arrangement, numerical normalization, field encoding, or time window truncation to match the data form with the input layer of the compressed neural network model.

[0065] Compressed neural network models preserve the inference network structure, quantized weight data, and quantized mapping parameters. After the target input data enters the model, the quantized weight data participates in convolution, matrix multiplication, feature fusion, or output mapping operations to generate the model's inference output data. The model's inference output data is then converted into target prediction results based on the output structure. These prediction results can include category, probability, region location, risk level, or numerical prediction value.

[0066] When the target input data is an image, an image structure transformation method is used to transform the image size, channel order, and pixel value range, and then input into a compressed neural network model to obtain category prediction data, region response data, or numerical prediction data.

[0067] When the target input data is a time-series signal, the continuous signal is divided into segments by window truncation, and the sampled values ​​within the segments are normalized and transformed before being input into a compressed neural network model to obtain the prediction results of state category, risk level or trend.

[0068] When the target input data contains multiple data sources, field encoding and feature concatenation are used to convert image features, temporal features, or structured fields into a unified input form, which is then input into a compressed neural network model to obtain the target prediction result.

[0069] In the fintech business, insurance claims platforms use vehicle inspection images, vehicle part information, or structured claim fields as target input data. After size conversion, field encoding, and numerical normalization, the data is input into a compressed neural network model to generate damage category, damage area, candidate repair items, or predicted damage amount.

[0070] In the field of healthcare, health management platforms use body surface images, physiological time-series signals, or health management records as target input data. After image conversion, time-series window truncation, and field encoding, the data is input into a compressed neural network model to generate prediction results of abnormal states, risk levels, or health status.

[0071] This embodiment transforms the input form of the target input data and inputs the transformed data into a compressed neural network model for prediction. This allows the quantized weight data to directly participate in the calculation during the inference stage, generating a target prediction result that conforms to the output structure. This reduces inference bias caused by inconsistent input forms and improves the prediction stability of the compressed neural network model during edge operation.

[0072] In one embodiment, such as Figure 3 As shown, step S10 above includes: S101, obtain the initial neural network model, training sample set and model deployment constraint data, extract model definition information and parameter index information from the initial neural network model, and determine the network layer sequence and data flow relationship between each network layer based on the model definition information; S102, Traverse the network layer sequence, identify the layer type of each network layer, and determine the depth level of each network layer according to the arrangement position of each network layer in the network layer sequence; S103, extract parameter description data for each network layer based on the parameter index information; S104, extract sample input structure data from the training sample set, and establish a correspondence between the sample input structure data and the input layer in the network layer sequence to generate input layer boundary data; S105, extract deployment resource boundary data from the model deployment constraint data, and establish a reference relationship between the deployment resource boundary data and the network layer sequence to generate deployment constraint reference data; S106, combine the layer type, depth level, parameter description data, input layer boundary data, and deployment constraint reference data of each network layer with the data flow relationship to generate network layer structure data.

[0073] In this embodiment, after the initial neural network model is read, the structural description section and the parameter storage section of the model file are separated. The structural description section is extracted as model definition information, which records the input nodes, network layer nodes, output nodes, node connection ports, tensor names, and node arrangement relationships. The parameter storage section is extracted as parameter index information, which records the mapping position between network layer identifiers and parameter data blocks, enabling network layer structure parsing to simultaneously locate network connection relationships and parameter storage locations.

[0074] After the model definition information is parsed by nodes, a network layer sequence is generated according to the input-output reference relationships between nodes. The network layer sequence records not only the arrangement of network layers but also the data flow relationships between each network layer. For single-input, single-output structures, an order relationship can be established based on the names of the output tensors of the previous layer and the input tensors of the next layer. For branching, merging, or cross-layer connection structures, a branching relationship can be generated based on the situation where the same output tensor is referenced by multiple network layers, and a merging relationship can be generated based on the situation where multiple input tensors merge into the same network layer. The resulting network layer sequence reflects the connection direction, dependencies, and output propagation range of each network layer within the model.

[0075] When traversing the network layer sequence, the layer name, operator type, number of input tensors, number of output tensors, and parameter reference fields are read for each network layer to identify its type. Layer types can include convolutional layers, fully connected layers, normalized layers, activation layers, pooling layers, attention layers, feature concatenation layers, feature fusion layers, and output layers. The depth level is determined based on the network layer's position in the network layer sequence and the data flow relationship. For sequential structures, the depth level increases with the data propagation direction; for branching structures, network layers in the same propagation round after the start of the same branch can be registered as the same level; for merging structures, the depth level of the merged layer can continue to increase based on the highest-level network layer in the input source. The depth level is used to express the relative position of the network layer within the model, facilitating subsequent parameter and output structure location according to the layer range.

[0076] Parameter description data is extracted through parameter index information. After the parameter name, parameter offset address, parameter data type, and parameter array format in the parameter index information are read, a mapping is established with the corresponding network layer identifier. Weight data dimension information is used to represent the number of dimensions, channels, convolution kernel size, or matrix row and column count of the weight tensor; bias data dimension information is used to represent the bias vector length and its corresponding output channel or output node; parameter storage location information is used to represent the read location of the parameter data in the model parameter file, model package, or in-memory parameter table. Through the above parameter description data, whether each network layer contains readable parameters, the parameter size, and the parameter storage area can all be recorded in a structured manner.

[0077] When extracting sample input structure data from the training sample set, the dimensions, channel order, data type, number of input fields, and arrangement of input fields are read. After establishing a correspondence between the sample input structure data and the input layers in the network layer sequence, input layer boundary data is generated. This boundary data records the data format received by the input layer, the range of input dimensions, the order of input fields, and the tensor entry names. This ensures that the network layer structure data is consistent with the actual input data structure, avoiding inaccuracies in the input layer boundaries caused by relying solely on the default input description in the model file.

[0078] When extracting deployment resource boundary data from model deployment constraint data, the storage capacity boundary, inference overhead boundary, memory usage boundary, supported operator range, supported data precision range, and input / output response boundary are read. After establishing reference relationships between the deployment resource boundary data and the network layer sequence, deployment constraint reference data is generated. Reference relationships can be established according to the overall model dimension, or according to network layer type, layer range, or operator type. For network layers significantly affected by storage limitations, deployment constraint reference data can be associated with the storage capacity boundary; for network layers with high computational demands, deployment constraint reference data can be associated with the inference overhead boundary; for network layers requiring specific data precision support, deployment constraint reference data can be associated with the supported data precision range.

[0079] Network layer structure data is derived from a combination of layer type, depth level, parameter description data, input layer boundary data, deployment constraint reference data, and data flow relationships. During the combination process, the network layer identifier is used as an index to write the layer type, layer position, parameter dimension, parameter storage location, and deployment constraint reference information corresponding to the same network layer into the same structure record. Then, the data flow relationships connect these structure records, enabling the network layer structure data to express both the parameter state of a single network layer and the transmission direction between network layers. The generated network layer structure data can be stored using a table structure, a graph structure, or a collection of layer objects. Table structures facilitate retrieval by network layer identifier, graph structures facilitate the analysis of branching and merging relationships, and collections of layer objects facilitate reading the model structure by depth level.

[0080] This embodiment separates model definition information and parameter index information from the initial neural network model, and generates network layer sequences and data flow relationships based on the model definition information. This transforms the model structure from a general file state into a searchable hierarchical record. By traversing the network layer sequence to identify layer types and determine depth levels, the position and connection range of each network layer within the model can be clearly defined. By extracting weight data dimension information, bias data dimension information, and parameter storage location information from the parameter index information, the network layer structure can be accurately bound to the parameter data. By generating input layer boundary data from sample input structure data and deployment constraint reference data from deployment resource boundary data, the network layer structure data can simultaneously include input constraints and resource constraints. The network layer structure data generated after the above data combination can reduce the parsing bias caused by the scattered storage of model structure, parameter positions, input boundaries, and deployment constraints, and improve the accuracy of subsequent network layer localization and quantization object determination.

[0081] In one embodiment, such as Figure 4 As shown, step S20 above includes: S201, extract the layer type, parameter description data, data flow relationship and network layer identifier from the network layer structure data; S202, based on the layer type and the parameter description data, determine a set of candidate network layers containing quantifiable parameters, and based on the data flow relationship, determine the path position data of each candidate network layer in the set of candidate network layers containing quantifiable parameters in the forward propagation path; S203, based on the path location data, select candidate network layers located in the forward propagation path from the candidate network layer set containing quantizable parameters, generate a set of network layers to be quantized, and determine the network layer identifier corresponding to each network layer to be quantized in the set of network layers to be quantized based on the network layer identifier. S204, Generate parameter access description data corresponding to the set of network layers to be quantized based on the parameter description data, generate feature transfer description data corresponding to the set of network layers to be quantized based on the data flow relationship, and generate hierarchical operation description data corresponding to the set of network layers to be quantized based on the layer type. S205, the parameter access description data, the feature transfer description data, and the hierarchical operation description data are bound according to the identifier of the network layer to be quantized to generate runtime data; S206, Extract calibration sample data from the training sample set, and input the calibration sample data into the initial neural network model to generate benchmark output data; S207, Set a set of candidate quantization bit widths. For each network layer to be quantized in the set of network layers to be quantized and each candidate quantization bit width in the set of candidate quantization bit widths, determine the currently traversed network layer to be quantized as the current network layer to be quantized, determine the currently traversed candidate quantization bit width as the current candidate quantization bit width, keep the parameter states of network layers other than the current network layer to be quantized unchanged, convert the original weight data contained in the current network layer to be quantized into probe quantization weight data according to the current candidate quantization bit width, and generate a hierarchical probe model corresponding to the current network layer to be quantized and the current candidate quantization bit width based on the probe quantization weight data. S208, the calibration sample data is input into each level of the detection model to generate detection output data corresponding to each level of the detection model, and the output offset data between the detection output data and the reference output data is determined. Quantization sensitivity data is generated based on the output offset data.

[0082] In this embodiment, after reading the layer type, parameter description data, data flow relationship, and network layer identifier from the network layer structure data, each network layer is converted into a filterable structure record. The layer type is used to distinguish different layer objects such as convolutional layers, fully connected layers, attention projection layers, normalization layers, activation layers, pooling layers, splicing layers, and output layers; the parameter description data is used to determine whether the network layer contains weight data, bias data, or other parameters that can be converted by bit width; the data flow relationship is used to determine whether the network layer output is received by subsequent layers and participates in the generation of model output; the network layer identifier is used to maintain the consistency of network layer identity during the filtering, cost statistics, and sensitivity detection processes.

[0083] The set of candidate network layers containing quantifiable parameters is derived based on layer type and parameter description data. Convolutional layers, fully connected layers, attention projection layers, and parameterized feature fusion layers typically contain weight data and can be included in the candidate pool; activation layers, pooling layers, shape transformation layers, or network layers that only perform tensor transpose do not contain quantifiable weights and are not included in the candidate pool. The parameter description data records the weight dimension, bias dimension, and parameter storage location, which can further exclude network layers with missing parameters, frozen parameters, or parameters that do not participate in the inference output.

[0084] Path location data is generated from data flow relationships and is used to record the position of candidate network layers in the forward propagation path. Based on the tensor reference relationship between the network layer's input and output, it can be determined whether the candidate network layer is located on the main path, branch path, merging point, or before output. If the output of a candidate network layer is not consumed by subsequent network layers or does not participate in the generation of the final output node, it will not enter the set of network layers to be quantized. The identifier of the network layer to be quantized is inherited from the network layer identifier, enabling subsequent parameter access, feature transfer, operation description, and sensitivity detection to be bound according to the same identifier.

[0085] Parameter access description data is generated based on the parameter description data, mainly recording the size of the weight tensor, the size of the bias tensor, the parameter read position, and the number of parameter accesses for the network layer to be quantized. Feature propagation description data is generated based on the data flow relationship, mainly recording the input tensor shape, output tensor shape, tensor propagation direction, number of branch inflows, and the number of times the output is called. Layer operation description data is generated based on the layer type, mainly recording the operation forms such as convolution, matrix multiplication, normalization transformation, attention projection, or feature fusion. After the above three types of data are bound to the network layer identifier to be quantized, runtime overhead data is generated. The runtime overhead data can express the differences in resource consumption of each network layer to be quantized from three dimensions: parameter read, feature propagation, and layer operation.

[0086] Calibration sample data is extracted from the training sample set. The number and type of samples should cover the model input structure so that the baseline output data can reflect the output state of the initial neural network model before probe quantization. After the calibration sample data is input into the initial neural network model, the baseline output data is obtained. The baseline output data can be classification probabilities, regression values, feature response maps, or results with multiple output fields, used for subsequent comparison with the probe output data.

[0087] The candidate quantization bit width set is used to provide multiple bit widths to be tested. For each network layer to be quantized and each candidate quantization bit width, only the original weight data of the current network layer to be quantized is changed. The original weight data is converted into probe quantization weight data according to the current candidate quantization bit width, while the parameter states of other network layers remain unchanged. After generating a hierarchical probe model based on the probe quantization weight data, the hierarchical probe model can reflect the impact of a single network layer on the model output at a specified quantization bit width, avoiding the mixed sources of output offset caused by the simultaneous quantization of multiple network layers.

[0088] After calibration sample data is input into the detection model at each level, detection output data is generated. The detection output data is compared with the baseline output data to obtain output offset data. Output offset data can be obtained based on differences in category distribution, regression numerical differences, output vector distance, or response map offset. After the output offset data is grouped according to the identifier of the network layer to be quantized and the candidate quantization bit width, quantization sensitivity data is generated. Quantization sensitivity data can express the degree of influence of different network layers to be quantized on output stability at different bit widths.

[0089] For example, the formula for summing the operating costs of the network layers to be quantized is:

[0090] in, This represents the summary data of the runtime overhead corresponding to the set of network layers to be quantized. This represents the set of network layers to be quantized. Describes the first layer in the set of network layers to be quantized. One network layer to be quantized. Indicates the first The hierarchical operating overhead corresponding to each network layer to be quantized can be obtained from parameter access description data, feature transfer description data, and hierarchical operation description data.

[0091] This embodiment filters the network layers to be quantized by layer type, parameter description data, data flow relationship, and network layer identifier. This allows the quantization target to be controlled within the range of network layers that contain quantizable parameters and participate in the forward propagation output generation, reducing interference from parameterless network layers or invalid output paths on the selection of quantization targets. By binding parameter access description data, feature transfer description data, and hierarchical operation description data, runtime overhead data is generated, which can express the differences in network layer resource consumption from multiple dimensions such as parameter reading, tensor transfer, and hierarchical operations. By generating baseline output data using calibration sample data and generating hierarchical probing models for individual network layers to be quantized and candidate quantization bit widths, output offsets can be attributed to specific network layers and specific bit widths, obtaining quantization sensitivity data that can be used to distinguish the degree of influence of network layer quantization, thereby improving the accuracy of network layer selection and bit width allocation in subsequent mixed-precision quantization.

[0092] In one embodiment, such as Figure 5 As shown, step S30 above includes: S301, extract available storage boundary data and inference overhead boundary data from the model deployment constraint data, and obtain the identifier of each network layer to be quantized in the set of network layers to be quantized from the network layer structure data; S302, Generate layer compression priority data corresponding to the set of network layers to be quantized based on the runtime overhead data, and generate layer precision preservation priority data corresponding to the set of network layers to be quantized based on the quantization sensitivity data; S303, based on the available storage boundary data, the inference overhead boundary data, the layer compression priority data, and the layer precision preservation priority data, generate quantization bit width data for the set of network layers to be quantized; S304, extract the original weight data from the set of network layers to be quantized, determine the numerical distribution characteristics of the original weight data, and generate weight quantization mapping parameter data based on the numerical distribution characteristics and the quantization bit width data; S305, Based on the weight quantization mapping parameter data, the original weight data is converted into quantized weight data; S306, the quantization bit width data, the original weight data, the weight quantization mapping parameter data, and the quantization weight data are bound according to the identifier of the network layer to be quantized to generate weight quantization associated data; S307, the quantized weight data is written into the network layer to be quantized in the initial neural network model corresponding to the identifier of the network layer to be quantized, and the parameter states of the network layers outside the set of network layers to be quantized in the initial neural network model are kept unchanged, thereby generating a quantization candidate model.

[0093] In this embodiment, after reading the available storage boundary data and inference overhead boundary data from the model deployment constraint data, the storage usage limit, inference time limit, memory access limit, and supported bit width range are organized into comparable constraint terms. The available storage boundary data is used to control the total usage of quantized weight data, and the inference overhead boundary data is used to control the resource consumption caused by quantized network layer operations and parameter access. The identifier of the network layer to be quantized is obtained from the network layer structure data and is used to write back the subsequently generated priority, bit width, mapping parameters, and quantization weights to the corresponding network layer.

[0094] Overhead data is converted into layer-level compression priority data. Network layers with high parameter access, high feature transfer, or high computational complexity have a more significant impact on overall storage and inference resources and thus receive higher compression priority. Quantization sensitivity data is converted into layer-level precision preservation priority data. Network layers with larger output offsets are more sensitive to changes in quantization bit width and thus receive higher precision preservation priority. Compression priority drives bit width reduction, while precision preservation priority limits the magnitude of bit width reduction; both types of priorities participate in bit width allocation.

[0095] When generating quantization bit width data, available storage boundary data and inference overhead boundary data are used as overall constraints, while layer compression priority data and layer precision preservation priority data are used as the basis for layer allocation. Lower bit widths are allocated to network layers with high compression priority and low precision preservation priority, and higher bit widths are allocated to network layers with high precision preservation priority. Storage usage and inference overhead are summarized based on the bit width results of all network layers to be quantized until the bit width combination meets the resource boundaries.

[0096] After the raw weight data is read from the set of network layers to be quantized, the range of weight values, positive and negative distribution, extreme value distribution, proportion of zero values, and amplitude differences between channels are analyzed to obtain the numerical distribution characteristics. These characteristics are used to select the quantization mapping method and, combined with the quantization bit width data, generate weight quantization mapping parameter data. The weight quantization mapping parameter data may include scaling factors, zeros, truncation boundaries, grouping mapping information, or channel mapping information, used to convert continuous weight values ​​into discrete quantized weight values ​​at the target bit width.

[0097] After the original weight data is converted into quantized weight data according to the weight quantization mapping parameter data, the quantization bit width data, original weight data, weight quantization mapping parameter data, and quantized weight data are bound to the identifier of the network layer to be quantized, generating weight quantization associated data. This weight quantization associated data records the quantization source, bit width configuration, mapping relationship, and conversion result for each network layer, making the quantized parameter state traceable, verifiable, and writable. After the quantized weight data is written into the corresponding network layer to be quantized in the initial neural network model, the parameter states of network layers outside the set of network layers to be quantized remain unchanged, thus obtaining a quantization candidate model. This model only changes the weight state of the selected network layer, facilitating subsequent training and output comparison based on the same model structure.

[0098] For example, the formula for allocating the quantization bit width of the network layer to be quantized is:

[0099] in, Indicates the first The quantization bit width data corresponding to each network layer to be quantized. This represents the set of candidate quantization bit widths, which can be determined based on the range of supported quantization bit widths in the model deployment constraint data. For example, {4,8,16} can be used as an example of a set of candidate quantization bit widths. This represents the index of the network layer in the set of network layers to be quantized. This represents the set of network layers to be quantized.

[0100] The formula for hierarchical weight quantization mapping is:

[0101] in, Indicates the first The quantization weight data corresponding to each network layer to be quantized. Indicates the first The original weight data corresponding to each network layer to be quantized. Indicates the first The quantization bit width data corresponding to each network layer to be quantized. Indicates the first The weight quantization mapping parameter data corresponding to each network layer to be quantized may include scaling factors, zeros, truncation boundaries, channel mapping parameters, or grouping mapping parameters. Q represents the weight quantization mapping function, which is used to convert the original weight data into quantized weight data based on the quantization bit width data and the weight quantization mapping parameter data.

[0102] This embodiment constrains the bit width combination by using available storage boundary data and inference overhead boundary data, making the quantization result controllable by edge resource conditions. It generates hierarchical compression priority data using runtime overhead data and hierarchical precision preservation priority data using quantization sensitivity data, reducing resource consumption while preserving the precision of network layers that significantly impact the output. It generates weighted quantization mapping parameter data using numerical distribution characteristics and quantization bit width data, reducing quantization offset caused by differences in numerical ranges during weight conversion. By binding network layer identifiers, bit widths, original weights, mapping parameters, and quantization weights with weighted quantization association data, it improves the accuracy of quantization result write-back and subsequent verification, ensuring that the generated quantization candidate model possesses both compression controllability and parameter consistency.

[0103] In one embodiment, such as Figure 6 As shown, step S40 above includes: S401, extract sample input data, sample annotation data and sample identification data from the training sample set, and establish sample correspondence data between the sample input data and the sample annotation data based on the sample identification data; S402, extract the network layer identifier to be quantized, the original weight data, the quantized weight data, and the quantization bit width data from the weight quantization association data; S403, the sample input data is input into the quantization candidate model for forward propagation, and the quantization weight data is used in the forward propagation process of the network layer corresponding to the network layer identifier to be quantized in the quantization candidate model to generate prediction result data carrying sample identifier data; S404, Based on the sample corresponding data and the sample identification data, the prediction result data and the sample annotation data are aligned in terms of output fields to generate prediction annotation aligned data; S405, using a preset loss function, determine the basic loss data based on the aligned predicted values ​​and labeled values ​​in the predicted labeled alignment data; S406, Based on the identifier of the network layer to be quantized, establish weight correspondence data between the original weight data and the quantized weight data, and perform element-by-element difference measurement on the corresponding original weight values ​​and quantized weight values ​​in the weight correspondence data to generate weight offset data. S407, the weight offset data and the quantization bit width data are bound according to the identifier of the network layer to be quantized to generate layer-by-layer error constraint data, and the layer-by-layer error constraint data corresponding to each network layer to be quantized is summarized to generate error constraint data.

[0104] In this embodiment, after reading the sample input data, sample annotation data, and sample identifier data from the training sample set, the sample input data serves as the input object for quantizing the candidate model, the sample annotation data serves as the comparison object for the prediction results, and the sample identifier data is used to maintain the correspondence between the input object, the annotation object, and the output object. Sample correspondence data can be established from the sample identifier data, ensuring that sample input data and sample annotation data under the same identifier are recorded as the same training unit, avoiding mismatches between prediction results and annotation objects during batch input, out-of-order loading, or batch output.

[0105] The weighted quantization associated data retrieves the identifier of the network layer to be quantized, the original weight data, the quantized weight data, and the quantization bit width data. The identifier of the network layer to be quantized is used to locate the network layer in the quantization candidate model that has already had its quantization weight data written into it. The original weight data is used to retain the parameter reference before quantization. The quantized weight data is used to participate in the forward propagation operation, and the quantization bit width data is used to record the quantization accuracy state corresponding to each network layer. After the above data is separated, the forward propagation operation and error constraint generation can access their respective data sources.

[0106] After the sample input data is fed into the quantization candidate model, the model performs forward propagation operations according to the network layer connections. The network layer corresponding to the identifier of the network layer to be quantized calls the quantization weight data to complete convolution, matrix multiplication, feature mapping, or output transformation. Network layers not corresponding to the identifier of the network layer to be quantized participate in the operation with their parameter states preserved. The prediction result data carries sample identifier data during generation, enabling the output results to be traced back to the corresponding sample records. The prediction result data may include classification probabilities, regression values, region response data, or multiple output fields.

[0107] When aligning the output fields of the prediction results data with the sample labeled data, the prediction objects and labeled objects under the same sample identifier are determined based on the corresponding sample data. Then, the corresponding positions between the predicted values ​​and labeled values ​​are determined based on the output field names, field order, output dimensions, or task branch identifiers. The aligned prediction-label data records the paired predicted values ​​and labeled values, so that subsequent loss generation is not affected by changes in the arrangement of output fields, changes in batch order, or multi-task output structures.

[0108] The preset loss function generates basic loss data based on the aligned predicted labels. Classification outputs can generate loss values ​​based on the difference between the predicted class distribution and the class labels; regression outputs can generate loss values ​​based on the offset between the predicted and labeled values; multi-field outputs can generate multiple loss components separately and then summarize them. The basic loss data reflects the output bias of the quantified candidate model under the current sample input.

[0109] The original weight data and quantized weight data are used to establish weight mapping data according to the identifier of the network layer to be quantized. The weight mapping data needs to maintain consistency in network layer identity, tensor dimension, channel position, and element position, so that each original weight value can correspond to the quantized weight value. The element-wise difference measure can generate weight offset data according to absolute offset, squared offset, cumulative offset within the channel, or overall tensor offset. The weight offset data reflects the magnitude of parameter changes caused by the quantization transformation.

[0110] Weight offset data and quantization bit width data are bound to the identifier of the network layer to be quantized to generate layer-by-layer error constraint data. Quantization bit width data is used to distinguish the quantization accuracy state of different network layers, and weight offset data is used to express the quantization offset degree of the corresponding network layer. The layer-by-layer error constraint data of each network layer to be quantized are summarized to generate error constraint data, which can simultaneously reflect the parameter offset distribution and bit width differences of multiple network layers.

[0111] For example, the formula for quantized weights participating in forward propagation is:

[0112] in, This represents the prediction result data corresponding to the input data of the i-th sample. This represents the input data for the i-th sample. This represents the forward propagation mapping process corresponding to the quantization candidate model. This represents the set of quantization weights that participate in the forward propagation in the quantization candidate model, including the quantization weights corresponding to the identifier of the network layer to be quantized.

[0113] The formula for the basic loss of the prediction annotation is:

[0114] in, This represents the basic loss data. This represents the cross-entropy loss function, which is suitable for classification prediction output scenarios. This represents the sample labeling data corresponding to the i-th sample. This represents the prediction result data corresponding to the i-th sample.

[0115] The formula for summarizing the layer-by-layer quantization error constraints is as follows:

[0116] in, This represents the error constraint data. This represents the set of network layers to be quantized. Indicates the first The original weight data corresponding to each network layer to be quantized. Indicates the first The quantization weight data corresponding to each network layer to be quantized. Indicates the first The weight offset measure between the original weight data and the quantized weight data in a network layer to be quantized.

[0117] This embodiment establishes a correspondence between sample input data and sample labeled data through sample identification data, and enables the prediction result data to carry sample identification data, which can reduce the loss bias caused by the mismatch between the prediction result and the labeled object in batch forward propagation; by aligning the output fields to generate prediction labeled aligned data, the classification, regression or multi-field output can generate basic loss data under a unified data structure; by establishing a correspondence between the original weight data and the quantized weight data through the identifier of the network layer to be quantized, and performing element-by-element difference measurement, the parameter offset caused by quantization conversion can be transformed into error constraint data; the basic loss data and error constraint data reflect the output bias and parameter offset respectively, so that the subsequent model update can be constrained by both the prediction result and the quantized weight state, improving the output stability during the quantization training process.

[0118] In one embodiment, such as Figure 7 As shown, step S50 above includes: S501, generate loss scale alignment data based on the basic loss data and the error constraint data, and determine basic loss weight data and error constraint weight data based on the loss scale alignment data; S502, adjust the basic loss data according to the basic loss weight data to generate weighted basic loss data, and adjust the error constraint data according to the error constraint weight data to generate weighted error constraint data; S503, the weighted basic loss data and the weighted error constraint data are merged to generate the overall loss data; S504, determine trainable model parameters from the quantization candidate models, and perform backpropagation in the quantization candidate models based on the overall loss data to generate parameter update data corresponding to the trainable model parameters; S505, Update the trainable model parameters based on the parameter update data to generate a parameter update model; S506, extract the network layer identifier to be quantized, quantization bit width data, and weight quantization mapping parameter data from the weight quantization association data, and based on the quantization bit width data and the weight quantization mapping parameter data, convert the updated trainable model parameters corresponding to the network layer identifier to be quantized in the parameter update model into training round quantization weight data. S507, Based on the training round quantized weight data, replace the quantized weight data of the network layer corresponding to the network layer identifier to be quantized in the parameter update model to generate the replaced parameter update model, and based on the training round quantized weight data, replace the quantized weight data in the weight quantization association data to generate the training round weight quantization association data. S508, determine whether the parameter update model after replacement meets the preset training termination condition; S509, when the preset training termination condition is met, the replaced parameter update model is determined as the quantization training model; S510, when the preset training termination condition is not met, the replaced parameter update model is used as the quantization candidate model for the next training round. Forward propagation is performed again based on the training sample set and the quantization candidate model for the next training round to regenerate prediction result data and basic loss data. Error constraint data is regenerated based on the weight quantization correlation data of the training round. Overall loss data is regenerated based on the regenerated basic loss data and the regenerated error constraint data. The trainable model parameters of the quantization candidate model for the next training round are updated based on the regenerated overall loss data until the preset training termination condition is met.

[0119] In this embodiment, the base loss data and error constraint data come from different sources, and their numerical scales, ranges of variation, and gradient effects may differ. The base loss data reflects the output deviation between the predicted and labeled values, while the error constraint data reflects the parameter offset between the quantized weights and the original weights. When generating loss scale-aligned data, the numerical range, mean, variance, gradient magnitude, or intra-batch fluctuation range of both types of loss data can be read, and the two types of loss data can be converted into a scale state that can participate in fusion together. The base loss weight data and error constraint weight data are determined based on the loss scale-aligned data, ensuring that the predicted output deviation and weight quantization offset have a controllable impact ratio during training.

[0120] The basic loss data is weighted according to the basic loss weights to obtain weighted basic loss data; the error constraint data is weighted according to the error constraint weights to obtain weighted error constraint data. The two types of weighted loss data are then fused to generate the overall loss data. Fusion can employ linear weighting, staged weighting, or gradient magnitude balancing. Linear weighting is suitable for training processes where the output task is stable; staged weighting is suitable for training processes that emphasize prediction output in the early stages and strengthen quantization constraints in the later stages; and gradient magnitude balancing is suitable for training processes where the gradients of the two types of losses differ significantly. The overall loss data simultaneously retains both prediction output constraints and quantization error constraints, preventing the trainable model parameters from simply fitting to the sample labels while ignoring the quantization weight shift.

[0121] The trainable model parameters in the quantization candidate model can include convolutional layer weights, fully connected layer weights, attention projection weights, bias parameters, normalization parameters, and output layer parameters. For network layers participating in quantization, there is a correspondence between the trainable model parameters and the quantization weight data; for network layers not participating in quantization, the trainable model parameters are updated according to their preserved parameter states. During backpropagation based on the overall loss data, the quantization candidate model backpropagates gradients according to the computational graph formed during forward propagation, generating parameter update data corresponding to the trainable model parameters. The parameter update data can include gradient values, update directions, update step sizes, and parameter momentum states, and is used to update the trainable model parameters to obtain the parameter-updated model.

[0122] The updated trainable model parameters corresponding to the identifier of the network layer to be quantized in the parameter update model need to be further converted into training round quantized weight data according to the quantization bit width data and weight quantization mapping parameter data. This conversion ensures that the trained parameter state falls back into the corresponding bit width representation range, preventing the parameters updated after backpropagation from deviating from the quantization constraints of the quantization candidate model. The weight quantization mapping parameter data provides scaling factors, zeros, truncation boundaries, or grouping mapping information, while the quantization bit width data provides the discrete representation range. Together, they control the conversion result from the updated trainable model parameters to the training round quantized weight data.

[0123] After replacing the quantized weights of the corresponding network layer in the training round with the quantized weights in the updated model, a new model with replaced parameters is generated. Simultaneously, the quantized weights in the weight quantization association data are replaced with the quantized weights from the training round, resulting in training round weight quantization association data. This ensures consistency between the model's parameter state and the association records. If only the weights within the model are replaced without updating the association records, subsequent error constraint data will be generated based on outdated quantized weights, leading to inconsistencies between parameter offset statistics and the current model state.

[0124] The preset training termination conditions are used to determine whether the replaced parameter update model has completed training. Termination conditions may include convergence of the overall loss data, the basic loss data falling below a loss threshold, the error constraint data falling below a bias threshold, the output change in consecutive epochs falling below a change threshold, or the training epochs reaching the upper limit. When the preset training termination conditions are met, the replaced parameter update model is determined as the quantized training model. If not met, the replaced parameter update model serves as the quantization candidate model for the next training epoch, and the quantized correlation data of the training epoch weights serves as the quantization record source for the next training epoch. Forward propagation, basic loss generation, error constraint generation, overall loss generation, and parameter updates are then performed again until the termination conditions are met.

[0125] The quantization training model preserves the network layer connections of the quantization candidate model. The network layer to be quantized uses the quantized weight data updated after training rounds, while the network layers not involved in quantization retain their corresponding parameter states. During training, input data passes through the input layer to the feature extraction layer, feature fusion layer, and output layer. The quantized weights generate prediction results after participating in forward propagation. The overall loss data then affects the trainable model parameters through backpropagation, and the quantized weight data from each training round is written back to the network layer to be quantized, ensuring that the model output fit and quantized parameter constraints are continuously updated within the same training process.

[0126] For example, the weighted fusion formula for total loss is:

[0127] in, This represents the total loss data. This represents the basic loss data. This represents the error constraint data. This represents the basic loss weight data. This represents the error constraint weight data.

[0128] This embodiment determines the basic loss weight data and error constraint weight data by using loss scale alignment data, which can reduce the interference caused by the difference in the numerical range of the two types of loss data on the training direction. By fusing the weighted basic loss data and weighted error constraint data to generate overall loss data, the trainable model parameters can be simultaneously constrained by prediction output bias and weight quantization offset. By generating parameter update data through backpropagation and re-converting the quantization weight data of the training round, the parameter update results can continuously meet the requirements of quantization bit width and weight quantization mapping parameters. By synchronously replacing the quantization weight data in the model and the quantization weight data in the weight quantization association data, the current parameter state of the model can be kept consistent with the source of error constraints. By controlling the repeated training process through preset training termination conditions, a quantized training model that takes into account both output stability and quantization consistency can be generated.

[0129] In one embodiment, such as Figure 8 As shown, step S60 above includes: S601, extract initial parameter storage data from the initial neural network model, and extract training parameter storage data from the quantization training model; S602, extract the network layer identifier to be quantized, quantization bit width data, weight quantization mapping parameter data and quantization weight data from the weight quantization association data; S603, based on the identifier of the network layer to be quantized, extract the initial parameter entries and training parameter entries corresponding to each network layer to be quantized from the initial parameter storage data and the training parameter storage data, respectively, and perform hierarchical alignment of the initial parameter entries, the training parameter entries, the quantization bit width data and quantization weight data of each network layer to be quantized to generate hierarchical compression comparison data; S604, Generate model storage compressed data based on the hierarchical compressed comparison data, Generate inference overhead analysis data based on the quantization training model and the quantization bit width data, and Generate quantization integrity analysis data based on the weighted quantization mapping parameter data and the quantization training model; S605, generate compressed analysis data based on the model storage compressed data, the inference overhead analysis data, and the quantization integrity analysis data; S606, extract available storage boundary data, inference overhead boundary data, and model structure boundary data from the model deployment constraint data, and generate deployment satisfaction markers based on the compressed analysis data, the available storage boundary data, the inference overhead boundary data, and the model structure boundary data; S607, when the deployment satisfies the flag indicating that the compressed analysis data satisfies the model deployment constraint data, inference parameter configuration data is generated based on the quantization training model, the quantization bit width data and the weight quantization mapping parameter data; S608, Remove training auxiliary state data from the quantization training model, generate inference model structure data, and combine the inference parameter configuration data and the inference model structure data to generate a compressed neural network model.

[0130] In this embodiment, the initial parameter storage data is read from the initial neural network model to obtain the parameter entries, parameter data types, parameter dimensions, parameter usage, and parameter storage locations of each network layer before compression. The training parameter storage data is read from the quantized training model to obtain the parameter entries, quantized parameter forms, parameter usage, and derivable states of each network layer after training. Both types of parameter storage data are retrieved using network layer identifiers, enabling comparison of the parameter states before compression and after training within the same network layer.

[0131] The weight quantization associated data retrieves the identifier of the network layer to be quantized, the quantization bit width data, the weight quantization mapping parameter data, and the quantization weight data. The identifier of the network layer to be quantized is used to locate the network layer to be quantized; the quantization bit width data records the bit width state used by each network layer; the weight quantization mapping parameter data records the numerical mapping relationship between the original weights and the quantized weights; and the quantization weight data records the discrete parameter results that can participate in inference after training. These data, together with the initial parameter storage data and the training parameter storage data, can simultaneously check the parameter compression ratio, the quantization weight writing status, and whether the parameters required for inference are complete.

[0132] The hierarchical compression comparison data is generated using the identifier of the network layer to be quantized as an index. For each network layer to be quantized, initial parameter entries are extracted from the initial parameter storage data, and training parameter entries are extracted from the training parameter storage data. The initial parameter entries, training parameter entries, quantization bit width data, and quantization weight data are then placed into the same hierarchical record. The hierarchical record may include the parameter occupancy before compression, the parameter occupancy after training, bit width type, number of quantization weights, mapping parameter status, and network layer derived status. This record reflects the parameter changes of a single network layer before and after compression.

[0133] Model storage compressed data is obtained based on hierarchical compression comparison data. The parameter usage before compression and after training for each network layer to be quantized is summarized to generate overall parameter usage changes, hierarchical compression ratios, and quantization weight usage. Inference overhead analysis data is obtained based on the quantization training model and quantization bit width data. It can be combined with the network layer input / output tensor shape, parameter access volume, quantization bit width, and hierarchical operation type to record the resource consumption of the compressed model during the inference phase. Quantization integrity analysis data is obtained based on weight quantization mapping parameter data and the quantization training model. It is used to verify whether each network layer to be quantized has complete quantization bit width, mapping parameters, and quantization weight writing results.

[0134] The compressed analysis data is generated by combining model storage compressed data, inference overhead analysis data, and quantization integrity analysis data. During the combination, the storage compressed data provides a basis for judging model size and parameter usage, the inference overhead analysis data provides a basis for judging inference resources, and the quantization integrity analysis data provides a basis for checking model structure and parameter writes. The compressed analysis data not only records the compression ratio but also whether the compressed model possesses complete inference capabilities.

[0135] The model deployment constraint data includes available storage boundary data, inference overhead boundary data, and model structure boundary data. Available storage boundary data is compared with compressed model storage data, inference overhead boundary data is compared with inference overhead analysis data, and model structure boundary data is compared with quantization integrity analysis data. The comparison results are summarized to generate deployment satisfaction markers. These markers record the storage boundary state, inference overhead boundary state, structural integrity state, and overall satisfaction state, facilitating the determination of whether the quantized training model can proceed to the compressed model generation stage.

[0136] When deploying compressed analysis data that satisfies model deployment constraints, inference parameter configuration data is generated based on the quantized training model, quantization bit width data, and weight quantization mapping parameter data. This inference parameter configuration data can include network layer quantization bit width, weight quantization mapping parameters, quantization weight read addresses, input / output data precision, and parameter read methods during the inference phase. After removing training auxiliary state data from the quantized training model, inference model structure data is generated. This training auxiliary state data can include gradient cache, optimizer state, training epoch records, loss records, and intermediate correction states. The inference parameter configuration data and the inference model structure data are combined to generate a compressed neural network model, ensuring the model retains the structure and quantization parameters required for inference while removing data only needed during the training phase.

[0137] For example, the formula for the storage usage of the compressed model is:

[0138] in, This represents the storage footprint of the compressed model. L represents the set of network layers in the compressed neural network model. Indicates the first The number of parameters corresponding to each network layer. Indicates the first The actual storage bit width of each network layer in the compressed model; for the network layer to be quantized. The allocated quantization bit width; for network layers that do not participate in quantization, The storage bit width corresponding to the reserved parameter state can be specified.

[0139] The available storage boundary satisfies the following formula:

[0140] in, This indicates the storage space occupied by the compressed model. This represents the available storage boundary data in the model deployment constraint data.

[0141] This embodiment compares the initial parameter storage data and training parameter storage data hierarchically to accurately reflect the changes in parameter usage before and after compression of each network layer to be quantized. By extracting the quantization bit width, mapping parameters, and quantization weights from the weight quantization correlation data, it can verify whether the quantization parameters in the compressed model are completely written. By generating compressed analysis data from the model storage compressed data, inference overhead analysis data, and quantization integrity analysis data, it can simultaneously determine the model volume, inference resources, and structural integrity. By generating deployment satisfaction markers from the compressed analysis data and model deployment constraint data, models that do not meet resource or structural boundaries can be excluded before generating the compressed neural network model. By removing training auxiliary state data and combining inference parameter configuration data and inference model structure data, a compressed neural network model oriented towards the inference stage can be obtained, improving the resource matching degree and structural availability before model deployment.

[0142] In one embodiment, the insurance claims platform deploys a vehicle damage recognition model for mobile surveying, rapid vehicle insurance damage assessment, and edge-assisted review scenarios. The platform first acquires a pre-trained initial neural network model, a set of vehicle insurance surveying training samples, and model deployment constraint data. The initial neural network model is used to identify vehicle parts, damage areas, damage severity, and candidate repair items. The training sample set includes survey images, partial vehicle images, damage assessment labels, repair item labels, and damage severity labels. The model deployment constraint data comes from mobile surveying terminals, in-vehicle devices, or claims adjuster handheld devices and includes available storage boundaries, inference overhead boundaries, power consumption boundaries, supported quantization bit width range, and model structure boundaries. The platform reads model definition information and parameter index information to determine the sequence of network layers such as input layers, convolutional layers, feature fusion layers, and output layers, as well as the data flow relationships between each network layer. It then identifies the layer type and depth level of each network layer and extracts the weight data dimension, bias data dimension, and parameter storage location. The platform also extracts sample input structure data from the training sample set, establishes a correspondence between the survey image size, channel order, and input fields and the model input layer, generates input layer boundary data, and establishes a reference relationship between the deployment resource boundaries such as terminal storage, inference time, and supported bit width and the network layer sequence, generating deployment constraint reference data. After combining layer type, depth level, parameter description data, input layer boundary data, deployment constraint reference data, and data flow relationships, network layer structure data that can describe the internal hierarchical structure, parameter distribution, and edge deployment boundaries of the vehicle damage recognition model is formed.

[0143] The platform further determines the set of network layers to be quantized based on the network layer structure data. From the network layer structure data, the platform extracts layer type, parameter description data, data flow relationships, and network layer identifiers. It filters convolutional layers, fully connected layers, attention projection layers, and feature fusion layers containing quantifiable parameters, excluding network layers that only perform activation, pooling, or size transformation. For the selected candidate network layers, the platform determines the path position of each candidate network layer in the forward propagation path based on the data flow relationships, retaining the network layers that participate in vehicle damage prediction output, generating the set of network layers to be quantized, and assigning a corresponding identifier to each network layer. The platform generates parameter access description data based on the parameter description data, such as the size of each layer's weight tensor, parameter read location, and parameter access count; it generates feature transfer description data based on the data flow relationships, such as the transfer direction of vehicle body region features, damage texture features, and fused features between network layers; and it generates hierarchical operation description data based on the layer type, such as convolution operations, matrix multiplication, and feature fusion operations. The platform binds parameter access description data, feature transfer description data, and hierarchical operation description data according to the identifier of the network layer to be quantized, and generates runtime overhead data to identify computational bottlenecks and network layers with large storage consumption in the vehicle damage recognition model.

[0144] To obtain quantization sensitivity data, the platform extracts calibration sample data from the training sample set. This calibration sample data covers different vehicle parts, such as the front, rear, doors, bumpers, and fenders, and covers different damage states, including minor scratches, dents, breaks, and component replacements. The platform inputs the calibration sample data into the initial neural network model to generate baseline output data. The platform sets a set of candidate quantization bit widths, such as low, medium, and high bit widths, and performs hierarchical probing for each network layer to be quantized and each candidate quantization bit width. During each probing, only the original weight data of the current network layer to be quantized is converted into probe quantization weight data, while the parameters of other network layers remain unchanged, thus generating the corresponding hierarchical probing model. The platform inputs the same calibration sample data into each hierarchical probing model to generate probe output data and compares the degree of deviation between the probe output data and the baseline output data. If quantizing a certain network layer results in a significant shift in the boundary of the damage area, a significant change in the probability of damage classification, or a large change in the candidate results of repair items, then that network layer has high quantization sensitivity; if the output change is small, then that network layer can withstand stronger compression. The platform generates quantization sensitivity data based on the output offset data, providing a basis for subsequent mixed precision allocation.

[0145] In mixed-precision quantization, the platform extracts available storage boundary data and inference overhead boundary data from model deployment constraint data, and obtains the identifiers of each network layer to be quantized from the network layer structure data. Based on runtime overhead data, the platform generates hierarchical compression priority data, increasing the compression priority for network layers with high parameter access, high feature transfer, or high hierarchical computation. Based on quantization sensitivity data, the platform generates hierarchical precision preservation priority data, increasing the precision preservation priority for network layers that significantly affect damage boundaries, vehicle part identification, and damage assessment output. The platform integrates available storage boundary data, inference overhead boundary data, hierarchical compression priority data, and hierarchical precision preservation priority data to generate quantization bit width data for the set of network layers to be quantized. Network layers with high runtime overhead and low quantization sensitivity can be deeply compressed using lower bit widths, while key layers with high quantization sensitivity retain higher bit widths, avoiding the loss of key feature representation caused by uniform low-bit quantization.

[0146] The platform extracts raw weight data from the set of network layers to be quantized, determines the numerical distribution characteristics of the raw weight data, such as weight value range, extreme value distribution, zero value ratio, and inter-channel amplitude differences, and generates weight quantization mapping parameter data based on the numerical distribution characteristics and quantization bit width data. The weight quantization mapping parameter data may include scaling factors, zeros, truncation boundaries, channel mapping parameters, or grouping mapping parameters. Based on the weight quantization mapping parameter data, the platform converts the raw weight data into quantized weight data, and then binds the quantization bit width data, raw weight data, weight quantization mapping parameter data, and quantized weight data according to the identifier of the network layer to be quantized, generating weight quantization associated data. Subsequently, the platform writes the quantized weight data into the corresponding network layer to be quantized in the initial neural network model. The parameter states of network layers not included in the set of network layers to be quantized remain unchanged, generating a quantization candidate model. This quantization candidate model has formed a mixed-precision structure, capable of simulating the low-precision parameter state during edge deployment.

[0147] The platform utilizes a training sample set and a quantized candidate model for forward propagation. The training sample set extracts sample input data, sample annotation data, and sample identification data. Sample input data can be vehicle insurance inspection images, magnified images of vehicle parts, or multi-angle damage images. Sample annotation data can include damage location, damage extent, repair method, and damage assessment label. Based on the sample identification data, the platform establishes sample correspondence data between the sample input data and sample annotation data, and extracts the network layer identifier to be quantized, original weight data, quantized weight data, and quantization bit width data from the weighted quantization correlation data. After the sample input data is input into the quantized candidate model, the network layer corresponding to the network layer identifier to be quantized participates in forward propagation using the quantized weight data, generating prediction result data carrying the sample identification data. The prediction result data and sample annotation data are aligned according to the sample correspondence data and output fields to generate prediction-annotation-aligned data. The platform uses a preset loss function to determine basic loss data based on the aligned prediction values ​​and annotation values, reflecting the prediction bias of the quantized candidate model in damage identification, repair item prediction, and damage assessment result output.

[0148] The platform also generates error constraint data based on weighted quantization correlation data. The platform establishes a weight correspondence between the original weight data and the quantized weight data according to the identifier of the network layer to be quantized. It then measures the element-wise difference between the corresponding original weight values ​​and quantized weight values ​​to generate weight offset data. The platform binds the weight offset data and quantization bit width data according to the identifier of the network layer to be quantized, generating layer-by-layer error constraint data. Finally, it summarizes the layer-by-layer error constraint data corresponding to each network layer to be quantized, generating the final error constraint data. This error constraint data is used to express the weight offset caused by quantization function modeling, ensuring that the training process focuses not only on the loss prediction result but also on the parameter difference before and after quantization.

[0149] The platform generates overall loss data based on basic loss data and error constraint data. Since the numerical ranges and gradient effects of the basic loss data and error constraint data may differ, the platform first generates loss scale alignment data and then determines the basic loss weight data and error constraint weight data based on this data. The basic loss data is adjusted according to the basic loss weight data to generate weighted basic loss data, and the error constraint data is adjusted according to the error constraint weight data to generate weighted error constraint data. The two types of weighted loss data are then fused to generate the overall loss data. This training process is equivalent to simultaneously introducing task loss and quantization error constraints in quantization-aware training, enabling the model to adapt to low-precision representations during the training phase and reducing accuracy loss during the inference phase.

[0150] The platform determines trainable model parameters from candidate quantization models and performs backpropagation within these models based on the overall loss data, generating parameter update data corresponding to the trainable model parameters. The platform updates the trainable model parameters based on this update data, generating a parameter-updated model. Subsequently, the platform extracts the identifier of the network layer to be quantized, quantization bit width data, and weight quantization mapping parameter data from the weight quantization association data. Based on the quantization bit width data and weight quantization mapping parameter data, it converts the updated trainable model parameters corresponding to the identifier of the network layer to be quantized in the parameter-updated model into training round quantization weight data. The training round quantization weight data replaces the quantization weight data of the corresponding network layer in the parameter-updated model, generating a replaced parameter-updated model. Simultaneously, it replaces the quantization weight data in the weight quantization association data, generating training round weight quantization association data. The platform determines whether the replaced parameter-updated model meets preset training termination conditions, such as overall loss convergence, prediction result fluctuation below a threshold, quantization error constraint stability, or the training rounds reaching their limit. When the conditions are met, the replaced parameter update model is determined as the quantization training model; when the conditions are not met, the replaced parameter update model is used as the quantization candidate model for the next training round, and forward propagation, loss generation, error constraint generation, overall loss generation and parameter update are continued until the training termination conditions are met.

[0151] After quantization training is completed, the platform generates compressed analysis data based on the initial neural network model, weight quantization correlation data, and the quantization training model. The platform extracts initial parameter storage data from the initial neural network model, training parameter storage data from the quantization training model, and the identifier of the network layer to be quantized, quantization bit width data, weight quantization mapping parameter data, and quantization weight data from the weight quantization correlation data. Based on the identifier of the network layer to be quantized, the platform extracts the corresponding initial parameter entries and training parameter entries for each network layer from the initial parameter storage data and training parameter storage data, and aligns the initial parameter entries, training parameter entries, quantization bit width data, and quantization weight data hierarchically to generate hierarchical compressed comparison data. The platform generates model storage compressed data based on the hierarchical compressed comparison data, inference overhead analysis data based on the quantization training model and quantization bit width data, and quantization integrity analysis data based on the weight quantization mapping parameter data and the quantization training model. The model storage compressed data reflects the model compression ratio, the inference overhead analysis data reflects changes in edge inference speed and memory usage, and the quantization integrity analysis data verifies whether the quantization bit width, quantization mapping parameters, and quantization weights have been completely written into the model.

[0152] The platform generates compressed analysis data based on model storage compressed data, inference overhead analysis data, and quantization integrity analysis data. It also extracts available storage boundary data, inference overhead boundary data, and model structure boundary data from model deployment constraint data. The platform compares the compressed analysis data with these boundary data to generate deployment compliance markers. If a compliance marker indicates that the compressed analysis data meets the model deployment constraints, the platform generates inference parameter configuration data based on the quantized training model, quantization bit width data, and weight quantization mapping parameter data. It then removes training auxiliary state data from the quantized training model, such as gradient cache, optimizer state, training epoch records, and loss records, generating inference model structure data. The inference parameter configuration data and inference model structure data are combined to generate a compressed neural network model. This compressed neural network model can be deployed to mobile survey terminals, in-vehicle devices, or edge-side claims workstations, enabling rapid completion of vehicle damage identification, repair item candidate selection, and damage assessment amount prediction at the edge.

[0153] In practical operation, the platform can compare the original floating-point model, the unified low-bit quantization model, the mixed-precision unquantized perception training model, and the mixed-precision quantized perception training model. The original floating-point model maintains good prediction accuracy, but its model size and memory consumption are large, making it difficult to meet the real-time inference requirements of the edge. The unified low-bit quantization model has a high compression ratio, but the accuracy of key layers decreases significantly, which can easily cause deviations in damage boundary recognition. The mixed-precision unquantized perception training model can reduce some accuracy loss by using different quantization bit widths in different layers, but it does not fully simulate quantization errors during the training phase. The mixed-precision quantized perception training model maintains high prediction accuracy while reducing storage consumption and inference overhead through mixed-precision quantization, quantization function modeling, participation of quantization parameters in forward propagation, and joint training of basic loss and quantization error constraints. For mobile claims and rapid damage assessment scenarios in auto insurance, this compressed neural network model can reduce reliance on cloud computing, improve edge response speed, and maintain stable output in damage image recognition, repair plan recommendation, and damage assessment result auxiliary review.

[0154] In one embodiment, a target prediction device based on mixed-precision quantization is provided, which corresponds one-to-one with the target prediction method based on mixed-precision quantization described in the above embodiments. (Refer to...) Figure 9 , Figure 9 This is a schematic diagram of the functional modules of a preferred embodiment of the target prediction device based on mixed-precision quantization of the present invention. The target prediction device based on mixed-precision quantization includes: a hierarchical structure analysis module 10, a layer analysis module 20, a mixed-precision quantization module 30, a training loss generation module 40, a quantization training update module 50, a compressed model generation module 60, and a target prediction module 70. Detailed descriptions of each functional module are as follows: The hierarchical structure parsing module 10 is used to obtain the initial neural network model, training sample set and model deployment constraint data, and to perform hierarchical structure parsing on the initial neural network model to generate network layer structure data. The quantization layer analysis module 20 is used to determine the set of network layers to be quantized based on the network layer structure data, and to obtain the running overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized. The mixed-precision quantization module 30 is used to perform mixed-precision quantization processing on the set of network layers to be quantized based on the model deployment constraint data, the runtime overhead data and the quantization sensitivity data, to generate weighted quantization correlation data and quantization candidate models. The training loss generation module 40 is used to perform forward propagation based on the training sample set and the quantized candidate model to generate prediction result data, generate basic loss data based on the prediction result data and the sample labeling data in the training sample set, and generate error constraint data based on the weight quantization correlation data. The quantization training update module 50 is used to generate overall loss data based on the basic loss data and the error constraint data, and update the trainable model parameters of the quantization candidate model based on the overall loss data to generate a quantization training model. The compressed model generation module 60 is used to generate compressed analysis data based on the initial neural network model, the weight quantization correlation data, and the quantization training model, and to generate a compressed neural network model based on the quantization training model under the condition that the compressed analysis data satisfies the model deployment constraint data. The target prediction module 70 is used to acquire target input data, input the target input data into the compressed neural network model for prediction, and generate target prediction results.

[0155] In one embodiment, the hierarchical structure parsing module 10 is specifically used for: The process involves acquiring an initial neural network model, a training sample set, and model deployment constraint data; extracting model definition information and parameter index information from the initial neural network model; and determining the sequence of network layers and the data flow relationship between each network layer based on the model definition information. Traverse the network layer sequence, identify the layer type of each network layer, and determine the depth level of each network layer according to its position in the network layer sequence; Based on the parameter index information, extract the parameter description data of each network layer; Extract sample input structure data from the training sample set, and establish a correspondence between the sample input structure data and the input layer in the network layer sequence to generate input layer boundary data; Deployment resource boundary data is extracted from the model deployment constraint data, and a reference relationship is established between the deployment resource boundary data and the network layer sequence to generate deployment constraint reference data; The network layer structure data is generated by combining the layer type, depth level, parameter description data, input layer boundary data, and deployment constraint reference data of each network layer with the data flow relationship.

[0156] In one embodiment, the layer analysis module 20 to be quantized is specifically used for: Extract layer type, parameter description data, data flow relationship, and network layer identifier from the network layer structure data; Based on the layer type and the parameter description data, a set of candidate network layers containing quantifiable parameters is determined, and based on the data flow relationship, the path position data of each candidate network layer in the set of candidate network layers containing quantifiable parameters in the forward propagation path is determined. Based on the path location data, candidate network layers located in the forward propagation path are selected from the candidate network layer set containing quantizable parameters to generate a set of network layers to be quantized, and the network layer identifier corresponding to each network layer in the set of network layers to be quantized is determined based on the network layer identifier. Based on the parameter description data, parameter access description data corresponding to the set of network layers to be quantized is generated; based on the data flow relationship, feature transfer description data corresponding to the set of network layers to be quantized is generated; and based on the layer type, hierarchical operation description data corresponding to the set of network layers to be quantized is generated. The parameter access description data, the feature transfer description data, and the hierarchical operation description data are bound according to the identifier of the network layer to be quantized to generate runtime overhead data; Calibration sample data is extracted from the training sample set and input into the initial neural network model to generate benchmark output data; A candidate quantization bit width set is defined. For each network layer to be quantized in the set of network layers to be quantized and each candidate quantization bit width in the candidate quantization bit width set, the network layer to be quantized currently being traversed is determined as the current network layer to be quantized, and the candidate quantization bit width to be traversed currently is determined as the current candidate quantization bit width. The parameter states of the network layers other than the current network layer to be quantized remain unchanged. The original weight data contained in the current network layer to be quantized is converted into probe quantization weight data according to the current candidate quantization bit width. Based on the probe quantization weight data, a hierarchical probe model corresponding to the current network layer to be quantized and the current candidate quantization bit width is generated. The calibration sample data is input into each level of the detection model to generate detection output data corresponding to each level of the detection model, and the output offset data between the detection output data and the reference output data is determined. Quantization sensitivity data is generated based on the output offset data.

[0157] In one embodiment, the mixed precision quantization module 30 is specifically used for: The available storage boundary data and inference overhead boundary data are extracted from the model deployment constraint data, and the identifier of each network layer to be quantized in the set of network layers to be quantized is obtained from the network layer structure data. Based on the runtime overhead data, generate layer compression priority data corresponding to the set of network layers to be quantized, and generate layer precision preservation priority data corresponding to the set of network layers to be quantized based on the quantization sensitivity data. Based on the available storage boundary data, the inference overhead boundary data, the layer compression priority data, and the layer precision preservation priority data, quantization bit width data is generated for the set of network layers to be quantized. Extract the original weight data from the set of network layers to be quantized, determine the numerical distribution characteristics of the original weight data, and generate weight quantization mapping parameter data based on the numerical distribution characteristics and the quantization bit width data; Based on the weight quantization mapping parameter data, the original weight data is converted into quantized weight data; The quantization bit width data, the original weight data, the weight quantization mapping parameter data, and the quantization weight data are bound according to the identifier of the network layer to be quantized to generate weight quantization associated data; The quantized weight data is written into the network layer to be quantized in the initial neural network model corresponding to the network layer identifier to be quantized, and the parameter states of the network layers outside the set of network layers to be quantized in the initial neural network model are kept unchanged to generate a quantization candidate model.

[0158] In one embodiment, the training loss generation module 40 is specifically used for: Extract sample input data, sample annotation data, and sample identification data from the training sample set, and establish sample correspondence data between the sample input data and the sample annotation data based on the sample identification data; Extract the network layer identifier to be quantized, the original weight data, the quantized weight data, and the quantization bit width data from the weight quantization correlation data; The sample input data is input into the quantization candidate model for forward propagation, and the network layer corresponding to the network layer identifier to be quantized in the quantization candidate model uses the quantization weight data to participate in the forward propagation process to generate prediction result data carrying sample identifier data. Based on the sample corresponding data and the sample identification data, the prediction result data and the sample annotation data are aligned in terms of output fields to generate prediction annotation aligned data; Using a preset loss function, the basic loss data is determined based on the aligned predicted values ​​and labeled values ​​in the predicted labeled alignment data; Based on the identifier of the network layer to be quantized, establish weight correspondence data between the original weight data and the quantized weight data, and perform element-by-element difference measurement on the corresponding original weight values ​​and quantized weight values ​​in the weight correspondence data to generate weight offset data. The weight offset data and the quantization bit width data are bound according to the identifier of the network layer to be quantized to generate layer-by-layer error constraint data, and the layer-by-layer error constraint data corresponding to each network layer to be quantized are summarized to generate error constraint data.

[0159] In one embodiment, the quantization training update module 50 is specifically used for: Based on the basic loss data and the error constraint data, loss scale alignment data is generated, and based on the loss scale alignment data, basic loss weight data and error constraint weight data are determined. Adjust the basic loss data according to the basic loss weight data to generate weighted basic loss data, and adjust the error constraint data according to the error constraint weight data to generate weighted error constraint data; The weighted basic loss data and the weighted error constraint data are combined to generate overall loss data; Trainable model parameters are determined from the quantization candidate models, and backpropagation is performed on the quantization candidate models based on the overall loss data to generate parameter update data corresponding to the trainable model parameters; The trainable model parameters are updated based on the parameter update data to generate a parameter update model; Extract the network layer identifier to be quantized, quantization bit width data, and weight quantization mapping parameter data from the weight quantization association data, and based on the quantization bit width data and the weight quantization mapping parameter data, convert the updated trainable model parameters corresponding to the network layer identifier to be quantized in the parameter update model into training round quantization weight data. The quantized weight data of the network layer corresponding to the network layer identifier to be quantized is replaced in the parameter update model based on the quantized weight data of the training round, and the quantized weight data in the weight quantization association data is replaced in the weight quantization association data based on the quantized weight data of the training round, and the weight quantization association data of the training round is generated. Determine whether the parameter update model after the replacement meets the preset training termination condition; When the preset training termination condition is met, the replaced parameter update model is determined as the quantization training model; When the preset training termination condition is not met, the replaced parameter-updated model is used as the quantization candidate model for the next training round. Forward propagation is performed again based on the training sample set and the quantization candidate model for the next training round. Prediction result data and basic loss data are regenerated. Error constraint data is regenerated based on the weight quantization correlation data of the training round. Overall loss data is regenerated based on the regenerated basic loss data and the regenerated error constraint data. The trainable model parameters of the quantization candidate model for the next training round are updated based on the regenerated overall loss data until the preset training termination condition is met.

[0160] In one embodiment, the compressed model generation module 60 is specifically used for: Initial parameter data is extracted and stored from the initial neural network model, and training parameter data is extracted and stored from the quantized training model; Extract the network layer identifier to be quantized, quantization bit width data, weight quantization mapping parameter data, and quantization weight data from the weight quantization association data; Based on the identifier of the network layer to be quantized, the initial parameter entries and training parameter entries corresponding to each network layer to be quantized are extracted from the initial parameter storage data and the training parameter storage data, respectively. The initial parameter entries, the training parameter entries, the quantization bit width data and quantization weight data of each network layer to be quantized are then hierarchically aligned to generate hierarchical compression comparison data. Based on the hierarchical compression comparison data, a model is generated to store compressed data; based on the quantization training model and the quantization bit width data, inference overhead analysis data is generated; and based on the weighted quantization mapping parameter data and the quantization training model, quantization integrity analysis data is generated. Compressed analysis data is generated based on the model storage compressed data, the inference overhead analysis data, and the quantization integrity analysis data; Available storage boundary data, inference overhead boundary data, and model structure boundary data are extracted from the model deployment constraint data, and deployment satisfaction markers are generated based on the compressed analysis data, available storage boundary data, inference overhead boundary data, and model structure boundary data. When the deployment satisfies the flag indicating that the compressed analysis data satisfies the model deployment constraint data, inference parameter configuration data is generated based on the quantized training model, the quantized bit width data, and the weight quantization mapping parameter data. The training auxiliary state data is removed from the quantized training model to generate inference model structure data. The inference parameter configuration data and the inference model structure data are then combined to generate a compressed neural network model.

[0161] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements server-side functions or steps of a target prediction method based on mixed-precision quantization.

[0162] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of a target prediction method based on mixed-precision quantization.

[0163] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the initial neural network model, training sample set, and model deployment constraint data; perform hierarchical structure analysis on the initial neural network model to generate network layer structure data. Based on the network layer structure data, determine the set of network layers to be quantized, and obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized; Based on the model deployment constraint data, the runtime overhead data, and the quantization sensitivity data, the set of network layers to be quantized is subjected to mixed precision quantization processing to generate weighted quantization correlation data and quantization candidate models. Forward propagation is performed based on the training sample set and the quantized candidate model to generate prediction result data. Basic loss data is generated based on the prediction result data and the sample labeling data in the training sample set. Error constraint data is generated based on the weight quantization correlation data. Based on the basic loss data and the error constraint data, overall loss data is generated, and based on the overall loss data, the trainable model parameters of the quantization candidate model are updated to generate a quantization training model. Compressed analysis data is generated based on the initial neural network model, the weighted quantization correlation data, and the quantization training model. A compressed neural network model is then generated based on the quantization training model, provided that the compressed analysis data satisfies the model deployment constraints. Obtain the target input data, input the target input data into the compressed neural network model for prediction, and generate the target prediction result.

[0164] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: Obtain the initial neural network model, training sample set, and model deployment constraint data; perform hierarchical structure analysis on the initial neural network model to generate network layer structure data. Based on the network layer structure data, determine the set of network layers to be quantized, and obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized; Based on the model deployment constraint data, the runtime overhead data, and the quantization sensitivity data, the set of network layers to be quantized is subjected to mixed precision quantization processing to generate weighted quantization correlation data and quantization candidate models. Forward propagation is performed based on the training sample set and the quantized candidate model to generate prediction result data. Basic loss data is generated based on the prediction result data and the sample labeling data in the training sample set. Error constraint data is generated based on the weight quantization correlation data. Based on the basic loss data and the error constraint data, overall loss data is generated, and based on the overall loss data, the trainable model parameters of the quantization candidate model are updated to generate a quantization training model. Compressed analysis data is generated based on the initial neural network model, the weighted quantization correlation data, and the quantization training model. A compressed neural network model is then generated based on the quantization training model, provided that the compressed analysis data satisfies the model deployment constraints. Obtain the target input data, input the target input data into the compressed neural network model for prediction, and generate the target prediction result.

[0165] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0167] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0168] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0169] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A target prediction method based on hybrid precision quantization, characterized in that, Includes the following steps: Obtain the initial neural network model, training sample set, and model deployment constraint data; perform hierarchical structure analysis on the initial neural network model to generate network layer structure data. Based on the network layer structure data, determine the set of network layers to be quantized, and obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized; Based on the model deployment constraint data, the runtime overhead data, and the quantization sensitivity data, the set of network layers to be quantized is subjected to mixed precision quantization processing to generate weighted quantization correlation data and quantization candidate models. Forward propagation is performed based on the training sample set and the quantized candidate model to generate prediction result data. Basic loss data is generated based on the prediction result data and the sample labeling data in the training sample set. Error constraint data is generated based on the weight quantization correlation data. Based on the basic loss data and the error constraint data, overall loss data is generated, and based on the overall loss data, the trainable model parameters of the quantization candidate model are updated to generate a quantization training model. Compressed analysis data is generated based on the initial neural network model, the weighted quantization correlation data, and the quantization training model. A compressed neural network model is then generated based on the quantization training model, provided that the compressed analysis data satisfies the model deployment constraints. Obtain the target input data, input the target input data into the compressed neural network model for prediction, and generate the target prediction result.

2. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Obtain the initial neural network model, training sample set, and model deployment constraint data; perform hierarchical structure parsing on the initial neural network model to generate network layer structure data, including: The initial neural network model, training sample set, and model deployment constraint data are obtained. Model definition information and parameter index information are extracted from the initial neural network model. Based on the model definition information, the sequence of network layers contained in the initial neural network model and the data flow relationship between each network layer are determined. Traverse the network layer sequence, identify the layer type of each network layer, and determine the depth level of each network layer according to its position in the network layer sequence; Based on the parameter index information, extract the parameter description data of each network layer; Extract sample input structure data from the training sample set, and establish a correspondence between the sample input structure data and the input layer in the network layer sequence to generate input layer boundary data; Deployment resource boundary data is extracted from the model deployment constraint data, and a reference relationship is established between the deployment resource boundary data and the network layer sequence to generate deployment constraint reference data; The network layer structure data is generated by combining the layer type, depth level, parameter description data, input layer boundary data, and deployment constraint reference data of each network layer with the data flow relationship.

3. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Based on the network layer structure data, a set of network layers to be quantized is determined, and the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized are obtained, including: Extract layer type, parameter description data, data flow relationship, and network layer identifier from the network layer structure data; Based on the layer type and the parameter description data, a set of candidate network layers containing quantifiable parameters is determined, and based on the data flow relationship, the path position data of each candidate network layer in the set of candidate network layers containing quantifiable parameters in the forward propagation path is determined. Based on the path location data, candidate network layers located in the forward propagation path are selected from the candidate network layer set containing quantizable parameters to generate a set of network layers to be quantized, and the network layer identifier corresponding to each network layer in the set of network layers to be quantized is determined based on the network layer identifier. Based on the parameter description data, parameter access description data corresponding to the set of network layers to be quantized is generated; based on the data flow relationship, feature transfer description data corresponding to the set of network layers to be quantized is generated; and based on the layer type, hierarchical operation description data corresponding to the set of network layers to be quantized is generated. The parameter access description data, the feature transfer description data, and the hierarchical operation description data are bound according to the identifier of the network layer to be quantized to generate runtime overhead data; Calibration sample data is extracted from the training sample set and input into the initial neural network model to generate benchmark output data; A candidate quantization bit width set is defined. For each network layer to be quantized in the set of network layers to be quantized and each candidate quantization bit width in the candidate quantization bit width set, the network layer to be quantized currently being traversed is determined as the current network layer to be quantized, and the candidate quantization bit width to be traversed currently is determined as the current candidate quantization bit width. The parameter states of the network layers other than the current network layer to be quantized remain unchanged. The original weight data contained in the current network layer to be quantized is converted into probe quantization weight data according to the current candidate quantization bit width. Based on the probe quantization weight data, a hierarchical probe model corresponding to the current network layer to be quantized and the current candidate quantization bit width is generated. The calibration sample data is input into each level of the detection model to generate detection output data corresponding to each level of the detection model, and the output offset data between the detection output data and the reference output data is determined. Quantization sensitivity data is generated based on the output offset data.

4. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Based on the model deployment constraint data, the runtime overhead data, and the quantization sensitivity data, mixed-precision quantization processing is performed on the set of network layers to be quantized to generate weighted quantization correlation data and quantization candidate models, including: The available storage boundary data and inference overhead boundary data are extracted from the model deployment constraint data, and the identifier of each network layer to be quantized in the set of network layers to be quantized is obtained from the network layer structure data. Based on the runtime overhead data, generate layer compression priority data corresponding to the set of network layers to be quantized, and generate layer precision preservation priority data corresponding to the set of network layers to be quantized based on the quantization sensitivity data. Based on the available storage boundary data, the inference overhead boundary data, the layer compression priority data, and the layer precision preservation priority data, quantization bit width data is generated for the set of network layers to be quantized. The original weight data is extracted from the set of network layers to be quantized, the numerical distribution characteristics of the original weight data are determined, and weight quantization mapping parameter data is generated based on the numerical distribution characteristics and the quantization bit width data. Based on the weight quantization mapping parameter data, the original weight data is converted into quantized weight data; The quantization bit width data, the original weight data, the weight quantization mapping parameter data, and the quantization weight data are bound according to the identifier of the network layer to be quantized to generate weight quantization associated data; The quantized weight data is written into the network layer to be quantized in the initial neural network model corresponding to the network layer identifier to be quantized, and the parameter states of the network layers outside the set of network layers to be quantized in the initial neural network model are kept unchanged to generate a quantization candidate model.

5. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Forward propagation is performed based on the training sample set and the quantized candidate model to generate prediction result data. Basic loss data is generated based on the prediction result data and the sample labeling data in the training sample set. Error constraint data is generated based on the weighted quantization correlation data, including: Extract sample input data, sample annotation data, and sample identification data from the training sample set, and establish sample correspondence data between the sample input data and the sample annotation data based on the sample identification data; Extract the network layer identifier to be quantized, the original weight data, the quantized weight data, and the quantization bit width data from the weight quantization correlation data; The sample input data is input into the quantization candidate model for forward propagation, and the network layer corresponding to the network layer identifier to be quantized in the quantization candidate model uses the quantization weight data to participate in the forward propagation process to generate prediction result data carrying sample identifier data. Based on the sample corresponding data and the sample identification data, the prediction result data and the sample annotation data are aligned in terms of output fields to generate prediction annotation aligned data; Using a preset loss function, the basic loss data is determined based on the aligned predicted values ​​and labeled values ​​in the predicted labeled alignment data; Based on the identifier of the network layer to be quantized, establish weight correspondence data between the original weight data and the quantized weight data, and perform element-by-element difference measurement on the corresponding original weight values ​​and quantized weight values ​​in the weight correspondence data to generate weight offset data. The weight offset data and the quantization bit width data are bound according to the identifier of the network layer to be quantized to generate layer-by-layer error constraint data, and the layer-by-layer error constraint data corresponding to each network layer to be quantized are summarized to generate error constraint data.

6. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Generate overall loss data based on the basic loss data and the error constraint data, and update the trainable model parameters of the quantization candidate model based on the overall loss data to generate a quantization training model, including: Based on the basic loss data and the error constraint data, loss scale alignment data is generated, and based on the loss scale alignment data, basic loss weight data and error constraint weight data are determined. Adjust the basic loss data according to the basic loss weight data to generate weighted basic loss data, and adjust the error constraint data according to the error constraint weight data to generate weighted error constraint data; The weighted basic loss data and the weighted error constraint data are combined to generate overall loss data; Trainable model parameters are determined from the quantization candidate models, and backpropagation is performed on the quantization candidate models based on the overall loss data to generate parameter update data corresponding to the trainable model parameters; The trainable model parameters are updated based on the parameter update data to generate a parameter update model; Extract the network layer identifier to be quantized, quantization bit width data, and weight quantization mapping parameter data from the weight quantization association data, and based on the quantization bit width data and the weight quantization mapping parameter data, convert the updated trainable model parameters corresponding to the network layer identifier to be quantized in the parameter update model into training round quantization weight data. The quantized weight data of the network layer corresponding to the network layer identifier to be quantized is replaced in the parameter update model based on the quantized weight data of the training round, and the quantized weight data in the weight quantization association data is replaced in the weight quantization association data based on the quantized weight data of the training round, and the weight quantization association data of the training round is generated. Determine whether the parameter update model after the replacement meets the preset training termination condition; When the preset training termination condition is met, the replaced parameter update model is determined as the quantization training model; When the preset training termination condition is not met, the replaced parameter-updated model is used as the quantization candidate model for the next training round. Forward propagation is performed again based on the training sample set and the quantization candidate model for the next training round. Prediction result data and basic loss data are regenerated. Error constraint data is regenerated based on the weight quantization correlation data of the training round. Overall loss data is regenerated based on the regenerated basic loss data and the regenerated error constraint data. The trainable model parameters of the quantization candidate model for the next training round are updated based on the regenerated overall loss data until the preset training termination condition is met.

7. The target prediction method based on hybrid precision quantization as described in claim 1, characterized in that, Based on the initial neural network model, the weighted quantized correlation data, and the quantized training model, compressed analysis data is generated. Then, under the condition that the compressed analysis data satisfies the model deployment constraints, a compressed neural network model is generated based on the quantized training model, including: Initial parameter data is extracted and stored from the initial neural network model, and training parameter data is extracted and stored from the quantized training model; Extract the network layer identifier to be quantized, quantization bit width data, weight quantization mapping parameter data, and quantization weight data from the weight quantization association data; Based on the identifier of the network layer to be quantized, the initial parameter entries and training parameter entries corresponding to each network layer to be quantized are extracted from the initial parameter storage data and the training parameter storage data, respectively. The initial parameter entries, the training parameter entries, the quantization bit width data and quantization weight data of each network layer to be quantized are then hierarchically aligned to generate hierarchical compression comparison data. Based on the hierarchical compression comparison data, a model is generated to store compressed data; based on the quantization training model and the quantization bit width data, inference overhead analysis data is generated; and based on the weighted quantization mapping parameter data and the quantization training model, quantization integrity analysis data is generated. Compressed analysis data is generated based on the model storage compressed data, the inference overhead analysis data, and the quantization integrity analysis data; Available storage boundary data, inference overhead boundary data, and model structure boundary data are extracted from the model deployment constraint data, and deployment satisfaction markers are generated based on the compressed analysis data, available storage boundary data, inference overhead boundary data, and model structure boundary data. When the deployment satisfies the flag indicating that the compressed analysis data satisfies the model deployment constraint data, inference parameter configuration data is generated based on the quantized training model, the quantized bit width data, and the weight quantization mapping parameter data. The training auxiliary state data is removed from the quantized training model to generate inference model structure data. The inference parameter configuration data and the inference model structure data are then combined to generate a compressed neural network model.

8. A target prediction device based on hybrid precision quantization, characterized in that, The target prediction device based on hybrid precision quantization includes: The hierarchical structure parsing module is used to obtain the initial neural network model, training sample set and model deployment constraint data, and to perform hierarchical structure parsing on the initial neural network model to generate network layer structure data. The quantization layer analysis module is used to determine the set of network layers to be quantized based on the network layer structure data, and to obtain the runtime overhead data and quantization sensitivity data corresponding to the set of network layers to be quantized. The mixed-precision quantization module is used to perform mixed-precision quantization on the set of network layers to be quantized based on the model deployment constraint data, the runtime overhead data and the quantization sensitivity data, and generate weighted quantization correlation data and quantization candidate models. The training loss generation module is used to perform forward propagation based on the training sample set and the quantized candidate model to generate prediction result data, generate basic loss data based on the prediction result data and the sample labeling data in the training sample set, and generate error constraint data based on the weight quantization correlation data. The quantization training update module is used to generate overall loss data based on the basic loss data and the error constraint data, and update the trainable model parameters of the quantization candidate model based on the overall loss data to generate a quantization training model. The compressed model generation module is used to generate compressed analysis data based on the initial neural network model, the weighted quantization correlation data, and the quantization training model, and to generate a compressed neural network model based on the quantization training model under the condition that the compressed analysis data satisfies the model deployment constraint data. The target prediction module is used to acquire target input data, input the target input data into the compressed neural network model for prediction, and generate target prediction results.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a target prediction program based on mixed precision quantization stored in the memory and executable on the processor, wherein the target prediction program based on mixed precision quantization, when executed by the processor, implements the steps of the target prediction method based on mixed precision quantization as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a target prediction program based on mixed precision quantization, which, when executed by a processor, implements the steps of the target prediction method based on mixed precision quantization as described in any one of claims 1-7.

Images