Quantization methods, target detection methods and devices for neural networks

Abstract

This disclosure provides a method for quantizing a neural network, a method for detecting targets, and an apparatus. The method for quantizing a neural network includes: acquiring a neural network to be quantized and training data; quantizing the network layers to be quantized in the neural network to obtain an initial quantized network; for a first target network layer to be adjusted in the initial quantized network, processing the training data based on an adjusted second target network layer preceding the first target network layer to obtain intermediate output data, and processing the intermediate output data based on the first target network layer to obtain first output data; processing the training data based on the network layers to be quantized corresponding to the first and second target network layers in the neural network to obtain second output data; and adjusting the network parameters of the first target network layer based on the first and second output data to obtain the quantized target neural network corresponding to the neural network to be quantized.

Description

Technical Field

[0001] This disclosure relates to the field of neural network technology, and more specifically, to a method for quantizing neural networks, a method for object detection, and an apparatus. Background Art

[0002] Deep learning is a computationally intensive algorithm. As the diversity and complexity of processing tasks increase, the requirements for algorithm accuracy and real-time performance continue to rise. As a result, the scale of neural networks is getting larger and larger, which requires more computing and storage resources, thus putting pressure on the deployment of neural networks.

[0003] In related technologies, neural networks are often compressed by quantizing them. However, after quantization, the accuracy of the neural network drops significantly. Therefore, how to recover the network accuracy lost due to quantization has become one of the urgent problems to be solved in this field. Summary of the Invention

[0004] This disclosure provides at least one method for quantizing neural networks, a method for object detection, and an apparatus.

[0005] In a first aspect, embodiments of this disclosure provide a method for quantizing a neural network, comprising:

[0006] Acquire the neural network to be quantized and the training data, and perform quantization processing on the network layers to be quantized of the neural network to be quantized to obtain the initial quantized network;

[0007] For the first target network layer to be adjusted in the initial quantization network, based on the adjusted second target network layer before the first target network layer, the training data is processed to obtain intermediate output data, and based on the first target network layer, the intermediate output data is processed to obtain the first output data.

[0008] Based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively, the training data is processed to obtain the second output data;

[0009] Based on the first output data and the second output data, the network parameters of the first target network layer are adjusted to obtain the quantized target neural network corresponding to the network to be quantized.

[0010] In this way, after quantizing the neural network to be quantized to obtain an initial quantized network, for the first target network layer in the initial quantized network, based on the adjusted second target network layer preceding the first target network layer, the training data is processed to obtain intermediate output data, and the first output data of the first target network layer is determined based on the intermediate output data; then, based on the network layers corresponding to the first and second target network layers in the neural network to be quantized, the training data is processed to obtain second output data; based on the first and second output data, the first target network layer is adjusted to obtain the quantized target neural network corresponding to the network to be quantized. This allows for the adjustment of the target network layer without using the labeled data corresponding to the training data, saving the manpower costs required for training; and during the adjustment of the first target network layer, the output data of the already adjusted second target network layer is incorporated, thus taking into account the error introduced by the adjustment of the second target network layer when adjusting the first target network layer, reducing error accumulation, and improving the network accuracy of the quantized target neural network.

[0011] In one possible implementation, before quantizing the network layer to be quantized in the neural network to be quantized, the method further includes:

[0012] Based on boundary parameters, the network parameters of the network layer to be quantized in the neural network to be quantized are initially adjusted within a certain range, wherein the boundary parameters are the training parameters in the initial training process of the neural network to be quantized.

[0013] In this way, by using boundary parameters to perform preliminary range adjustment on the network layer to be quantized before quantization, the adjusted network parameters and the output of each layer can be made more stable during quantization, thereby reducing potential quantization loss.

[0014] In one possible implementation, the quantization process of the network layer to be quantized in the neural network includes:

[0015] At least one of the weight values, bias values, and activation values ​​of the network layer to be quantized is quantized.

[0016] In one possible implementation, the step of quantizing the unquantized network layer of the neural network to obtain an initial quantized network includes:

[0017] For any of the network layers to be quantized, the network layer to be quantized is quantized based on a plurality of preset quantization parameters corresponding to the network layer to be quantized, thereby obtaining a plurality of quantized network layers to be screened.

[0018] The quantization loss corresponding to each network layer to be screened is determined, and based on the quantization loss, the quantized target network layer corresponding to the network layer to be quantized is determined from the plurality of network layers to be screened. The quantized target network layers corresponding to each network layer to be quantized constitute the initial quantization network.

[0019] In this way, by using multiple quantization parameters to quantize the same network layer to be quantized, and by filtering the network layers to be screened after quantization based on the quantization loss, the quantization loss of each target network layer in the final initial quantized network can be reduced, thereby improving the quantization effect of the neural network to be quantized in the quantization process.

[0020] In one possible implementation, the quantization loss corresponding to the network layer to be screened is determined according to the following method:

[0021] The quantization loss is determined based on the output data of the network layer to be screened and the output data of the network layer to be quantized corresponding to the network layer to be screened.

[0022] The output data of the network layer to be screened is determined based on the output data of the target network layer after quantization processing.

[0023] In this way, when calculating the quantization loss, the output data of the target network layer to be screened is determined based on the output data of the target network layer after quantization processing before the network layer to be screened. This allows the quantization loss for the network layer to be screened to be accumulated from the quantization loss of the quantization process before the network layer to be screened. This enables the location of the large quantization loss during the quantization process to be located based on the quantization loss, facilitating timely adjustments to the quantization process to obtain better quantization results.

[0024] In one possible implementation, adjusting the network parameters of the first target network layer based on the first output data and the second output data includes:

[0025] The bias value of the first target network layer is adjusted based on the first output data and the second output data; and / or the weight value of the first target network layer is adjusted based on the first output data, the second output data and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight value of the first target network layer.

[0026] In this way, the adjustment of the first target network layer can be completed without introducing additional annotation information, saving the manpower cost required for network parameter adjustment (i.e., training).

[0027] In one possible implementation, adjusting the bias value of the first target network layer based on the first output data and the second output data includes:

[0028] Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer;

[0029] The bias value of the first target network layer is adjusted based on the aforementioned bias adjustment value;

[0030] Determine the first loss value of the first target network layer, and update the first output data based on the first target network layer after the bias value adjustment if the first loss value does not meet the first preset condition;

[0031] Based on the updated first output data and second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition.

[0032] In one possible implementation, the training data includes sample images, the first output data includes a first feature map, and the second output data includes a second feature map.

[0033] The step of determining the bias adjustment value corresponding to the first target network layer based on the first output data and the second output data includes:

[0034] The difference between the first mean of each channel in the second feature map and the second mean of each channel in the first feature map is used as the bias adjustment value; or...

[0035] Based on the values ​​of each channel of the first feature map and the second feature map after activation processing, and the mask, the bias adjustment value is determined. The mask is used to filter the channel values ​​of the first feature map and the second feature map when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

[0036] In this way, by using the mask to filter the channel values ​​involved in calculating the first and second means, channels that were adjusted in the preliminary range adjustment stage can be excluded from the calculation process, thereby avoiding the influence of the preliminary range adjustment operation on the determination of the bias adjustment value and ensuring the accuracy of the bias adjustment value.

[0037] In one possible implementation, adjusting the weight values ​​of the first target network layer based on the first output data, the second output data, and the rounding mask parameters to be trained includes:

[0038] Based on the quantization parameters corresponding to the first target network layer, the weight values ​​in the first target network layer are rounded to determine the target weight values ​​corresponding to each weight value.

[0039] The target weight values ​​corresponding to each weight value are updated based on the rounding mask parameters, and the first output data is updated based on the updated weight values.

[0040] The second loss value is determined based on the updated first output data and the second output data. If the second loss value does not meet the second preset condition, the process returns to the step of determining the target weight value until the second loss value meets the second preset condition.

[0041] In this way, by introducing a rounding mask parameter to adjust the weight values ​​of the first target network layer, the network accuracy of the final target neural network can be improved.

[0042] In one possible implementation, after obtaining the quantized target neural network corresponding to the network to be quantized, the method further includes:

[0043] The target neural network is trained based on the training data and the labeled data of the training data.

[0044] In this way, by training the target neural network with labeled training data, the network accuracy of the target neural network can be further improved.

[0045] Secondly, embodiments of this disclosure also provide a target detection method, including:

[0046] Acquire the point cloud data to be detected;

[0047] The target neural network, quantized by the neural network quantization method described in any of the first aspects, is used to detect the point cloud data to be detected, and the detection result of the point cloud data to be detected is determined.

[0048] The target vehicle's movement is controlled based on the detection results.

[0049] Thirdly, embodiments of this disclosure also provide a quantization device for a neural network, comprising:

[0050] The quantization module is used to acquire the neural network to be quantized and the training data, and to perform quantization processing on the network layers to be quantized of the neural network to obtain the initial quantized network.

[0051] The determining module is used to process the training data to obtain intermediate output data based on the adjusted second target network layer before the first target network layer in the initial quantization network, and to process the intermediate output data based on the first target network layer to obtain first output data.

[0052] The processing module is used to process the training data to obtain second output data based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer, respectively.

[0053] The adjustment module is used to adjust the network parameters of the first target network layer based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized.

[0054] In one possible implementation, before quantizing the network layer to be quantized in the neural network to be quantized, the quantization module is further configured to:

[0055] Based on boundary parameters, the network parameters of the network layer to be quantized in the neural network to be quantized are initially adjusted within a certain range, wherein the boundary parameters are the training parameters in the initial training process of the neural network to be quantized.

[0056] In one possible implementation, the quantization module, when performing quantization processing on the network layer to be quantized in the neural network to be quantized, is used to:

[0057] At least one of the weight values, bias values, and activation values ​​of the network layer to be quantized is quantized.

[0058] In one possible implementation, the quantization module, when performing quantization processing on the unquantized network layer of the neural network to be quantized to obtain an initial quantized network, is used to:

[0059] For any of the network layers to be quantized, the network layer to be quantized is quantized based on a plurality of preset quantization parameters corresponding to the network layer to be quantized, thereby obtaining a plurality of quantized network layers to be screened.

[0060] The quantization loss corresponding to each network layer to be screened is determined, and based on the quantization loss, the quantized target network layer corresponding to the network layer to be quantized is determined from the plurality of network layers to be screened. The quantized target network layers corresponding to each network layer to be quantized constitute the initial quantization network.

[0061] In one possible implementation, the quantization module is used to determine the quantization loss corresponding to the network layer to be screened according to the following method:

[0062] The quantization loss is determined based on the output data of the network layer to be screened and the output data of the network layer to be quantized corresponding to the network layer to be screened.

[0063] The output data of the network layer to be screened is determined based on the output data of the target network layer after quantization processing.

[0064] In one possible implementation, the adjustment module, when adjusting the network parameters of the first target network layer based on the first output data and the second output data, is used to:

[0065] The bias value of the first target network layer is adjusted based on the first output data and the second output data; and / or the weight value of the first target network layer is adjusted based on the first output data, the second output data and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight value of the first target network layer.

[0066] In one possible implementation, the adjustment module, when adjusting the bias value of the first target network layer based on the first output data and the second output data, is used to:

[0067] Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer;

[0068] The bias value of the first target network layer is adjusted based on the aforementioned bias adjustment value;

[0069] Determine the first loss value of the first target network layer, and update the first output data based on the first target network layer after the bias value adjustment if the first loss value does not meet the first preset condition;

[0070] Based on the updated first output data and second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition.

[0071] In one possible implementation, the training data includes sample images, the first output data includes a first feature map, and the second output data includes a second feature map.

[0072] The adjustment module, when determining the bias adjustment value corresponding to the first target network layer based on the first output data and the second output data, is used to:

[0073] The difference between the first mean of each channel in the second feature map and the second mean of each channel in the first feature map is used as the bias adjustment value; or...

[0074] Based on the values ​​of each channel of the first feature map and the second feature map after activation processing, and the mask, the bias adjustment value is determined. The mask is used to filter the channel values ​​of the first feature map and the second feature map when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

[0075] In one possible implementation, the adjustment module, when adjusting the weight values ​​of the first target network layer based on the first output data, the second output data, and the rounding mask parameters to be trained, is used to:

[0076] Based on the quantization parameters corresponding to the first target network layer, the weight values ​​in the first target network layer are rounded to determine the target weight values ​​corresponding to each weight value.

[0077] The target weight values ​​corresponding to each weight value are updated based on the rounding mask parameters, and the first output data is updated based on the updated weight values.

[0078] The second loss value is determined based on the updated first output data and the second output data. If the second loss value does not meet the second preset condition, the process returns to the step of determining the target weight value until the second loss value meets the second preset condition.

[0079] In one possible implementation, after obtaining the quantized target neural network corresponding to the network to be quantized, the adjustment module is further configured to:

[0080] The target neural network is trained based on the training data and the labeled data of the training data.

[0081] Fourthly, embodiments of this disclosure also provide a target detection device, comprising:

[0082] The acquisition module is used to acquire the point cloud data to be detected;

[0083] The detection module is used to detect the point cloud data to be detected based on the target neural network obtained by the quantization method of the neural network as described in any of the first aspects, and to determine the detection result of the point cloud data to be detected.

[0084] The control module is used to control the driving of the target vehicle based on the detection results.

[0085] Fifthly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, they perform the steps in any of the possible implementations of the first or second aspect described above.

[0086] In a sixth aspect, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps in any of the possible implementations of the first or second aspect described above.

[0087] For a description of the effects of the quantization device, computer equipment, and computer-readable storage medium of the aforementioned neural network, please refer to the description of the quantization method of the aforementioned neural network, which will not be repeated here.

[0088] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. Attached Figure Description

[0089] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0090] Figure 1 A flowchart of a quantization method for a neural network provided in an embodiment of this disclosure is shown;

[0091] Figure 2 A flowchart illustrating a specific method for obtaining an initial quantized network in the quantization method of a neural network provided in an embodiment of this disclosure is shown.

[0092] Figure 3 A flowchart is shown illustrating a specific method for adjusting the bias value in the quantization method of the neural network provided in this embodiment of the present disclosure;

[0093] Figure 4 A flowchart illustrating a specific method for adjusting weight values ​​in the quantization method of a neural network provided in this embodiment of the present disclosure is shown.

[0094] Figure 5A flowchart of a target detection method provided by an embodiment of this disclosure is shown;

[0095] Figure 6 A flowchart illustrating another method for quantizing a neural network provided in an embodiment of this disclosure is shown;

[0096] Figure 7 This diagram illustrates the adjustment of model parameters in the quantization method of the neural network provided in this embodiment of the present disclosure.

[0097] Figure 8 A schematic diagram of the architecture of a quantization device for a neural network provided in an embodiment of this disclosure is shown;

[0098] Figure 9 This diagram illustrates the architecture of a target detection device provided in an embodiment of the present disclosure;

[0099] Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0101] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0102] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0103] Research has found that related technologies often compress neural networks by quantizing them. However, after quantization, the network accuracy drops significantly. Therefore, how to recover from the loss of network accuracy caused by quantization has become one of the urgent problems to be solved in this field.

[0104] Based on the above research, this disclosure provides a method for quantizing neural networks, a method for detecting targets, and an apparatus. After quantizing the neural network to be quantized to obtain an initial quantized network, for a first target network layer in the initial quantized network, based on an adjusted second target network layer preceding the first target network layer, the training data is processed to obtain intermediate output data, and a first output data for the first target network layer is determined based on the intermediate output data. Then, based on the network layers corresponding to the first and second target network layers in the neural network to be quantized, the training data is processed to obtain second output data. Based on the first and second output data, the first target network layer is adjusted to obtain the quantized target neural network corresponding to the network to be quantized. In this way, the adjustment of the target network layer can be completed without using labeled data corresponding to the training data, saving the manpower cost required for training. Furthermore, during the adjustment of the first target network layer, the output data of the already adjusted second target network layer is incorporated, which allows the error introduced by the adjustment of the second target network layer to be considered when adjusting the first target network layer, reducing error accumulation and improving the network accuracy of the quantized target neural network.

[0105] To facilitate understanding of this embodiment, a detailed description of the neural network quantization method disclosed in this disclosure is provided first. The execution entity of the neural network quantization method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, or a terminal, etc. In some possible implementations, the neural network quantization method can be implemented by a processor calling computer-readable instructions stored in memory.

[0106] See Figure 1 The diagram shows a flowchart of a neural network quantization method provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein:

[0107] S101: Obtain the neural network to be quantized and the training data, and perform quantization processing on the network layer to be quantized of the neural network to obtain the initial quantized network.

[0108] S102: For the first target network layer to be adjusted in the initial quantization network, based on the adjusted second target network layer before the first target network layer, the training data is processed to obtain intermediate output data, and based on the first target network layer, the intermediate output data is processed to obtain first output data.

[0109] S103: Based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively, the training data is processed to obtain the second output data.

[0110] S104: Adjust the network parameters of the first target network layer based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized.

[0111] The following is a detailed explanation of the steps described above.

[0112] Regarding S101, when quantizing the network layer to be quantized in the neural network to be quantized, the network layer to be quantized can be quantized according to preset quantization parameters. The quantization parameters are used to characterize the quantization range of each network layer to be quantized in the neural network to be quantized, and may include the range of fixed-point numbers when quantizing floating-point values ​​to fixed-point numbers, as well as the quantization step size when performing quantization processing. The quantization step size may be, for example, 8 bits, 16 bits, etc. The network layer to be quantized may be all the network layers of the neural network to be quantized, or it may be a part of the network layers of the neural network to be quantized, such as the convolutional layer in the neural network to be quantized.

[0113] For example, taking autonomous driving as an application scenario, the neural network to be quantized can be a point cloud target detection network (the network type can be SECOND model, PointPillars model, PointRCNN model, PV-RCNN model, etc.), used to perform target detection on the point cloud data of the LiDAR deployed on the autonomous driving device; the training data can be sample point cloud data collected in the autonomous driving scenario.

[0114] Specifically, when quantizing the network layer to be quantized in the neural network to be quantized, at least one of the weight value, bias value and activation value of the network layer to be quantized can be quantized.

[0115] For example, for a given floating-point tensor X to be quantized (which can be at least one of weight values, bias values, and activation values), by determining a quantized fixed-point range tensorScale and a pre-specified fixed-point bit width (i.e., quantization step size) bitWidth, its quantized fixed-point tensor Xq can be represented as:

[0116]

[0117] The function clamp(X, min, max) truncates the values ​​of each element in the tensor to a certain preset closed interval. That is, values ​​less than the lower bound of the preset interval will be truncated to the lower bound, and values ​​greater than the upper bound of the preset interval will be truncated to the upper bound.

[0118] In one possible implementation, the neural network to be quantized is a point cloud target detection network, which consists of multiple network structures such as a point cloud feature extraction network, a feature fusion network, and a prediction head network.

[0119] Here, the point cloud feature extraction network is used to extract features from the input point cloud data to obtain feature vectors (or tensors) corresponding to the point cloud data; the feature fusion network is used to fuse the features of the point cloud data extracted by the point cloud feature extraction network, including tensor concatenation operations on the input tensors; the prediction head network is used to predict the features of the fused point cloud data. The prediction head network contains an activation layer, which can be located at the end of the prediction head network and is used to activate the input data of the activation layer to improve the expressive power of the prediction head network.

[0120] Specifically, when quantizing the network layer to be quantized, the network parameters corresponding to each tensor that needs to be spliced ​​can be quantized using the same quantization parameters to reduce the quantization loss caused by subsequent tensor splicing operations. In addition, the quantization range can be set for the activation layer at the end of the prediction head network based on experimental results to improve the expressive power of the point cloud feature extraction network.

[0121] In one possible implementation, before quantizing the network layer of the neural network to be quantized, the network parameters of the network layer to be quantized can be initially adjusted based on boundary parameters.

[0122] Here, the boundary parameters are the training parameters in the initial training process of the neural network to be quantized. Different boundary parameters can be used for preliminary range adjustment for different quantization objects in the network layer to be quantized.

[0123] Specifically, the neural network to be quantized undergoes initial training before quantization to improve its accuracy. During the initial training, a boundary parameter is added to the loss function to determine the loss value of the neural network. Based on this boundary parameter, the network layer to be quantized is preliminarily adjusted (i.e., network parameter adjustment during training) based on the loss value. The boundary parameter can be used to determine the adjustment range when adjusting the parameters during the preliminary adjustment.

[0124] In one possible implementation, a first boundary parameter can be set for the output data of the active layer in the network layer to be quantized. When performing preliminary range adjustment, a first numerical range can be determined based on the first boundary parameter, and the data in the output data of the active layer that are outside the first numerical range can be adjusted to the boundary value corresponding to the first numerical range.

[0125] Here, the first boundary parameter can be a learnable parameter that is updated in real time during the initial range adjustment, and the first boundary parameter can be trained using a weight decay mechanism (i.e., the first boundary parameter is updated).

[0126] For example, taking a first numerical range determined based on the first boundary parameter as -10 to 10, and the output data containing the data 8, 9, and 11 as an example, since 11 is outside the first numerical range, the data can be adjusted to the corresponding boundary value 10, so that the output data of the activation layer after the initial range adjustment is 8, 9, and 10.

[0127] For example, taking the adjustment of the output data of the ReLU activation layer as an example, the output of the ReLU activation layer after the initial range adjustment can be expressed as:

[0128] ClipReLU(X) = clamp(X, 0, α′) where α′ is the upper bound of the first numerical range calculated based on the learnable parameter α. When calculating the upper bound of the first numerical range, the specific calculation formula can be set according to the actual use situation, and this embodiment does not limit it.

[0129] In another possible implementation, a second boundary parameter can be set for the weight values. When making the initial range adjustment, the second numerical range corresponding to the initial range adjustment of the network layer to be quantized can be determined based on the second boundary parameter and the weight value of any of the network layers to be quantized, and the data in the weight values ​​that are outside the second numerical range can be adjusted to the boundary value corresponding to the second numerical range.

[0130] Here, the second boundary parameter can be a parameter that is preset based on experimental results.

[0131] Specifically, when determining the second numerical range, the mean of the absolute values ​​of the weights of any of the network layers to be quantized, and the second boundary parameter, can be used to determine the second numerical range corresponding to the initial range adjustment of the network layer to be quantized.

[0132] For example, taking the weight values ​​of the network layer to be quantized as 1, 2, 3, and 4, and the second boundary parameter as 1.2, the mean absolute value of the weight values ​​of the network layer to be quantized can be determined to be 2.5. Based on the product of the mean and the second boundary parameter, the second numerical range can be determined to be -3 to 3. The weight value 4 can be adjusted to 3, resulting in the weight values ​​of the network layer to be quantized after the initial range adjustment as 1, 2, 3, and 3.

[0133] For example, taking the adjustment of the weights of a convolutional layer as an example, the weights of the convolutional layer after the initial range adjustment can be expressed as:

[0134] W′=clamp(W,-avg(|W|)·β,avg(|W|)·β)

[0135] Where β is the second boundary parameter, avg represents the average value, and W represents the weight value of the convolutional layer before adjustment.

[0136] In this way, by using boundary parameters to perform preliminary range adjustment on the network layer to be quantized before quantization, the adjusted network parameters and the output of each layer can be made more stable during quantization, thereby reducing potential quantization loss.

[0137] One possible implementation, such as Figure 2 As shown, the initial quantization network can be obtained through the following steps:

[0138] S201: For any of the network layers to be quantized, perform quantization processing on the network layer to be quantized based on a plurality of preset quantization parameters corresponding to the network layer to be quantized, to obtain a plurality of network layers to be screened after quantization processing corresponding to the network layer to be quantized.

[0139] S202: Determine the quantization loss corresponding to each network layer to be screened, and based on the quantization loss, determine the quantized target network layer corresponding to the network layer to be quantized from the plurality of network layers to be screened. The quantized target network layers corresponding to each network layer to be quantized constitute the initial quantization network.

[0140] Here, the quantization loss can be characterized by a similarity index between the output data of the network layer before and after quantization. The similarity index can be an index that can characterize similarity, such as L1 distance or cosine distance.

[0141] In this way, by using multiple quantization parameters to quantize the same network layer to be quantized, and by filtering the network layers to be screened after quantization based on the quantization loss, the quantization loss of each target network layer in the final initial quantized network can be reduced, thereby improving the quantization effect of the neural network to be quantized in the quantization process.

[0142] In one possible implementation, when determining the quantization loss corresponding to the network layer to be screened, the quantization loss can be determined based on the output data of the network layer to be screened and the output data of the network layer to be quantized corresponding to the network layer to be screened.

[0143] Here, the output data of the network layer to be screened is determined based on the output data of the target network layer after quantization processing.

[0144] Specifically, the formula for calculating the quantization loss can be:

[0145]

[0146] In the above formula, This represents the output data of the i-th quantized network layer, where none of the network layers before the i-th quantized layer have undergone quantization processing. This represents the output data of the i-th network layer to be screened. All target network layers before the i-th network layer to be screened have been quantized. "similarity" indicates that a similarity index is calculated.

[0147] In this way, when calculating the quantization loss, the output data of the target network layer to be screened is determined based on the output data of the target network layer after quantization processing before the network layer to be screened. This allows the quantization loss determined for the network layer to be screened to accumulate the quantization loss of the quantization process before the network layer to be screened. As a result, the position of large quantization loss in the quantization process can be located based on the quantization loss, which facilitates timely adjustment of the quantization process to obtain better quantization results.

[0148] S102: For the first target network layer to be adjusted in the initial quantization network, based on the adjusted second target network layer before the first target network layer, the training data is processed to obtain intermediate output data, and based on the first target network layer, the intermediate output data is processed to obtain first output data.

[0149] Here, when adjusting the target network layers in the initial quantization network, the adjustment can be performed from front to back according to the connection order of each target network layer in the initial quantization network. Thus, the adjustment of each second target network layer before the first target network layer is completed before the output data of the first target network layer is determined.

[0150] Specifically, the output data of the second target network layer can be input to the first target network layer to obtain the first output data of the first target network layer. The second target network layer may contain multiple layers, and the output data of the second target network layer input to the first target network layer is the output data of the last second target network layer.

[0151] For example, if the first target network layer is the 6th convolutional layer in the initial quantization network and the second target network layer is the 1st to 5th convolutional layers, then the output data of the second target network layer input to the first target network layer is the output data of the 5th convolutional layer.

[0152] S103: Based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively, the training data is processed to obtain the second output data.

[0153] Here, the second output data is the output data of the network layer to be quantized corresponding to the first target network layer after the training data is input into the neural network to be quantized.

[0154] Continuing with the previous example, taking the first target network layer as the 6th convolutional layer in the initial quantization network, the second output data is the output data of the 6th convolutional layer after the training data is input into the neural network to be quantized.

[0155] It should be noted that the execution order of S102 and S103 can be S102 first and then S103, or S103 first and then S102, or S102 and S103 can be executed simultaneously. This embodiment does not limit the execution order.

[0156] S104: Adjust the network parameters of the first target network layer based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized.

[0157] Here, since no additional labeled data is used when adjusting the first target network layer, only the first output data and the second output data are used for adjustment, the adjustment process of adjusting the first target network layer based on the first output data and the second output data is unlabeled tuning. Using unlabeled tuning can save the manpower cost required for network parameter adjustment (i.e., training).

[0158] Specifically, adjustments can be made using any of the following methods:

[0159] Method 1: Adjust the bias value of the first target network layer.

[0160] Here, the training data may include sample images, the first output data may include a first feature map, and the second output data may include a second feature map; when adjusting the bias value of the first target network layer, the bias value of the first target network layer may be adjusted based on the first output data and the second output data.

[0161] One possible implementation, such as Figure 3 As shown, the bias value can be adjusted through the following steps:

[0162] S301: Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer.

[0163] Specifically, the bias adjustment value can be determined using any of the following methods:

[0164] Method A: The difference between the first mean of each channel in the second output data and the second mean of each channel in the first output data is used as the bias adjustment value.

[0165] Here, since the feature map in the output of the first target network layer after quantization may have an overall channel distribution offset error, resulting in a large quantization loss, in order to address this quantization loss, the difference between the first mean of each channel in the second output data (corresponding to the output data before quantization) and the second mean of each channel in the first output data (corresponding to the output data after quantization) can be determined, and the difference can be used as the bias adjustment value.

[0166] For example, the formula for calculating the bias adjustment value can be:

[0167]

[0168] Where, d [i] Avg(Y) represents the bias adjustment value corresponding to the first target network layer of the i-th layer; [i]() represents the first mean of each channel in the second output data; This represents the second mean of each channel in the first output data.

[0169] Specifically, the difference represents the offset of the overall channel distribution of the first target network layer during quantization processing. The corresponding quantization loss can be reduced by using the difference as the bias adjustment value to adjust the bias value.

[0170] Method B: Determine the offset adjustment value based on the values ​​of each channel of the first output data and the second output data after activation processing, as well as the mask.

[0171] Here, the mask is used to filter the channel values ​​of the first output data and the second output data when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

[0172] Specifically, before quantization, boundary parameters can be used to determine the numerical range for initial range adjustment, and data outside the range in the output data can be adjusted. However, the adjusted data may affect the calculation of the bias adjustment value when calculating the mean, causing the calculated bias adjustment value to be unable to accurately represent the overall channel distribution offset of the first target network layer during quantization. Therefore, the mask can be used to filter the channel values ​​participating in the calculation of the first mean and the second mean, thereby ensuring the accuracy of the bias adjustment value and improving the calculation speed when determining the bias adjustment value.

[0173] Specifically, the first feature map can have dimensions of C×H×W, where C represents the number of channels in the feature map, H represents the height of the feature map, and W represents the width of the feature map. The mask has the same dimensions as the first feature map, which is also C×H×W. Each channel in the mask corresponds one-to-one with each channel in the first feature map. The channel value of each channel in the mask can be 0 or 1. For any target channel of the mask, if the channel value in the first feature map corresponding to the target channel is not adjusted in the preliminary range adjustment stage, then the channel value of the target channel is 1, indicating that the channel value in the first feature map corresponding to the target channel participates in the calculation process of the difference of means in method A above. If the channel value in the first feature map corresponding to the target channel is adjusted in the preliminary range adjustment stage, then the channel value of the target channel is 0, indicating that the channel value in the first feature map corresponding to the target channel does not participate in the calculation process of the difference of means in method A above.

[0174] For example, taking the adjustment of the ReLU activation layer as an example, the formula for calculating the bias adjustment value can be:

[0175]

[0176] Where, d [i] This represents the bias adjustment value corresponding to the i-th network layer to be adjusted (i.e., the ReLU activation layer); This indicates the calculation of the difference between each channel; M represents the mask corresponding to the ReLU activation layer; Avg represents the averaging operation.

[0177] Thus, based on method A above, by multiplying the difference of means by a mask using 0 and 1 as channel values, channels that are adjusted in the initial range adjustment stage can be excluded from the calculation process, thereby avoiding the influence of the initial range adjustment operation on the determination of the bias adjustment value.

[0178] S302: Adjust the bias value of the first target network layer based on the bias adjustment value.

[0179] Here, when adjusting the bias value of the first target network layer based on the adjusted bias value, a warmup update method can be used for gradual adjustment.

[0180] For example, taking the bias value of the first target network layer as 1 and the bias adjustment value as 0.5, the bias value of the first target network layer can be adjusted to 1.1 instead of directly to 1.5 according to the adjustment method of warmup update, thereby avoiding the problem of network accuracy decrease caused by excessive adjustment in a single time.

[0181] For example, the formula for adjusting the bias value can be:

[0182]

[0183] In the above formula, the sum of the original bias value and the bias adjustment value can be used as the adjustment target, and the warmup update adjustment method can be used during adjustment.

[0184] S303: Determine the first loss value of the first target network layer, and if the first loss value does not meet the first preset condition, update the first output data based on the first target network layer after the bias value adjustment.

[0185] S304: Based on the updated first output data and the second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition.

[0186] Specifically, the first preset condition may be that the number of executions reaches a preset number, and the first loss value may be a similarity index between the output data of the first target network layer before and after adjustment, such as L1 distance, cosine distance, and other indicators that can characterize similarity. The calculation process is similar to the process of determining the quantization loss in S202 above, and will not be repeated here.

[0187] Method 2: Adjust the weight values ​​of the first target network layer.

[0188] Here, the weight values ​​of the first target network layer can be adjusted based on the first output data, the second output data, and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight values ​​of the first target network layer.

[0189] One possible implementation, such as Figure 4 As shown, the weight values ​​can be adjusted through the following steps:

[0190] S401: Based on the quantization parameters corresponding to the first target network layer, the weight values ​​in the first target network layer are rounded to determine the target weight values ​​corresponding to each weight value.

[0191] S402: Update the target weight values ​​corresponding to each weight value based on the rounding mask parameters, and update the first output data based on the updated weight values.

[0192] Here, the formula used to round down the weight values ​​in the first target network layer based on the quantization parameters corresponding to the first target network layer can be:

[0193]

[0194] in, This represents the target weight value after rounding, tensorScale represents the quantization range, and W... [i] M represents the unquantized weight values. rounding The rounding mask parameter represents the rounding process. The above formula can be used to round each weight value and determine the target weight value corresponding to each weight value.

[0195] Specifically, the rounding mask parameter can be a learnable parameter that is updated in real time when the target weight value is adjusted. The rounding mask parameter can be trained using a loss function that increases the penalty weight with the number of training iterations (that is, the first boundary parameter is updated).

[0196] For example, the loss function for training the rounding mask parameters can be:

[0197] L reg =λ·Σ(1-|clamp((ζ-γ)·sigmoid(M rounding )+γ,0,1)-0.5| β )

[0198] Where ζ, γ, β, and λ are all preset parameters, and β gradually decreases as the number of training iterations increases.

[0199] S403: Determine a second loss value based on the updated first output data and the second output data, and if the second loss value does not meet the second preset condition, return to the step of determining the target weight value until the second loss value meets the second preset condition.

[0200] Here, the second loss function used to determine the second loss value can be the same as the first loss function used to determine the first loss value; or, the loss function corresponding to the rounding mask parameter can be added to the first loss function, so that the second loss function can better match the real-time changes of the rounding mask parameter. The second preset condition can be that the number of executions has reached a preset number, etc.

[0201] In this way, by adjusting the bias value of the first target network layer based on the first output data and the second output data; and / or by adjusting the weight value of the first target network layer based on the first output data, the second output data, and the rounding mask parameter to be trained, the adjustment of the first target network layer can be completed without introducing additional annotation information, saving the manpower cost required for network parameter adjustment (i.e., training).

[0202] In one possible implementation, when adjusting the first target network layer based on the first output data and the second output data, the first target network layer can be adjusted based on the above-described method 1 and method 2 respectively, and the input data to the next target network layer can be re-determined based on the updated network parameters of the first target network layer. The above adjustment and update steps are repeated to complete the update of all target network layers in the initial quantization network, so that the adjustment of the initial quantization network can effectively combine multiple adjustment methods at the same time, resulting in a better adjustment effect of the initial quantization network.

[0203] In practical applications, after adjusting the first target network layer through the above-mentioned unlabeled optimization process to obtain the target neural network, the network accuracy may fail to meet the preset network accuracy requirements because no labeling information was used during the adjustment process.

[0204] In one possible implementation, after obtaining the quantized target neural network corresponding to the network to be quantized, the target neural network can be trained based on the training data and the labeled data of the training data.

[0205] Specifically, the target neural network can be trained using a preset gradient estimation method based on the training data and the labeled data of the training data, so as to further improve the network accuracy of the target network.

[0206] See Figure 5 The diagram shows a flowchart of a target detection method provided in an embodiment of this disclosure. The method includes steps S501 to S503, wherein:

[0207] S501: Acquire the point cloud data to be detected.

[0208] S502: The target neural network, quantized by the neural network quantization method described in any embodiment of this disclosure, is used to detect the point cloud data to be detected, and the detection result of the point cloud data to be detected is determined.

[0209] S503: Control the target vehicle's movement based on the detection results.

[0210] The control of the target vehicle's movement may include, for example, controlling the target vehicle's acceleration, deceleration, steering, and braking, or playing voice prompts to prompt the driver to control the target vehicle's acceleration, deceleration, steering, and braking.

[0211] The quantization method for neural networks provided in this disclosure will now be described with reference to specific examples, such as... Figure 6 As shown, the quantization method for neural networks provided in this disclosure embodiment may include the following steps:

[0212] Step 1: Obtain the dataset containing the calibration data

[0213] Here, the correction data refers to the training data used to adjust the network parameters of the neural network.

[0214] Step 2: Perform stable quantization training on the neural network to be quantized to obtain a stable quantized floating-point model.

[0215] Here, the stable quantization training is used to perform preliminary range adjustment of the network parameters of the network layer to be quantized in the neural network to be quantized.

[0216] Specifically, the detailed description of stable quantization training can be found in section S101 above, which describes the preliminary range adjustment of the network parameters of the network layer to be quantized in the neural network to be quantized based on boundary parameters. It will not be repeated here.

[0217] Step 3: Perform quantization processing on the floating-point model according to the preset quantization parameters to obtain a preliminary quantization model.

[0218] Specifically, the detailed description of the quantization process for the floating-point model can be found in the relevant content in S101 above, and will not be repeated here.

[0219] Step 4: Based on the preliminary quantization model and the floating-point model, adjust the model parameters of the preliminary quantization model to obtain the optimized quantization model.

[0220] Here, after adjusting the model parameters of the initial quantization model, a preset gradient estimation method can be used to fine-tune the quantization model to further improve the network accuracy of the fine-tuned quantization model.

[0221] In one possible implementation, when adjusting the model parameters of the preliminary quantization model based on the preliminary quantization model and the floating-point model, it can be done as follows: Figure 7 As shown, using Figure 7 The steps in the process involve adjusting the model parameters.

[0222] Here, when adjusting the model parameters of the preliminary quantization model, the adjustment can be carried out sequentially from layer 1 to layer n according to the forward propagation topology order. When adjusting to layer i, the network parameters of layer i can be adjusted by bias value correction and adaptive weight rounding based on the quantized feature map (i.e., the output data of layer i in the quantized network) and the floating-point feature map before quantization (i.e., the output data of layer i in the network before quantization). The quantization model is then updated to synchronize the adjustment to the quantization model and complete the adjustment of the network parameters of layer i.

[0223] Specifically, the relevant descriptions of bias value correction processing can be found in the S301 and S302 sections above, and the relevant descriptions of adaptive weight rounding processing can be found in the S401 and S402 sections above, and will not be repeated here.

[0224] The neural network quantization method provided in this disclosure, after quantizing the neural network to be quantized to obtain an initial quantized network, processes the training data based on the adjusted second target network layer preceding the first target network layer in the initial quantized network to obtain intermediate output data, and determines the first output data of the first target network layer based on the intermediate output data; then, based on the network layers corresponding to the first and second target network layers in the neural network to be quantized, processes the training data to obtain second output data; and adjusts the first target network layer based on the first and second output data to obtain the quantized target neural network corresponding to the network to be quantized. In this way, the adjustment of the target network layer can be completed without using the labeled data corresponding to the training data, saving the manpower cost required for training; and during the adjustment of the first target network layer, the output data of the already adjusted second target network layer is incorporated, so that the error brought by the adjustment of the second target network layer can be taken into account when adjusting the first target network layer, reducing error accumulation and improving the network accuracy of the quantized target neural network.

[0225] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0226] Based on the same inventive concept, this disclosure also provides a quantization device for a neural network corresponding to the quantization method of the neural network. Since the principle of the device in this disclosure for solving the problem is similar to the quantization method of the neural network described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0227] Reference Figure 8 The diagram shown is an architectural schematic of a quantization device for a neural network provided in an embodiment of this disclosure. The device includes: a quantization module 801, a determination module 802, a processing module 803, and an adjustment module 804; wherein,

[0228] The quantization module 801 is used to acquire the neural network to be quantized and the training data, and to perform quantization processing on the network layer to be quantized of the neural network to obtain the initial quantized network.

[0229] The determining module 802 is used to process the training data to obtain intermediate output data based on the adjusted second target network layer before the first target network layer in the initial quantization network, and to process the intermediate output data based on the first target network layer to obtain first output data.

[0230] Processing module 803 is used to process the training data to obtain second output data based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively.

[0231] The adjustment module 804 is used to adjust the network parameters of the first target network layer based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized.

[0232] In one possible implementation, before quantizing the network layer to be quantized in the neural network to be quantized, the quantization module 801 is further configured to:

[0233] Based on boundary parameters, the network parameters of the network layer to be quantized in the neural network to be quantized are initially adjusted within a certain range, wherein the boundary parameters are the training parameters in the initial training process of the neural network to be quantized.

[0234] In one possible implementation, the quantization module 801, when performing quantization processing on the network layer to be quantized in the neural network to be quantized, is used to:

[0235] At least one of the weight values, bias values, and activation values ​​of the network layer to be quantized is quantized.

[0236] In one possible implementation, the quantization module 801, when performing quantization processing on the unquantized network layer of the neural network to be quantized to obtain an initial quantized network, is used to:

[0237] For any of the network layers to be quantized, the network layer to be quantized is quantized based on a plurality of preset quantization parameters corresponding to the network layer to be quantized, thereby obtaining a plurality of quantized network layers to be screened.

[0238] The quantization loss corresponding to each network layer to be screened is determined, and based on the quantization loss, the quantized target network layer corresponding to the network layer to be quantized is determined from the plurality of network layers to be screened. The quantized target network layers corresponding to each network layer to be quantized constitute the initial quantization network.

[0239] In one possible implementation, the quantization module 801 is used to determine the quantization loss corresponding to the network layer to be screened according to the following method:

[0240] The quantization loss is determined based on the output data of the network layer to be screened and the output data of the network layer to be quantized corresponding to the network layer to be screened.

[0241] The output data of the network layer to be screened is determined based on the output data of the target network layer after quantization processing.

[0242] In one possible implementation, the adjustment module 804, when adjusting the network parameters of the first target network layer based on the first output data and the second output data, is used to:

[0243] The bias value of the first target network layer is adjusted based on the first output data and the second output data; and / or the weight value of the first target network layer is adjusted based on the first output data, the second output data and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight value of the first target network layer.

[0244] In one possible implementation, the adjustment module 804, when adjusting the bias value of the first target network layer based on the first output data and the second output data, is used to:

[0245] Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer;

[0246] The bias value of the first target network layer is adjusted based on the aforementioned bias adjustment value;

[0247] Determine the first loss value of the first target network layer, and update the first output data based on the first target network layer after the bias value adjustment if the first loss value does not meet the first preset condition;

[0248] Based on the updated first output data and second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition.

[0249] In one possible implementation, the training data includes sample images, the first output data includes a first feature map, and the second output data includes a second feature map.

[0250] The adjustment module 804, when determining the bias adjustment value corresponding to the first target network layer based on the first output data and the second output data, is used to:

[0251] The difference between the first mean of each channel in the second feature map and the second mean of each channel in the first feature map is used as the bias adjustment value; or...

[0252] Based on the values ​​of each channel of the first feature map and the second feature map after activation processing, and the mask, the bias adjustment value is determined. The mask is used to filter the channel values ​​of the first feature map and the second feature map when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

[0253] In one possible implementation, the adjustment module 804, when adjusting the weight values ​​of the first target network layer based on the first output data, the second output data, and the rounding mask parameters to be trained, is used to:

[0254] Based on the quantization parameters corresponding to the first target network layer, the weight values ​​in the first target network layer are rounded to determine the target weight values ​​corresponding to each weight value.

[0255] The target weight values ​​corresponding to each weight value are updated based on the rounding mask parameters, and the first output data is updated based on the updated weight values.

[0256] The second loss value is determined based on the updated first output data and the second output data. If the second loss value does not meet the second preset condition, the process returns to the step of determining the target weight value until the second loss value meets the second preset condition.

[0257] In one possible implementation, after obtaining the quantized target neural network corresponding to the network to be quantized, the adjustment module 804 is further configured to:

[0258] The target neural network is trained based on the training data and the labeled data of the training data.

[0259] Reference Figure 9 The diagram shown is a schematic representation of the architecture of a target detection device according to an embodiment of this disclosure. The device includes: an acquisition module 901, a detection module 902, and a control module 903; wherein,

[0260] Module 901 is used to acquire point cloud data to be detected;

[0261] The detection module 902 is used to detect the point cloud data to be detected based on the target neural network obtained by the quantization method of the neural network provided in any of the embodiments of the present disclosure, and to determine the detection result of the point cloud data to be detected.

[0262] The control module 903 is used to control the driving of the target vehicle based on the detection results.

[0263] The quantization apparatus for neural networks provided in this disclosure, after quantizing the neural network to be quantized to obtain an initial quantized network, processes the training data based on the adjusted second target network layer preceding the first target network layer in the initial quantized network to obtain intermediate output data, and determines the first output data of the first target network layer based on the intermediate output data; then, based on the network layers corresponding to the first and second target network layers in the neural network to be quantized, processes the training data to obtain second output data; and adjusts the first target network layer based on the first and second output data to obtain the quantized target neural network corresponding to the network to be quantized. In this way, the adjustment of the target network layer can be completed without using the labeled data corresponding to the training data, saving the manpower cost required for training; and during the adjustment of the first target network layer, the output data of the already adjusted second target network layer is incorporated, so that the error caused by the adjustment of the second target network layer can be taken into account when adjusting the first target network layer, reducing error accumulation and improving the network accuracy of the quantized target neural network.

[0264] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0265] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 10 The diagram shows the structure of a computer device 1000 provided in this embodiment of the present disclosure, including a processor 1001, a memory 1002, and a bus 1003. The memory 1002 stores execution instructions and includes main memory 10021 and external memory 10022. The main memory 10021, also called internal memory, is used to temporarily store computational data in the processor 1001 and data exchanged with external memory 10022 such as a hard disk. The processor 1001 exchanges data with the external memory 10022 through the main memory 10021. When the computer device 1000 is running, the processor 1001 and the memory 1002 communicate through the bus 1003, causing the processor 1001 to execute the following instructions:

[0266] Acquire the neural network to be quantized and training data, and quantize the network layers of the neural network to be quantized based on quantization parameters that match the neural network to be quantized to obtain an initial quantized network;

[0267] For the first target network layer in the initial quantization network, based on the adjusted second target network layer preceding the first target network layer, the training data is processed to obtain output data, and the first output data of the first target network layer is determined based on the output data.

[0268] Based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively, the training data is processed to obtain the second output data;

[0269] The first target network layer is adjusted based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized.

[0270] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the neural network quantization method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0271] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the neural network quantization method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0272] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0273] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0274] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0275] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0276] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0277] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A point cloud target detection method, characterized in that, include: Acquire the neural network to be quantized and the training data, and perform quantization processing on the network layers to be quantized of the neural network to be quantized to obtain the initial quantized network; For the first target network layer to be adjusted in the initial quantization network, based on the adjusted second target network layer before the first target network layer, the training data is processed to obtain intermediate output data, and based on the first target network layer, the intermediate output data is processed to obtain the first output data. Based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer respectively, the training data is processed to obtain the second output data; Based on the first output data and the second output data, the network parameters of the first target network layer are adjusted to obtain the quantized target neural network corresponding to the network to be quantized. The target neural network is used to perform target detection on the lidar point cloud data collected by the lidar deployed on the autonomous driving device; The adjustment of the network parameters of the first target network layer based on the first output data and the second output data includes: The bias value of the first target network layer is adjusted based on the first output data and the second output data; and / or the weight value of the first target network layer is adjusted based on the first output data, the second output data and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight value of the first target network layer. The adjustment of the bias value of the first target network layer based on the first output data and the second output data includes: Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer; The bias value of the first target network layer is adjusted based on the aforementioned bias adjustment value; Determine the first loss value of the first target network layer, and update the first output data based on the first target network layer after the bias value adjustment if the first loss value does not meet the first preset condition; Based on the updated first output data and second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition; The training data includes sample images, the first output data includes a first feature map, and the second output data includes a second feature map. The step of determining the bias adjustment value corresponding to the first target network layer based on the first output data and the second output data includes: Based on the values ​​of each channel of the first feature map and the second feature map after activation processing, and the mask, the bias adjustment value is determined. The mask is used to filter the channel values ​​of the first feature map and the second feature map when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

2. The method according to claim 1, characterized in that, Before quantizing the network layer to be quantized in the neural network to be quantized, the method further includes: Based on boundary parameters, the network parameters of the network layer to be quantized in the neural network to be quantized are initially adjusted within a certain range, wherein the boundary parameters are the training parameters in the initial training process of the neural network to be quantized.

3. The method according to claim 1, characterized in that, The quantization process for the network layer to be quantized in the neural network to be quantized includes: At least one of the weight values, bias values, and activation values ​​of the network layer to be quantized is quantized.

4. The method according to claim 1, characterized in that, The step of quantizing the unquantized network layer of the neural network to be quantized to obtain the initial quantized network includes: For any of the network layers to be quantized, the network layer to be quantized is quantized based on a plurality of preset quantization parameters corresponding to the network layer to be quantized, thereby obtaining a plurality of quantized network layers to be screened. The quantization loss corresponding to each network layer to be screened is determined, and based on the quantization loss, the quantized target network layer corresponding to the network layer to be quantized is determined from the plurality of network layers to be screened. The quantized target network layers corresponding to each network layer to be quantized constitute the initial quantization network.

5. The method according to claim 4, characterized in that, The quantization loss corresponding to the network layer to be screened is determined using the following method: The quantization loss is determined based on the output data of the network layer to be screened and the output data of the network layer to be quantized corresponding to the network layer to be screened. The output data of the network layer to be screened is determined based on the output data of the target network layer after quantization processing.

6. The method according to claim 1, characterized in that, The adjustment of the weight values ​​of the first target network layer based on the first output data, the second output data, and the rounded mask parameters to be trained includes: Based on the quantization parameters corresponding to the first target network layer, the weight values ​​in the first target network layer are rounded to determine the target weight values ​​corresponding to each weight value. The target weight values ​​corresponding to each weight value are updated based on the rounding mask parameters, and the first output data is updated based on the updated weight values. The second loss value is determined based on the updated first output data and the second output data. If the second loss value does not meet the second preset condition, the process returns to the step of determining the target weight value until the second loss value meets the second preset condition.

7. The method according to any one of claims 1 to 6, characterized in that, After obtaining the quantized target neural network corresponding to the network to be quantized, the method further includes: The target neural network is trained based on the training data and the labeled data of the training data.

8. A vehicle control method, characterized in that, include: Acquire the point cloud data to be detected; The target neural network obtained by the point cloud target detection method as described in any one of claims 1 to 7 is used to detect the point cloud data to be detected, and the detection result of the point cloud data to be detected is determined. The target vehicle's movement is controlled based on the detection results.

9. A point cloud target detection device, characterized in that, include: The quantization module is used to acquire the neural network to be quantized and the training data, and to perform quantization processing on the network layers to be quantized of the neural network to obtain the initial quantized network. The determining module is used to process the training data to obtain intermediate output data based on the adjusted second target network layer before the first target network layer in the initial quantization network, and to process the intermediate output data based on the first target network layer to obtain first output data. The processing module is used to process the training data to obtain second output data based on the network layers to be quantized in the neural network to be quantized, which correspond to the first target network layer and the second target network layer, respectively. An adjustment module is used to adjust the network parameters of the first target network layer based on the first output data and the second output data to obtain the quantized target neural network corresponding to the network to be quantized; and to use the target neural network to perform target detection on the lidar point cloud data collected by the lidar deployed on the autonomous driving device. The adjustment module is specifically used for: The bias value of the first target network layer is adjusted based on the first output data and the second output data; and / or the weight value of the first target network layer is adjusted based on the first output data, the second output data and the rounding mask parameter to be trained, wherein the rounding mask parameter is used to round the weight value of the first target network layer. The adjustment module is specifically used for: Based on the first output data and the second output data, determine the bias adjustment value corresponding to the first target network layer; The bias value of the first target network layer is adjusted based on the aforementioned bias adjustment value; Determine the first loss value of the first target network layer, and update the first output data based on the first target network layer after the bias value adjustment if the first loss value does not meet the first preset condition; Based on the updated first output data and second output data, return to the step of determining the bias adjustment value until the first loss value satisfies the first preset condition; The training data includes sample images, the first output data includes a first feature map, and the second output data includes a second feature map. The adjustment module is specifically used for: Based on the values ​​of each channel of the first feature map and the second feature map after activation processing, and the mask, the bias adjustment value is determined. The mask is used to filter the channel values ​​of the first feature map and the second feature map when calculating the bias adjustment value. The mask is determined based on the boundary parameters of the first target network layer when performing preliminary range adjustment.

10. A target detection device, characterized in that, include: The acquisition module is used to acquire the point cloud data to be detected; The detection module is used to detect the point cloud data to be detected based on the target neural network obtained by the point cloud target detection method as described in any one of claims 1 to 7, and to determine the detection result of the point cloud data to be detected. The control module is used to control the driving of the target vehicle based on the detection results.

11. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the point cloud target detection method as described in any one of claims 1 to 7, or the steps of the target detection method as described in claim 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the point cloud target detection method as described in any one of claims 1 to 7, or performs the steps of the target detection method as described in claim 8.

Images