Vehicle-mounted message anomaly detection method and device, electronic equipment and storage medium

Abstract

The invention relates to a vehicle-mounted message anomaly detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a training sample which comprises a sample vehicle-mounted message and a hard tag indicating whether the sample vehicle-mounted message is abnormal or not; the training sample is adopted to construct and train a teacher model, a MobileNet layer in the teacher model is used for extracting spatial features of the sample vehicle-mounted message, and a gating circulation unit in the teacher model is used for capturing time sequence features of the sample vehicle-mounted message; according to the hard label of the training sample and a soft label output by the teacher model, knowledge distillation learning is carried out on an initial student model, and the soft label refers to category probability distribution of the sample vehicle-mounted message; and detecting whether the target vehicle-mounted message is abnormal or not by adopting the trained student model. According to the invention, both the message detection accuracy and the high calculation efficiency are considered.

Description

Technical Field

[0001] The present application relates to the field of data detection technology, and in particular to a method, device, electronic device and storage medium for detecting anomalies in vehicle-mounted messages. Background Art

[0002] Existing methods for in-vehicle message detection mainly rely on rule-based, statistical or traditional machine learning models. Although these methods are simple to implement, they have obvious limitations. Rule-based methods have poor flexibility and are difficult to adapt to new abnormal patterns; statistical methods are insensitive to sudden abnormalities and require a large amount of historical data; and complex deep learning models can improve detection accuracy, but due to their high computational complexity and resource consumption, they are difficult to deploy efficiently in resource-constrained in-vehicle environments. Therefore, existing technologies cannot simultaneously take into account both the accuracy of message detection and the computational efficiency of the model. Summary of the invention

[0003] The present application provides a method, device, electronic device and storage medium for detecting anomalies in vehicle messages to solve the problem of being unable to take into account both the accuracy of message detection and the computational efficiency of the model at the same time.

[0004] In a first aspect, the present application provides a method for detecting anomalies of vehicle-mounted messages, the method comprising: Acquire a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; The training samples are used to construct and train a teacher model, wherein the MobileNet layer in the teacher model is used to extract the spatial features of the sample vehicle messages, and the gated recurrent unit in the teacher model is used to capture the time series features of the sample vehicle messages; Performing knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle-borne messages; Use the trained student model to detect whether the target vehicle message is abnormal; Wherein, using the training samples to construct and train the teacher model includes: Mapping the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; Extracting global and local features of the image using the MobileNet layer of the teacher model, and converting the image into a MobileNet output vector; Capturing the long-range dependency of the MobileNet output vector through an update gate and a reset gate in the gated recurrent unit to obtain a gated output vector; Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain a trained teacher model; Wherein, performing knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model includes: Determine the initial MobileNet-GRU student model with a single layer of convolutional kernels and a single layer of gated recurrent units; The output of the teacher model is converted into a soft label by adjusting the temperature coefficient and using a softmax function to obtain a distillation loss function, wherein the temperature coefficient being greater than a first threshold value indicates that the distribution area of ​​the soft label is uniform, and the temperature coefficient being less than the first threshold value indicates that the distribution of the soft label is close to the distribution of the hard label; Training the initial student model according to the hard labels of the training samples to obtain a hard label loss function; A total loss function is obtained by weighting the distillation loss function and the hard label loss function, and a trained student model is determined according to a loss value of the total loss function.

[0005] Optionally, extracting the overall features and local features of the image using the MobileNet layer of the teacher model, and converting the image into a MobileNet output vector comprises: Extracting local features of the image through the deep convolution kernel of the MobileNet layer to obtain a first feature map; Performing a point-by-point convolution operation on the first feature map to obtain a second feature map, wherein the feature expression capability of the second feature map is greater than the feature expression capability of the first feature map; Performing a global maximum pooling operation on the second feature map to obtain a third feature map containing important features; Performing depth convolution on the third feature map using the depth convolution kernel until a fourth feature map having a set number of channels is obtained; A dropout operation and a global average pooling operation are performed on the fourth feature map to obtain a MobileNet output vector.

[0006] Optionally, capturing the long-range dependency of the MobileNet output vector through an update gate and a reset gate in the gated recurrent unit to obtain a gated output vector includes: Processing the MobileNet output vector and the hidden state at the previous moment through an update gate in the gated recurrent unit to obtain a first weight value at the current moment, wherein the first weight value is used to determine how to mix the MobileNet output vector and the hidden state; Processing the MobileNet output vector and the hidden state at the previous moment through a reset gate in the gated recurrent unit to obtain a second weight value at the current moment, wherein the second weight value is used to determine non-important information that can be ignored in the hidden state; Adjusting the hidden state at the previous moment by using the second weight value, and determining a new candidate hidden state according to the MobileNet output vector; Performing a weighted combination of the hidden state at the previous moment and the candidate hidden state by using the first weight value to determine the hidden state at the current moment; Repeat the above steps in sequence to obtain the hidden state at each moment, and concatenate the hidden states at each moment to obtain the gated output vector.

[0007] Optionally, using the trained student model to detect whether the target vehicle message is abnormal includes: Inputting the target vehicle-borne message into the trained student model, wherein the trained student model is deployed in a vehicle gateway; Obtain a message category output by the trained student model, wherein the message category is used to indicate whether the target vehicle-mounted message is normal or abnormal.

[0008] Optionally, obtaining a training sample includes: Aligning initial vehicle-borne messages of different lengths, and converting character labels of the aligned messages into numerical labels through encoding to obtain sample vehicle-borne messages; The sample vehicle-borne message is subjected to base-to-base conversion and normalization processing to obtain a hard label.

[0009] In a second aspect, the present application provides a vehicle-mounted message anomaly detection device, the device comprising: An acquisition module, configured to acquire a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; A construction and training module, used to construct and train a teacher model using the training samples, wherein the MobileNet layer in the teacher model is used to extract the spatial features of the sample vehicle message, and the gated recurrent unit in the teacher model is used to capture the time series features of the sample vehicle message; A learning module, configured to perform knowledge distillation learning on an initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle-borne messages; A detection module, used to detect whether the target vehicle message is abnormal using the trained student model; Wherein, the construction and training module is used for: Mapping the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; Extracting global and local features of the image using the MobileNet layer of the teacher model, and converting the image into a MobileNet output vector; Capturing the long-range dependency of the MobileNet output vector through an update gate and a reset gate in the gated recurrent unit to obtain a gated output vector; Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain a trained teacher model; Wherein, the learning module is used for: Determine the initial MobileNet-GRU student model with a single layer of convolutional kernels and a single layer of gated recurrent units; The output of the teacher model is converted into a soft label by adjusting the temperature coefficient and using a softmax function to obtain a distillation loss function, wherein the temperature coefficient being greater than a first threshold value indicates that the distribution area of ​​the soft label is uniform, and the temperature coefficient being less than the first threshold value indicates that the distribution of the soft label is close to the distribution of the hard label; Training the initial student model according to the hard labels of the training samples to obtain a hard label loss function; A total loss function is obtained by weighting the distillation loss function and the hard label loss function, and a trained student model is determined according to a loss value of the total loss function.

[0010] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0011] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the method for detecting anomalies of in-vehicle messages described in any one of the above items of the present application.

[0012] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: In the present application, a complex MobileNet-GRU teacher model is first constructed, which can simultaneously extract the spatial and temporal features of the vehicle-borne message to ensure the accuracy of message anomaly detection. Then, through the knowledge distillation technology, a lightweight student model is trained using the soft labels generated by the complex teacher model and the original hard labels. The student model can obtain richer feature information from the soft labels while learning the real category information of the hard labels, and better understand the similarities between categories, thereby improving classification accuracy. The lightweight student model can not only use less computing power to improve computing efficiency, but also ensure the accuracy of message detection, achieving a balance between message detection accuracy and high computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0016] Figure 1 A flow chart of a method for abnormal detection of vehicle-mounted messages provided in an embodiment of the present application; Figure 2 A schematic diagram of the training sample preprocessing process provided in the embodiment of the present application; Figure 3 A schematic diagram of a process for constructing a student model provided in an embodiment of the present application; Figure 4 A schematic diagram of a process for abnormal detection of an in-vehicle message provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an abnormality detection device for vehicle-mounted messages provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0018] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0019] In order to solve the problem in the prior art that it is difficult to balance message detection accuracy and model calculation efficiency, the embodiment of the present application adopts a lightweight MobileNet-GRU model for message anomaly detection. The lightweight model can improve the calculation efficiency while also improving the message detection accuracy according to the MobileNet-GRU model.

[0020] A method for detecting anomalies of an in-vehicle message in an embodiment of the present application may be executed by a processor or controller of the vehicle.

[0021] Some terms used in the embodiments of the present application are explained below, including the following: On-board message: A data packet transmitted via the CAN (Controller Area Network) bus or the LIN (Local Interconnect Network) bus, containing communication information between the vehicle's various sensors and controllers, such as speed, temperature, throttle position, etc.

[0022] Hard label: A binary label indicating whether a sample vehicle message is abnormal, usually 0 (normal) or 1 (abnormal).

[0023] Soft label: The category probability distribution output by the teacher model reflects the possibility that the sample vehicle message belongs to different categories and provides richer similarity information between categories.

[0024] Knowledge distillation learning: A model compression technique that achieves similar performance at a lower computational cost by allowing the student model to learn the knowledge of the teacher model.

[0025] Teacher model: A complex, high-performance model that generates high-quality soft labels to guide the learning of the student model.

[0026] Student model: A simplified model that learns the knowledge of the teacher model through knowledge distillation. It has lower computational complexity and is suitable for resource-constrained environments.

[0027] MobileNet layer: An efficient convolutional neural network architecture used to extract spatial features of images or data and reduce computing resource consumption.

[0028] Gated Recurrent Unit (GRU): A variant of recurrent neural network that effectively captures long-range dependencies in time series data by introducing a gating mechanism to control the flow of information.

[0029] The following will be combined with a specific implementation method to describe in detail a method for detecting abnormality of an in-vehicle message provided by an embodiment of the present application, taking application to a processor as an example. Figure 1 As shown, the specific steps are as follows: Step 101: obtaining a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; Step 102: constructing and training a teacher model using the training samples, wherein the MobileNet layer in the teacher model is used to extract the spatial features of the sample vehicle messages, and the gated recurrent unit in the teacher model is used to capture the time series features of the sample vehicle messages; Step 103: Perform knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle messages; Step 104: Use the trained student model to detect whether the target vehicle message is abnormal.

[0030] In step 101, the processor collects a large number of messages from the vehicle communication system as training samples. These messages contain real-time communication information between various sensors and controllers of the vehicle, such as speed, temperature, throttle position, etc. Each message has a hard label to indicate whether the message is abnormal. The hard label is usually a binary value, where 0 represents a normal message and 1 represents an abnormal message. These labels can be annotated by experts or automatically generated through historical data analysis.

[0031] In step 102, the processor builds a complex teacher model, which includes two parts: a MobileNet layer and a gated recurrent unit (GRU). The MobileNet layer is used to extract the spatial features of the vehicle message and capture the static pattern in the message, while the GRU is used to capture the time series features and process the time dependency of the message. The processor uses the cross entropy loss function to train the teacher model to optimize its classification performance for normal and abnormal messages.

[0032] The processor uses the trained teacher model to predict the training samples and generate soft labels. Soft labels refer to the probability distribution of sample vehicle messages belonging to different categories, reflecting the similarities and differences between categories. The trained teacher model has powerful feature extraction and sequence modeling capabilities. On the one hand, it can provide high-quality soft labels to help the student model better learn the subtle differences between categories. On the other hand, it can make full use of rich features to improve detection accuracy.

[0033] In step 103, the student model is a simplified version of the MobileNet-GRU model with fewer parameters and lower computational complexity. The embodiment of the present application combines the soft label with the original hard label to jointly train the initial student model. Through knowledge distillation, the student model not only learns the true category information of the hard label, but also learns the soft label provided by the teacher model, thereby enhancing the robustness to noisy data and classification accuracy. The present application ensures that the student model can inherit the powerful feature extraction capability of the teacher model to accurately classify messages, and can simplify the calculation through a lightweight model to improve the calculation efficiency.

[0034] In step 104, the trained student model is converted to the TensorFlow Lite format (a lightweight model format for running TensorFlow models on specific devices), which is designed for running models on mobile and embedded devices, supports a variety of hardware accelerators, and is suitable for use on resource-limited automotive gateway devices. The trained student model is deployed on devices such as vehicle gateways to receive and process new target vehicle messages in real time. Based on the input target vehicle message, the student model outputs a prediction result of whether it is abnormal, providing timely feedback to the vehicle's safety monitoring system. If an abnormal message is detected, the system will trigger an alarm or take corresponding measures to ensure the stability and safety of the vehicle system.

[0035] The trained student model can run efficiently on embedded devices and quickly respond to potential security threats. The lightweight student model reduces computing resource consumption, allowing anomaly detection to achieve high accuracy and low latency in resource-constrained environments.

[0036] Exemplarily, there is a set of vehicle message data, each message contains an 8-byte data field. Some of these messages are normal and some are abnormal. The processor first uses the teacher model (a complex MobileNet-GRU model) to classify these messages and generate soft labels. For example, for a message, the teacher model may output [0.9, 0.1], indicating that the message has a 90% chance of being normal and a 10% chance of being abnormal. Next, these soft labels are used together with the original hard labels (0 or 1) to train a simplified student model. By learning the soft labels of the teacher model, the student model can not only learn how to distinguish between normal and abnormal messages, but also understand the subtle differences between the two. Ultimately, this lightweight student model can run efficiently on the vehicle gateway, detect whether the newly received message is abnormal in real time, and ensure the safety and stability of the vehicle system.

[0037] In this application, we first build a complex MobileNet-GRU teacher model, which can simultaneously extract the spatial and temporal features of vehicle-borne messages to ensure the accuracy of message anomaly detection. Then, through knowledge distillation technology, we use the soft labels generated by the complex teacher model and the original hard labels to jointly train a lightweight student model, so that the student model can better understand the similarities between categories, thereby improving classification accuracy. The trained lightweight student model can not only use less computing power to improve computational efficiency, but also ensure message detection accuracy, achieving both message detection accuracy and high computational efficiency.

[0038] In addition, the lightweight student model takes up less storage space and memory, and requires less computing resources during inference, making it suitable for deployment on devices with limited storage and memory, such as mobile devices and embedded systems. Reduced computation and memory access times mean lower energy consumption, making it suitable for battery-powered devices.

[0039] As an optional implementation, in step 101, obtaining training samples includes the following contents: aligning initial vehicle messages of different lengths, and converting the character labels of the aligned messages into numerical labels through encoding to obtain sample vehicle messages; performing base conversion and normalization on the sample vehicle messages to obtain hard labels.

[0040] Figure 2It is the preprocessing process of training samples. The message supports multiple different data lengths. For example, the DLC (Data Length Code) of the CAN message can be up to 8 bytes. In order to facilitate the training of the neural network model, the initial vehicle messages of different lengths need to be standardized. First, determine whether the length of the initial vehicle message is equal to 8. If it is equal to 8, one-hot encoding is performed directly. If it is less than 8, zero is added to the end of its payload to ensure that all messages are of the same length.

[0041] The label field of the message data set is identified by character data, for example, normal represents normal messages and anomaly represents abnormal messages. In order to facilitate model processing, these character labels need to be converted into numeric labels to obtain sample vehicle messages. The encoding conversion rule is any one of the following rules: one rule is: normal is encoded as 0, indicating a normal message; anomaly is encoded as 1, indicating an abnormal message; another one-hot encoding rule is: normal is encoded as [1,0], indicating a normal message; anomaly is encoded as [0,1], indicating an abnormal message.

[0042] The sample vehicle message is converted to binary format to ensure the consistency of the data format; in order to scale the message data to the [0,1] interval and improve the efficiency and stability of model training, the maximum-minimum value normalization method is used. The specific conversion formula is as follows: ,in, represents the i-th feature, represents the specific value of the i-th feature in the k-th sample, Represents the normalized value. The normalized value is the hard label.

[0043] As an optional implementation, in step 102, using training samples to construct and train a teacher model includes the following: Step S11: Mapping the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; Step S12: using the MobileNet layer of the teacher model to extract the overall features and local features of the image, and converting the image into a MobileNet output vector; Step S13: Capture the long-range dependency of the MobileNet output vector through the update gate and reset gate in the gated recurrent unit to obtain a gated output vector; Step S14: Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain a trained teacher model.

[0044] Each vehicle message can be up to 8 bytes, such as [0x1A, 0x2B, 0x3C, 0x4D, 0x5E, 0x6F, 0x70, 0x81]. The 8 bytes are the one-dimensional time series features to be processed. The processor maps the one-dimensional time series features of each vehicle message to a two-dimensional space to form a grayscale or color image. For example, each byte of data can be converted into a pixel value and arranged in a matrix form according to specific rules. This ensures that all generated images have the same size (such as 224*224) and are normalized so that the pixel values ​​are in the [0,1] interval, which is convenient for subsequent convolutional neural network processing.

[0045] For example, there is an 8-byte vehicle message data as follows: [0x1A, 0x2B, 0x3C, 0x4D, 0x5E, 0x6F, 0x70, 0x81]. Each byte can be expanded into a 16*16 small block. All pixel values ​​in each block are set to the value of the byte, so a total of 16*16*8 small blocks are obtained. These blocks are arranged into a 16*128 matrix with 16*128=2048 pixels. The interpolation method is used to expand the 16*128 matrix to 224*224. The image is normalized so that the pixel value is in the [0,1] interval.

[0046] The embodiment of the present application converts one-dimensional time series data into two-dimensional images, and can use mature image processing technology and convolutional neural networks (such as MobileNet) to extract complex spatiotemporal features and improve feature expression capabilities. In addition, data in image form can more intuitively display patterns and structures in messages, which helps to capture potential abnormal patterns.

[0047] The processor uses the MobileNet layer in the teacher model to perform multi-step convolution operations on the generated image to extract its overall and local features, and finally converts the extracted feature map into a one-dimensional feature vector (MobileNet output vector) for subsequent processing. The MobileNet layer can efficiently extract rich spatiotemporal features, significantly improving the model's ability to understand complex patterns. By converting images into feature vectors, it provides strong support for subsequent time series modeling.

[0048] The processor then inputs the MobileNet output vector into the gated recurrent unit (GRU) in the teacher model. GRU can capture dynamic changes in time series by updating the gate and resetting the gate mechanism, ensuring that the model has a good understanding of long-term dependencies. By introducing the gating mechanism, the model can flexibly choose which historical information to retain and which information to ignore, thereby improving classification accuracy.

[0049] Finally, the processor converts the gated output vector into a prediction result representing the message category, and evaluates the gap between the model prediction result and the true label through a loss function (such as binary cross entropy loss). The loss function gives a value based on the difference between the predicted value and the true label. The smaller the value, the more accurate the model's prediction. By continuously adjusting the model parameters, the model performs better and better on the training data, and finally converges to a better state. Strict evaluation and verification ensure that the model not only performs well on the training data, but also maintains a high accuracy on unseen data.

[0050] This application converts one-dimensional time series features into two-dimensional images to facilitate feature extraction by the MobileNet network, and then extracts the overall and local features of the image through a convolutional neural network, and then captures the dynamic changes in the time series through a gating mechanism. This application constructs an efficient hybrid model by combining the MobileNet layer and the gated recurrent unit (GRU). This combination enables the model to simultaneously utilize the spatial and temporal features of the vehicle-borne messages to improve the accuracy and robustness of intrusion detection.

[0051] As an optional implementation, in step 103, performing knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model includes the following contents: Step S21: determining an initial MobileNet-GRU student model having a single-layer convolution kernel and a single-layer gated recurrent unit; Step S22: by adjusting the temperature coefficient, the output of the teacher model is converted into a soft label using the softmax function to obtain a distillation loss function, wherein a temperature coefficient greater than the first threshold indicates that the distribution area of ​​the soft label is uniform, and a temperature coefficient less than the first threshold indicates that the distribution of the soft label is close to the distribution of the hard label; Step S23: training the initial student model according to the hard labels of the training samples to obtain a hard label loss function; Step S24: Obtain a total loss function based on the weighted addition of the distillation loss function and the hard label loss function, and determine the trained student model based on the loss value of the total loss function.

[0052] Figure 3 The flowchart for constructing the student model shows that after the teacher model outputs the soft label, the initial student model is trained using the hard label and the soft label. The total loss function of the student model is obtained based on the distillation loss function and the hard label loss function, thereby training the student model. The process of student model training is as follows.

[0053] The processor builds a lightweight student model that consists of a single layer of convolutional kernels (for preliminary feature extraction) and a single layer of gated recurrent units (for capturing time series features).

[0054] Then the temperature coefficient T is introduced to adjust the output distribution of the teacher model. The temperature coefficient T controls the hardness of the probability distribution output by the Softmax function (soft maximum function). When T is larger, the Softmax output is smoother and the category probability distribution area is uniform, indicating that the soft label is softer, such as [0.9, 0.1]. When T is smaller, the Softmax output is sharper and close to the distribution of hard labels, indicating that the soft label is closer to the true label, such as 0 or 1. Using the adjusted temperature coefficient T, the output of the teacher model is converted to soft labels, and the distillation loss function is calculated based on these soft labels. The distillation loss function measures the difference between the output of the student model and the soft label of the teacher model. Through temperature coefficient adjustment, the soft label can provide more similarity information between categories, helping the student model to better understand the nuances of the category and improve classification accuracy.

[0055] The calculation formula of the soft label is: ,in, is a soft label, T is the temperature coefficient, represents the unnormalized prediction score of the input sample belonging to the i-th category, represents the unnormalized prediction score that the input sample belongs to the jth category, and k is the total number of samples.

[0056] The formula for the distillation loss function is:

[0057] in, represents the distillation loss function, is the i-th output of the teacher model at temperature T, is the i-th output of the student model under the temperature coefficient T, and k is the total number of samples.

[0058] The processor uses the true labels (i.e. hard labels) of the training samples to supervise the training of the student model. The hard label is a binary value indicating whether the message is abnormal (0 for normal, 1 for abnormal). An appropriate loss function (such as binary cross entropy loss) can be defined to evaluate the gap between the student model's prediction results and the hard label. The loss function gives a numerical value based on the difference between the predicted value and the true label. The smaller the value, the more accurate the model's prediction.

[0059] The formula for the hard label loss function is:

[0060] in, is the hard label loss function, is a hard label vector, represents the i-class output of the unsoftened student model, and k is the total number of samples.

[0061] The distillation loss function and the hard label loss function are weighted together to form a total loss function. The choice of weights can be adjusted according to the specific application scenario to balance the impact of soft labels and hard labels. By combining the distillation loss and the hard label loss, the total loss function can comprehensively evaluate the performance of the student model to ensure that it not only inherits the knowledge of the teacher model, but also can accurately classify actual samples.

[0062] The formula for the total loss function is:

[0063] in, is the total loss function, is a hyperparameter that can be tuned empirically to a reference or dynamically.

[0064] In this application, the processor first determines an initial student model with a single-layer convolution kernel and a single-layer gated recurrent unit to ensure its efficiency and adaptability, and then adjusts the temperature coefficient T and uses the Softmax function to convert the output of the teacher model into a soft label to generate a distillation loss function to flexibly control the hardness of the soft label. The student model is then trained according to the hard labels of the training samples to obtain a hard label loss function to ensure that the model can accurately classify actual samples. Finally, a total loss function is formed by weighted combination of the distillation loss function and the hard label loss function, and the trained student model is determined based on the loss value of the total loss function.

[0065] As an optional implementation, in step S12, determining the MobileNet output vector includes the following contents: Step S121: extracting local features of the image through the deep convolution kernel of the MobileNet layer to obtain a first feature map; Step S122: performing a point-by-point convolution operation on the first feature map to obtain a second feature map, wherein the feature expression capability of the second feature map is greater than the feature expression capability of the first feature map; Step S123: performing a global maximum pooling operation on the second feature map to obtain a third feature map containing important features; Step S124: performing depth convolution on the third feature map using a depth convolution kernel until a fourth feature map having a set number of channels is obtained; Step S125: Perform a dropout operation and a global average pooling operation on the fourth feature map to obtain a MobileNet output vector.

[0066] The processor converts the sample message data into an image as input to ensure that the input format is uniform and suitable for subsequent processing. Use the deep convolution kernel to perform preliminary extraction of local features of the image. Deep convolution processes each pixel channel by channel to capture local texture and edge information in the image and generate the first feature map. Deep convolution can preliminarily extract local features in the image while maintaining computational efficiency, laying the foundation for subsequent more complex feature extraction. This stage focuses on capturing the basic structure and pattern of the image.

[0067] The processor applies a point-by-point convolution operation to expand the number of channels of the first feature map to obtain a second feature map. Point-by-point convolution not only improves the feature expression capability, but also maintains computational efficiency. By increasing the number of channels, the model can capture more complex and rich feature information, enhance the feature expression capability, and enable the model to better understand different types of local features. Point-by-point convolution significantly enhances the feature expression capability, allowing the model to capture a wider variety of local features, providing richer information for subsequent feature fusion and classification tasks.

[0068] The processor performs a global maximum pooling operation on the second feature map to extract the most important features and obtain a third feature map. Maximum pooling selects the maximum value of each local area, reducing the spatial dimension of the feature map while retaining key information. Through the maximum pooling operation, the model can effectively reduce the spatial dimension of the feature map while retaining the most significant features, improving computational efficiency and enhancing the robustness of the features. Global maximum pooling reduces the spatial dimension of the feature map while retaining the most important features, improving the computational efficiency and robustness of the model, and enabling the model to focus on the most representative features.

[0069] The processor applies the deep convolution operation again to convolve the third feature map, gradually extracting deeper features. Finally, the fourth feature map is obtained. Further extract more complex features and increase the number of channels to capture more details. Through multiple steps of deep convolution operations, the model can gradually extract deeper features, enhance the ability to understand complex patterns, and ensure that the model can capture more detailed information, thereby improving classification accuracy.

[0070] The processor performs a dropout operation on the fourth feature map, randomly setting some neurons to zero to prevent overfitting and improve the generalization ability of the model. Dropout, as a regularization technique, can effectively prevent the model from over-relying on certain specific features during training and enhance the robustness of the model. The fourth feature map after the dropout operation is converted into a one-dimensional feature vector through global average pooling, which not only reduces the spatial dimension of the feature map, but also provides a smoother and more stable feature representation, ensuring the quality of the feature vector.

[0071] Exemplarily, the processing flow of the MobileNet layer includes the following: 1. Initial input: Convert the CAN message data into an image with a dimension of 225*225*3 as input, ready for feature extraction.

[0072] 2. Deep convolution: Use a 3*3*3 deep convolution kernel to extract local features and obtain a 222*222*3 feature map. This step preliminarily extracts local features in the image.

[0073] 3. Point-by-point convolution: Apply a 1*1*32 point-by-point convolution operation to expand the number of channels from 3 to 32, and obtain a 222*222*32 feature map. Point-by-point convolution enhances feature expression capabilities while maintaining computational efficiency.

[0074] 4. Global maximum pooling: Perform global maximum pooling on the 222*222*32 feature map to extract the most important features and obtain a 111*111*32 feature map. Maximum pooling reduces the spatial dimension while retaining key information.

[0075] 5. Repeated convolution: Apply the depthwise convolution operation again to convolve the 111*111*32 feature map, and finally get a 108*108*64 feature map. This step further extracts more complex features and increases the number of channels to capture more details.

[0076] 6. Dropout operation: Perform dropout operation on the 108*108*64 feature map, randomly set some neurons to zero, prevent overfitting, and improve the generalization ability of the model.

[0077] 7. Global average pooling and reshaping: The feature map is converted into a one-dimensional feature vector through global average pooling, and adjusted to a format suitable for GRU network input (1*64 tensor) through reshape operation. This step provides a smoother and more stable feature representation.

[0078] MobileNet is an efficient convolutional neural network architecture that uses deep separable convolution to reduce the amount of calculation and parameters. The embodiment of the present application can efficiently and accurately extract the spatial features of the vehicle-mounted CAN message through the MobileNet layer.

[0079] As an optional implementation, in step S13, determining the gate output vector includes: Step S131: Processing the MobileNet output vector and the hidden state at the previous moment through the update gate in the gated recurrent unit to obtain a first weight value at the current moment, wherein the first weight value is used to determine how to mix the MobileNet output vector and the hidden state; Step S132: Processing the MobileNet output vector and the hidden state at the previous moment through the reset gate in the gated recurrent unit to obtain a second weight value at the current moment, wherein the second weight value is used to determine non-important information that can be ignored in the hidden state; Step S133: adjusting the hidden state at the previous moment by the second weight value, and determining a new candidate hidden state according to the MobileNet output vector; Step S134: performing a weighted combination of the hidden state at the previous moment and the candidate hidden state by using the first weight value to determine the hidden state at the current moment; Step S135: Repeat the above steps in sequence to obtain the hidden state at each moment, and connect the hidden state at each moment in series to obtain the gated output vector.

[0080] In the GRU layer, the Update Gate is first introduced, which is responsible for determining how the output vector from the current moment of the MobileNet layer and the hidden state of the previous moment are combined. This process involves linear transformation of the two and applying an activation function (usually a Sigmoid function) to generate the first weight value zt. The first weight value zt represents how much the model should retain the old state information (hidden state) at the current point in time, and how much it should adopt the new input information (i.e., the MobileNet output vector). In other words, zt determines the mixing ratio between new information and old information. Through a carefully adjusted information fusion mechanism, it is ensured that the model can not only learn long-term dependencies from past hidden states, but also appropriately introduce new feature information, avoid forgetting important historical data or accepting new information too early, thereby improving the expressiveness and accuracy of the model.

[0081] Next is the role of the reset gate, which also calculates the second weight value rt based on the current MobileNet output vector and the hidden state of the previous moment. This weight value is used to evaluate which parts of the old state can be ignored because it indicates the degree of unimportant information in the old state. By applying rt to the hidden state of the previous moment, GRU can selectively forget less relevant past information in order to better focus on the novelty and changes brought about by the current input. The reset gate enables GRU to have more sophisticated memory management capabilities, which can effectively filter out unnecessary information, reduce the burden on the model, enhance its memory and understanding of key information, and improve the model's depth of understanding of sequence data.

[0082] Using the hidden state adjusted by the reset gate and the current MobileNet output vector, GRU calculates a new candidate hidden state This step usually includes the application of a tanh activation function to ensure that the candidate states fall within a reasonable range of values.

[0083] Finally, by updating the first weight value zt of the gate, GRU will update the hidden state of the previous moment and the candidate hidden state just calculated Perform a weighted combination to determine the final hidden state ht at the current moment.

[0084] Through the above steps, GRU can flexibly adjust its internal state at each time point, which not only maintains effective modeling of long time series, but also can quickly respond to short-term changes. This dynamic characteristic makes GRU very suitable for processing data sets with complex spatiotemporal structures, such as video frame sequences or time series prediction tasks.

[0085] Exemplarily, the calculation process of the gated recurrent unit is as follows: 1. Update the door to calculate the door state information at time t.

[0086] in, represents the first weight value, is the sigmoid function, is the parameter matrix of the update gate, is the hidden state at time t-1, is the final 1*64 tensor obtained in the MobileNet network layer. is the bias vector for the update gate.

[0087] 2. Reset the gate to calculate the gate state information at time t.

[0088] in, represents the second weight value, is the sigmoid function, To reset the gate parameter matrix, is the hidden state at time t-1, is the final 1*64 tensor obtained in the MobileNet network layer. is the bias vector for the reset gate.

[0089] 3. Reset the gate to calculate the candidate hidden state.

[0090]

[0091] in, represents the candidate hidden state, W is the parameter matrix for linear transformation reset gate, represents the second weight value, is the hidden state at time t-1, is the final 1*64 tensor obtained in the MobileNet network layer, and b is the bias vector used to adjust the result after linear transformation.

[0092] 4. The update gate calculates the current hidden state.

[0093] in, Indicates the current hidden state. represents the first weight value, is the hidden state at time t-1, represents a candidate hidden state.

[0094] 5. MobileNet-GRU model loss function.

[0095]

[0096] Among them, loss is the loss function, n is the total number of samples, is the hard label value (0 or 1) of the i-th sample, It is the predicted value of the model for the i-th sample, that is, the probability that the model predicts the label value of the i-th sample is 1. In the hard label, 0 represents normal CAN message information and 1 represents abnormal message information.

[0097] This application processes the MobileNet output vector through the update gate and reset gate in GRU, which can efficiently capture the long-range dependencies of image features evolving over time.

[0098] As an optional implementation, in step 104, using the trained student model to detect whether the target vehicle-mounted message is abnormal includes: inputting the target vehicle-mounted message into the trained student model, wherein the trained student model is deployed in the vehicle gateway; obtaining the message category output by the trained student model, wherein the message category is used to indicate whether the target vehicle-mounted message is normal or abnormal.

[0099] The student model trained by knowledge distillation is deployed in the vehicle gateway, and the pre-processed target vehicle message is directly input into the student model. Since the student model is optimized and can run efficiently in a resource-constrained environment, real-time processing can be achieved. Based on the input target vehicle message, the student model outputs a probability distribution, indicating the possibility that the message belongs to the normal or abnormal category, and then feeds back the judgment result to the vehicle safety monitoring system or other related modules so that further measures can be taken (such as triggering alarms, recording logs, automatic repair, etc.).

[0100] This application provides a schematic diagram of an abnormality detection process for vehicle-mounted messages, such as Figure 4 As shown, the following steps are included.

[0101] Step 401: Preprocess the initial vehicle-borne message to obtain a training sample.

[0102] Step 402: Convert training samples into images.

[0103] Step 403: Use the image to build and train the teacher model of MobileNet-GRU.

[0104] Step 404: Determine whether the model accuracy meets the requirements. If not, return to step 403 to retrain the model. If yes, execute step 405.

[0105] Step 405: Save the teacher model.

[0106] Step 406: Use the soft labels output by the teacher model and the hard labels of the training samples to perform knowledge distillation and train the student model.

[0107] Step 407: The trained student model is deployed to the vehicle gateway.

[0108] Step 408: Use the deployed student model to determine whether the message data is abnormal.

[0109] Based on the same technical concept, the present application provides an abnormality detection device for vehicle-mounted messages, such as Figure 5 As shown, the device comprises: An acquisition module 501 is used to acquire a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; A construction and training module 502 is used to construct and train a teacher model using training samples, wherein the MobileNet layer in the teacher model is used to extract spatial features of sample vehicle messages, and the gated recurrent unit in the teacher model is used to capture time series features of the sample vehicle messages; A learning module 503 is used to perform knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle-borne messages; A detection module 504 is used to detect whether the target vehicle message is abnormal using the trained student model; Optionally, the construction and training module 502 is used to: Map the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; The MobileNet layer of the teacher model is used to extract the overall and local features of the image and convert the image into a MobileNet output vector; The long-range dependency of the MobileNet output vector is captured by the update gate and reset gate in the gated recurrent unit to obtain the gated output vector. Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain the trained teacher model; Optionally, the learning module 503 is used to: Determine the initial MobileNet-GRU student model with a single layer of convolutional kernels and a single layer of gated recurrent units; By adjusting the temperature coefficient, the output of the teacher model is converted into soft labels using the softmax function to obtain the distillation loss function, where a temperature coefficient greater than the first threshold indicates that the distribution area of ​​the soft labels is uniform, and a temperature coefficient less than the first threshold indicates that the distribution of the soft labels is close to that of the hard labels; Train the initial student model according to the hard labels of the training samples to obtain the hard label loss function; The total loss function is obtained by weighting the distillation loss function and the hard label loss function, and the trained student model is determined according to the loss value of the total loss function.

[0110] Optionally, the construction and training module 502 is used to: The local features of the image are extracted through the deep convolution kernel of the MobileNet layer to obtain the first feature map; Performing a point-by-point convolution operation on the first feature map to obtain a second feature map, wherein the feature expression capability of the second feature map is greater than the feature expression capability of the first feature map; Perform a global maximum pooling operation on the second feature map to obtain a third feature map containing important features; Performing depth convolution on the third feature map using a depth convolution kernel until a fourth feature map with a set number of channels is obtained; Perform dropout operation and global average pooling operation on the fourth feature map to obtain the MobileNet output vector.

[0111] Optionally, the construction and training module 502 is used to: Processing the MobileNet output vector and the hidden state at the previous moment through an update gate in the gated recurrent unit to obtain a first weight value at the current moment, wherein the first weight value is used to determine how to mix the MobileNet output vector and the hidden state; The MobileNet output vector and the hidden state at the previous moment are processed by the reset gate in the gated recurrent unit to obtain a second weight value at the current moment, wherein the second weight value is used to determine non-important information that can be ignored in the hidden state; The hidden state at the previous moment is adjusted by the second weight value, and a new candidate hidden state is determined according to the MobileNet output vector; Performing a weighted combination of the hidden state at the previous moment and the candidate hidden state by using a first weight value to determine the hidden state at the current moment; Repeat the above steps in sequence to obtain the hidden state at each moment, and concatenate the hidden states at each moment to obtain the gated output vector.

[0112] Optionally, the detection module 504 is used to: Inputting the target vehicle message into the trained student model, wherein the trained student model is deployed in the vehicle gateway; Get the message category output by the trained student model, where the message category is used to indicate whether the target vehicle message is normal or abnormal.

[0113] Optionally, the acquisition module 501 is used to: Aligning initial vehicle-borne messages of different lengths, and converting character labels of the aligned messages into numerical labels through encoding to obtain sample vehicle-borne messages; The sample vehicle messages are converted and normalized to obtain hard labels.

[0114] like Figure 6 As shown, an embodiment of the present application provides an electronic device, including a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0115] The memory 603 is used to store computer programs.

[0116] In one embodiment of the present application, the processor 601 is used to implement the vehicle-borne message anomaly detection method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 603.

[0117] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for detecting anomalies of in-vehicle messages provided in any of the aforementioned method embodiments are implemented.

[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0120] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0121] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for detecting anomalies of vehicle-borne messages, characterized in that: The method comprises: Acquire a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; The training samples are used to construct and train a teacher model, wherein the MobileNet layer in the teacher model is used to extract the spatial features of the sample vehicle messages, and the gated recurrent unit in the teacher model is used to capture the time series features of the sample vehicle messages; Performing knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle-borne messages; Use the trained student model to detect whether the target vehicle message is abnormal; Wherein, using the training samples to construct and train the teacher model includes: Mapping the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; Extracting global and local features of the image using the MobileNet layer of the teacher model, and converting the image into a MobileNet output vector; Capturing the long-range dependency of the MobileNet output vector through an update gate and a reset gate in the gated recurrent unit to obtain a gated output vector; Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain a trained teacher model; Wherein, performing knowledge distillation learning on the initial student model according to the hard labels of the training samples and the soft labels output by the teacher model includes: Determine the initial MobileNet-GRU student model with a single layer of convolutional kernels and a single layer of gated recurrent units; The output of the teacher model is converted into a soft label by adjusting the temperature coefficient and using a softmax function to obtain a distillation loss function, wherein the temperature coefficient being greater than a first threshold value indicates that the distribution area of ​​the soft label is uniform, and the temperature coefficient being less than the first threshold value indicates that the distribution of the soft label is close to the distribution of the hard label; Training the initial student model according to the hard labels of the training samples to obtain a hard label loss function; A total loss function is obtained by weighting the distillation loss function and the hard label loss function, and a trained student model is determined according to a loss value of the total loss function.

2. The method according to claim 1, characterized in that Using the MobileNet layer of the teacher model to extract the overall features and local features of the image, and converting the image into a MobileNet output vector includes: Extracting local features of the image through the deep convolution kernel of the MobileNet layer to obtain a first feature map; Performing a point-by-point convolution operation on the first feature map to obtain a second feature map, wherein the feature expression capability of the second feature map is greater than the feature expression capability of the first feature map; Performing a global maximum pooling operation on the second feature map to obtain a third feature map containing important features; Performing depth convolution on the third feature map using the depth convolution kernel until a fourth feature map having a set number of channels is obtained; A dropout operation and a global average pooling operation are performed on the fourth feature map to obtain a MobileNet output vector.

3. The method according to claim 1, characterized in that: The long-range dependency of the MobileNet output vector is captured by the update gate and the reset gate in the gated recurrent unit, and the gated output vector is obtained, including: Processing the MobileNet output vector and the hidden state at the previous moment through an update gate in the gated recurrent unit to obtain a first weight value at the current moment, wherein the first weight value is used to determine how to mix the MobileNet output vector and the hidden state; Processing the MobileNet output vector and the hidden state at the previous moment through a reset gate in the gated recurrent unit to obtain a second weight value at the current moment, wherein the second weight value is used to determine non-important information that can be ignored in the hidden state; Adjusting the hidden state at the previous moment by using the second weight value, and determining a new candidate hidden state according to the MobileNet output vector; Performing a weighted combination of the hidden state at the previous moment and the candidate hidden state by using the first weight value to determine the hidden state at the current moment; Repeat the above steps in sequence to obtain the hidden state at each moment, and concatenate the hidden states at each moment to obtain the gated output vector.

4. The method according to claim 1, characterized in that: The trained student model is used to detect whether the target vehicle message is abnormal, including: Inputting the target vehicle-borne message into the trained student model, wherein the trained student model is deployed in a vehicle gateway; Obtain a message category output by the trained student model, wherein the message category is used to indicate whether the target vehicle-mounted message is normal or abnormal.

5. The method according to claim 1, characterized in that Obtaining training samples includes: Aligning initial vehicle-borne messages of different lengths, and converting character labels of the aligned messages into numerical labels through encoding to obtain sample vehicle-borne messages; The sample vehicle-borne message is subjected to base-to-base conversion and normalization processing to obtain a hard label.

6. A vehicle-borne message anomaly detection device, characterized in that: The device comprises: An acquisition module, configured to acquire a training sample, wherein the training sample includes a sample vehicle message and a hard label indicating whether the sample vehicle message is abnormal; A construction and training module, used to construct and train a teacher model using the training samples, wherein the MobileNet layer in the teacher model is used to extract the spatial features of the sample vehicle message, and the gated recurrent unit in the teacher model is used to capture the time series features of the sample vehicle message; A learning module, configured to perform knowledge distillation learning on an initial student model according to the hard labels of the training samples and the soft labels output by the teacher model, wherein the soft labels refer to the category probability distribution of the sample vehicle-borne messages; A detection module, used to detect whether the target vehicle message is abnormal using the trained student model; Wherein, the construction and training module is used for: Mapping the one-dimensional time series features in the sample vehicle message into a two-dimensional space to form an image; Extracting global and local features of the image using the MobileNet layer of the teacher model, and converting the image into a MobileNet output vector; Capturing the long-range dependency of the MobileNet output vector through an update gate and a reset gate in the gated recurrent unit to obtain a gated output vector; Adjust the loss value based on the difference between the detection result corresponding to the gated output vector and the true label to obtain a trained teacher model; Wherein, the learning module is used for: Determine the initial MobileNet-GRU student model with a single layer of convolutional kernels and a single layer of gated recurrent units; The output of the teacher model is converted into a soft label by adjusting the temperature coefficient and using a softmax function to obtain a distillation loss function, wherein the temperature coefficient being greater than a first threshold value indicates that the distribution area of ​​the soft label is uniform, and the temperature coefficient being less than the first threshold value indicates that the distribution of the soft label is close to the distribution of the hard label; Training the initial student model according to the hard labels of the training samples to obtain a hard label loss function; A total loss function is obtained by weighting the distillation loss function and the hard label loss function, and a trained student model is determined according to a loss value of the total loss function.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-5 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images