Training method of image classification model, determination method of driving state, and vehicle

Abstract

The application provides a training method of an image classification model, a driving state determination method and a vehicle. The method comprises the following steps: obtaining a plurality of actual face images of a plurality of users; performing feature extraction on the actual face images by using a pre-constructed image classification model to obtain actual face features; performing a feature conversion operation on the actual face features, and constructing a synthetic face image according to the actual face features subjected to the feature conversion operation; determining other actual face images with the same classification label as the synthetic face image from the plurality of actual face images as associated face images; and updating model parameters of the image classification model based on the synthetic face image and the associated face images, so that the classification labels obtained by the image classification model when classifying the actual face images and the associated face images are different. The technical problem that the image classification model is not accurate in classification in the prior art is solved, and the accuracy of the image classification model in classification is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a training method for an image classification model, a method for determining driving status, and a vehicle. Background Technology

[0002] With the continuous development of science and technology, the demand for image classification is growing rapidly, and its application areas are expanding. Image classification often relies on image classification models. However, traditional image classification models suffer from inaccurate classification. They have poor recognition capabilities for images with minimal differences in the input, failing to accurately distinguish between them, thus resulting in inaccurate classification results. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a training method for an image classification model, a method for determining driving state, and a vehicle, so as to solve the technical problem that the image classification model is not accurate.

[0004] To achieve the above objectives, this application provides a training method for an image classification model, comprising: acquiring multiple actual facial images of multiple users; extracting features from the actual facial images using a pre-constructed image classification model to obtain actual facial features; performing feature transformation operations on the actual facial features; constructing a synthetic facial image based on the actual facial features after the feature transformation operation, wherein the classification labels of the synthetic facial image and its corresponding actual facial image after the feature transformation operation are different; identifying other actual facial images with the same classification labels as the synthetic facial image as associated facial images among the multiple actual facial images; updating the model parameters of the image classification model based on the synthetic facial image and the associated facial image, so that the classification labels obtained by the image classification model when classifying the actual facial images and the associated facial images are different.

[0005] Optionally, the actual facial features include appearance features and state features; the step of performing feature transformation operation on the actual facial features and constructing a synthetic facial image based on the actual facial features after the feature transformation operation includes: performing a vector transformation operation on the state features to obtain state transformation features, wherein the state transformation features are the opposite of the state described by the state features; and constructing the synthetic facial image based on the appearance features and the state transformation features.

[0006] Optionally, updating the model parameters of the image classification model based on the synthesized facial image and the associated facial image includes: constructing a loss function for the image classification model with the goal of maximizing the similarity between the synthesized facial image and the associated facial image; and minimizing the loss function to update the model parameters of the image classification model.

[0007] Optionally, the method further includes: acquiring a predetermined number of target facial images of the target user, and labeling the target facial images to generate labeled tags; using the target facial images to perform a first iteration of training on the image classification model, so that the predicted classification tag obtained by the image classification model after the first iteration of training when classifying the target facial images is the same as the labeled tag.

[0008] Optionally, before acquiring a predetermined number of target facial images of the target user, the method further includes: acquiring multiple sets of actual facial images of at least one user, wherein the number of actual facial images in each set is different; performing a second iteration of training on the image classification model using each set of actual facial images, so that the classification accuracy of the image classification model after the second iteration of training for each set of actual facial images exceeds a preset threshold; and determining the number of images in the actual facial image set corresponding to the maximum classification accuracy as the predetermined number.

[0009] This application also provides a method for determining driving status, comprising: acquiring at least one target facial image of a target user; inputting the at least one target facial image into an image classification model trained using the above-described image classification model training method, and outputting a classification label for each target facial image through the image classification model; and determining the driving status of the target user based on at least one classification label.

[0010] Optionally, determining the driving status of the target user based on at least one classification label includes: determining the driving status of the target user according to the frequency corresponding to each classification label.

[0011] Optionally, determining the driving state of the target user based on at least one classification label includes: inputting at least one classification label into a pre-built driving state classification model, and outputting the driving state of the target user via the driving state classification model.

[0012] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor, when executing the computer program, implements methods such as a training method for an image classification model or a method for determining a driving state.

[0013] Based on the same inventive concept, this application also provides a vehicle that includes the electronic device.

[0014] As can be seen from the above, the image classification model training method, driving state determination method, and vehicle provided in this application include acquiring multiple actual facial images of multiple users, extracting features from the actual facial images using a pre-constructed image classification model to obtain actual facial features, which accurately describe all features contained in the actual facial images. A feature transformation operation is performed on the actual facial features, and a synthetic facial image is constructed based on the actual facial features after the feature transformation operation. The synthetic facial image and its corresponding actual facial image after the feature transformation operation have different classification labels; that is, the feature transformation operation yields a synthetic facial image with a different classification label than the actual facial image, so that the image classification model can be trained subsequently based on the synthetic facial image. Among multiple real facial images, other real facial images with the same classification label as the synthesized facial image are identified as associated facial images. An image classification model is then used to learn the feature relationships between the synthesized facial images and associated facial images with the same classification label, improving the feature extraction capability of the image classification model. The model parameters are then updated so that the updated model can recognize that the synthetic facial images and associated facial images have the same classification label. Since the synthetic facial images and real facial images have different classification labels, the image classification model can accurately identify that the associated facial images and real facial images have different classification labels. When the associated facial images and real facial images are only slightly different, the training method of the image classification model proposed in this application can still ensure that the classification labels output by the image classification model are different from those of the real facial images, achieving the goal of accurately identifying facial images with small differences, thereby improving the classification accuracy of the image classification model. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the training method of the image classification model according to an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating the method for determining the driving state according to an embodiment of this application;

[0018] Figure 3This is a schematic diagram of the structure of a training device for an image classification model according to an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the structure of the driving state determination device according to an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] As described in the background section, with the rapid development of computer technology, image classification technology has an increasingly wide range of applications. For example, in the field of intelligent driving, accurately identifying the state category of a driver's facial image and issuing corresponding warnings helps improve driver safety. Image classification needs are often met by image classification models. The training process of an image classification model is a supervised learning process, typically consisting of two phases: a training phase and a testing phase. In the training phase, images labeled with pre-defined categories are used as input, and a classifier is generated by learning the relationship between image features and category labels. In the testing phase, the trained classifier is used to classify new images.

[0024] In existing technologies, traditional image classification models suffer from inaccurate classification. Traditional image classification models have poor recognition performance for input images with minimal differences. For example, when the input image is a facial image with relatively small eyes, a traditional image classification model may not accurately identify facial images with relatively small eyes or images with closed eyes, thus resulting in inaccurate classification results.

[0025] In view of this, embodiments of this application propose a training method for an image classification model, referring to... Figure 1 This includes the following steps:

[0026] Step 101: Obtain multiple actual facial images of multiple users, and extract features from the actual facial images using a pre-built image classification model to obtain actual facial features.

[0027] In this step, multiple actual facial images from multiple users are acquired. The relatively large number of actual facial images from different users enriches the samples used in the training process of the image classification model, thereby improving the accuracy of the model. Before inputting these multiple actual facial images into the image classification model, each image can undergo preprocessing. The appropriate preprocessing operation is determined according to the training requirements of the image classification model. Preprocessing operations can be one or more of the following: scaling, cropping, rotation, and flipping. This ensures that the multiple actual facial images meet the input requirements of the image classification model while increasing sample diversity, thus laying the foundation for improving the training efficiency of the image classification model. It should be noted that all facial images input into the image classification model in this application require preprocessing operations, which will not be elaborated further below.

[0028] The actual facial image includes the facial features that the image classification model needs to classify. For example, when the facial feature that the image classification model needs to classify is the eyes, the actual facial image is a facial image that includes the eyes; when the facial feature that the image classification model needs to classify is the mouth, the actual facial image is a facial image that includes the mouth. If the actual facial image includes other facial features besides the target facial features needed to train the image classification model, then the target facial features need to be cropped from the actual facial image. For example, if the actual facial image includes all facial features, and the target facial feature is the eyes, then the user's left and right eye regions can be cropped using a face detection algorithm and a face feature point localization algorithm, and the cropped left and right eye images of the user are labeled respectively to distinguish between the user's left and right eyes. The actual facial image can be an image including the left eye in an open state or an image including the left eye in a closed state. Correspondingly, the actual facial image can be an image including the right eye in an open state or an image including the right eye in a closed state. Cropping facial features other than the eye area eliminates interference from irrelevant image information, allowing subsequent image classification models to accurately extract features from the target face. This improves both the accuracy and training efficiency of the image classification model.

[0029] After inputting the actual facial image into the image classification model, the image classification model is also needed to encode the actual facial image in order to perform lossless compression of the actual facial image. This reduces the amount of data that the image learning model needs to process and improves the processing efficiency of the image learning model. In this way, the actual facial features of the compressed actual facial image are obtained. The actual facial features can accurately describe all the features contained in the actual facial image.

[0030] Step 102: Perform feature transformation on the actual facial features, and construct a synthetic facial image based on the actual facial features after the feature transformation operation, wherein the classification labels of the synthetic facial image and its corresponding actual facial image after the feature transformation operation are different.

[0031] In this step, the actual facial features corresponding to the actual facial image undergo feature transformation to construct a synthetic facial image. It's important to note that feature transformation is required for each actual facial feature corresponding to the input image classification model. The purpose of constructing the synthetic facial image is to obtain a facial image with a different classification label than its corresponding feature-transformed actual facial image. Specifically, feature transformation is performed on the actual facial features corresponding to the actual facial image to obtain transformed features that differ from the actual facial features. The transformed features describe a different morphological state of the facial image than the actual facial features; therefore, the synthetic facial image constructed using the transformed features has a different classification label than the actual facial image. For example, the actual facial image is an image with open eyes, and the classification label is "eyes open"; the actual facial features describe an open-eye state, and the classification label is "eyes closed". The transformed features obtained through feature transformation change this state to a closed-eye state, and the synthetic facial image constructed using the transformed features is an image with closed eyes.

[0032] It's important to note that for users with small eye areas, the image classification model extracts features with and without eyes closed, resulting in low discriminative power between the two features. This indicates a poor feature extraction capability for similar images, making it unable to distinguish between open and closed eyes. In this step, the transformed feature is the one furthest from the actual facial features, meaning it completely differs from the actual eye state described by the facial features. The image classification model can learn from the transformed feature to improve its feature extraction capability for similar images, while obtaining two spatially distant features for images with completely different states.

[0033] Step 103: Among multiple actual facial images, identify other actual facial images with the same classification label as the synthesized facial image as associated facial images.

[0034] In this step, the synthesized facial image is constructed from the actual facial features corresponding to the actual facial image through a feature transformation operation. Because the actual facial features undergo feature transformation, the classification label of the synthesized facial image is not the same as that of the actual facial image. However, since multiple actual facial images contain multiple classification labels, some of these images contain actual facial images with the same classification label as the synthesized facial image. These actual facial images with the same classification label as the synthesized facial image are identified as associated facial images, and the image classification model can then be trained using the synthesized facial images and associated facial images. For example, if the synthesized facial image is an image with eyes open and the classification label is "eyes open," then an actual facial image with the same classification label "eyes open" is searched among the multiple actual facial images and identified as an associated facial image. Furthermore, since this application requires feature transformation operations for the actual facial features corresponding to each actual facial image input to the image classification model, multiple synthesized facial images can be obtained. For each synthesized facial image, it is necessary to determine its associated associated facial image.

[0035] It should be noted that, in order to ensure that the subsequent image classification model can accurately learn the relationship between the synthesized facial image and the associated facial image, the synthesized facial image and the associated facial image are facial images belonging to the same user.

[0036] Step 104: Update the model parameters of the image classification model based on the synthesized facial image and the associated facial image, so that the classification labels obtained by the image classification model when classifying the actual facial image and the associated facial image are different.

[0037] In this step, the image classification model learns the relationship between the features of the synthesized facial image and the associated facial image. As mentioned in the previous steps, the synthesized facial image and the associated facial image have the same classification label. The features corresponding to the synthesized facial image are obtained through a transformation operation, while the features of the associated facial image are obtained by encoding by the image classification model. Due to the poor feature extraction capability of the image classification model at this point, there are certain differences between these two features. To make the features extracted by the image classification model from the associated facial image closer to the features of the synthesized facial image, the model parameters of the image classification model need to be updated based on the synthesized facial image and the associated facial image. After the parameter update, the features extracted by the image classification model from the associated facial image have a high similarity to the features of the synthesized facial image. Consequently, the classification label output by the image classification model for the associated facial image is the same as the classification label for the synthesized facial image. Then, when the image classification model classifies the actual facial image and the associated facial image, the classification labels obtained are different, thus improving the recognition accuracy of the image classification model for facial images with high similarity.

[0038] The above scheme acquires multiple actual facial images from multiple users. A pre-built image classification model is used to extract features from these actual facial images to obtain actual facial features that accurately describe all the features contained in the actual facial images. Feature transformation is performed on these actual facial features, and a synthetic facial image is constructed based on these transformed features. The synthetic facial image and its corresponding actual facial image after feature transformation have different classification labels; that is, the feature transformation results in a synthetic facial image with a different classification label than the actual facial image. This allows for subsequent training of the image classification model based on the synthetic facial image. Other actual facial images with the same classification label as the synthetic facial image are identified as associated facial images. The image classification model is then used to learn the feature relationship between the synthetic facial images and the associated facial images with the same classification label, improving the feature extraction capability of the image classification model. The model parameters are updated so that the updated image classification model can recognize that the synthetic facial images and the associated facial images have the same classification label. Because the synthetic facial images have different classification labels than the actual facial images, the image classification model can accurately identify that the associated facial images have different classification labels than the actual facial images. When the associated facial image and the actual facial image are facial images with little difference, the training method of the image classification model proposed in this application can still make the classification labels of the associated facial image and the actual facial image output by the image classification model different, thus achieving the goal of accurately identifying facial images with little difference and improving the classification accuracy of the image classification model.

[0039] The method for constructing synthetic facial images will be explained in detail through the following examples.

[0040] In some embodiments, the actual facial features include appearance features and state features; the step of performing feature transformation operations on the actual facial features and constructing a synthetic facial image based on the actual facial features after the feature transformation operations includes: performing a vector transformation operation on the state features to obtain state transformation features, wherein the state transformation features are the opposite of the state described by the state features; and constructing the synthetic facial image based on the appearance features and the state transformation features.

[0041] In this embodiment, the actual facial features include appearance features and state features. For example, when the actual facial image is an eye image, appearance features are those characterizing eye size, and state features are those characterizing whether the eyes are open or closed. A vector transformation operation is performed on the state features. For example, a feature transformation formula is used to perform a vector transformation operation on the state features to obtain state features opposite to the current state features; that is, a state transformation feature is obtained. For example, the state features of the eyes are transformed from open state features to closed state features. The feature transformation formula is: f d ' ,x =f d,x ×M, Among them, f d ' ,x For the transformation feature, f d,x Let M be the state feature, M be the transpose of the feature vector, and x be the feature dimension of the state feature (e.g., the feature dimension can be 2). Through the feature transformation formula, the state transformation feature is made to be the opposite of the state described by the state feature. Based on the appearance features and the transformation feature, a synthetic facial image can be constructed. The synthetic facial image has the same appearance features as the actual facial image, but the state described by the state feature is reversed. For example, if the actual facial image is an image of the user with their eyes open, the synthetic facial image will be an image of the same user with their eyes closed. It should be noted that the image classification model can be a GAN (generative model) or a Stable diffusion model, etc.

[0042] The better the feature extraction capability, the higher the recognition accuracy of the image classification model. Therefore, in order to improve the recognition accuracy of the image classification model, it is necessary to construct a loss function for the image classification model and use the loss function to adjust the model parameters.

[0043] In some embodiments, updating the model parameters of the image classification model based on the synthesized facial image and the associated facial image includes: constructing a loss function for the image classification model with the goal of maximizing the similarity between the synthesized facial image and the associated facial image; and minimizing the loss function to update the model parameters of the image classification model.

[0044] In this embodiment, to improve the feature extraction capability of the image classification model, the spatial distance between the features of the synthesized facial image and the associated facial image is minimized by minimizing the loss function, which in turn maximizes the similarity between the features of the synthesized facial image and the associated facial image. By minimizing the loss function, the model parameters of the image classification model are updated, continuously improving the feature extraction accuracy of the image classification model. This makes the constructed synthesized facial image closely resemble the directly obtained actual facial image, thus improving the image classification model's ability to recognize facial images and consequently enhancing its classification accuracy. It should be noted that the loss function can be selected according to the actual training requirements of the model. For example, the loss function can be the gradient descent loss function; this embodiment does not limit the specific type of loss function.

[0045] It should be noted that, in order to simultaneously improve the feature extraction and classification capabilities of the image classification model, a fully connected layer can be added after the feature extraction layer. This fully connected layer determines the classification label and loss of the image classification model. With the goal of maximizing the similarity between the synthesized facial image and the associated facial image, and maximizing the accuracy of the classification category of the actual facial image, a loss function for the image classification model is constructed. This allows both the feature extraction and classification capabilities of the image classification model to be improved, further enhancing the classification accuracy.

[0046] The image classification model described above is trained based on actual facial images of multiple users. To improve the specificity of feature extraction of the image classification model, target facial images of specific target users can be obtained to further train the image classification model, so that the image classification model can accurately identify the features of the target user.

[0047] In some embodiments, the method further includes: acquiring a predetermined number of target facial images of a target user, and labeling the target facial images to generate labeled tags; using the target facial images to perform a first iteration of training on the image classification model, so that the predicted classification tag obtained by the image classification model after the first iteration of training when classifying the target facial images is the same as the labeled tag.

[0048] In this embodiment, after obtaining the trained image classification model, to improve the transferability of the image classification model and enable it to accurately classify the facial images of the target user, a predetermined number of target facial images of the target user can be obtained. These predetermined number of target facial images include images of the target user with the same appearance but opposite states; that is, the predetermined number of target facial images includes images of the same user with open eyes and images with closed eyes. For example, when the image classification model needs to classify eye images, it can guide the target user to perform corresponding open and closed eye actions through audio and video, thereby obtaining the predetermined number of target facial images of the target user. The target facial images are then labeled according to their states to obtain labeled tags. For example, if the target facial image is an open-eye image, the labeled tag is "open-eye"; if the target facial image is a closed-eye image, the labeled tag is "closed-eye". The image classification model is trained in its first iteration using a predetermined number of target facial images and pre-labeled images. During this first iteration, the model parameters are continuously updated until the output of the pre-defined classification label matches the pre-labeled label. The first iteration then stops. At this point, the image classification model can accurately identify the target user image, providing a reliable basis for judging the target user's driving status.

[0049] It should be noted that the two parameter updates for the image classification model described above can be performed on the same device or on different devices, depending on the user's actual application scenario. For example, in the field of intelligent driving, when classifying facial images of drivers, to quickly complete the first iteration of training, the pre-built image classification model can be pre-trained on a remote server. The first iteration of training can then be completed on a local vehicle infotainment system or edge device such as a mobile phone. Before the first iteration of training begins, the pre-built image classification model is obtained using the local vehicle infotainment system or edge device. This also helps protect the information security of the target user.

[0050] Furthermore, to ensure the image classification model can accurately learn the features of the target facial image, before the first iteration of training, the acquired target facial image needs to be inspected to determine whether it meets the input criteria of the image classification model. For example, if the target facial image is an image with closed eyes, it is necessary to detect whether the eye-closing action in the image is a preset eye-closing action, which is the action of completely closing the eyes. Iterative training of the image classification model using a predetermined number of target facial images of the target user updates the model parameters, enabling it to learn the features of the target user's target facial images more accurately, thus obtaining more accurate classification results and improving the user experience.

[0051] Since the number of target facial images for a target user is limited, determining a reasonable number of target facial images to input into the image classification model is crucial. The number of target facial images can be determined using a meta-learning model, and the process for determining the number of target facial images is described below.

[0052] In some embodiments, before acquiring a predetermined number of target facial images of a target user, the method further includes: acquiring multiple sets of actual facial images of at least one user, wherein the number of actual facial images in each set is different; performing a second iteration of training on the image classification model using each set of actual facial images, so that the classification accuracy of the image classification model after the second iteration of training for each set of actual facial images exceeds a preset threshold; and determining the number of images in the actual facial image set corresponding to the maximum classification accuracy as the predetermined number.

[0053] In this embodiment, to determine the optimal number of input target facial images, i.e., to determine a predetermined number, multiple sets of actual facial images need to be acquired, with a different number of actual facial images in each set. The actual facial images in each set are divided into a training set and a test set. Both the training set and the test set include a support set and a query set. The support set contains N1 classification labels, with K1 classified actual facial images for each label. The query set contains Q1 query actual facial images. The task of the training set is to use the learned features of the K1 actual facial images in the support set to classify the actual facial images in the query set into the corresponding categories. The support set in the test set contains N2 classification labels, with K2 classified actual facial images for each label. The query set contains Q2 query actual facial images. The task of the test set is to use the learned features of the K2 actual facial images in the support set to classify the actual facial images in the query set into the corresponding categories. It should be noted that the classification labels in the training set and the test set are not the same. The support set and query set in the training set are mainly used to learn model parameters and increase the generalization ability of the model parameters. The support set in the test set is mainly used to fine-tune the model parameters, while the query set is mainly used to test the performance of the final model. Even with a relatively small number of actual facial images, the image classification model has the ability to learn and can accurately classify images.

[0054] The image classification model is trained in a second iteration using each set of actual facial images to learn the features of each set of actual facial images. When the classification accuracy of the image classification model for each set of actual facial images reaches a preset threshold during the second iteration training, the second iteration training is stopped. At this point, it can be considered that the classification accuracy of the image classification model for each set of actual facial images is relatively high.

[0055] For each set of actual facial images, the number of actual facial images in the support set of the training set is k1, and the number of actual facial images in the query set is k2, where k1, k2 ∈ [k min k max For example, k min ≥2,k maxThe number of images in the second iteration of training is set to ≤20. When testing the performance of the image classification model using the query set in the test set, the model can be tested separately using the test set of each group of actual facial images to obtain the classification accuracy for each group. Higher classification accuracy indicates better second-iteration training of the image classification model. Therefore, the number of actual facial images in the group corresponding to the maximum classification accuracy is determined to be a predetermined number. This enhances the transferability of the image classification model, enabling it to achieve accurate classification results when facing new tasks, while also improving the training efficiency of the first iteration.

[0056] Based on the foregoing embodiments, the image classification model trained using the image classification model training method can accurately output the classification labels of facial images. Having determined the classification labels, this application proposes a method for determining driving status, referring to... Figure 2 This includes the following steps:

[0057] Step 201: Obtain at least one target facial image of the target user.

[0058] In this step, the user's driving status determines their driving safety; therefore, accurately determining the user's driving status is essential. The driving status is determined by analyzing the target user's facial images captured while driving the vehicle. The image acquisition device in the vehicle can capture the target user's facial images in real time during driving, and can also begin capturing facial images upon receiving a trigger operation. For example, the trigger operation could be receiving a command to activate the Driver Monitor System (DMS), or detecting that the vehicle's speed exceeds a predetermined speed.

[0059] Step 202: Input at least one target facial image into an image classification model trained using an image classification model training method, and output a classification label for each target facial image through the image classification model.

[0060] In this step, the image classification model described above can accurately extract features from the target facial image, thereby determining the correct classification label corresponding to the target facial image. The classification label can reflect the target user's driving state; for example, the classification label can be a label for open or closed eyes, and a label for open or closed mouth. By accurately extracting and classifying features from the target facial image, the subsequent determination of the target user's driving state becomes more accurate.

[0061] Step 203: Determine the driving status of the target user based on at least one classification label.

[0062] In this step, the driving state of the target user can be determined by analyzing at least one classification label. For example, the classification label may include a label indicating the target user's eyes are open or closed, and a label indicating the target user's mouth is open or closed. The driving state of the target user can be determined by analyzing the classification labels of the target user's facial image. Driving state is used to characterize the degree of fatigue driving of the target user, such as normal driving, slightly fatigued driving, and severely fatigued driving. Based on the user's driving state, the vehicle can issue corresponding prompts. For example, after determining that the target user's driving state is slightly fatigued, the driver monitoring system can provide a voice reminder to the target user; after determining that the target user's driving state is severely fatigued, the driver monitoring system can interact with the target user via voice. Prompts will only cease when the target user's driving state is detected to be normal or the vehicle is powered off. The system alerts the user to abnormal driving states, allowing the target user to make appropriate adjustments based on their driving state. Accurately determining the target user's driving state can help improve the user's driving safety.

[0063] The above scheme acquires at least one target facial image of the target user. This image is then input into an image classification model trained using the aforementioned method. The model outputs a classification label for each target facial image. Accurate feature extraction and classification of the target facial images improves the accuracy of determining the target user's driving status. Based on at least one classification label, the target user's driving status is determined. Accurately determining the target user's driving status can improve driving safety.

[0064] The following examples will provide a detailed explanation of the specific details of the method for determining driving status.

[0065] In some embodiments, determining the driving status of the target user based on at least one classification label includes: determining the driving status of the target user according to the frequency corresponding to each classification label.

[0066] In this embodiment, when the category label is the target user's open eyes label, if the target user's open eyes label is relatively infrequent within a predetermined time, the frequency of open eyes is relatively low, indicating that the target user's eye-closing time is relatively long. Therefore, the target user's driving status can be determined based on the frequency of open eyes. The lower the frequency of open eyes, the more dangerous the driving status. To improve the accuracy of determining driving status, the target user's eye and mouth states can be analyzed simultaneously. If the frequency of mouth opening is relatively high within a predetermined time, it indicates that the user may be frequently yawning, indicating relative fatigue. Therefore, the target user's driving status can be determined based on the frequency corresponding to each category label, allowing for subsequent appropriate reminders based on the driving status.

[0067] This application can determine the driving status of a target user not only by using the frequency corresponding to the classification label, but also by using a driving status classification model.

[0068] In some embodiments, determining the driving state of the target user based on at least one classification label includes: inputting at least one classification label into a pre-built driving state classification model, and outputting the driving state of the target user via the driving state classification model.

[0069] In this embodiment, the target user's driving state can also be determined based on a pre-built driving state classification model. This model can accurately learn the features of the classification labels, thereby obtaining a relatively accurate classification result of the driving state. By accurately determining the target user's driving state, the user's driving safety is ensured.

[0070] It should be noted that the above two methods for assessing the driving status of the target user can be selected based on the user's actual needs.

[0071] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0072] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0073] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a training device for an image classification model.

[0074] refer to Figure 3 The training device for the image classification model includes:

[0075] The feature extraction module 10 is configured to acquire multiple actual facial images of multiple users, and extract features from the actual facial images using a pre-built image classification model to obtain actual facial features.

[0076] The construction module 20 is configured to perform feature transformation operations on the actual facial features and construct a synthetic facial image based on the actual facial features after the feature transformation operation, wherein the classification labels of the synthetic facial image and its corresponding actual facial image after the feature transformation operation are different.

[0077] The first determining module 30 is configured to determine, among multiple actual facial images, other actual facial images with the same classification label as the synthetic facial image as associated facial images.

[0078] The update module 40 is configured to update the model parameters of the image classification model based on the synthesized facial image and the associated facial image, so that the image classification model obtains different classification labels when classifying the actual facial image and the associated facial image.

[0079] The aforementioned device acquires multiple actual facial images from multiple users. A pre-built image classification model is used to extract features from these actual facial images to obtain actual facial features that accurately describe all the features contained in the actual facial images. A feature transformation operation is performed on these actual facial features, and a synthetic facial image is constructed based on these transformed features. The synthetic facial image and its corresponding actual facial image after the feature transformation have different classification labels; that is, the feature transformation operation yields a synthetic facial image with a different classification label than the actual facial image, which is then used to train the image classification model. Other actual facial images with the same classification label as the synthetic facial image are identified as associated facial images. The image classification model is then used to learn the feature relationship between the synthetic facial images and the associated facial images with the same classification label, improving the feature extraction capability of the image classification model. The model parameters are then updated so that the updated image classification model can recognize that the synthetic facial images and the associated facial images have the same classification label. Because the synthetic facial images and the actual facial images have different classification labels, the image classification model can accurately identify that the associated facial images have different classification labels than the actual facial images. When the associated facial image and the actual facial image are facial images with little difference, the training method of the image classification model proposed in this application can still make the classification labels of the associated facial image and the actual facial image output by the image classification model different, thus achieving the goal of accurately identifying facial images with little difference and improving the classification accuracy of the image classification model.

[0080] In some embodiments, the construction module 20 is further configured to: the actual facial features include appearance features and state features; perform a vector transformation operation on the state features to obtain state transition features, wherein the state transition features are the opposite of the state described by the state features; and construct the synthetic facial image based on the appearance features and the state transition features.

[0081] In some embodiments, the update module 40 is further configured to construct a loss function for the image classification model with the goal of maximizing the similarity between the synthesized facial image and the associated facial image; and to minimize the loss function to update the model parameters of the image classification model.

[0082] In some embodiments, the system further includes a first training module, which is configured to acquire a predetermined number of target facial images of a target user, and to annotate the target facial images to generate labeled tags; and to perform a first iteration of training on the image classification model using the target facial images, so that the predicted classification tags obtained by the image classification model after the first iteration of training when classifying the target facial images are the same as the labeled tags.

[0083] In some embodiments, a second training module is further configured to acquire multiple sets of actual facial images of at least one user before acquiring a predetermined number of target facial images of the target user, wherein the number of actual facial images in each set is different; perform a second iteration training on the image classification model using each set of actual facial images, so that the classification accuracy of the image classification model after the second iteration training exceeds a preset threshold for each set of actual facial images; and determine the number of images in the actual facial image set corresponding to the maximum classification accuracy as the predetermined number.

[0084] refer to Figure 4 The device for determining the driving state includes:

[0085] The acquisition module 50 is configured to acquire at least one target facial image of the target user.

[0086] The output module 60 is configured to input at least one target facial image into an image classification model trained using the above-described image classification model training method, and output a classification label for each target facial image through the image classification model.

[0087] The second determining module 70 is configured to determine the driving status of the target user based on at least one classification label.

[0088] The aforementioned device acquires at least one target facial image of a target user. This image is then input into an image classification model trained using the aforementioned image classification model training method. The image classification model outputs a classification label for each target facial image. By accurately extracting and classifying the target facial images, the subsequent determination of the target user's driving status becomes more accurate. Based on at least one classification label, the target user's driving status is determined. Accurately determining the target user's driving status can improve driving safety.

[0089] In some embodiments, the second determining module 70 is further configured to determine the driving status of the target user based on the frequency corresponding to each category label.

[0090] In some embodiments, the second determining module 70 is further configured to input at least one classification label into a pre-built driving state classification model and output the driving state of the target user via the driving state classification model.

[0091] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0092] The apparatus of the above embodiments is used to implement the training method of the corresponding image classification model in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0093] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the training method of the image classification model described in any of the above embodiments.

[0094] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0095] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0096] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0097] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0098] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0099] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0100] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0101] The electronic devices described above are used to implement the training method of the corresponding image classification model in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0102] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the training method of the image classification model as described in any of the above embodiments.

[0103] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0104] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the training method of the image classification model as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0105] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0106] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0107] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0108] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A training method for an image classification model, characterized in that, include: Multiple actual facial images of multiple users are acquired, and features are extracted from the actual facial images using a pre-built image classification model to obtain actual facial features, wherein the actual facial features include appearance features and state features; The actual facial features are subjected to a feature transformation operation, which includes: performing a vector transformation operation on the state features to obtain state transformation features, wherein the transformed features are different from the morphological state of the facial image described by the actual facial features; A synthetic facial image is constructed based on the actual facial features after the feature transformation operation, wherein the classification labels of the synthetic facial image and the corresponding actual facial image after the feature transformation operation are different, the appearance features of the synthetic facial image are the same as those of the actual facial image, and the states described by the state features are opposite. Among multiple real facial images, other real facial images with the same classification labels as the synthesized facial image are identified as associated facial images; The model parameters of the image classification model are updated based on the synthesized facial image and the associated facial image, so that the image classification model obtains different classification labels when classifying the actual facial image and the associated facial image; The step of updating the model parameters of the image classification model based on the synthesized facial image and the associated facial image includes: constructing a loss function for the image classification model with the goal of maximizing the similarity between the synthesized facial image and the associated facial image; and minimizing the loss function to update the model parameters of the image classification model.

2. The method according to claim 1, characterized in that, The process involves performing feature transformation on the actual facial features and constructing a synthetic facial image based on the transformed facial features. include: The state transition feature is the opposite of the state described by the state feature; The synthetic facial image is constructed based on the appearance features and the state transition features.

3. The method according to claim 1, characterized in that, The method further includes: Acquire a predetermined number of target facial images of the target user, and annotate the target facial images to generate labeled tags; The image classification model is trained in a first iteration using the target facial image, so that the predicted classification label obtained by the image classification model after the first iteration of training is the same as the labeled label.

4. The method according to claim 3, characterized in that, Before acquiring a predetermined number of target facial images of the target user, the method further includes: Acquire multiple sets of actual facial images of at least one user, wherein the number of actual facial images in each set is different; The image classification model is trained in a second iteration using each set of actual facial images, so that the classification accuracy of the image classification model after the second iteration exceeds a preset threshold for each set of actual facial images. The number of images in the actual facial image group corresponding to the maximum classification accuracy is determined as the predetermined number.

5. A method for determining driving state, characterized in that, include: Acquire at least one target facial image of the target user; At least one target facial image is input into an image classification model trained using the image classification model training method as described in any one of claims 1-4, and the image classification model outputs a classification label for each target facial image. The driving status of the target user is determined based on at least one classification label.

6. The method according to claim 5, characterized in that, Determining the driving status of the target user based on at least one classification label includes: The driving status of the target user is determined based on the frequency of each category label.

7. The method according to claim 5, characterized in that, Determining the driving status of the target user based on at least one classification label includes: At least one classification label is input into a pre-built driving state classification model, and the driving state of the target user is output through the driving state classification model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4 or 5 to 7.

9. A vehicle, characterized in that, Includes the electronic device as described in claim 8.

Images