Image processing model training method and device, equipment and medium

Abstract

The invention discloses an image processing model training method and device, equipment and a medium. The method comprises the following steps: inputting sample face data into a face feature extraction unit in a to-be-trained image processing model to obtain sample face features; the image processing model further comprises a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit; inputting the sample face features into a gender judgment unit for gender judgment; inputting the age label into a style feature extraction unit to obtain a style feature representing the age; inputting the style features and the sample face features into a feature enhancement unit to obtain enhanced features; inputting the enhanced features into an age prediction unit for age prediction; and determining the final loss of the image processing model according to the real age, the gender judgment result and the age prediction result, and updating the parameters of the image processing model according to the final loss. According to the embodiment of the invention, the accuracy of age prediction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a training method, device, equipment and medium for an image processing model. Background Art

[0002] Existing in-vehicle intelligent human-computer interaction mainly focuses on voice recognition technology. Users issue commands through voice, and the in-vehicle terminal recognizes the voice commands and performs corresponding operations. However, existing in-vehicle recommendation systems have not yet fully utilized the personal information of drivers and passengers to predict user preferences and make effective recommendations. Summary of the invention

[0003] The present invention provides a training method, device, equipment and medium for an image processing model to solve the problem that an in-vehicle system cannot perform effective personalized recommendations using only voice information.

[0004] According to one aspect of the present invention, a method for training an image processing model is provided, comprising:

[0005] Obtain sample facial data of drivers and passengers;

[0006] Inputting the sample face data into a face feature extraction unit in an image processing model to be trained to obtain sample face features; the image processing model further includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit;

[0007] Inputting the sample facial features into the gender determination unit to perform gender determination;

[0008] Inputting the age label into the style feature extraction unit to obtain the style feature representing the age;

[0009] Inputting the style feature and the sample face feature into the feature enhancement unit to obtain enhanced features;

[0010] Inputting the enhanced features into the age prediction unit to perform age prediction;

[0011] According to the real age, gender judgment result and age prediction result, the final loss of the image processing model is determined, and the parameters of the image processing model are updated according to the final loss.

[0012] According to another aspect of the present invention, there is provided a training device for an image processing model, comprising:

[0013] A data acquisition module is used to acquire sample facial data of the driver and passenger; the sample facial data is input into a facial feature extraction unit in the image processing model to be trained to obtain sample facial features; the image processing model also includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit;

[0014] A gender determination module, used for inputting the sample facial features into the gender determination unit for gender determination;

[0015] A style feature acquisition module, used for inputting the age label into the style feature extraction unit to obtain a style feature representing the age;

[0016] An enhanced feature acquisition module, used for inputting the style feature and the sample face feature into the feature enhancement unit to obtain an enhanced feature;

[0017] An age prediction module, used for inputting the enhanced features into the age prediction unit to perform age prediction;

[0018] A parameter updating module is used to determine the final loss of the image processing model according to the real age, gender judgment result and age prediction result, and to update the parameters of the image processing model according to the final loss.

[0019] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the training method of the image processing model according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the image processing model described in any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the training method of the image processing model described in any embodiment of the present invention when executed by a processor.

[0025] The embodiment of the present invention can simultaneously handle gender judgment and age prediction tasks by constructing an image processing model to perform gender judgment and age prediction on facial images of vehicle occupants. A style feature extraction unit is introduced into the age prediction task, and the age label is input into the style feature extraction unit to obtain style features that characterize age. The style features that characterize age and sample facial features are input into the feature enhancement unit to obtain enhanced features. Since the enhanced features incorporate more age-related details, they can more accurately reflect the actual age information of the face, so that the model can more accurately capture these subtle differences when predicting age, thereby improving the accuracy of age prediction.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1A is a first flow chart of a training method for an image processing model provided by an embodiment of the present invention;

[0029] Figure 1B is a structural schematic diagram of an image processing model provided by an embodiment of the present invention;

[0030] Figure 2 is a second flow chart of the training method of the image processing model provided by an embodiment of the present invention;

[0031] Figure 3 is a flow chart of a method for processing an image of a person on board a vehicle provided by an embodiment of the present invention;

[0032] Figure 4 It is a structural schematic diagram of a training device for an image processing model provided by an embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of the structure of an electronic device implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Figure 1A This is the first flow chart of the training method of the image processing model provided by the embodiment of the present invention. This embodiment is applicable to the case of training the image processing model. The method can be executed by the training device of the image processing model. The device can be implemented in the form of hardware and / or software. The device can be configured in an electronic device with corresponding data processing capabilities. Figure 1A As shown, the method includes:

[0037] S110, obtaining sample facial data of the driver and passenger; inputting the sample facial data into a facial feature extraction unit in an image processing model to be trained to obtain sample facial features; the image processing model also includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit.

[0038] The facial images of the drivers and passengers in the vehicle are captured by the camera, and the captured facial images are preprocessed, including grayscale, face detection, face alignment, etc. to obtain facial data. Specifically, facial images of different age groups and genders are collected, and the age and gender are annotated. The annotated data is: [face image, age, gender], where age (age) is annotated as any integer [0, A], that is, age is annotated as any integer between 0 and A, including 0 and A; gender (gender) is annotated as 0 or 1. Using advanced face detection algorithms, the collected facial images are analyzed, the faces are accurately located and cropped, and [x-min, y-min, x-max, y-max] are annotated in the facial images. (x-min, y-min) and (x-max, y-max) are used to represent the face frame for subsequent processing. The facial key point detection algorithm is used to detect multiple key points of the face, including the left and right eyes, nose, and both sides of the mouth, a total of five key points. The head posture is determined by the position information of key points to ensure accurate recognition when the head posture changes. The size, grayscale, equalization and other conditions of the face image are adjusted to ensure the consistency of the input data. After data normalization, the data set is divided into training set: test set = 8:2 or 9:1.

[0039] Build an image processing model to determine the gender and predict the age of the input face data. Figure 1B The image processing model includes a face feature extraction unit, a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit. Sample face data in the test set is obtained, and the sample face data is input into the face feature extraction unit in the image processing model to be trained to obtain sample face features.

[0040] S120: Input the sample facial features into the gender determination unit to perform gender determination.

[0041] The gender judgment unit includes at least one classification layer, which is used to classify the facial features to obtain the gender and probability of the facial data.

[0042] S130: Input the age label into the style feature extraction unit to obtain a style feature representing the age.

[0043] S140: Input the style feature and the sample face feature into the feature enhancement unit to obtain enhanced features.

[0044] Exemplarily, the style feature extraction unit includes an encoder and a multi-layer perceptron; the encoder encodes the age label and transmits the encoding result to the multi-layer perceptron; the multi-layer perceptron learns the style features corresponding to each value of the age label.

[0045] Among them, the age labels corresponding to the sample faces are all integers of [0, A], that is, the age labels are all integers between 0 and A, including 0 and A. The style feature extraction unit encodes the age labels and then passes them to the multi-layer perceptron for in-depth age feature learning to obtain style features that characterize the styles of each age. The style features that characterize the styles of each age and the sample face features are input into the feature enhancement unit to obtain enhanced features. Since the enhanced features incorporate more age-related details, they can more accurately reflect the actual age information of the face. The model can more accurately capture these subtle differences when predicting age, thereby improving the accuracy of age prediction.

[0046] Optionally, the step of inputting the style feature and the sample facial feature into the feature enhancement unit to obtain an enhanced feature includes: performing matrix multiplication on the style feature and the sample facial feature to obtain an enhanced feature.

[0047] Through matrix multiplication, the style features representing the styles of each age can be directly injected into the sample face features. This helps to enhance the features in the face image that are closely related to age, such as skin texture, wrinkle distribution, facial contour, etc., so that the sample face features can represent the age more finely and discriminately. At the same time, matrix multiplication is an efficient mathematical operation method that can complete the processing of a large amount of data in a short time. Therefore, the method of matrix multiplication of the style features representing age with the sample face features has high computational efficiency and can meet the needs of real-time processing or large-scale data processing. Moreover, this method has high flexibility and can adjust the style feature matrix according to different application scenarios and needs. For example, when it is necessary to emphasize the characteristics of a specific age group, the weights and parameters in the matrix can be adjusted accordingly. This flexibility makes this method applicable to a variety of different computer vision tasks.

[0048] S150: Input the enhanced features into the age prediction unit to perform age prediction.

[0049] S160. Determine the final loss of the image processing model according to the actual age, gender judgment result and age prediction result, and update the parameters of the image processing model according to the final loss.

[0050] Gender judgment is regarded as a binary classification problem. According to the gender judgment result, a loss function is defined for the gender judgment task. For age prediction, a combination of classification and regression is adopted. Classification is used to predict age, and regression is used to predict age by weight. According to the real age, the age classification prediction result and the age regression prediction result, a loss function is defined for the age prediction task. The final loss function of the image processing model is determined according to the loss function of the gender judgment task and the loss function of the age prediction task, and the parameters of the image processing model are updated according to the final loss of the image processing model.

[0051] The embodiment of the present invention can simultaneously handle gender judgment and age prediction tasks by constructing an image processing model to perform gender judgment and age prediction on facial images of vehicle occupants. A style feature extraction unit is introduced into the age prediction task, and the age label is input into the style feature extraction unit to obtain style features that characterize age. The style features that characterize age and sample facial features are input into the feature enhancement unit to obtain enhanced features. Since the enhanced features incorporate more age-related details, they can more accurately reflect the actual age information of the face, so that the model can more accurately capture these subtle differences when predicting age, thereby improving the accuracy of age prediction.

[0052] Figure 2 2 is a second flow chart of the training method of the image processing model provided by the embodiment of the present invention. This embodiment optimizes and improves the final loss of the image processing model according to the real age, gender judgment result and age prediction result based on the above embodiment. Figure 2 As shown, the method includes:

[0053] S210, obtaining sample facial data of the driver and passenger; inputting the sample facial data into a facial feature extraction unit in an image processing model to be trained to obtain sample facial features; the image processing model also includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit.

[0054] S220: Input the sample facial features into the gender determination unit to perform gender determination.

[0055] S230: Input the age label into the style feature extraction unit to obtain a style feature representing the age.

[0056] S240: Input the style feature and the sample face feature into the feature enhancement unit to obtain enhanced features.

[0057] S250: Input the enhanced features into the age prediction unit to perform age prediction.

[0058] S260. Determine a first loss based on the gender judgment result in the current iteration round; and obtain a mean absolute error between the current predicted age and the actual age in the current iteration round as the second loss.

[0059] Refer to the following formula (1), treat gender judgment as a binary classification problem, and calculate the cross entropy loss function based on the gender judgment result as the loss function of the gender judgment task. Calculate the cross entropy loss based on the gender judgment result in the current iteration.

[0060] L Gender =-Σ i y i log(p i ) (1)

[0061] where y i is the real gender label, with a value of 0 or 1; p i The model predicts the result to be y i The probability of i is 0 or 1, indicating the value of the true gender label. The first loss is determined based on the cross entropy loss corresponding to the gender judgment results of the sample face data of all training sets.

[0062] Construct age labels, which are all integers in [0, A], that is, all integers between 0 and A, including 0 and A. The age prediction task is regarded as a multi-label classification task. In each iteration, the predicted probability of the current sample face data belonging to each value of the age label is predicted, and the age label value with the highest prediction probability is taken as the predicted age. The error is calculated based on the real age and the predicted age. Refer to the following formula (2) and use the mean absolute error (MAE) as the loss function of age prediction, i.e., the second loss.

[0063]

[0064] where y n is the real age, y n ' is the predicted age, and N is the total number of samples.

[0065] S270. Determine a third loss according to the current prediction probability, the reference distribution probability and the first distribution function in the current iteration round.

[0066] The value of the first distribution function corresponding to the age label is used as the reference distribution probability; the third loss is determined according to the relative entropy between the current predicted probability and the reference distribution probability. Exemplarily, the age label x is 0, 1, 2, 3, ..., A; the first distribution function of any sample face data in the current iteration round is:

[0067] Among them, σ is 2, μ is 3, which is the real age corresponding to the current sample face data. The value of the first distribution function corresponding to each value of the age label is determined, which is the reference distribution probability.

[0068] The prediction probability of any sample face data in the current iteration round is the probability of predicting the current sample face data as the corresponding value of the age label. For example, the prediction result of the current sample face data is (0: 0.01, 1: 0.03, 2: 0.05, 3: 0.6, ..., A: 0.01), which means that the probability of predicting the current sample face data as 0 is 0.01, the probability of predicting the current sample face data as 1 is 0.03, ..., the probability of predicting the current sample face data as A is 0.01, and the sum of the prediction probabilities corresponding to the various values ​​of the age label is 1. The third loss is determined according to the relative entropy between the reference distribution probability and the prediction probability corresponding to each value of the age label.

[0069] By taking age distribution as a new prediction target for face age prediction, the age prediction task essentially becomes a multi-label classification problem, which enables the model to better handle the uncertainty in age prediction and improve the accuracy of prediction. Traditional age prediction methods often regard age as a single regression value, while the introduction of age distribution function allows the model to optimize on multiple adjacent ages to find a more appropriate prediction result. Fuzzy learning from real age and adjacent ages not only considers the actual age, but also the information of adjacent ages. The distribution function can represent the degree of description of each age to the corresponding face image. Each face image can not only help learn its actual age, but also help learn the adjacent ages, which improves the effectiveness of the training images of each age. By assigning a distribution function to each training sample, each image can provide training information for multiple adjacent ages. The model can increase the number of training images for each age without increasing the total number of training images. Since the distribution function contains the information of each age, the model can learn age characteristics more comprehensively during the training process and reduce the overfitting problem caused by insufficient or unbalanced data.

[0070] In an optional embodiment, if the current iteration is not the first iteration of the image processing model, the third loss is determined based on the current prediction probability, the reference distribution probability and the first distribution function in the current iteration round, including: obtaining the previous prediction probability in the previous iteration round; determining the first distribution function based on the age label and the previous prediction probability; taking the value of the first distribution function corresponding to the age label as the reference distribution probability; and obtaining the third loss based on the relative entropy between the current prediction probability and the reference distribution probability.

[0071] Specifically, referring to formulas (3)-(5), the previous prediction probabilities corresponding to the various values ​​of the age labels in the prediction results of the previous iteration round are taken as the reference distribution probabilities, and the mean and variance are calculated according to the age labels and the reference distribution probabilities, so as to construct the age distribution function of the previous iteration round, and take the age distribution function of the previous iteration round as the first distribution function.

[0072] μ t-t =∑ i x(i)*p t-1 (i),i∈[0,A] (3)

[0073]

[0074] Among them, x(i) represents the values ​​of age labels, p t-1 (i) is the predicted probability of the age label taking the value i in the prediction result of the previous iteration round t-1. t-1 (i) is the first distribution function.

[0075] The values ​​of the first distribution function corresponding to each value of the age label are used as reference distribution probabilities. The third loss is obtained based on the relative entropy between each reference distribution probability and the current prediction probability corresponding to each value of the age label in the current iteration round. The specific formula is shown in (6):

[0076]

[0077] Where N is the number of training samples; p t (i) is the predicted probability of the age label taking the value i in the prediction result of the current iteration t-1.

[0078] In an optional embodiment, if the current iteration is the first iteration of the image processing model, the third loss is determined based on the current predicted probability, the reference distribution probability and the first distribution function in the current iteration round, including: taking the initial distribution function of the real age as the first distribution function; taking the value of the first distribution function corresponding to the age label as the reference distribution probability; and obtaining the third loss based on the relative entropy between the current predicted probability and the reference distribution probability.

[0079] If the current iteration is the first iteration of the image processing model, refer to formula (7), the real age can be used as the mean, the variance can be 2, and the initial distribution function can be constructed as the reference distribution function. The value of the first distribution function corresponding to the age label is used as the reference distribution probability. According to the relative entropy between the current prediction probability and the reference distribution probability, the third loss is obtained.

[0080]

[0081] Where σ is set to 2; μ is the real age corresponding to the current sample face data; x(i) represents the various values ​​of the age label.

[0082] By introducing the label distribution self-learning training method in the model training process, the neural network can adaptively and gradually estimate the age label distribution during the refinement process. The prediction results of the previous iteration on the age label are combined with the prediction results of the current iteration to construct the loss function, and the estimation of the age label distribution is gradually adjusted to better reflect the distribution of the real age label. This inheritance and adaptive learning process promotes the performance improvement of model training in age prediction and improves the accuracy of model prediction.

[0083] S280. Determine a fourth loss according to the current predicted probability, the current predicted distribution probability and a second distribution function; wherein the second distribution function is determined according to the age label and the current predicted probability.

[0084] Specifically, referring to formulas (8)-(11), the mean and variance are calculated based on the age label and the current prediction probability corresponding to each value of the age label in the prediction result of the current iteration round, so as to construct the age distribution function of the current iteration round, and use the age distribution function of the current iteration round as the second distribution function. The values ​​of the second distribution function corresponding to each value of the age label are used as the current prediction distribution probability. The fourth loss is obtained based on the relative entropy between each current prediction distribution probability and the current prediction probability corresponding to each value of the age label in the current iteration round. The specific formula is as follows:

[0085] μ t =∑ i x(i)*p t (i),i∈[0,A] (8)

[0086]

[0087] Among them, x(i) represents the values ​​of age labels, p t (i) is the predicted probability of the age label taking the value i in the prediction result of the current iteration round t. t (i) is the second distribution function. N is the number of training samples.

[0088] S290. Determine a final loss of the image processing model according to the first loss, the second loss, the third loss, and the fourth loss.

[0089] Specifically, the third loss and the fourth loss are weighted and summed to obtain the joint divergence loss, as shown in the following formula (12):

[0090]

[0091] A final loss of the image processing model is determined according to the first loss, the second loss and the joint divergence loss.

[0092] By taking age distribution as a new prediction target for face age prediction, the age prediction task essentially becomes a multi-label classification problem, which enables the model to better handle the uncertainty in age prediction and improve the accuracy of prediction. The model training considers the accuracy of classification prediction, that is, the degree of closeness between the specific age value output by the model and the real age, and pays attention to the prediction quality of the entire age distribution, which can capture the uncertainty or ambiguity in the prediction results. Combining the two, the performance of the model in both accurate age prediction and age range prediction can be comprehensively considered, so as to more comprehensively evaluate the model performance and improve the model prediction accuracy. The joint divergence loss constrains the prediction of age distribution, which helps to reduce the overfitting of the model on the training data and improve the generalization performance of the model. In some cases, the model may tend to predict some age groups more accurately and other age groups relatively poorly. By combining the prediction loss and the joint divergence loss, this error distribution can be balanced, so that the prediction performance of the model on different age groups is more balanced. The joint divergence loss enables the model to output an age distribution instead of a single age value, which provides richer prediction information for practical applications. The combination of classification prediction loss and joint divergence loss can form a joint loss function. This joint optimization helps the model converge to the optimal solution faster. At the same time, by introducing a self-learning training method in the model training process, the neural network adaptively and gradually estimates the age label distribution during the refinement process, combines the prediction results of the previous iteration on the age label with the prediction results of the current iteration to construct a loss function, and gradually adjusts the estimate of the age label distribution to better reflect the distribution of the real age label. This inheritance and adaptive learning process promotes the performance improvement of model training in age prediction and improves the accuracy of model prediction.

[0093] Optionally, if the current iteration is not the first iteration of the image processing model, determining the final loss of the image processing model based on the real age, gender judgment result and age prediction result includes: determining the current distribution age based on the age label and the second distribution function; determining the reference distribution age based on the age label and the first distribution function; determining the initial distribution age based on the age label and the initial distribution function of the real age; determining the loose loss based on the current distribution age, the reference distribution age and the initial distribution age; determining the final loss of the image processing model based on the first loss, the second loss, the third loss and the fourth loss includes: determining the final loss of the image processing model based on the loose loss, the first loss, the second loss, the third loss and the fourth loss.

[0094] Specifically, for each iteration round other than the first iteration, the model parameters are adaptively optimized based on the results of the previous iteration round and the real data. In order to avoid the error accumulation of the current iteration result to the previous iteration result, a loose loss function is introduced in the iterative optimization process of the model.

[0095] The value of the second distribution function corresponding to each value of the age label is used as the current distribution probability of each value of the age label, and each value of the age label is multiplied and summed by its current distribution probability to obtain the current distribution age. The value of the first distribution function corresponding to each value of the age label is used as the reference distribution probability of each value of the age label, and each value of the age label is multiplied and summed by its reference distribution probability to obtain the reference distribution age. The value of the initial distribution function corresponding to each value of the age label is used as the initial distribution probability of each value of the age label, and each value of the age label is multiplied and summed by its initial distribution probability to obtain the initial distribution age. Referring to formula (13), the error is calculated based on the reference distribution age and the current distribution age respectively with the initial distribution age. When the error of the current iteration is greater than the error of the previous iteration round, the error is applied for optimization, otherwise it is not used.

[0096]

[0097] Where a(n) is the initial distribution age; b t (n) is the current distribution age; b t-1 (n) is the reference distribution age, and N is the number of training samples.

[0098] Accordingly, the final loss of the image processing model is determined according to the first loss, the second loss, the loose loss and the joint divergence loss. The final loss function is shown in the following formula (14):

[0099] L=w0L Gender +w1L age-MAE +w2L slack +w3L age_kld (14)

[0100] Among them w0, w1, w2, w3 are hyperparameters.

[0101] Since the initial distribution function constructed in the first iteration of the training process is determined based on experience, the initial distribution may be different from the actual age label distribution, which affects the accuracy of age prediction. By introducing loose loss in the non-first iteration training process, the error accumulation of the current iteration result on the previous iteration result can be avoided, thus improving the accuracy of the model.

[0102] In order to make the model more robust in different environments and in situations where training data is lacking, the following optimizations can be performed: Data enhancement: Increase the diversity of training samples and improve the generalization ability of the model by rotating, scaling, flipping, adding noise, etc.; Transfer learning: Use a model pre-trained on a large-scale face dataset as a starting point, and fine-tune it to adapt to age and gender recognition tasks in specific scenarios; Ensemble learning: Combine the prediction results of multiple classifiers to improve overall recognition performance through voting or weighted averaging; Post-processing: Smooth the classification results to reduce the impact of noise and improve the stability and accuracy of the final recognition.

[0103] Optionally, the method further includes: testing the image processing model; wherein, for gender judgment, accuracy is used as an evaluation indicator; for age prediction, mean absolute error and cumulative error are used as evaluation indicators.

[0104]

[0105] Where M is the number of test samples; K d To determine the number of test samples with an absolute estimation error less than d, CA(3), CA(5), and CA(7) can be set for evaluation.

[0106] By setting the mean absolute error and cumulative error to evaluate the predictive ability of the image processing model, we can not only evaluate the overall prediction of the model, but also understand the deviation of the age prediction results. For example, we can calculate the proportion of samples under different thresholds such as 3, 5, and 7 that differ from the actual age. This helps us understand to what extent the model can accurately predict age and the distribution of prediction errors.

[0107] The embodiment of the present invention uses age distribution as a new prediction target for face age prediction tasks, and the age prediction task essentially becomes a multi-label classification problem, so that the model can better handle the uncertainty in age prediction and improve the accuracy of prediction; traditional age prediction methods often regard age as a single regression value, while the introduction of age distribution function allows the model to optimize on multiple adjacent ages, so as to find a more suitable prediction result. Fuzzy learning from real age and adjacent ages not only considers the actual age, but also considers the information of adjacent ages; the distribution function can represent the degree of description of each age to the corresponding face image, and each face image can not only help learn its actual age, but also help learn the adjacent ages, thereby improving the effectiveness of the training images of each age. By assigning a distribution function to each training sample, each image can provide training information for multiple adjacent ages, and the model can increase the number of training images for each age without increasing the total number of training images. Since the distribution function contains the information of each age, the model can learn the age characteristics more comprehensively during the training process, reducing the overfitting problem caused by insufficient or unbalanced data. By introducing the label distribution self-learning training method in the model training process, the neural network can adaptively and gradually estimate the age label distribution during the refinement process. The prediction results of the previous iteration on the age label are combined with the prediction results of the current iteration to construct the loss function, and the estimation of the age label distribution is gradually adjusted to better reflect the distribution of the real age label. This inheritance and adaptive learning process promotes the performance improvement of model training in age prediction and improves the accuracy of model prediction.

[0108] Figure 3 This is a flow chart of a vehicle-borne personnel image processing method provided by an embodiment of the present invention. This embodiment is applicable to the case of processing the facial images of vehicle-borne personnel. The method can be executed by a vehicle-borne personnel image processing device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device with corresponding data processing capabilities. Figure 3 As shown, the method includes:

[0109] S310, acquiring facial data of the vehicle occupants; and performing gender determination and age prediction on the facial data using an image processing model.

[0110] S320, feeding back the results of the gender determination and age prediction to the vehicle-mounted system, and the vehicle-mounted system adjusting the vehicle-mounted equipment according to the gender and age of the vehicle-mounted occupant;

[0111] The image processing model is trained by the image processing model training method provided by any of the aforementioned embodiments.

[0112] In the vehicle cabin environment, accurate identification of the age and gender of the vehicle occupants is crucial for providing personalized services and recommendations. For example, drivers of different age groups may have different preferences for music, air conditioning temperature, etc. The facial image of the vehicle occupants is captured by the high-definition camera in the vehicle, and the captured facial image is preprocessed by grayscale, face detection, face alignment, etc. to obtain the facial data of the vehicle occupants. The facial data is input into the image processing model for gender judgment and age prediction, and the gender judgment results and age prediction results are fed back to the vehicle system. The vehicle system adjusts the vehicle equipment according to the gender and age of the vehicle occupants, so that the vehicle occupants have a better riding experience. For example, the age-based music recommendation system can play songs that are more suitable for the driver's age group, and the gender-based air conditioning temperature control system can automatically adjust to a more comfortable temperature for the driver. By introducing the age prediction and gender judgment methods based on computer vision, the vehicle can more accurately understand the needs of the vehicle occupants and adjust the settings of the vehicle entertainment system, air conditioning system and other equipment accordingly, so as to provide more personalized services. Age and gender recognition can also be used to improve the safety of passengers. For example, for older drivers, the vehicle can automatically adjust settings such as seat position and steering wheel height to provide a more comfortable driving environment; at the same time, the vehicle's safety settings, such as braking distance and collision warning, can also be adjusted according to the driver's gender and age.

[0113] Image processing models can be trained using large-scale datasets on servers or workstations. After training, export the model to a format supported by the accelerated machine learning inference platform, such as Caffe or ONNX. Use the tools provided by the accelerated machine learning inference platform to convert the trained model to a format that can be run on the target hardware. During the conversion process, the model may need to be optimized to adapt to the computing power of the target hardware. Deploy the converted model to the hardware of the in-vehicle system and use the accelerated machine learning inference platform runtime environment to perform model inference on the target hardware.

[0114] The embodiment of the present invention captures the facial image of the vehicle occupants through the vehicle camera, and processes and analyzes it using computer vision technology. Specifically, a deep learning-based image processing model is used to extract and identify facial features in the image, thereby obtaining the age and gender information of the vehicle occupants. There is no need for the vehicle occupants to issue voice commands, which is more convenient and efficient; the age and gender information of the vehicle occupants can be accurately identified to provide more personalized services; the accuracy and efficiency of identification can be improved by continuously learning and optimizing algorithms. It can effectively solve the limitations of existing vehicle-mounted AI human-computer interaction, enhance the intelligence level of human-computer interaction, optimize the driving experience, and improve driving safety.

[0115] Figure 4A schematic diagram of the structure of a training device for an image processing model provided by an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0116] The data acquisition module 410 is used to acquire sample facial data of the driver and passenger; the sample facial data is input into the facial feature extraction unit in the image processing model to be trained to obtain sample facial features; the image processing model also includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit;

[0117] A gender determination module 420, configured to input the sample facial features into the gender determination unit for gender determination;

[0118] A style feature acquisition module 430, used for inputting the age label into the style feature extraction unit to obtain a style feature representing the age;

[0119] Enhanced feature acquisition module 440, used for inputting the style feature and the sample face feature into the feature enhancement unit to obtain enhanced features;

[0120] An age prediction module 450, configured to input the enhanced features into the age prediction unit for age prediction;

[0121] The parameter updating module 460 is used to determine the final loss of the image processing model according to the real age, gender judgment result and age prediction result, and update the parameters of the image processing model according to the final loss.

[0122] The image processing model training device provided in the embodiment of the present invention can execute the image processing model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] Optionally, the parameter updating module is specifically used for:

[0124] Determine the first loss based on the gender judgment result in the current iteration round;

[0125] The mean absolute error between the current predicted age and the true age in the current iteration round is obtained as the second loss;

[0126] Determining a third loss according to the current predicted probability, the reference distribution probability and the first distribution function in the current iteration round;

[0127] Determine a fourth loss according to the current predicted probability, the current predicted distribution probability and a second distribution function; wherein the second distribution function is determined according to the age label and the current predicted probability;

[0128] A final loss of the image processing model is determined according to the first loss, the second loss, the third loss and the fourth loss.

[0129] Optionally, if the current iteration is the first iteration of the image processing model, the parameter updating module is further used to:

[0130] The initial distribution function of the real age is used as the first distribution function;

[0131] Using the value of the first distribution function corresponding to the age label as a reference distribution probability;

[0132] A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

[0133] Optionally, if the current iteration is not the first iteration of the image processing model, the parameter updating module is further configured to:

[0134] Obtaining the previous prediction probability in the previous iteration round; determining the first distribution function according to the age label and the previous prediction probability;

[0135] Using the value of the first distribution function corresponding to the age label as a reference distribution probability;

[0136] A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

[0137] Optionally, if the current iteration is not the first iteration of the image processing model, the parameter updating module is further used to:

[0138] determining the current distribution age according to the age label and the second distribution function;

[0139] determining a reference distribution age according to the age label and the first distribution function;

[0140] Determine the initial distribution age according to the initial distribution function of the age label and the real age;

[0141] determining a loose loss based on the current distribution age, the reference distribution age, and the initial distribution age;

[0142] Determining a final loss of the image processing model according to the first loss, the second loss, the third loss, and the fourth loss includes:

[0143] A final loss of the image processing model is determined according to the loose loss, the first loss, the second loss, the third loss and the fourth loss.

[0144] Optionally, the style feature extraction unit includes an encoder and a multi-layer perceptron; the encoder encodes the age label and transmits the encoding result to the multi-layer perceptron; the multi-layer perceptron learns the style features corresponding to each value of the age label.

[0145] Optionally, the enhanced feature acquisition module is specifically used to:

[0146] The style feature is matrix-multiplied with the sample face feature to obtain an enhanced feature.

[0147] Optionally, the device further comprises:

[0148] A testing module, used for testing the image processing model;

[0149] Among them, for gender judgment, accuracy is used as the evaluation indicator; for age prediction, mean absolute error and cumulative error are used as evaluation indicators.

[0150] The image processing model training device further described can also execute the image processing model training method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] Figure 5 A schematic diagram of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0152] like Figure 5As shown, the electronic device 50 includes at least one processor 51, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0153] A number of components in the electronic device 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0154] The processor 51 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 51 performs the various methods and processes described above, such as a training method for an image processing model.

[0155] In some embodiments, the training method of the image processing model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the training method of the image processing model described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to execute the training method of the image processing model by any other appropriate means (e.g., by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0163] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for an image processing model, characterized in that: The method comprises: Obtain sample facial data of drivers and passengers; Inputting the sample face data into a face feature extraction unit in an image processing model to be trained to obtain sample face features; the image processing model further includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit; Inputting the sample facial features into the gender determination unit to perform gender determination; Inputting the age label into the style feature extraction unit to obtain the style feature representing the age; Inputting the style feature and the sample face feature into the feature enhancement unit to obtain enhanced features; Inputting the enhanced features into the age prediction unit to perform age prediction; According to the real age, gender judgment result and age prediction result, the final loss of the image processing model is determined, and the parameters of the image processing model are updated according to the final loss.

2. The method according to claim 1, characterized in that The determining of the final loss of the image processing model according to the actual age, the gender judgment result and the age prediction result comprises: Determine the first loss based on the gender judgment result in the current iteration round; The mean absolute error between the current predicted age and the true age in the current iteration round is obtained as the second loss; Determining a third loss according to the current predicted probability, the reference distribution probability and the first distribution function in the current iteration round; Determine a fourth loss according to the current predicted probability, the current predicted distribution probability and a second distribution function; wherein the second distribution function is determined according to the age label and the current predicted probability; A final loss of the image processing model is determined according to the first loss, the second loss, the third loss and the fourth loss.

3. The method according to claim 2, characterized in that If the current iteration is the first iteration of the image processing model, determining the third loss according to the current prediction probability, the reference distribution probability and the first distribution function in the current iteration round includes: The initial distribution function of the real age is used as the first distribution function; Using the value of the first distribution function corresponding to the age label as a reference distribution probability; A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

4. The method according to claim 2, characterized in that: If the current iteration is not the first iteration of the image processing model, determining the third loss according to the current prediction probability, the reference distribution probability and the first distribution function in the current iteration round includes: Obtaining the previous prediction probability in the previous iteration round; determining the first distribution function according to the age label and the previous prediction probability; Using the value of the first distribution function corresponding to the age label as a reference distribution probability; A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

5. The method according to claim 2, characterized in that: If the current iteration is not the first iteration of the image processing model, determining the final loss of the image processing model according to the real age, gender judgment result and age prediction result further includes: determining the current distribution age according to the age label and the second distribution function; determining a reference distribution age according to the age label and the first distribution function; Determine the initial distribution age according to the initial distribution function of the age label and the real age; determining a loose loss based on the current distribution age, the reference distribution age, and the initial distribution age; Determining a final loss of the image processing model according to the first loss, the second loss, the third loss, and the fourth loss includes: A final loss of the image processing model is determined according to the loose loss, the first loss, the second loss, the third loss and the fourth loss.

6. The method according to claim 1, characterized in that The style feature extraction unit includes an encoder and a multi-layer perceptron; the encoder encodes the age label and transmits the encoding result to the multi-layer perceptron; the multi-layer perceptron learns the style features corresponding to each value of the age label.

7. The method according to claim 1, characterized in that The step of inputting the style feature and the sample face feature into the feature enhancement unit to obtain the enhanced feature comprises: The style feature is matrix-multiplied with the sample face feature to obtain an enhanced feature.

8. The method according to claim 1, characterized in that The method further comprises: Testing the image processing model; Among them, for gender judgment, accuracy is used as the evaluation indicator; for age prediction, mean absolute error and cumulative error are used as evaluation indicators.

9. A method for processing an image of a person in a vehicle, characterized in that: The method comprises: Obtain facial data of passengers in the vehicle; Performing gender determination and age prediction on the face data through an image processing model; Feeding back the results of the gender determination and age prediction to the vehicle-mounted system, the vehicle-mounted system adjusting the vehicle-mounted equipment according to the gender and age of the vehicle-mounted occupant; Wherein, the image processing model is trained by the image processing model training method described in any one of claims 1-8.

10. A training device for an image processing model, characterized in that: The device comprises: A data acquisition module is used to acquire sample facial data of the driver and passenger; the sample facial data is input into a facial feature extraction unit in the image processing model to be trained to obtain sample facial features; the image processing model also includes a gender judgment unit, a style feature extraction unit, a feature enhancement unit and an age prediction unit; A gender determination module, used for inputting the sample facial features into the gender determination unit for gender determination; A style feature acquisition module, used for inputting the age label into the style feature extraction unit to obtain a style feature representing the age; An enhanced feature acquisition module, used for inputting the style feature and the sample face feature into the feature enhancement unit to obtain an enhanced feature; An age prediction module, used for inputting the enhanced features into the age prediction unit to perform age prediction; A parameter updating module is used to determine the final loss of the image processing model according to the real age, gender judgment result and age prediction result, and to update the parameters of the image processing model according to the final loss.

11. The device according to claim 10, characterized in that The parameter updating module is specifically used for: Determine the first loss based on the gender judgment result in the current iteration round; The mean absolute error between the current predicted age and the true age in the current iteration round is obtained as the second loss; Determining a third loss according to the current predicted probability, the reference distribution probability and the first distribution function in the current iteration round; Determine a fourth loss according to the current predicted probability, the current predicted distribution probability and a second distribution function; wherein the second distribution function is determined according to the age label and the current predicted probability; A final loss of the image processing model is determined according to the first loss, the second loss, the third loss and the fourth loss.

12. The device according to claim 11, characterized in that If the current iteration is the first iteration of the image processing model, the parameter updating module is further used to: The initial distribution function of the real age is used as the first distribution function; Using the value of the first distribution function corresponding to the age label as a reference distribution probability; A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

13. The method according to claim 11, characterized in that If the current iteration is not the first iteration of the image processing model, the parameter updating module is further configured to: Obtaining the previous prediction probability in the previous iteration round; determining the first distribution function according to the age label and the previous prediction probability; Using the value of the first distribution function corresponding to the age label as a reference distribution probability; A third loss is obtained according to the relative entropy between the current prediction probability and the reference distribution probability.

14. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the image processing model described in any one of claims 1-8 or the vehicle-mounted personnel image processing method described in claim 9.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a processor to implement the training method of the image processing model described in any one of claims 1 to 8 or the vehicle-mounted personnel image processing method described in claim 9 when executed.

Images