In-vehicle person gender and age detection method and device, electronic equipment and storage medium
By extracting and aligning facial landmarks from the original facial images of people inside the vehicle, and combining this with a multi-branch gender and age detection model, the problem of slow speed and low accuracy in gender and age recognition of people inside the vehicle in existing technologies has been solved. This has enabled fast and accurate gender and age recognition, and improved the personalized service capabilities of the smart cockpit.
Patent Information
- Application Number
- CN202411079141.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-26
AI Technical Summary
Existing technologies for identifying the gender and age of occupants in vehicles are slow to predict and have low accuracy, failing to meet the rapid and accurate identification requirements of smart cockpits.
By acquiring the original facial images of the people inside the vehicle, facial landmarks are extracted and aligned. A multi-branch gender and age detection model, including a backbone network and multiple detection heads, is used. Combining classification and regression methods, the mean of multiple gender and age detection values is calculated to obtain the final result.
It improves the speed and accuracy of gender and age detection. By combining face alignment and multi-branch models, it achieves fast and accurate gender and age recognition, and supports personalized driving style switching and recommendations.
Smart Images

Figure CN119028000A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, electronic device and storage medium for detecting the gender and age of people in a vehicle. Background Art
[0002] With the rapid development of smart cockpit technology, the in-car environment is no longer a simple internal space of a vehicle, but has gradually evolved into a private mobile space that integrates personalization, intelligence, and comfort. With the deep integration of artificial intelligence, big data analysis, and the Internet of Things, the smart cockpit system can accurately capture and understand the diverse needs of passengers, thereby providing more intimate and customized services. Among them, gender and age, as important factors affecting individual behavioral habits, aesthetic preferences, and safety awareness, play a vital role in driving style conversion and personalized recommendations.
[0003] People of different genders and ages have different driving styles and hobbies. Changing driving styles and making personalized recommendations based on the gender and age of the occupants can greatly improve the driving experience. There are also many methods for predicting the gender and age of occupants in related technologies, but most of them use multiple models to predict gender and age, and the prediction speed is slow and the prediction accuracy is low. Therefore, how to accurately and quickly identify the gender and age of occupants is very important for smart cockpits. Summary of the invention
[0004] In view of the shortcomings of the related technologies mentioned above, the present application provides a method, device, electronic device and storage medium for detecting the gender and age of people in a vehicle to solve the technical problem of being unable to accurately identify the gender and age of people in the vehicle.
[0005] The present application provides a method for detecting the gender and age of a person in a vehicle, the method comprising: obtaining an original face image of the person in the vehicle; extracting face key points from the original face image, and performing face alignment on the original face image based on the face key points to obtain a face image to be detected; inputting the face image to be detected into a gender and age detection model to obtain multiple gender detection values and multiple age detection values, the gender and age detection model including multiple detections for outputting gender detection values and age detection values. The gender detection result of the face image to be detected is obtained based on the average of the multiple gender detection values, and the age detection result of the face image to be detected is obtained based on the average of the multiple age detection values.
[0006] In one embodiment of the present application, the face image to be detected is input into a gender and age detection model to obtain multiple gender detection values and multiple age detection values, including: after extracting the facial features of the face image to be detected through a backbone network, the facial features are input into the detection head, the detection head includes at least a first detection head, a second detection head and a third detection head, and the gender and age detection model includes the backbone network and multiple detection heads connected to the backbone network; the first detection head outputs a first gender detection value and a first age detection value based on the face image to be detected; the second detection head outputs a second gender detection value and a second age detection value based on the face image to be detected; the third detection head outputs a third gender detection value and a third age detection value based on the face image to be detected.
[0007] In one embodiment of the present application, the face image to be detected is input into a gender and age detection model, and multiple gender detection values and multiple age detection values are obtained, including: the first detection head obtains a first gender detection value and a first probability value corresponding to a first age classification based on the face image to be detected, the first age classification is to divide a preset age range into multiple age groups based on a preset first classification rule, the first probability value is a probability value corresponding to each age group in the first age classification, and a first age detection value is obtained based on the first probability value corresponding to the first age classification; the second detection head obtains a second gender detection value and a second probability value corresponding to the second age classification based on the face image to be detected, the second age classification is to divide the preset age range into multiple age groups based on a preset second classification rule, the second probability value is a probability value corresponding to each age group in the second age classification, and a second age detection value is obtained based on the second probability value corresponding to the second age classification; the third detection head obtains a third gender detection value and an age regression value based on the face image to be detected, and uses the age regression value as the third age detection value.
[0008] In one embodiment of the present application, a first age detection value is obtained based on a first probability value corresponding to the first age classification, and a second age detection value is obtained based on a second probability value corresponding to the second age classification, including: calculating the age group mean corresponding to each age group in the first age classification, and performing a weighted sum operation on the age group mean and the probability value corresponding to the age group to obtain the age detection value of the first detection head; calculating the age group mean corresponding to each age group in the second age classification, and performing a weighted sum operation on the age group mean and the probability value corresponding to the age group to obtain the age detection value of the second detection head.
[0009] In one embodiment of the present application, performing face alignment on the original face image based on the face key points includes: obtaining multiple historical face images, calculating an average key point position based on the face key point positions of the multiple historical face images, and using the average key point position as the standard key point position; calculating an affine transformation matrix based on the face key point positions of the original face image and the standard key point positions, so as to perform face alignment on the original face image based on the standard key points through the affine transformation matrix.
[0010] In one embodiment of the present application, after obtaining the original facial image of the person in the car, it also includes: if the original facial image contains multiple faces, the multiple faces in the original facial image are classified based on the facial recognition frame coordinates and the preset area coordinates to obtain the corresponding in-car areas of the multiple faces, the preset area coordinates include the driver area coordinates, the co-pilot area coordinates, the left rear passenger area coordinates, the middle rear passenger area coordinates and the right rear passenger area coordinates, and the in-car areas include the driver area, the co-pilot area, the left rear passenger area, the middle rear passenger area and the right rear passenger area; facial key points are extracted for the faces in each in-car area, and facial alignment is performed on the faces in each in-car area based on the facial key points to obtain the facial images to be detected corresponding to each in-car area; the facial images to be detected corresponding to each in-car area are respectively input into the gender and age detection model to obtain the gender detection results and the age detection results corresponding to the facial images to be detected in each in-car area.
[0011] In one embodiment of the present application, before obtaining the original facial image of the person in the car, it also includes: obtaining an initial training data set, the initial training data set including multiple training facial images and corresponding age labels and gender labels; performing lighting enhancement processing on the training facial images in the initial training data set, the lighting enhancement processing including normalizing the pixel values in the training facial images, and performing lighting enhancement on the normalized training facial images based on a preset lighting coefficient; inputting the initial training data set after the lighting enhancement processing as a facial training set into a preset initial detection model, and training the initial detection model to obtain the gender and age detection model.
[0012] In one embodiment of the present application, after obtaining the gender detection result of the face image to be detected based on the average of multiple gender detection values and obtaining the age detection result of the face image to be detected based on the average of multiple age detection values, it also includes: matching the corresponding driving strategy in a preset strategy library based on the gender detection result and the age detection result; when it is detected that the vehicle speed is zero and the door is opened and closed once, obtaining the face image to be detected of the person in the car again to detect the gender and age of the person in the car.
[0013] The embodiment of the present application also provides a device for detecting the gender and age of a person in a vehicle, the device comprising: an image input module for acquiring an original face image of a person in the vehicle; a face alignment module for extracting face key points from the original face image, and performing face alignment on the original face image based on the face key points to obtain a face image to be detected; a model detection module for inputting the face image to be detected into a gender and age detection model to obtain a plurality of gender detection values and a plurality of age detection values, the gender and age detection model comprising a plurality of detection heads for outputting gender detection values and age detection values. A result output module for obtaining a gender detection result of the face image to be detected based on the average of a plurality of gender detection values, and obtaining an age detection result of the face image to be detected based on the average of a plurality of age detection values.
[0014] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the method for detecting the gender and age of a person in a vehicle as described in any of the above embodiments.
[0015] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the method for detecting the gender and age of a person in a vehicle as described in any of the above embodiments.
[0016] Beneficial effects of the present application: The embodiments of the present application provide a method, device, electronic device and storage medium for detecting the gender and age of a person in a vehicle. The method obtains an original face image of the person in the vehicle, extracts face key points from the original face image, and performs face alignment on the original face image based on the face key points to obtain a face image to be detected, inputs the face image to be detected into a gender and age detection model to obtain multiple gender detection values and multiple age detection values, obtains a gender detection result of the face image to be detected based on the average of the multiple gender detection values, obtains an age detection result of the face image to be detected based on the average of the multiple age detection values, improves the speed of gender and age detection through face alignment, outputs gender and age through a multi-branch combination of classification and regression, and improves the accuracy of the gender and age detection results by sharing network weights for gender and age detection tasks.
[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1is a schematic diagram of an implementation environment of a method for detecting the gender and age of a person in a vehicle, shown in an exemplary embodiment of the present application;
[0019] Figure 2 is a flow chart of a method for detecting the gender and age of a person in a vehicle, shown in an exemplary embodiment of the present application;
[0020] Figure 3 is a specific flow chart of detecting the gender and age of a person in a vehicle as shown in an exemplary embodiment of the present application;
[0021] Figure 4 is a schematic diagram of a framework of a gender and age detection model shown in an exemplary embodiment of the present application;
[0022] Figure 5 is a block diagram of a device for detecting the gender and age of a person in a vehicle, shown in an exemplary embodiment of the present application;
[0023] Figure 6 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0024] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0025] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0026] It should be noted that in this application, "first", "second", etc. are only used to distinguish similar objects, and are not used to limit the order or precedence of similar objects. The variations of "including", "having", etc. described above indicate that the scope covered by the subject of the word is not exclusive except for the examples shown by the word.
[0027] It is understood that the various numbers, step numbers, etc. recorded in this application are distinguished for the convenience of description and are not used to limit the scope of this application. The size of the numbers in this application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic.
[0028] In the following description, a lot of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.
[0029] The embodiments of the present application respectively propose a method for detecting the gender and age of a person in a vehicle, a device for detecting the gender and age of a person in a vehicle, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.
[0030] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a method for detecting the gender and age of a person in a vehicle, as shown in an exemplary embodiment of the present application. Figure 1 As shown, the implementation environment of the method for detecting the gender and age of people in a vehicle includes an image acquisition device 101 and an intelligent cockpit system 102, wherein the image acquisition device 101 is a camera that can be installed in the vehicle, such as an OMS camera, and the intelligent cockpit system 102 is a system loaded with a face detection model and a gender and age detection model. The image acquisition device 101 outputs the collected original face image to the intelligent cockpit system 102, and the intelligent cockpit system 102 performs face alignment on the original face image based on the face detection model to obtain a face image to be detected, and outputs the face image to be detected to the gender and age detection model to perform gender and age detection on the people in the vehicle in the face image to be detected.
[0031] See also Figure 2 , Figure 2 is a flowchart of a method for detecting the gender and age of a person in a vehicle, shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is as follows. The method may also be applicable to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.
[0032] like Figure 2 As shown, in an exemplary embodiment, the method for detecting the gender and age of a person in a vehicle includes at least steps S210 to S240, which are described in detail as follows:
[0033] Step S210, obtaining the original face image of the person in the car.
[0034] In one embodiment of the present application, original facial images of people in the car are obtained by a camera installed in the car. In this embodiment, various areas in the car are also classified to obtain multiple areas in the car. The areas in the car include the driver area, the co-pilot area, the left rear passenger, the middle rear passenger and the right rear passenger. Gender and age detection is performed on facial images in different predefined areas to obtain the gender and age of people in different areas.
[0035] In one embodiment of the present application, before obtaining the original facial image of the person in the car, it also includes: obtaining an initial training data set, the initial training data set includes multiple training facial images and corresponding age labels and gender labels; performing lighting enhancement processing on the training facial images in the initial training data set, the lighting enhancement processing includes normalizing the pixel values in the training facial images, and performing lighting enhancement on the normalized training facial images based on a preset lighting coefficient; inputting the initial training data set after the lighting enhancement processing as a facial training set into a preset initial detection model, and training the initial detection model to obtain a gender and age detection model.
[0036] In one embodiment of the present application, a gender and age model is obtained based on a preset initial detection model training. The preset initial detection model uses a ShuffleNetv2 network designed for mobile terminal deployment and reasoning acceleration in order to reduce the resource occupation of the gender and age model on the mobile terminal and improve the feature extraction capability of the model. The model outputs three branches after extracting facial features through the backbone network. Each branch outputs gender detection values and age detection values through a convolutional network, an activation function, and a fully connected layer. For age detection, the model uses multi-branch outputs of classification and regression to effectively improve the accuracy of age prediction. Because the gender and age tasks are strongly coupled, the gender and age of the model share network weights. The three detection heads of the model output 72 neurons, 12 neurons, and 3 neurons respectively. The first two neurons output by each detection head are used for gender detection, and the remaining neurons are used for age detection. The gender detection uses the Binary Cross Entropy (BCE) loss function to calculate the gender loss, and the age detection uses the L1 loss function to calculate the age loss, so that the total loss is obtained based on the age loss and the gender loss to achieve the training of the model.
[0037] In one embodiment of the present application, the illumination enhancement processing of the training face images in the initial training data set includes using the Phong model to generate training face images under different illumination conditions as the face images to be detected. Compared with ordinary illumination data enhancement, the images generated by the model are more realistic and detailed. In this embodiment, the final illumination intensity is calculated by three components: ambient light, diffuse light, and specular light. The calculation method is as follows:
[0038]
[0039] In formula (1), I is the final light intensity, I a To calculate the ambient light intensity, I d To calculate the diffuse light intensity, I s To calculate the intensity of specular light, k a is the preset ambient light coefficient, k d is the preset diffuse reflection coefficient, k s is the preset ambient specular reflection coefficient, I ambient is the ambient light intensity of the image, L is the direction vector of the image light source, N is the surface normal vector of the image, and I diffuse is the diffuse light intensity of the image, R is the direction vector of the reflected light of the image, V is the direction vector of the sight line of the image, n is the preset image highlight index, I specular The intensity of the image's specular reflection.
[0040] In one embodiment of the present application, each pixel in the initial detection image is traversed, and the pixel value is normalized between 0 and 1 to represent the basic color in the illumination model. Based on the normalized initial monitoring image, the image ambient light intensity, the image diffuse light intensity, and the image specular light intensity are obtained. The light source direction and normal vector are set in the initial detection image to obtain the image light source direction vector and the image surface normal vector. Based on the line of sight direction and reflection direction in the image, the image line of sight direction vector and the image reflected light direction vector are obtained.
[0041] In one embodiment of the present application, before the initial training data set is subjected to illumination enhancement processing, it also includes creating a facial image data set with expression labels such as happy, surprised, sad, frustrated and angry based on the initial training data set, and training the ExprGAN model to learn the potential features of different expressions. After the model training is completed, a multi-expression generation model is obtained, and the facial image and the target expression label are input into the multi-expression generation model to generate a new image with the target expression. Inputting the initial training data set into the multi-expression generation model to obtain more facial images with different expressions, so as to expand the data and increase the facial data of multiple expressions can effectively improve the robustness and accuracy of the gender and age detection model.
[0042] In one embodiment of the present application, the initial training data set is based on an open source data set to produce a data set suitable for gender and age prediction of the yellow race. The age distribution of the MegaAge_Asian open source data set is 0-70 years old and is for Asian faces, but lacks gender labels. For this reason, an open source gender prediction model is used to gender-label the data. The Asian Face Age Dataset (AFAD) contains gender and age labels, and the age distribution is 19-40 years old. Based on the age distribution characteristics of the above two open source data sets, the initial training data set is expanded through a multi-expression generation model and light enhancement processing to obtain a large number of bisexual face training data sets with an age range of 0-70 years old.
[0043] In one embodiment of the present application, before training the preset initial model with a face training data set, it also includes performing face alignment processing on the face training images in the face training data set, enhancing the data set by random erasing, randomly adding noise, etc., and outputting the enhanced face training data set to the preset initial model to train a gender and age model.
[0044] Step S220, extracting facial key points from the original facial image, and performing facial alignment on the original facial image based on the facial key points to obtain a facial image to be detected.
[0045] In one embodiment of the present application, face alignment of original face images based on face key points includes: acquiring multiple historical face images, calculating an average key point position based on the face key point positions of the multiple historical face images, and using the average key point position as the standard key point position; calculating an affine transformation matrix based on the face key point positions of the original face image and the standard key point positions, so as to face align the original face image based on the standard key points through the affine transformation matrix.
[0046] In one embodiment of the present application, aligning the original face image based on the standard key points through the affine transformation matrix includes: the face key points of the original face image can be selected from key position points such as eyes, nose and mouth in the face, calculating the affine change matrix according to the correspondence between the face key points of the original face image and the standard key points, and translating, rotating and scaling the original face image according to the calculated affine change matrix to achieve face alignment of the original face image based on the standard key points. In this embodiment, the calculation of the affine change matrix can be obtained by solving the least squares problem to obtain the optimal change matrix as the affine change matrix, so that the face key points in the original face image can be aligned with the standard key points as much as possible.
[0047] In one embodiment of the present application, the original face image is aligned with the face detection model to obtain the face image to be detected, wherein the face detection model is based on the face frame information of the WIDER FACE open source data set, and the face key point detection algorithm of Dlib is used to obtain the position of the face key points in the data set and manually fine-tune it; based on the OMS camera to collect the cabin data, the face detection and key point detection model of Dlib is used to pre-label the collected data, and the pre-labeling result is fine-tuned using the labeling tool. A regression module for face key points is added based on the output head of the YOLOX model. In view of the problem that the commonly used L2 loss function is insensitive to small errors and leads to poor key point recognition effect, this embodiment uses the Wingloss loss function to adjust the face detection model.
[0048] In one embodiment of the present application, ShufflenetV2 is used to replace the backbone network of YOLOX; ShufflenetV2 conducts experiments based on hardware with industrial-grade optimized neural network libraries, and proposes four criteria for efficient network operation: keeping the number of channels constant to reduce memory access costs, avoiding excessive group convolution, using multi-branch structures with caution, and avoiding element-by-element operations, and ShufflenetV2 is designed based on the above four criteria. In order to increase the generalization ability of the model, data enhancement methods such as random rotation, random brightness change, and random erasure are used. The face detection network is pre-trained based on the annotated WIDER FACE open source dataset, and some network weights of the pre-trained face detection model are frozen. Fine-tuning is performed based on the self-collected cockpit dataset, which is a historical face image acquired through an image acquisition device.
[0049] Step S230, inputting the face image to be detected into the gender and age detection model to obtain multiple gender detection values and multiple age detection values, the gender and age detection model including multiple detection heads for outputting gender detection values and age detection values.
[0050] In one embodiment of the present application, a facial image to be detected is input into a gender and age detection model to obtain multiple gender detection values and multiple age detection values, including: after extracting facial features of the facial image to be detected through a backbone network, the facial features are input into a detection head, the detection head includes at least a first detection head, a second detection head and a third detection head, and the gender and age detection model includes a backbone network and multiple detection heads connected to the backbone network; the first detection head outputs a first gender detection value and a first age detection value based on the facial image to be detected; the second detection head outputs a second gender detection value and a second age detection value based on the facial image to be detected; the third detection head outputs a third gender detection value and a third age detection value based on the facial image to be detected.
[0051] The face image to be detected is input into the gender and age detection model, and multiple gender detection values and multiple age detection values are obtained, including: the first detection head obtains a first gender detection value and a first probability value corresponding to the first age classification based on the face image to be detected, the first age classification is to divide the preset age range into multiple age groups based on the preset first classification rule, the first probability value is the probability value corresponding to each age group in the first age classification, and the first age detection value is obtained based on the first probability value corresponding to the first age classification; the second detection head obtains a second gender detection value and a second probability value corresponding to the second age classification based on the face image to be detected, the second age classification is to divide the preset age range into multiple age groups based on the preset second classification rule, the second probability value is the probability value corresponding to each age group in the second age classification, and the second age detection value is obtained based on the second probability value corresponding to the second age classification; the third detection head obtains a third gender detection value and an age regression value based on the face image to be detected, and uses the age regression value as the third age detection value. A first age detection value is obtained based on a first probability value corresponding to a first age classification, and a second age detection value is obtained based on a second probability value corresponding to a second age classification, including: calculating the age group mean corresponding to each age group in the first age classification, and performing a weighted sum operation on the age group mean and the probability value corresponding to the age group to obtain the age detection value of the first detection head; calculating the age group mean corresponding to each age group in the second age classification, and performing a weighted sum operation on the age group mean and the probability value corresponding to the age group to obtain the age detection value of the second detection head.
[0052] In one embodiment of the present application, the first age classification is 70 age categories, and the preset first classification rule is to divide the preset age range into 70 age groups. In this embodiment, the preset age range is 0-70 years old. Based on the first classification rule, the probability value corresponding to each age group in the first age classification includes the 0-year-old age group and the corresponding probability value, the 1-year-old age group and the corresponding probability value, the 2-year-old age group and the corresponding probability value, ..., the 69-year-old age group and the corresponding probability value and the 70-year-old age group and the corresponding probability value.
[0053] In one embodiment of the present application, the second age classification is divided into 10 age categories, and the preset second classification rule is to divide the preset age range into 10 age groups. In this embodiment, the preset age range is 0-70 years old. Based on the preset second classification rule, the probability values corresponding to each age group in the second age classification include 0-6 age groups and corresponding probability values, 7-13 age groups corresponding probability values, 14-19 age groups corresponding probability values, ..., 57-63 age groups and corresponding probability values, 64-70 age groups and corresponding probability values.
[0054] In one embodiment of the present application, the framework of the gender and age detection model is as follows: Figure 4 As shown, Figure 4is a schematic diagram of a framework of a gender and age detection model shown in an exemplary embodiment of the present application. Figure 4 It can be seen that the face image to be detected is input into the backbone network of the gender and age detection model for face extraction. The backbone network outputs the facial features to the first detection head, the second detection head and the third detection head. The output of the first detection head is 2 gender classifications and 70 age categories, among which 70 age categories are the first age classifications, and 2 gender classifications are to output two gender values, and the average of the two gender values is used as the first gender detection value. The output of the age 70 classification is the 0-year-old age group and the corresponding probability value, the 1-year-old age group and the corresponding probability value, the 2-year-old age group and the corresponding probability value, ..., the 69-year-old age group and the corresponding probability value and the 70-year-old age group and the corresponding probability value. The age 70 category divides the preset age range (0-70 years old in this embodiment) into 70 parts, and the age group corresponding to each part is 1 age value. The age of each age group is multiplied by its corresponding probability value and then added as the first age detection value, for example, 1*1-year-old age group and the corresponding probability value+2*2-year-old age group and the corresponding probability value+....+69*69-year-old age group and the corresponding probability value+70*70-year-old age group and the corresponding probability value. The output of the second detection head is 2 gender categories and 10 age categories, among which 10 age categories are the second age categories, and the 2 gender categories are to output two gender values, and the average of the two gender values is used as the second gender detection value. The output of the 10 age categories is the 0-6 age group and the corresponding probability value, the probability value corresponding to the 7-13 age group, the probability value corresponding to the 14-19 age group, ..., the 57-63 age group and the corresponding probability value, the 64-70 age group and the corresponding probability value. The 10 age categories are to divide the preset age range (0-70 years old in this embodiment) into 10 parts, and the age group corresponding to each part is 7 age values. The age average of each age group is multiplied by its corresponding probability value and then added as the second age detection value, for example, 3*0-6 age group and the corresponding probability value + 10*7-13 age group corresponding probability value + 16*14-19 age group corresponding probability value + 60*57-63 age group and the corresponding probability value + 68*64-70 age group and the corresponding probability value. The output of the third detection head is gender 2 classification and age regression value. Gender 2 classification outputs two gender values, and the average of the two gender values is used as the third gender detection value, and the age regression value is used as the third age detection value.
[0055] Step S240, obtaining a gender detection result of the face image to be detected based on the average of the multiple gender detection values, and obtaining an age detection result of the face image to be detected based on the average of the multiple age detection values.
[0056] In one embodiment of the present application, if the average of the multiple gender detection values is greater than the preset gender threshold, the gender detection result is female, otherwise it is male. The average of the multiple age detection values is used as the age detection result.
[0057] In one embodiment of the present application, after obtaining the gender detection result of the face image to be detected based on the average of multiple gender detection values and obtaining the age detection result of the face image to be detected based on the average of multiple age detection values, it also includes: matching the corresponding driving strategy in a preset strategy library based on the gender detection result and the age detection result; when it is detected that the vehicle speed is zero and the door is opened and closed once, obtaining the face image to be detected of the person in the car again to detect the gender and age of the person in the car.
[0058] In one embodiment of the present application, in order to reduce resource usage, the gender and age model reasoning operation is stopped after the gender and age information of the occupants is obtained through the gender and age detection model. When adaptive cruise control is turned on, if the driver is a young male, a shorter following distance can be set to improve road traffic efficiency; if the driver is an older driver or a woman, a longer following distance is set to give the driver more reaction time. When the occupants of the car play random music through the voice assistant, the location of the occupants can be determined by the location of the voice, and then the gender and age of the passengers can be obtained, and personalized music recommendations can be provided in combination with the music recommendation algorithm. When the vehicle speed is zero and the door is reopened and closed, the gender and age model re-acquires the gender and age of the occupants.
[0059] In one embodiment of the present application, if the original face image contains multiple faces, the multiple faces in the original face image are classified based on the face recognition frame coordinates and the preset area coordinates to obtain the corresponding in-car areas of the multiple faces, the preset area coordinates include the driver area coordinates, the co-pilot area coordinates, the left rear passenger area coordinates, the middle rear passenger area coordinates and the right rear passenger area coordinates, and the in-car areas include the driver area, the co-pilot area, the left rear passenger area, the middle rear passenger area and the right rear passenger area; facial key points are extracted from the faces in each in-car area, and face alignment is performed on the faces in each in-car area based on the facial key points to obtain the face images to be detected corresponding to each in-car area; the face images to be detected corresponding to each in-car area are respectively input into the gender and age detection model to obtain the gender detection results and age detection results corresponding to the face images to be detected in each in-car area.
[0060] In one embodiment of the present application, the original face image is divided into regions based on the seating area of the occupants of the vehicle to obtain a plurality of in-vehicle regions, including the driver region, the co-pilot region, the left rear passenger region, the middle rear passenger region, and the right rear passenger region, and preset region coordinates are set for each in-vehicle region. If the face recognition frame coordinates in the original face image are within a preset region coordinate, the face corresponding to the face recognition frame is considered to be the passenger face in the in-vehicle region. For example, if the face recognition frame coordinates are within the co-pilot region coordinates, the face corresponding to the face recognition frame is the passenger face in the co-pilot region, the face image corresponding to the co-pilot region is extracted, the face image to be detected corresponding to the co-pilot region is obtained, and the face image to be detected corresponding to the co-pilot region is input into the gender and age detection model, thereby obtaining the age detection result and gender detection result of the passenger in the co-pilot region.
[0061] In one embodiment of the present application, different intelligent driving strategies can be matched according to the gender and age detection results of people in different areas of the vehicle. For example, if it is detected that the person in the driver's area is a young male, a shorter following distance can be set when the adaptive cruise control is turned on to improve road traffic efficiency. If it is detected that the person in the driver's area is an older driver or a woman, a longer following distance can be set when the adaptive cruise control is turned on to give the driver more reaction time. When the person in the car plays random music through the voice assistant, the gender and age of the person in the car corresponding to the voice emitting area can be detected to provide personalized music recommendations in combination with the music recommendation algorithm.
[0062] See also Figure 3 , Figure 3 is a specific flow chart of the gender and age detection of the vehicle occupants shown in an exemplary embodiment of the present application. Figure 3 As shown, the original face image is obtained based on the OMS camera, and the original face image is input into the face detection model to perform face alignment based on the key point face frame, and then the illumination enhancement processing and facial expression data enhancement are performed to obtain the face image to be detected, and the face image to be detected is input into the gender and age model, and the output gender detection result is based on the sliding average window algorithm to obtain the final gender detection result. The sliding average window algorithm is based on the detection results of multiple frames of face images to be detected to obtain the final detection result. If the gender detection result of the previous frame image is greater than the gender detection result of the next frame image and the gender detection result of the previous frame image is greater than the set gender threshold, the output gender is female, otherwise it is male. The output age and gender detection result is based on the sliding average window algorithm to obtain the final age detection result, and the final age detection result is obtained based on the average value of the age detection results of multiple frames of face images to be detected and output, and a driving strategy that is more suitable for the occupants is selected based on the vehicle information and the output gender and age detection result to improve the driving experience of the occupants.
[0063] See also Figure 5 , Figure 5 is a block diagram of a device for detecting the gender and age of a person in a vehicle, as shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown is as follows. The device may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.
[0064] like Figure 5 As shown, the exemplary device for detecting the gender and age of a person in a vehicle includes: an image input module 501 , a face alignment module 502 , a model detection module 503 and a result output module 504 .
[0065] The image input module 501 is used to obtain the original face image of the person in the car;
[0066] The face alignment module 502 is used to extract the face key points of the original face image and align the original face image based on the face key points to obtain the face image to be detected;
[0067] The model detection module 503 is used to input the face image to be detected into the gender and age detection model to obtain multiple gender detection values and multiple age detection values. The gender and age detection model includes multiple detection heads for outputting gender detection values and age detection values.
[0068] The result output module 504 is used to obtain the gender detection result of the face image to be detected based on the average of multiple gender detection values, and to obtain the age detection result of the face image to be detected based on the average of multiple age detection values.
[0069] Figure 6 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0070] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603, such as executing the method described in the above embodiment. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, ROM 602 and RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0071] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.
[0072] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, various functions defined in the system of the present application are executed.
[0073] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0074] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0075] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0076] Another aspect of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the above-mentioned method for detecting the gender and age of a person in a vehicle. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.
[0077] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method for detecting the gender and age of a person in a vehicle provided in each of the above embodiments.
[0078] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A method for detecting the gender and age of people in a vehicle, characterized in that: The method for detecting the gender and age of the person in the vehicle comprises: Obtain the original face image of the person in the car; Extracting facial key points from the original facial image, and performing facial alignment on the original facial image based on the facial key points to obtain a facial image to be detected; Inputting the face image to be detected into a gender and age detection model to obtain a plurality of gender detection values and a plurality of age detection values, wherein the gender and age detection model includes a plurality of detection heads for outputting gender detection values and age detection values; A gender detection result of the face image to be detected is obtained based on an average of multiple gender detection values, and an age detection result of the face image to be detected is obtained based on an average of multiple age detection values.
2. The method for detecting the gender and age of a person in a vehicle according to claim 1, characterized in that: Inputting the face image to be detected into the gender and age detection model, obtaining multiple gender detection values and multiple age detection values includes: After extracting the facial features of the face image to be detected through the backbone network, the facial features are input into the detection head, the detection head at least includes a first detection head, a second detection head and a third detection head, and the gender and age detection model includes the backbone network and a plurality of detection heads connected to the backbone network; The first detection head outputs a first gender detection value and a first age detection value based on the face image to be detected; The second detection head outputs a second gender detection value and a second age detection value based on the face image to be detected; The third detection head outputs a third gender detection value and a third age detection value based on the face image to be detected.
3. The method for detecting the gender and age of a person in a vehicle according to claim 2, characterized in that: Inputting the face image to be detected into the gender and age detection model, obtaining multiple gender detection values and multiple age detection values includes: The first detection head obtains a first gender detection value and a first probability value corresponding to a first age classification based on the face image to be detected, wherein the first age classification is to divide a preset age range into multiple age groups based on a preset first classification rule, and the first probability value is a probability value corresponding to each age group in the first age classification, and a first age detection value is obtained based on the first probability value corresponding to the first age classification; The second detection head obtains a second gender detection value and a second probability value corresponding to a second age classification based on the face image to be detected, wherein the second age classification is to divide the preset age range into multiple age groups based on a preset second classification rule, and the second probability value is a probability value corresponding to each age group in the second age classification, and a second age detection value is obtained based on the second probability value corresponding to the second age classification; The third detection head obtains a third gender detection value and an age regression value based on the face image to be detected, and uses the age regression value as the third age detection value.
4. The method for detecting the gender and age of a person in a vehicle according to claim 3, characterized in that: Obtaining a first age detection value based on a first probability value corresponding to the first age category, and obtaining a second age detection value based on a second probability value corresponding to the second age category, including: Calculating the age group mean corresponding to each age group in the first age classification, and performing a weighted sum operation on the age group mean and the probability value corresponding to the age group to obtain the age detection value of the first detection head; The age group mean corresponding to each age group in the second age classification is calculated, and the age group mean and the probability value corresponding to the age group are weighted summed to obtain the age detection value of the second detection head.
5. The method for detecting the gender and age of a person in a vehicle according to claim 1, characterized in that: Performing face alignment on the original face image based on the face key points includes: Acquire multiple historical face images, calculate an average key point position based on the facial key point positions of the multiple historical face images, and use the average key point position as a standard key point position; An affine transformation matrix is calculated based on the facial key point positions of the original facial image and the standard key point positions, so as to perform facial alignment on the original facial image based on the standard key points through the affine transformation matrix.
6. The method for detecting the gender and age of a person in a vehicle according to claim 1, characterized in that: After obtaining the original face image of the person in the car, it also includes: If the original face image contains multiple faces, classify the multiple faces in the original face image based on the face recognition frame coordinates and the preset area coordinates to obtain the in-vehicle areas corresponding to the multiple faces, the preset area coordinates include the driver area coordinates, the co-pilot area coordinates, the left rear passenger area coordinates, the middle rear passenger area coordinates and the right rear passenger area coordinates, and the in-vehicle areas include the driver area, the co-pilot area, the left rear passenger area, the middle rear passenger area and the right rear passenger area; Extracting facial key points of faces in each in-car area, and aligning faces in each in-car area based on the facial key points to obtain a face image to be detected corresponding to each in-car area; The face images to be detected corresponding to the various in-vehicle areas are respectively input into the gender and age detection model to obtain the gender detection results and the age detection results corresponding to the face images to be detected in the various in-vehicle areas.
7. The method for detecting the gender and age of a person in a vehicle according to claim 1, characterized in that: Before obtaining the original face image of the person in the car, it also includes: Obtain an initial training data set, which includes multiple training face images and corresponding age labels and gender labels; Performing illumination enhancement processing on the training face images in the initial training data set, wherein the illumination enhancement processing includes normalizing pixel values in the training face images and performing illumination enhancement on the normalized training face images based on a preset illumination coefficient; The initial training data set after the illumination enhancement processing is input into a preset initial detection model as a face training set, and the initial detection model is trained to obtain the gender and age detection model.
8. The method for detecting the gender and age of a person in a vehicle according to claim 1, characterized in that: After obtaining the gender detection result of the face image to be detected based on the average of the multiple gender detection values, and obtaining the age detection result of the face image to be detected based on the average of the multiple age detection values, the method further includes: Matching a corresponding driving strategy in a preset strategy library based on the gender detection result and the age detection result; When it is detected that the vehicle speed is zero and the door is opened and closed once, the face image of the person in the vehicle to be detected is obtained again to detect the gender and age of the person in the vehicle.
9. A device for detecting the gender and age of people in a vehicle, characterized in that: The in-vehicle personnel gender and age detection device comprises: An image input module is used to obtain the original face image of the person in the car; A face alignment module, used to extract face key points from the original face image, and perform face alignment on the original face image based on the face key points to obtain a face image to be detected; A model detection module, used for inputting the face image to be detected into a gender and age detection model to obtain a plurality of gender detection values and a plurality of age detection values, wherein the gender and age detection model includes a plurality of detection heads for outputting gender detection values and age detection values; The result output module is used to obtain the gender detection result of the face image to be detected based on the average of multiple gender detection values, and to obtain the age detection result of the face image to be detected based on the average of multiple age detection values.
10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method for detecting the gender and age of a person in a vehicle as described in any one of claims 1-7.
