A method, device, vehicle and storage medium for detecting in-vehicle makeup

Abstract

The present application discloses a method, device, vehicle and storage medium for detecting in-vehicle makeup. The method includes: obtaining the ambient light intensity inside the vehicle; if the ambient light intensity is less than a preset light intensity threshold, controlling the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtaining the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; if it is determined based on the face image that the user's makeup does not meet the predetermined conditions, outputting makeup prompt information for prompting the user to touch up the makeup. The present invention ensures that the ambient light intensity is greater than the preset light intensity threshold, uses the camera to capture the user's face image under this ambient light intensity condition, detects the user's makeup, and when it is determined that the user's makeup does not meet the predetermined conditions, prompts the user to touch up the makeup, so that the user can touch up the makeup without being affected by the ambient light intensity, saving the user's time and allowing the user to have a perfect makeup.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and more specifically, to a method, device, vehicle and storage medium for detecting in-vehicle makeup. Background Art

[0002] Keeping beautiful at all times is an important part of the lives of women of all ages. Women's makeup has different requirements for ambient light, temperature, social scenes, etc. In the related art, a method for detecting the user's makeup and guiding the user to put on makeup through a camera is proposed. However, due to the different makeup environments of the users, the technology has the problem of inaccurate detection of the makeup effect, which affects the user's makeup effect and wastes the user's makeup time. Summary of the invention

[0003] The present invention proposes a method, device, vehicle and storage medium for detecting in-vehicle makeup to improve the above-mentioned problems.

[0004] In the first aspect, the embodiment of the present application provides a method for detecting makeup in a vehicle, the method comprising: obtaining the ambient light intensity inside the vehicle; if the ambient light intensity is less than a preset light intensity threshold, controlling the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtaining a face image of a user captured by a first camera when the ambient light intensity is greater than the preset light intensity threshold; if it is determined according to the face image that the makeup of the user does not meet a predetermined condition, outputting makeup prompt information for prompting the user to touch up the makeup. .

[0005] In the second aspect, the embodiment of the present application also provides an in-vehicle makeup detection device, which includes: a light intensity acquisition unit, a fill light unit, a face image acquisition unit, and a makeup touch-up prompt unit. Among them, the light intensity acquisition unit is used to obtain the ambient light intensity inside the vehicle; the fill light unit is used to control the fill light to light up if the ambient light intensity is less than a preset light intensity threshold, so that the ambient light intensity after fill light is greater than the preset light intensity threshold; the face image acquisition unit is used to obtain the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; the makeup touch-up prompt unit is used to output makeup touch-up prompt information for prompting the user to touch up the makeup if it is determined based on the face image that the user's makeup does not meet the predetermined conditions.

[0006] In a third aspect, the embodiment of the present application further provides a vehicle, comprising: a fill light, a light intensity detector, a first camera, one or more processors, a memory, and one or more applications. The light intensity detector is used to detect the ambient light intensity in the environment in which it is located; the first camera is used to collect facial images; one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to be executed to implement the method described in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the method described in the first aspect above.

[0008] The technical solution provided by the present invention is to obtain the ambient light intensity inside the vehicle; if the ambient light intensity is less than the preset light intensity threshold, control the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtain the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; if it is determined based on the face image that the user's makeup does not meet the predetermined conditions, output makeup touch-up prompt information for prompting the user to touch up makeup. Therefore, the above method of the present application is adopted to obtain the ambient light intensity inside the vehicle, and to perform fill light when the ambient light intensity is less than the preset light intensity threshold, so as to ensure that the facial image for detecting the user's makeup captured by the first camera is under the condition of an ambient light intensity greater than the preset light intensity threshold, and when it is determined that the user's makeup does not meet the predetermined conditions, prompt the user to touch up makeup, so that the user can touch up makeup without being affected by the ambient light intensity, so as to save the user's time and enable the user to have a perfect makeup. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram showing a flow chart of a method for detecting in-vehicle makeup according to another embodiment of the present application is shown;

[0011] Figure 2 A schematic diagram showing a flow chart of a method for detecting in-vehicle makeup according to another embodiment of the present application is shown;

[0012] Figure 3 A schematic diagram showing a flow chart of a method for detecting in-vehicle makeup according to another embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram showing a flow chart of a method for detecting in-vehicle makeup according to another embodiment of the present application is shown;

[0014] Figure 5 A schematic diagram of the structure of a vehicle interior makeup detection device proposed in one embodiment of the present application is shown;

[0015] Figure 6 A structural block diagram of a vehicle proposed in an embodiment of the present application is shown;

[0016] Figure 7 A structural block diagram of a computer-readable storage medium proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] With the accelerated pace of life and frequent overtime work, people have less and less time to put on makeup. Sometimes people want to sleep a little longer in the morning, so the time for makeup in the morning is significantly compressed. Many times when people don’t have time to put on makeup at home, they will put on makeup or touch up makeup on the way to transportation.

[0019] The inventor has discovered through research that in the related technologies, there are control methods that use cameras to detect user makeup and guide users to put on makeup, but they are unable to solve the problem of putting on makeup while driving (such as passing through underground garages, tunnels, shaded areas, etc.). Due to changes in light, the camera cannot accurately judge the makeup effect when detecting the user's makeup, which affects the user's makeup effect and wastes the user's makeup time.

[0020] In order to alleviate the above problems, the present application proposes a method, device, vehicle and storage medium for in-vehicle makeup detection. By obtaining the ambient light intensity inside the vehicle, fill light is performed when the ambient light intensity is less than a preset light intensity threshold, ensuring that the face image for detecting the user's makeup collected by the first camera is under an ambient light intensity greater than the preset light intensity threshold, and when it is determined that the user's makeup does not meet the predetermined conditions, the user is prompted to touch up the makeup, so that the user can touch up the makeup without being affected by the ambient light intensity, saving the user's time and giving the user a perfect makeup.

[0021] The following will describe the various embodiments of the present application in detail with reference to the accompanying drawings. Figure 1 In one embodiment of the present application, a method for detecting in-vehicle makeup is provided, which can be applied to a vehicle equipped with a camera, a light intensity detector, a central processing unit, and a fill light. The method is not limited here, and can be specifically controlled and executed by an application installed in the vehicle. The method for detecting in-vehicle makeup may include steps S110 to S140.

[0022] Step S110: Acquire the ambient light intensity inside the vehicle.

[0023] In an embodiment of the present application, before obtaining the ambient light intensity inside the vehicle, the method further includes: turning on a smart makeup touch-up function.

[0024] In some embodiments, the vehicle's smart makeup function is actively turned on by the user. Specifically, the vehicle receives the user's instruction to turn on the smart makeup function, which may be a voice instruction from the user to turn on the smart makeup function, or a touch / button control instruction from the user to turn on the smart makeup function, or a wireless communication technology / SPI (serial peripheral interface) to obtain the user's instruction to turn on the smart makeup function from an associated device. After the vehicle turns on the smart makeup function, it will prompt the user (such as a voice prompt, an indicator light prompt, or a prompt interface that the smart makeup function has been turned on) that the smart makeup function has been turned on.

[0025] In the embodiment of the present application, after the vehicle turns on the intelligent makeup touch-up function, the ambient light intensity inside the vehicle is obtained. The method of obtaining the ambient light intensity inside the vehicle can be to obtain the ambient light intensity using a photosensitive sensor, or to obtain the ambient light intensity inside the vehicle using a light intensity detector, or to obtain the ambient light intensity inside the vehicle based on a neural network, which is not limited here.

[0026] For example, the ambient light intensity inside the vehicle is measured using a photosensitive sensor, wherein the photosensitive sensor can be a photoresistor type / photodiode type / photovoltaic cell type / thermal effect infrared light type, and its working principle is to use a photosensitive element to convert a light signal into an electrical signal, and it has the characteristics of non-contact, fast response, and reliable performance. For example, the ambient light intensity is obtained using a light sensor, which is developed based on the principle of the photoelectric effect of semiconductors, and can detect the ambient light intensity of the surrounding environment, and can also detect the difference in light on surfaces of different colors.

[0027] In some embodiments, when a vehicle interior makeup detection method provided by an embodiment of the present application is applied to a vehicle equipped with a makeup mirror, obtaining the ambient light intensity inside the vehicle may be obtaining the ambient light intensity inside the vehicle when it is detected that the makeup mirror is being used.

[0028] Specifically, the way to detect whether the makeup mirror is in use may be to use relevant sensors to detect the light on the mirror surface, the position of the makeup mirror, the temperature of the makeup mirror, and whether the makeup mirror is unfolded, etc. to detect whether the makeup mirror is in use, or to use a camera to capture an image in the makeup mirror to detect whether the makeup mirror is in use. For example, a position sensor is used to detect whether the makeup mirror is in use. When the position sensor detects that the position of the makeup mirror has changed, it is determined that the makeup mirror is in use. It should be understood that the makeup mirror may be a mirror, a display, or other equipment that can show the user's makeup to the user and assist the user in putting on makeup.

[0029] In some embodiments, when it is detected that the makeup mirror is being used, an image in the makeup mirror is obtained. If the image in the makeup mirror does not include a complete facial image, the seat position of the user corresponding to the facial image is adjusted so that the image in the makeup mirror obtained after adjusting the seat position includes a complete facial image.

[0030] As an embodiment, the vehicle includes a display, and a first camera is arranged on the display. The display can show the user's makeup to the user. When it is detected that the display is in use, the first camera captures the image in the display. If the image in the display does not include a complete facial image, the position of the seat opposite to the display is adjusted so that after the seat position is adjusted, the image in the display captured by the first camera includes a complete facial image.

[0031] As another embodiment, the vehicle also includes a second camera, which is arranged at the position of the headrest of the vehicle seat and is used to capture the image in the makeup mirror; when it is detected that the makeup mirror is in use, the image in the makeup mirror captured by the second camera is obtained; if the image in the makeup mirror captured by the second camera does not include a complete facial image, the position of the seat where the second camera is located is adjusted so that after the seat position is adjusted, the image in the makeup mirror captured by the second camera includes a complete facial image.

[0032] Step S120: If the ambient light intensity is less than the preset light intensity threshold, the fill light is controlled to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold.

[0033] In the embodiment of the present application, the preset light intensity threshold may be pre-stored in the storage unit of the vehicle, or obtained from the associated cloud / device through wireless communication technology, or obtained from the associated device through SPI; the size of the preset light intensity threshold may be obtained from third-party experimental data. For example, the preset light intensity threshold is 900 lux pre-stored in the storage unit of the vehicle. It should be understood that the preset light intensity threshold can be actually changed according to user needs.

[0034] Among them, the fill light can be a makeup mirror light, a camera fill light, or other lighting devices on the vehicle. The fill light can be one light or multiple lights, and the position of the vehicle can be set according to the needs of the user. It should be noted that after the fill light is turned on, it can ensure that the ambient light intensity after the fill light is greater than the preset light intensity threshold. For example, the fill light is a makeup mirror light, and the makeup mirror light can adjust the brightness. Its main function is to provide auxiliary light when the ambient light intensity is less than the preset ambient light intensity, so as to obtain an image that meets the user's needs. The makeup mirror light can use an LED light emitting diode as the light source, and can be powered by a rechargeable battery or by external AC power.

[0035] Step S130: Acquire a facial image of the user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold.

[0036] As an implementation method, when the acquired ambient light intensity is greater than a preset light intensity threshold, the first camera captures a facial image of the user, and the vehicle obtains basic facial information of the user captured by the camera and recorded by the first camera sensor, thereby obtaining the facial image captured by the first camera.

[0037] As another implementation, when the acquired ambient light intensity is less than a preset light intensity threshold, the fill light is controlled to light up, and when the ambient light intensity after fill light is greater than the preset light intensity threshold, the first camera captures the user's facial image, and the vehicle obtains the basic facial information of the user captured by the camera and recorded by the first camera sensor, thereby obtaining the facial image captured by the first camera.

[0038] It should be noted that when obtaining the facial image of the user captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold, the camera range of the first camera can ensure that the full face image of the user is captured. For example, the first camera can be set on the upper part of the A-pillar of the vehicle or on the base of the rearview mirror to ensure that the full face image of the front passenger is obtained.

[0039] Step S140: if it is determined based on the face image that the makeup of the user does not meet a predetermined condition, output makeup touch-up prompt information for prompting the user to touch up the makeup.

[0040] In an embodiment of the present application, a method for determining whether the user's makeup meets predetermined conditions based on the facial image may be, after obtaining a full-face image of the user using a first camera, using an AI processor to evaluate the facial makeup of the full-face image, and comparing the evaluation result with the predetermined condition; when the evaluation result meets the predetermined condition, it is determined that the user's makeup meets the predetermined condition; otherwise, it is determined that the user's makeup does not meet the predetermined condition.

[0041] Wherein, the facial makeup may include facial makeup such as facial blemishes, skin color uniformity, eyebrow shape, lipstick gloss, etc. It should be noted that the facial makeup of the full face image is evaluated using an AI processor, wherein the makeup dimensions evaluated may be basic dimensions for makeup evaluation, or may be expanded to include more dimensions, such as shadows, hairlines, blush, etc., which may be set according to user needs. The evaluation result may be a score for facial makeup (such as a score for facial blemishes, a score for skin color uniformity, a score for eyebrow shape, and a score for lipstick gloss). The predetermined condition may be that the evaluation score for facial makeup is greater than a standard evaluation score. The standard evaluation score may be pre-stored in a storage unit of the vehicle, or may be obtained from an associated cloud / device through wireless communication technology, or may be obtained from an associated device through SPI. It should be noted that the predetermined condition may be adjusted and customized according to user needs.

[0042] In the embodiment of the present application, the makeup prompt information output for prompting the user to touch up the makeup may be that the vehicle directly outputs voice prompts, plays videos, generates text display interfaces, etc. to prompt the user to touch up the makeup, or the vehicle outputs voice prompts, plays videos, generates text display interfaces, etc. prompt information to the associated device through wireless communication technology, and the associated device outputs the makeup prompt information for prompting the user to touch up the makeup. Wherein, the makeup prompt information for prompting the user to touch up the makeup may include at least one of the name of the part to be touched up and the color of the cosmetics required for the makeup. Wherein, the name of the part to be touched up may be eyebrows, eyelashes, lips, eye makeup, etc.; the color of the cosmetics required for the makeup may be the color of the eyebrow pencil, the color of the blush, the color of the lipstick, the color of the eye shadow, etc. For example, the makeup prompt information for prompting the user to touch up the makeup is voice information: lips, lipstick, bright red.

[0043] In some embodiments, if it is determined based on the facial image that the makeup of the user meets a predetermined condition, information is output to prompt the user that the makeup is completed. The information output to prompt the user that the makeup is completed may be that the vehicle directly outputs a voice prompt, plays a video, generates a text display interface, etc. to prompt the user that the makeup is completed. It may also be that the vehicle outputs a voice prompt, plays a video, generates a text display interface, or other prompt information through wireless communication technology to an associated device, and the associated device outputs an alert to prompt the user that the makeup is completed.

[0044] Exemplarily, the vehicle includes a sun visor makeup mirror, a makeup mirror light, a central processing unit, a light sensor, a first camera, and a display screen. The storage unit of the vehicle stores a preset light intensity threshold of 900 lux and a predetermined condition that the standard score of facial makeup (facial blemishes, skin color uniformity, eyebrow shape, lipstick gloss) is 80 points. After the user turns on the smart makeup touch-up function, he opens the sun visor makeup mirror to touch up his makeup. The central processing unit obtains the ambient light intensity X inside the vehicle from the in-vehicle light sensor. When X>900 lux, or when X<900 lux and the makeup mirror light is turned on, the obtained ambient light intensity Y>900 lux, the first camera continuously collects the user's facial image, the central processing unit continuously performs AI processing on the facial image obtained by the first camera, and the AI ​​processor scores the facial makeup (facial blemishes, skin color uniformity, eyebrow shape, lipstick gloss) of the facial image, and displays the facial image. The screen displays the score. When the facial blemish score is greater than 80 points, it displays "Facial blemish OK"; when the skin tone uniformity score is greater than 80 points, it displays "Skin tone uniformity OK"; when the eyebrow shape score is greater than 80 points, it displays "Eyebrow shape OK"; when the lipstick gloss score is greater than 80 points, it displays "Lipstick gloss OK", and a voice reminds the user that the makeup touch-up is completed; if the score of the facial makeup (facial blemishes, skin tone uniformity, eyebrow shape, lipstick gloss) is ≤80 points, the user is reminded to touch up the makeup, and the large screen displays the specific facial makeup score below 80 points, and a voice prompts the user of the specific makeup touch-up area.

[0045] In some embodiments, the vehicle includes a vanity mirror, and a vanity mirror light is provided on the vanity mirror, and the brightness of the vanity mirror light can be adjusted; when the user uses the vanity mirror to assist in makeup, the brightness of the vanity mirror light provided on the vanity mirror can be adjusted according to the ambient light intensity when the user is applying makeup and the preset light intensity threshold. Exemplarily, when the preset light intensity threshold is set to 900 lux, when the acquired ambient light intensity is X, when X>900 lux, the vanity mirror light is not turned on; when X<900 lux, the vanity mirror light is turned on for fill light, and the brightness of the vanity mirror light is adjusted to Y (Y=900-X).

[0046] The embodiment of the present application proposes a method for detecting makeup in a car, which obtains the ambient light intensity inside the vehicle; if the ambient light intensity is less than a preset light intensity threshold, controls the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtains the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; if it is determined based on the face image that the user's makeup does not meet the predetermined conditions, outputs makeup touch-up prompt information for prompting the user to touch up makeup. By adopting the above method, it is ensured that under the condition that the ambient light intensity inside the vehicle is greater than the preset light intensity threshold, the user's face image is captured by the first camera, and the makeup of the face image is detected. When it is determined that the makeup does not meet the predetermined conditions, the user is prompted to touch up makeup, so that the user can touch up makeup without being affected by the ambient light intensity inside the car, saving the user's time and allowing the user to have a perfect makeup.

[0047] See also Figure 2 Another embodiment of the present application provides a method for detecting in-vehicle makeup, which includes steps S210 to S250.

[0048] Step S210: Acquire the ambient light intensity inside the vehicle.

[0049] Step S220: If the ambient light intensity is less than the preset light intensity threshold, the fill light is controlled to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold.

[0050] Step S230: Acquire a facial image of the user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold.

[0051] The specific execution process of step S210 to step S230 can refer to the specific description of step S110 to step S130 above, which will not be repeated here.

[0052] Step S240: Calculate the similarity between the face image and a preset image to obtain a similarity value.

[0053] In the embodiment of the present application, the similarity calculation between the face image and the preset image to obtain the similarity value may be: segmenting the face image to obtain a lip image, an eyebrow image, an eye image and a facial image, wherein the facial image does not include lips, eyebrows and eyes. The lip image is calculated to obtain a lip similarity value by similarity calculation between the lip image and the preset lip image, the eyebrow image is calculated to obtain an eyebrow similarity value by similarity calculation between the eye image and the preset eye image, the eye similarity value is calculated by similarity calculation between the eye image and the preset eye image, and the facial image is calculated to obtain a facial similarity value by similarity calculation between the facial image and the preset facial image.

[0054] The facial image may be segmented by using image processing methods such as threshold segmentation, region segmentation, edge segmentation, and histogram. For example, the facial image may be segmented by using a k-means algorithm (a clustering-based edge detection method), image clustering, boundary tracking, and face location, and finally, a lip image, an eyebrow image, an eye image, and a facial image in the facial image may be obtained, wherein the facial image does not include lips, eyebrows, and eyes.

[0055] In an embodiment of the present application, the preset lip image, preset eyebrow image, preset eye image, and preset facial image may be obtained through third-party experimental data; may be pre-stored in a storage unit of the vehicle, or may be obtained from an associated cloud / device through wireless communication technology, or may be obtained from an associated device through SPI.

[0056] Among them, the method of calculating the similarity between the facial image and the preset image can be to use a feature point detection algorithm such as SURF algorithm, Susan operator algorithm, RANSC algorithm, hash algorithm, etc. to calculate the similarity between the facial image and the preset image, such as using a hash algorithm to obtain a comparison array of grayscale pixels between the facial image and the preset image, obtain the Hamming distance between the facial image and the preset image, and calculate the similarity of the two images through the Hamming distance.

[0057] Step S250: If the similarity value is less than a preset similarity threshold, the makeup of the user does not meet the predetermined condition, and makeup touch-up prompt information is output to prompt the user to touch up the makeup.

[0058] In an embodiment of the present application, the preset similarity threshold may be obtained through a third-party data experiment or may be set independently by a user; it may be pre-stored in a storage unit of the vehicle or obtained from an associated cloud / device through wireless communication technology or obtained from an associated device through SPI.

[0059] It should be noted that if the similarity value is less than the preset similarity threshold, including if there is a target similarity value less than the corresponding preset similarity threshold among the lip similarity value, the eyebrow similarity value, the eye similarity value and the face similarity value, then the user's makeup does not meet the predetermined condition. Among them, the target similarity value is the similarity value less than the preset similarity threshold.

[0060] Exemplarily, the preset similarity thresholds obtained through third-party data experiments include: a lip preset similarity threshold of 80%, an eyebrow preset similarity threshold of 80%, an eye preset similarity threshold of 80%, and a face preset similarity threshold of 80%. The above preset similarity thresholds are pre-stored in a storage unit of the terminal device, and the similarity values ​​obtained by calculating the similarity values ​​between the facial image and the preset image are a lip similarity value of 76% (less than 80%), an eyebrow similarity value of 86% (greater than 80%), an eye similarity value of 82% (greater than 80%), and a face similarity value of 83% (greater than 80%). Because the lip similarity value of 76% (less than 80%) is less than the corresponding lip preset similarity threshold, the user's makeup does not meet the predetermined conditions.

[0061] The specific execution process of outputting the makeup touch-up prompt information for prompting the user to touch up makeup in step S250 can refer to the specific description of step S140 above, which will not be repeated here. It should be noted that in the embodiment of the present application, the makeup touch-up prompt information can also include displaying the similarity value obtained by calculating the similarity value between the face image and the preset image.

[0062] The embodiment of the present application proposes a method for detecting makeup in a car, which obtains the ambient light intensity inside the car; if the ambient light intensity is less than the preset light intensity threshold, controls the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtains the face image of the user captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; calculates the similarity between the face image and the preset image to obtain a similarity value; if the similarity value is less than the preset similarity threshold, the makeup of the user does not meet the preset condition; and outputs makeup touch-up prompt information for prompting the user to touch up the makeup. By adopting the above method, it is ensured that the face image captured by the first camera for detecting the makeup of the user is under the condition that the ambient light intensity inside the vehicle is greater than the preset light intensity threshold, and by comparing the similarity between the face image and the preset image, it is determined whether the makeup of the user meets the preset condition. At the same time, when the makeup of the user does not meet the preset condition, the user is prompted to touch up the makeup, so that the user can touch up the makeup in the car without being affected by the ambient light intensity, saving the user's time and allowing the user to have a perfect makeup.

[0063] See also Figure 3 Another embodiment of the present application provides a method for detecting in-vehicle makeup, which includes steps S310 to S360.

[0064] Step S310: Acquire the ambient light intensity inside the vehicle.

[0065] Step S320: If the ambient light intensity is less than the preset light intensity threshold, the fill light is controlled to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold.

[0066] Step S330: Acquire a facial image of the user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold.

[0067] The specific execution process of step S310 to step S330 can refer to the specific description of step S110 to step S130 above, which will not be repeated here.

[0068] Step S340: inputting the face image into a preset makeup detection model to obtain makeup information corresponding to the face image.

[0069] In an embodiment of the present application, the preset makeup detection model can be pre-stored in a storage unit of the vehicle, or obtained from a cloud or device associated with the vehicle through wireless communication technology, or obtained from an associated device using SPI; the preset makeup detection model can be obtained by training with third-party experimental data, and the preset makeup detection model can be an AlexNet model, a VGG19 (fast neural network) model, a ResNet_152 (residual network) model, an InceptionV4 model, a DenseNet model, an OverFeat model, an R-CNN model, a SPP-net model, a FAST R-CNN model, an R-FCN model, a YOLO model, or other models that can perform image detection. For example, the Fast R-CNN model is used as the preset makeup detection model to detect face images. The main steps of the detection are: input the entire face image into the model, and use CNN (convolutional neural network) to obtain the feature layer of the image; use Selective Search and other methods to extract candidate frames from the original image, and project these candidate frames one by one to the final feature layer; perform RoI Pooling operation on each candidate region candidate frame on the feature layer to obtain a fixed-size feature representation; then pass through two fully connected layers, use Softmax (logistic regression model) multi-classification for target recognition, use the regression model to fine-tune the border position and size, and obtain the correspondence between the face image and the makeup information, such as obtaining the eyebrow shape, eye shadow color concentration, lipstick color depth, blush color concentration, etc. in the face image.

[0070] Step S350: Determine whether the makeup of the user meets a predetermined condition according to the makeup information.

[0071] In the embodiment of the present application, the predetermined condition may be pre-stored in a storage unit of the vehicle, or may be obtained from a cloud or device associated with the vehicle through wireless communication technology, or may be obtained from an associated device using SPI. The predetermined condition is obtained through training with third-party experimental data, and the predetermined condition may include standard eyebrow shape, standard eyeshadow color concentration, standard lipstick color depth, standard blush color concentration, etc.

[0072] Determining whether the makeup of the user meets the predetermined condition according to the makeup information may be to compare the correspondence between the face image and the makeup information obtained by the preset makeup detection model with the predetermined condition, and if the makeup information of the face image meets the predetermined condition, then it is determined that the makeup of the user meets the predetermined condition; otherwise, the makeup of the user does not meet the predetermined condition. For example, the predetermined condition is: the standard concentration of eyelashes in the correspondence between the face image and the makeup information is greater than or equal to 50%, the standard eyebrow shape of eyebrows in the correspondence between the face image and the makeup information is willow-leaf eyebrows, and the standard concentration of the color is greater than or equal to 60%, the standard color of eyeshadow in the correspondence between the face image and the makeup information is earth color and the standard concentration is 40%, the standard color of lips in the correspondence between the face image and the makeup information is bright red and the standard concentration is 60%, and the standard color of blush in the correspondence between the face image and the makeup information is bright red and the standard concentration is 20%, then it is determined that the makeup of the user meets the predetermined condition, otherwise, the makeup of the user does not meet the predetermined condition. When the preset makeup detection model obtains that the concentration of eyelashes in the correspondence between the facial image and the makeup information is 80% (greater than 50%), the eyebrow shape in the correspondence between the facial image and the makeup information is willow-leaf eyebrow and the concentration of eyebrow color is 78% (greater than 60%), the eyeshadow color in the correspondence between the facial image and the makeup information is earth color and the concentration is 30% (greater than 40%), the lip color in the correspondence between the facial image and the makeup information is bright red and the concentration is 56% (greater than 60%), and the blush color in the correspondence between the facial image and the makeup information is bright red and the concentration is 28% (greater than 20%), it is determined that the user's makeup meets the predetermined conditions.

[0073] Step S360: if it is determined according to the makeup information that the makeup of the user does not meet the predetermined condition, output makeup touch-up prompt information for prompting the user to touch up the makeup.

[0074] The specific execution process of step S360 can refer to the specific description of step S140 above, which will not be repeated here. It should be noted that in the embodiment of the present application, the output of the makeup prompt information for prompting the user to touch up the makeup can also include displaying the corresponding relationship between the face image and the makeup information.

[0075] The embodiment of the present application proposes a method for detecting makeup in a car, which includes obtaining the ambient light intensity inside the car; if the ambient light intensity is less than a preset light intensity threshold, controlling the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtaining a facial image of a user captured by a first camera when the ambient light intensity is greater than the preset light intensity threshold; inputting the facial image into a preset makeup detection model to obtain makeup information corresponding to the facial image; determining whether the makeup of the user meets a predetermined condition according to the makeup information; and outputting makeup touch-up prompt information for prompting the user to touch up makeup if it is determined according to the makeup information that the makeup of the user does not meet the predetermined condition. By adopting the above method, it is ensured that the facial image for detecting the makeup of the user captured by the first camera is under the condition that the ambient light intensity inside the car is greater than the preset light intensity threshold, and the facial image is input into the preset makeup detection model to determine whether the makeup of the user meets the predetermined condition. At the same time, when the makeup of the user does not meet the preset condition, the user is prompted to touch up makeup, so that the user can touch up makeup in the car without being affected by the ambient light intensity, saving the user's time and enabling the user to have a perfect makeup.

[0076] See also Figure 4 Another embodiment of the present application provides a method for detecting in-vehicle makeup, which includes steps S410 to S460.

[0077] Step S410: Acquire the user image captured by the first camera.

[0078] In an embodiment of the present application, after the smart makeup touch-up function is turned on, the first camera captures the user's full face, and the vehicle obtains the basic facial information of the user captured by the first camera and recorded by the camera sensor, thereby obtaining the facial image captured by the first camera.

[0079] Exemplarily, when the first camera is installed on the A-pillar of the vehicle for capturing images of the driver's seat and the front passenger seat area in the vehicle, when there is a user in the driver's seat / front passenger seat area of ​​the vehicle, the first camera can capture a full facial image of the user; when the first camera is installed on the B-pillar of the vehicle for capturing images of the rear seat area in the vehicle, when there is a user in the rear seat area of ​​the vehicle, the first camera can capture a full facial image of the user.

[0080] Step S420: Detect whether the user image is a female user image.

[0081] In some embodiments, the detection of whether the user image is a female user image may be to scan the entire user image using a model such as an EigenFace (Eigenface) algorithm, a FisherFace algorithm, a LBPH algorithm, a CNN (Convolutional Neural Network), a RNN (Recurrent Neural Network), a GAN (Generative Adversarial Network), etc., to detect whether the face user image is a female user image. For example, the FisherFace algorithm mainly uses the idea of ​​LDA (Linear Projection Analysis) to project male and female samples in the sample space onto a straight line passing through the origin, and ensure that the projection distance of the sample on the line is the smallest within the class and the distance between the classes is the largest, thereby separating the dividing line for identifying males and females, and distinguishing the gender of the user image based on the dividing line. When the FisherFace algorithm is used to scan the user image, the user image is projected onto a straight line passing through the origin, and based on the relationship between the straight line and the dividing line, it is detected whether the user image is a female user image.

[0082] Step S430: If yes, obtain the ambient light intensity.

[0083] Step S440: If the ambient light intensity is less than the preset light intensity threshold, the fill light is controlled to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold.

[0084] Step S450: Acquire a facial image of the user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold.

[0085] Step S460: if it is determined based on the face image that the makeup of the user does not meet a predetermined condition, output makeup touch-up prompt information for prompting the user to touch up the makeup.

[0086] The specific execution process of step S430 to step S460 can refer to the specific description of step S110 to step S140 above, which will not be repeated here.

[0087] The embodiment of the present application proposes a method for detecting makeup in a car, which is to obtain a user image captured by a first camera; detect whether the user image is a female user image; if so, obtain the ambient light intensity; if the ambient light intensity is less than a preset light intensity threshold, control the fill light to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; obtain the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; if it is determined based on the face image that the user's makeup does not meet the predetermined conditions, output makeup touch-up prompt information for prompting the user to touch up the makeup. By adopting the above method, when it is detected that the user is a female, it is ensured that the face image captured by the first camera for detecting the user's makeup is under the condition that the ambient light intensity is greater than the preset light intensity threshold, and when it is determined that the user's makeup does not meet the predetermined conditions, the user is prompted to touch up the makeup, so that the user can touch up the makeup in the car without being affected by the ambient light intensity, saving the user's time and allowing the user to have a perfect makeup.

[0088] See also Figure 5 , which shows an in-vehicle makeup detection device proposed in an embodiment of the present application, the device 500 includes: a light intensity acquisition unit 510, a fill light unit 520, a face image acquisition unit 530 and a touch-up makeup prompt unit 540. Specifically, the light intensity acquisition unit 510 is used to acquire the ambient light intensity inside the vehicle; the fill light unit 520 is used to control the fill light to light up if the ambient light intensity is less than a preset light intensity threshold, so that the ambient light intensity after fill light is greater than the preset light intensity threshold; the face image acquisition unit 530 is used to acquire the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold; the touch-up makeup prompt unit 540 is used to output a touch-up makeup prompt information for prompting the user to touch up makeup if it is determined from the face image that the user's makeup does not meet a predetermined condition.

[0089] In some embodiments, the light intensity acquisition unit 510 includes a light sensor. After the user turns on the smart makeup touch-up function, when it is detected that the user uses a makeup mirror, the ambient light intensity is acquired using the light sensor. The fill light unit 520 includes a makeup mirror light. If the acquired ambient light intensity is less than a preset light intensity threshold, the makeup mirror light is controlled to light up for fill light, so that the ambient light intensity after fill light is greater than the preset light intensity threshold; when the user touches up the makeup, the brightness of the makeup mirror light set on the makeup mirror is adjusted according to the ambient light intensity of the user when touching up the makeup and the preset light intensity threshold. The face image acquisition unit 530 includes a first camera, which acquires the user's face image captured by the first camera when the ambient light intensity is greater than the preset light intensity threshold. The makeup touch-up prompt unit 540 is used to segment the face image to obtain a lip image, an eyebrow image, an eye image and a facial image, wherein the facial image does not include lips, eyebrows and eyes; perform similarity calculation between the lip image and a preset lip image to obtain a lip similarity value, perform similarity calculation between the eyebrow image and a preset eyebrow image to obtain an eyebrow similarity value, perform similarity calculation between the eye image and a preset eye image to obtain an eye similarity value, and perform similarity calculation between the facial image and a preset facial image to obtain a facial similarity value; if there is a target similarity value less than the corresponding preset threshold value among the lip similarity value, the eyebrow similarity value, the eye similarity value and the facial similarity value, the user's makeup does not meet the predetermined condition, and output makeup touch-up prompt information for prompting the user to touch up the makeup.

[0090] It should be noted that each embodiment of this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0091] See also Figure 6Based on the above-mentioned in-vehicle makeup detection method, the present application also provides another vehicle 600 including a processor that can execute the above-mentioned in-vehicle makeup detection method, the vehicle 600 includes: a fill light 610, a light intensity detector 620, a first camera 630, one or more processors 640, a memory 650, and one or more applications. Among them, the fill light 610 is used to provide auxiliary light for fill light when the acquired ambient light intensity inside the vehicle 600 is less than a preset light intensity threshold, so that the ambient light intensity after fill light is greater than the preset light intensity threshold; the light intensity detector 620 is used to monitor the ambient light intensity in the environment in which it is located; the first camera 630 can collect the face image in the above-mentioned embodiment; the processor 640 can execute the program stored in the memory 650, and the memory 650 stores the program that can execute the content of the above-mentioned embodiment.

[0092] Among them, the light intensity detector 620 can be composed of a photosensitive sensor, such as a light sensor. Among them, the photosensitive sensor can be a photoresistor type / photodiode type / photovoltaic cell type / thermal effect infrared light type, and its working principle is to use a photosensitive element to convert a light signal into an electrical signal to obtain the ambient light intensity.

[0093] The camera 630 may include a lens part, an LED part, a chip part, a PCB and a component part. The optical image of the user's face generated through the lens is projected onto the surface of the image sensor, and then converted into an electrical signal and transformed into a digital image signal after A / D conversion. The signal is then sent to a digital signal processing chip for processing and then transmitted to a processor such as a computer through a USB interface to generate a facial image.

[0094] The processor 640 may include one or more cores for processing data and a message matrix unit. The processor 640 uses various interfaces and lines to connect various parts of the entire vehicle 600, and executes various functions of the vehicle 600 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 650, and calling data stored in the memory 650, such as AI processing of facial makeup of the acquired user's face image. Optionally, the processor 640 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 640 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 640, but may be implemented separately through a communication chip.

[0095] The memory 650 may include a random access memory (RAM) or a read-only memory (ROM). The memory 6550 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 650 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a navigation destination, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the terminal during use (such as effect diagrams of different makeup types, different makeup tutorial videos, predetermined conditions, etc.).

[0096] In some embodiments, the vehicle 600 further includes a second camera, a makeup mirror, and a makeup mirror light. The second camera is arranged on the seat of the vehicle 600, and is used to capture the image of the makeup mirror. The makeup mirror light is arranged on the makeup mirror, and the makeup mirror is used to assist the user in applying makeup. The makeup mirror light is used to fill in light, so that the ambient light intensity after fill in light is greater than a preset light intensity threshold. The first camera 630 is arranged on the vehicle 600, and is used to capture the image of the designated area in the vehicle 600. Exemplarily, when the first camera 630 is installed on the upper part of the A-pillar of the vehicle 600 or on the base of the rearview mirror to capture the image of the driver's seat and the co-pilot seat area in the vehicle 600, when there is a user in the driver's seat / co-pilot seat area of ​​the vehicle 600, the first camera 630 can capture the full face image of the user; when the first camera 630 is installed on the B-pillar of the vehicle 600 to capture the image of the rear seat area in the vehicle 600, when there is a user in the rear seat area of ​​the vehicle 600, the first camera 630 can capture the full face image of the user.

[0097] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 700 stores a program code 710, which can be called by a processor to execute the method described in the above method embodiment.

[0098] The computer readable storage medium 700 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer readable storage medium 700 has storage space for program codes 710 that perform any method steps in the above method. These program codes 710 can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting makeup in a car, characterized in that: The method comprises: When it is detected that the vanity mirror is in use, obtaining the ambient light intensity inside the vehicle and obtaining an image in the vanity mirror, wherein the vanity mirror is used to display the user's makeup; If the image in the vanity mirror does not include a complete face image, adjusting the seat position of the vehicle so that the image in the vanity mirror obtained after adjusting the seat position includes a complete face image; If the ambient light intensity is less than the preset light intensity threshold, the fill light is controlled to light up so that the ambient light intensity after fill light is greater than the preset light intensity threshold; Acquire a facial image of a user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold; If it is determined according to the face image that the makeup of the user does not meet a predetermined condition, makeup touch-up prompt information is output to prompt the user to touch up the makeup.

2. The makeup detection method according to claim 1, characterized in that: If it is determined according to the face image that the makeup of the user does not meet the predetermined condition, before outputting makeup touch-up prompting information for prompting the user to touch up the makeup, the method includes: Calculating the similarity between the face image and a preset image to obtain a similarity value; If the similarity value is less than a preset similarity threshold, the makeup of the user does not meet the predetermined condition.

3. The makeup detection method according to claim 1, characterized in that: If it is determined according to the face image that the makeup of the user does not meet the predetermined condition, before outputting makeup touch-up prompting information for prompting the user to touch up the makeup, the method includes: Inputting the face image into a preset makeup detection model to obtain makeup information corresponding to the face image; Determine whether the makeup of the user meets a predetermined condition according to the makeup information.

4. The makeup detection method according to claim 1, characterized in that: After outputting makeup touch-up prompting information for prompting the user to touch up makeup, the method further includes: When the user is touching up the makeup, the brightness of the makeup mirror light provided on the makeup mirror is adjusted according to the ambient light intensity of the user when touching up the makeup and the preset light intensity threshold.

5. The makeup detection method according to claim 1, characterized in that: Get the image in the vanity mirror, including: An image of a makeup mirror captured by a second camera is acquired, where the second camera is disposed on a seat of the vehicle.

6. The makeup detection method according to claim 1, characterized in that: Before obtaining the ambient light intensity inside the vehicle, the method further includes: Obtaining a user image captured by the first camera; Detecting whether the user image is a female user image; If yes, the step of obtaining the ambient light intensity is performed.

7. The makeup detection method according to any one of claims 1 to 6, characterized in that: The makeup touch-up prompting information for prompting the user to touch up the makeup includes at least one of the name of the part to be touched up and the color of the cosmetics required for the touch-up.

8. A vehicle interior makeup detection device, characterized in that: include: a light intensity acquisition unit, configured to acquire the ambient light intensity inside the vehicle and acquire an image in the makeup mirror when detecting that the makeup mirror is in use, wherein the makeup mirror is used to display the makeup of the user; if the image in the makeup mirror does not include a complete face image, adjust the seat position of the vehicle so that the image in the makeup mirror acquired after adjusting the seat position includes a complete face image; A fill light unit, used for controlling the fill light to light up if the ambient light intensity is less than a preset light intensity threshold, so that the ambient light intensity after fill light is greater than the preset light intensity threshold; A face image acquisition unit, used to acquire a face image of a user captured by the first camera when the ambient light intensity is greater than a preset light intensity threshold; The makeup touch-up prompting unit is used to output makeup touch-up prompting information for prompting the user to touch up the makeup if it is determined based on the face image that the makeup of the user does not meet the predetermined condition.

9. A vehicle, characterized in that: include: Fill light; A light intensity detector for detecting the ambient light intensity in the environment in which it is located; A first camera, used for collecting facial images; one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. The vehicle according to claim 9, characterized in that The vehicle also includes a second camera, a vanity mirror, and a vanity mirror light; The second camera is arranged on the seat of the vehicle and is used to capture the image of the makeup mirror; the makeup mirror light is arranged on the makeup mirror, the makeup mirror is used to assist the user to touch up the makeup, and the makeup mirror light is used to fill in the light so that the ambient light intensity after the fill in light is greater than the preset light intensity threshold.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images