Skin health monitoring method based on cosmetic mirror, cosmetic mirror and storage medium

Abstract

The invention discloses a skin health monitoring method based on a cosmetic mirror, the cosmetic mirror and a storage medium, and belongs to the technical field of cosmetic mirrors. The skin health monitoring method is applied to the cosmetic mirror, the cosmetic mirror comprises an image acquisition device, and the skin health monitoring method based on the cosmetic mirror comprises the following steps: controlling the image acquisition device to acquire user image data of a target user, and determining first face data and hand data of the target user according to the user image data; adjusting the first face data according to the hand data to obtain second face data; and determining a health monitoring result of the target user according to the second face data and a recognition algorithm. The technical effect of improving the accuracy of judging the health degree of the user is achieved.

Description

Technical Field

[0001] The present invention relates to the field of cosmetic mirrors, and in particular to a cosmetic mirror-based skin health monitoring method, a cosmetic mirror and a storage medium. Background Art

[0002] With the development of technology, the requirements for the intelligence of makeup mirrors are gradually increasing. At present, smart makeup mirrors generally take pictures of faces and directly determine the user's health status through image recognition algorithms. However, this method often requires a standardized shooting process, and when the user is older or the user's skin color is more distinctive, the recognition result is prone to errors, resulting in a decrease in the accuracy of judging the user's health status.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a skin health monitoring method based on a cosmetic mirror, a cosmetic mirror and a storage medium, aiming to improve the accuracy of judging the health status of a user.

[0005] To achieve the above object, the present invention provides a skin health monitoring method based on a cosmetic mirror, which is applied to a cosmetic mirror. The cosmetic mirror comprises: an image acquisition device. The skin health monitoring method based on the cosmetic mirror comprises the following steps:

[0006] Controlling the image acquisition device to acquire user image data of a target user, and determining first facial data and hand data of the target user according to the user image data;

[0007] adjusting the first facial data according to the hand data to obtain second facial data;

[0008] The monitoring result of the health of the target user is determined according to the second facial data and the recognition algorithm.

[0009] Optionally, the number of the user image data is multiple, and the step of determining the first face data and hand data of the target user according to the user image data includes:

[0010] Identify a hand region and a face region in the image data of each user to obtain a plurality of hand regions and a plurality of face regions;

[0011] determining the hand data according to the plurality of hand regions and the plurality of face regions;

[0012] The first face data is determined from the plurality of user image data according to the hand data.

[0013] Optionally, the step of determining the hand data according to the plurality of hand regions and the plurality of face regions comprises:

[0014] Calculating a first distance according to the hand area and the face area in each user image data to obtain a plurality of first distances, wherein the first distance is the distance between the hand area and the face area;

[0015] Identify the hand orientation direction in the hand area in each user image data to obtain multiple hand orientation directions;

[0016] The hand data is determined according to a plurality of the first distances and the hand orientation directions.

[0017] Optionally, the step of determining the first facial data from the plurality of user image data according to the hand data comprises:

[0018] determining facial comparison data according to the hand data;

[0019] Determine a target face region from the plurality of face regions according to face region analysis, wherein the target face region includes all organ features of the face;

[0020] Calculate similarity data between the facial comparison data and the data of the plurality of target face regions respectively;

[0021] The first facial data is determined according to the similarity data.

[0022] Optionally, the step of adjusting the first facial data according to the hand data to obtain the second facial data includes:

[0023] determining a filter parameter of a filter according to the hand data;

[0024] The first facial data is filtered according to the filtering parameter to obtain the second facial data.

[0025] Optionally, the hand data includes hand skin color, the first facial data includes first facial skin color, and the step of adjusting the first facial data according to the hand data to obtain the second facial data includes:

[0026] Correcting the first facial skin color according to the hand skin color to obtain a second facial skin color;

[0027] The second facial skin color is used as the second facial data.

[0028] Optionally, the recognition algorithm is a color matching algorithm, and the step of determining the monitoring result of the health of the target user according to the second facial data and the recognition algorithm includes:

[0029] Matching the second facial skin color with skin color data in a skin color database according to a color matching algorithm to obtain a matching result, wherein the skin color database includes an association relationship between skin color and medical record features;

[0030] The monitoring result is determined according to the matching result.

[0031] Optionally, the step of controlling the image acquisition device to acquire user image data of the target user includes:

[0032] Acquire the face processing type of the user at the current moment, and determine the acquisition time of the image acquisition device to acquire the user image data of the target user according to the face processing type;

[0033] Controlling the image acquisition device to acquire user image data of a target user according to the acquisition time;

[0034] The facial treatment type includes: one of makeup, makeup removal, and skin care.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a makeup mirror, which includes: a memory, a processor, and a makeup mirror-based skin health monitoring program stored in the memory and executable on the processor, wherein the makeup mirror-based skin health monitoring program is configured to implement the steps of any of the above-mentioned makeup mirror-based skin health monitoring methods.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also provides a storage medium, characterized in that a skin health monitoring program based on a makeup mirror is stored on the storage medium, and when the skin health monitoring program based on a makeup mirror is executed by a processor, the steps of the skin health monitoring method based on a makeup mirror are implemented in any one of the above-mentioned steps.

[0037] The present invention proposes a skin health monitoring method based on a makeup mirror. The method controls the image acquisition device to acquire user image data of a target user, and determines first facial data and hand data of the target user according to the user image data; the first facial data is adjusted according to the hand data to obtain second facial data, thereby effectively avoiding inaccurate monitoring results caused by aging or changes in the user's overall skin, thereby improving the accuracy of judging the user's health status. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of the structure of a makeup mirror in a hardware operating environment involved in an embodiment of the present invention;

[0039] Figure 2 It is a flowchart of a first embodiment of a skin health monitoring method based on a cosmetic mirror of the present invention;

[0040] Figure 3 It is a flowchart of a second embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention;

[0041] Figure 4 4 is a flow chart of a third embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention.

[0042] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a makeup mirror in the hardware operating environment involved in the embodiment of the present invention.

[0045] like Figure 1 As shown, the makeup mirror may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, an interactive device 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The interactive device 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional interactive device 1003 may also be connected to the communication bus through a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0046] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the vanity mirror, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0047] like Figure 1As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a skin health monitoring program based on a cosmetic mirror.

[0048] exist Figure 1 In the makeup mirror shown, the network interface 1004 is mainly used for data communication with other devices; the interactive device 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the makeup mirror of the present invention can be set in the makeup mirror, and the makeup mirror calls the skin health monitoring program based on the makeup mirror stored in the memory 1005 through the processor 1001, and executes the skin health monitoring method based on the makeup mirror provided in an embodiment of the present invention.

[0049] The embodiment of the present invention provides a skin health monitoring method based on a cosmetic mirror, referring to Figure 2 , Figure 2 This is a flow chart of a first embodiment of a skin health monitoring method based on a cosmetic mirror according to the present invention.

[0050] In this embodiment, the cosmetic mirror is applied, and the cosmetic mirror includes: an image acquisition device, and the skin health monitoring method based on the cosmetic mirror includes:

[0051] Step S1, controlling the image acquisition device to acquire user image data of a target user, and determining first face data and hand data of the target user according to the user image data;

[0052] In this embodiment, the image acquisition device may be a camera, which may be arranged inside the makeup mirror, and may be made by punching holes on the mirror surface so that the camera can capture images outside the makeup mirror. The target user here is the user who uses the makeup mirror, and the camera may acquire images as the user image data. Optionally, feature recognition is used to determine whether there are faces and hands in the user image data. When there are faces and hands at the same time, the first face data and hand data in the user image data are acquired. The first face data here may include the center position of the face, the image of the face, and the feature data of each organ of the face. It should be clear that the face image here is a color image, which can include more data than black and white images, thereby facilitating the determination of the monitoring results of the target user's skin health.

[0053] Step S2, adjusting the first facial data according to the hand data to obtain second facial data;

[0054] It should be noted that the inventor creatively realized that the user's hands do not need to be directly cropped and deleted when photographed during the shooting process. This is because the hand data in the captured image is actually correlated with the facial skin. For example, skin atrophy, dry skin, pigment changes, etc. of middle-aged and elderly people actually exist on the face and hands at the same time. Such common skin problems of middle-aged and elderly people often directly affect the accuracy of judgment using facial image data. Therefore, it is necessary to adjust the first facial data using the hand data. Optionally, the color of the first facial data can be adjusted according to the hand data, and the parameters of the filter for the first facial data can also be adjusted according to the hand data to obtain the second facial data.

[0055] Step S3, determining the monitoring result of the health of the target user according to the second facial data and the recognition algorithm.

[0056] The recognition algorithm used here is not limited to a color feature matching algorithm. Optionally, the skin color of the second facial data is matched with a preset skin color, and the corresponding one is determined after a successful match. Alternatively, a recognition model can be trained using an algorithm and a training set, and the second facial data can be input into the recognition model to determine the detection result.

[0057] In this embodiment, the image acquisition device is controlled to acquire user image data of the target user, and the first facial data and hand data of the target user are determined based on the user image data; the first facial data is adjusted based on the hand data to obtain the second facial data, thereby effectively avoiding inaccurate monitoring results caused by aging or changes in the user's overall skin, thereby improving the accuracy of judging the user's health status.

[0058] Further, based on the first embodiment, a second embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention is proposed. In this embodiment, referring to Figure 3 , the number of the user image data is multiple, and the step of determining the first face data and hand data of the target user according to the user image data includes:

[0059] Step S11, identifying a hand region and a face region in each user image data to obtain a plurality of hand regions and a plurality of face regions;

[0060] In this embodiment, multiple photos can be taken continuously. Optionally, the hand area and the face area are determined in the user image data through feature recognition, and the hand area and the face area are determined separately, so that the hand area of ​​the hand image and the face area of ​​the face image can be located.

[0061] Step S12, determining the hand data according to the plurality of hand regions and the plurality of face regions;

[0062] Specifically, a corresponding index is determined according to the plurality of hand regions and the plurality of face regions, and the hand data is determined according to data corresponding to the index. The index here can be the direction of the hand or the distance between the hand and the face.

[0063] Step S13: determining the first facial data from the plurality of user image data according to the hand data.

[0064] After the hand data is determined, the first face data is determined from the plurality of user image data according to the hand data. It should be noted that, in this embodiment, the first face data and the hand data may be from two different user image data. In other embodiments, the number of the user image data is one, and the first face data and the hand data of the target user are determined according to one user image data.

[0065] In this embodiment, by identifying the hand area and the face area in each user image data, multiple hand areas and multiple face areas are obtained and the hand data is determined based on the multiple hand areas and the multiple face areas, so that the hand data can be selected from the image data corresponding to the multiple hands, thereby improving the representativeness of the hand data, and the first face data is determined in the multiple user image data based on the hand data, thereby improving the correlation between the hand data and the first face data, thereby improving the accuracy of the monitoring result.

[0066] Further, the step of determining the hand data according to the plurality of hand regions and the plurality of face regions comprises:

[0067] Calculating a first distance according to the hand area and the face area in each user image data to obtain a plurality of first distances, wherein the first distance is the distance between the hand area and the face area;

[0068] Identify the hand orientation direction in the hand area in each user image data to obtain multiple hand orientation directions;

[0069] The hand data is determined according to a plurality of the first distances and the hand orientation directions.

[0070] The first distance here is the distance between the hand area and the face area belonging to the same user image data; the inventor of the present application creatively discovered that in the process of using a makeup mirror, there are a large number of image data in which the hands and faces overlap in the collected multiple user image data. Conventionally, this type of data is actually directly abandoned during the data cleaning process due to incomplete faces. However, in this embodiment, this type of data is actually used. Specifically, the distance between the center coordinates of the hand area and the center coordinates of the face area is calculated as the first distance. It should be noted that the smaller the first distance is, the larger the overlap area of ​​the face area and the hand area is. The hand orientation direction here can be the direction of the palm or the direction of the back of the hand. In this embodiment, the back of the hand is used for explanation. After determining the hand orientation direction, the hand data is determined according to the hand orientation direction and the first distance; specifically, the distance interval and the orientation angle interval can be set, and the data corresponding to the hand area that satisfies both the distance interval and the orientation angle interval are selected as the hand data. Preferably, the hand orientation direction of the back of the hand should be close to the orientation of the face of the target user.

[0071] In this embodiment, the hand area is determined by the first distance and the hand direction, and hand data with the same lighting as the face can be selected, thereby ensuring that the environmental conditions of the hand data obtained are the same as those of the face, thereby improving the accuracy of the monitoring results determined in subsequent steps.

[0072] Further, the step of determining the first facial data from the plurality of user image data according to the hand data includes:

[0073] determining facial comparison data according to the hand data;

[0074] Determine a target face region from the plurality of face regions according to face region analysis, wherein the target face region includes all organ features of the face;

[0075] Calculate similarity data between the facial comparison data and the data of the plurality of target face regions respectively;

[0076] The first facial data is determined according to the similarity data.

[0077] Preferably, the face comparison data is data of the face area corresponding to the user image data belonging to the same hand data; it should be explained that the target face area here is an area including complete face features, so as to filter out the face area corresponding to the user image data where the hand area and the face area overlap. Then, similarity data is calculated based on the face comparison data and the data of multiple target face areas, and the first face data is determined in the target face area based on the similarity data, so as to ensure that the first face data includes all organ features while taking into account the correlation with the hand data.

[0078] In this embodiment, facial comparison data is determined based on hand data, and a target facial area is determined from multiple facial areas based on facial area analysis. Similarity data is calculated between the facial comparison data and the data of the multiple target facial areas, and the first facial data is determined based on the similarity data, thereby improving the accuracy of the selected first facial data.

[0079] Further, based on the first embodiment or the second embodiment, a third embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention is proposed. In this embodiment, referring to Figure 4 The step of adjusting the first facial data according to the hand data to obtain the second facial data comprises:

[0080] Step S21, determining filtering parameters of a filter according to the hand data;

[0081] The hand data here is specifically a hand image, and the type of filter, the order of the filter, the bandwidth of the filter, etc. are determined according to the number of wrinkles in the hand image and the depth data of the wrinkles. The depth data of the wrinkles here may be related to the clarity of the edge of the wrinkles. It should be noted that the higher the depth of the wrinkles, the higher the filtering effect required.

[0082] Step S22: filtering the first facial data according to the filtering parameters to obtain the second facial data.

[0083] The filter is adjusted according to the filter parameter, and the first facial data is filtered using the filter, so as to obtain the second facial data.

[0084] In this embodiment, the filtering parameters of the filter are determined by the hand data, and the first facial data are filtered according to the filtering parameters, so that different first facial data can be filtered, thereby improving the filtering effect.

[0085] Further, based on the first embodiment or the second embodiment, a fourth embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention is proposed. In this embodiment, the hand data includes hand skin color, the first facial data includes first facial skin color, and the step of adjusting the first facial data according to the hand data to obtain the second facial data includes:

[0086] Correcting the first facial skin color according to the hand skin color to obtain a second facial skin color;

[0087] Specifically, the first facial skin color is corrected according to the hand skin color, a color deviation is calculated between the hand skin color and a standard hand skin color, and the first facial skin color is adjusted according to the color deviation to obtain a skin color corresponding to the first facial skin color.

[0088] The second facial skin color is used as the second facial data.

[0089] In this embodiment, the first facial skin color is corrected according to the hand skin color, thereby obtaining the second facial data.

[0090] Furthermore, the recognition algorithm is a color matching algorithm, and the step of determining the monitoring result of the skin health of the target user according to the second facial data and the recognition algorithm includes:

[0091] Matching the second facial skin color with skin color data in a skin color database according to a color matching algorithm to obtain a matching result, wherein the skin color database includes an association relationship between skin color and medical record features;

[0092] The monitoring result is determined according to the matching result.

[0093] The color matching algorithm here can be a histogram matching algorithm or a similarity calculation algorithm based on image colors. The skin color database includes the association between skin color and medical record features. Specifically, it can be as follows: yellow indicates spleen and stomach, and there is an abnormality in the digestive system; white indicates lungs, and there is an abnormality in the respiratory system; red indicates an abnormality in the cardiovascular system; cyan indicates liver and gallbladder, and there is an abnormality in the immune system; black indicates an abnormality in the kidney and bladder system. Of course, different abnormalities can be represented by more specific colors.

[0094] Further, based on any of the above embodiments, a fifth embodiment of the skin health monitoring method based on a cosmetic mirror of the present invention is proposed. In this embodiment, the step of controlling the image acquisition device to acquire user image data of the target user includes:

[0095] Acquire the face processing type of the user at the current moment, and determine the acquisition time of the image acquisition device to acquire the user image data of the target user according to the face processing type;

[0096] Controlling the image acquisition device to acquire user image data of a target user according to the acquisition time;

[0097] The facial treatment type includes: one of makeup, makeup removal, and skin care.

[0098] In this embodiment, there is no need to remind the user that they are not wearing makeup. Instead, the collection time is determined through motion recognition to avoid interfering with the user's need to use the mirror.

[0099] In addition, an embodiment of the present invention also proposes a makeup mirror, which includes: a memory, a processor, and a makeup mirror-based skin health monitoring program stored in the memory and executable on the processor, wherein the makeup mirror-based skin health monitoring program is configured to implement the steps of any of the above-mentioned embodiments of the makeup mirror-based skin health monitoring method.

[0100] In addition, an embodiment of the present invention also proposes a storage medium, on which a skin health monitoring program based on a makeup mirror is stored. When the skin health monitoring program based on a makeup mirror is executed by a processor, the steps of any of the above-mentioned embodiments of the skin health monitoring method based on a makeup mirror are implemented.

[0101] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0102] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0104] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A skin health monitoring method based on a cosmetic mirror, characterized in that: Applied to a makeup mirror, the makeup mirror comprises: an image acquisition device, and the skin health monitoring method based on the makeup mirror comprises the following steps: Controlling the image acquisition device to acquire user image data of a target user, and determining first facial data and hand data of the target user according to the user image data; adjusting the first facial data according to the hand data to obtain second facial data; The monitoring result of the health of the target user is determined according to the second facial data and the recognition algorithm.

2. The skin health monitoring method based on a cosmetic mirror according to claim 1, characterized in that: The number of the user image data is multiple, and the step of determining the first face data and hand data of the target user according to the user image data includes: Identify a hand region and a face region in the image data of each user to obtain a plurality of hand regions and a plurality of face regions; determining the hand data according to the plurality of hand regions and the plurality of face regions; The first face data is determined from the plurality of user image data according to the hand data.

3. The skin health monitoring method based on a cosmetic mirror as claimed in claim 2, characterized in that: The step of determining the hand data according to the plurality of hand regions and the plurality of face regions comprises: Calculating a first distance according to the hand area and the face area in each user image data to obtain a plurality of first distances, wherein the first distance is the distance between the hand area and the face area; Identify the hand orientation direction in the hand area in each user image data to obtain multiple hand orientation directions; The hand data is determined according to a plurality of the first distances and the hand orientation directions.

4. The skin health monitoring method based on a cosmetic mirror as claimed in claim 3, characterized in that: The step of determining the first face data from the plurality of user image data according to the hand data comprises: determining facial comparison data according to the hand data; Determine a target face region from the plurality of face regions according to face region analysis, wherein the target face region includes all organ features of the face; Calculate similarity data between the facial comparison data and the data of the plurality of target face regions respectively; The first facial data is determined according to the similarity data.

5. The skin health monitoring method based on a cosmetic mirror according to claim 1, characterized in that: The step of adjusting the first facial data according to the hand data to obtain the second facial data comprises: determining a filter parameter of a filter according to the hand data; The first facial data is filtered according to the filtering parameter to obtain the second facial data.

6. The skin health monitoring method based on a cosmetic mirror according to claim 1, characterized in that: The hand data includes hand skin color, the first facial data includes first facial skin color, and the step of adjusting the first facial data according to the hand data to obtain second facial data includes: Correcting the first facial skin color according to the hand skin color to obtain a second facial skin color; The second facial skin color is used as the second facial data.

7. The skin health monitoring method based on a cosmetic mirror according to claim 6, characterized in that: The recognition algorithm is a color matching algorithm, and the step of determining the monitoring result of the health of the target user according to the second facial data and the recognition algorithm comprises: Matching the second facial skin color with skin color data in a skin color database according to a color matching algorithm to obtain a matching result, wherein the skin color database includes an association relationship between skin color and medical record features; The monitoring result is determined according to the matching result.

8. The skin health monitoring method based on a cosmetic mirror according to any one of claims 1 to 7, characterized in that: The step of controlling the image acquisition device to acquire user image data of the target user comprises: Acquire the face processing type of the user at the current moment, and determine the acquisition time of the image acquisition device to acquire the user image data of the target user according to the face processing type; Controlling the image acquisition device to acquire user image data of a target user according to the acquisition time; The facial treatment type includes: one of makeup, makeup removal, and skin care.

9. A cosmetic mirror, characterized in that: The makeup mirror comprises: a memory, a processor, and a makeup mirror-based skin health monitoring program stored in the memory and executable on the processor, wherein the makeup mirror-based skin health monitoring program is configured to implement the steps of the makeup mirror-based skin health monitoring method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a skin health monitoring program based on a cosmetic mirror, and when the skin health monitoring program based on a cosmetic mirror is executed by a processor, the steps of the skin health monitoring method based on a cosmetic mirror as described in any one of claims 1 to 8 are implemented.

Images