Vision-based Face Region Localization Method, Device and Wearable Device

Abstract

The present application discloses a vision-based face area positioning method, device, and wearable device. The method includes: acquiring an image of a beauty care device collected by a camera on the wearable device and performing recognition using an image recognition algorithm; determining spatial attitude information of the beauty care device relative to the wearable device according to the image recognition result; determining spatial attitude information of the face three-dimensional model relative to the wearable device according to the spatial attitude information of the beauty care device relative to the wearable device and preset face key points; and determining a target face area where the beauty care device acts according to the spatial attitude information of the beauty care device and the face three-dimensional model relative to the wearable device. The present application uses vision positioning technology implemented based on a wearable device to position the face area, providing a basis for automatic switching of the beauty care mode, eliminating the need to coordinate the relative positions between the user and the device, reducing the user operation difficulty, and improving the skin care experience.

Description

Technical Field

[0001] The present application relates to the technical field of beauty care, and in particular, to a vision-based face area positioning method, device, and wearable device. Background Art

[0002] Traditional medical beauty care devices mainly achieve the switching of different beauty care modes by setting buttons for different regions of the human face. This solution is relatively reliable but has a poor experience. The main reasons are as follows:

[0003] 1) The user needs to first familiarize themselves with the beauty care mode corresponding to each button and needs to switch when the beauty care device's working head reaches the corresponding area of the human face. This process is relatively cumbersome, and the user is prone to confusing the buttons. At the same time, manually switching modes back and forth easily adds user fatigue; 2) Due to individual differences and different usage habits of users, the uniform button configuration causes trouble for the usual grasping postures of some users. The user needs some time to get used to using the product, reducing the product's friendliness.

[0004] To solve the problem that traditional medical beauty care devices cannot achieve automatic switching of beauty care modes, the automatic positioning of the face area where the beauty care device acts becomes the entry point. The main solution provided in the prior art is to achieve the positioning of the face area through the interaction between the user, a terminal device such as a mobile phone, and the beauty care device. By aligning the camera on the terminal device with the user's face and taking a face image when the beauty care device is working, and then using image recognition technology and a pre-generated face map to determine the face area where the beauty care device is currently acting.

[0005] However, to ensure the accuracy of image recognition, this vision-based positioning solution requires coordinating the relative positions of the user, the terminal device, and the beauty care device, increasing the operation difficulty of the user and the device, and thus affecting the user's skin care experience. Summary of the Invention

[0006] Embodiments of the present application provide a vision-based face area positioning method, device, and wearable device to achieve automatic positioning of the face area through vision recognition technology, reduce the operation difficulty and operation cost of the user, and improve the user's skin care experience.

[0007] Embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, embodiments of the present application provide a vision-based face area positioning method, and the vision-based face area positioning method includes:

[0009] Obtain an image of a beauty care device collected by a camera on a wearable device and use an image recognition algorithm to recognize the image of the beauty care device to obtain an image recognition result. The image of the beauty care device contains a preset recognizable identifier;

[0010] Determine the spatial attitude information of the beauty care device in the coordinate system of the wearable device according to the image recognition result;

[0011] Determine the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the preset face key points;

[0012] Determine the target face area corresponding to the beauty care device in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device.

[0013] Optionally, the preset recognizable identifier includes a preset coding identifier, and obtaining the image of the beauty care device collected by the camera on the wearable device and using an image recognition algorithm to recognize the image of the beauty care device, the obtained image recognition result includes:

[0014] Convert the image of the beauty care device into a grayscale image and a target channel image respectively;

[0015] Use an image detection algorithm to detect the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image respectively;

[0016] Determine a coding identifier recognition result according to the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image, and the coding identifier recognition result includes at least three 2D feature points.

[0017] Optionally, the image recognition result includes 2D feature points of a target recognizable identifier, and determining the spatial attitude information of the beauty care device in the coordinate system of the wearable device according to the image recognition result includes:

[0018] Obtain the correspondence between the 2D feature points and the 3D feature points of the preset recognizable identifier, where the 3D feature points are the 3D feature points of the preset recognizable identifier in the coordinate system of the beauty care device;

[0019] Determine the 3D feature points corresponding to the 2D feature points of the target recognizable identifier according to the correspondence between the 2D feature points and the 3D feature points of the preset recognizable identifier;

[0020] Determine the spatial attitude information of the camera in the coordinate system of the beauty care device according to the 3D feature points corresponding to the 2D feature points of the target recognizable identifier by using a camera pose solving algorithm;

[0021] Determine the spatial attitude information of the beauty care device in the coordinate system of the wearable device according to the spatial attitude information of the camera in the coordinate system of the beauty care device.

[0022] Optionally, the determining the spatial pose information of the three-dimensional face model in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and the preset face key points includes:

[0023] Tracking the spatial pose information of the beauty care device in the wearable device coordinate system based on the spatial pose information of the beauty care device in the wearable device coordinate system and the preset face key points;

[0024] Determining the spatial pose information of the three-dimensional face model in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system.

[0025] Optionally, the determining the spatial pose information of the three-dimensional face model in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system includes:

[0026] Determining the spatial pose information of the preset face key points in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system;

[0027] Performing fitting using a three-dimensional face model fitting algorithm according to the spatial pose information of the preset face key points in the wearable device coordinate system to obtain the spatial pose information of the three-dimensional face model in the wearable device coordinate system.

[0028] Optionally, the determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the three-dimensional face model in the wearable device coordinate system includes:

[0029] Determining a three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the three-dimensional face model in the wearable device coordinate system;

[0030] Determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model.

[0031] Optionally, the image of the beauty care device is collected in real time by a camera. The determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the three-dimensional face model in the wearable device coordinate system includes:

[0032] Update the spatial attitude information of the beauty care device in the wearable device coordinate system according to the image recognition result corresponding to the image of the beauty care device collected by the camera in real time;

[0033] Obtain the spatial attitude change information of the preset human face key points, and update the spatial attitude information of the human face three-dimensional model in the wearable device coordinate system according to the spatial attitude change information of the preset human face key points;

[0034] Determine the target human face area corresponding to the beauty care device in the wearable device coordinate system according to the updated spatial attitude information of the beauty care device in the wearable device coordinate system and the updated spatial attitude information of the human face three-dimensional model in the wearable device coordinate system.

[0035] Optionally, the target human face partition is one of multiple human face partitions, and the multiple human face partitions include at least one of a five-partition and a seven-partition.

[0036] In a second aspect, an embodiment of the present application further provides a vision-based human face area positioning device, which is characterized in that the vision-based human face area positioning device includes:

[0037] An identification unit, configured to obtain an image of a beauty care device collected by a camera on a wearable device and perform identification on the image of the beauty care device by using an image recognition algorithm to obtain an image recognition result, where the image of the beauty care device includes a preset recognizable identifier;

[0038] A first determination unit, configured to determine the spatial attitude information of the beauty care device in the wearable device coordinate system according to the image recognition result;

[0039] A second determination unit, configured to determine the spatial attitude information of the human face three-dimensional model in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and preset human face key points;

[0040] A third determination unit, configured to determine the target human face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the human face three-dimensional model in the wearable device coordinate system.

[0041] In a third aspect, an embodiment of the present application further provides a wearable device, where the wearable device includes:

[0042] A camera, configured to collect an image of a beauty care device;

[0043] A processor; and

[0044] A memory arranged to store computer-executable instructions that, when executed, cause the processor to execute any one of the foregoing vision-based human face region localization methods.

[0045] In a fourth aspect, an embodiment of the present application further provides a beauty care system, which includes a beauty care device and the wearable device as described above.

[0046] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs that, when executed by a wearable device including a plurality of application programs, cause the wearable device to execute any one of the foregoing methods.

[0047] The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects: In the vision-based human face region localization method of the embodiment of the present application, first, an image of the beauty care device collected by a camera on the wearable device is obtained, and the image of the beauty care device is recognized using an image recognition algorithm to obtain an image recognition result, and the image of the beauty care device contains a preset recognizable identifier; then, the spatial pose information of the beauty care device in the wearable device coordinate system is determined according to the image recognition result; afterwards, the spatial pose information of the human face three-dimensional model in the wearable device coordinate system is determined according to the spatial pose information of the beauty care device in the wearable device coordinate system and the preset human face key points; finally, the target human face region corresponding to the beauty care device in the wearable device coordinate system is determined according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the human face three-dimensional model in the wearable device coordinate system. The vision localization technology implemented by the wearable device worn by the user in the embodiment of the present application automatically locates the human face region, provides a basis for the automatic switching of the beauty care mode and precise skin care, eliminates the need to coordinate the relative positions among the user, the wearable device, and the beauty care device, reduces the difficulty of image acquisition, and eliminates the need to additionally set up a hardware structure on the beauty care device, improving the compatibility with different beauty care devices. Description of the Drawings

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0049] Figure 1 It is a flowchart of a vision-based human face region localization method in an embodiment of the present application;

[0050] Figure 2 It is a schematic diagram of human face region division in an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of human face key points in an embodiment of the present application;

[0052] Figure 4 It is a schematic diagram of a preset recognizable identifier in the embodiment of the present application;

[0053] Figure 5 It is a schematic diagram of an image recognition process in the embodiment of the present application;

[0054] Figure 6 It is a schematic diagram of constructing a coordinate system of a beauty care device in the embodiment of the present application;

[0055] Figure 7 It is a schematic diagram of the reconstruction process of a three-dimensional face morphology in the embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of the process of determining a target face area in the embodiment of the present application;

[0057] Figure 9 It is a schematic diagram of the structure of a vision-based face area positioning device in the embodiment of the present application;

[0058] Figure 10 It is a schematic diagram of the structure of a wearable device in the embodiment of the present application. Detailed implementation manners

[0059] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0060] The following will detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0061] Based on the problem that the existing solution needs to increase the user's usage burden to achieve the mode switching of the beauty care device, the embodiment of the present application provides a vision-based face area positioning method. As Figure 1 shown, it provides a schematic diagram of the process of a vision-based face area positioning method in the embodiment of the present application. The vision-based face area positioning method at least includes the following steps S110 to step S140:

[0062] Step S110, obtain an image of the beauty care device collected by a camera on the wearable device and use an image recognition algorithm to recognize the image of the beauty care device to obtain an image recognition result. The image of the beauty care device contains a preset recognizable identifier.

[0063] The "face area" defined in the embodiment of the present application may be a facial skin area divided according to the functional requirements of the beauty care device, such as Figure 2 As shown, a schematic diagram of face area division in an embodiment of the present application is provided, which may include, for example, a forehead area, an eye area, and a cheek area.

[0064] When locating the face area, the user needs to wear the wearable device first to obtain the image of the beauty care device captured by the camera on the wearable device. The wearable device of the embodiment of the present application can be regarded as an auxiliary positioning device independent of the beauty care device. A wearable device in any form such as a headband, hairpin, hairpin, necklace, neck ring, bracelet, or wristband can be used to fix it on the user's head, neck, wrist, etc. A camera is set at a specific position of the wearable device. The specific position only needs to ensure that the camera can capture the beauty care device in front of the entire face area. For example, if a wearable device in the form of a headband is used, the camera can be set at the position directly above the forehead corresponding to the wearable device, or it can be set at the positions on both sides of the forehead. The number of cameras can be one or more. Therefore, how to set the camera on the wearable device can be flexibly set by technicians in this field according to needs, and no specific limitation is made here.

[0065] It should be noted that since the specific structure of the wearable device is designed in advance based on the user's wearing method, for example, if the wearable device is in the form of a headband, the user can simply wear it normally according to the habit of wearing a headband. After wearing it, the relative position between the user and the wearable device is fixed, and there is no need to consider or coordinate the user's position relative to the wearable device in the image acquisition link, which reduces the user's operation difficulty.

[0066] After obtaining the image of the beauty protection device, the image recognition algorithm can be used to identify the preset identifiable logo in the image of the beauty protection device. The "preset identifiable logo" refers to a specific logo that is pre-set on the body of the beauty protection device and has determined the position information. Its main function is to provide a basis for determining the spatial posture information of the beauty protection device in the coordinate system of the wearable device. The specific form of expression can be flexibly determined by technical personnel in this field in combination with existing technologies, and is not specifically limited here.

[0067] Step S120, determining the spatial posture information of the beauty care device in the wearable device coordinate system according to the image recognition result.

[0068] The image recognition algorithm can recognize the preset recognizable identifiers contained in the image, determine the two-dimensional position information corresponding to the preset recognizable identifiers, and then solve the camera pose in the coordinate system of the beauty care device based on the three-dimensional position information corresponding to the two-dimensional position information of the preset recognizable identifiers determined in advance. Finally, the spatial attitude information of the beauty care device in the coordinate system of the wearable device is inversely calculated, that is, the spatial attitude of the beauty care device is calibrated to the coordinate system of the wearable device. The reason for calibrating the spatial attitude of the beauty care device to the coordinate system of the wearable device is mainly to consider the relatively fixed characteristics of the wearable device relative to the user's head.

[0069] The coordinate system of the wearable device in the embodiments of the present application can be flexibly defined according to actual needs. For example, the origin of the coordinate system and the corresponding axis directions can be defined based on any point on the wearable device, or a camera coordinate system can be directly constructed with the camera position on the wearable device as the origin. The spatial attitude information of the beauty care device in the coordinate system of the wearable device can specifically refer to the six-degree-of-freedom information of the beauty care device relative to the wearable device. Six degrees of freedom refer to the degrees of freedom of a rigid body moving in three-dimensional space, especially the translation of the rigid body along three mutually perpendicular coordinate axes in the front-back, up-down, and left-right directions, and it can also rotate its direction around the three perpendicular axes. The three rotation directions are respectively called pitch, yaw, and roll.

[0070] Step S130: Determine the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the preset face key points.

[0071] After determining the spatial attitude information of the beauty care device in the coordinate system of the wearable device, it is also necessary to further determine the spatial attitude information of the three-dimensional face morphology in the coordinate system of the wearable device. For this, the embodiments of the present application have predefined face key points. Here, the face key points mainly refer to the face features in areas such as eyebrows, eyelids, lips, and nose, as Figure 3 shown, a schematic diagram of face key points in the embodiments of the present application is provided. The black dots in the figure are the defined face key points. The detection of face key points is the basis for other face-related tasks such as face recognition, expression analysis, and three-dimensional face reconstruction. In the embodiments of the present application, the three-dimensional face morphology is mainly restored based on the preset face key points.

[0072] Since the spatial attitude information of the beauty care device in the coordinate system of the wearable device is calculated in the foregoing steps, the three-dimensional positions of the face key points in the coordinate system of the wearable device can be determined according to the relative movements of the beauty care device on each preset face key point, and thus the three-dimensional face model in the coordinate system of the wearable device can be fitted, that is, the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device.

[0073] In the embodiment of the present application, the determination of the spatial pose of the three-dimensional face model does not require an additional camera facing the face area to be set on the beauty care device, which reduces the hardware transformation cost of the beauty care device and improves the compatibility with other different types of beauty care devices at the same time, that is, the purpose of enabling a wearable device to be simultaneously connected to multiple different beauty care devices is achieved.

[0074] Step S140: Determine the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the three-dimensional face model in the wearable device coordinate system.

[0075] The spatial pose information of the beauty care device and the three-dimensional face model in the wearable device coordinate system indicates that the three-dimensional position and pose information of the two are unified in the same coordinate system. Furthermore, the spatial overlapping area can be calculated through the spatial pose information of the two in the same coordinate system, so as to determine the target face area that the beauty care device currently needs to act on.

[0076] Since the spatial pose information changes with the real-time acquisition and recognition of camera images, etc., the corresponding calculated target face area will also be updated in real time. Thus, the beauty care device can be controlled to switch different beauty care modes for different target face areas.

[0077] The vision-based face area positioning method in the embodiment of the present application uses the vision positioning technology implemented by the wearable device worn by the user to automatically locate the face area, which provides a basis for the automatic switching of beauty care modes and precise skin care. It is not necessary to coordinate the relative positions among the user, the wearable device, and the beauty care device, reduces the difficulty of image acquisition, and does not require an additional hardware structure to be set on the beauty care device, improving the compatibility with different beauty care devices.

[0078] In some embodiments of the present application, the preset recognizable identifier includes a preset coding identifier. The method of acquiring the image of the beauty care device collected by the camera on the wearable device and using an image recognition algorithm to recognize the image of the beauty care device, and the obtained image recognition result includes: converting the image of the beauty care device into a grayscale image and a target channel image respectively; using an image detection algorithm to detect the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image respectively; determining the coding identifier recognition result according to the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image, and the coding identifier recognition result includes at least three 2D feature points.

[0079] To facilitate the extraction of coding identifiers and improve the recognition accuracy, an embodiment of this application designs a coding identifier of binary square. The form of each coding identifier can be, for example, four circles embedded in a rectangular frame, with one circle being black and the rest being white. This is mainly because the subsequent PnP (Perspective-n-Point) solution algorithm requires at least three feature points. Designing the coding identifier as four circles is on the one hand because the rectangle recognition and dot recognition algorithms are relatively mature, and on the other hand because there is a problem of directionality with four circles, and a special circle can be used to represent directionality.

[0080] It should be noted that the above coding identifier can be implemented based on conventional coding methods in computer vision such as Aurcomarker, etc., or a coding identifier can be independently designed according to its own needs. The coding requirement is that the coding surrounding the body of the beauty care device has a unique minimum recognition unit at any perspective. Since the relative spatial positions of each coding identifier are defined in advance, as long as one coding identifier is recognized from any perspective, the spatial positions of other coding identifiers can be known. In addition, adopting the above recognizable coding identifier does not require additional hardware structures to be set on the beauty care device, thus enabling compatibility with more beauty care device products.

[0081] Considering that the holding posture of the user holding the beauty care device will affect the recognition of the coding identifier, multiple rectangular coding identifiers can be arranged in a spiral manner around the body of the beauty care device, so as to ensure that at least one coding identifier is recognized. However, generally, 3 to 4 coding identifiers will be recognized. In addition, a ring-shaped identifier can be set at one end of the body of the beauty care device to locate the recognition of the rectangular coding identifier. As Figure 4 shown, a schematic diagram of a preset recognizable identifier in an embodiment of this application is provided.

[0082] Based on the above-defined coding identifier, when performing coding identifier recognition, the image of the beauty device can be first converted into a grayscale image and an image for extracting the color of the target channel respectively. The color of the target channel mainly depends on the color of the ring-shaped identifier, and can be set to green, for example. Then, the ring-shaped identifier in the green channel map and the coding identifier and ring-shaped identifier in the grayscale image are extracted respectively according to shape detection and binary differentiation. After removing the ring-shaped identification code from the green channel map, a rectangle where all four circles are satisfied and the corresponding circle positions are obtained, thereby recognizing one coding identifier. If there are other identifiers that meet the conditions, the same processing can be performed. Subsequently, the three-dimensional spatial positions and two-dimensional image positions of the four circles in a single coding identifier can be used to solve through PnP, so at least one coding identifier needs to be recognized. As Figure 5 shown, a schematic diagram of an image recognition process in an embodiment of this application is provided.

[0083] In some embodiments of the present application, the image recognition result includes 2D feature points of a target recognizable identifier. The determining of the spatial pose information of the beauty care device in the wearable device coordinate system according to the image recognition result includes: obtaining the correspondence between the 2D feature points and 3D feature points of a preset recognizable identifier, where the 3D feature points are the 3D feature points of the preset recognizable identifier in the beauty care device coordinate system; determining the 3D feature points corresponding to the 2D feature points of the target recognizable identifier according to the correspondence between the 2D feature points and 3D feature points of the preset recognizable identifier; determining the spatial pose information of the camera in the beauty care device coordinate system by using a camera pose solving algorithm according to the 3D feature points corresponding to the 2D feature points of the target recognizable identifier; and determining the spatial pose information of the beauty care device in the wearable device coordinate system according to the spatial pose information of the camera in the beauty care device coordinate system.

[0084] In the embodiments of the present application, the calculation of the spatial pose information of the beauty care device in the wearable device coordinate system can be obtained based on a camera pose solving algorithm such as the PnP algorithm. The PnP algorithm is an algorithm for solving the camera pose by combining the coordinates of given 3D points, the corresponding 2D point coordinates, and the camera internal parameter matrix. The camera internal parameters generally refer to the focal length value, principal point value, coordinate axis tilt parameter, and camera distortion parameter in the pinhole camera model, which can be calibrated in advance.

[0085] Based on this, in the stage of defining the preset recognizable identifier in the embodiments of the present application, the three-dimensional spatial coordinates of the feature points in the preset recognizable identifier in the world coordinate system, that is, the beauty care device coordinate system, have been determined. Therefore, the correspondence between the 2D feature points and 3D feature points is also determined accordingly. Then, based on the correspondence between the 2D feature point values and the 3D feature points and the 2D feature points of the target recognizable identifier currently recognized from the image, the 3D feature points corresponding to the 2D feature points of the target recognizable identifier can be determined. Further, the PnP algorithm can be used to calculate the camera pose, that is, the spatial change of the current camera relative to the beauty care device coordinate system. Finally, through conversion, the spatial change of the beauty care device relative to the wearable device coordinate system can be obtained, that is, the six-degree-of-freedom information of the current beauty care device is obtained, and it also provides a basis for the subsequent detection of facial key points and the establishment of a facial three-dimensional model.

[0086] For the convenience of understanding the embodiments of the present application, such as Figure 6As shown, a schematic diagram of constructing a coordinate system for a beauty and care device in an embodiment of the present application is provided. For example, assuming that the body of the beauty and care device is a cylinder, then when the body of the beauty and care device is parallel to the ground, the center of the bottom surface of the body can be used as the origin of the coordinate system, the direction parallel to the ground and horizontally backward is the X-axis, the direction perpendicular to the ground and upward is the Y-axis, and the direction parallel to the ground and horizontally to the right is the Z-axis. Of course, this is only an exemplary description of the coordinate system of the beauty and care device in the embodiment of the present application. Specifically, how to construct the coordinate system of the beauty and care device can be flexibly adjusted by those skilled in the art according to the actual situation, and no specific limitation is made here.

[0087] In some embodiments of the present application, the determining the spatial pose information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial pose information of the beauty and care device in the coordinate system of the wearable device and the preset face key points includes: tracking the spatial pose information of the beauty and care device in the coordinate system of the wearable device based on the spatial pose information of the beauty and care device in the coordinate system of the wearable device and the preset face key points; determining the spatial pose information of the three-dimensional face model in the coordinate system of the wearable device according to the tracked spatial pose information of the beauty and care device in the coordinate system of the wearable device.

[0088] Since the camera in the embodiment of the present application is set facing the beauty and care device and there is no camera facing the face area, it is impossible to directly determine the spatial pose information of each face key point in the face area in the coordinate system of the wearable device by means of image recognition. However, since there is less muscle distribution in the frontal area of the face, the spatial changes caused by facial muscle movement of the camera bound to the frontal area of the face can be ignored. Based on the stable spatial relationship formed by the camera on the forehead and the three-dimensional face morphology, the spatial pose information of the beauty and care device in the coordinate system of the wearable device can be tracked, and the spatial pose information of the face key points in the coordinate system of the wearable device can be indirectly determined through the tracking result of the spatial pose information of the beauty and care device.

[0089] For example, when the user first uses the beauty and care device, the user is controlled to move the beauty and care device in a certain way so that the sensing head of the beauty and care device sequentially lands on each preset face key point in the face area. When the sensing head of the beauty and care device contacts the preset face key point, the spatial pose information of the beauty and care device in the coordinate system of the wearable device tracked at this time is the spatial pose information of the face key point in the coordinate system of the wearable device. Based on this, the spatial pose information of each preset face key point in the coordinate system of the wearable device can be obtained respectively, and then the three-dimensional face morphology in the coordinate system of the wearable device can be restored therefrom.

[0090] The above process can be regarded as a face calibration step, which indirectly calculates the spatial pose information of facial key points in the wearable device coordinate system based on the tracking of the spatial pose information of the beauty care device in the wearable device coordinate system, so as to calibrate the three-dimensional facial morphology to the wearable device coordinate system, serving as the basis for calculating the spatial overlapping area between the three-dimensional facial morphology and the beauty care device in the wearable device coordinate system subsequently.

[0091] In some embodiments of the present application, determining the spatial pose information of the three-dimensional facial model in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system includes: determining the spatial pose information of the preset facial key points in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system; and performing fitting using a three-dimensional facial model fitting algorithm according to the spatial pose information of the preset facial key points in the wearable device coordinate system to obtain the spatial pose information of the three-dimensional facial model in the wearable device coordinate system.

[0092] When the sensing head of the beauty care device contacts the preset facial key points, a feedback signal will be generated, that is, the feedback signal indicates that the beauty care device has currently contacted the preset facial key points. Then, at this moment, the spatial positions of the beauty care device and the preset facial key points in the wearable device coordinate system coincide. Therefore, the spatial pose information of the beauty care device tracked at this time is the spatial pose information of the facial key points in the wearable device coordinate system.

[0093] After obtaining the spatial pose information of the preset facial key points in the wearable device coordinate system, the three-dimensional facial model can be fitted according to the spatial pose information of the preset facial key points in the wearable device coordinate system using a three-dimensional facial model fitting algorithm to restore the three-dimensional facial morphology in the wearable device coordinate system. For the convenience of understanding the embodiments of the present application, as Figure 7 shown, a schematic diagram of the reconstruction process of a three-dimensional facial morphology in an embodiment of the present application is provided. The three-dimensional facial model fitting algorithm can be implemented based on existing three-dimensional facial reconstruction models such as 3D MM, Flame, etc. The specific implementation process can be determined in combination with the prior art and will not be elaborated here.

[0094] In some embodiments of the present application, determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system includes: determining a three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system; determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model.

[0095] Based on the spatial attitude information of the beauty care device and the three-dimensional face model in the wearable device coordinate system obtained in the foregoing embodiments, the three-dimensional morphology of the beauty care device and the three-dimensional face morphology are determined. Thus, by calculating the overlapping area between the beauty care device and the three-dimensional face morphology, it can be calculated which target face area the sensing head of the current beauty care device acts on, thereby providing a basis for automatically switching the beauty care mode. As Figure 8 shown, a schematic diagram of a process for determining a target face area in an embodiment of the present application is provided.

[0096] Since the beauty care device and the three-dimensional face morphology can be regarded as three-dimensional point clouds composed of multiple 3D points respectively, the calculation of the overlapping area can be implemented based on algorithms for overlapping areas of three-dimensional point clouds such as the PCL algorithm. Of course, those skilled in the art can also flexibly select how to calculate the overlapping area according to actual needs, and no specific limitation is made here.

[0097] In some embodiments of the present application, the image of the beauty care device is collected in real time by a camera. Determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system includes: updating the spatial attitude information of the beauty care device in the wearable device coordinate system according to the image recognition result corresponding to the image of the beauty care device collected in real time by the camera; obtaining the spatial attitude change information of preset face key points, and updating the spatial attitude information of the three-dimensional face model in the wearable device coordinate system according to the spatial attitude change information of the preset face key points; determining the target face area corresponding to the beauty care device in the wearable device coordinate system according to the updated spatial attitude information of the beauty care device in the wearable device coordinate system and the updated spatial attitude information of the three-dimensional face model in the wearable device coordinate system.

[0098] The three-dimensional facial morphology in the foregoing embodiments is obtained when the face does not make any expression changes, or the influence of facial expression changes on the subsequent determination of the target face area is ignored. However, in fact, during the use of the beauty care device by the user, the movement of facial muscles is random, and the facial key points will also change accordingly, thereby causing the corresponding three-dimensional facial morphology to change.

[0099] Based on this, in order to further improve the accuracy of face area positioning, the embodiments of the present application can update the relative spatial relationship between the beauty care device and the three-dimensional facial morphology in the coordinate system of the wearable device in real time, thereby providing more accurate input for the subsequent positioning of the face area where the beauty care device is located.

[0100] The real-time update of the spatial attitude information of the beauty care device can be achieved based on a camera that is always on, and the spatial attitude change of the preset facial key points can be achieved based on an electromyogram device. The electromyogram device can be set at positions such as the forehead and outer ear area of the face, for example, and can be integrated with the camera on a device similar to a "headband" and work in a multi-threaded parallel mode with the camera. The electromyogram generated by the facial muscle movement can be detected by the electromyogram device, and the spatial attitude change information of the current facial key points can be calculated through an algorithm, so as to update the spatial attitude information of the three-dimensional facial model in the coordinate system of the wearable device, and then update the three-dimensional facial morphology.

[0101] In addition, considering that the beauty care device itself also has the function of generating electromyogram, the signal captured by the electromyogram device may be interfered. Therefore, it is possible to further consider excluding the interference of the electromyogram signal generated by the beauty care device itself. The closer the relative distance between the beauty care device and the electromyogram device, the greater the interference. Therefore, the electromyogram device can be controlled to turn on and off by detecting the relative distance between the beauty care device and the electromyogram device, so as to ensure that the electromyogram device detects the change of facial key points as much as possible when not interfered or the interference can be ignored, thereby ensuring the accuracy of the update of the three-dimensional facial morphology.

[0102] For example, it can be set that the electromyogram device pauses working after the beauty care device touches the facial skin and starts working after the beauty care device leaves the face, so as to ensure that the electromyogram device detects the change of facial key points as much as possible when not interfered or the interference can be ignored. At the same time, the power consumption of the overall system is also considered, and the switch of the electromyogram device is dynamically controlled to avoid unnecessary operations.

[0103] In some embodiments of the present application, the target face partition is one of multiple face partitions, and the multiple face partitions include at least one of a five-partition and a seven-partition.

[0104] As described above Figure 2As shown in the figure, the face partition in the embodiments of the present application may include a forehead area, an eye area, and a cheek area. This partition method specifically corresponds to a five-partition, including a forehead area, a left eye area, a right eye area, a left cheek area, and a right cheek area. Of course, the face area can be further divided into more detailed areas. For example, the cheek area can be further divided into a jawline area and a cheekbone area. This partition method corresponds to a seven-partition. Of course, specifically how to divide it can be flexibly defined by those skilled in the art according to actual needs, and no specific limitation is made here.

[0105] The more detailed the division of the face area, the higher the accuracy requirement for area positioning. And the face area positioning scheme based on vision implemented in the embodiments of the present application can support a sufficiently fine-grained division of the face area, providing strong support for the accurate skin care of users according to their partitions.

[0106] The face area positioning method based on vision positioning technology in the present application enables the beauty care device to automatically switch the beauty care mode according to the current face area where it is located, liberating the user's hands while achieving accurate skin care. In addition, the positioning of the face area can also endow the beauty care device with more application values, such as real-time temperature detection, humidity detection, impedance detection, etc. at any position in the face area. These partition data can establish the user's personalized face care data, thereby automatically providing personalized instrument parameters, modes, and care suggestions to achieve accurate care.

[0107] It should be noted that the attached drawings mentioned in the embodiments of the present application are for reference only, and some detailed information is not fully shown.

[0108] The embodiments of the present application also provide a vision-based face area positioning device 900, as Figure 9 shown, which provides a structural schematic diagram of a vision-based face area positioning device in the embodiments of the present application. The vision-based face area positioning device 900 at least includes: an identification unit 910, a first determination unit 920, a second determination unit 930, and a third determination unit 940, where:

[0109] The identification unit 910 is configured to obtain an image of the beauty care device collected by a camera on the wearable device and use an image recognition algorithm to recognize the image of the beauty care device to obtain an image recognition result. The image of the beauty care device contains a preset recognizable identifier;

[0110] The first determination unit 920 is configured to determine the spatial attitude information of the beauty care device in the wearable device coordinate system according to the image recognition result;

[0111] The second determination unit 930 is configured to determine the spatial attitude information of the face three-dimensional model in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and preset face key points;

[0112] A third determination unit 940, configured to determine a target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system.

[0113] In some embodiments of the present application, the preset recognizable identifier includes a preset coding identifier, and the recognition unit is specifically configured to: convert the image of the beauty care device into a grayscale image and a target channel image respectively; use an image detection algorithm to detect the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image respectively; determine a coding identifier recognition result according to the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image, and the coding identifier recognition result includes at least three 2D feature points.

[0114] In some embodiments of the present application, the image recognition result includes 2D feature points of a target recognizable identifier, and the first determination unit 920 is specifically configured to: obtain the correspondence between the 2D feature points and 3D feature points of the preset recognizable identifier, where the 3D feature points are the 3D feature points of the preset recognizable identifier in the beauty care device coordinate system; determine the 3D feature points corresponding to the 2D feature points of the target recognizable identifier according to the correspondence between the 2D feature points and 3D feature points of the preset recognizable identifier; determine the spatial attitude information of the camera in the beauty care device coordinate system according to the 3D feature points corresponding to the 2D feature points of the target recognizable identifier; determine the spatial attitude information of the beauty care device in the wearable device coordinate system according to the spatial attitude information of the camera in the beauty care device coordinate system.

[0115] In some embodiments of the present application, the second determination unit 930 is specifically configured to: track the spatial attitude information of the beauty care device in the wearable device coordinate system based on the spatial attitude information of the beauty care device in the wearable device coordinate system and the preset face key points; determine the spatial attitude information of the three-dimensional face model in the wearable device coordinate system according to the tracked spatial attitude information of the beauty care device in the wearable device coordinate system.

[0116] In some embodiments of the present application, the second determination unit 930 is specifically configured to: determine the spatial attitude information of the preset face key points in the wearable device coordinate system according to the tracked spatial attitude information of the beauty care device in the wearable device coordinate system; perform fitting using a three-dimensional face model fitting algorithm according to the spatial attitude information of the preset face key points in the wearable device coordinate system to obtain the spatial attitude information of the three-dimensional face model in the wearable device coordinate system.

[0117] In some embodiments of the present application, the third determination unit 940 is specifically configured to: determine a three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system; determine a target face area corresponding to the beauty care device in the wearable device coordinate system according to the three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model.

[0118] In some embodiments of the present application, the image of the beauty care device is collected in real time by a camera, and the third determination unit 940 is specifically configured to: update the spatial attitude information of the beauty care device in the wearable device coordinate system according to the image recognition result corresponding to the image of the beauty care device collected in real time by the camera; obtain the spatial attitude change information of preset face key points, and update the spatial attitude information of the three-dimensional face model in the wearable device coordinate system according to the spatial attitude change information of the preset face key points; determine a target face area corresponding to the beauty care device in the wearable device coordinate system according to the updated spatial attitude information of the beauty care device in the wearable device coordinate system and the updated spatial attitude information of the three-dimensional face model in the wearable device coordinate system.

[0119] It can be understood that the above vision-based face area positioning device can implement each step of the vision-based face area positioning method provided in the foregoing embodiments. The relevant explanations of the vision-based face area positioning method are applicable to the vision-based face area positioning device, and will not be elaborated here.

[0120] An embodiment of the present application further provides a wearable device, which includes: a camera for collecting an image of a beauty care device; a processor; and

[0121] a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute any one of the foregoing vision-based face area positioning methods.

[0122] In some embodiments of the present application, the wearable device further includes an electromyogram device for detecting the change information of face key points.

[0123] The wearable device in the foregoing embodiments can be regarded as an auxiliary device for realizing the mode switching of the beauty care device. For the convenience of users, the wearable device can adopt a head-mounted device in a form similar to a "hair hoop", integrated with a camera facing the beauty care device and the vision-based face area positioning device in the foregoing embodiments, and an electromyogram device can also be provided. The camera can be one or more RGB cameras or infrared cameras, etc., and can be set at positions such as the forehead according to needs, and is used to collect images of the beauty care device with a preset recognizable identifier. The vision-based face area positioning device can be regarded as a processor, which mainly calculates the spatial poses of the beauty care device and the three-dimensional shape of the face, as well as the target face area. The electromyogram device can be set in areas such as the forehead and outer ear of the face, and is mainly used to detect the dynamic expression changes of the face in real time and obtain the change information of the key points of the face.

[0124] When the user uses the beauty care device, he only needs to wear the wearable device on the head or neck, and the subsequent face area positioning and beauty care mode switching are both automatically realized processes, thus greatly reducing the user's usage burden and improving the user's usage experience.

[0125] The embodiment of the present application also provides a beauty care system, and the beauty care system includes a beauty care device and the wearable device as described above.

[0126] Figure 10 It is a schematic structural diagram of a wearable device in an embodiment of the present application. Please refer to Figure 10 At the hardware level, the wearable device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the wearable device may also include other hardware required for other services.

[0127] The processor, network interface, and memory can be interconnected through an internal bus, and the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 10 only a bidirectional arrow is used in

[0128] A memory for storing programs. Specifically, the program may include program codes, and the program codes include computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0129] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a vision-based face region positioning device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0130] Obtain an image of the beauty care device collected by a camera on the wearable device and use an image recognition algorithm to recognize the image of the beauty care device to obtain an image recognition result, where the image of the beauty care device contains a preset recognizable identifier;

[0131] Determine the spatial attitude information of the beauty care device in the coordinate system of the wearable device according to the image recognition result;

[0132] Determine the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and preset face key points;

[0133] Determine the target face region corresponding to the beauty care device in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device.

[0134] The above as in this application Figure 1The method executed by the vision-based face region localization device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0135] The wearable device can also execute Figure 1 the method executed by the vision-based face region localization device in Figure 1 the illustrated embodiment and implement the functions of the vision-based face region localization device in

[0136] The embodiments of the present application also propose a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions that, when executed by a wearable device including multiple application programs, can enable the wearable device to execute Figure 1 the method executed by the vision-based face region localization device in the illustrated embodiment, and specifically used to execute:

[0137] Obtain an image of the beauty care device collected by the camera on the wearable device and use an image recognition algorithm to recognize the image of the beauty care device to obtain an image recognition result, where the image of the beauty care device contains a preset recognizable identifier;

[0138] Determine the spatial attitude information of the beauty care device in the coordinate system of the wearable device according to the image recognition result;

[0139] Determine the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the preset face key points;

[0140] Determine the target face area corresponding to the beauty care device in the coordinate system of the wearable device according to the spatial attitude information of the beauty care device in the coordinate system of the wearable device and the spatial attitude information of the three-dimensional face model in the coordinate system of the wearable device.

[0141] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0143] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for realizing the functions in the process Figure 1One or more processes and / or blocks Figure 1 Steps of functions specified in one or more blocks.

[0145] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0146] Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0147] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0148] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0149] Those skilled in the art will appreciate that the embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0150] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A vision-based face region localization method, characterized in that, The vision-based face region positioning method includes: Obtaining an image of the beauty care device collected by a camera on a wearable device and using an image recognition algorithm to recognize the image of the beauty care device, obtaining an image recognition result. The image of the beauty care device contains a preset recognizable identifier, and the preset recognizable identifier refers to a specific identifier that is pre-set on the body of the beauty care device and whose position information is determined; Determining the spatial pose information of the beauty care device in the wearable device coordinate system according to the image recognition result; Determining the spatial pose information of the face three-dimensional model in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and preset face key points; Determining the target face region corresponding to the beauty care device in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and the spatial pose information of the face three-dimensional model in the wearable device coordinate system.

2. The vision-based face region localization method according to claim 1, wherein The preset recognizable identifier includes a preset coding identifier. The obtaining an image of the beauty care device collected by a camera on a wearable device and using an image recognition algorithm to recognize the image of the beauty care device, obtaining an image recognition result includes: Converting the image of the beauty care device into a grayscale image and a target channel image respectively; Using an image detection algorithm to detect the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image respectively; Determining a coding identifier recognition result according to the preset coding identifier in the grayscale image and the preset coding identifier in the target channel image, and the coding identifier recognition result includes at least three 2D feature points.

3. The vision-based face region localization method according to claim 1, wherein The image recognition result includes 2D feature points of the target recognizable identifier. The determining the spatial pose information of the beauty care device in the wearable device coordinate system according to the image recognition result includes: Obtaining the correspondence between the 2D feature points and 3D feature points of the preset recognizable identifier, where the 3D feature points are the 3D feature points of the preset recognizable identifier in the beauty care device coordinate system; Determining the 3D feature points corresponding to the 2D feature points of the target recognizable identifier according to the correspondence between the 2D feature points and 3D feature points of the preset recognizable identifier; Using a camera pose solving algorithm to determine the spatial pose information of the camera in the beauty care device coordinate system according to the 3D feature points corresponding to the 2D feature points of the target recognizable identifier; Determining the spatial pose information of the beauty care device in the wearable device coordinate system according to the spatial pose information of the camera in the beauty care device coordinate system.

4. The vision-based face region localization method according to claim 1, characterized in that, The determining the spatial pose information of the face three-dimensional model in the wearable device coordinate system according to the spatial pose information of the beauty care device in the wearable device coordinate system and preset face key points includes: Tracking the spatial pose information of the beauty care device in the wearable device coordinate system based on the spatial pose information of the beauty care device in the wearable device coordinate system and the preset face key points; Determining the spatial pose information of the face three-dimensional model in the wearable device coordinate system according to the tracked spatial pose information of the beauty care device in the wearable device coordinate system.

5. The vision-based face region localization method according to claim 4, characterized in that, The step of determining the spatial posture information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial posture information of the beauty and protection device in the coordinate system of the wearable device that is tracked includes: Determine the spatial posture information of the preset facial key points in the wearable device coordinate system according to the tracked spatial posture information of the beauty and protection device in the wearable device coordinate system; According to the spatial posture information of the preset facial key points in the wearable device coordinate system, a facial three-dimensional model fitting algorithm is used to perform fitting to obtain the spatial posture information of the facial three-dimensional model in the wearable device coordinate system.

6. The vision-based face region localization method according to claim 1, characterized in that The step of determining the target face region corresponding to the beauty care device in the wearable device coordinate system according to the spatial posture information of the beauty care device in the wearable device coordinate system and the spatial posture information of the three-dimensional face model in the wearable device coordinate system comprises: Determine a three-dimensional overlapping area corresponding to the beauty care device and the three-dimensional face model according to the spatial posture information of the beauty care device in the wearable device coordinate system and the spatial posture information of the three-dimensional face model in the wearable device coordinate system; The target face area corresponding to the beauty protection device in the wearable device coordinate system is determined according to the three-dimensional overlapping area corresponding to the beauty protection device and the three-dimensional face model.

7. The vision-based face region localization method according to claim 1, wherein The image of the beauty care device is collected by a camera in real time, and the determining of the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial posture information of the beauty care device in the wearable device coordinate system and the spatial posture information of the three-dimensional face model in the wearable device coordinate system includes: updating the spatial posture information of the beauty care device in the coordinate system of the wearable device according to the image recognition result corresponding to the image of the beauty care device collected by the camera in real time; Acquire spatial posture change information of preset facial key points, and update spatial posture information of the three-dimensional face model in the coordinate system of the wearable device according to the spatial posture change information of the preset facial key points; According to the updated spatial posture information of the beauty and protection device in the wearable device coordinate system and the updated spatial posture information of the three-dimensional face model in the wearable device coordinate system, the target face area corresponding to the beauty and protection device in the wearable device coordinate system is determined.

8. The vision-based face region localization method according to any one of claims 1 to 7, characterized in that The target face area is one of a plurality of face partitions, and the plurality of face partitions include at least one of five partitions and seven partitions.

9. A vision-based face region localization device, characterized in that, The vision-based face area positioning device comprises: an identification unit, used for acquiring an image of the beauty care device captured by a camera on the wearable device and identifying the image of the beauty care device using an image recognition algorithm to obtain an image recognition result, wherein the image of the beauty care device contains a preset identifiable mark, and the preset identifiable mark refers to a specific mark that is pre-set on the body of the beauty care device and determines the position information; A first determining unit, configured to determine the spatial posture information of the beauty and protection device in the coordinate system of the wearable device according to the image recognition result; A second determination unit, configured to determine the spatial attitude information of the three-dimensional face model in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and preset face key points; A third determination unit, configured to determine the target face area corresponding to the beauty care device in the wearable device coordinate system according to the spatial attitude information of the beauty care device in the wearable device coordinate system and the spatial attitude information of the three-dimensional face model in the wearable device coordinate system.

10. A wearable device, characterized in that, The wearable device includes: A camera, configured to collect an image of the beauty care device; A processor; and A memory arranged to store computer-executable instructions, the executable instructions, when executed, causing the processor to execute the vision-based face area positioning method according to any one of claims 1 to 8.

11. A beauty care system, characterized in that, The beauty care system includes a beauty care device and the wearable device according to claim 10.

Images