Image processing method, device and electronic device
By obtaining and using the information on the key points of the teeth to stretch the tooth, the deformation effect is added to the teeth, and the problem of difficulty in effectively simulating tooth stretching in the prior art is solved, and the visual presentation effect of short videos is improved.
Patent Information
- Application Number
- CN202110376320.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-07
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-04-07
AI Technical Summary
The prior art is difficult to effectively add deformation effects to teeth, especially in short video shooting, and there is a lack of image processing methods that can truly simulate tooth stretching.
By obtaining the teeth key point information in the target face image, determining the stretching information, and using this information to stretch the tooth area, the deformation effect of the teeth is achieved.
It has achieved the addition of deformation effects to teeth, enhanced the visual presentation effect of short videos, and met the creative needs of users.
Smart Images

Figure CN112926539B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to an image processing method, device, and electronic device. Background Art
[0002] As smartphone functions continue to improve, various short video shooting platforms have emerged. Users can use various short video shooting platforms to create short video works with personal characteristics. In order to make the short video function produce better visual presentation effects or meet the creative needs of users, these short video platforms have created a need to provide users with various special effects. Summary of the invention
[0003] This disclosure section is provided to introduce concepts in a brief form, which will be described in detail in the detailed description section below. This disclosure section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] The embodiments of the present disclosure provide an image processing method, device and electronic device, which realize the function of adding deformation effects to teeth.
[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising: obtaining a target facial image, wherein the teeth in the mouth area of the target facial image are exposed; obtaining tooth key point information of a target tooth area in the target facial image; based on the tooth key point information, determining stretching information for stretching the target tooth area; using the stretching information, stretching the target tooth area to obtain a stretched facial image.
[0006] In a second aspect, an embodiment of the present disclosure provides an image processing device, which includes: a first acquisition unit, used to acquire a target facial image, wherein the state of the teeth in the mouth area of the target facial image is an exposed state; a second acquisition unit, used to acquire tooth key point information of a target tooth area in the target facial image; a first determination unit, used to determine stretching information for stretching the target tooth area based on the tooth key point information; and a stretching unit, used to stretch the target tooth area using the stretching information to obtain a stretched facial image.
[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in the first aspect.
[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method as described in the first aspect.
[0009] The image processing method, device and electronic device provided by the embodiments of the present disclosure obtain a target face image in which the teeth in the mouth area are exposed; then obtain the tooth key point information of the target tooth area in the target face image; then, based on the tooth key point information, determine the stretching information for stretching the target tooth area; finally, use the stretching information to stretch the target tooth area to obtain the stretched face image. In this way, the function of adding deformation effects to the teeth is realized. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0011] Figure 1 is an exemplary system architecture diagram in which various embodiments of the present disclosure may be applied;
[0012] Figure 2 is a flow chart of an embodiment of an image processing method according to the present disclosure;
[0013] Figure 3 is a schematic diagram of an application scenario of the image processing method according to the present disclosure;
[0014] Figure 4 is a flowchart of another embodiment of the image processing method according to the present disclosure;
[0015] Figure 5 is a flowchart of an embodiment of determining a stretching origin, a stretching end point and a stretching radius in an image processing method according to the present disclosure;
[0016] Figure 6 is a flowchart of an embodiment of a method for determining a target face image in an image processing method according to the present disclosure;
[0017] Figure 7 is a flowchart of an embodiment of extracting lip key point information according to the image processing method of the present disclosure;
[0018] Figure 8 is a schematic structural diagram of an embodiment of an image processing device according to the present disclosure;
[0019] Fig. 9It is a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0021] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0022] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0026] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the image processing method of the present disclosure may be applied.
[0027] like Figure 1As shown, the system architecture 100 may include terminal devices 1011, 1012, 1013, a network 102, and a server 103. The network 102 is used to provide a medium for communication links between the terminal devices 1011, 1012, 1013 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0028] The user can use the terminal devices 1011, 1012, 1013 to interact with the server 103 through the network 102 to send or receive messages, etc. For example, the terminal devices 1011, 1012, 1013 can obtain a pre-trained lip key point extraction network from the server 103. Various communication client applications can be installed on the terminal devices 1011, 1012, 1013, such as image processing applications, short video social applications, instant messaging software, etc.
[0029] Terminal devices 1011, 1012, and 1013 can first obtain a target facial image in which the teeth in the mouth area are exposed; then, they can obtain the tooth key point information of the target tooth area in the above target facial image; then, based on the above tooth key point information, they can determine the stretching information for stretching the above target tooth area; finally, they can use the above stretching information to stretch the above target tooth area to obtain a stretched facial image.
[0030] Terminal devices 1011, 1012, 1013 can be hardware or software. When terminal devices 1011, 1012, 1013 are hardware, they can be various electronic devices with display screens and supporting information interaction, including but not limited to smart phones, tablet computers, laptop portable computers, etc. When terminal devices 1011, 1012, 1013 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.
[0031] The server 103 may be a server that provides various services, for example, it may be a background server that provides a pre-trained lip key point extraction network to the terminal devices 1011 , 1012 , and 1013 .
[0032] It should be noted that the server 103 can be hardware or software. When the server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 103 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.
[0033] It should also be noted that the image processing method provided in the embodiments of the present disclosure is usually executed by the terminal devices 1011 , 1012 , and 1013 , and the image processing apparatus is usually arranged in the terminal devices 1011 , 1012 , and 1013 .
[0034] It should also be noted that if the terminal devices 1011, 1012, and 1013 have local storage of pre-trained lip key point extraction network and other information, the exemplary system architecture 100 may not have the network 102 and the server 103.
[0035] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.
[0036] Continue to refer Figure 2 , shows a process 200 of an embodiment of an image processing method according to the present disclosure. The image processing method comprises the following steps:
[0037] Step 201, obtaining a target face image.
[0038] In this embodiment, the execution subject of the image processing method (for example Figure 1 The terminal device shown in the figure can obtain the target face image. The state of the teeth in the mouth area of the above target face image is usually exposed, so the above target face image usually presents a tooth area. Here, the state of the teeth can include an exposed state or a non-exposed state.
[0039] Step 202, obtaining tooth key point information of a target tooth area in a target face image.
[0040] In this embodiment, the execution subject may obtain the tooth key point information of the target tooth area in the target face image. The target tooth area may be a pre-specified area of the teeth to be processed, for example, the area where the central incisor (also called "front teeth") or the canine (also called "tiger teeth") are located. The tooth key point information of the target tooth area may be the position information of the key points of the teeth indicated by the target tooth area. As an example, the position information may be represented by coordinates (x, y). At this time, the coordinates of the first pixel point in the upper left corner of the target face image may be set to (0, 0).
[0041] Here, the shape of the tooth region can be approximated as a rectangle. As an example, the key point of the tooth can be the center point of the tooth.
[0042] In this embodiment, the execution subject may extract the contour curve of the target tooth area in the target face image through the active contour model algorithm, and then use the contour curve to obtain key point information of the target tooth area.
[0043] Step 203: Determine stretching information for stretching the target tooth region based on the tooth key point information.
[0044] In this embodiment, the execution subject may determine the stretching information for stretching the target tooth region based on the tooth key point information. Here, the execution subject may input the tooth key point information into a pre-trained tooth stretching model to obtain the stretching information for stretching the target tooth region. The tooth stretching model may be used to characterize the correspondence between the tooth key point information and the stretching information for stretching the tooth.
[0045] As an example, the stretching information may include the stretching length of the edge points in the lower edge of the tooth indicated by the target tooth region. It should be noted that the more edge points the stretching lengths included in the stretching information correspond to, the better the stretching effect.
[0046] Step 204, using the stretching information, stretching the target tooth region to obtain a stretched face image.
[0047] In this embodiment, the execution subject may use the stretching information to stretch the target tooth region to obtain a stretched face image. As an example, for an edge point in the lower edge of the tooth indicated by the target tooth region, the execution subject may stretch the edge point by a corresponding stretching length, and then the stretched points may be sequentially connected in a left-to-right or right-to-left order to obtain a stretched face image.
[0048] The method provided by the above embodiment of the present disclosure obtains a target face image in which the teeth of the mouth area are exposed; then obtains the tooth key point information of the target tooth area in the above target face image; then, based on the above tooth key point information, determines the stretching information for stretching the above target tooth area; finally, using the above stretching information, stretches the above target tooth area to obtain the stretched face image. In this way, the function of adding deformation effects to the teeth is realized.
[0049] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the image processing method according to this embodiment. Figure 3In the application scenario, the user terminal 301 may first obtain a target face image 302 in which the teeth in the mouth area are exposed. Afterwards, the tooth key point information 303 of the target tooth area in the target face image 302 may be obtained. Here, the target tooth area is "tiger teeth", and the key point of the teeth may be the center point of the teeth. Then, the user terminal 301 may determine the stretching information for stretching the "tiger teeth" area based on the tooth key point information 303. The user terminal 301 may input the tooth key point information 303 into a pre-trained tooth stretching model to obtain the stretching information for stretching the "tiger teeth" area. Here, the stretching information may include the stretching length of the edge point in the lower edge of the "tiger teeth". Finally, for the edge point in the lower edge of the "tiger teeth", the user terminal 301 may stretch the edge point to the corresponding stretching length, and then the stretched points may be sequentially connected in a left-to-right or right-to-left order to obtain a stretched face image, as shown in icon 304.
[0050] Further references Figure 4 , which shows a process 400 of another embodiment of an image processing method. The process 400 of the image processing method includes the following steps:
[0051] Step 401, obtaining a target face image.
[0052] Step 402, obtaining tooth key point information of a target tooth area in a target face image.
[0053] In this embodiment, steps 401-402 may be performed in a manner similar to steps 201-202, and will not be described in detail herein.
[0054] Step 403: Determine the stretching origin, stretching end point, and stretching radius for stretching the target tooth region based on the tooth key point information.
[0055] In this embodiment, the execution subject of the image processing method (for example Figure 1 The terminal device shown in the figure can determine the stretching origin, stretching end point and stretching radius for stretching the target tooth area based on the above tooth key point information. The stretching origin can be called the stretching starting point, the stretching end point can be called the stretching target point, and the stretching radius can be equivalent to the brush size of liquefaction in image processing software (for example, PS (Adobe Photoshop) software).
[0056] Here, the execution subject can input the tooth key point information into the pre-trained stretching model to obtain the stretching origin, stretching end point and stretching radius for stretching the target tooth area. The stretching model can be used to characterize the correspondence between the tooth key point information and the stretching origin, stretching end point and stretching radius for stretching the tooth.
[0057] Step 404: stretch the target tooth region from the stretching origin to the stretching end point according to the stretching radius to obtain a stretched face image.
[0058] In this embodiment, the execution subject may stretch the target tooth region from the stretching origin to the stretching end point according to the stretching radius to obtain a stretched face image.
[0059] from Figure 4 It can be seen that Figure 2 Compared with the corresponding embodiment, the process 400 of the image processing method in this embodiment embodies the step of stretching the target tooth area from the stretching origin to the stretching end point according to the stretching radius. Therefore, the scheme described in this embodiment provides a stretching method for the tooth area, which makes the tooth stretching effect better.
[0060] Further references Figure 5 , which shows a process 500 of an embodiment of determining a stretching origin, a stretching end point, and a stretching radius in an image processing method. The process 500 of determining the stretching origin, the stretching end point, and the stretching radius includes the following steps:
[0061] Step 501: Determine the width of the tooth indicated by the target tooth region by using the position information of the two lower end points of the tooth indicated by the target tooth region.
[0062] Here, the target tooth region may be a pre-specified region of the tooth to be processed, for example, the region where the central incisor or canine is located. The tooth key point information of the target tooth region may include the position information of the lower two endpoints of the tooth indicated by the target tooth region.
[0063] In this embodiment, the execution subject of the image processing method (for example Figure 1 The terminal device shown in the figure can determine the width of the tooth indicated by the target tooth area by using the position information of the two lower endpoints of the tooth indicated by the target tooth area. Specifically, the execution subject can determine the distance between the two lower endpoints of the tooth (i.e., the length of the line connecting the two lower endpoints) as the width of the tooth.
[0064] Step 502: determine the midpoint of the line connecting the two lower end points of the tooth indicated by the target tooth region as the stretching origin.
[0065] In this embodiment, the execution subject may determine the midpoint of the line connecting the two lower end points of the tooth indicated by the target tooth region as the stretching origin. The stretching origin may be called the stretching starting point.
[0066] Step 503: determine a point below the stretching origin and at a distance from the stretching origin equal to the target length as the stretching end point.
[0067] In this embodiment, the execution entity may determine a point below the stretching origin and at a distance of a target length from the stretching origin as the stretching end point. Here, the target length may be a preset first ratio of the width, for example, the target length may be the width multiplied by 0.6.
[0068] Step 504: determine the preset second ratio of the width as the stretching radius.
[0069] In this embodiment, the execution entity may determine the preset second ratio of the width as the stretching radius. For example, the stretching radius may be the width multiplied by 0.5.
[0070] The method provided by the above embodiment of the present disclosure determines the width of the tooth indicated by the above target tooth area by using the position information of the two lower endpoints of the tooth indicated by the above target tooth area; then, the midpoint of the line connecting the two lower endpoints of the tooth indicated by the above target tooth area is determined as the stretching origin; then, the point located below the above stretching origin and the distance between the point and the above stretching origin being the target length is determined as the stretching end point; finally, the preset second ratio of the width is determined as the stretching radius. Therefore, the scheme described in this embodiment provides a method for determining the stretching origin, stretching end point and stretching radius, so that the determined stretching origin, stretching end point and stretching radius are more accurate.
[0071] Further references Figure 6 , which shows a process 600 of an embodiment of a method for determining a target face image in an image processing method. The process 600 of the method for determining a target face image includes the following steps:
[0072] Step 601, obtaining a face image.
[0073] In this embodiment, the execution subject of the image processing method (for example Figure 1 The terminal device shown in the figure can obtain a face image. When using an image processing application, a user can take a face image to be processed.
[0074] Step 602: extract key point information of at least one tooth area in the face image to obtain a set of tooth key point information.
[0075] In this embodiment, the execution subject can extract key point information of at least one tooth region in the face image to obtain a set of tooth key point information. The at least one tooth region generally includes the target tooth region, and the target tooth region can be a pre-specified region of the teeth to be processed, for example, it can be the region where the central incisor (also called "front teeth") or the canine (also called "tiger teeth") are located. The key point information of the tooth region can be the position information of the key points of the teeth indicated by the tooth region. As an example, the position information can be represented by coordinates (x, y), and the coordinate value of the first pixel in the upper left corner of the image can be set to (0, 0).
[0076] Here, the shape of the tooth region can be approximated as a rectangle. As an example, the key point of the tooth can be the center point of the tooth.
[0077] As an example, the execution subject may extract a contour curve of at least one tooth region in the face image by an active contour model algorithm, and then use the contour curve to obtain key point information of each tooth region in the at least one tooth region, thereby forming a key point information set of at least one tooth region.
[0078] As another example, the execution subject may input the face image into a pre-trained tooth key point extraction network to obtain key point information of the at least one tooth region. As an example, the face image may be input into a trained regression neural network, such as ResNet (Residual Neural Network), VGG (Visual Geometry Group, deep convolutional neural network), ShuffleNet (lightweight neural network), etc.
[0079] Step 603: extract key point information of the lip area in the face image to obtain lip key point information.
[0080] In this embodiment, the execution subject may extract key point information of the lip region in the face image to obtain lip key point information.
[0081] As an example, the execution subject can detect key points of the lip area based on a traditional algorithm, such as ASM (Active Shape Model), to obtain position information of lip edge feature points. As an example, the lip edge feature points can be the intersection of the lip edge and the teeth.
[0082] Among them, the ASM model is a point distribution model. In the point distribution model, the geometric shapes of objects with similar appearances, such as the geometric shape of the lip area, are represented by a shape vector formed by connecting the coordinates of key feature points in series. Then, a shape model is obtained through training. Finally, the shape model is used to determine the key feature point information in the new image.
[0083] As another example, the execution subject inputs the facial image into a pre-trained lip key point extraction network to obtain lip key point information. The lip key point information includes: position information of the intersection of the lip edge and the teeth. As an example, the position information can be represented by coordinates (x, y), and the coordinate value of the first pixel in the upper left corner of the image can be set to (0, 0). As an example, the lip key point extraction network can be a trained regression neural network, such as ResNet, VGG, and ShuffleNet.
[0084] Step 604: Detect the state of the teeth based on the tooth key point information set and the lip key point information.
[0085] In this embodiment, the execution subject may detect the state of the teeth based on the set of teeth key point information obtained in step 602 and the lip key point information obtained in step 603. The state of the teeth may include: an exposed state or a non-exposed state.
[0086] As an example, the above-mentioned execution subject can first select any tooth key point information from the tooth key point information set. As an example, the tooth key point information is the position information of the center point of each tooth area in at least one tooth area shown in the above-listed teeth, expressed by the coordinates (x, y), and the coordinate value of the first pixel point in the upper left corner of the face image can be recorded as (0, 0). Randomly select a tooth key point information, and the selected tooth key point information can be (26, 20). Afterwards, the lip key point information whose x coordinate value in the lip key point information is close to a certain x coordinate value in the above-selected tooth key point information can be found from the lip key point information. As an example, the lip key point information can be the position information of the intersection of the upper lip edge and the teeth. Expressed by the coordinates (x, y). For example, the lip key point information is (13, 7), (19, 9), (27, 8), (34, 6). Select the one closest to the x coordinate value in the tooth key point information, such as (27, 8). Finally, the y coordinate value in the position information of the selected tooth key point information and the selected lip key point whose x coordinate value is closest to the y coordinate value in the selected lip key point information can be subtracted from the y coordinate value in the selected lip key point information to obtain a difference value. In response to determining that the absolute value of the difference value is greater than a preset first target value, the above-mentioned execution subject can determine that the teeth are in an exposed state. Otherwise, it can be determined that the teeth are not exposed.
[0087] As another example, the execution subject may first average the y coordinate values in the set of teeth key point information. Then, the information of a key point in the lip key point information may be selected. Finally, the average value may be subtracted from the y coordinate value in the selected key point information to obtain a difference value. In response to determining that the absolute value of the difference value is greater than a preset second target value, the execution subject may determine that the teeth are in an exposed state. Otherwise, it may be determined that the teeth are not exposed.
[0088] Step 605: In response to detecting that the teeth are in an exposed state, the face image is determined as a target face image.
[0089] In this embodiment, if it is detected in step 604 that the teeth are in an exposed state, the execution subject may determine the facial image as a target facial image.
[0090] The method provided by the above-mentioned embodiment of the present disclosure obtains a set of key point information of teeth by extracting key point information of at least one tooth area in a face image, which provides a basis for the subsequent processing of a single tooth. Afterwards, the state of the teeth can be detected using the key point information of the lip area extracted from the face image, thereby preparing for subsequent tooth special effects. Finally, in response to detecting that the state of the teeth is exposed, the above-mentioned face image is determined as a target face image. In this way, face images with exposed teeth can be screened out from face images.
[0091] Further references Figure 7 , which shows a process 700 of an embodiment of extracting lip key point information in an image processing method. The process 700 of extracting lip key point information includes the following steps:
[0092] Step 701: input the face image into a pre-trained lip key point extraction network to obtain initial key point information of the lip area in the face image.
[0093] In this embodiment, the execution subject of the image processing method (for example Figure 1 The terminal device shown in the figure can input the face image into a pre-trained lip key point extraction network to obtain the initial key point information of the lip area in the face image. The above-mentioned lip key point extraction network can be used to characterize the correspondence between the face image and the initial key point information of the lip area in the face image. Here, the initial key point information of the lip area can be the position information of the intersection of the upper lip edge and the teeth. The number of initial key points of the lip area can be a preset number.
[0094] Step 702: Determine the inner contour of the upper lip in the lip region using the initial key point information.
[0095] In this embodiment, the execution subject may determine the inner contour of the upper lip in the lip region by using the initial key point information. The execution subject may connect the initial key points in order from left to right or from right to left to obtain the inner contour of the upper lip in the lip region.
[0096] It should be noted that the more the initial key points are, the more accurate the inner contour of the upper lip is.
[0097] Step 703, determining whether the presentation direction of the lip region is the target direction.
[0098] In this embodiment, the execution entity can determine whether the presentation direction of the lip area is the target direction. The presentation direction of the lip area can be the direction of the line between the left and right corners of the lips. The target direction is usually the direction of the upper and lower edges of the face image. That is, the execution entity can determine whether the direction of the line between the left and right corners of the lip area is parallel to the directions of the upper and lower edges of the face image. Generally speaking, if the direction of the line between the left and right corners of the lip area is parallel to the directions of the upper and lower edges of the face image, it means that the presentation direction of the lip area is the standard direction.
[0099] If it is determined that the presentation direction of the lip region is the target direction, the execution entity may execute step 704 .
[0100] Step 704, if the presentation direction of the lip area is the target direction, a vertical upward ray is drawn with the two lower endpoints of the teeth indicated by at least one tooth area as the starting point, and the intersection of the ray and the inner contour is determined as the lip key point to generate the lip key point information.
[0101] Here, the tooth key point information set may include position information of two lower endpoints of each tooth in the upper row of teeth indicated by the at least one tooth region.
[0102] In this embodiment, if it is determined in step 703 that the presentation direction of the above-mentioned lip area is the above-mentioned target direction, the above-mentioned execution entity can draw a vertical upward ray with the lower two end points of the teeth indicated by the above-mentioned at least one tooth area as the starting point, and the intersection of the above-mentioned ray and the above-mentioned inner contour can be determined as the lip key point to generate the lip key point information.
[0103] Here, if there are 4 teeth in the tooth region, the number of the lip key points is usually 8.
[0104] The method provided by the above embodiment of the present disclosure obtains the initial key point information of the lip area in the above face image by inputting the above face image into a pre-trained lip key point extraction network; then, using the above initial key point information, the inner contour of the upper lip in the above lip area is determined; then, whether the presentation direction of the above lip area is the target direction is determined; if so, a vertical upward ray is made with the lower two endpoints of the teeth indicated by the above at least one tooth area as the starting point, and the intersection of the above ray and the above inner contour is determined as the lip key point to generate the lip key point information. Thus, a method for extracting lip key point information is provided, and more accurate lip key point information can be extracted in this way.
[0105] In some optional implementations, if it is determined in step 703 that the presentation direction of the lip area is not the target direction, the execution entity may rotate the facial image to adjust the presentation direction of the lip area to the target direction. That is, the direction of the line between the left and right corners of the lip area may be made parallel to the direction of the upper and lower edges of the facial image. Afterwards, a vertical upward ray may be made with the lower two endpoints of the teeth indicated by at least one tooth area in the adjusted facial image as the starting point, and the intersection of the ray and the inner contour of the lip area in the adjusted facial image may be determined as the lip key point to generate the lip key point information. In this way, when the lip area presented in the facial image is crooked, the lip area in the facial image may be rotated, so that the presentation direction of the lip area is adjusted to the target direction, so as to facilitate the subsequent extraction of the lip key point information.
[0106] In some optional implementations, the execution subject may detect the state of the teeth based on the tooth key point information set and the lip key point information in the following manner: the execution subject may determine the width of the preset tooth area in the at least one tooth area using the tooth key point information set and the lip key point information. As an example, the preset tooth area may be the lateral incisor of the upper teeth. In this case, the execution subject may determine the width of the lateral incisor of the upper teeth. Since there are usually two lateral incisors, the average value of the widths of the two lateral incisors may be calculated as the width of the lateral incisor of the upper teeth. The minimum value of the widths of the two lateral incisors may also be determined as the width of the lateral incisor of the upper teeth. Afterwards, the execution subject may determine whether the width of the preset tooth area is greater than a preset width threshold. If the width of the second target tooth area is greater than the width threshold, the execution subject may determine that the state of the teeth is an exposed state. If the width of the second target tooth area is less than or equal to the width threshold, the execution subject may determine that the state of the teeth is a non-exposed state. Thus, a method for determining the state of the teeth is provided, and the state of the teeth may be determined more accurately in this way.
[0107] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an image processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0108] like Figure 8 As shown, the image processing device 800 of this embodiment includes: a first acquisition unit 801, a second acquisition unit 802, a first determination unit 803 and a stretching unit 804. The first acquisition unit 801 is used to acquire a target face image, wherein the state of the teeth in the mouth area of the target face image is an exposed state; the second acquisition unit 802 is used to acquire the tooth key point information of the target tooth area in the target face image; the first determination unit 803 is used to determine the stretching information for stretching the target tooth area based on the tooth key point information; the stretching unit 804 is used to stretch the target tooth area using the stretching information to obtain the stretched face image.
[0109] In this embodiment, the specific processing of the first acquisition unit 801, the second acquisition unit 802, the first determination unit 803 and the stretching unit 804 of the image processing device 800 can refer to Figure 2 This corresponds to step 201, step 202, step 203 and step 204 in the embodiment.
[0110] In some optional implementations, the stretching information may include a stretching origin, a stretching end point and a stretching radius; and the stretching unit 804 may be further used to use the stretching information in the following manner to stretch the target tooth area to obtain a stretched facial image: the stretching unit 804 may stretch the target tooth area from the stretching origin to the stretching end point according to the stretching radius to obtain a stretched facial image.
[0111] In some optional implementations, the above-mentioned tooth key point information may include position information of the two lower endpoints of the tooth indicated by the above-mentioned target tooth area; and the above-mentioned first determination unit 803 may be further used to determine the stretching information for stretching the above-mentioned target tooth area based on the above-mentioned tooth key point information in the following manner: the above-mentioned first determination unit 803 may first use the position information of the two lower endpoints of the tooth indicated by the above-mentioned target tooth area to determine the width of the tooth indicated by the above-mentioned target tooth area; then, the midpoint of the line connecting the two lower endpoints of the tooth indicated by the above-mentioned target tooth area may be determined as the stretching origin; then, the point located below the above-mentioned stretching origin and the distance between the point and the above-mentioned stretching origin is the target length may be determined as the stretching end point, wherein the above-mentioned target length is a preset first proportion of the above-mentioned width; finally, the preset second proportion of the above-mentioned width may be determined as the stretching radius.
[0112] In some optional implementations, the image processing device 800 may include: a third acquisition unit (not shown in the figure), a first extraction unit (not shown in the figure), a second extraction unit (not shown in the figure), a detection unit (not shown in the figure), and a second determination unit (not shown in the figure). The third acquisition unit may be used to acquire a face image; the first extraction unit may be used to extract key point information of at least one tooth region in the face image to obtain a set of tooth key point information, wherein the at least one tooth region includes the target tooth region; the second extraction unit may be used to extract key point information of a lip region in the face image to obtain lip key point information; the detection unit may be used to detect the state of the teeth based on the set of tooth key point information and the lip key point information, wherein the state of the teeth includes an exposed state or a non-exposed state; the second determination unit may be used to determine the face image as a target face image in response to detecting that the state of the teeth is an exposed state.
[0113] In some optional implementations, the above-mentioned tooth key point information set may include the position information of the lower two endpoints of each tooth in the upper row of teeth indicated by the above-mentioned at least one tooth region; and the above-mentioned second extraction unit may be further used to extract the key point information of the lip region in the above-mentioned face image in the following manner to obtain the lip key point information: the above-mentioned second extraction unit may first input the above-mentioned face image into a pre-trained lip key point extraction network to obtain the initial key point information of the lip region in the above-mentioned face image; thereafter, the above-mentioned initial key point information may be used to determine the inner contour of the upper lip in the above-mentioned lip region; then, it may be determined whether the presentation direction of the above-mentioned lip region is the target direction; if so, a vertical upward ray is made with the lower two endpoints of the teeth indicated by the above-mentioned at least one tooth region as the starting point, and the intersection of the above-mentioned ray and the above-mentioned inner contour is determined as the lip key point to generate the lip key point information.
[0114] In some optional implementations, the second extraction unit may also be used for: if the presentation direction of the lip area is not the target direction, rotating the facial image to adjust the presentation direction of the lip area to the target direction, drawing a vertical upward ray with the two lower endpoints of the teeth indicated by the at least one tooth area in the adjusted facial image as the starting point, determining the intersection of the ray and the inner contour of the lip area in the adjusted facial image as the lip key point, and generating lip key point information.
[0115] In some optional implementations, the detection unit can be further used to detect the state of the teeth based on the tooth key point information set and the lip key point information in the following manner: the detection unit can use the tooth key point information set and the lip key point information to determine the width of a preset tooth area in the at least one tooth area; then, it can be determined whether the width of the preset tooth area is greater than a preset width threshold; if so, it can be determined that the state of the teeth is an exposed state.
[0116] Reference below Fig. 9 , which shows an electronic device (eg, Figure 1 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0117] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0118] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Fig. 9 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0119] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus, or a device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.
[0120] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a target face image, wherein the teeth in the mouth area of the target face image are exposed; obtains the tooth key point information of the target tooth area in the target face image; determines the stretching information for stretching the target tooth area based on the tooth key point information; and stretches the target tooth area using the stretching information to obtain a stretched face image.
[0121] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0123] According to one or more embodiments of the present disclosure, there is provided an image processing method, the method comprising: obtaining a target facial image, wherein the teeth in the mouth area of the target facial image are exposed; obtaining tooth key point information of the target tooth area in the target facial image; based on the tooth key point information, determining stretching information for stretching the target tooth area; using the stretching information, stretching the target tooth area to obtain a stretched facial image.
[0124] According to one or more embodiments of the present disclosure, the above-mentioned stretching information includes a stretching origin, a stretching end point and a stretching radius; and the above-mentioned stretching information is used to stretch the above-mentioned target tooth area to obtain a stretched facial image, including: according to the above-mentioned stretching radius, the above-mentioned target tooth area is stretched from the above-mentioned stretching origin to the above-mentioned stretching end point to obtain a stretched facial image.
[0125] According to one or more embodiments of the present disclosure, the above-mentioned tooth key point information includes the position information of the two lower endpoints of the teeth indicated by the above-mentioned target tooth area; and the above-mentioned stretching information for stretching the above-mentioned target tooth area based on the above-mentioned tooth key point information includes: using the position information of the two lower endpoints of the teeth indicated by the above-mentioned target tooth area to determine the width of the teeth indicated by the above-mentioned target tooth area; determining the midpoint of the line connecting the two lower endpoints of the teeth indicated by the above-mentioned target tooth area as the stretching origin; determining the point below the above-mentioned stretching origin and the point whose distance from the above-mentioned stretching origin is the target length as the stretching end point, wherein the above-mentioned target length is a preset first ratio of the above-mentioned width; and determining the preset second ratio of the above-mentioned width as the stretching radius.
[0126] According to one or more embodiments of the present disclosure, before acquiring the target facial image, the method includes: acquiring a facial image; extracting key point information of at least one tooth region in the facial image to obtain a tooth key point information set, wherein the at least one tooth region includes the target tooth region; extracting key point information of a lip region in the facial image to obtain lip key point information; detecting the state of the teeth based on the tooth key point information set and the lip key point information, wherein the state of the teeth includes an exposed state or a non-exposed state; in response to detecting that the state of the teeth is an exposed state, determining the facial image as a target facial image.
[0127] According to one or more embodiments of the present disclosure, the above-mentioned tooth key point information set includes the position information of the lower two endpoints of each tooth in the upper row of teeth indicated by the above-mentioned at least one tooth region; and the above-mentioned extraction of key point information of the lip region in the above-mentioned face image to obtain lip key point information, including: inputting the above-mentioned face image into a pre-trained lip key point extraction network to obtain initial key point information of the lip region in the above-mentioned face image; using the above-mentioned initial key point information to determine the inner contour of the upper lip in the above-mentioned lip region; determining whether the presentation direction of the above-mentioned lip region is the target direction; if so, making a vertical upward ray with the lower two endpoints of the teeth indicated by the above-mentioned at least one tooth region as the starting point, determining the intersection of the above-mentioned ray and the above-mentioned inner contour as the lip key point, and generating lip key point information.
[0128] According to one or more embodiments of the present disclosure, after determining whether the presentation direction of the lip area is the target direction, the method further includes: if the presentation direction of the lip area is not the target direction, rotating the facial image to adjust the presentation direction of the lip area to the target direction, drawing a vertical upward ray with the two lower endpoints of the teeth indicated by the at least one tooth area in the adjusted facial image as the starting point, determining the intersection of the ray and the inner contour of the lip area in the adjusted facial image as a lip key point, and generating lip key point information.
[0129] According to one or more embodiments of the present disclosure, the above-mentioned detection of the state of teeth based on the above-mentioned tooth key point information set and the above-mentioned lip key point information includes: using the above-mentioned tooth key point information set and the above-mentioned lip key point information to determine the width of the preset tooth area in the above-mentioned at least one tooth area; determining whether the width of the above-mentioned preset tooth area is greater than a preset width threshold; if so, determining that the state of the teeth is an exposed state.
[0130] According to one or more embodiments of the present disclosure, there is provided an image processing device, which includes: a first acquisition unit, used to acquire a target facial image, wherein the state of teeth in the mouth area of the target facial image is an exposed state; a second acquisition unit, used to acquire tooth key point information of a target tooth area in the target facial image; a first determination unit, used to determine stretching information for stretching the target tooth area based on the tooth key point information; and a stretching unit, used to stretch the target tooth area using the stretching information to obtain a stretched facial image.
[0131] According to one or more embodiments of the present disclosure, the above-mentioned stretching information includes a stretching origin, a stretching end point and a stretching radius; and the above-mentioned stretching unit is further used to use the above-mentioned stretching information in the following manner to stretch the above-mentioned target tooth area to obtain a stretched facial image: according to the above-mentioned stretching radius, the above-mentioned target tooth area is stretched from the above-mentioned stretching origin to the above-mentioned stretching end point to obtain a stretched facial image.
[0132] According to one or more embodiments of the present disclosure, the above-mentioned tooth key point information includes the position information of the two lower endpoints of the teeth indicated by the above-mentioned target tooth area; and the above-mentioned first determination unit is further used to determine the stretching information for stretching the above-mentioned target tooth area based on the above-mentioned tooth key point information in the following manner: using the position information of the two lower endpoints of the teeth indicated by the above-mentioned target tooth area, determine the width of the teeth indicated by the above-mentioned target tooth area; determine the midpoint of the line connecting the two lower endpoints of the teeth indicated by the above-mentioned target tooth area as the stretching origin; determine the point below the above-mentioned stretching origin and the point whose distance from the above-mentioned stretching origin is the target length as the stretching end point, wherein the above-mentioned target length is a preset first ratio of the above-mentioned width; and determine the preset second ratio of the above-mentioned width as the stretching radius.
[0133] According to one or more embodiments of the present disclosure, the device includes: a third acquisition unit, used to acquire a facial image; a first extraction unit, used to extract key point information of at least one tooth area in the above facial image to obtain a tooth key point information set, wherein the at least one tooth area includes the above target tooth area; a second extraction unit, used to extract key point information of a lip area in the above facial image to obtain lip key point information; a detection unit, used to detect the state of the teeth based on the above tooth key point information set and the above lip key point information, wherein the state of the teeth includes an exposed state or a non-exposed state; a second determination unit, used to determine the above facial image as a target facial image in response to detecting that the state of the teeth is an exposed state.
[0134] According to one or more embodiments of the present disclosure, the above-mentioned tooth key point information set includes the position information of the lower two endpoints of each tooth in the upper row of teeth indicated by the above-mentioned at least one tooth region; and the above-mentioned second extraction unit is further used to extract the key point information of the lip region in the above-mentioned face image in the following manner to obtain the lip key point information: input the above-mentioned face image into a pre-trained lip key point extraction network to obtain the initial key point information of the lip region in the above-mentioned face image; use the above-mentioned initial key point information to determine the inner contour of the upper lip in the above-mentioned lip region; determine whether the presentation direction of the above-mentioned lip region is the target direction; if so, make a vertical upward ray with the lower two endpoints of the teeth indicated by the above-mentioned at least one tooth region as the starting point, determine the intersection of the above-mentioned ray and the above-mentioned inner contour as the lip key point, and generate the lip key point information.
[0135] According to one or more embodiments of the present disclosure, the second extraction unit is also used for: if the presentation direction of the lip area is not the target direction, rotating the facial image to adjust the presentation direction of the lip area to the target direction, drawing a vertical upward ray with the two lower end points of the teeth indicated by the at least one tooth area in the adjusted facial image as the starting point, determining the intersection of the ray and the inner contour of the lip area in the adjusted facial image as the lip key point, and generating lip key point information.
[0136] According to one or more embodiments of the present disclosure, the detection unit is further used to detect the state of the teeth based on the tooth key point information set and the lip key point information in the following manner: using the tooth key point information set and the lip key point information to determine the width of a preset tooth area in the at least one tooth area; determining whether the width of the preset tooth area is greater than a preset width threshold; if so, determining that the state of the teeth is an exposed state.
[0137] According to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned image processing method.
[0138] According to one or more embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned image processing method are implemented.
[0139] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a first acquisition unit, a second acquisition unit, a first determination unit, and a stretching unit. The names of these units do not constitute limitations on the units themselves in some cases, for example, the first acquisition unit may also be described as a "unit for acquiring a target face image."
[0140] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.
Claims
1. An image processing method, characterized in that: include: Acquire a target face image, wherein the teeth in the mouth area of the target face image are exposed; Acquire tooth key point information of a target tooth area in the target face image; Based on the tooth key point information, determining stretching information for stretching the target tooth area, wherein the stretching information includes a stretching origin, a stretching end point, and a stretching radius; Using the stretching information, stretching the target tooth region to obtain a stretched face image, including: stretching the target tooth region from the stretching origin to the stretching end point according to the stretching radius to obtain a stretched face image, wherein the stretching length and the stretching radius of the target tooth region are determined based on the width of the teeth indicated by the target tooth region; The tooth key point information includes position information of two lower end points of the tooth indicated by the target tooth region; and The step of determining stretching information for stretching the target tooth region based on the tooth key point information includes: Determine the width of the tooth indicated by the target tooth region by using the position information of the lower two end points of the tooth indicated by the target tooth region; Determine the midpoint of the line connecting the two lower end points of the tooth indicated by the target tooth region as the stretching origin; Determine a point below the stretching origin and at a target distance from the stretching origin as a stretching end point; A preset second ratio of the width is determined as a stretching radius.
2. The method according to claim 1, characterized in that The target length is a preset first ratio of the width.
3. The method according to claim 1, characterized in that Before acquiring the target face image, the method includes: Get face image; Extracting key point information of at least one tooth region in the face image to obtain a set of tooth key point information, wherein the at least one tooth region includes the target tooth region; Extracting key point information of the lip area in the face image to obtain lip key point information; Based on the tooth key point information set and the lip key point information, detecting the state of the teeth, wherein the state of the teeth includes an exposed state or a non-exposed state; In response to detecting that the state of the teeth is an exposed state, the face image is determined as a target face image.
4. The method according to claim 3, characterized in that The tooth key point information set includes position information of two lower endpoints of each tooth in the upper row of teeth indicated by the at least one tooth region; as well as The step of extracting key point information of the lip region in the face image to obtain the lip key point information includes: Inputting the face image into a pre-trained lip key point extraction network to obtain initial key point information of the lip area in the face image; Determine the inner contour of the upper lip in the lip area by using the initial key point information; Determining whether the presentation direction of the lip region is a target direction; If so, a vertical upward ray is drawn with the two lower end points of the tooth indicated by the at least one tooth region as the starting point, and the intersection of the ray and the inner contour is determined as the lip key point to generate the lip key point information.
5. The method according to claim 4, characterized in that After determining whether the presentation direction of the lip region is the target direction, the method further includes: If the presentation direction of the lip area is not the target direction, the facial image is rotated to adjust the presentation direction of the lip area to the target direction, and a vertical upward ray is drawn with the two lower endpoints of the teeth indicated by the at least one tooth area in the adjusted facial image as the starting point, and the intersection of the ray and the inner contour of the lip area in the adjusted facial image is determined as the lip key point to generate lip key point information.
6. The method according to any one of claims 3 to 5, characterized in that: The detecting the state of the teeth based on the teeth key point information set and the lips key point information comprises: Determine a width of a preset tooth region in the at least one tooth region by using the tooth key point information set and the lip key point information; Determining whether the width of the preset tooth area is greater than a preset width threshold; If so, it is determined that the state of the tooth is an exposed state.
7. An image processing device, characterized in that: include: A first acquisition unit is used to acquire a target face image, wherein the teeth in the mouth area of the target face image are exposed; A second acquisition unit is used to acquire tooth key point information of a target tooth area in the target face image; A first determining unit, configured to determine stretching information for stretching the target tooth region based on the tooth key point information, wherein the stretching information includes a stretching origin, a stretching end point, and a stretching radius; A stretching unit, configured to use the stretching information to stretch the target tooth region to obtain a stretched face image; The stretching unit is further used to use the stretching information to stretch the target tooth area to obtain a stretched face image in the following manner: According to the stretching radius, stretching the target tooth region from the stretching origin to the stretching end point to obtain a stretched face image, wherein the stretching length and the stretching radius of the target tooth region are determined based on the width of the teeth indicated by the target tooth region; The tooth key point information includes position information of two lower end points of the tooth indicated by the target tooth region; and The first determining unit is further configured to determine stretching information for stretching the target tooth region based on the tooth key point information in the following manner: Determine the width of the tooth indicated by the target tooth region by using the position information of the lower two end points of the tooth indicated by the target tooth region; Determine the midpoint of the line connecting the two lower end points of the tooth indicated by the target tooth region as the stretching origin; Determine a point below the stretching origin and at a target distance from the stretching origin as a stretching end point; A preset second ratio of the width is determined as a stretching radius.
8. The device according to claim 7, characterized in that The target length is a preset first ratio of the width.
9. The device according to claim 7, characterized in that The device comprises: A third acquisition unit, used to acquire a face image; A first extraction unit is used to extract key point information of at least one tooth region in the face image to obtain a set of tooth key point information, wherein the at least one tooth region includes the target tooth region; A second extraction unit is used to extract key point information of the lip area in the face image to obtain lip key point information; A detection unit, configured to detect a state of teeth based on the set of teeth key point information and the lip key point information, wherein the state of the teeth includes an exposed state or a non-exposed state; The second determining unit is configured to determine the face image as a target face image in response to detecting that the teeth are in an exposed state.
10. The device according to claim 9, characterized in that The tooth key point information set includes position information of two lower endpoints of each tooth in the upper row of teeth indicated by the at least one tooth region; and The second extraction unit is further used to extract key point information of the lip area in the face image in the following manner to obtain lip key point information: Inputting the face image into a pre-trained lip key point extraction network to obtain initial key point information of the lip area in the face image; Determine the inner contour of the upper lip in the lip area by using the initial key point information; Determining whether the presentation direction of the lip region is a target direction; If so, a vertical upward ray is drawn with the two lower end points of the tooth indicated by the at least one tooth region as the starting point, and the intersection of the ray and the inner contour is determined as the lip key point to generate the lip key point information.
11. The device according to claim 10, characterized in that The second extraction unit is also used for: If the presentation direction of the lip area is not the target direction, the facial image is rotated to adjust the presentation direction of the lip area to the target direction, and a vertical upward ray is drawn with the two lower endpoints of the teeth indicated by the at least one tooth area in the adjusted facial image as the starting point, and the intersection of the ray and the inner contour of the lip area in the adjusted facial image is determined as the lip key point to generate lip key point information.
12. The device according to any one of claims 9 to 11, characterized in that The detection unit is further configured to detect the state of the teeth based on the teeth key point information set and the lip key point information in the following manner: Determine a width of a preset tooth region in the at least one tooth region by using the tooth key point information set and the lip key point information; Determining whether the width of the preset tooth area is greater than a preset width threshold; If so, it is determined that the state of the tooth is an exposed state.
13. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
