Stain detection method, electronic device, storage medium, and program product

Abstract

Embodiments of the present application provide a kind of color spot detection method, electronic equipment, storage medium and program product.The method comprises: obtaining image to be processed, and the image to be processed includes the skin area to be measured;Target region in the image to be processed is converted to LAB color space, and LAB image is obtained;Wherein, target region includes at least part of the skin area to be measured on the image to be processed;Based on the L component, A component and B component of each pixel on LAB image, the target brown component of each pixel on LAB image is determined;Based on the target brown component of each pixel on LAB image, the color spot detection of target region is carried out, and color spot detection result is obtained.According to the above technical solution, the distribution state of the pigmentation of the skin area to be measured can be more accurately represented, which helps to accurately obtain the color spot detection result of the image to be processed.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically to a method for detecting color spots, electronic devices, storage media, and computer program products. Background Technology

[0002] With the development of the times, the "appearance economy" has received increasing attention. Young women's pursuit of beauty and anxiety about their appearance make them more concerned about the condition of their facial skin, especially issues that directly reflect aging and facial pigmentation, such as various types of blemishes like sunspots, freckles, melasma, and age spots. However, current algorithms struggle to effectively capture information such as the depth and distribution of these blemishes. Artificial intelligence (AI)-based blemish detection algorithms often have low recall rates and cannot accurately determine the extent of blemishes on a person's face. Therefore, a new blemish detection solution is needed to address these technical problems. Summary of the Invention

[0003] This application is made in view of the above-mentioned problems. This application provides a method for detecting color spots, an electronic device, a storage medium, and a computer program product.

[0004] According to one aspect of this application, a method for detecting pigmentation spots is provided, comprising: acquiring an image to be processed, the image to be processed containing a skin region to be tested; converting the target region in the image to be processed to the LAB color space to obtain a LAB image; wherein the target region includes at least a portion of the skin region to be tested on the image to be processed; determining the target brown component of each pixel in the LAB image based on the L component, A component and B component of each pixel in the LAB image; and performing pigmentation spot detection on the target region based on the target brown component of each pixel in the LAB image to obtain a pigmentation spot detection result.

[0005] According to another aspect of this application, an electronic device is also provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the above-described stain detection method.

[0006] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored, wherein the program instructions are used to execute the above-described stain detection method when running.

[0007] According to another aspect of this application, a computer program product is also provided, the computer program product comprising a computer program, wherein the computer program, when running, is used to perform the above-described stain detection method.

[0008] According to the stain detection method, electronic device, storage medium, and computer program product of this application, a LAB image is obtained by converting the target area in the image to be processed to the LAB color space. A target brown component is determined based on the L, A, and B components of each pixel in the LAB image, and a stain detection result for the target area is obtained based on this target brown component. In this scheme, since the colors presented by the L, A, and B components are highly consistent with human visual perception, the target brown component determined based on the L, A, and B components also has high consistency with the user's visual perception, which can accurately characterize the distribution of pigmentation in the skin area to be tested. This helps to accurately obtain the stain detection result of the image to be processed. Attached Figure Description

[0009] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 A schematic block diagram of an example electronic device for implementing the stain detection method and apparatus according to embodiments of this application is shown;

[0011] Figure 2 A schematic flowchart of a stain detection method according to an embodiment of this application is shown;

[0012] Figure 3 A schematic diagram showing the results of facial landmark detection according to an embodiment of this application;

[0013] Figure 4 A schematic block diagram of a stain detection device according to an embodiment of this application is shown; and

[0014] Figure 5 A schematic block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0015] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include facial recognition, image processing, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beauty filters, cosmetics, medical aesthetics, and intelligent temperature measurement.

[0016] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.

[0017] This application provides a method for detecting pigmentation spots, an electronic device, a storage medium, and a computer program product. The pigmentation spot detection method according to this application can effectively detect pigmentation spots in a test area of ​​skin. The pigmentation spot detection technology according to this application can be applied to any field involving pigmentation spot detection, such as live streaming, video effects, and medical aesthetics.

[0018] First, refer to Figure 1 This describes an example electronic device 100 for implementing the stain detection method and apparatus according to embodiments of this application.

[0019] like Figure 1As shown, the electronic device 100 includes one or more processors 102 and one or more storage devices 104. Optionally, the electronic device 100 may also include an input device 106, an output device 108, and an image capturing device 110, these components being interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 1 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may also have other components and structures as needed.

[0020] The processor 102 may be implemented in at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), and microprocessor. The processor 102 may be one or a combination of several of the following: central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), or other processing units with data processing capabilities and / or instruction execution capabilities. It may also control other components in the electronic device 100 to perform the desired functions.

[0021] The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of this application described below, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.

[0022] The input device 106 may be a device used by a user to input commands, and may include one or more of the following: keyboard, mouse, microphone, and touch screen.

[0023] The output device 108 can output various information (e.g., images and / or sound) to the outside (e.g., a user), and may include one or more of a display, speaker, etc. Optionally, the input device 106 and the output device 108 can be integrated together and implemented using the same interactive device (e.g., a touch screen).

[0024] The image acquisition device 110 can acquire images and store the acquired images in the storage device 104 for use by other components. The image acquisition device 110 can be a standalone camera or a camera in a mobile terminal, etc. It should be understood that the image acquisition device 110 is only an example, and the electronic device 100 may not include the image acquisition device 110. In this case, other devices with image acquisition capabilities can be used to acquire images and send the acquired images to the electronic device 100.

[0025] For example, the example electronic device used to implement the stain detection method and apparatus according to the embodiments of this application can be implemented on devices such as personal computers, terminal devices, time and attendance machines, panel displays, cameras, or remote servers. Terminal devices include, but are not limited to, tablet computers, mobile phones, PDAs (Personal Digital Assistants), touchscreen all-in-one machines, wearable devices, etc.

[0026] Below, we will refer to Figure 2 A method for detecting discoloration according to an embodiment of this application is described. Figure 2 A schematic flowchart of a stain detection method 200 according to an embodiment of this application is shown. Figure 2 As shown, the stain detection method 200 includes the following steps S210, S220, S230 and S240.

[0027] Step S210: Obtain the image to be processed, which contains the skin region to be tested.

[0028] For example, the image to be processed can be any type of image containing the skin region to be tested. The skin region to be tested can be a skin region at any location, including but not limited to the skin region of a face and / or the skin region on body parts other than the face. The image to be processed can be an image containing the skin region to be tested (which may be called the foreground) and other regions (which may be called the background), or it can be an image excluding the background. For example, if the skin region to be tested is the skin region of a face, the image to be processed can contain only the user's head, or it can be a half-body photo of the user, etc. The image to be processed can be the original image captured by the image acquisition device (such as the image acquisition device 110 described above), or an image obtained after preprocessing the original image captured by the image acquisition device. Preprocessing can include normalization, scaling, smoothing, etc. Preprocessing can also include the operation of extracting a portion of the image region containing the skin region to be tested from the original image captured by the image acquisition device to obtain the image to be processed.

[0029] The image to be processed can come from an external device and be transmitted to the electronic device 100 for color spot detection. Alternatively, the image to be processed can also be acquired by the electronic device 100 itself. For example, the electronic device 100 can use an image acquisition device 110 (e.g., a stand-alone camera) to acquire the image to be processed. The image acquisition device 110 can transmit the acquired image to the processor 102, where the processor 102 performs color spot detection.

[0030] Step S220: Convert the target region in the image to be processed to the LAB color space to obtain a LAB image; wherein, the target region may include at least a portion of the skin region to be tested on the image to be processed.

[0031] For example, at least a portion of the skin region to be tested can be all or part of the skin region to be tested in the image to be processed; however, it is preferable that at least a portion of the skin region to be tested is all the skin regions to be tested in the image to be processed. For example, when the skin region to be tested is a facial skin region, any suitable existing or future facial region recognition network model can be used end-to-end to determine the facial skin region or the facial region containing the face in the image to be processed. In the case of determining the facial region, the facial skin region can be determined from the facial region. For example, and not limitingly, the facial region recognition network model can be implemented using an image segmentation network. For example, the facial region recognition network model can be implemented using one or more of the following networks: Fully Convolutional Networks (FCN), U-shaped network (Unet), DeepLab series, V-shaped network (Vnet), etc. For example, keypoint detection can also be performed on the image to be processed to determine facial keypoints, and then facial regions or facial skin regions can be segmented based on the facial keypoints. For example, a facial landmark detection model can be used to detect landmarks on a face, and based on the results, at least a portion of the facial skin region (e.g., the cheek region) can be identified as at least a portion of the skin region to be tested in the image to be processed. Assuming that the at least portion of the skin region to be tested is the cheek region, when determining the cheek region, cheek boundary points can be determined based on the detected facial landmarks, and Bézier curves can be used to connect the various cheek boundary points to determine the contour of the cheek region, thus determining the location of the cheek region.

[0032] The image to be processed can be in RGB or BGR format and can be converted to LAB format. For example, the R, G, and B values ​​of each pixel in the RGB color space can first be converted to the CIE XYZ color space, and then the X, Y, and Z values ​​of each pixel in the CIE XYZ color space can be converted to the LAB color space. Exemplarily, but not limitingly, the cv.cvtColor and cv.inRange functions in the OpenCV library can be used to convert the pixel values ​​of each pixel in the target area to the LAB color space to obtain a LAB image. It can be understood that the LAB image includes pixels that correspond one-to-one with each pixel in the target area, and the L, A, and B components of each pixel in the target area are the L, A, and B components of the corresponding pixel in the LAB image. In one embodiment, the size of the LAB image can be the same as the size of the image to be processed, and the pixels in the LAB image can correspond one-to-one with the pixels in the image to be processed. The pixel values ​​of each pixel in the area of ​​the image to be processed other than the target area can be converted to the LAB color space as the pixel values ​​of the corresponding pixels in the LAB image. Alternatively, the pixel values ​​of pixels in regions of the image to be processed other than the target region may not be converted to the LAB color space. Instead, their pixel values ​​in the image to be processed or any default pixel values ​​may be used as the pixel values ​​of the corresponding pixels in the LAB image. In another embodiment, the size of the LAB image may not be the same as the size of the image to be processed. For example, the size of the LAB image may be smaller than the size of the image to be processed, and the LAB image may contain only the pixels corresponding to the target region. In either case, the LAB image may have pixels corresponding to the pixels in the image to be processed located within the target region.

[0033] Color spots are typically brown. After converting the image to the LAB color space, the visual component of brown can be constructed based on the physical meaning of the L, A, and B channels. In the RGB color space, the values ​​of R, G, and B are not visually consistent with the color presented by the pixel (for example, increasing the R value does not necessarily make the pixel redder), and the chromaticity distribution in the RGB color space is uneven (changing the same value in R, G, and B results in inconsistent perceived color changes). The LAB color space, however, separates brightness and color. The L channel represents brightness, the A channel represents the components from green to red, and the B channel represents the components from blue to yellow. The values ​​of A and B have strong visual consistency with the color presented by the pixel. Furthermore, the chromaticity in the LAB color space changes uniformly; that is, if the changes in L, A, and B are the same, the perceived color changes are also similar. Based on these characteristics, the LAB color space is very suitable for extracting components of a specific color. Therefore, to effectively determine the pigmentation status of the skin area to be tested, the image to be processed can be converted to the LAB color space, and a target brown component can be constructed based on the L, A, and B components. By obtaining the target brown component, the distribution of pigmentation in the skin area to be tested can be effectively captured, highlighting the shallow pigmentation that is not obvious in the original image, reflecting the depth of different pigmentation, and thus effectively determining the pigmentation status of at least part of the skin area to be tested.

[0034] Step S230: Based on the L component, A component and B component of each pixel in the LAB image, determine the target brown component of each pixel in the LAB image.

[0035] By way of example, and not limitation, for each pixel in a LAB image, the target brown component of that pixel can be obtained by linearly combining the L, A, and B components of that pixel according to the target weights.

[0036] Step S240: Based on the target brown component of each pixel in the LAB image, perform color spot detection on the target region to obtain the color spot detection result.

[0037] For example, based on the target brown component of each pixel in the determined LAB image, the color spot detection result within the target area can be obtained. For instance, based on the size and distribution of the target brown component, the region where the color spot is located can be determined. Furthermore, color spot information such as size, color intensity, and severity can be determined based on the color spot region. In addition, a brown area map can be generated based on the target brown component of each pixel in the LAB image. The color spot detection result can include the aforementioned color spot information and / or brown area map.

[0038] According to the pigmentation detection method provided in this application, a LAB image is obtained by converting the target region in the image to be processed to the LAB color space. A target brown component is determined based on the L, A, and B components of each pixel in the LAB image, and the pigmentation detection result of the target region is obtained based on this target brown component. In this scheme, since the colors presented by the L, A, and B components are highly consistent with human visual perception, the target brown component determined based on the L, A, and B components also has high consistency with the user's visual perception. This can accurately characterize the distribution of pigmentation in the skin area to be tested, which helps to accurately obtain the pigmentation detection result of the image to be processed.

[0039] For example, the stain detection method according to the embodiments of this application can be implemented in a device, apparatus or system having a memory and a processor.

[0040] The spot detection method according to the embodiments of this application can be deployed at the image acquisition end, for example, at a personal terminal or a server.

[0041] Alternatively, the stain detection method according to embodiments of this application can also be deployed distributedly on a server (or cloud) and at a personal terminal. For example, an image can be acquired on the client side, and the client can transmit the acquired image to the server (or cloud) side for stain detection.

[0042] For example, performing spot detection on a target region based on the target brown component of each pixel in the LAB image to obtain spot detection results may include: generating a brown area map corresponding to the target region based on the target brown component of each pixel in the LAB image, wherein the spot detection results may include a brown area map, which is used to characterize the spot situation in the target region, and the brown area map has target pixels that correspond one-to-one with each pixel in the LAB image, and the pixel value of any target pixel in the brown area map is determined based on the target brown component of the corresponding pixel in the LAB image; after performing spot detection on the target region based on the target brown component of each pixel in the LAB image to obtain spot detection results, the method may further include: displaying the brown area map on a display interface.

[0043] A brown area map can be used to indicate the distribution of brown within a target area, that is, to indicate the presence of color patches. In one embodiment, the size of the brown area map can be the same as the size of the LAB image, and their pixels can correspond one-to-one. Therefore, the brown area map can also include pixels that correspond one-to-one with pixels within the target area. For ease of description, the pixels in the brown area map that correspond to pixels in the LAB image are called target pixels. The pixel value of any (or every) target pixel in the brown area map can be determined based on the target brown component of the corresponding pixel in the LAB image. For example, the pixel value of the pixel at coordinates (10, 20) in the brown area map can be determined based on the target brown component of the pixel at coordinates (10, 20) in the LAB image. As described above, when the size of the LAB image is the same as the size of the image to be processed, for any (or every) pixel in any region other than the target region in the image to be processed, the target brown component of the pixel corresponding to that pixel in the LAB image can be set to a default pixel value (i.e., a default brown component value). Therefore, the pixel values ​​corresponding to the pixels in these regions can optionally be set to default pixel values ​​in the brown area image, for example, to the pixel values ​​corresponding to a specific color (e.g., white), so that these regions appear as a specific color in the brown area image, making it easier to highlight information such as the color depth of the color spots. Of course, optionally, the size of the brown area image can also be the same as the size of the target region involved in steps S220, S230, and S240, and each pixel in the brown area image can correspond one-to-one with each pixel in the target region.

[0044] After generating the brown area map, it can be displayed on the interface for users to easily view the pigmentation detection results. It can be understood that for the target brown component, the darker the corresponding brown, the more severe the pigmentation. Generating and displaying a brown area map based on the target brown component allows for a clear and intuitive presentation of the pigmentation status, including the color intensity of each spot on the tested skin area.

[0045] Based on the generated brown area map, users can clearly and intuitively understand the distribution of brown within the target area, and the intensity of the brown indicates the nature of the color spots. For example, the darker the brown, the darker the color of the spots. Using the above technical solution, users can easily and clearly understand the distribution and intensity of color spots.

[0046] For example, generating a brown area map corresponding to a target region based on the target brown component of each pixel in the LAB image may include: normalizing the target brown component of each pixel in the LAB image; performing a weighted summation operation to obtain the brown area map. The weighted summation operation may include: for each pixel in the LAB image, performing a weighted summation of the target white RGB value and the target brown RGB value based on the first weight and the second weight corresponding to that pixel to obtain the pixel value of the target pixel corresponding to that pixel in the brown area map; wherein, in the weighted summation, the weight of the target white RGB value is the first weight, and the weight of the target brown RGB value is the second weight; the first weight corresponding to any pixel is negatively correlated with the normalized target brown component of that pixel; and the second weight corresponding to any pixel is positively correlated with the normalized target brown component of that pixel.

[0047] In one embodiment, due to varying acquisition conditions such as light intensity, color saturation, and exposure in different image acquisition scenarios, the numerical distribution range corresponding to the target brown component calculated from the LAB image in different images to be processed is scattered. Therefore, the target brown component of each pixel in the LAB image can be normalized before generating the brown area map. Optionally, after obtaining the target brown component of each pixel in the LAB image, the formula can be used... These target brown components are normalized. Here, δ represents the normalized brown component corresponding to each pixel (e.g., the i-th pixel) in the LAB image, e represents the natural logarithm base, mean(brown) represents the average of the target brown components of each pixel in the LAB image, brown is the target brown component of each pixel (e.g., the i-th pixel) in the LAB image obtained in the previous embodiment, and "10." represents a floating-point number. Based on the normalization result, the values ​​of the target brown components can be mapped to the range [0,1].

[0048] It's understandable that if the target brown component of each pixel in the LAB image isn't normalized, the first and second weights corresponding to that pixel can be directly determined based on the target brown component. Then, the first weight is used as the weight for the target white RGB value, and the second weight is used as the weight for the target brown RGB value. A weighted sum of the target white and target brown RGB values ​​is then performed to obtain the pixel value of the target pixel corresponding to that pixel in the brown area image. Alternatively, if the target brown component of each pixel in the LAB image is normalized, the first and second weights corresponding to that pixel can be determined based on the normalized target brown component. Then, the first weight is used as the weight for the target white RGB value, and the second weight is used as the weight for the target brown RGB value. A weighted sum of the target white and target brown RGB values ​​is then performed to obtain the pixel value of the target pixel corresponding to that pixel in the brown area image.

[0049] For example, a weighted summation operation can be performed to obtain a brown area map. The weighted summation operation may include the following steps. In one embodiment, a first weight and a second weight can be determined based on the target brown component of any (or every) pixel. The first weight can be used as the weight of the target white RGB value, and the second weight can be used as the weight of the target brown RGB value. Both the target white RGB value and the target brown RGB value can be preset. For example, the target white RGB value can be the RGB value corresponding to regular white, for example, R = 255, G = 225, B = 255. The target brown RGB value can be set as needed; for example, a larger R value results in a darker brown, so the user can set the R value according to their needs to control the shade of brown. For example, the target brown RGB value could be R = 210, G = 180, B = 140, which corresponds to dark brown. For example, and not limitingly, Formula I can be used... brown =δ1*color brown +(1-δ1)*color white The target white RGB value and the target brown RGB value are weighted and summed to obtain the pixel value of the target pixel corresponding to any pixel on the brown area map. Where, color brown and color white These are the target brown RGB value and the target white RGB value, respectively, with δ1 representing the target brown component of the current pixel.

[0050] According to the above technical solution, by normalizing the obtained target brown component, the differences in the depth of color spots in different images to be processed can be quantitatively compared, thus obtaining a more accurate and reliable brown area map. In the brown area map obtained by the above method, the darker the color spot, the darker the brown color, while normal skin areas appear lighter, closer to white. This can intuitively show the shape, size, and color depth of the color spots.

[0051] For example, performing spot detection on a target region based on the target brown component of each pixel in the LAB image to obtain spot detection results may include: for any pixel in the LAB image, comparing the target brown component of that pixel with a target threshold; if the target brown component of that pixel is greater than the target threshold, then determining that the pixel corresponding to that pixel in the target region belongs to a spot; otherwise, determining that the pixel corresponding to that pixel in the target region does not belong to a spot. The spot detection results may include classification information about whether each pixel in the target region belongs to a spot.

[0052] In one embodiment, a target threshold can be preset. If the value of the target brown component is between [0,1], then the target threshold can be any value between 0 and 1, such as 0.4. If the value of the target brown component is between [0,255], then the target threshold can be any value between 0 and 255, such as 100. The target brown component of each pixel in the LAB image can be compared with the target threshold. Pixels with a target brown component greater than the target threshold are determined to belong to a color spot; otherwise, the pixel is determined not to belong to a color spot. As mentioned above, the LAB image has pixels that correspond one-to-one with the pixels in the target area. Therefore, in the above manner, it can be determined whether each pixel in the target area belongs to a color spot. The color spot detection result can include classification information about whether each pixel in the target area belongs to a color spot. The classification information can be represented in any form, for example, 'a' indicates that the pixel belongs to a color spot, and 'b' indicates that the pixel does not belong to a color spot.

[0053] The technical solution in this embodiment can be combined with the "generating a brown area map corresponding to the target region based on the target brown component of each pixel on the LAB image" in the previous embodiment to obtain the classification information and brown area map of the target region, or only one of the solutions can be selected for implementation.

[0054] According to the above technical solution, each pixel in the target region is filtered based on a target threshold to determine the color spot information within the target region. This method does not require complex calculations, thus it is highly efficient and easy to implement.

[0055] For example, determining the target brown component of each pixel in a LAB image based on the L component, A component, and B component of each pixel in the LAB image may include: for any pixel in the LAB image, determining the initial brown component of the pixel by linearly combining the L component, A component, and B component of the pixel, wherein the initial brown component is positively correlated with the A component and the B component and negatively correlated with the L component; wherein the target brown component of any pixel in the LAB image is the initial brown component of the pixel; or, determining the target brown component of each pixel in the LAB image based on the L component, A component, and B component of each pixel in the LAB image may further include: determining the target brown component of the pixel based on the initial brown component of any pixel in the LAB image.

[0056] As mentioned above, chromaticity in the LAB color space varies uniformly; that is, if the L, A, and B components change by the same magnitude, the perceived color change by the user will also be similar. Therefore, the target brown component constructed based on a linear combination (or linear superposition) of the L, A, and B components also exhibits a uniform chromaticity distribution, and its value effectively describes the lightness or darkness of the corresponding color (i.e., brown). For example, the initial brown component brown1 is positively correlated with the A and B components and negatively correlated with the L component. For instance, the initial brown component... Where k1, k2, and k3 are positive constants, and a, b, and l represent the A, B, and L components in the LAB color space, respectively. Based on the initial brown component of the pixel, it can be determined as the target brown component of the pixel.

[0057] In one example, the initial brown component of any (or every) pixel can be directly determined as its target brown component. In another example, the initial brown component of any (or every) pixel can be modified to obtain the target brown component.

[0058] According to the above technical solution, the initial brown component of a pixel is determined by linearly combining its L, A, and B components. The resulting target brown component exhibits a relatively uniform chromaticity distribution, and its value effectively describes the depth of brown.

[0059] For example, determining the target brown component of any pixel based on the initial brown component of any pixel in the LAB image may include: for any pixel in the LAB image, correcting the initial brown component of the pixel based on a correction coefficient to obtain the target brown component of the pixel, wherein the correction coefficient is positively correlated with the A and B components of the pixel and negatively correlated with the L component.

[0060] In one embodiment, due to the characteristics of the LAB color space, where the L component represents only luminance information and the A and B components represent color information, the visual effect of brown is enhanced when the values ​​of the A and B components are relatively larger than the L component. Therefore, the obtained initial brown component can be corrected to obtain the target brown component for that pixel. The correction coefficient is positively correlated with the A and B components of the pixel and negatively correlated with the L component. The obtained initial brown component brown1 can be multiplied by the correction coefficient to correct it.

[0061] According to the above technical solution, based on the correction coefficient, when the values ​​of components A and B are relatively larger than those of component L, the visual effect of brown can be enhanced. This can more accurately fit the visual effect of brown in different environments, which is conducive to more accurately determining the color spot detection results.

[0062] For example, the target brown component of any (which may be each) pixel satisfies the following formula:

[0063]

[0064] Where k1, k2, and k3 are positive constants, and a, b, and l represent the A, B, and L components in the LAB color space, respectively. Indicates the initial brown component. This represents the correction factor.

[0065] As described above, through Partially applicable to the initial brown component The correction is made to obtain the target brown component corresponding to that pixel. Therefore, based on this formula, the target brown component can be determined more accurately.

[0066] For example,

[0067] In one embodiment, the parameters can be set as follows:

[0068] Using the parameters in this embodiment, the target brown component is extracted from the skin region to be tested using the aforementioned formula for calculating brown. Exemplarily, the extracted target brown component can be presented as a grayscale image. Exemplarily, the grayscale image can be the same size as the image to be processed, and their pixels can correspond one-to-one. The pixel value of any pixel in the grayscale image can be equal to the target brown component of the corresponding pixel in the image to be processed. For regions other than the target region, the target brown component corresponding to these regions can be directly set to a default value, such as 0. Thus, in the grayscale image, regions other than the target region can be uniformly presented as black.

[0069] Experiments have shown that the above parameters have a wide range of applicability and the extracted target brown component is relatively accurate.

[0070] For example, before converting the target region in the image to be processed to the LAB color space to obtain a LAB image, the method may further include: performing key point detection on the image to be processed to obtain facial key points; and determining the target region based on the facial key points.

[0071] For example, the facial key points described herein may include one or more of the following: a set of eye key points, a set of nose key points, a set of mouth key points, a set of eyebrow key points, a set of facial midline key points, a set of ear key points, and a set of facial contour key points, etc. It is understood that the set of eye key points may include eye key points located on either side of the face, the set of eyebrow key points may include eyebrow key points located on either side of the face, and the set of ear key points may include ear key points located on either side of the face. For example, the set of eye key points may include the set of eye key points corresponding to the left eye and / or the set of eye key points corresponding to the right eye. For example, the set of eye key points on either side may include the set of key points on the outer rim of the eye and / or the set of key points on the inner rim of the eye. Those skilled in the art will understand that the set of key points on the outer rim of the eye may include key points on the outer rim contour of the eye, and the set of key points on the inner rim of the eye may include key points on the outer rim contour of the eye. The inner rim contour of the eye may be a contour line that generally surrounds the eyeball. The outer contour of the eye can be a line roughly surrounding the eyeball, inner canthus, and outer canthus. That is, the outer contour of the eye is larger than the inner contour, essentially enclosing it. Both contours reflect the position, shape, and size of the eye. The set of key points for facial contours can include the entire face contour set, or it can include the upper face contour set and / or the lower face contour set. The upper face contour set can primarily include key points along the contour line from one ear through the hairline to the other ear (upper face contour line), and the lower face contour set can primarily include key points along the contour line from one ear through the chin to the other ear (lower face contour line). The positions of the upper and lower face contour lines can be defined and differentiated as needed.

[0072] Any keypoint detection algorithm can be used to detect keypoints in the image to be processed. For example, Active Shape Model (ASM), Active Appearance Models (AAM), Cascaded Pose Regression (CPR), and deep learning-based methods can be used. Deep learning-based methods can be implemented using a facial keypoint detection model, which can be a neural network model. Exemplarily, but not limitingly, the aforementioned facial keypoint detection model can include one or more of the following: Task-Constrained Deep Convolutional Network (TCDCN), Deep Alignment Networks (DAN), Residual Networks (ResNet), Rep-VisualGeometry Group (RepVGG), etc. In embodiments using a facial keypoint detection model, the facial keypoint detection models used for any two facial parts (e.g., eyes and nose) can be the same neural network model or different neural network models. Facial key points detected by key point detection can be dense points, meaning their density can exceed a certain threshold.

[0073] Figure 3 A schematic diagram illustrating the facial landmark detection results according to one embodiment of this application is shown. Figure 3 As shown, key point detection can obtain the key points corresponding to the mouth, nose, eyes, eyebrows, facial contours, and facial midline in a face image. Figure 3 The set of facial contour key points shown is a set of key points for the entire face. For example, the facial key point detection results may include: 128 key points corresponding to the mouth, 128 key points corresponding to the nose, 128 key points corresponding to each of the left and right eyes (for example, for the left eye, this may include 63 key points contained in the inner circle of the left eye, 64 key points contained in the outer circle of the left eye, and 1 key point corresponding to the pupil of the left eye), 64 key points corresponding to each of the left and right eyebrows, 145 key points corresponding to the upper contour of the face (which can be represented by the hairline), 128 key points corresponding to the lower contour of the face, and 128 key points corresponding to the midline of the face.

[0074] For example, the target region may include a unilateral or bilateral cheek region and / or forehead region. For example, and not limitingly, the cheek region may be determined based on the locations of the obtained sets of eye keypoints, nose keypoints, and facial contour keypoints. The set of facial contour keypoints used to determine the cheek region may be the lower facial contour keypoint set or the full facial contour keypoint set. For example, and not limitingly, the forehead region may be determined based on the locations of the obtained sets of eyebrow keypoints and facial contour keypoints. The set of facial contour keypoints used to determine the forehead region may be the upper facial contour keypoint set or the full facial contour keypoint set. See reference. Figure 3 The following description uses the right side of the face as an example. The left side of the face can be determined in the same way, defining the cheek area of ​​the left cheek. The location of the key points around the right eye, the right nose, and the lower contour of the right face determines the boundary points of the right cheek. That is, the right cheek boundary points are contained within the area enclosed by the key points of the right eye, the right nostril, and the lower contour of the right face. Based on these boundary points, Bézier curves can be used to connect them, thus defining the right cheek area. Similarly, the left cheek area, forehead area, etc., can be determined in the same way. See also... Figure 3 This shows the cheek areas 310 and the forehead area 320 on both sides. Note that... Figure 3 The shapes of the various regions and the arrangement of key points shown are just examples; the shapes of the regions and the arrangement of key points can be set as needed.

[0075] For example, determining the cheek region in an image to be processed may include: performing keypoint detection on the image to be processed to obtain facial keypoints, including a set of eye keypoints, a set of nose keypoints, and a set of lower facial contour keypoints; selecting a first set of eye keypoints and a second set of eye keypoints from the set of eye keypoints, wherein the first set of eye keypoints may include at least one eye keypoint located on the lower eyelid and whose distance from the inner corner of the eye keypoint is less than or equal to a first distance threshold, and the second set of eye keypoints includes at least one eye keypoint located on the lower eyelid and whose distance from the outer corner of the eye keypoint is less than or equal to a second distance threshold, wherein the inner corner of the eye keypoint and the outer corner of the eye keypoint are eye... The set of key points for the eyes includes key points located at the inner and outer corners of the eyes. Each key point in the selected set of key points is moved to produce a first target displacement, and the moved key points are defined as the first set of key points for the cheeks. The angle between the first target displacement and a first direction is less than 90 degrees, and the first direction is downward along the midline of the face in the image to be processed. A first set of key points and a second set of key points for the nose are selected from the set of key points for the nose. The first set of key points for the nose includes at least one key point located on a first side and whose distance from the key point on the top of the nose is less than or equal to a third distance threshold. The second set of key points for the nose may include at least one key point. Nose keypoints located on the first side and whose distance from the nasal wing keypoint is less than or equal to the fourth distance threshold; when the eye keypoint set belongs to the left eye, the first side is the left side of the nose; when the eye keypoint set belongs to the right eye, the first side is the right side of the nose; the nasal roof keypoint is the nose keypoint located at the top of the first side; the nasal wing keypoint is the nose keypoint located on the first side and on the nasal wing; move each of the selected nose keypoints to produce a second target displacement, and determine the moved nose keypoints as the second cheek keypoint set, wherein the angle between the second target displacement and the second direction is less than 90 degrees, and the second direction is the direction perpendicular to the first direction and towards the second side; according to the target A subset of lower facial contour keypoints is selected from the set of lower facial contour keypoints at intervals. For each selected lower facial contour keypoint, interpolation is performed between the target point and the lower facial contour keypoint based on the distance between the target point and the target point to obtain a third set of cheek keypoints. The distance between the target point and the midline of the face in the image to be processed is less than or equal to a fifth distance threshold. The first set of cheek keypoints, the second set of cheek keypoints, and the third set of cheek keypoints are merged together to obtain a total set of cheek keypoints. A cheek boundary line is generated based on each keypoint in the total set of cheek keypoints, and the area within the cheek boundary line is defined as the cheek region.

[0076] Through keypoint detection in the previous embodiments, sets of keypoints for the eyes, nose, and facial contours can be obtained. Based on these keypoints, the cheek region can be determined. See again. Figure 3 The image shows a cheek region 310. Exemplarily, the cheek region may avoid the dividing lines of the facial features and be appropriately recessed inwards relative to the outer contour of the face, a scheme described below.

[0077] When detecting the left cheek region, the set of eye key points used in this embodiment can be either the set of key points around the outer rim of the left eye or the set of key points around the inner rim of the left eye. Conversely, when detecting the right cheek region, the set of eye key points used in this embodiment can be either the set of key points around the outer rim of the right eye or the set of key points around the inner rim of the right eye. The following describes an exemplary method for determining the cheek region using the left cheek region as an example; the right cheek region can be determined in the same way. For example, the detected set of eye key points is the set of key points around the outer rim of the left eye, which may include the inner and outer corner key points of the left eye. The inner and outer corner key points of the left eye are located at the inner and outer corners of the left eye, respectively. That is, the inner corner key point of the left eye can be the rightmost (i.e., closest to the midline of the face) key point on the left eye. The outer corner key point of the left eye can be the leftmost (i.e., furthest from the midline of the face) key point on the left eye.

[0078] A first set of eye keypoints and a second set of eye keypoints can be selected from the set of keypoints for the left eye. Each set of eye keypoints can include one or more eye keypoints. For the first set of eye keypoints, at least one eye keypoint is located on the lower eyelid of the left eye and its distance from the inner corner of the left eye keypoint is less than or equal to a first distance threshold. The first distance threshold can be any suitable value, which can be set as needed. For example, the first distance threshold can be in the range of [0, 3] mm, such as equal to 2 mm. That is, the keypoint in the first set of eye keypoints is a keypoint on the lower eyelid near the inner corner of the eye. For the second set of eye keypoints, at least one eye keypoint is located on the lower eyelid and its distance from the outer corner of the eye keypoint is less than or equal to a second distance threshold. The second distance threshold can also be any suitable value. For example, the second distance threshold can be in the range of [0, 3] mm, such as equal to 2 mm. That is, the keypoint in the second set of eye keypoints is a keypoint on the lower eyelid near the outer corner of the eye. It is understood that the first distance threshold and the second distance threshold can be the same or different.

[0079] In one embodiment, the first set of eye keypoints may include the inner corner of the eye keypoint, and the second set of eye keypoints may include the outer corner of the eye keypoint. This will be illustrated below using this as an example. The first and second sets of eye keypoints can be moved. For example, the selected inner and outer corner of the eye keypoints can be moved to produce a first target displacement. The direction of the first target displacement can be parallel to the midline of the face and downwards, or an acute angle between the first target displacement and the downward direction along the midline of the face. It should be noted that during the movement, the first target displacement corresponding to any two different eye keypoints can be the same or different. That is, the selected eye keypoints move downwards overall, but the direction and / or distance of movement can be equal or unequal. It is understood that the directional terms such as "up," "down," "left," and "right" described herein are based on the definition of a face. For example, the midline of the face can be determined based on a set of midline keypoints or based on any other method. The moved inner and outer corner of the eye keypoints can be determined as the first set of cheek keypoints.

[0080] Before selecting the first and second sets of nose keypoints from the set of nose keypoints, it is known that the nasal roof keypoint is the topmost nose keypoint on the first side. The nasal ala keypoint is the nose keypoint located on the first side and on the nasal ala. For example, if the eye keypoint set belongs to the left eye, then the first side is the left side of the nose; if the eye keypoint set belongs to the right eye, then the first side is the right side of the nose. For example, when detecting the left cheek region, the nasal roof keypoint can be the topmost keypoint on the left, and the nasal ala keypoint can be the keypoint on the left furthest from the midline of the face. When detecting the right cheek region, the nasal roof keypoint can be the topmost keypoint on the right, and the nasal ala keypoint can be the keypoint on the right furthest from the midline of the face. The first and second sets of nose keypoints can each contain one or more keypoints.

[0081] For the first group of nasal keypoints, it may include at least one nasal keypoint located on the first side and whose distance from the nasal roof keypoint is less than or equal to a third distance threshold. The third distance threshold can be any suitable value, which can be set as needed. For example, the third distance threshold can be in the range of [0, 3] mm, such as equal to 2 mm. For the second group of nasal keypoints, it may include at least one nasal keypoint located on the first side and whose distance from the alar keypoint is less than or equal to a fourth distance threshold. The third and fourth distance thresholds can be any suitable values, which can be set as needed. For example, the third and fourth distance thresholds can each be in the range of [0, 3] mm, such as equal to 2 mm. Referring to the above description, it can be seen that the first group of nasal keypoints are keypoints close to the nasal roof, and the second group of nasal keypoints are keypoints close to the alar.

[0082] Similar to eye keypoints, a second set of cheek keypoints can be obtained by moving selected nose keypoints. For example, each nose keypoint in the selected set of nose keypoints can be moved to produce a second target displacement, and the moved nose keypoints are defined as the second set of cheek keypoints. The angle between the second target displacement and a second direction is less than 90 degrees, and the second direction can represent a direction perpendicular to the first direction and pointing towards a second side. That is, if the eye keypoint set belongs to the left eye, then the second side is the left side of the face. If the eye keypoint set belongs to the right eye, then the second side is the right side of the face. For example, for the left cheek region, nose keypoints selected from the left nose keypoints can be moved by a second target displacement towards the left side of the face. In one embodiment, the first set of nose keypoints and the second set of nose keypoints each include one keypoint, such as the nasal roof keypoint and the nasal ala keypoint, which can be moved by a second target displacement towards the left side of the face. The second target displacements corresponding to any two different nose keypoints can be the same or different. In other words, the selected key points of the nose are generally moved toward the second side, but the direction and / or distance of movement may be equal or unequal.

[0083] As mentioned above, the cheek area can avoid the dividing lines of facial features and be appropriately recessed inwards compared to the outer contour of the face. This avoids color distortion in the outer contour area of ​​the face due to its irregular shape. This solution is described below.

[0084] In one embodiment, for multiple key points on the lower contour of the face, the user can pre-set a target interval to select a subset of key points from the key points on the lower contour of the face as initial cheek outer boundary points according to the target interval. The target interval can be any value greater than 0, such as an interval of 5 key points, 8 key points, 10 key points, etc. The user can also select any key point on the face as the target point, which can be any point such as the center point of both eyes, the center point of both eyebrows, etc. For example, the target point can be the center point of the line connecting the pupils of both eyes, or the center point of the line connecting the two inner corners of the eyes, or the intersection of any of the above connecting lines with the midline of the face. The distance between the target point and the midline of the face is less than or equal to a fifth distance threshold. The fifth distance threshold can be set to any suitable value as needed, for example, the fifth distance threshold can be in the range of [0,3] mm, for example, equal to 2 mm. For any (or each) initial cheek outer boundary point among multiple initial cheek outer boundary points, interpolation is performed between the target point and the initial cheek outer boundary point according to the target ratio based on the distance between the target point and the initial cheek outer boundary point. By way of example, and not limitation, the target ratio can be the ratio of the distance between the target point and the new cheek outer boundary point to the distance between the initial cheek outer boundary point and the target point. The target ratio can range from [0, 1], for example, 0.9. That is, the ratio of the distance between the interpolated new cheek outer boundary point and the target point to the distance between the initial cheek outer boundary point and the target point is 0.9. Similarly, interpolation can be performed between the target point and each initial cheek outer boundary point to obtain new cheek outer boundary points. Finally, the determined new cheek outer boundary points can be used as the third set of cheek keypoints.

[0085] The obtained sets of first, second, and third cheek keypoints are merged together to form the overall cheek keypoint set. Optionally, the keypoints in the overall cheek keypoint set can be directly connected or connected using Bézier curves to generate the cheek boundary line. Alternatively, the overall cheek keypoint set can be interpolated before directly connecting the interpolated keypoints or connecting them using Bézier curves to obtain the cheek boundary line. The area within the cheek boundary line can be defined as the cheek region.

[0086] Any two of the aforementioned first, second, third, fourth, and fifth distance thresholds can be the same or different. It is understood that the first to fifth distance thresholds in the preceding embodiments can be adjusted based on facial differences, and this application does not impose any restrictions on this.

[0087] According to the above technical solution, facial key points can be obtained by performing key point detection on the image to be processed, and then the cheek region can be determined based on these key points. This method can avoid the inaccuracy of the obtained cheek region due to facial differences, ensuring the effectiveness of the obtained cheek region and improving the efficiency of pigmentation detection. Furthermore, the cheek outer boundary points obtained through interpolation are more accurate and can avoid errors caused by facial differences, further ensuring the effectiveness of the pigmentation detection results.

[0088] For example, before moving each of the selected eye keypoints to produce a first target displacement, the method may further include: determining a first reference distance based on the positional difference between the inner corner eye keypoint and the outer corner eye keypoint; or, if the face keypoints also include a set of face midline keypoints, selecting two face midline keypoints from the set of face midline keypoints and determining a first reference distance based on the positional difference between the two face midline keypoints; determining a first target displacement based on the first reference distance, wherein the magnitude of the first target displacement is proportional to the first reference distance by a first target.

[0089] Before moving each of the selected eye keypoints to produce a first target displacement, a first reference distance can be determined to control the eye keypoints to produce appropriate displacement based on this distance. The first reference distance can be determined based on any two or more detected facial keypoints. For example, the positional difference between the inner and outer corner keypoints can be calculated, and the first reference distance can be determined based on this positional difference. The first target displacement can then be set according to a first target ratio proportional to the first reference distance. The first target ratio can be set to any suitable value as needed. For example, the first target ratio can fall within the range of (0, 0.2) for the positional difference between the inner corner of the eye and the outer corner of the eye. For instance, the magnitude of the first target displacement can be equal to 0.2 times the first reference distance, and the direction of the first target displacement can be set according to the direction in which the eye key points are desired to move. Furthermore, if the acquired facial key points include facial midline key points, then any two facial midline key points can be selected from the set of facial midline key points, such as any two adjacent facial midline key points, and the positional difference between these two key points can be determined as the first reference distance. For example, the first target ratio can fall within the range of (0, 5) for the positional difference between two adjacent facial midline key points.

[0090] Using the above method, a first reference distance suitable for each individual can be determined, thereby determining the first target displacement. Since different people typically have different facial shapes, this method can determine a first target displacement that is suitable for each individual, making the obtained first target displacement more targeted, which can effectively improve the accuracy of cheek area detection.

[0091] For example, before moving each of the selected nose keypoints to produce a second target displacement, the method may further include: calculating the positional difference between the inner corner eye keypoint and the outer corner eye keypoint to obtain a second reference distance; or, if the facial keypoints also include a set of facial midline keypoints, selecting two facial midline keypoints from the set of facial midline keypoints and calculating the distance between the two facial midline keypoints to obtain a second reference distance; determining a second target displacement based on the second reference distance, wherein the second target displacement is proportional to the second reference distance as a second target.

[0092] Before moving at least one of the selected nasal keypoints to produce a second target displacement, a second reference distance can also be determined to control the appropriate displacement of the nasal keypoints based on this distance. The method for obtaining the second reference distance is similar to that for obtaining the first reference distance, and will not be repeated here for simplicity. The magnitude of the second target displacement is determined by multiplying the obtained second reference distance by a second target ratio. The second target ratio can be set to any suitable value as needed. For example, for the positional difference between the inner and outer corners of the eye, the second target ratio can fall within the range of (0, 0.2). For example, the magnitude of the second target displacement can be equal to 0.2 times the second reference distance. Furthermore, for the positional difference between two adjacent facial midline keypoints, the second target ratio can fall within the range of (0, 5).

[0093] Using the above method, a second reference distance suitable for each individual can be determined, thereby determining the second target displacement. Since different people typically have different facial shapes, this method can determine a second target displacement that is suitable for each individual, making the obtained second target displacement more targeted, which can effectively improve the accuracy of cheek region detection.

[0094] The method for determining the forehead region is similar to that for the cheek region, except for the key points used and the direction of key point movement. For example, when determining the forehead region, at least some key points from the eyebrow key point sets on both sides can be moved upwards to obtain a first forehead key point set, and at least some key points from the upper facial contour key point set can be moved downwards to obtain a second forehead key point set. Subsequently, the first and second forehead key point sets can be merged into a total forehead key point set. Based on the key points in the total forehead key point set, a forehead boundary line is generated, and the area within the forehead boundary line is defined as the forehead region. Those skilled in the art can understand the method for determining the forehead region by referring to the above method for determining the cheek region; it will not be elaborated upon here.

[0095] In the embodiment of keypoint detection using the aforementioned facial keypoint detection model, the model can be trained using a training dataset. The training dataset may include multiple sample face images and ground truth information corresponding one-to-one with each of the sample face images. The ground truth information may include the location information of facial keypoints. The facial keypoints in the ground truth information may include one or more of the following keypoints: eye keypoints, nose keypoints, mouth keypoints, eyebrow keypoints, facial contour keypoints, facial midline keypoints, and ear keypoints. For example, the facial keypoints in the ground truth information may include: multiple keypoints corresponding to the eye area (e.g., 64 keypoints annotated around each of the outer edges of the left and right eyes), keypoints corresponding to the nose (e.g., 126 keypoints annotated on the outer contour of the nose), and keypoints corresponding to the upper and lower contours of the face (e.g., 128 keypoints annotated on the lower contour and 145 keypoints annotated on the upper contour). It is understood that the number of keypoints annotated for each part of the face in this embodiment is merely exemplary, and the number of annotated keypoints can be arbitrary. Multiple sample face images are input into an initial facial landmark detection model to obtain corresponding landmark prediction results. This initial model has the same network structure as the model used in the actual landmark detection operation, but the parameters may differ. After training the parameters of the initial model, the resulting model is the one used in the actual landmark detection operation. The landmark prediction results and the annotation information of the multiple sample face images are substituted into a preset loss function to calculate the predicted loss value. Then, based on the predicted loss value, the parameters of the initial model are optimized using backpropagation and gradient descent algorithms. Parameter optimization can be iteratively performed until the model converges. After training, the obtained model can be used for subsequent landmark detection; this stage can be called the model inference stage. Of course, when different models are used to predict landmarks in different areas, each model can be trained based on sample face images annotated with landmarks for the corresponding areas. Those skilled in the art will understand the implementation method, which will not be elaborated further.

[0096] According to the above technical solution, by performing key point detection on the image to be processed, facial key points can be obtained, and then the facial skin region can be determined based on the facial key points. This method can avoid the inaccuracy of the target region obtained due to the differences in faces, and can ensure the effectiveness of the obtained target region, thereby improving the accuracy of pigmentation detection.

[0097] For example, the method may further include setting the target brown component of the region in the image to be processed, excluding the target region, to a default value.

[0098] In one embodiment, the target brown component of the region outside the identified target area in the image to be processed can be set to a default value. This default value can be any value, such as 0. This eliminates interference from the background and locations such as facial features, avoiding their influence on the color spot detection results, while increasing the contrast with the target area, allowing users to intuitively observe the color spot detection results.

[0099] For example, the spot detection result includes the size of the spot. The spot detection is performed on the target region based on the target brown component of each pixel on the LAB image to obtain the spot detection result, including: determining the spot region where the spot is located in the target region based on the target brown component of each pixel on the LAB image; calculating the area and / or perimeter of any spot region as the size of the spot contained in the spot region.

[0100] For example, for any (or every) pixel within a LAB image, it can be determined whether the pixel belongs to a color patch based on the target brown component of that pixel, as can be understood with reference to the embodiments described above. After determining whether each pixel belongs to a color patch, adjacent pixels belonging to the same color patch within the target area can be grouped together as the same color patch, thereby determining the color patch region where each color patch is located. Subsequently, the area and / or perimeter of any color patch region can be calculated, and the calculated area and / or perimeter can be regarded as the size of the color patch contained in that color patch region.

[0101] This method allows for a simple and quick determination of the size of the pigmentation.

[0102] For example, the spot detection result includes information on the color depth of the spot. The spot detection is performed on the target area based on the target brown component of each pixel in the LAB image to obtain the spot detection result, including: determining the spot area where the spot is located in the target area based on the target brown component of each pixel in the LAB image; and calculating the average value of the target brown component of each pixel in any spot area as the color depth information of the spot contained in the spot area.

[0103] The method for determining the color spot region can be referred to the description above, and will not be repeated here. For any color spot region, the average value of the target brown component of the pixels contained therein can be used as the color depth information of the color spot contained in that region. That is, the average value of the target brown component is used to characterize the color depth of the current color spot. The larger the average value, the darker the brown, and the darker the color of the spot. This method can intuitively represent the color depth information of the color spot.

[0104] For example, the spot detection result includes the severity of the spot. Spot detection is performed on the target region based on the target brown component of each pixel in the LAB image to obtain the spot detection result, including: determining the spot region where the spot is located in the target region based on the target brown component of each pixel in the LAB image; calculating the area and / or perimeter of any spot region as the size of the spot contained in the spot region, and / or calculating the average value of the target brown component of each pixel in any spot region as the color depth information of the spot contained in the spot region; determining the severity of the spot contained in the spot region based on the size of the spot contained in any spot region and / or the color depth information of the spot contained in the spot region.

[0105] The method for determining the blemish area can be referred to the description above, and will not be repeated here. In one embodiment, the severity of the blemish can be determined based on the size information or the color intensity information of the blemish area, or it can be determined based on both the size information and the color intensity information of the blemish area. For example, threshold ranges can be set for the size information and the color intensity information respectively, where different threshold ranges correspond to different blemish severity. Taking the size of the blemish as an example, where the value range of the blemish size is [0,1], a first threshold range of [0,0.2] can be set to represent no blemish, a second threshold range of [0.2,0.5] to represent mild blemish, a third threshold range of [0.5,0.7] to represent moderate blemish, and a fourth threshold range of [0.7,1] to represent severe blemish. It should be understood that the above-mentioned threshold range setting method is merely exemplary, and this application does not impose any limitations on it. For example, in another embodiment, the threshold range for no spots can be [0, 0.1], the threshold range for mild spots can be [0.1, 0.3], the threshold range for moderate spots can be [0.3, 0.6], and the threshold range for severe spots can be [0.6, 1]. Similarly, threshold ranges can be set for the color intensity of spots to distinguish their severity; for simplicity, this will not be elaborated further here.

[0106] According to the above technical solution, the severity of pigmentation is determined based on the size and / or color intensity of the pigmentation. This method can intuitively and accurately determine the severity of pigmentation.

[0107] According to another aspect of this application, a stain detection device is provided. Figure 4 A schematic block diagram of a stain detection device 400 according to one embodiment of this application is shown.

[0108] like Figure 4As shown, the stain detection device 400 according to an embodiment of this application includes an acquisition module 410, a conversion module 420, a determination module 430, and an acquisition module 440. Each module can respectively perform the functions described above. Figure 2 The steps of the stain detection method are described below. Only the main functions of each component of the stain detection device 400 are described below, omitting the details already described above.

[0109] The acquisition module 410 is used to acquire the image to be processed, which contains the skin region to be tested. The acquisition module 410 can be... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0110] The conversion module 420 is used to convert the target region in the image to be processed to the LAB color space to obtain a LAB image; wherein, the target region includes at least a portion of the skin region to be tested on the image to be processed. The conversion module 420 can be... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0111] The determination module 430 is used to determine the target brown component of each pixel in the LAB image based on the L component, A component, and B component of each pixel in the LAB image. The determination module 430 can be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0112] The acquisition module 440 is used to perform color spot detection on the target region based on the target brown component of each pixel in the LAB image, and obtain the color spot detection result. The acquisition module 440 can be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0113] Figure 5 A schematic block diagram of an electronic device 500 according to an embodiment of this application is shown. The electronic device 500 includes a memory 510 and a processor 520.

[0114] The memory 510 stores computer program instructions for implementing the corresponding steps in the stain detection method according to the embodiments of this application.

[0115] The processor 520 is used to run computer program instructions stored in the memory 510 to perform corresponding steps of the stain detection method according to the embodiments of this application.

[0116] In one embodiment, computer program instructions executed by processor 520 are used to perform the following steps: acquiring an image to be processed, the image to be processed containing a skin region to be tested; converting the target region in the image to be processed to the LAB color space to obtain a LAB image; wherein the target region includes at least a portion of the skin region to be tested on the image to be processed; determining the target brown component of each pixel on the LAB image based on the L component, A component and B component of each pixel on the LAB image; and performing spot detection on the target region based on the target brown component of each pixel on the LAB image to obtain spot detection results.

[0117] For example, the electronic device 500 may also include an image acquisition device 530. The image acquisition device 530 is used to acquire an image to be processed. The image acquisition device 530 is optional, and the electronic device 500 may also exclude the image acquisition device 530. In this case, the processor 520 may acquire the image to be processed by other means, such as from an external device or from the memory 510.

[0118] Furthermore, according to embodiments of this application, a storage medium is also provided, on which program instructions are stored. When the program instructions are run by a computer or processor, they are used to execute corresponding steps of the stain detection method of this application embodiment and to implement corresponding modules in the stain detection device according to embodiments of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0119] In one embodiment, when the program instructions are executed by a computer or processor, the computer or processor may implement the various functional modules of the stain detection device according to the embodiments of this application, and / or may execute the stain detection method according to the embodiments of this application.

[0120] In one embodiment, the program instructions are used to perform the following steps at runtime: acquiring an image to be processed, the image to be processed containing a skin region to be tested; converting the target region in the image to be processed to the LAB color space to obtain a LAB image; wherein the target region includes at least a portion of the skin region to be tested on the image to be processed; determining the target brown component of each pixel in the LAB image based on the L component, A component, and B component of each pixel in the LAB image; and performing spot detection on the target region based on the target brown component of each pixel in the LAB image to obtain spot detection results.

[0121] Furthermore, according to an embodiment of this application, a computer program product is also provided, which includes a computer program that, when running, performs the above-described stain detection method 200.

[0122] Each module in the electronic device according to the embodiments of this application can be implemented by the processor of the electronic device implementing spot detection according to the embodiments of this application running computer program instructions stored in the memory, or by computer instructions stored in a computer-readable storage medium of the computer program product according to the embodiments of this application being executed by a computer.

[0123] Furthermore, according to an embodiment of this application, a computer program is also provided, which, when running, is used to execute the above-described stain detection method 200.

[0124] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0125] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0127] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0128] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more aspects of the various applications, features of this application are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the inventive point lies in solving the corresponding technical problem with fewer features than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0129] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0130] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0131] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the stain detection device according to the embodiments of this application. This application can also be implemented as a device program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0132] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0133] The above are merely specific embodiments or descriptions of specific embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A method for detecting pigmentation spots, comprising: Acquire an image to be processed, the image containing the skin region to be tested; The target region in the image to be processed is converted to the LAB color space to obtain a LAB image; wherein, the target region includes at least a portion of the skin region to be tested in the image to be processed; Based on the L component, A component and B component of each pixel in the LAB image, the target brown component of each pixel in the LAB image is determined, and the target brown component is used to reflect the depth of the color spot. Based on the target brown component of each pixel in the LAB image, color spot detection is performed on the target region to obtain color spot detection results; The step of determining the target brown component of each pixel in the LAB image based on the L component, A component, and B component of each pixel in the LAB image includes: For any pixel in the LAB image, the initial brown component of the pixel is determined by linearly combining the L component, A component and B component of the pixel, wherein the initial brown component is positively correlated with the A component and B component and negatively correlated with the L component. The target brown component of any pixel is determined based on the initial brown component of that pixel in the LAB image; The step of determining the target brown component of any pixel based on its initial brown component in the LAB image includes: For any pixel in the LAB image, the initial brown component of the pixel is corrected based on a correction coefficient to obtain the target brown component of the pixel, wherein the correction coefficient is positively correlated with the A and B components of the pixel and negatively correlated with the L component.

2. The method as described in claim 1, wherein, The step of detecting color spots in the target region based on the target brown component of each pixel in the LAB image to obtain color spot detection results includes: A brown area map corresponding to the target region is generated based on the target brown component of each pixel in the LAB image. The color spot detection result includes the brown area map, which is used to characterize the color spot situation in the target region. The brown area map has target pixels that correspond one-to-one with each pixel in the LAB image. The pixel value of any target pixel in the brown area map is determined based on the target brown component of the corresponding pixel in the LAB image. After performing spot detection on the target region based on the target brown component of each pixel in the LAB image to obtain the spot detection result, the method further includes: The brown area map is displayed on the display interface.

3. The method as described in claim 2, wherein, The step of generating a brown area map corresponding to the target region based on the target brown component of each pixel in the LAB image includes: Normalize the target brown component of each pixel in the LAB image; A weighted summation operation is performed to obtain the brown area image. The weighted summation operation includes: for each pixel in the LAB image, based on the first weight and the second weight corresponding to the pixel, a weighted summation is performed on the target white RGB value and the target brown RGB value to obtain the pixel value of the target pixel corresponding to the pixel in the brown area image; wherein, during the weighted summation, the weight of the target white RGB value is the first weight, and the weight of the target brown RGB value is the second weight; the first weight corresponding to any pixel is negatively correlated with the normalized target brown component of the pixel; the second weight corresponding to any pixel is positively correlated with the normalized target brown component of the pixel.

4. The method according to any one of claims 1-3, wherein, The step of detecting color spots in the target region based on the target brown component of each pixel in the LAB image to obtain color spot detection results includes: For any pixel in the LAB image, the target brown component of the pixel is compared with a target threshold. If the target brown component of the pixel is greater than the target threshold, the pixel corresponding to the pixel in the target area is determined to be a color spot; otherwise, the pixel corresponding to the pixel in the target area is determined not to be a color spot. The color spot detection result includes classification information on whether each pixel in the target area belongs to a color spot.

5. The method according to any one of claims 1-3, wherein, The target brown component of any pixel satisfies the following formula: ； Where k1, k2, and k3 are positive constants, and a, b, and l represent the A, B, and L components in the LAB color space, respectively. This represents the initial brown component. This represents the correction coefficient.

6. The method of claim 5, wherein, ， ， ， 。 7. The method as described in any one of claims 1-3, wherein, The color spot detection result includes the size of the color spot. The color spot detection of the target region based on the target brown component of each pixel on the LAB image to obtain the color spot detection result includes: The color patch region within the target area is determined based on the target brown component of each pixel in the LAB image; Calculate the area and / or perimeter of any color spot region as the size of the color spot contained in that region; And / or, The spot detection result includes information on the color depth of the spots. The step of detecting spots in the target region based on the target brown component of each pixel in the LAB image to obtain the spot detection result includes: The color patch region within the target area is determined based on the target brown component of each pixel in the LAB image; Calculate the average value of the target brown component of each pixel in any color spot region as the color depth information of the color spot contained in that color spot region; And / or, The stain detection result includes the severity of the stain. The stain detection is performed on the target region based on the target brown component of each pixel in the LAB image to obtain the stain detection result, including: The color patch region within the target area is determined based on the target brown component of each pixel in the LAB image; Calculate the area and / or perimeter of any color spot region as the size of the color spot contained in the color spot region, and / or calculate the average value of the target brown component of each pixel in any color spot region as the color depth information of the color spot contained in the color spot region. The severity of the pigmentation in any pigmented region is determined based on the size of the pigmentation and / or the color intensity of the pigmentation contained in that region.

8. An electronic device comprising a processor and a memory, wherein, The memory stores computer program instructions, which, when executed by the processor, are used to perform the stain detection method as described in any one of claims 1 to 7.

9. A storage medium on which program instructions are stored, wherein, The program instructions, when executed, are used to perform the stain detection method as described in any one of claims 1 to 7.

10. A computer program product, the computer program product comprising a computer program, wherein, The computer program, when running, is used to perform the stain detection method as described in any one of claims 1 to 7.

Images