Apparatus and method for determining skin condition or skin type by using artificial neural network, and recording medium having instructions recorded thereon
By building an artificial neural network, combining user skin images and survey answers, the individual differences and accuracy of skin condition assessment in the prior art are solved, and the accurate assessment of user skin condition and type is achieved, and the consistency of the assessment is improved. Convenience.
Patent Information
- Application Number
- CN202380026880.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-14
- Filing Date
- 2023-03-13
- Publication Date
- 2024-11-19
AI Technical Summary
Existing methods have problems with personality differences and difficulty in assessing certain skin conditions (such as contact dermatitis) when determining the user's skin condition or skin type, and traditional methods have difficulty achieving consistent and accurate assessment.
Using an artificial intelligence technology device, by building an artificial neural network, using the user's skin image and the correlation model of investigating problems, determine the skin condition or type of the target user, and combine user answers and expert evaluation to improve the accuracy and consistency of the evaluation. sex.
Accurate and reliable assessment of user skin status and type is achieved, reducing the number of user answers, and improving the convenience and interpretability of evaluation.
Smart Images

Figure CN118984673A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to technology for determining a user's skin condition or skin type based on artificial intelligence technology. Background Art
[0002] In a previous method for determining a user's skin condition or skin type, the skin condition may be determined by the user directly answering a survey or questionnaire. However, in this method, there is such a difference depending on the individual's personality: the individual directly determines his / her own condition; and there is such a problem that the skin condition cannot be logically evaluated due to such a difference. In addition, conventionally, the skin condition may be determined by an expert who examines the user's skin condition based on the user's image. The problem with this method is that skin conditions (e.g., contact dermatitis, etc.) that are difficult to determine by an image cannot be evaluated. Therefore, there is an increasing need in the art for a method of more consistently and accurately determining skin conditions. Summary of the invention
[0003] The present disclosure provides a technology for determining a user's skin condition or skin type based on artificial intelligence technology.
[0004] According to aspects of the present disclosure, a device for determining a user's skin condition or skin type based on artificial intelligence technology can be provided. The device according to the present disclosure may include one or more processors; and one or more memories, the one or more memories being configured to store instructions and artificial neural networks, which, when executed by the processors, cause the one or more processors to perform calculations, the artificial neural network being constructed by modeling the correlation between multiple images of the skin of multiple users and scores of multiple survey questions included in a survey about skin provided to multiple users, wherein the one or more processors can be configured to perform the following operations according to the instructions: determine a first survey score group for a survey about the skin of a target user; input an image of the skin of the target user into the artificial neural network; determine a second survey score group for the survey based on the output of the artificial neural network; and determine a score for each of one or more indicators indicating the skin type of the target user or the skin condition of the target user based on the first survey score group and the second survey score group.
[0005] In one embodiment, the survey may include a group of questions for at least one target to be determined, and the group of questions for at least one target to be determined may correspond to one or more indicators, respectively.
[0006] In one embodiment, one or more processors may be configured to: obtain survey results from a target user, the survey results comprising an answer to each of a plurality of survey questions included in a survey; and determine a first survey score group by converting the answer to each of the plurality of survey questions included in the survey into a score.
[0007] In one embodiment, the second set of survey scores may be determined after the artificial neural network receives an input of an image of the target user's skin and outputs a score for each of a plurality of survey questions included in the survey.
[0008] In one embodiment, one or more processors may be configured to: obtain scores of questions corresponding to a first type from a plurality of survey questions included in a survey from a first survey score group; obtain scores of questions corresponding to a second type from a plurality of survey questions included in the survey from a second survey score group; determine a final survey score group based on the scores of the questions corresponding to the first type and the scores of the questions corresponding to the second type; and determine a score for a skin type or each of the one or more indicators based on the final survey score group.
[0009] In one embodiment, questions corresponding to the first type may be questions having a score predicted by the artificial neural network with an accuracy less than a predetermined threshold, while questions corresponding to the second type are questions having a score predicted by the artificial neural network with an accuracy greater than or equal to the predetermined threshold.
[0010] In one embodiment, the one or more processors may be configured to: determine a final survey score group based on the first survey score group and the second survey score group; determine a score for each of the one or more indicators by adding the scores of the questions included in the final survey score group according to each question group included in the survey; and determine a skin type based on the score for each of the one or more indicators.
[0011] In one embodiment, the one or more processors may be configured to: determine an expert score set for the image of the target user's skin based on the output of the artificial neural network; and determine a score for the skin type or each of the one or more indicators additionally based on the expert score set.
[0012] In one embodiment, the expert score group may include scores of at least one target to be determined among redness, pigmentation, pores, wrinkles, and acne determined based on the image of the target user's skin.
[0013] In one embodiment, the one or more processors may be configured to: determine a final survey score group based on the first survey score group and the second survey score group; and determine a score for the skin type or each of the one or more indicators by calculating a weighted sum of the final survey score group and the expert score group.
[0014] According to another aspect of the present disclosure, a method for determining a user's skin condition or skin type based on artificial intelligence technology may be provided. A method executed by a computer may include one or more processors and one or more memories storing instructions, which are executed by the one or more processors, the one or more memories may store an artificial neural network, which is constructed by modeling the correlation between multiple images of the skin of multiple users and scores of multiple survey questions included in a survey about skin provided to multiple users, and the method may include the one or more processors performing the following operations according to these instructions: determining a first survey score group for a survey about the skin of a target user; inputting an image of the skin of the target user into the artificial neural network; determining a second survey score group for the survey based on the output of the artificial neural network; and determining a score for each of one or more indicators indicating the skin type of the target user or the skin condition of the target user based on the first survey score group and the second survey score group.
[0015] In one embodiment, the survey may include a group of questions for at least one target to be determined, and the group of questions for at least one target to be determined may correspond to one or more indicators respectively.
[0016] In one embodiment, the method may also include performing the following operations by one or more processors: obtaining a survey result from a target user, the survey result including an answer to each of a plurality of survey questions included in the survey; and determining a first survey score group by converting the answer to each of the plurality of survey questions included in the survey into a score.
[0017] In one embodiment, the method may further include performing the following operations by one or more processors: obtaining scores of questions corresponding to the first type among multiple survey questions included in the survey from a first survey score group; obtaining scores of questions corresponding to the second type among multiple survey questions included in the survey from a second survey score group; determining a final survey score group based on the scores of the questions corresponding to the first type and the scores of the questions corresponding to the second type; and determining a score for the skin type or each of the one or more indicators based on the final survey score group.
[0018] In one embodiment, questions corresponding to the first type may be questions having a score predicted by the artificial neural network with an accuracy less than a predetermined threshold, while questions corresponding to the second type are questions having a score predicted by the artificial neural network with an accuracy greater than or equal to the predetermined threshold.
[0019] In one embodiment, the method may further include performing the following operations by one or more processors: determining a final survey score group based on the first survey score group and the second survey score group; determining a score for each of the one or more indicators by adding the scores of the questions included in the final survey score group according to each question group included in the survey; and determining a skin type based on the score for each of the one or more indicators.
[0020] In one embodiment, the method may also include performing the following operations by one or more processors: determining an expert score group for the image of the target user's skin based on the output of the artificial neural network; and additionally determining a score for the skin type or each of the one or more indicators based on the expert score group.
[0021] In one embodiment, the expert score group may include scores of at least one target to be determined among redness, pigmentation, pores, wrinkles, and acne determined based on the image of the target user's skin.
[0022] In one embodiment, the method may further include performing the following operations by one or more processors: determining a final survey score group based on the first survey score group and the second survey score group, wherein determining the score of the skin type or each of the one or more indicators may include determining the score of the skin type or each of the one or more indicators by calculating a weighted sum of the final survey score group and the expert score group.
[0023] According to another aspect of the present disclosure, a non-transitory computer-readable recording medium for determining a user's skin condition or skin type based on artificial intelligence technology may be provided. A non-transitory computer-readable recording medium records instructions that are executed by a computer, and the non-transitory computer-readable recording medium may include: one or more memories configured to store instructions that, when executed by a processor, cause one or more processors to perform calculations; and an artificial neural network constructed by modeling the correlation between multiple images of the skin of multiple users and scores of multiple survey questions included in a survey about skin provided to multiple users, wherein when executed by one or more processors, the instructions may cause the one or more processors to perform the following operations: determine a first survey score group for a survey about the skin of a target user; input an image of the skin of the target user into the artificial neural network; determine a second survey score group for the survey based on the output of the artificial neural network; and determine a score for each of one or more indicators indicating the skin type of the target user or the skin condition of the target user based on the first survey score group and the second survey score group.
[0024] According to various embodiments of the present disclosure, compared to an artificial neural network for immediately predicting a user's skin condition from a user image, an explainable artificial neural network can be provided by inputting a user image into the artificial neural network and predicting a score for each survey question included in a survey.
[0025] According to various embodiments of the present disclosure, the reliability of the score of the survey can be improved by predicting a survey score group (e.g., a second survey score group) of the survey through an artificial neural network and correcting the survey score group (e.g., the first survey score group) according to the actual answers of the user through the second survey score group.
[0026] According to various embodiments of the present disclosure, user convenience can be increased by reducing the number of survey questions to be acquired from a user. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 The process of operating a computing device according to an embodiment of the present disclosure is shown.
[0028] Figure 2 is a block diagram illustrating a computing device according to an embodiment of the present disclosure.
[0029] Figure 3 is a diagram showing user images and survey results according to an embodiment of the present disclosure.
[0030] Figure 4is a conceptual diagram illustrating a process in which a computing device determines a second survey score group based on an output of an artificial neural network according to an embodiment of the present disclosure.
[0031] Figure 5 is a conceptual diagram illustrating a process in which a computing device determines an expert score group and a second survey score group based on an output of an artificial neural network according to another embodiment of the present disclosure.
[0032] Figure 6 Some training data for training an artificial neural network according to an embodiment of the present disclosure are shown.
[0033] Figure 7 is a flow chart illustrating the operation of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0034] The various embodiments of the present disclosure are examples for clearly describing the technical ideas of the present disclosure, without limiting the present disclosure to a specific embodiment form. The technical ideas of the present disclosure include various modifications, equivalents and substitutions of the various embodiments disclosed in this document, as well as embodiments in which all or some of the various embodiments are selectively combined. The scope of the claims of the technical ideas of the present disclosure is not limited to the various embodiments or their specific implementations presented below.
[0035] Unless otherwise defined, terms (including technical or scientific terms) used in the present disclosure may have meanings that are commonly understood by those skilled in the art to which the present disclosure belongs.
[0036] The expressions "including", "may include", "provided with", "may be provided with", "have", "may have", etc. used in the present disclosure indicate the existence of the following features (e.g., functions, operations, elements, etc.), but do not exclude the existence of other additional features. That is, these expressions should be understood as open terms that mean the possibility of including other embodiments.
[0037] Unless otherwise mentioned in the context, singular expressions used in the present disclosure may include plural meanings, and the same applies to singular expressions recited in the claims.
[0038] Unless otherwise mentioned in the context, the expressions "first", "second", etc. used in the present disclosure are used to distinguish one entity from another entity among multiple entities, and are not intended to limit the order or importance of the corresponding entities. In an embodiment, multiple question groups according to the present disclosure can be distinguished from each other by expressing "first question group" and "second question group". In an embodiment, multiple indicators indicating skin conditions according to the present disclosure can be distinguished from each other by expressing "first indicator" and "second indicator".
[0039] As used in the present disclosure, the expressions "A, B, and C", "A, B or C", "A, B and / or C", "at least one of A, B, and C", "at least one of A, B, or C", or "at least one of A, B, and / or C" may mean each of the listed items or all available combinations of the listed items. For example, "at least one of A or B" may mean all of the following: (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.
[0040] The term "unit" used in the present disclosure may refer to software or hardware components, such as field programmable gate arrays (FPGAs) and application specific integrated circuits (ASICs). However, a "unit" is not limited to hardware and software. A "unit" may be configured to be stored in an addressable storage medium, and may be configured to run one or more processors. In an embodiment, a "unit" may include software components, object-oriented software components, components such as class components and task components, processors, functions, processes, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays, and parameters.
[0041] The expression "based on" used in this article is used to describe one or more factors that influence the decision, judgment action or operation described in the phrase or sentence including the relevant expression, and the expression does not exclude additional factors that influence the decision, judgment action or operation.
[0042] Expressions used in the present disclosure indicating that any element (e.g., a first element) is “connected to” or “coupled to” another element (e.g., a second element) may mean not only that any element is directly connected or coupled to the other element but also that any element is connected or coupled to the other element via a new element (e.g., a third element).
[0043] The expression "configured to" used in the present disclosure may have meanings such as "set to", "having the ability to", "changed to", "made to", "able to", etc. in the context. The corresponding expression is not limited to the meaning of "specifically designed in hardware", for example, a processor configured to perform a specific operation may refer to a special-purpose computer structured to perform the corresponding specific operation by programming.
[0044] In the present disclosure, artificial intelligence (AI) may refer to a technology that imitates human learning ability, reasoning ability, and perception ability and implements these abilities on a computer. Artificial intelligence may include machine learning and component technologies that use machine learning. Machine learning may mean an algorithm that extracts features of at least one training data to classify input data. In addition, technologies that use machine learning algorithms to imitate human brain functions (such as recognition and determination) may also be understood as part of the scope of artificial intelligence. For example, technical fields such as language understanding, visual understanding, inference / prediction, knowledge representation, and operational control may be included.
[0045] In the present disclosure, an artificial neural network may be designed to simulate a human brain structure on a computer, and may include multiple network nodes that simulate neurons of a human neural network and have weighted values. Multiple network nodes may simulate the synaptic activity of neurons that exchange signals through synapses between neurons, and multiple network nodes may have a connection relationship between them. In an artificial neural network, multiple network nodes are located on layers with different depths, and data may be exchanged according to a convolutional connection relationship. For example, the artificial neural network may be a convolutional neural network. In the present disclosure, an artificial neural network is a model trained according to a predetermined machine learning scheme, and may refer to a model that determines the weighted value of at least one network node included in an untrained model through machine learning. Machine learning may mean that computer software uses data and experience in processing data to improve data processing capabilities through training. An artificial neural network is constructed by modeling the correlation between data, and the correlation may be expressed by multiple parameters. An artificial neural network derives the correlation between data by extracting and analyzing features from given data, and machine learning may optimize the parameters of the artificial neural network by repeating the process. For example, an artificial neural network may train a mapping (correlation) between input and output for data given in the form of input and output pairs. Alternatively, when only input data is given, the artificial neural network can learn the relationship by deriving regularities between the given data.In this disclosure, "artificial neural network" may be used interchangeably with the term "artificial neural network model".
[0046] Hereinafter, various embodiments of the present disclosure are described with reference to the accompanying drawings. In the accompanying drawings and the accompanying drawings, the same reference numerals may be assigned to the same or substantially equivalent elements. In addition, in the following description of various embodiments, overlapping descriptions of the same or corresponding elements may be omitted, but this does not mean that the corresponding elements do not belong to the embodiment.
[0047] Figure 1The process of operating the computing device 100 according to an embodiment of the present disclosure is shown. The computing device 100 can obtain a user image 310 and a survey result 330 from the user terminal 110. The user image 310 can be an image of the skin of a "target user" who expects to determine the skin type through the computing device 100 or determine the score of one or more indicators indicating the skin type through the computing device 100. In the following, for ease of description, the "target user" may be referred to as the "user", and the "image of the target user's skin" may be referred to as the "user image". The survey result 330 may be hierarchical data obtained from the user for the survey of the present disclosure. The survey result 330 may be data having a predetermined document structure (e.g., a table, a list, a tuple, a dictionary, etc.).
[0048] The computing device 100 may determine a score for each of one or more indicators indicating a skin type of the user and / or a skin condition of the user based on the user image 310 and the survey results 330. The computing device 100 may determine a first survey score group for the survey based on the survey results 330. The computing device 100 may input the user image 310 into an artificial neural network and determine a second survey score group for the survey based on the output of the artificial neural network. The computing device 100 may determine a final survey score group based on the first survey score group and the second survey score group. The computing device 100 may determine a score for each of one or more indicators indicating a skin condition based on the final survey score group. The computing device 100 may determine the skin type of the user through the determined scores for each of the one or more indicators.
[0049] The computing device 100 may transmit the determined skin type and the score for each of the one or more indicators to the user terminal 110 .
[0050] In the present disclosure, one or more indicators indicating the skin condition of the user may include indicators such as oiliness, sensitivity, moisture, pigment, wrinkles, pores, etc. The computing device 100 may determine the score of each of the one or more indicators indicating the skin condition by a predetermined calculation. At this time, the score of a specific indicator may indicate the quality of the skin condition of the corresponding indicator. For example, when the score of the oiliness indicator is low, the user's skin lacks oil, and when the score of the oiliness indicator is high, the user's skin has too much oil. When the score of the moisture indicator is low, the user's skin lacks moisture, and when the score of the moisture indicator is high, the user's skin is in a sufficient moisture state. When the score of the sensitivity indicator is low, the user's skin is not sensitive, and when the score of the sensitivity indicator is high, the user's skin is sensitive. When the score of the pigment indicator is low, the user's skin is almost pigment-free, and when the score of the pigment indicator is high, the user's skin has high pigment. When the score of the wrinkle indicator is low, the user's skin has no wrinkles, and when the score of the wrinkle indicator is high, the user's skin has many wrinkles. When the score of the pore index is low, the user's skin is in a good pore state (e.g., a small number of pores or small pores), and when the score of the pore index is high, the user's skin is in a poor pore state (e.g., a large number of pores or large pores). One or more indicators indicating skin conditions are for description only, and the present disclosure includes various types of indicators that may indicate skin conditions.
[0051] Figure 2 is a block diagram of a computing device 100 according to an embodiment of the present disclosure. The computing device 100 may include one or more processors 210 and / or one or more memories 220 as elements. In some embodiments, at least one element of the computing device 100 may be omitted, or other elements may be added to the computing device 100. In some embodiments, additionally or alternatively, some elements may be integrated or may be implemented as a single or multiple entities. In the present disclosure, one or more processors 210 may be expressed as processor 210. Unless the context clearly indicates otherwise, the expression "processor 210" may refer to a collection of one or more processors. In the present disclosure, one or more memories 220 may be expressed as memory 220. Unless the context clearly indicates otherwise, the expression "memory 220" may refer to a collection of one or more memories.
[0052] At least some elements inside or outside the computing device 100 may be connected to each other through a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), a mobile industry processor interface (MIPI), etc., and exchange data and / or signals.
[0053] The processor 210 may control at least one element connected to the processor 210 in the computing device 100 through driving software (e.g., instructions, programs, etc.). The processor 210 may perform various calculations, processing, data generation, manufacturing, etc. operations related to the present disclosure. The processor 210 may load data, etc. from the memory 320 and store it in the memory 320. The processor 210 may process the calculation of the artificial neural network.
[0054] The memory 220 may store various data. The data stored in the memory 220 may be acquired or processed by at least one element of the computing device 100, or may include software (e.g., instructions, programs, etc.) as the data used. The memory 220 may include volatile memory and / or non-volatile memory. In the present disclosure, instructions or programs are software stored in the memory 220, and may include an operating system, an application, and / or middleware for providing various functions to allow the application to use the resources of the computing device 100 for controlling the resources of the computing device 100. The memory 220 may store instructions that, when executed by the processor 210, cause the processor 210 to perform calculations. The memory 220 may store surveys provided to users. The memory 220 may store an artificial neural network that is constructed by modeling the correlation between multiple images of the skin of multiple users (i.e., multiple predetermined users) and scores of multiple survey questions included in the survey for the skin provided to the multiple users (i.e., multiple predetermined users). The survey and artificial neural network according to the present disclosure are described in detail below.
[0055] In an embodiment, the computing device 100 may further include a communication circuit 230. The communication circuit 230 may perform wireless or wired communication between the computing device 100 and a server or between the computing device 100 and other devices. For example, the communication circuit 230 may perform wireless communication according to schemes such as enhanced mobile broadband (eMBB), ultra-reliable low-latency communication (URLLC), massive machine-type communication (MMTC), long-term evolution (LTE), LTE-Advanced (LTE-A), new radio (NR), universal mobile telecommunications system (UMTS), global system for mobile communications (GSM), code division multiple access (CDMA), wideband CDMA (WCDMA), wireless broadband (WiBro), wireless fidelity (Wi-Fi), Bluetooth, near field communication (NFC), global positioning system (GPS), or global navigation satellite system (GNSS). For example, the communication circuit 230 may perform wired communication according to schemes such as universal serial bus (USB), high-definition multimedia interface (HDMI), recommended standard-232 (RS-232), or plain old telephone service (POTS). In an embodiment, the communication circuit 230 may communicate with another device. In this case, the other device may be the user terminal 110 .
[0056] In an embodiment, the computing device 100 may further include an input unit 240. The input unit 240 may be an element for transferring data physically input from the outside to at least one element included in the computing device 100. For example, the input unit 240 may include a mouse, a keyboard, a touch pad, and the like.
[0057] The computing device 100 according to various embodiments of the present disclosure may be various types of devices. For example, the computing device may be a portable communication device, a portable multimedia device, a wearable device, a home appliance, or a device according to a combination of one or more of the above devices. The computing device of the present disclosure is not limited to the devices listed above.
[0058] A survey according to an embodiment of the present disclosure may include a plurality of predetermined survey questions. In the present disclosure, "survey questions" may be used interchangeably with "questions". The survey may include a group of questions for at least one target to be determined. Each group of questions may include one or more survey questions. The target to be determined may be, for example, oiliness, sensitivity, pigmentation, wrinkles, etc. The survey may include at least one group of questions for oily skin, a group of questions for sensitive skin, a group of questions for pigmented skin, or a group of questions for wrinkled skin. For example, the group of questions for oily skin may include the questions shown in Table 1.
[0059] [Table 1]
[0060]
[0061]
[0062] A question set for sensitive skin may include the questions shown in Table 2.
[0063] [Table 2]
[0064]
[0065] A question set for pigmented skin may include the questions shown in Table 3.
[0066] [Table 3]
[0067]
[0068] A question set for wrinkled skin may include the questions shown in Table 4.
[0069] [Table 4]
[0070]
[0071] As described above, the contents of survey questions included in a survey, the types and number of surveys, the types and number of question groups, and the like are examples for describing the present disclosure and do not limit the present disclosure.
[0072] In an embodiment of the present disclosure, each question group in the question group for at least one target to be determined included in the survey may correspond to each indicator in one or more indicators. For example, the question group for "oily skin" may be a question group for determining the skin condition for "oiliness" in one or more indicators indicating the skin condition of the user. The question group for "sensitive skin" may be a question group for determining the skin condition for "sensitivity" in one or more indicators indicating the skin condition of the user.
[0073] Figure 3A user image 310 and a survey result 330 according to an embodiment of the present disclosure are shown. The user image 310 can be obtained from the user and can be, for example, a facial photo of the user, a full-body photo, or a photo obtained by capturing the user's skin. The survey result 330 may include an answer to each of the multiple survey questions included in the survey. For example, the survey result 330 may include answers to questions 1 to 5 in the first question group and answers to questions 1 to 5 in the second question group. The computing device 100 can obtain the survey result 330 from the external user terminal 110 through the communication circuit 230. The computing device 100 can obtain the survey result 330 by directly receiving an input signal from the user through the input unit 240. In this case, the computing device 100 may also include a display unit (not shown) for visually showing the survey.
[0074] Computing device 100 may determine a first survey score group for the survey based on survey results 330. In the present disclosure, a survey score group may refer to data including a score for each of a plurality of survey questions included in the survey. A "first survey score group" may refer to a survey score group derived based on survey results 330 acquired from a user. A "second survey score group" may refer to a survey score group derived based on an output of an artificial neural network. The first survey score group and the second survey score group may be collectively referred to as "survey score groups."
[0075] Processor 210 may obtain survey result 330 from a user, survey result 330 including an answer to each of a plurality of survey questions included in the survey. Processor 210 may obtain survey result 330 by receiving data from an external device (e.g., a user terminal) through communication circuit 230. Processor 210 may obtain survey result 330 by receiving an input of data from a user through input unit 240. Processor 210 may determine a first survey score group by converting the answer to each of a plurality of survey questions included in the survey result. Table 5 shows at least some of the first survey score groups. The score of each question in Table 5 may be obtained by converting Figure 3 The survey result 330 is converted into a score to obtain a value.
[0076] [Table 5]
[0077]
[0078] The survey result may include information about the number selected by the user for the multiple-choice question, and the first survey score group may be data obtained by converting the information about the number selected by the user into a score. The score conversion table used by the processor 210 to convert the survey result 330 into the first survey score group of Table 5 may be, for example, as shown in Table 6.
[0079] [Table 6]
[0080]
[0081] The score conversion table shown in Table 6 can be predetermined considering the number of answers to each question, the importance of the answer, etc. For example, some of the questions included in the survey may be questions asking about the symptoms or severity of a condition (e.g., what happens to my skin if I don't use a moisturizer?). The answer to this question can be configured to make the symptoms from no.1 to no.4 become severe or alleviated (e.g., no.1 is always dry, no.2 is frequently dry, no.3 is sometimes dry, and no.4 is no different from usual), and the score conversion table for the corresponding question can be configured to assign a higher score or a lower score from no.1 to no.4. In another example, some of the questions included in the survey may be questions that the user cannot know the exact answer or that the user is difficult to answer (e.g., I _________ blemishes on my skin). The answer to this question can be configured to make the symptoms from no.1 to no.4 become severe or alleviated, but no.5 corresponding to "don't know" can also be included, and the score conversion table for the corresponding question can be configured to assign a higher score or a lower score from no.1 to no.4, but the intermediate score is assigned to no.5. In another example, some of the questions included in the survey may be questions with two choices (e.g., true / false, yes / no, or positive / negative). The answer to the question may include two responses, and the score conversion table for the corresponding question may be configured to assign different scores to the two responses.
[0082] Computing device 100 may input the user image into an artificial neural network and determine a second survey score group for the survey based on an output of the artificial neural network.
[0083] Figure 4 4 is a conceptual diagram showing a process in which the computing device 100 determines a second survey score group based on the artificial neural network 410 according to an embodiment of the present disclosure. The artificial neural network 410 may be an artificial neural network model trained to determine a second survey score group 450 of the survey based on the input user image 310. The artificial neural network 410 may be trained by the computing device 100 of the present disclosure. After the artificial neural network 410 is trained by an external device, the artificial neural network 410 may be transmitted to the computing device 100 and used by the computing device 100. In the following, it is described that the artificial neural network 410 is trained by the computing device 100, but this is only for the convenience of description and does not limit the present disclosure.
[0084] The artificial neural network 410 according to an embodiment of the present disclosure may be trained based on one or more training data, the one or more training data including training images and survey score groups determined for the training images based on surveys. The training images are images pre-received from predetermined users and may be images stored in the memory 220. For example, the training images included in the training data may be images corresponding to the training images. Figure 3 The survey score group determined for the training image may be based on data generated by a predetermined user shown in the corresponding training image in response to the survey results of the survey. The survey score group determined for the training image may be based on data generated by another user (e.g., a skin expert) in response to the survey results of the survey based on the corresponding training image. Each piece of training data may be expressed by an ordered pair of X (input value) and Y (label value). X is a training image, and Y may be expressed as a survey score group. That is, the training data may be an ordered pair (training image, survey score group). The artificial neural network 410 may be trained based on the training data.
[0085] Processor 210 may input user image 310 into trained artificial neural network 410, perform calculations based on at least one node, and determine a second survey score set 450 for a survey including a plurality of survey questions. That is, second survey score set 450 may be determined after trained artificial neural network 410 receives user image 310 and outputs a score for each of the plurality of survey questions included in the survey. Table 7 shows at least some of the second survey score sets. The score for each question shown in Table 7 may be a score determined by artificial neural network 410 based on the user image 310. Figure 3 The value of the user image 310 output.
[0086] [Table 7]
[0087]
[0088] According to various embodiments of the present disclosure, in which the processor 210 determines the second survey score group 450 through the artificial neural network 410, the present disclosure has the advantage of providing an interpretable artificial neural network compared to an artificial neural network for immediately predicting the skin condition of a user from a user image by predicting a score for each survey question with respect to the user image 310. In addition, the present disclosure improves the reliability of the survey score by predicting the survey score group (i.e., the second survey group) of the survey through the artificial neural network, thereby correcting the survey score group (i.e., the first survey score group) according to the actual user response through the second survey score group. In addition, when the survey score group of the survey is predicted using the artificial neural network according to the present disclosure, the convenience of the user can be increased by reducing the number of survey questions that should be obtained from the user.
[0089] The computing device 100 may determine a score for each of one or more indicators indicating a skin type of the user or a skin condition of the user based on the first survey score group and the second survey score group. The computing device 100 may determine a final survey score group based on the first survey score group and the second survey score group. The computing device 100 may determine a score for each of one or more indicators indicating a skin condition based on the final survey score group. The computing device 100 may determine the skin type of the user through the determined score for each of the one or more indicators. Hereinafter, various embodiments are described in which the computing device 100 determines the final survey score group based on the first survey score group and the second survey score group according to the present disclosure.
[0090] In the first embodiment in which the computing device 100 determines the final survey score group, the processor 210 may determine the final survey score group by adding or deriving an average value of the first survey score group and the second survey score group. The processor 210 may determine the final survey score group by adding or averaging the scores included in the first survey score group determined based on the survey result obtained from the user and the second survey score group determined based on the output of the artificial neural network 410 for the user image. Table 8 shows the final survey score group calculated by averaging the first survey score group shown in Table 5 and the second survey score group shown in Table 7.
[0091] [Table 8]
[0092]
[0093] In a second embodiment in which the computing device 100 determines the final survey score group, the processor 210 may determine the final survey score group by calculating a weighted average of the first survey score group and the second survey score group. The weight value of each survey score group used for the weighted average may be a value predetermined for each of the first survey group and the second survey group. For example, a weighted value of "0.3" may be determined for the first survey score group, and a weighted value of "0.7" may be determined for the second survey score group. The weight value of each survey score group may be determined in consideration of the accuracy of the artificial neural network 410. The weight value of each survey score group used for the weighted average may be based on the reliability of the artificial neural network 410. The reliability output by the artificial neural network 410 for the input image together with the second survey score group may be, for example, a real number in the range [0, 1]. When the reliability output by the artificial neural network 410 is higher, the processor 210 may calculate the weighted average of the first survey score group and the second survey score group by assigning a higher weighted value to the second survey score group. For example, when the reliability output by the artificial neural network 410 is "P (P is a real number higher than or equal to 0 and equal to or less than 1)", the processor 210 may determine the final survey score group based on the equation "(1-P)*(first survey score group)+(second survey score group)". Therefore, when the artificial neural network 410 has a higher certainty for the output second survey score group, the processor 210 may reflect more of the second survey score group in the final survey score group. In addition, the artificial neural network 410 may output the reliability for each question to output the second survey score group. When the artificial neural network 410 derives the reliability for each question, calculating the weighted average of the first survey score group and the second survey score group may be performed for each question. Table 9 shows the final survey score group calculated by assigning a weighted value of 0.3 to the first survey score group shown in Table 5 and assigning a weighted value of 0.7 to the second survey score group shown in Table 7.
[0094] [Table 9]
[0095]
[0096] In a third embodiment in which the computing device 100 determines a final survey score group, the processor 210 may determine the final survey score group by obtaining a score of the survey question according to each question from the first survey score group or the second survey score group. Specifically, the processor 210 may divide the plurality of survey questions included in the survey into two or more types, and obtain scores of questions corresponding to the first type from the first survey score group and obtain scores of questions corresponding to the second type from the second survey score group. In the present disclosure, the scores of questions corresponding to the first type (e.g., first type questions) may be included in the final survey score group and obtained from the first survey score group. In addition, in the present disclosure, the scores of questions corresponding to the second type (e.g., second type questions) may be included in the final survey score group and obtained from the second survey score group. The processor 210 may determine whether each question corresponds to the first type or the second type based on a predetermined calculation.
[0097] In an embodiment of determining whether each of the plurality of questions included in the survey corresponds to the first type or the second type, the processor 210 may determine the type of each question according to a predetermined reference. The predetermined reference may be stored in the memory 220 and may include a plurality of questions corresponding to the first type and a plurality of questions corresponding to the second type. For example, the survey may include questions no.1 to no.10, and the processor 210 may determine questions no.1 to no.5 as the first type and questions no.6 to no.10 as the second type with reference to the predetermined reference. In addition, the processor 210 may obtain the scores of questions no.1 to no.5 from the first survey score group and the scores of questions no.6 to no.10 from the second survey score group based on the determined types to determine a final survey score group for all questions.
[0098] In another embodiment of determining whether each of the plurality of questions included in the survey corresponds to the first type or the second type, the processor 210 may determine the type of each question according to the accuracy of the artificial neural network 410. The question corresponding to the first type may be a question for which the accuracy of the artificial neural network 410 for predicting the score of the question is less than a predetermined threshold. The question corresponding to the second type may be a question for which the accuracy of the artificial neural network 410 for predicting the score of the question is higher than or equal to a predetermined threshold. The accuracy of the artificial neural network 410 according to an embodiment of the present disclosure may be measured during a validation or testing process performed after training the artificial neural network model. The accuracy of the artificial neural network 410 may be calculated for each question. For ease of description, it is assumed that there is a validation data set and that the validation data set includes 10 pairs of data. The data pairs included in the validation data set may have the form of a tuple (user image, first survey score group). In addition, for ease of description, it is assumed that the validation data set includes (image A, survey score group 1-A (first survey score group corresponding to image A)). Based on this assumption, the artificial neural network 410 may receive an input of image A and output a survey score group 2-A (a second survey score group output for image A). The processor 210 may calculate the accuracy of the artificial neural network 410 based on the errors in the survey score group 2-A and the survey score group 1-A for the image A. For example, the processor 210 may obtain the accuracy of the artificial neural network 410 by calculating the relative error considering the “true value” of the survey score group 1-A and the “measured value” of the survey score group 2-A. For example, when the survey score group 1-A made based on the survey including two questions and the image A is [3, 4], and the output survey score group 2-A is [2.4, 3.6], the accuracy of question no. 1 may be calculated as 0.8 (=1-|3–2.4| / 3), and the accuracy of question no. 2 may be calculated as 0.9 (=1-|4–3.6| / 4). In addition, when such a process is performed on all 10 pairs of data included in the validation data set, the processor 210 may calculate the accuracy of the validation data set of the artificial neural network 410. For example, the processor 210 may calculate the accuracy of the validation data set by averaging the accuracy calculated for each validation data set. Therefore, the processor 210 may determine the accuracy of the artificial neural network 410. The processor 210 according to the present disclosure may determine each of the plurality of questions as the first type or the second type by comparing the accuracy calculated for each question by the artificial neural network 410 with a predetermined threshold.For example, when the accuracy of the questions calculated by the artificial neural network 410 for the survey including question no.1 and question no.2 is 0.8 and 0.9 and the predetermined threshold is 0.85, the processor 210 may determine question no.1 whose accuracy is less than the predetermined threshold as the first type, and question no.2 whose accuracy is higher than the predetermined threshold as the second type. Therefore, the processor 210 may obtain the score of question no.1 from the first survey score group and obtain the score of question no.2 from the second survey score group to determine the final survey score group for both question no.1 and question no.2. The predetermined threshold to be compared with the accuracy calculated by the artificial neural network 410 for each question may be configured for each question. When the threshold is configured for each question, the processor 210 may determine the type of the question in consideration of different thresholds for the corresponding question.
[0099] The processor 210 may classify each of the multiple questions included in the survey into one of the first type or the second type based on the at least one method described above, and obtain the score of the corresponding question from the first survey score group or the second survey score group according to the type of the question, thereby determining the final survey score group. For the first survey score group shown in Table 5 and the second survey score group shown in Table 7, Table 10 shows the final survey score group determined according to the type of each question. For ease of description, it is assumed that questions no. 2, no. 3, and no. 5 of the first question group and questions no. 3 and no. 4 of the second question group are first type questions, while questions no. 1 and no. 4 of the first question group and questions no. 1, no. 2, and no. 5 of the second question group are second type questions.
[0100] [Table 10]
[0101]
[0102]
[0103] As described above, some embodiments for determining the final survey score group and some embodiments for determining the type of questions described with reference to Tables 8 to 10 are merely examples for describing the present disclosure and do not limit the present disclosure, and the present disclosure includes various embodiments for determining the final survey score group based on a first survey score group and a second survey score group, the first survey score group being determined based on a user input, and the second survey score group being determined based on an output of an artificial neural network.
[0104] The computing device 100 may determine the score of each of the one or more indicators indicating the user's skin type or the user's skin condition based on the final survey score group. In the present disclosure, the score of each of the one or more indicators indicating the user's skin condition may include a "total score" and an "indicator score". The processor 210 may determine the total score by adding the scores of the questions included in the final survey score group according to at least one question group included in the survey. For example, referring to the final survey score group of Table 10, the total score of the first question group may be 22.863, and the total score of the second question group may be 18.781. When the first question group is a question group for determining oily skin and the second question group is a question group for determining sensitivity, the processor 210 may determine that the total score of the oiliness indicator is 22.863 and the total score of the sensitivity indicator is 18.781. The processor 210 may determine the indicator score of each of the one or more indicators indicating the user's skin condition based on the total score. The processor 210 may convert the total score into an indicator score through a scaling scheme. In the present disclosure, a scaling scheme may refer to a scheme for converting a score range of a determined total score into a predetermined score range (eg, higher than or equal to 0 and equal to or less than 100). For example, the processor 210 may perform calculations based on Equation 1 for scaling.
[0105] [Equation 1]
[0106]
[0107] For example, when the “total score” calculated by the processor 210 for a specific indicator is 13 and the processor 210 performs the scaling scheme according to Equation 1, the “indicator score” of the corresponding specific indicator may be 90. According to the present disclosure, the processor 210 may determine which skin condition corresponds to the corresponding indicator score within a predetermined total score range by converting the score range that the specific indicator may have (i.e., [the minimum total value of the scores of the theoretically available questions, the maximum total value of the scores of the theoretically available questions]) into a predetermined score range (e.g., [0, 100]).
[0108] The processor 210 may determine the user's skin condition indicated by the corresponding indicator based on the indicator score of each indicator in one or more indicators. For example, when within a total score range predetermined for a specific indicator (e.g., higher than or equal to 0 and equal to or less than 100), the first range (e.g., higher than or equal to 0 and less than 20) is classified as "very bad", the second range (e.g., higher than or equal to 20 and less than 40) is classified as "bad", the third range (e.g., higher than or equal to 40 and less than 60) is classified as "normal", the fourth range (e.g., higher than or equal to 60 and less than 80) is classified as "good", the fifth range (e.g., higher than or equal to 80 and less than 100) is classified as "very good", and the indicator score of the specific indicator is 55, the processor 210 may determine the user's skin condition indicated by the corresponding specific indicator as "normal".
[0109] The processor 210 may determine the user's skin type based on the user's skin condition for each of the one or more indicators. For example, it is assumed that the first indicator included in the one or more indicators is an oiliness indicator, the second indicator is a moisture indicator, and the third indicator is a sensitive indicator. The processor 210 may determine the skin condition for each indicator by comparing the indicator score of each indicator with the average score of the corresponding indicator. For example, in the oiliness indicator corresponding to the first indicator, when the user's oiliness indicator score is less than the average score of the oiliness indicator, the user's oiliness condition may be determined as "dry (D)", and when the user's oiliness indicator score is higher than the average score of the oiliness indicator, the user's oiliness condition may be determined as "oily (O)". In the moisture indicator corresponding to the second indicator, when the user's moisture indicator score is less than the average score of the moisture indicator, the user's moisture condition may be determined as "insufficient moisture (-)", and when the user's moisture indicator score is higher than the average score of the moisture indicator, the user's moisture condition may be determined as "sufficient moisture (+)". When the user's sensitivity index score is less than the average score of the sensitivity index, the user's sensitive condition can be determined as "sensitive (S)", and when the user's sensitivity index score is higher than the average score of the sensitivity index, the user's sensitive condition can be determined as "insensitive (N)". The processor 210 can determine the user's skin type by adding the skin conditions according to the determined indicators. That is, when the oily condition (i.e., the skin condition according to the first indicator) determined for the user is "dry (D)", the moisture condition (i.e., the skin condition according to the second indicator) is "insufficient moisture (-)", and the sensitive condition (i.e., the skin condition according to the third indicator) is "sensitive (S)", the processor 210 can determine the corresponding user skin type as "DS-". In this case, "DS-" can be a general reference to skin types of dry, insufficient moisture, and sensitive skin. As described above, the processor 210 can determine the skin condition of the corresponding indicator based on the indicator score of each indicator in one or more indicators, and combine the determined skin conditions to determine the user's skin type. For example, when there are six indicators indicating the skin condition and each indicator has two conditions, the user's skin type can include a total of 64 (=2 6 ) types. The number of conditions for each indicator may be three or more. In another embodiment, the processor 210 may determine the user's skin type based on a main indicator included in the one or more indicators. For example, when the one or more indicators indicating the skin condition include the first to sixth indicators, the processor 210 may determine the user's skin type by combining only the skin conditions for the one or more indicators (e.g., the first to third indicators) corresponding to the main indicators. For example, when the user's skin type is determined based on three main indicators indicating the skin condition among the six indicators, a total of 8 (=23 ) types can be derived as the user skin type. In this case, the processor 210 according to the present disclosure has the following advantages: the user skin type is transmitted in the form of shorter and more intuitive information, and additional information is generated by an indicator score or skin condition determined by an indicator that is not a primary indicator.
[0110] In the following, reference Figure 5 and Figure 6 Input and output data and operation of an artificial neural network 510 according to another embodiment of the present disclosure are described. In the present disclosure, unless otherwise expressed, the above description of the artificial neural network 410 may be applied to the artificial neural network 510, and overlapping descriptions may be omitted hereinafter.
[0111] Figure 5 is a conceptual diagram illustrating a process in which the computing device 100 determines an expert score group 551 and a second investigation score group 553 based on an output of an artificial neural network 510 according to another embodiment of the present disclosure.
[0112] The computing device 100 may determine an expert score group 551 for the user image 310 based on the output of the artificial neural network 510. That is, the computing device 100 may determine a second survey score group for the survey based on the output of the artificial neural network 510, and additionally determine the expert score group 551 for the user image 310. The expert score group 551 may include scores for one or more targets to be determined, which may be visually determined based on the user image 310. The expert score group 551 may include a score for at least one target to be determined among redness, pigment, pores, wrinkles, and acne. For example, the artificial neural network 510 may determine the expert score group 551 by determining the redness score as 1, the pigment score as 1.981, the pore score as 2.975, the wrinkle score as 1.993, and the inflammation score as 1.019 based on the user image 310.
[0113] Figure 6 Some training data for training the artificial neural network 510 according to an embodiment of the present disclosure are shown. The artificial neural network 510 can be trained based on one or more data including a survey score group 630 and an expert score group 610, wherein the survey score group 630 is determined for the training image based on the survey, and the expert score group 610 is determined for the training image. When the artificial neural network 510 is trained by the training data, the artificial neural network 510 can determine the expert score group 551 and the second survey score group 553 based on the input user image 310. The training images and configurations included in the training data and / or the method for generating the survey score group 630 have been described above with reference to Figure 3 and Figure 4 Described. The expert score group 610 included in the training data may be data generated as a result of an expert (e.g., a health care provider, a skin-related expert, etc.) observing the corresponding training image with the naked eye. For example, the expert score group 610 included in the training data may include a score determined for at least one target to be determined among redness, pigmentation, pores, wrinkles, and acne after the expert views the training image with the naked eye. When the training data of the artificial neural network 510 is expressed by an ordered pair of X (input value) and Y (label value), X may be a training image, and Y may be a value including the expert score group 610 and the survey score group 630. That is, the training data may be an ordered pair (X: training image, Y: [expert score group, survey score group]). In this embodiment, Y included in the training data of the artificial neural network 510 may be data in a form that connects the expert score group 610 and the survey score group 630 together. For example, when the survey score group 630 is a matrix of size 1×35 and the expert score group is a matrix of size 1×5, Y included in the training data of the artificial neural network 510 may be matrix data of size 1×40. The artificial neural network 510 may be trained based on the training data.
[0114] The processor 210 may input the user image 310 into the trained artificial neural network 510 , perform calculation based on at least one node, and determine an expert score group 551 for the user image 310 and a second survey score group 553 for the survey.
[0115] The processor 210 may determine a score for each of one or more indicators indicating a user skin type or a user skin condition based on the final survey score group and additionally based on the expert score group, the expert score group being determined for the user image based on the artificial neural network 510. The processor 210 may determine the score for the user skin type or each of the one or more indicators by calculating a weighted sum of the final survey score group and the expert score group. For example, when the score of the first indicator included in the final survey score group is "5", the score of the first indicator included in the expert score group is "15", and the weighted values of the final survey score group and the expert score group are "0.5" and "0.5", the final score of the first indicator may be determined to be "10 (=0.5*5+0.5*15)".
[0116] The computing device 100 according to the present disclosure can determine a second survey score group 553 and an expert score group 551 based on the artificial neural network 510, and determine a score of the user's skin condition and skin type by considering both the survey score group 553 and the expert score group 551 to generate skin-related information by combining the user's subjective answers to the survey and the expert's objective evaluation.
[0117] In another embodiment of the present disclosure, the computing device 100 may detect at least one target symptom area in the user image 310 based on an artificial neural network. Hereinafter, for ease of description, the artificial neural network for detecting the target symptom area may be referred to as a "second artificial neural network". The second artificial neural network may be trained by the computing device 100 of the present disclosure. The second artificial neural network may be transmitted to the computing device 100 and used by the computing device 100 after being trained by an external device. The second artificial neural network may receive an image to be trained to detect a target symptom area. The target symptom area may be a symptom area such as acne, pigmentation, redness, pores, or wrinkles. For example, the second artificial neural network may generate bounding box data including the target symptom area within the input image according to a predetermined detection method. In another example, the second artificial neural network may determine one or more pixels corresponding to the target symptom area among a plurality of pixels included in the input image according to a predetermined segmentation method. The processor 210 may detect at least one target symptom area within the user image 310 through the fully trained second artificial neural network, and display the detected area through a display unit.
[0118] Figure 7is a flowchart showing the operation of a computing device according to an embodiment of the present disclosure. In operation S710, the computing device 100 according to an embodiment of the present disclosure may determine a first survey score group for a survey on the skin of a target user. The target user may refer to a user who desires to determine the skin type through the computing device 100 or to determine the score of one or more indicators indicating the skin condition through the computing device 100. The survey may include a plurality of predetermined survey questions. The survey may include a question group for at least one target to be determined. The survey may include at least one question group in a question group for oily skin, a question group for sensitive skin, a question group for pigmented skin, or a question group for wrinkled skin. Each question group included in the survey may include a plurality of survey questions. The processor 210 may obtain a survey result as a user's answer to the survey. The processor 210 may obtain a survey result 330 from the user through the input unit 240. The processor 210 may obtain the survey result 330 from the external user terminal 110 through the communication unit 230. The processor 210 may determine the first survey score group for the survey based on the obtained survey result 330. The “survey score group” may refer to data including a score for each of a plurality of survey questions included in the survey. The “first survey score group” may refer to a survey score group derived based on the survey result 330 acquired from the user. For example, the survey result 330 may include an answer (e.g., a response number) selected for each objective question, and the first survey score group may be data obtained by converting the survey result 330 into a score. The processor 210 may determine the first survey score group by converting the answer to each of the plurality of survey questions included in the survey result.
[0119] In operation S720, the computing device 100 according to an embodiment of the present disclosure may input an image of the skin of the target user into an artificial neural network. The artificial neural network may be trained to determine a survey score group for a survey based on the input user image. The training data for training the artificial neural network may include a training image and a survey score group corresponding to the training image.
[0120] In operation S730, the computing device 100 according to an embodiment of the present disclosure may determine a second survey score group of the survey based on the output of the artificial neural network. The "second survey score group" may refer to a survey score group derived based on the output of the artificial neural network.
[0121] In operation S740, the computing device 100 according to an embodiment of the present disclosure may determine the scores of one or more indicators indicating the skin type of the target user or the skin condition of the target user based on the first survey score group and the second survey score group. The processor 210 may determine the final survey score group by calculating the sum, average, or weighted sum of the first survey score group and the second survey score group. The processor 210 may determine the final survey score group by determining the scores of a plurality of survey questions included in the survey from one of the first survey score group or the second survey score group. In this case, the question type may be determined according to a predetermined reference in the memory 220. In addition, the question type may be determined based on the accuracy of the artificial neural network for each question. The processor 210 may determine the score of each of the one or more indicators indicating the skin type of the user or the skin condition of the user based on the final survey score group. In the present disclosure, the score of each of the one or more indicators indicating the skin condition of the user may include a "total score" and an "indicator score". The processor 210 may determine the total score by summing the scores of the questions included in the final survey score group according to at least one question group included in the survey. The processor 210 may determine an indicator score for each of the one or more indicators indicating the user's skin condition based on the total score. The processor 210 may convert the total score into an indicator score through a scaling scheme. The processor 210 may determine the user's skin condition indicated by the corresponding indicator based on the indicator score for each of the one or more indicators. The processor 210 may determine the user's skin type based on the user's skin condition for each of the one or more indicators.
[0122] Although each step of the method or algorithm according to the present disclosure is described in the order of sequence in each flow chart of the present disclosure, each step can be performed in the order randomly combined by the present disclosure and in the order of sequence. The description according to the flow chart does not exclude changes or modifications in the method or algorithm, and does not mean that the predetermined steps are necessary or preferred. In an embodiment, at least some operations can be performed in parallel, repeatedly or heuristically. In an embodiment, at least some operations can be omitted or other operations can be added.
[0123] Various embodiments of the present disclosure may be implemented as software in a machine-readable storage medium. The software may be software for implementing various embodiments of the present disclosure. The software may be inferred by a programmer in the field of the present disclosure according to various embodiments of the present disclosure. For example, the software may be a program including machine-readable instructions (e.g., code or code segments). The machine may be a device capable of operating according to instructions called from a storage medium, and may be, for example, a computer. In an embodiment, the machine may be a computing device 100 according to an embodiment of the present disclosure. In an embodiment, the processor of the machine may allow elements of the machine to perform functions corresponding to corresponding instructions by executing called instructions. In an embodiment, the processor may be a processor 210 according to an embodiment of the present disclosure. The storage medium may be all types of recording media storing machine-readable data. The storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. In an embodiment, the storage medium may be a memory 320. In an embodiment, the storage medium may be implemented in the form of being distributed in a computer system connected via a network. The software may be distributed and stored in a computer system and executed. The storage medium may be a non-transitory storage medium. Non-transitory storage media are tangible media, regardless of whether data is stored semi-permanently or temporarily, and non-transitory storage media do not include transitory propagating signals.
[0124] Although the technical idea of the present disclosure has been described through various embodiments, the technical idea of the present disclosure includes various substitutions, modifications and changes that can be made within the scope that can be understood by those skilled in the art to which the present disclosure belongs. In addition, these substitutions, modifications and changes should be understood to be included in the scope of the attached claims.
Claims
1. A device comprising: one or more processors; as well as one or more memories configured to store instructions that, when executed by the processors, cause the one or more processors to perform computations and an artificial neural network constructed by modeling correlations between a plurality of images of a plurality of users' skin and scores for a plurality of survey questions included in a survey about skin provided to the plurality of users, The one or more processors are configured to perform the following operations according to the instructions: determining a first survey score group for a survey related to the target user's skin; inputting an image of the target user's skin into the artificial neural network; determining a second survey score set for the survey based on the output of the artificial neural network; and A score for each of one or more indicators indicating a skin type of the target user or a skin condition of the target user is determined based on the first survey score group and the second survey score group.
2. The device according to claim 1, wherein: The survey includes a set of questions for at least one objective to be determined, and The question groups for at least one target to be determined correspond to one or more indicators respectively.
3. The device according to claim 1, wherein: The one or more processors are configured to: obtaining survey results from the target user, the survey results comprising responses to each of the plurality of survey questions included in the survey; as well as The first set of survey scores is determined by converting responses to each of the plurality of survey questions included in the survey into a score.
4. The device according to claim 1, wherein: The second set of survey scores is determined after the artificial neural network receives an input of an image of the target user's skin and outputs a score for each of the plurality of survey questions included in the survey.
5. The device according to claim 1, wherein: The one or more processors are configured to: obtaining, from the first survey score group, scores of questions corresponding to a first type among the plurality of survey questions included in the survey; obtaining, from the second survey score group, scores of questions corresponding to a second type among the plurality of survey questions included in the survey; determining a final survey score group based on the scores of questions corresponding to the first type and the scores of questions corresponding to the second type; as well as A score for the skin type or each of the one or more indicators is determined based on the final survey score set.
6. The device according to claim 5, wherein: Questions corresponding to the first type are questions having scores predicted by the artificial neural network with an accuracy less than a predetermined threshold, while questions corresponding to the second type are questions having scores predicted by the artificial neural network with an accuracy greater than or equal to the predetermined threshold.
7. The device according to claim 1, wherein: The one or more processors are configured to: determining a final survey score group based on the first survey score group and the second survey score group; determining a score for each of the one or more indicators by summing the scores of the questions included in the final survey score group according to each group of questions included in the survey; as well as The skin type is determined based on a score for each of the one or more indicators.
8. The device according to claim 1, wherein: The one or more processors are configured to: Determining an expert score group for the image of the target user's skin based on the output of the artificial neural network; and A score for the skin type or each of the one or more indicators is additionally determined based on the expert score set.
9. The device according to claim 8, wherein: The expert score group includes scores of at least one target to be determined among redness, pigmentation, pores, wrinkles, and acne determined based on the image of the target user's skin.
10. The device according to claim 8, wherein: The one or more processors are configured to: determining a final survey score group based on the first survey score group and the second survey score group; and The score for the skin type or each of the one or more indicators is determined by calculating a weighted sum of the final survey score group and the expert score group.
11. A method executed by a computer, the computer comprising one or more processors and one or more memories storing instructions executed by the one or more processors, The one or more memories store an artificial neural network constructed by modeling correlations between a plurality of images of a plurality of users' skin and scores of a plurality of survey questions included in a survey about skin provided to the plurality of users, and The method is performed by the one or more processors according to the instructions, and the operations include: determining a first survey score group for a survey related to the target user's skin; inputting an image of the target user's skin into the artificial neural network; determining a second survey score set for the survey based on the output of the artificial neural network; as well as A score for each of one or more indicators indicating a skin type of the target user or a skin condition of the target user is determined based on the first survey score group and the second survey score group.
12. The method according to claim 11, wherein: The survey includes a question group for at least one target to be determined, and the question group for at least one target to be determined corresponds to the one or more indicators respectively.
13. The method according to claim 11, further comprising executing, by the one or more processors: obtaining a survey result from the target user, the survey result comprising a response to each of the plurality of survey questions included in the survey; and The first survey score set is determined by converting responses to each survey question of a plurality of survey questions included in the survey into a score.
14. The method according to claim 11, further comprising performing, by the one or more processors, the following operations: obtaining, from the first survey score group, scores of questions corresponding to a first type among the plurality of survey questions included in the survey; obtaining, from the second survey score group, scores of questions corresponding to a second type among the plurality of survey questions included in the survey; determining a final survey score group based on the scores of questions corresponding to the first type and the scores of questions corresponding to the second type; as well as A score for the skin type or each of the one or more indicators is determined based on the final survey score set.
15. The method according to claim 14, wherein: Questions corresponding to the first type are questions having scores predicted by the artificial neural network with an accuracy less than a predetermined threshold, while questions corresponding to the second type are questions having scores predicted by the artificial neural network with an accuracy greater than or equal to the predetermined threshold.
16. The method according to claim 11, further comprising performing, by the one or more processors, the following operations: determining a final survey score group based on the first survey score group and the second survey score group; determining a score for each of the one or more indicators by summing the scores of the questions included in the final survey score group according to each group of questions included in the survey; and The skin type is determined based on a score for each of the one or more indicators.
17. The method according to claim 11, further comprising performing, by the one or more processors, the following operations: Determining an expert score group for the image of the target user's skin based on the output of the artificial neural network; and A score for the skin type or each of the one or more indicators is additionally determined based on the expert score set.
18. The method according to claim 17, wherein: The expert score group includes scores of at least one target to be determined among redness, pigmentation, pores, wrinkles, and acne determined based on the image of the target user's skin.
19. The method of claim 17, further comprising determining, by the one or more processors, a final survey score set based on the first survey score set and the second survey score set, in, Determining the score for the skin type or each of the one or more indicators includes determining the score for the skin type or each of the one or more indicators by calculating a weighted sum of the final survey score group and the expert score group.
20. A non-transitory computer-readable recording medium, the non-transitory computer-readable recording medium recording instructions executed by a computer, the non-transitory computer-readable recording medium comprising: one or more memories configured to store instructions that, when executed by the processors, cause the one or more processors to perform computations; and an artificial neural network constructed by modeling correlations between a plurality of images of a plurality of users' skin and scores of a plurality of survey questions included in a survey about skin provided to the plurality of users, Wherein, when the instruction is executed by the one or more processors, the one or more processors perform the following operations: determining a first survey score group for a survey related to the target user's skin; inputting an image of the target user's skin into the artificial neural network; determining a second survey score set for the survey based on the output of the artificial neural network; and A score for each of one or more indicators indicating a skin type of the target user or a skin condition of the target user is determined based on the first survey score group and the second survey score group.
