A skin condition detection method and system based on image recognition

Abstract

The present invention provides a skin condition detection method and system based on image recognition, which relate to the field of image recognition technology. In the present invention, first, a target skin image and target skin description data are obtained; secondly, skin image semantic vectors of at least two dimensions of the target skin image are mined, and the skin image semantic vectors of at least two dimensions are aggregated to form a first skin condition semantic vector; then, the skin description data semantic vector of the target skin description data is mined, and the skin description data semantic vector and the first skin condition semantic vector are associated with each other to form a skin condition associated semantic vector; finally, a skin condition detection operation is performed on the skin condition associated semantic vector through a target skin condition detection network, and a skin condition detection result is output. Based on the above method, the problem of relatively low reliability of skin condition detection in the prior art can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a skin condition detection method and system based on image recognition. Background Art

[0002] The detection of skin condition is generally carried out arbitrarily based on professional experience, and this detection method has the problem of relatively low efficiency. With the continuous maturity of image recognition technology, the application of skin condition detection based on image recognition is becoming more and more common, and this method can fully improve the efficiency of detection. However, the inventors have found through research that the recognition and detection of skin condition based on existing image recognition technology generally has the problem of relatively low detection reliability. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a skin condition detection method and system based on image recognition, so as to improve the problem of relatively low reliability of skin condition detection in the prior art.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] A skin condition detection method based on image recognition, comprising:

[0006] Obtaining a target skin image and target skin description data, wherein the target skin description data is formed in response to an input operation of a skin condition detection user on a skin condition corresponding to the target skin image, and belongs to a description of a skin condition by the skin condition detection user;

[0007] Mining skin image semantic vectors of at least two dimensions of the target skin image, and aggregating the skin image semantic vectors of at least two dimensions to form a first skin state semantic vector corresponding to the target skin image;

[0008] Mining out a skin description data semantic vector of the target skin description data, and performing an association mining operation on the skin description data semantic vector and the first skin state semantic vector to form a corresponding skin state association semantic vector;

[0009] Through the target skin state detection network, a skin state detection operation is performed on the skin state associated semantic vector, and a skin state detection result corresponding to the target skin image is output.

[0010] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the step of performing a skin state detection operation on the skin state associated semantic vector through a target skin state detection network and outputting a skin state detection result corresponding to the target skin image includes:

[0011] According to the target mapping vector, the skin state associated semantic vector is updated to form a corresponding target skin state semantic vector, wherein the target mapping vector is formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training while maintaining other parameters of the target skin state detection network unchanged;

[0012] Through the target skin state detection network, a skin state detection operation is performed on the target skin state semantic vector, and a skin state detection result corresponding to the target skin image is output.

[0013] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the step of updating the skin state associated semantic vector according to the target mapping vector to form a corresponding target skin state semantic vector includes:

[0014] According to the first mapping vector included in the target mapping vector, a linear mapping operation is performed on the skin state associated semantic vector to form a corresponding skin state mapping semantic vector;

[0015] Performing a fusion operation on the skin state mapping semantic vector and the skin description data semantic vector, and outputting a corresponding skin state fusion semantic vector;

[0016] Performing a nonlinear operation on the skin state fusion semantic vector, and outputting a corresponding skin state nonlinear semantic vector;

[0017] According to the second mapping vector included in the target mapping vector, a linear mapping operation is performed on the skin state nonlinear semantic vector to form a corresponding target skin state semantic vector, wherein the second mapping vector and the first mapping vector are both formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training while maintaining other parameters of the target skin state detection network unchanged.

[0018] In some preferred embodiments, in the above-mentioned skin condition detection method based on image recognition, the target skin condition detection network includes a plurality of semantic mining units connected in sequence;

[0019] The step of performing a skin state detection operation on the target skin state semantic vector through the target skin state detection network and outputting a skin state detection result corresponding to the target skin image includes:

[0020] Using the plurality of the semantic mining units, performing a semantic mining operation on the target skin state semantic vector, and outputting a corresponding skin state semantic mining vector;

[0021] Performing a skin state detection operation according to the skin state semantic mining vector, and outputting a skin state detection result corresponding to the target skin image;

[0022] Wherein, the input vector of the first semantic mining unit among the plurality of sequentially connected semantic mining units includes the target skin state semantic vector;

[0023] For each of the semantic mining units except the last one among the multiple semantic mining units connected in sequence, an association mining operation is performed on the semantic mining vector corresponding to the semantic mining unit and the first skin state semantic vector, and the corresponding skin state association mining vector is output. Moreover, a linear mapping operation is performed on the skin state association mining vector according to the first mapping vector in the semantic mining unit, and a fusion operation is performed on the skin state association mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit, and a non-linear operation is performed on the vector formed by the fusion operation, and a linear mapping operation is performed on the vector formed by the non-linear operation according to the second mapping vector in the semantic mining unit, and the vector formed by the linear mapping operation is loaded into the next semantic mining unit of the semantic mining unit for semantic mining operation.

[0024] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, each of the semantic mining units carries its own first mapping vector and second mapping vector; wherein the skin state detection method based on image recognition further includes:

[0025] Extracting a plurality of training skin data, wherein each of the training skin data includes a training skin image, training skin description data and a skin state label;

[0026] According to the first part of the training skin data among the multiple training skin data, the target skin state detection network is trained in the first stage, wherein, in the first stage of training, the target mapping vector does not participate, so that the corresponding skin state associated semantic vector is directly used as the target skin state semantic vector, and the first mapping vector and the second mapping vector in each of the semantic mining units do not participate, so that the corresponding skin state associated mining vector is directly used as the skin state associated mining vector after the linear mapping operation, and the vector formed by the corresponding nonlinear operation is directly used as the vector formed by the linear mapping operation, or, multiple semantic mining units do not participate, and the corresponding target skin state semantic vector is directly used as the skin state semantic mining vector;

[0027] Maintaining the network parameters of the target skin state detection network that participate in the first stage of training unchanged, and configuring the target mapping vector, the first mapping vector and the second mapping vector of each semantic mining unit so that they are all configured as zero vectors or random vectors;

[0028] Based on the second part of the training skin data among the multiple training skin data, the target mapping vector and the first mapping vector and the second mapping vector of each semantic mining unit are trained to form the final target mapping vector, the first mapping vector and the second mapping vector.

[0029] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the steps of mining the skin description data semantic vector of the target skin description data, and performing an association mining operation on the skin description data semantic vector and the first skin state semantic vector to form a corresponding skin state association semantic vector include:

[0030] In a pre-built semantic vector library, a second skin state semantic vector matching the first skin state semantic vector is searched, wherein each historical skin state semantic vector in the semantic vector library is formed by performing semantic mining and aggregation operations based on other skin images;

[0031] In a pre-constructed skin state library, skin state detection data corresponding to the second skin state semantic vector is determined, wherein each historical skin state data in the skin state library is used to reflect the actual skin state corresponding to other corresponding skin images;

[0032] Merging the skin condition detection data and the target skin description data, and outputting corresponding merged skin description data;

[0033] Performing a semantic mining operation on the merged skin description data, and outputting a skin description data semantic vector corresponding to the target skin description data;

[0034] performing a vector connection operation on the second skin state semantic vector and the first skin state semantic vector, and outputting a corresponding skin state connection vector;

[0035] An association mining operation is performed on the skin description data semantic vector and the skin state connection vector to form a corresponding skin state associated semantic vector.

[0036] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the step of searching for a second skin state semantic vector matching the first skin state semantic vector in a pre-built semantic vector library comprises:

[0037] For each historical skin state semantic vector in the semantic vector library, determining a vector distance between the historical skin state semantic vector and the first skin state semantic vector;

[0038] In the semantic vector library, each historical skin state semantic vector whose vector distance with the first skin state semantic vector is less than a predetermined vector threshold distance is selected to form a corresponding candidate matching vector set;

[0039] When the candidate matching vector set belongs to a non-empty set, the candidate matching vector set is used as a target matching vector set, or, when the candidate matching vector set belongs to a non-empty set, a historical skin state semantic vector having a minimum vector distance with the first skin state semantic vector is assigned to the candidate matching vector set to form a corresponding target matching vector set;

[0040] Based on the historical skin state semantic vectors in the target matching vector set, a second skin state semantic vector matching the first skin state semantic vector is determined.

[0041] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the step of performing an association mining operation on the skin description data semantic vector and the skin state connection vector to form a corresponding skin state associated semantic vector includes:

[0042] Performing an association mining operation on a skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector, and outputting a corresponding local first association vector;

[0043] Performing an association mining operation on a skin description data local vector corresponding to the target skin description data in the skin description data semantic vector and the first skin state semantic vector in the skin state connection vector, and outputting a corresponding local second association vector;

[0044] Based on the local first association vector and the local second association vector, a corresponding skin state association semantic vector is formed.

[0045] In some preferred embodiments, in the above-mentioned skin state detection method based on image recognition, the step of performing an association mining operation on the skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector to output the corresponding local first association vector includes:

[0046] Performing a first spatial mapping operation on the skin description data semantic vector to form a corresponding skin data mapping vector;

[0047] Performing a second spatial mapping operation on the skin state connection vector to form a corresponding skin state mapping vector;

[0048] Performing an associative fusion operation on the skin data mapping vector and the skin state mapping vector after the row and column parameters are swapped to form a corresponding skin fusion associative vector;

[0049] According to a predetermined target vector, the skin fusion association vector is adjusted, and based on the skin fusion association vector after the adjustment operation, a corresponding correlation parameter distribution is determined, wherein the target vector is used to replace vector parameters other than the vector parameters to be processed in the skin fusion association vector, and the vector parameters to be processed are formed by: associating and fusing a skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector;

[0050] performing a third spatial mapping operation on the second skin state semantic vector to form a corresponding skin state semantic mapping vector;

[0051] An association fusion operation is performed on the association parameter distribution and the skin state semantic mapping vector to form a corresponding skin state association semantic vector.

[0052] An embodiment of the present invention also provides a skin condition detection system based on image recognition, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program, in order to implement the above-mentioned skin condition detection method based on image recognition.

[0053] The embodiment of the present invention provides a skin state detection method and system based on image recognition. First, a target skin image and target skin description data are obtained; secondly, the skin image semantic vector of at least two dimensions of the target skin image is mined, and the skin image semantic vector of at least two dimensions is aggregated to form a first skin state semantic vector; then, the skin description data semantic vector of the target skin description data is mined, and the skin description data semantic vector and the first skin state semantic vector are associated with the mining operation to form a skin state associated semantic vector; finally, through the target skin state detection network, the skin state associated semantic vector is subjected to skin state detection operation, and the skin state detection result is output. Based on the above method, since not only the two semantic vectors representing the target skin image and the target skin description data are associated with the mining operation, so that the semantic representation ability of the formed skin state associated semantic vector is better (such as the semantic information represented is richer), therefore, the reliability of the skin state detection operation based on the skin state associated semantic vector can be improved to a certain extent. In addition, since the semantic information mining of two dimensions is performed for the target skin image, the richness of the mined semantic vector can be further improved, thereby improving the problem of relatively low reliability of skin state detection existing in the prior art.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a structural block diagram of a skin condition detection system based on image recognition provided by an embodiment of the present invention.

[0056] Figure 2 A flowchart of the steps of the skin condition detection method based on image recognition provided in an embodiment of the present invention.

[0057] Figure 3 A schematic diagram of mining by multiple semantic mining units provided in an embodiment of the present invention.

[0058] Figure 4 A flowchart of other steps included in the skin condition detection method based on image recognition provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] like Figure 1 As shown, an embodiment of the present invention provides a skin condition detection system based on image recognition. Among them, the skin condition detection system based on image recognition may include a memory and a processor. In detail, the memory and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The memory may store at least one software function module (computer program) that can exist in the form of software or firmware. The processor can be used to execute the executable computer program stored in the memory, thereby realizing the skin condition detection method based on image recognition provided by the embodiment of the present invention (as described later).

[0062] Optionally, the memory may be, but not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0063] and, Figure 1 The structure shown is for illustration only. The skin condition detection system based on image recognition may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may include a communication unit for information exchange with other devices. In an alternative example, the skin condition detection system based on image recognition may be a server with data processing capabilities.

[0064] Combination Figure 2 The embodiment of the present invention further provides a skin condition detection method based on image recognition, which can be applied to the above-mentioned skin condition detection system based on image recognition. The method steps defined in the process related to the skin condition detection method based on image recognition can be implemented by the skin condition detection system based on image recognition (including the processor).

[0065] The following will Figure 2 The specific process shown is explained in detail.

[0066] Step S110, obtaining a target skin image and target skin description data.

[0067] In an embodiment of the present invention, the skin condition detection system based on image recognition can obtain a target skin image and target skin description data. The target skin description data is formed in response to an input operation of a skin condition detection user on the condition of the skin corresponding to the target skin image, and belongs to the description of the skin condition by the skin condition detection user. Exemplarily, the skin condition detection user can perform image acquisition on the target skin through an image acquisition device to form the target skin image, and the skin condition detection user can perform a conventional description of the condition of the target skin based on his or her professional knowledge to form the corresponding target skin description data (the specific form of expression can be text data or voice data). For example, the target skin description data can be color, shape, pore size, etc.

[0068] Step S120, mining skin image semantic vectors of at least two dimensions of the target skin image, and aggregating the skin image semantic vectors of at least two dimensions to form a first skin state semantic vector corresponding to the target skin image.

[0069] In an embodiment of the present invention, the skin state detection system based on image recognition can mine skin image semantic vectors of at least two dimensions of the target skin image, and aggregate the skin image semantic vectors of at least two dimensions to form a first skin state semantic vector corresponding to the target skin image. Exemplarily, the skin image semantic vector of the first dimension can be formed by performing a convolution operation on the three color channel values ​​of the target skin image, and the skin image semantic vector of the second dimension can be formed by performing a convolution operation on the contour lines in the target skin image. Specifically, some existing contour line detection technologies can be used to identify the contour lines in the target skin image, and then, each pixel point in the target skin image is assigned a value, such as assigning a value of 1 to the points on the contour line and assigning a value of 0 to other points. In this way, the corresponding contour line representation image can be obtained, and then, the contour line representation image can be convolved to obtain the skin image semantic vector of the second dimension. In addition, the specific manner of aggregating the skin image semantic vectors of at least two dimensions is not limited. For example, the skin image semantic vectors of at least two dimensions may be subjected to an association mining operation, or the skin image semantic vectors of at least two dimensions may be connected (such as splicing) to obtain a corresponding first skin state semantic vector, such as (skin image semantic vector 1, skin image semantic vector 2). Specifically, this may be achieved using a convolutional unit in a target skin state detection network.

[0070] Step S130, mining the skin description data semantic vector of the target skin description data, and performing an association mining operation on the skin description data semantic vector and the first skin state semantic vector to form a corresponding skin state association semantic vector.

[0071] In an embodiment of the present invention, the image recognition-based skin condition detection system can mine the skin description data semantic vector of the target skin description data, and perform association mining operations on the skin description data semantic vector and the first skin condition semantic vector to form a corresponding skin condition associated semantic vector. In this way, the skin condition associated semantic vector can carry both the semantic information of the skin description data semantic vector and the semantic information of the first skin condition semantic vector, thereby improving the representation capability of the semantic information.

[0072] Step S140, performing a skin state detection operation on the skin state associated semantic vector through a target skin state detection network, and outputting a skin state detection result corresponding to the target skin image.

[0073] In an embodiment of the present invention, the skin state detection system based on image recognition can perform skin state detection operations on the skin state associated semantic vector through a target skin state detection network, and output the skin state detection result corresponding to the target skin image. It should be noted that the output function of the target skin state detection network can be a binary classification function or a multi-classification function, and the specific content of the corresponding skin state detection result can also be different. For example, the skin state detection result that can be output based on the binary classification function is "good skin state, poor skin state" or "skin disease, no skin disease", etc., and the skin state detection result that can be output based on the multi-classification function is "excellent skin state, good skin state, general skin state, poor skin state, extremely poor skin state" or "acne, eczema, psoriasis, spots, freckles, pigmentation", etc. Alternatively, in other embodiments, the output function of the target skin state detection network can also be a regression function, so that the output is a continuous value, such as skin smoothness 0-10, etc.

[0074] Based on the above method, since not only the two semantic vectors representing the target skin image and the target skin description data are associated with each other and mined, the semantic representation ability of the formed skin state associated semantic vector is better (such as the represented semantic information is richer), the reliability of the skin state detection operation based on the skin state associated semantic vector can be improved to a certain extent. In addition, since two-dimensional semantic information mining is performed on the target skin image, the richness of the mined semantic vector can be further improved, thereby improving the relatively low reliability of skin state detection in the prior art.

[0075] In the above-mentioned step S130, the specific method of mining the corresponding skin state associated semantic vector is not limited. For example, in a specific embodiment, in order to make the mined skin state associated semantic vector have a more accurate and reliable semantic information representation capability, the above-mentioned step S130 may include step S131, step S132, step S133, step S134, step S135 and step S136, and the specific contents are described as follows.

[0076] Step S131: searching a pre-built semantic vector library for a second skin state semantic vector that matches the first skin state semantic vector.

[0077] In an embodiment of the present invention, a second skin state semantic vector matching the first skin state semantic vector can be found in a pre-built semantic vector library, wherein each historical skin state semantic vector in the semantic vector library is formed by performing semantic mining and aggregation operations based on other skin images (the specific processing process is as described in the aforementioned step S120).

[0078] Step S132: determining skin state detection data corresponding to the second skin state semantic vector in a pre-built skin state library.

[0079] In an embodiment of the present invention, skin state detection data corresponding to the second skin state semantic vector can be determined in a pre-constructed skin state library, wherein each historical skin state data in the skin state library is used to reflect the actual skin state corresponding to the corresponding other skin images (e.g., using a target skin state detection network, performing a skin state detection operation on the corresponding skin state associated semantic vector, and outputting the corresponding actual skin state).

[0080] Step S133, merging the skin condition detection data and the target skin description data, and outputting corresponding merged skin description data.

[0081] In an embodiment of the present invention, the skin condition detection data and the target skin description data can be merged to output corresponding merged skin description data; exemplarily, the skin condition detection data and the target skin description data can be merged into one text data, wherein, in the text data, the skin condition detection data and the target skin description data can be separated by a separator to avoid distinguishing different data.

[0082] Step S134, performing a semantic mining operation on the merged skin description data, and outputting a skin description data semantic vector corresponding to the target skin description data.

[0083] In an embodiment of the present invention, after the merged skin description data is formed, a semantic mining operation can be performed on the merged skin description data to output a skin description data semantic vector corresponding to the target skin description data. Exemplarily, the merged skin description data can be first segmented by a trained word embedding model, and then each word obtained can be word embedded to obtain a corresponding word embedding vector, and finally, each word embedding vector can be spliced ​​to obtain a corresponding skin description data semantic vector (or, the spliced ​​vector can be convoluted and pooled to obtain a corresponding skin description data semantic vector). Based on this, the skin description data semantic vector not only carries the descriptive semantic information of the skin state detection user, but also carries the semantic information of the matching skin state detection data, thereby achieving further constraints on the skin description data semantic vector, that is, improving the representation ability of the skin description data semantic vector. For example, the merged skin description data can be "dark yellow color, large pores, poor skin smoothness", and the corresponding word embedding vectors can be:

[0084] color: [0.12, 0.45, -0.33, ...];

[0085] Dark yellow: [-0.25, 0.18, 0.29, ...];

[0086] Pores: [0.33, -0.27, 0.11, ...];

[0087] Coarse: [-0.18, 0.22, -0.14, ...];

[0088] Skin: [0.27, -0.12, 0.39, ...];

[0089] Smoothness: [-0.21, 0.34, -0.29, ...];

[0090] Difference: [-0.19, 0.41, -0.35, ...];

[0091] The skin description data semantic vector may be:

[0092] {[0.12,0.45,-0.33,...]; [-0.25,0.18,0.29,...]; [0.33,-0.27,0.11,...]; [-0.18,0.22,-0.14,...]; [0.27,-0.12,0.39,...]; [-0.21,0.34,-0.29,...]; [-0.19,0.41,-0.35,...];}

[0093] It should be noted that the above-mentioned word embedding vectors are only examples, and the word embedding process may be different depending on the training data. In addition, the specific dimensions of each word embedding vector (such as 100 dimensions, 300 dimensions) may also vary depending on the training data and parameters of the word embedding model.

[0094] Step S135, performing a vector connection operation on the second skin state semantic vector and the first skin state semantic vector, and outputting a corresponding skin state connection vector.

[0095] In an embodiment of the present invention, a vector connection operation may be performed on the second skin state semantic vector and the first skin state semantic vector to output a corresponding skin state connection vector. Exemplarily, the second skin state semantic vector and the first skin state semantic vector may be concatenated together, such as (second skin state semantic vector, first skin state semantic vector). In addition, if the second skin state semantic vector is 100-dimensional and the first skin state semantic vector is 200-dimensional, the corresponding skin state connection vector is 300-dimensional.

[0096] Step S136, performing an association mining operation on the skin description data semantic vector and the skin state connection vector to form a corresponding skin state associated semantic vector.

[0097] In an embodiment of the present invention, the skin description data semantic vector and the skin state connection vector may be subjected to an association mining operation to form a corresponding skin state associated semantic vector, so that full fusion of semantic information can be achieved between the skin description data semantic vector and the skin state connection vector, thereby obtaining a more semantically rich and accurate skin state associated semantic vector.

[0098] In the above-mentioned step S131, the specific method of finding the second skin state semantic vector matching the first skin state semantic vector is not limited. For example, in a specific embodiment, in order to make the found second skin state semantic vector provide more reliable constraints in subsequent steps, the above-mentioned step S131 may include step S131a, step S131b, step S131c and step S131d, and the specific contents are described as follows.

[0099] Step S131a: for each historical skin state semantic vector in the semantic vector library, determine the vector distance between the historical skin state semantic vector and the first skin state semantic vector.

[0100] In an embodiment of the present invention, for each historical skin state semantic vector in the semantic vector library, the vector distance between the historical skin state semantic vector and the first skin state semantic vector can be determined. For example, the cosine distance between the historical skin state semantic vector and the first skin state semantic vector can be calculated as the corresponding vector distance.

[0101] Step S131b, in the semantic vector library, each historical skin state semantic vector whose vector distance with the first skin state semantic vector is less than a predetermined vector threshold distance is selected to form a corresponding candidate matching vector set.

[0102] In an embodiment of the present invention, each historical skin state semantic vector whose vector distance with the first skin state semantic vector is less than a predetermined vector threshold distance (the specific size of the vector threshold distance can be selected according to actual needs and is not specifically limited here) can be screened out in the semantic vector library to form a corresponding candidate matching vector set, that is, some relatively similar historical skin state semantic vectors are screened out as candidates.

[0103] Step S131c, when the candidate matching vector set belongs to a non-empty set, the candidate matching vector set is used as the target matching vector set, or, when the candidate matching vector set belongs to a non-empty set, a historical skin state semantic vector having a minimum vector distance with the first skin state semantic vector is assigned to the candidate matching vector set to form a corresponding target matching vector set.

[0104] In an embodiment of the present invention, when the candidate matching vector set belongs to a non-empty set, the candidate matching vector set can be directly used as a target matching vector set. Alternatively, when the candidate matching vector set belongs to a non-empty set, a historical skin state semantic vector having a minimum vector distance with the first skin state semantic vector can be assigned to the candidate matching vector set to form a corresponding target matching vector set.

[0105] Step S131d: determining a second skin state semantic vector matching the first skin state semantic vector based on the historical skin state semantic vectors in the target matching vector set.

[0106] In an embodiment of the present invention, a second skin state semantic vector matching the first skin state semantic vector can be determined based on the historical skin state semantic vector in the target matching vector set. Exemplarily, any one of the historical skin state semantic vectors in the target matching vector set can be used as the second skin state semantic vector, or the historical skin state semantic vectors in the target matching vector set can be fused, such as by weighted summing according to the corresponding vector distances to obtain the second skin state semantic vector, wherein, in the weighted summing process, the weighting coefficient and the vector distance can have a negatively correlated corresponding relationship.

[0107] In the above-mentioned step S136, the specific manner of performing the association mining operation on the skin description data semantic vector and the skin state connection vector is not limited. For example, in a specific embodiment, in order to make the mined skin state associated semantic vector have a more accurate and reliable semantic information representation capability, the above-mentioned step S136 may include step S136a, step S136b and step S136c, and the specific contents are described as follows.

[0108] Step S136a, performing an association mining operation on the skin description data local vector corresponding to the skin condition detection data in the skin description data semantic vector and the second skin condition semantic vector in the skin condition connection vector, and outputting a corresponding local first association vector.

[0109] In an embodiment of the present invention, the local vector of skin description data corresponding to the skin condition detection data in the skin description data semantic vector (such as the concatenation result of each word embedding vector corresponding to the skin condition detection data) and the second skin condition semantic vector in the skin condition connection vector can be associated with a mining operation to output a corresponding local first association vector. In this way, the local vector of skin description data corresponding to the skin condition detection data and the second skin condition semantic vector can be fully integrated to obtain a semantically rich local first association vector.

[0110] Step S136b, performing an association mining operation on the skin description data local vector corresponding to the target skin description data in the skin description data semantic vector and the first skin state semantic vector in the skin state connection vector, and outputting a corresponding local second association vector.

[0111] In an embodiment of the present invention, the skin description data local vector corresponding to the target skin description data in the skin description data semantic vector (such as the concatenation result of each word embedding vector corresponding to the target skin description data) and the first skin state semantic vector in the skin state connection vector can be associated with a mining operation to output a corresponding local second association vector. In this way, the skin description data local vector corresponding to the target skin description data and the first skin state semantic vector can be fully fused to obtain a semantically rich local second association vector.

[0112] Step S136c: forming a corresponding skin state associated semantic vector based on the local first associated vector and the local second associated vector.

[0113] In an embodiment of the present invention, a corresponding skin state associated semantic vector can be formed based on the local first association vector and the local second association vector; exemplarily, the local first association vector and the local second association vector can be spliced ​​together to form a corresponding skin state associated semantic vector, or, the local first association vector and the local second association vector can be subjected to an association mining operation to output a corresponding skin state associated semantic vector.

[0114] In the above step S136a, the specific manner of performing the association mining operation is not limited. For example, in a specific implementation, in order to make the mined local first association vector have a more accurate and reliable semantic information representation capability, the above S136a may include:

[0115] First, a first spatial mapping operation may be performed on the semantic vector of the skin description data to form a corresponding skin data mapping vector; illustratively, the semantic vector of the skin description data may be multiplied by a parameter matrix formed through training to obtain the skin data mapping vector;

[0116] Secondly, a second spatial mapping operation may be performed on the skin state connection vector to form a corresponding skin state mapping vector; illustratively, the skin state connection vector may be multiplied by a parameter matrix formed through training to obtain the skin state mapping vector;

[0117] Then, the skin data mapping vector and the skin state mapping vector after the row and column parameters are swapped (that is, the row vector parameters are used as column vector parameters) can be associated and fused (such as multiplied) to form a corresponding skin fusion association vector;

[0118] Afterwards, the skin fusion association vector can be adjusted according to a predetermined target vector, and the corresponding correlation parameter distribution can be determined based on the skin fusion association vector after the adjustment operation, wherein the target vector is used to replace vector parameters other than the vector parameters to be processed in the skin fusion association vector, and the vector parameters to be processed are formed in the following manner: a skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector are associated and fused (such as multiplied) to form; exemplarily, the target vector is a vector with the same shape and size as the skin fusion association vector, and the target vector The parameter at the position corresponding to the vector parameter to be processed is 1, and the parameters at other positions are negative infinity, and the target vector and the skin fusion association vector can be dot-multiplied to obtain the skin fusion association vector after the adjustment operation; in addition, the skin fusion association vector after the adjustment operation can be directly used as the corresponding correlation parameter distribution, or the parameters in the skin fusion association vector after the adjustment operation can be normalized to obtain the corresponding correlation parameter distribution. For example, the parameters in the skin fusion association vector after the adjustment operation can be used as the exponent of the natural base for corresponding calculation to obtain the parameters in the correlation parameter distribution. In this way, for the negative infinite parameter, it can be zero after calculation, so that the correlation parameter between two unrelated parameters is zero;

[0119] Further, a third spatial mapping operation may be performed on the second skin state semantic vector to form a corresponding skin state semantic mapping vector. For example, the second skin state semantic vector may be multiplied by a parameter matrix formed through training to obtain the skin state semantic mapping vector. It should be noted that the parameter matrices corresponding to the above-mentioned spatial mapping operations may be different.

[0120] Finally, an association fusion operation (such as multiplication) may be performed on the association parameter distribution and the skin state semantic mapping vector to form a corresponding skin state association semantic vector.

[0121] In the above-mentioned step S140, the specific manner of performing the skin state detection operation on the skin state associated semantic vector is not limited. For example, in a specific implementation, in order to make the output skin state detection result more accurate and reliable, the above-mentioned step S140 may include step S141 and step S142, and the specific contents are described as follows.

[0122] Step S141, updating the skin state associated semantic vector according to the target mapping vector to form a corresponding target skin state semantic vector.

[0123] In an embodiment of the present invention, the skin state associated semantic vector can be updated according to the target mapping vector to form a corresponding target skin state semantic vector. The target mapping vector is formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training, while maintaining other parameters of the target skin state detection network unchanged. In this way, the richness of the obtained semantic vector can be improved.

[0124] Step S142, performing a skin state detection operation on the target skin state semantic vector through the target skin state detection network, and outputting a skin state detection result corresponding to the target skin image.

[0125] In an embodiment of the present invention, after obtaining the target skin state semantic vector, the target skin state detection network may be used to perform a skin state detection operation on the target skin state semantic vector to output a skin state detection result corresponding to the target skin image.

[0126] In the above-mentioned step S141, the specific method of updating the skin state associated semantic vector is not limited. For example, in a specific implementation, in order to make the mined target skin state semantic vector have richer semantic information representation capabilities, the above-mentioned step S141 may include step S141a, step S141b, step S141c and step S141d, and the specific contents are described as follows.

[0127] Step S141a: performing a linear mapping operation on the skin state associated semantic vector according to the first mapping vector included in the target mapping vector to form a corresponding skin state mapping semantic vector.

[0128] In an embodiment of the present invention, a linear mapping operation may be performed on the skin state associated semantic vector according to the first mapping vector included in the target mapping vector to form a corresponding skin state mapping semantic vector. Exemplarily, the first mapping vector and the skin state associated semantic vector may be multiplied to obtain a corresponding skin state mapping semantic vector.

[0129] Step S141b, performing a fusion operation on the skin state mapping semantic vector and the skin description data semantic vector, and outputting a corresponding skin state fusion semantic vector.

[0130] In an embodiment of the present invention, the skin state mapping semantic vector and the skin description data semantic vector may be fused to output a corresponding skin state fused semantic vector. Exemplarily, the skin state mapping semantic vector and the skin description data semantic vector may be superimposed to obtain a corresponding skin state fused semantic vector.

[0131] Step S141c, performing a nonlinear operation on the skin state fusion semantic vector, and outputting a corresponding skin state nonlinear semantic vector.

[0132] In an embodiment of the present invention, the skin state fusion semantic vector may be subjected to a nonlinear operation to output a corresponding skin state nonlinear semantic vector. That is, through the nonlinear operation, nonlinear information may be introduced, thereby improving the richness of the skin state nonlinear semantic vector. Exemplarily, the skin state fusion semantic vector may be first subjected to an upsampling process, and then the result of the upsampling process may be activated by an activation function (such as ReLU, etc.), and then the result of the activation process may be subjected to a downsampling process, so that the skin state nonlinear semantic vector and the skin state fusion semantic vector have the same size.

[0133] Step S141d: performing a linear mapping operation on the skin state nonlinear semantic vector according to the second mapping vector included in the target mapping vector to form a corresponding target skin state semantic vector.

[0134] In an embodiment of the present invention, a linear mapping operation can be performed on the skin state nonlinear semantic vector according to the second mapping vector included in the target mapping vector to form a corresponding target skin state semantic vector. Exemplarily, the second mapping vector and the skin state nonlinear semantic vector can be multiplied to obtain a corresponding target skin state semantic vector. Wherein, the second mapping vector and the first mapping vector are formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training, while maintaining other parameters of the target skin state detection network unchanged. The specific process is described below.

[0135] In the above step S142, the specific manner of performing the skin state detection operation on the target skin state semantic vector is not limited. For example, in a specific implementation, in order to make the output skin state detection result more accurate and reliable, Figure 3 The target skin condition detection network includes a plurality of semantic mining units connected in sequence. The above-mentioned step S142 may include step S142a and step S142b, and the specific contents are as follows.

[0136] Step S142a, using the plurality of semantic mining units, performs semantic mining operations on the target skin condition semantic vector, and outputs a corresponding skin condition semantic mining vector.

[0137] In the embodiment of the present invention, multiple semantic mining units can be used to perform semantic mining operations on the target skin state semantic vector and output the corresponding skin state semantic mining vector. Among them, since the multiple semantic mining units are connected in sequence, that is, the output vector of the previous semantic mining unit will be used as the output vector of the next semantic mining unit, so that the output vectors of different semantic mining units have different depths, that is, the semantic information of the target skin state semantic vector at different levels can be mined, and the skin state detection result can be output more accurately.

[0138] Step S142b, performing skin condition detection operation according to the skin condition semantic mining vector, and outputting the skin condition detection result corresponding to the target skin image.

[0139] In an embodiment of the present invention, after obtaining the skin state semantic mining vector, a skin state detection operation can be performed based on the skin state semantic mining vector to output the skin state detection result corresponding to the target skin image; exemplarily, the skin state semantic mining vector can be fully connected, and then the skin state semantic mining vector after the full connection processing can be output, such as by processing through the output function described in the previous text.

[0140] In the above step S142a, the specific manner of performing semantic mining operation on the target skin state semantic vector is not limited. For example, in a specific implementation, in order to make the mined skin state semantic mining vector have more accurate and reliable semantic information representation capability, the above step S142a may include:

[0141] For each of the semantic mining units except the last one among the multiple semantic mining units connected in sequence, an association mining operation is performed on the semantic mining vector corresponding to the semantic mining unit and the first skin state semantic vector (as described above), and the corresponding skin state association mining vector is output. Moreover, according to the first mapping vector in the semantic mining unit, a linear mapping operation is performed on the skin state association mining vector (as described above), and a fusion operation is performed on the skin state association mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit (as described above), and a non-linearization operation is performed on the vector formed by the fusion operation (as described above), and according to the second mapping vector in the semantic mining unit, a linear mapping operation is performed on the vector formed by the non-linearization operation, and the vector formed by the linear mapping operation is loaded into the next semantic mining unit of the semantic mining unit for semantic mining operation, wherein the input vector of the first semantic mining unit among the multiple semantic mining units connected in sequence includes the target skin state semantic vector.

[0142] For the above step S142a, for example:

[0143] In the first semantic mining unit, the target skin state semantic vector may be focused to obtain a semantic mining vector corresponding to the first semantic mining unit (for example, the row and column parameters of the target skin state semantic vector may be swapped to form a swapped vector, and then the target skin state semantic vector and the swapped vector may be multiplied to obtain a corresponding focus correlation parameter distribution, and then the focus correlation parameter distribution and the target skin state semantic vector may be multiplied to obtain a corresponding semantic mining vector). Then, the semantic mining vector corresponding to the first semantic mining unit and the first skin state semantic vector may be associated with each other to output a corresponding skin state associated mining vector, and a linear mapping operation may be performed on the skin state associated mining vector according to the first mapping vector in the semantic mining unit, and a fusion operation may be performed on the skin state associated mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit, and a nonlinearization operation may be performed on the vector formed by the fusion operation, and a linear mapping operation may be performed on the vector formed by the nonlinearization operation according to the second mapping vector in the semantic mining unit to obtain an output vector corresponding to the first semantic mining unit;

[0144] In the second semantic mining unit, the output vector corresponding to the first semantic mining unit can be focused to obtain the semantic mining vector corresponding to the second semantic mining unit. Then, the semantic mining vector corresponding to the second semantic mining unit and the first skin state semantic vector can be associated with each other to output the corresponding skin state associated mining vector. According to the first mapping vector in the semantic mining unit, a linear mapping operation is performed on the skin state associated mining vector. Furthermore, a fusion operation is performed on the skin state associated mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit. A nonlinearization operation is performed on the vector formed by the fusion operation. Furthermore, according to the second mapping vector in the semantic mining unit, a linear mapping operation is performed on the vector formed by the nonlinearization operation to obtain the output vector corresponding to the second semantic mining unit.

[0145] In the last semantic mining unit, the output vector corresponding to the previous semantic mining unit (the penultimate semantic mining unit) can be focused to obtain the semantic mining vector corresponding to the last semantic mining unit. Then, the semantic mining vector corresponding to the last semantic mining unit and the first skin state semantic vector can be associated with each other to output the corresponding skin state associated mining vector. Furthermore, a linear mapping operation can be performed on the skin state associated mining vector based on the first mapping vector in the semantic mining unit. Furthermore, a fusion operation can be performed on the skin state associated mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit. A nonlinearization operation can be performed on the vector formed by the fusion operation. Furthermore, a linear mapping operation can be performed on the vector formed by the nonlinearization operation based on the second mapping vector in the semantic mining unit to obtain the output vector corresponding to the last semantic mining unit, wherein the output vector can be used as the corresponding skin state semantic mining vector.

[0146] On the basis of the above content, it is also necessary to explain the first mapping vector and the second mapping vector, wherein each of the semantic mining units carries its own first mapping vector and second mapping vector. Figure 4 The skin condition detection method based on image recognition may further include step S150, step S160, step S170 and step S180, the specific contents of which are described below and may be implemented by the skin condition detection system based on image recognition.

[0147] Step S150, extracting a plurality of training skin data.

[0148] In an embodiment of the present invention, the skin condition detection system based on image recognition can extract a plurality of training skin data (also referred to as sample data, used in training). Each of the training skin data includes a training skin image (corresponding to the target skin image), training skin description data (corresponding to the target skin description data) and a skin condition label (i.e., the actual skin condition, which can be formed by evaluation and annotation by corresponding experts).

[0149] Step S160, performing a first stage of training on the target skin condition detection network according to the first part of the training skin data among the plurality of training skin data.

[0150] In an embodiment of the present invention, the skin state detection system based on image recognition can perform the first stage of training on the target skin state detection network based on the first part of the training skin data among the multiple training skin data (which can be a part of the data randomly selected from the multiple training skin data). In the first stage of training, the target mapping vector does not participate, so that the corresponding skin state associated semantic vector is directly used as the target skin state semantic vector, and the first mapping vector and the second mapping vector in each of the semantic mining units do not participate, so that the corresponding skin state associated mining vector is directly used as the skin state associated mining vector after the linear mapping operation, and the vector formed by the corresponding nonlinear operation is directly used as the vector formed by the linear mapping operation, or, multiple semantic mining units do not participate, and the corresponding target skin state semantic vector is directly used as the skin state semantic mining vector. Among them, "not participating" means that in the corresponding processing process of the aforementioned steps S110-step S140, the corresponding input is directly used as the output, so that the corresponding vector or unit does not participate in the operation. For example, in step S141, the skin state associated semantic vector will be updated according to the target mapping vector to form the corresponding target skin state semantic vector. In this way, the skin state associated semantic vector can be directly used as the target skin state semantic vector, so that the target mapping vector does not participate in the operation. In addition, after performing a skin state detection operation based on the target skin state detection network and obtaining the corresponding training skin state detection result, the corresponding network parameters (excluding the above-mentioned parameters that do not participate in the operation) can be adjusted and updated based on the error between the training skin state detection result and the skin state label (the error calculation method is not limited and can be selected according to actual needs) until the error converges, and the first stage of training is completed.

[0151] Step S170, maintain the network parameters participating in the first stage of training in the target skin condition detection network unchanged, and configure the target mapping vector, the first mapping vector and the second mapping vector of each semantic mining unit so that they are all configured as zero vectors or random vectors.

[0152] In an embodiment of the present invention, after completing the first stage of training, the image recognition-based skin condition detection system can maintain the network parameters of the target skin condition detection network that participated in the first stage of training unchanged, and configure the target mapping vector, each semantic mining unit's own first mapping vector and second mapping vector so that they are all configured as a zero vector or a random vector (i.e., a randomly generated vector).

[0153] Step S180, based on the second part of the training skin data among the multiple training skin data, the target mapping vector and the first mapping vector and the second mapping vector of each semantic mining unit are trained to form the final target mapping vector, the first mapping vector and the second mapping vector.

[0154] In an embodiment of the present invention, the skin state detection system based on image recognition can train the target mapping vector and the first mapping vector and the second mapping vector of each semantic mining unit according to the training skin data of the second part of the plurality of training skin data (which can be all the training skin data other than the training skin data of the first part), so as to form the final target mapping vector, the first mapping vector and the second mapping vector. That is to say, in the training process of the second stage, the network parameters participating in the training of the first stage and the network parameters not participating in the training of the first stage will participate in the calculation (that is, all network parameters are parameterized, corresponding to the above-mentioned steps S110-S140), so as to obtain the corresponding training skin state detection result, and then, based on the error between the training skin state detection result and the skin state label, the target mapping vector and the first mapping vector and the second mapping vector of each semantic mining unit can be trained (that is, the parameters other than the parameters trained in the first stage) and adjusted and updated until the error converges, and then the training of the second stage is completed.

[0155] In summary, the present invention provides a skin state detection method and system based on image recognition. First, a target skin image and target skin description data are obtained; secondly, the skin image semantic vector of at least two dimensions of the target skin image is mined, and the skin image semantic vector of at least two dimensions is aggregated to form a first skin state semantic vector; then, the skin description data semantic vector of the target skin description data is mined, and the skin description data semantic vector and the first skin state semantic vector are associated with the mining operation to form a skin state associated semantic vector; finally, through the target skin state detection network, the skin state associated semantic vector is subjected to skin state detection operation, and the skin state detection result is output. Based on the above method, since not only the two semantic vectors representing the target skin image and the target skin description data are associated with the mining operation, so that the semantic representation ability of the formed skin state associated semantic vector is better (such as the semantic information represented is richer), therefore, the reliability of the skin state detection operation based on the skin state associated semantic vector can be improved to a certain extent. In addition, since two dimensions of semantic information mining are performed for the target skin image, the richness of the mined semantic vector can be further improved, thereby improving the problem of relatively low reliability of skin state detection in the prior art.

[0156] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0158] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A skin condition detection method based on image recognition, characterized in that: include: Obtaining a target skin image and target skin description data, wherein the target skin description data is formed in response to an input operation of a skin condition detection user on a skin condition corresponding to the target skin image, and belongs to a description of a skin condition by the skin condition detection user; Mining skin image semantic vectors of at least two dimensions of the target skin image, and aggregating the skin image semantic vectors of at least two dimensions to form a first skin state semantic vector corresponding to the target skin image; Mining out a skin description data semantic vector of the target skin description data, and performing an association mining operation on the skin description data semantic vector and the first skin state semantic vector to form a corresponding skin state association semantic vector; Through the target skin state detection network, a skin state detection operation is performed on the skin state associated semantic vector, and a skin state detection result corresponding to the target skin image is output.

2. The skin condition detection method based on image recognition according to claim 1, characterized in that: The step of performing a skin state detection operation on the skin state associated semantic vector through a target skin state detection network and outputting a skin state detection result corresponding to the target skin image comprises: According to the target mapping vector, the skin state associated semantic vector is updated to form a corresponding target skin state semantic vector, wherein the target mapping vector is formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training while maintaining other parameters of the target skin state detection network unchanged; Through the target skin state detection network, a skin state detection operation is performed on the target skin state semantic vector, and a skin state detection result corresponding to the target skin image is output.

3. The skin condition detection method based on image recognition according to claim 2, characterized in that: The step of updating the skin state associated semantic vector according to the target mapping vector to form a corresponding target skin state semantic vector comprises: According to the first mapping vector included in the target mapping vector, a linear mapping operation is performed on the skin state associated semantic vector to form a corresponding skin state mapping semantic vector; Performing a fusion operation on the skin state mapping semantic vector and the skin description data semantic vector, and outputting a corresponding skin state fusion semantic vector; Performing a nonlinear operation on the skin state fusion semantic vector, and outputting a corresponding skin state nonlinear semantic vector; According to the second mapping vector included in the target mapping vector, a linear mapping operation is performed on the skin state nonlinear semantic vector to form a corresponding target skin state semantic vector, wherein the second mapping vector and the first mapping vector are both formed by performing a second stage of training after the target skin state detection network has undergone the first stage of training while maintaining other parameters of the target skin state detection network unchanged.

4. The skin condition detection method based on image recognition according to claim 2, characterized in that: The target skin state detection network includes a plurality of semantic mining units connected in sequence; The step of performing a skin state detection operation on the target skin state semantic vector through the target skin state detection network and outputting a skin state detection result corresponding to the target skin image includes: Using the plurality of the semantic mining units, performing a semantic mining operation on the target skin state semantic vector, and outputting a corresponding skin state semantic mining vector; Performing a skin state detection operation according to the skin state semantic mining vector, and outputting a skin state detection result corresponding to the target skin image; Wherein, the input vector of the first semantic mining unit among the plurality of sequentially connected semantic mining units includes the target skin state semantic vector; For each of the semantic mining units except the last one among the multiple semantic mining units connected in sequence, an association mining operation is performed on the semantic mining vector corresponding to the semantic mining unit and the first skin state semantic vector, and the corresponding skin state association mining vector is output. Moreover, a linear mapping operation is performed on the skin state association mining vector according to the first mapping vector in the semantic mining unit, and a fusion operation is performed on the skin state association mining vector after the linear mapping operation and the semantic mining vector corresponding to the semantic mining unit, and a non-linear operation is performed on the vector formed by the fusion operation, and a linear mapping operation is performed on the vector formed by the non-linear operation according to the second mapping vector in the semantic mining unit, and the vector formed by the linear mapping operation is loaded into the next semantic mining unit of the semantic mining unit for semantic mining operation.

5. The skin condition detection method based on image recognition according to claim 4, characterized in that: Each of the semantic mining units carries its own first mapping vector and second mapping vector; wherein the skin state detection method based on image recognition also includes: Extracting a plurality of training skin data, wherein each of the training skin data includes a training skin image, training skin description data and a skin state label; According to the first part of the training skin data among the multiple training skin data, the target skin state detection network is trained in the first stage, wherein, in the first stage of training, the target mapping vector does not participate, so that the corresponding skin state associated semantic vector is directly used as the target skin state semantic vector, and the first mapping vector and the second mapping vector in each of the semantic mining units do not participate, so that the corresponding skin state associated mining vector is directly used as the skin state associated mining vector after the linear mapping operation, and the vector formed by the corresponding nonlinear operation is directly used as the vector formed by the linear mapping operation, or, multiple semantic mining units do not participate, and the corresponding target skin state semantic vector is directly used as the skin state semantic mining vector; Maintaining the network parameters of the target skin state detection network that participate in the first stage of training unchanged, and configuring the target mapping vector, the first mapping vector and the second mapping vector of each semantic mining unit so that they are all configured as zero vectors or random vectors; Based on the second part of the training skin data among the multiple training skin data, the target mapping vector and the first mapping vector and the second mapping vector of each semantic mining unit are trained to form the final target mapping vector, the first mapping vector and the second mapping vector.

6. The skin condition detection method based on image recognition according to claim 1, characterized in that: The steps of mining the skin description data semantic vector of the target skin description data, and performing an association mining operation on the skin description data semantic vector and the first skin state semantic vector to form a corresponding skin state associated semantic vector include: In a pre-built semantic vector library, a second skin state semantic vector matching the first skin state semantic vector is searched, wherein each historical skin state semantic vector in the semantic vector library is formed by performing semantic mining and aggregation operations based on other skin images; In a pre-constructed skin state library, skin state detection data corresponding to the second skin state semantic vector is determined, wherein each historical skin state data in the skin state library is used to reflect the actual skin state corresponding to other corresponding skin images; Merging the skin condition detection data and the target skin description data, and outputting corresponding merged skin description data; Performing a semantic mining operation on the merged skin description data, and outputting a skin description data semantic vector corresponding to the target skin description data; performing a vector connection operation on the second skin state semantic vector and the first skin state semantic vector, and outputting a corresponding skin state connection vector; An association mining operation is performed on the skin description data semantic vector and the skin state connection vector to form a corresponding skin state associated semantic vector.

7. The skin condition detection method based on image recognition according to claim 6, characterized in that: The step of searching for a second skin state semantic vector matching the first skin state semantic vector in a pre-built semantic vector library comprises: For each historical skin state semantic vector in the semantic vector library, determining a vector distance between the historical skin state semantic vector and the first skin state semantic vector; In the semantic vector library, each historical skin state semantic vector whose vector distance with the first skin state semantic vector is less than a predetermined vector threshold distance is selected to form a corresponding candidate matching vector set; When the candidate matching vector set belongs to a non-empty set, the candidate matching vector set is used as a target matching vector set, or, when the candidate matching vector set belongs to a non-empty set, a historical skin state semantic vector having a minimum vector distance with the first skin state semantic vector is assigned to the candidate matching vector set to form a corresponding target matching vector set; Based on the historical skin state semantic vectors in the target matching vector set, a second skin state semantic vector matching the first skin state semantic vector is determined.

8. The skin condition detection method based on image recognition according to claim 6, characterized in that: The step of performing an association mining operation on the skin description data semantic vector and the skin state connection vector to form a corresponding skin state associated semantic vector comprises: Performing an association mining operation on a skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector, and outputting a corresponding local first association vector; Performing an association mining operation on a skin description data local vector corresponding to the target skin description data in the skin description data semantic vector and the first skin state semantic vector in the skin state connection vector, and outputting a corresponding local second association vector; Based on the local first association vector and the local second association vector, a corresponding skin state association semantic vector is formed.

9. The skin condition detection method based on image recognition according to claim 8, characterized in that: The step of performing an association mining operation on the skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector to output a corresponding local first association vector comprises: Performing a first spatial mapping operation on the skin description data semantic vector to form a corresponding skin data mapping vector; Performing a second spatial mapping operation on the skin state connection vector to form a corresponding skin state mapping vector; Performing an associative fusion operation on the skin data mapping vector and the skin state mapping vector after the row and column parameters are swapped to form a corresponding skin fusion associative vector; According to a predetermined target vector, the skin fusion association vector is adjusted, and based on the skin fusion association vector after the adjustment operation, a corresponding correlation parameter distribution is determined, wherein the target vector is used to replace vector parameters other than the vector parameters to be processed in the skin fusion association vector, and the vector parameters to be processed are formed by: associating and fusing a skin description data local vector corresponding to the skin state detection data in the skin description data semantic vector and the second skin state semantic vector in the skin state connection vector; performing a third spatial mapping operation on the second skin state semantic vector to form a corresponding skin state semantic mapping vector; An association fusion operation is performed on the association parameter distribution and the skin state semantic mapping vector to form a corresponding skin state association semantic vector.

10. A skin condition detection system based on image recognition, characterized in that: It comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the skin condition detection method based on image recognition as described in any one of claims 1 to 9.

Images