Skin surface microparticle detection method and system

Abstract

The invention discloses a skin surface microparticle detection method and system, and relates to the technical field of optical detection, and the method comprises the steps: obtaining the relative position relation characteristics of various microparticles distributed on the skin surface, and the two-dimensional image characteristics and spectral characteristics of various microparticles; collecting a two-dimensional image and a hyperspectral image; identifying a microparticle type corresponding to each microparticle image in the two-dimensional image and outputting a first identification result; micro-particle types corresponding to all micro-particle images in the hyperspectral image are judged, and a second recognition result is output; marking the same microparticle image based on the first recognition result and the second recognition result, and if the category marks are the same, determining the category and performing associated storage on the category and the position coordinates; if the category marks are different, the category of the current microparticle image is judged and output according to the relative position relation characteristics, the two-dimensional image and the hyperspectral image are combined, recognition and judgment are conducted from the microparticle image characteristics and the chemical components of substances, and the recognition precision of the microparticles on the skin surface is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of optical detection technology, and more particularly to a method and system for detecting microparticles on the skin surface. Background Art

[0002] The detection of microparticles on the skin surface, such as the detection and identification of microparticles such as facial sebum, keratinous debris, microorganisms or exogenous particles, not only has important application value in the fields of skin health monitoring and environmental exposure assessment, but also plays an important role in the evaluation of the cleansing effect of facial cleansers, the evaluation of the facial protection effect of skin care products, and the detection of cosmetic residues.

[0003] Current detection and identification of microparticles on the skin's surface primarily relies on high-definition cameras capturing high-definition images of the skin's surface. These images are then counted using either manual or intelligent image recognition techniques to determine the number and type of microparticles present. In practice, due to the complex composition and numerous causes of microparticles on the skin's surface, current detection and identification methods are not only inefficient but also unable to analyze the specific composition of microparticles or the underlying material, reducing the accuracy of microparticle detection and identification. This also compromises the reliability of diagnostic assessments based on these results. Summary of the Invention

[0004] In response to the problems of low accuracy and efficiency in the detection and identification of microparticles on the skin surface in the prior art, the first object of this application is to provide a method for detecting microparticles on the skin surface. By collecting hyperspectral image data of the skin surface, the method can analyze the microparticle composition on the skin surface at the pixel level based on the spectral data, and compare the obtained analysis results with the distribution pattern of the microparticles on the skin surface, so that the final detection and identification results are more accurate. To implement the above-mentioned skin surface microparticle detection method, this application also proposes a skin surface microparticle detection system, the specific scheme of which is as follows: A method for detecting microparticles on the skin surface, comprising: Acquire and store the relative position relationship characteristics of each type of microparticles distributed on the skin surface, as well as the two-dimensional image characteristics and spectral characteristics of each type of microparticles; Collect two-dimensional images and hyperspectral images of the skin surface; Identify the type of microparticles corresponding to each microparticle image in the two-dimensional image based on an image recognition algorithm according to the two-dimensional image features of each type of microparticles, and output a first recognition result; Determining the type of microparticles corresponding to each microparticle image in the hyperspectral image based on the spectral characteristics of each type of microparticles using a data matching algorithm, and outputting a second recognition result; Marking each microparticle image in the two-dimensional image by category based on the first recognition result and the second recognition result, and associating the microparticle image with the position coordinates corresponding to the microparticle image to form a first calibration image and a second calibration image; Compare the first category mark and the second category mark associated with the microparticle image at the same position in the first verification image and the second verification image: If the first category mark and the second category mark are the same, determining the category corresponding to the current microparticle image according to either category mark and storing it in association with the position coordinates; If the first category mark and the second category mark are different, obtaining a relative positional relationship between the current microparticle image and the microparticle image of the determined category, determining the category of the current microparticle image based on the relative positional relationship feature, and storing the associated category with the position coordinates; Based on the categories and position coordinates of each stored microparticle image, the category, quantity and distribution area of ​​each microparticle on the skin surface are statistically output.

[0005] This technical solution analyzes the classification of microparticles on the skin surface using both RGB two-dimensional high-definition images and hyperspectral images, improving the accuracy of microparticle classification detection and identification. Furthermore, when image recognition and spectral recognition results differ, the relative positional relationship between microparticles is analyzed to provide auxiliary judgment, further improving the detection and identification accuracy of microparticles on the skin surface. Furthermore, the entire detection and identification process requires no physical contact with the user's skin, making it efficient and fast.

[0006] Furthermore, determining the category of the current micro-particle image according to the relative position relationship feature includes: Comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; Comparing first confirmation probability values ​​corresponding to the first category mark and the second category mark; Selecting a category mark with a large first confirmation probability value as the category of the current microparticle image and storing it in association with the position coordinates; The first confirmation probability value is set to be positively correlated with the relationship similarity.

[0007] Through the above technical solution, based on the relative position relationship between the microparticles of different categories in the first verification image and the second verification image, combined with the relative position relationship characteristics, further identification and judgment are made on the microparticle image category to be determined, which can effectively improve the accuracy of microparticle category recognition.

[0008] Furthermore, if the first category mark and the second category mark are different, the method further includes: Obtain and store the first correspondence between various types of microparticles on the skin surface and their number ratios, sampling time, and environmental parameters; Acquiring the sampling time and environmental parameters of the two-dimensional image and hyperspectral image of the skin surface to be inspected; According to the first corresponding relationship, the types and quantity ratios of microparticles on the skin to be inspected are estimated and generated; Assigning a second confirmation probability value to the first category marker and the second category marker respectively according to the estimated microparticle category and quantity ratio; Weights are assigned to the first confirmation probability value and the second confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i), where P(i) is the category decision value, w1 and w2 are the weights of the first confirmation probability value and the second confirmation probability value, respectively, P1 is the first confirmation probability value, P2 is the second confirmation probability value, and i is the category label number, which can be 1 or 2; Selecting the category mark with the larger category determination value as the category of the current microparticle image and storing it in association with the position coordinates; The environmental parameters include ambient temperature, humidity, and air quality index.

[0009] Through the above technical solution, when identifying and determining the types of microparticles on the skin surface, the probability of exogenous microparticles adhering to the skin under different sampling times and environmental parameters is comprehensively considered, which can improve the accuracy of identifying the types of microparticles.

[0010] Furthermore, if the first category mark and the second category mark are different, the method further includes: Acquire and store reference spectral features corresponding to each skin tissue component, and the association probability between each skin tissue component and the type of microparticles on the skin surface; Obtaining spectral characteristics of the location of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected; Comparing the above spectral features with the reference spectral features, and confirming the skin tissue composition at the current microparticle image position based on the comparison results; Generating a third confirmation probability value of the microparticle category corresponding to the category mark appearing at the current position based on the skin tissue components obtained by comparison and the association probability; A weight is assigned to the third confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i), where w3 is the third confirmation probability weight and P3 is the third confirmation probability; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

[0011] Through the above technical solution, the type of microparticles on the skin surface can be inferred based on the composition of the dermis and subcutaneous tissue, which can greatly improve the recognition accuracy of endogenous microparticles such as fat particles and small papules.

[0012] Furthermore, if the first category mark and the second category mark are different, the method further includes: Obtain and store the reference variation patterns of the spectral characteristics of each type of microparticles over time; Collect and store two-dimensional images and hyperspectral images of the skin to be inspected multiple times at a set frequency; Retrieve historical hyperspectral image data to obtain the spectral features corresponding to the current microparticle image at each sampling time, and generate the change trend of the above spectral features based on the sampling time analysis; Comparing the change trend with the reference change rule, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category label or the second category label; A weight is assigned to the fourth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i), where w4 is the weight of the fourth confirmation probability value and P4 is the fourth confirmation probability value; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

[0013] Through the above technical solution, the type of microparticles can be confirmed based on the law of change of the spectral characteristics of the set type of microparticles over time, and the recognition accuracy of the microparticle category can be further improved.

[0014] Furthermore, if the first category mark and the second category mark are different, the method further includes: Get the current user's historical skin parameter data; Establishing a second correspondence between each skin parameter and the type of microparticles on the skin surface; assigning fifth confirmation probability values ​​to the microparticle categories corresponding to the first category mark and the second category mark based on the historical skin parameters and according to the second corresponding relationship; A weight is assigned to the fifth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i), where w5 is the fifth confirmation probability weight and P5 is the fifth confirmation probability; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

[0015] Through the above technical solution, the category of microparticles on the user's current skin surface can be accurately inferred based on the user's previous skin disease case data, further improving the detection and identification accuracy of microparticle categories on the skin surface.

[0016] Furthermore, if the first category mark and the second category mark are different, the method further includes: Based on statistical analysis of the big data bureau, the probability values ​​of each type of microparticle appearing in each area of ​​the skin surface are obtained and stored; Obtain the position coordinates corresponding to the current microparticle image; Finding probability values ​​of microparticles of the first and second marker categories corresponding to the categories appearing at the aforementioned position coordinates to generate a sixth confirmation probability value; A weight is assigned to the sixth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i)+w6*P6(i), where w6 is the weight of the sixth confirmation probability value and P6 is the sixth confirmation probability value; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

[0017] Through the above technical solution, the microparticle category corresponding to the current microparticle image can be estimated based on the probability of different types of microparticles appearing in different areas of the skin surface, and the recognition accuracy of the microparticle image on the skin surface can be further improved after weighted calculation.

[0018] Furthermore, the method further includes a weight optimization step: The weight of each probability value in the category judgment value calculation formula is set as a parameter variable; Adjust the weights corresponding to the probability values ​​and associate and record the accuracy of the microparticle image category recognition results; At least one set of weight combinations is obtained so that the accuracy probability of the recognition result is within a set interval.

[0019] Through the above technical solution, the optimal weight combination can be obtained based on the weight changes of the above multiple probability values, thereby systematically improving the recognition accuracy of microparticles on the skin surface.

[0020] A skin surface microparticle detection system, comprising: a data storage unit configured to store relative position relationship characteristics of each type of microparticles distributed on the skin surface, two-dimensional image characteristics and spectral characteristics of each type of microparticles, and the two-dimensional image and hyperspectral image to be processed collected by the data collection unit; a data acquisition unit configured to acquire a two-dimensional image and a hyperspectral image of the skin surface; a microparticle recognition unit comprising a first recognition module and a second recognition module, wherein the first recognition module is configured to identify the type of microparticles corresponding to each microparticle image in the two-dimensional image based on the two-dimensional image features of each type of microparticle and an image recognition algorithm, and output a first recognition result; The second recognition module is configured to determine the type of microparticles corresponding to each microparticle image in the hyperspectral image based on the spectral characteristics of each type of microparticle and a data matching algorithm, and output a second recognition result; a category marking unit configured to mark each microparticle image in the two-dimensional image by category based on the first recognition result and the second recognition result, and associate the mark with the position coordinates corresponding to the microparticle image to form a first calibration image and a second calibration image; a category determination unit configured to compare a first category mark and a second category mark associated with a microparticle image at the same position in the first verification image and the second verification image; if the first category mark and the second category mark are the same, determining the category corresponding to the current microparticle image based on either category mark and storing the category in association with the position coordinates; and if the first category mark and the second category mark are different, obtaining a relative positional relationship between the current microparticle image and the microparticle image of the determined category, determining the category of the current microparticle image based on a feature of the relative positional relationship and storing the category in association with the position coordinates; The result output unit is configured to statistically output the category, quantity and distribution area of ​​each microparticle on the skin surface based on the category and position coordinates of each stored microparticle image.

[0021] Furthermore, the data storage unit also stores: The primary correspondence between each type of microparticle on the skin surface and its percentage, sampling time, and environmental parameters; the reference spectral characteristics corresponding to each skin tissue component; the association probability between each skin tissue component and the type of microparticle on the skin surface; the reference variation pattern of the spectral characteristics of each type of microparticle over time; the current user's historical skin parameter data; the secondary correspondence between each skin parameter and the type of microparticle on the skin surface; and the probability of each type of microparticle appearing in each area of ​​the skin surface. The data acquisition unit is further configured to acquire sampling time and environmental parameters for acquiring the two-dimensional image and the hyperspectral image of the skin surface to be inspected, spectral characteristics of the location of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected, and the two-dimensional image and the hyperspectral image of the skin to be inspected acquired multiple times at a set frequency; The category determination unit further includes: A class probability generation module configured to: Comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; According to the first correspondence, the categories and quantity ratios of the microparticles on the skin to be inspected are estimated, and according to the estimated categories and quantity ratios of the microparticles, a second confirmation probability value is assigned to the first category mark and the second category mark respectively; Comparing the spectral features with the reference spectral features, confirming the skin tissue composition at the current microparticle image location based on the comparison results, and generating a third confirmation probability value for the microparticle category corresponding to the category label at the current location based on the skin tissue composition obtained from the comparison and the association probability; Retrieving historical hyperspectral image data to obtain spectral features corresponding to the current microparticle image at each sampling time, generating a change trend of the spectral features based on sampling time analysis, comparing the change trend with the reference change pattern, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category label or the second category label; Assigning fifth confirmation probability values ​​to the microparticle categories corresponding to the first category mark and the second category mark respectively based on the historical skin parameters and according to the second corresponding relationship; and Finding probability values ​​of microparticles of the first and second marker categories corresponding to the categories appearing at the aforementioned position coordinates to generate a sixth confirmation probability value; A probability weight configuration module is configured to configure corresponding weights for each of the above probability values; The category determination module is configured to receive the probability values ​​and calculate the category of the pre-generated microparticle image based on a set algorithm according to the corresponding weights, and store the image in association with the position coordinates.

[0022] Through the above technical solution, when faced with microparticle images of uncertain categories, the microparticle categories can be further identified and determined based on various data and data associations, which can effectively improve the accuracy of microparticle category identification.

[0023] This application has at least one of the following beneficial effects: (1) By combining two-dimensional images with hyperspectral image recognition, it is possible to identify and determine the categories of microparticles not only based on their image features, but also based on their chemical composition, effectively improving the accuracy of identifying the categories of microparticles on the skin surface. (2) By analyzing the distribution patterns and relative position relationships of various types of microparticles on the skin surface, further judgments can be made on microparticle images of uncertain categories, thereby improving recognition accuracy; (3) This application solution adopts a non-contact identification method, which only requires the use of a hyperspectral camera and a high-definition camera to capture the surface image of the skin to be inspected, which is convenient and efficient; (4) By introducing the correlation between the type of microparticles and parameters such as sampling time and environmental parameters, dermis and subcutaneous tissue composition, we can further assist in confirming the category corresponding to the microparticle image and ensure the accuracy of the detection and recognition results; (5) Through flexible weight configuration, the optimal weight can be configured for each confirmation probability value for different users, making the final detection and recognition results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the method for detecting microparticles on the skin surface; Figure 2 Schematic diagram of a method for determining the category of microparticles based on relative position relationship characteristics; Figure 3 Schematic diagram of a method for assisting in determining the type of microparticles based on spectral information of skin tissue components; Figure 4 Schematic diagram of the functional modules of the skin surface microparticle detection system.

[0025] Figure numerals: 100, data acquisition unit; 200, data storage unit; 300, microparticle identification unit; 310, first identification module; 320, second identification module; 400, category marking unit; 500, category determination unit; 510, category probability generation module; 520, probability weight configuration module; 530, category determination module; 600, result output unit. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present application in detail, and examples of the embodiments are shown in the attached Figure 1-4 Shown in.

[0027] Throughout this specification, reference to the terms "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0028] A method for detecting microparticles on the skin surface, such as Figure 1 As shown, it mainly includes the following steps: S100, acquiring and storing relative position relationship characteristics of each type of microparticles distributed on the skin surface, as well as two-dimensional image characteristics and spectral characteristics of each type of microparticles; S200, collects two-dimensional images and hyperspectral images of the skin surface; S300, identifying the type of microparticles corresponding to each microparticle image in the two-dimensional image based on an image recognition algorithm according to the two-dimensional image features of each type of microparticle, and outputting a first recognition result; S400, determining the type of microparticles corresponding to each microparticle image in the hyperspectral image based on the spectral characteristics of each type of microparticles using a data matching algorithm, and outputting a second recognition result; S500, classifying each microparticle image in the two-dimensional image based on the first recognition result and the second recognition result, and storing the classified microparticle images in association with the corresponding position coordinates of the microparticle images to form a first calibration image and a second calibration image; S600, comparing the first category mark and the second category mark associated with the microparticle image at the same position in the first verification image and the second verification image: S610, if the first category mark and the second category mark are the same, determining the category corresponding to the current microparticle image according to either category mark and storing it in association with the position coordinates; S620: If the first category mark and the second category mark are different, obtaining a relative positional relationship between the current microparticle image and the microparticle image of a determined category, determining the category of the current microparticle image based on the relative positional relationship feature, and storing the associated category with the position coordinates; S700 , based on the categories and position coordinates of the stored microparticle images, statistically output the categories, quantities and distribution areas of the microparticles on the skin surface.

[0029] In the above step S100, the category of microparticles includes the type and size of microparticles. The types of microparticles described in the embodiments of the present application include exogenous microparticles and endogenous microparticles. Exogenous microparticles include environmental dust, pollen, microorganisms, cosmetic residues, etc., and endogenous microparticles include fat particles, keratin particles, etc. The relative position relationship characteristics of each category of microparticles distributed on the skin surface include the relative position relationship characteristics between microparticles of the same category and the relative position relationship characteristics between microparticles of different categories. The above relative position relationship characteristics include the average spacing, distribution trend, distribution shape and area of ​​each microparticle on the skin surface. For example, exogenous dust will be more evenly attached to the skin surface, while endogenous microparticles will usually be diffusely distributed with a certain area as the center.

[0030] In steps S100 and S200, the two-dimensional image of the microparticle refers to an image of the skin to be examined captured by a high-definition camera, while the spectrum refers to a spectral image captured by a hyperspectral camera. Two-dimensional image features refer to shape characteristics that can reflect the classification of microparticles, such as the shape of the microparticle's edge, while spectral features refer to the unique wavelength distribution of the microparticle to be identified in the spectrum, which is used to reflect the composition, structure, or energy state of the microparticle.

[0031] In step S300, the captured high-definition image of the microparticles is directly compared with the two-dimensional image features to identify the microparticle type. In practical applications, the image recognition algorithm is configured as an image recognition module generated by neural network training, such as a convolutional neural network (CNN) model. In step S400, the spectral characteristics of each microparticle are directly matched using a data matching algorithm, such as the correlation coefficient method, which measures the similarity by calculating the correlation coefficient between the peaks of the two spectral images to obtain the identification result.

[0032] In the above steps S300 and S400 , the first recognition result and the second recognition result include recognition results of surface microparticle categories of each area on the skin to be inspected.

[0033] In this embodiment of the present application, in step S500, a first calibration chart and a second calibration chart are generated based on the two-dimensional image and spectral image recognition results. Each microparticle image is labeled with its corresponding category in the calibration chart and the position coordinates of the microparticles are stored. To ensure that the position coordinates of each microparticle in the two calibration charts are consistent, the first and second calibration charts use the same reference coordinate system.

[0034] In step S600, if the category labels of the microparticle images at the same position in the two verification images are different, relative position relationship feature determination is introduced.

[0035] In detail, in step S600, the category of the current micro-particle image is determined according to the relative position relationship feature, such as Figure 2As shown, further comprising: S601, comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; S602, comparing first confirmation probability values ​​corresponding to the first category mark and the second category mark; S603: Select the category mark with the largest first confirmation probability value as the category of the current micro-particle image and store it in association with the position coordinates.

[0036] The above-mentioned first confirmation probability value is set to be positively correlated with the relationship similarity, that is, the more consistent the actual relative position relationship is with the relative position relationship feature, the greater the first confirmation probability value. For example, the relative position relationship between the microparticles of cosmetic residues and pollen attached to the skin surface is obviously different. Based on the comparison of the position relationship features, the type of microparticles can be quickly confirmed.

[0037] In actual applications, the type of microparticles attached to the skin surface is usually related to the sampling time and environmental parameters. For example, in an open outdoor environment, the microparticles on the skin surface are mostly exogenous microparticles, such as air dust; if the sampling time is in the early morning, the probability that the microparticles on the skin surface are fat particles is higher.

[0038] In order to further improve the accuracy of identifying microparticle categories and avoid the limitations of relying solely on relative position relationship features to assist in identifying microparticle categories, in this application, if the first category label and the second category label are different, that is, the results of spectral feature recognition and image recognition are inconsistent, the skin surface microparticle detection method described in this application also includes: A100 obtains and stores a first correspondence between each type of microparticle on the skin surface and its proportion, sampling time, and environmental parameters. The environmental parameters include ambient temperature and humidity, and air quality index, such as PM2.5 and PM10 concentrations.

[0039] A200, acquires the sampling time and environmental parameters for the two-dimensional image and hyperspectral image of the skin surface to be inspected.

[0040] A300, based on the first correspondence, estimates the types and quantity ratios of microparticles on the skin to be tested.

[0041] A400 assigns a second confirmation probability value to the first category mark and the second category mark based on the estimated microparticle category and quantity ratio. For example, if the number of category A microparticles accounts for 60% and the number of category B microparticles accounts for 10% during the set time period, then the second confirmation probability value corresponding to category A microparticles is 60%, and the second confirmation probability value corresponding to category B microparticles is 10%.

[0042] A500: Assign weights to the first confirmation probability value and the second confirmation probability value, and generate category determination values ​​of the first category mark and the second category mark using the following calculation formula: P(i)=w1*P1(i)+w2*P2(i).

[0043] Among them, w1 and w2 are the weights of the first confirmation probability value and the second confirmation probability value respectively, P1 is the first confirmation probability value, P2 is the second confirmation probability value, and i is the category tag number, which takes a value of 1 or 2. For example, P(1) represents the final category determination value of the first category tag, P1(1) represents the first confirmation probability value corresponding to the first category tag, and P1(2) represents the second confirmation probability value corresponding to the first category tag.

[0044] A600: Select the category mark with the larger category determination value as the category of the current microparticle image and store it in association with the position coordinates.

[0045] When using the above technical solution to identify and determine the type of microparticles on the skin surface, the probability of exogenous microparticles adhering to the skin under different sampling times and different environmental parameters is comprehensively considered, which can improve the accuracy of microparticle classification identification.

[0046] For endogenous microparticles, their causes are usually related to lesions in the dermis or subcutaneous tissue or changes in tissue composition. For example, fat particles are usually formed due to excessive secretion of sebaceous glands and clogging of pores. Therefore, when abnormal sebum secretion is confirmed, the microparticles at the location of abnormal sebum secretion are likely to be fat particles. Figure 3 As shown, if the first category mark and the second category mark are different, the method further includes: B100, obtaining and storing reference spectral features corresponding to each skin tissue component and the association probability between each skin tissue component and the category of microparticles on the skin surface.

[0047] B200, obtaining the spectral characteristics of the position of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected.

[0048] B300, compare the above spectral features with the reference spectral features, and confirm the skin tissue composition at the current microparticle image position based on the comparison results. The comparison of the spectral features of this application preferably uses the correlation coefficient method, and can also be combined with the angle cosine method for comparison.

[0049] B400 , generating a third confirmation probability value of the microparticle category corresponding to the category mark appearing at the current position based on the skin tissue components obtained through comparison and the association probability.

[0050] B500, assigning a weight to the third confirmation probability value, and generating a category determination value of the first category mark and the second category mark by the following calculation formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i). Where w3 is the weight of the third confirmation probability value, P3 is the third confirmation probability value, and the remaining parameters in the calculation formula have been explained above and will not be repeated here.

[0051] B600: Determine the category of the current microparticle image based on the category determination value and store it in association with the position coordinates.

[0052] The above steps B100-B600 can infer the type of microparticles on the skin surface based on the composition of the dermis and subcutaneous tissue, and can greatly improve the recognition accuracy of endogenous microparticles, such as fat particles and small papules.

[0053] In practical applications, except for the physicochemical properties of exogenous microparticles, which do not change frequently, the physicochemical composition of other microparticles usually changes over time. The main reason is that the protein or water content in the microparticles changes over time. Even the physicochemical composition of exogenous microparticles will change, such as the water content in pollen attached to the skin surface will be lost over time. The trends of the above physicochemical compositions corresponding to different microparticle categories over time are different. Therefore, the above change trends are used as the basis for determining the category of microparticles. To this end, in an embodiment of the present application, if the first category label and the second category label are different, the method further includes: C100 obtains and stores the reference variation patterns of the spectral characteristics of each type of microparticles over time.

[0054] C200 collects and stores two-dimensional images and hyperspectral images of the skin to be tested multiple times at a set frequency.

[0055] C300 retrieves historical hyperspectral image data, obtains the spectral features corresponding to the current microparticle image at each sampling time, and generates the change trend of the above spectral features based on sampling time analysis.

[0056] C400 , comparing the change trend with the reference change rule, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category mark or the second category mark.

[0057] C500, assigning a weight to the fourth confirmation probability value, and generating a category determination value of the first category mark and the second category mark by the following calculation formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i), where w4 is the weight of the fourth confirmation probability value and P4 is the fourth confirmation probability value.

[0058] C600 : Determine the category of the current microparticle image based on the category determination value and store it in association with the position coordinates.

[0059] The above technical solution can confirm the type of microparticles based on the law of change of the spectral characteristics of the set type of microparticles over time, and further improve the recognition accuracy of the microparticle category.

[0060] In practical applications, microparticles on the surface of human skin are usually related to the user's physical health, especially skin health. For example, whether there is endocrine disorder or inflammation, etc., will affect the type and number of endogenous microparticles on the skin surface. Therefore, in a further optimized embodiment of the present application, if the first category label and the second category label are different, the method further includes: D100: Obtain historical skin parameter data of the current user. The historical skin parameter data includes skin disease case information, such as skin inflammation, etc., and also includes historical skin surface microparticle type detection data, etc.

[0061] D200, establishes a second correspondence between various skin parameters and the types of microparticles on the skin surface. For example, skin inflammation often causes folliculitis, which in turn causes gray-black spots to appear on the skin surface.

[0062] D300 , assigning fifth confirmation probability values ​​to the microparticle categories corresponding to the first category mark and the second category mark based on the historical skin parameters and according to the second corresponding relationship.

[0063] D400, assigning a weight to the fifth confirmation probability value, and generating a category determination value of the first category mark and the second category mark by the following calculation formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i), where w5 is the fifth confirmation probability value weight and P5 is the fifth confirmation probability value.

[0064] D500: Determine the category of the current microparticle image based on the category determination value and store it in association with the position coordinates.

[0065] In addition to introducing the above-mentioned auxiliary discrimination parameters, considering that the distribution area of ​​microparticles on the skin surface is usually specific, for example, fat particles generally grow around the eyes, cheeks, forehead and other parts, and clinically manifest as small yellow-white papules. Therefore, in the embodiment of the present application, if the first category mark and the second category mark are different, the method further includes: E100 uses statistical analysis from a large data set to obtain and store the probability of each type of microparticle appearing in each area of ​​the skin surface. In practice, the probability of each area corresponding to the corresponding type of microparticle can be manually labeled.

[0066] E200, obtain the position coordinates corresponding to the current micro-particle image.

[0067] E300 , searching for probability values ​​of microparticles of the first and second marker categories corresponding to the categories appearing at the above position coordinates, and generating a sixth confirmation probability value.

[0068] E400, assigning a weight to the sixth confirmation probability value, and generating a category determination value of the first category mark and the second category mark by the following calculation formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i)+w6*P6(i), where w6 is the weight of the sixth confirmation probability value and P6 is the sixth confirmation probability value; E500: Determine the category of the current microparticle image based on the category determination value and store it in association with the position coordinates.

[0069] In actual applications, the above confirmation probability values ​​will be different for different users. For example, the distribution area of ​​fat particles on the user's face is closely related to the user himself. That is, in order to ensure the accuracy of the detection and recognition results, each of the above confirmation probability values ​​needs to be assigned a different weight. For this reason, it is more critical that in the present application scheme, the method further includes a weight optimization step: S810, set the weight of each probability value in the category judgment value calculation formula as a parameter variable, for example, set the weights of each item in the following calculation formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4* P4(i) +w5* P5(i) +w6* P6(i) as a variable.

[0070] S820: Adjust the weights corresponding to the probability values ​​and associate and record the accuracy of the microparticle image category recognition results.

[0071] S830: Obtain at least one set of weight combinations so that the accuracy probability of the recognition result is within a set range.

[0072] The above steps S820-S830 can be completed through neural network model training, and finally obtain weight combinations for different individuals, thereby systematically improving the recognition accuracy of microparticles on the skin surface.

[0073] In order to realize the above-mentioned skin surface microparticle detection method, the present application also proposes a skin surface microparticle detection system, such as Figure 4As shown, it mainly includes a data storage unit 200, a data acquisition unit 100, a microparticle identification unit 300, a category marking unit 400, a category determination unit 500 and a result output unit 600.

[0074] Data storage unit 200 is configured to store the relative positional distribution characteristics of various types of microparticles on the skin surface, the two-dimensional image characteristics and spectral characteristics of each type of microparticle, and the unprocessed two-dimensional images and hyperspectral images collected by data acquisition unit 100. In practical applications, data storage unit 200 includes local memory and a cloud database. The local memory, such as a memory chip built into the skin analyzer or a computer hard drive, is used to temporarily store relevant intermediate data. The cloud database is used to store historical data, various feature data, neural network models, etc.

[0075] The data acquisition unit 100 is configured to acquire two-dimensional images and hyperspectral images of the skin surface, and mainly includes a high-definition camera and a hyperspectral camera, which are data-connected to the data storage unit 200. The above-mentioned camera functions can be integrated into a skin detector to realize related functions.

[0076] The microparticle identification unit 300 includes a first identification module 310 and a second identification module 320. The first identification module 310 is configured to identify the microparticle type corresponding to each microparticle image in the two-dimensional image based on the two-dimensional image features of each type of microparticle using an image recognition algorithm, and output a first identification result. The second identification module 320 is configured to determine the microparticle type corresponding to each microparticle image in the hyperspectral image based on the spectral features of each type of microparticle using a data matching algorithm, and output a second identification result.

[0077] The category marking unit 400 is configured to be data-connected with the microparticle identification unit 300, and to perform category marking on each microparticle image in the two-dimensional image based on the first identification result and the second identification result, respectively. The marking content is the ID code corresponding to the microparticle category, and the ID code is associated with the position coordinates corresponding to the microparticle image to form a first calibration diagram and a second calibration diagram. In a specific embodiment, the first calibration diagram and the second calibration diagram are stored in the data storage unit 200 in the form of a data table. When the user needs to check the relevant detection and identification results, he can Figure 2 The two-dimensional image shown is presented for intuitive presentation.

[0078] The category determination unit 500 includes a data processor, preferably located in a cloud server for efficient data processing. The category determination unit 500 is configured to compare the first category mark and the second category mark associated with the microparticle image at the same location in the first and second verification images. If the first category mark and the second category mark are identical, the category corresponding to the current microparticle image is determined based on either category mark and stored in association with the location coordinates. If the first category mark and the second category mark are different, the relative positional relationship between the current microparticle image and the microparticle image whose category has been determined is determined, and the category of the current microparticle image is determined based on the relative positional relationship characteristics and stored in association with the location coordinates.

[0079] Result output unit 600 is configured to be data-connected to category determination unit 500 and, based on the category and location coordinates of each stored microparticle image, statistically outputs the category, quantity, and distribution area of ​​each microparticle on the skin surface. Result output unit 600 includes a data display device, such as a display or intelligent display terminal.

[0080] In order to improve the recognition accuracy of microparticles, the data storage unit 200 also stores: the first correspondence between each category of microparticles on the skin surface and their number ratio and the sampling time and environmental parameters, the reference spectral characteristics corresponding to each skin tissue component, and the association probability between each skin tissue component and the category of microparticles on the skin surface, the reference change law of the spectral characteristics of each category of microparticles over time, the current user's historical skin parameter data and the second correspondence between each skin parameter and the category of microparticles on the skin surface, and the probability value of each category of microparticles appearing in each area of ​​the skin surface.

[0081] Correspondingly, the data acquisition unit 100 is also configured to collect the sampling time and environmental parameters for acquiring the two-dimensional image and hyperspectral image of the skin surface to be inspected, the spectral characteristics of the location of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected, and the two-dimensional image and hyperspectral image of the skin to be inspected collected multiple times at a set frequency.

[0082] The category determination unit 500 further includes: a category probability generation module 510 , a probability weight configuration module 520 and a category determination module 530 .

[0083] The category probability generation module 510 is configured to implement the following functions as needed: A. Comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; B. estimating the categories and number ratios of the microparticles on the skin to be inspected based on the first correspondence, and assigning second confirmation probability values ​​to the first category marker and the second category marker based on the estimated categories and number ratios of the microparticles; C. Comparing the spectral features with the reference spectral features, determining the skin tissue composition at the current microparticle image location based on the comparison results, and generating a third confirmation probability value for the microparticle category corresponding to the category label at the current location based on the skin tissue composition obtained from the comparison and the association probability; D. Retrieving historical hyperspectral image data to obtain spectral features corresponding to the current microparticle image at each sampling time, analyzing and generating a variation trend of the spectral features based on the sampling time, comparing the variation trend with the reference variation pattern, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category label or the second category label; E. assigning a fifth confirmation probability value to each of the microparticle categories corresponding to the first category mark and the second category mark based on the historical skin parameters and according to the second corresponding relationship; F. Find the probability values ​​of the microparticles corresponding to the first and second marker categories appearing at the above position coordinates to generate a sixth confirmation probability value.

[0084] The probability weight configuration module 520 is configured to configure corresponding weights for the above-mentioned probability values.

[0085] The category determination module 530 is configured to receive the probability values ​​and calculate the category of the front microparticle image according to the corresponding weights based on a set algorithm and store it in association with the position coordinates. The set algorithm is configured as the type determination calculation formula as described above.

[0086] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for detecting microparticles on the skin surface, characterized in that: include: Acquire and store the relative position relationship characteristics of each type of microparticles distributed on the skin surface, as well as the two-dimensional image characteristics and spectral characteristics of each type of microparticles; Collect two-dimensional images and hyperspectral images of the skin surface; Identify the type of microparticles corresponding to each microparticle image in the two-dimensional image based on an image recognition algorithm according to the two-dimensional image features of each type of microparticles, and output a first recognition result; Determining the type of microparticles corresponding to each microparticle image in the hyperspectral image based on the spectral characteristics of each type of microparticles using a data matching algorithm, and outputting a second recognition result; Marking each microparticle image in the two-dimensional image by category based on the first recognition result and the second recognition result, and associating the microparticle image with the position coordinates corresponding to the microparticle image to form a first calibration image and a second calibration image; Compare the first category mark and the second category mark associated with the microparticle image at the same position in the first verification image and the second verification image: If the first category mark and the second category mark are the same, determining the category corresponding to the current microparticle image according to either category mark and storing it in association with the position coordinates; If the first category mark and the second category mark are different, obtaining a relative positional relationship between the current microparticle image and the microparticle image of the determined category, determining the category of the current microparticle image based on the relative positional relationship feature, and storing the associated category with the position coordinates; Based on the categories and position coordinates of each stored microparticle image, the category, quantity and distribution area of ​​each microparticle on the skin surface are statistically output.

2. The method for detecting microparticles on the skin surface according to claim 1, wherein: Determining the category of the current micro-particle image according to the relative position relationship feature includes: Comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; Comparing first confirmation probability values ​​corresponding to the first category mark and the second category mark; Selecting a category mark with a large first confirmation probability value as the category of the current microparticle image and storing it in association with the position coordinates; The first confirmation probability value is set to be positively correlated with the relationship similarity.

3. The method for detecting microparticles on the skin surface according to claim 2, wherein: If the first category mark and the second category mark are different, the method further includes: Obtain and store the first correspondence between various types of microparticles on the skin surface and their number ratios, sampling time, and environmental parameters; Acquiring the sampling time and environmental parameters of the two-dimensional image and hyperspectral image of the skin surface to be inspected; According to the first corresponding relationship, the types and quantity ratios of microparticles on the skin to be inspected are estimated and generated; Assigning a second confirmation probability value to the first category marker and the second category marker respectively according to the estimated microparticle category and quantity ratio; Weights are assigned to the first confirmation probability value and the second confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i); where P(i) is the category decision value, w1 and w2 are the weights of the first confirmation probability value and the second confirmation probability value, respectively, P1 is the first confirmation probability value, P2 is the second confirmation probability value, and i is the category label number, which can be 1 or 2; Selecting the category mark with the larger category determination value as the category of the current microparticle image and storing it in association with the position coordinates; The environmental parameters include ambient temperature, humidity, and air quality index.

4. The method for detecting microparticles on the skin surface according to claim 3, wherein: If the first category mark and the second category mark are different, the method further includes: Acquire and store reference spectral features corresponding to each skin tissue component, and the association probability between each skin tissue component and the type of microparticles on the skin surface; Obtaining spectral characteristics of the location of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected; Comparing the above spectral features with the reference spectral features, and confirming the skin tissue composition at the current microparticle image position based on the comparison results; Generating a third confirmation probability value of the microparticle category corresponding to the category mark appearing at the current position based on the skin tissue components obtained by comparison and the association probability; A weight is assigned to the third confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i), where w3 is the third confirmation probability weight and P3 is the third confirmation probability; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

5. The method for detecting microparticles on the skin surface according to claim 4, characterized in that: If the first category mark and the second category mark are different, the method further includes: Obtain and store the reference variation patterns of the spectral characteristics of each type of microparticles over time; Collect and store two-dimensional images and hyperspectral images of the skin to be inspected multiple times at a set frequency; Retrieve historical hyperspectral image data to obtain the spectral features corresponding to the current microparticle image at each sampling time, and generate the change trend of the above spectral features based on the sampling time analysis; Comparing the change trend with the reference change rule, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category label or the second category label; A weight is assigned to the fourth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i), where w4 is the weight of the fourth confirmation probability value and P4 is the fourth confirmation probability value; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

6. The method for detecting microparticles on the skin surface according to claim 5, characterized in that: If the first category mark and the second category mark are different, the method further includes: Get the current user's historical skin parameter data; Establishing a second correspondence between each skin parameter and the type of microparticles on the skin surface; assigning fifth confirmation probability values ​​to the microparticle categories corresponding to the first category mark and the second category mark based on the historical skin parameters and according to the second corresponding relationship; A weight is assigned to the fifth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i), where w5 is the fifth confirmation probability weight and P5 is the fifth confirmation probability; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

7. The method for detecting microparticles on the skin surface according to claim 6, wherein: If the first category label and the second category label are different, the method further includes: Based on statistical analysis of the big data bureau, the probability values ​​of each type of microparticle appearing in each area of ​​the skin surface are obtained and stored; Obtain the position coordinates corresponding to the current microparticle image; Finding probability values ​​of microparticles of the first and second marker categories corresponding to the categories appearing at the aforementioned position coordinates to generate a sixth confirmation probability value; A weight is assigned to the sixth confirmation probability value, and the category determination values ​​of the first category mark and the second category mark are calculated using the following formula: P(i)=w1*P1(i)+w2*P2(i)+w3*P3(i)+w4*P4(i)+w5*P5(i)+w6*P6(i), where w6 is the weight of the sixth confirmation probability value and P6 is the sixth confirmation probability value; The category of the current microparticle image is determined based on the category determination value and stored in association with the position coordinates.

8. The method for detecting microparticles on the skin surface according to claim 7, characterized in that: The method further comprises a weight optimization step: The weight of each probability value in the category judgment value calculation formula is set as a parameter variable; Adjust the weights corresponding to the probability values ​​and associate and record the accuracy of the microparticle image category recognition results; At least one set of weight combinations is obtained so that the accuracy probability of the recognition result is within a set interval.

9. A skin surface microparticle detection system, characterized in that: To implement the method for detecting microparticles on the skin surface according to any one of claims 1 to 8, the method comprises: The data storage unit (200) is configured to store relative position relationship characteristics of each type of microparticles distributed on the skin surface, two-dimensional image characteristics and spectral characteristics of each type of microparticles, and the two-dimensional image to be processed and the hyperspectral image collected by the data collection unit (100); A data acquisition unit (100) configured to acquire a two-dimensional image and a hyperspectral image of the skin surface; The microparticle recognition unit (300) comprises a first recognition module (310) and a second recognition module (320), wherein the first recognition module (310) is configured to recognize the type of microparticles corresponding to each microparticle image in the two-dimensional image based on the image recognition algorithm according to the two-dimensional image features of each type of microparticles, and output a first recognition result; The second recognition module (320) is configured to determine the type of microparticles corresponding to each microparticle image in the hyperspectral image based on the spectral characteristics of each type of microparticle and a data matching algorithm, and output a second recognition result; a category marking unit (400) configured to mark each microparticle image in the two-dimensional image by category based on the first recognition result and the second recognition result, and store the mark in association with the position coordinates corresponding to the microparticle image to form a first calibration image and a second calibration image; A category determination unit (500) is configured to compare a first category mark and a second category mark associated with a microparticle image at the same position in a first verification image and a second verification image: if the first category mark and the second category mark are the same, determining the category corresponding to the current microparticle image based on either category mark and storing it in association with the position coordinates; if the first category mark and the second category mark are different, obtaining a relative positional relationship between the current microparticle image and a microparticle image of a determined category, determining the category of the current microparticle image based on a feature of the relative positional relationship and storing it in association with the position coordinates; The result output unit (600) is configured to statistically output the category, quantity and distribution area of ​​each microparticle on the skin surface based on the category and position coordinates of each stored microparticle image.

10. The skin surface microparticle detection system according to claim 9, characterized in that: The data storage unit (200) also stores: The primary correspondence between each type of microparticle on the skin surface and its percentage, sampling time, and environmental parameters; the reference spectral characteristics corresponding to each skin tissue component; the association probability between each skin tissue component and the type of microparticle on the skin surface; the reference variation pattern of the spectral characteristics of each type of microparticle over time; the current user's historical skin parameter data; the secondary correspondence between each skin parameter and the type of microparticle on the skin surface; and the probability of each type of microparticle appearing in each area of ​​the skin surface. The data acquisition unit (100) is further configured to acquire sampling time and environmental parameters for acquiring a two-dimensional image and a hyperspectral image of the surface of the skin to be inspected, spectral characteristics of the location of the microparticle image to be confirmed in the hyperspectral image of the skin to be inspected, and two-dimensional images and hyperspectral images of the skin to be inspected acquired multiple times at a set frequency; The category determination unit (500) further includes: The class probability generation module (510) is configured to: Comparing the currently acquired relative position relationship with the stored relative position relationship features, and assigning a first confirmation probability value to the first category mark and the second category mark based on the position relationship similarity; According to the first correspondence, the categories and quantity ratios of the microparticles on the skin to be inspected are estimated, and according to the estimated categories and quantity ratios of the microparticles, a second confirmation probability value is assigned to the first category mark and the second category mark respectively; Comparing the spectral features with the reference spectral features, confirming the skin tissue composition at the current microparticle image location based on the comparison results, and generating a third confirmation probability value for the microparticle category corresponding to the category label at the current location based on the skin tissue composition obtained from the comparison and the association probability; Retrieving historical hyperspectral image data to obtain spectral features corresponding to the current microparticle image at each sampling time, generating a change trend of the spectral features based on sampling time analysis, comparing the change trend with the reference change pattern, and confirming, based on the comparison result, a fourth confirmation probability value that the microparticle category corresponding to the current microparticle image is consistent with the first category label or the second category label; Assigning fifth confirmation probability values ​​to the microparticle categories corresponding to the first category mark and the second category mark respectively based on the historical skin parameters and according to the second corresponding relationship; and Finding probability values ​​of microparticles of the first and second marker categories corresponding to the categories appearing at the aforementioned position coordinates to generate a sixth confirmation probability value; A probability weight configuration module (520) configured to configure corresponding weights for each of the above probability values; The category determination module (530) is configured to receive each of the probability values ​​and calculate the category of the front micro-particle image based on a set algorithm according to the corresponding weight, and store the category in association with the position coordinates.

Images