A method and system for automatic acquisition of clinical skin images

Abstract

A method and system for automatic acquisition of clinical skin images, comprising the following steps: S1, preprocessing; S2, generating and labeling a detection frame, a clinical skin image quality score result; S3, constructing a clinical skin image data set; S4, constructing and training a lesion detection and quality assessment model; S5, inputting the clinical skin image acquired in real time into the lesion detection and quality assessment model, obtaining a detection frame corresponding to the skin lesion, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and a clinical skin image quality score result; S6, verifying the detection frame area quality score result; S7, verifying the quality score result; S8, retaining the acquisition result. The present invention designs a set of automatic acquisition methods and systems for clinical skin images, realizing automatic lesion focusing and high-quality lesion image acquisition, and greatly improving the acquisition quality and efficiency of clinical skin images.

Description

Technical Field

[0001] The present invention belongs to the field of skin image detection and equipment automatic control, and in particular relates to a clinical skin image acquisition method and system. Background Art

[0002] In the practice of diagnosis and treatment of skin diseases, clinical images are an important source of information, especially in the process of remote diagnosis and treatment, which is often an important basis for skin disease screening. However, in telemedicine scenarios, primary hospitals or patients themselves often find it difficult to obtain high-quality clinical images. This is not because the current shooting performance of mobile phones, tablets and other camera devices is insufficient, but because the existing acquisition solutions are often shot manually or the equipment simply shoots automatically. When selecting and focusing the image acquisition device, a standardized acquisition method or acquisition system is not designed, resulting in the focus often being focused on other body parts or invalid backgrounds during the image acquisition process, making the lesion area blurred.

[0003] At present, there is no standardized collection scheme for clinical skin images. On some special equipment, such as dermatoscopes, there is no autofocus system designed specifically for the collection of skin lesion images, which results in the inability of existing image acquisition equipment to give full play to its high-quality shooting performance. The collected images have a high waste rate and low diagnostic and treatment value. At the same time, repeated manual shooting and collection increase labor costs and medical burdens. Therefore, in order to improve the screening accuracy of telemedicine, it is particularly important to design an autofocus method for collecting images of skin damage and lesions. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the existing acquisition scheme has a high waste rate, low diagnostic value and high labor cost. The present invention overcomes the deficiencies and defects mentioned in the above background technology and provides a clinical skin image automatic acquisition method and system.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] A method for automatically collecting clinical skin images comprises the following steps:

[0007] S1, preprocessing clinical skin images;

[0008] S2, generating a detection frame corresponding to the skin lesion in the clinical skin image;

[0009] The skin lesions include one or more of skin damage, lesions or defects.

[0010] Determine the detection frame area, calculate the proportion of the detection frame area in the overall area of ​​the clinical skin image and mark it;

[0011] Score the image quality corresponding to the skin lesion area within the detection frame, obtain the clinical skin image quality score result and mark it;

[0012] S3, constructing a clinical skin image dataset; the clinical skin image dataset includes clinical images, annotations of the ratio of the detection box area to the overall area of ​​the clinical skin image, and annotations of the clinical skin image quality score results;

[0013] S4. Based on the clinical skin image dataset, construct and train a lesion detection and quality assessment model; the lesion detection and quality assessment model takes the clinical skin image as input, and takes the detection box corresponding to the skin lesion, the proportion of the detection box area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result as output;

[0014] S5, inputting the clinical skin image acquired in real time into the lesion detection and quality assessment model to obtain the detection frame corresponding to the skin lesion, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result;

[0015] S6, setting the maximum and minimum thresholds of the proportion of the detection frame area to the overall area of ​​the clinical skin image, and comparing the proportion of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum thresholds; when the proportion is less than the minimum threshold or greater than the maximum threshold, it is determined to be unqualified, and the clinical skin image acquisition device is adjusted, and the image is re-acquired back to S5; when the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is determined to be qualified, and enter S7;

[0016] S7, setting a minimum threshold of the clinical skin image quality score result, comparing the real-time clinical skin image quality score result with the minimum threshold, when the quality score result is lower than the minimum threshold, it is judged as unqualified, adjusting the clinical skin image acquisition device, and returning to S5 to re-acquire the image; when the quality score result is not lower than the minimum threshold, it is judged as qualified, and entering S8;

[0017] S8. Retain the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and the corresponding clinical skin image as the acquisition result.

[0018] Preferably, the preprocessing specifically includes: retaining images containing skin lesions and filtering out other images.

[0019] Preferably, the image quality scoring criteria include image clarity and / or exposure.

[0020] Preferably, the constructing and training of the lesion detection and quality assessment model based on the clinical skin image dataset specifically includes:

[0021] According to a certain ratio, the clinical skin image dataset is divided into a training dataset and a test dataset;

[0022] Based on the clinical skin image training dataset, a lesion detection and quality assessment model was constructed and trained. Based on the clinical skin image test dataset, the lesion detection and quality assessment model was tested.

[0023] More preferably, clinical skin images for testing are collected, and the lesion site detection and quality assessment model is verified in the real world, and the lesion site detection and quality assessment model is obtained after successful verification.

[0024] Preferably, the lesion detection and quality assessment model comprises an encoder, a decoder, and a regression decoder and a classification decoder composed of a pooling layer and a linear layer;

[0025] The encoder encodes the clinical skin image, the decoder decodes the features encoded by the encoder to obtain a detection frame corresponding to the skin lesion site, the pooling layer pools the encoding, and the linear layer outputs the proportion of the detection frame area in the overall area of ​​the clinical skin image and the clinical skin image quality score result.

[0026] Preferably, in step S6, adjusting the clinical skin image acquisition device specifically includes: recording the shooting focal length, inputting instructions to the clinical skin image acquisition device, and adjusting the proportion of the detection frame area in the image acquired by the acquisition device to the overall area of ​​the clinical skin image;

[0027] In step S7, the adjusting of the clinical skin image acquisition device specifically includes: based on the shooting focal length recorded in step S6, inputting instructions to the clinical skin image acquisition device, adjusting the focus of the acquisition device to the center of the detection frame, and refocusing.

[0028] In this solution, it is necessary to first determine the shooting focal length, adjust the proportion of the detection frame area in the acquired image to the overall area of ​​the clinical skin image, and then further focus and adjust the image clarity, exposure, etc. at the determined focal length.

[0029] Preferably, the method of inputting instructions includes gesture zooming and / or manual focusing.

[0030] Preferably, the adjusting of the focal length and / or the zoom size specifically includes: adjusting the lesion site to the center of the lesion detection frame through manual focusing.

[0031] Preferably, in steps S5 to S8, after the clinical skin image acquired in real time is input into the lesion detection and quality assessment model, the detection frame corresponding to the skin lesion site is synchronously displayed on the shooting interface of the clinical skin image acquisition device.

[0032] Under the same technical concept, the present invention also provides a clinical skin image auto-focus acquisition system, the acquisition system includes a processor, and the processor includes the following modules:

[0033] Image collection preprocessing module: preprocess clinical skin images;

[0034] Annotation module: used to determine the detection frame area, calculate the proportion of the detection frame area in the overall area of ​​the clinical skin image and annotate it; score the image quality corresponding to the skin lesion part in the detection frame, obtain the bedside skin image quality score result and annotate it;

[0035] Lesion detection and quality assessment model training module: used to build and train a lesion detection and quality assessment model based on the clinical skin image dataset;

[0036] Lesion detection and quality assessment model test module: used to input the real-time acquired clinical skin image into the lesion detection and quality assessment model, obtain the detection frame corresponding to the skin lesion site, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result;

[0037] Lesion size supervision module: used to set the maximum and minimum thresholds of the proportion of the detection frame area to the overall area of ​​the clinical skin image, and compare the proportion of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum thresholds; when the proportion is less than the minimum threshold or greater than the maximum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image; when the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is judged as qualified and proceeds to the subsequent steps;

[0038] Quality score supervision module: set the minimum threshold of clinical skin image quality score results, compare the real-time clinical skin image quality score results with the minimum threshold, and when the quality score result is lower than the minimum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image; when the quality score result is not lower than the minimum threshold, it is judged as qualified, and the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and its corresponding clinical skin image are retained as acquisition results;

[0039] Shooting module: used to adjust the clinical skin image acquisition equipment to shoot and obtain clinical skin images.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The present invention designs a set of automatic clinical skin image acquisition methods and systems, which realize automatic lesion focusing and high-quality lesion image acquisition by combining a deep learning-based lesion detection network, a lesion image quality assessment network and a subsequent automatic control method. This method can avoid the image acquisition device focusing on the face, other parts or invalid background, which may cause the lesion in the acquired image to be out of focus or overexposed, thereby greatly improving the acquisition quality and efficiency of clinical skin images.

[0042] (2) This method sets the collection standard and quality supervision scheme for clinical skin images, and uses the lesion detection frame and lesion image quality for supervision. First, the lesion size is supervised to ensure that the proportion of the overall clinical skin area captured by the detection frame is reasonable. Then, the quality score supervision module is used to ensure the clarity of the clinical skin image in the detection frame. Automated supervision and adjustment are used throughout the process, ensuring that the size, clarity, exposure and other factors of the lesion image meet the needs of high-quality clinical diagnosis, and can better meet the actual medical standards, especially to improve the lesion screening accuracy in telemedicine scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a schematic diagram of the overall process of the automatic collection method of clinical skin images of the present invention;

[0045] Figure 2 It is a schematic diagram of the process of clinical skin image annotation and data set construction of the present invention;

[0046] Figure 3 is a schematic diagram of the flow chart of the lesion detection and quality assessment model encoding of the present invention; DETAILED DESCRIPTION

[0047] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively and meticulously below in conjunction with the accompanying drawings and preferred embodiments of the present invention, but the protection scope of the present invention is not limited to the following specific embodiments.

[0048] Unless otherwise defined, all professional terms used below have the same meanings as those generally understood by those skilled in the art. The professional terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention.

[0049] Unless otherwise specified, various raw materials, reagents, instruments and equipment used in the present invention can be purchased from the market or prepared by existing methods.

[0050] Example 1

[0051] This embodiment provides a method for automatically collecting clinical skin images, and its overall flow chart is as follows: Figure 1 As shown, the method realizes automatic high-quality image acquisition for skin clinical images by constructing a lesion detection model, a lesion image quality assessment model, and autofocus control based on the above two models. During the test, clinical skin images are obtained through camera devices including but not limited to mobile phones, tablets, etc., and the images are input into the lesion detection model to obtain the lesion detection frame, which can be displayed in real time during the shooting process;

[0052] The specific steps include:

[0053] S1. Preprocessing clinical skin images: cleaning the original clinical skin images, retaining the images containing skin lesions, and filtering out the rest of the images;

[0054] S2. Labeling and division of datasets;

[0055] Generate a detection frame corresponding to the skin lesion in the clinical skin image, score and annotate the ratio of the detection frame area to the overall area of ​​the clinical skin image and the image quality corresponding to the skin lesion in the detection frame; the image quality scoring criteria include image clarity and / or exposure;

[0056] A clinical skin image dataset is constructed based on clinical images, the ratio of the detection box area to the overall area of ​​the clinical skin image, and the annotation of the clinical skin image quality score results; the dataset contains the following contents: Figure 2 As shown;

[0057] The clinical skin image dataset is divided into a training dataset and a test dataset in a ratio of 2:8;

[0058] S3. Based on the clinical skin image training dataset, a lesion detection and quality assessment model is constructed and trained, and based on the clinical skin image test dataset, the lesion detection and quality assessment model is tested; the lesion detection and quality assessment model takes the clinical skin image as input, and the detection box corresponding to the skin lesion, the proportion of the detection box area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result are output;

[0059] The lesion detection and quality assessment model includes an encoder, a decoder, and a regression decoder and a classification decoder composed of a pooling layer and a linear layer; the encoding and decoding process of the encoder, decoder, regression decoder and classification decoder is as follows Figure 3 As shown;

[0060] The encoder encodes the clinical skin image, and the decoder decodes the features encoded by the encoder to obtain the detection frame corresponding to the skin lesion site;

[0061] The regression decoder includes a mean pooling layer and a linear layer. The mean pooling layer pools the encoding, and the linear layer outputs the proportion of the detection box area in the overall area of ​​the clinical skin image.

[0062] The classification decoder contains a mean pooling layer and a linear layer. The mean pooling layer pools the encoding, and the linear layer outputs the clinical skin image quality score result;

[0063] S4. Test the lesion detection and quality assessment model based on the clinical skin image test dataset;

[0064] S5. Collect clinical skin images for testing, input them into the lesion detection and quality assessment model, conduct real-world verification on the lesion detection and quality assessment model, verify the reliability of the detection frame corresponding to the skin lesion, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result, and obtain the lesion detection and quality assessment model after successful verification;

[0065] S6. Inputting the clinical skin image acquired in real time into the lesion detection and quality assessment model to obtain the detection frame corresponding to the skin lesion, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result;

[0066] S7, setting the maximum and minimum thresholds of the proportion of the detection frame area to the overall area of ​​the clinical skin image, and comparing the proportion of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum thresholds; when the proportion is less than the minimum threshold or greater than the maximum threshold, it is determined to be unqualified, and the clinical skin image acquisition device is adjusted, and the process returns to S6 to re-acquire the image; when the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is determined to be qualified, and the process proceeds to S8;

[0067] S8, setting a minimum threshold of the clinical skin image quality score result, comparing the real-time clinical skin image quality score result with the minimum threshold, when the quality score result is lower than the minimum threshold, it is judged as unqualified, adjusting the clinical skin image acquisition device, and entering S9, when the quality score result is not lower than the minimum threshold, it is judged as qualified, and entering S10;

[0068] S9, re-capture the image, and adjust the focus to the center of the detection frame based on the focal length of the detection frame determined in step S7, and re-focus;

[0069] S10, retaining the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and the corresponding clinical skin image as the acquisition result.

[0070] A clinical skin image auto-focus acquisition system used in this embodiment includes an acquisition system including a processor, and the processor includes the following modules:

[0071] Image collection preprocessing module: preprocess clinical skin images;

[0072] Annotation module: used to generate detection frames corresponding to skin lesions in clinical skin images, score and annotate the proportion of the detection frame area in the overall area of ​​the clinical skin image and the image quality corresponding to the skin lesion part in the detection frame; the image quality scoring criteria include image clarity and / or exposure, thereby obtaining a clinical skin image dataset;

[0073] Lesion detection and quality assessment model training module: used to build and train a lesion detection and quality assessment model based on the clinical skin image dataset;

[0074] Lesion detection and quality assessment model test module: used to input the real-time acquired clinical skin image into the lesion detection and quality assessment model, obtain the detection frame corresponding to the skin lesion site, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result;

[0075] Lesion size supervision module: used to set the maximum and minimum thresholds of the proportion of the detection frame area to the overall area of ​​the clinical skin image, and compare the proportion of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum thresholds; when the proportion is less than the minimum threshold or greater than the maximum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image; when the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is judged as qualified and proceeds to the subsequent steps;

[0076] Quality score supervision module: set the minimum threshold of clinical skin image quality score results, compare the real-time clinical skin image quality score results with the minimum threshold, and when the quality score result is lower than the minimum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image; when the quality score result is not lower than the minimum threshold, it is judged as qualified, and the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and its corresponding clinical skin image are retained as acquisition results;

[0077] Shooting module: used to adjust the clinical skin image acquisition equipment to shoot and obtain clinical skin images.

Claims

1. A method for automatic collection of clinical skin images, characterized in that: The following steps are involved: S1, preprocessing clinical skin images; S2, generating a detection frame corresponding to the skin lesion in the clinical skin image; Determine the detection frame area, calculate the proportion of the detection frame area in the overall area of ​​the clinical skin image and mark it; Score the image quality corresponding to the skin lesion area within the detection frame, obtain the clinical skin image quality score result and mark it; S3, build a clinical skin image dataset; S4. Based on the clinical skin image dataset, construct and train a lesion detection and quality assessment model; the lesion detection and quality assessment model takes the clinical skin image as input, and takes the detection box corresponding to the skin lesion, the proportion of the detection box area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result as output; S5, inputting the clinical skin image acquired in real time into the lesion detection and quality assessment model to obtain the detection frame corresponding to the skin lesion, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result; The lesion detection and quality assessment model includes an encoder, a decoder, and a regression decoder and a classification decoder composed of a pooling layer and a linear layer; The encoder encodes the clinical skin image, the decoder decodes the features encoded by the encoder to obtain a detection frame corresponding to the skin lesion site, the pooling layer pools the encoding, and the linear layer outputs the proportion of the detection frame area in the overall area of ​​the clinical skin image and the clinical skin image quality score result; S6, setting a maximum and minimum threshold value of the ratio of the detection frame area to the overall area of ​​the clinical skin image, and comparing the ratio of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum threshold values; When the ratio is less than the minimum threshold or greater than the maximum threshold, it is determined to be unqualified, the clinical skin image acquisition device is adjusted, and the process returns to S5 to re-acquire the image. When the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is determined to be qualified and the process proceeds to S7. S7, setting a minimum threshold of the clinical skin image quality score result, comparing the real-time clinical skin image quality score result with the minimum threshold, when the quality score result is lower than the minimum threshold, it is judged as unqualified, adjusting the clinical skin image acquisition device, and returning to S5 to re-acquire the image; when the quality score result is not lower than the minimum threshold, it is judged as qualified, and entering S8; S8. Retain the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and the corresponding clinical skin image as the acquisition result.

2. The method according to claim 1, characterized in that The image quality scoring criteria include image clarity and / or exposure.

3. The method according to claim 1, characterized in that In step S6, the adjusting of the clinical skin image acquisition device specifically includes: recording the shooting focal length, inputting instructions to the clinical skin image acquisition device, and adjusting the proportion of the detection frame area in the image acquired by the acquisition device to the overall area of ​​the clinical skin image; In step S7, the adjusting of the clinical skin image acquisition device specifically includes: based on the shooting focal length recorded in step S6, inputting instructions to the clinical skin image acquisition device, adjusting the focus of the acquisition device to the center of the detection frame, and refocusing.

4. The method according to claim 3, characterized in that The method of inputting the command includes gesture zooming and / or manual focusing.

5. The method according to any one of claims 1 to 4, characterized in that: In the steps S5 to S8, after the clinical skin image acquired in real time is input into the lesion detection and quality assessment model, the detection frame corresponding to the skin lesion part is synchronously displayed on the shooting interface of the clinical skin image acquisition device.

6. A clinical skin image auto-focus acquisition system for executing the clinical skin image auto-focus acquisition method according to any one of claims 1 to 5, characterized in that: The acquisition system includes a processor, and the processor includes the following modules: Image collection preprocessing module: preprocess clinical skin images; Annotation module: used to generate a detection frame corresponding to the skin lesion in the clinical skin image, determine the area of ​​the detection frame, calculate the proportion of the detection frame area in the overall area of ​​the clinical skin image, and annotate it; Score the image quality corresponding to the skin lesion area in the detection frame, obtain the bed skin image quality score result and mark it; Lesion detection and quality assessment model training module: used to build and train a lesion detection and quality assessment model based on the clinical skin image dataset; Lesion detection and quality assessment model test module: used to input the real-time acquired clinical skin image into the lesion detection and quality assessment model, obtain the detection frame corresponding to the skin lesion site, the proportion of the detection frame area in the overall area of ​​the clinical skin image, and the clinical skin image quality score result; Lesion size supervision module: used to set the maximum and minimum thresholds of the proportion of the detection frame area to the overall area of ​​the clinical skin image, and compare the proportion of the detection frame area to the overall area of ​​the clinical skin image acquired in real time with the maximum and minimum thresholds; When the ratio is less than the minimum threshold or greater than the maximum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image. When the detection frame size is not less than the minimum threshold and not greater than the maximum threshold, it is judged as qualified and proceeds to the subsequent steps; Quality score supervision module: set the minimum threshold of clinical skin image quality score results, compare the real-time clinical skin image quality score results with the minimum threshold, and when the quality score result is lower than the minimum threshold, it is judged as unqualified, and the clinical skin image acquisition device is adjusted to re-acquire the image; when the quality score result is not lower than the minimum threshold, it is judged as qualified, and the detection frame corresponding to the skin lesion site, the clinical skin image quality score result and its corresponding clinical skin image are retained as acquisition results; Shooting module: used to adjust the clinical skin image acquisition equipment to shoot and obtain clinical skin images.

Images