Wheal identification method, system and prick reading system

Abstract

The present invention discloses a wheal identification method, system and prick reading system, which can quickly and accurately identify and analyze the wheals generated at the prick site, reduce the steps of manual measurement, and speed up the process of skin prick test. The technical scheme is as follows: identifying positioning points: preprocessing the captured image, identifying the positioning points in the preprocessed image, and capturing the skin prick image based on the positioning points; identifying wheals: based on the captured skin prick image, identifying and screening the wheals to be measured and their corresponding confidences based on the deep learning method; calculating the wheal measurement value: measuring and calculating the identified and screened wheals, and obtaining the area, maximum diameter and vertical diameter of the wheals to be measured; secondary screening of wheals: based on the obtained confidence and the obtained area, secondary screening and supplementation of the wheals; wheal analysis; secondary screening of wheals: based on the obtained confidence and the obtained area, secondary screening and supplementation of the wheals; wheal analysis.

Description

Technical Field

[0001] The invention relates to a method and system applied to skin prick test, and in particular to a wheal identification method and system and a prick reading system for running the wheal identification method. Background Art

[0002] The skin prick test (SPT), also known as the needle prick test or intradermal test, is a method used to detect the body's sensitivity to specific allergens. The test involves gently pricking the skin (usually the inner forearm) with a prick solution containing a trace amount of allergen extract to observe whether a local allergic reaction is triggered. The test is based on the principle that if a patient is allergic to a substance, their immune system will produce specific IgE antibodies to the substance. When exposed again, these antibodies trigger mast cells and basophils to release inflammatory mediators such as histamine, resulting in the formation of a wheal (i.e., local redness, swelling, and bulge) and / or redness at the puncture site. The prick solution is a solution containing a trace amount of allergen extract that is usually used in the skin prick test. These extracts are carefully prepared to ensure safety and accuracy. Each prick solution targets one or a specific class of allergens, such as a specific type of pollen, food component, or insect venom. When performing the test, a medical professional will use a special prick needle or fine needle to gently puncture the surface of the skin, and then drop the prick solution on the puncture site to allow it to penetrate the skin. Wheal is a typical manifestation of allergic reaction in skin prick test, which refers to the local edema damage caused by temporary inflammatory congestion and large amount of fluid exudation of skin mucosal blood vessels due to allergen stimulation, resulting in local swelling and redness. The size and shape of the wheal vary depending on individual differences, the type of allergen and the degree of allergy. The appearance of wheal indicates that the patient has an allergic reaction to the corresponding allergen. The measurement of wheal (such as diameter) is an important basis for evaluating the intensity of allergic reaction and judging the positive and negative of test results.

[0003] Skin prick test (SPT) has become the most commonly used method for allergen detection in clinical practice because it is simple, convenient, rapid, sensitive and inexpensive. The traditional way to interpret the results of skin prick test is for the collector to trace the outline of the wheal with a marker, transfer it to the special report paper for the prick test with transparent tape, and measure the wheal with a measuring ruler. The positive or negative is determined based on the wheal measurement results.

[0004] Although the skin prick test is quick and easy to perform, the number of prick solutions used in clinical practice continues to increase, and the collector needs to measure the wheal produced by each prick solution in turn, which takes a long time and is highly subjective. Individual experience and judgment accuracy also vary. Summary of the invention

[0005] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0006] The purpose of the present invention is to solve the above-mentioned problems and provide a wheal identification method, system and prick reading system to quickly and accurately identify and analyze the wheals generated at the prick sites, reduce the tedious steps of manual measurement, and speed up the process of skin prick test.

[0007] The technical solution of the present invention is: the present invention discloses a method for identifying a wheal, comprising the following steps:

[0008] Step 1: Identify the positioning points: pre-process the captured image, identify the positioning points in the pre-processed image, and capture the skin prick image based on the positioning points;

[0009] Step 2: Identify wheals: Based on the captured skin prick images, identify and screen the wheals to be measured and their corresponding confidence levels based on the deep learning method;

[0010] Step 3: Calculation of wind group measurement values: Measure and calculate the wind groups identified and screened in step 2 to obtain the area, maximum diameter and vertical diameter of the wind group to be measured;

[0011] Step 4: Secondary screening of wind groups: Based on the confidence obtained in step 2 and the area obtained in step 3, secondary screening and supplementation of wind groups are performed;

[0012] Step 5: Wheal analysis.

[0013] According to an embodiment of the wheal identification method of the present invention, the preprocessing step in step 1 includes: converting the captured RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

[0014] According to one embodiment of the wind group identification method of the present invention, the identification of the positioning point in step 1 adopts a spot detection method, and after the positioning point is identified, it also includes checking the positioning point identification results, and the inspection process includes: checking whether the number of detection results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal coordinate and the vertical coordinate of the center point of each positioning point meets the preset value.

[0015] According to an embodiment of the wheal identification method of the present invention, step 2 further includes the following processing steps:

[0016] First, the skin prick image captured by the positioning point recognition in step 1 is input into the pre-trained wheal model to obtain the potential wheal or raised area;

[0017] The potential wheal or raised areas identified in the previous step are then matched to the pre-set puncture sites.

[0018] According to an embodiment of the wheal identification method of the present invention, the wheal model is a Mask-RCNN wheal model, and the model is trained in the following manner:

[0019] Step 1: Open the sample photo and determine the locations of all wind masses and bulges in the sample photo;

[0020] Step 2: Use the image annotation tool to mark a circle along the outer contours of all wheals and record the category as wheal;

[0021] Step 3: Mark a circle along the outer contours of all the ridges and record this category as ridges;

[0022] Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model. Then, select the sample files containing non-circular irregular shaped wind groups from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model. The second model is used to measure the irregular shaped wind groups that may appear in the samples. The deep learning model is the Mask-RCNN model, and the annotation result file type is json.

[0023] According to an embodiment of the wheal identification method of the present invention, the method for matching the identified potential wheal or raised area with the preset pricking site is:

[0024] First, the coordinates of the center point of each potential wheal or bulge area are obtained, and the puncture site closest to the center point of each potential wheal or bulge area is found, and the distance between the two is required to be less than a minimum distance threshold;

[0025] Then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

[0026] According to one embodiment of the wheal identification method of the present invention, the secondary screening is based on the relationship between any two or three of the wheal confidence, the wheal area or the puncture site attributes, wherein the puncture site attributes include the near-light side site, the negative site, the positive site and / or the specific allergen site.

[0027] According to an embodiment of the wheal identification method of the present invention, in step 3, the measured lengths of the maximum diameter and the vertical diameter of each wheal are calculated based on the true value conversion ratio of the length, and the measured area of ​​the inner region of the wheal contour is calculated based on the true value conversion ratio of the area.

[0028] According to an embodiment of the wheal identification method of the present invention, the maximum diameter and the true length of the vertical diameter and the true area of ​​each wheal are calculated based on the correction ratio.

[0029] According to an embodiment of the wheal identification method of the present invention, in step 4, the wheals identified and screened in step 2 are screened and supplemented for the second time according to the following rules, and "retaining the result" means retaining the binary mask, area, maximum diameter and vertical diameter corresponding to the wheal for wheal analysis, wherein:

[0030] (1) For wheals whose first model confidence is greater than the first confidence preset value and whose area is greater than the first area preset value, the first model recognition result is retained;

[0031] (2) For wheals whose first model confidence is greater than the second confidence preset value and whose area is greater than the second area preset value, the first model recognition result is retained;

[0032] (3) For wheals whose first model confidence is greater than the third confidence preset value and whose area is greater than the third area preset value, the first model recognition result is retained;

[0033] (4) For the wheal corresponding to the puncture site close to the light source, if the first confidence preset value > the first model confidence > the fourth confidence preset value and the area > the third area preset value, the first model recognition result is retained;

[0034] (5) For the positive control wheal, regardless of the confidence of the first model wheal, the first model result is retained; if there is no wheal corresponding to the positive control prick solution in the first model, return to step 2, select the first model and select the output category as "protuberance", repeat steps 2 and 3, find the protuberance corresponding to the positive control prick solution, regard the protuberance as a wheal and retain the result;

[0035] (6) For negative control wheals whose first model confidence is greater than the fifth confidence preset value, the first model recognition results are retained, and all allergen wheals whose first model confidence is greater than the fifth confidence preset value are retained;

[0036] (7) For a house dust mite allergen wheal of which the first model confidence is greater than the sixth confidence preset value and the area is greater than the fourth area preset value, if the house dust mite allergen wheal confidence is greater than the first confidence preset value, the house dust mite allergen wheal identification result of the first model is retained;

[0037] (8) For allergen wheals whose first model confidence is greater than the first model confidence of the positive control wheals, their first model identification results are retained;

[0038] (9) For a wheal with a second model confidence level greater than the seventh confidence level preset value and an area greater than the fifth area preset value, or a wheal with a second model confidence level greater than the eighth confidence level preset value and an area greater than the sixth area preset value at the same point in the first model, the point will be retained based on the recognition result of the second model;

[0039] (10) If there is no potential allergen wheal, that is, the confidence is less than the minimum confidence threshold, then return to step 2, select the first model and select the output category as "uplift", and repeat steps 2 and 3 to calculate all uplift results; if there is only one uplift that reaches the minimum confidence except the positive control, and the uplift confidence is greater than the ninth confidence preset value, then the uplift at this point will be regarded as a wheal and the result will be retained;

[0040] (11) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

[0041] According to one embodiment of the wheal identification method of the present invention, step 5 further includes calculating corresponding analysis indicators for the wheal according to the type of pricking solution, wherein the analysis indicators are selected from the ratio of the wheal area to the positive control area, the ratio of the average diameter of the wheal to the average diameter of the positive control, and the maximum diameter.

[0042] The present invention also discloses a wind group identification system, the system comprising:

[0043] A positioning point recognition module is used to pre-process the captured image, identify the positioning points in the pre-processed image, and capture the skin prick image based on the positioning points;

[0044] A wheal recognition module is used to identify and select the wheals to be measured and their corresponding confidence levels based on the captured skin prick images based on a deep learning method;

[0045] The wind mass measurement value calculation module is used to measure and calculate the wind masses identified and screened by the wind mass identification module to obtain the area, maximum diameter and vertical diameter of the wind mass to be measured;

[0046] A wheal secondary screening module is used to perform secondary screening and supplement of wheals based on the confidence obtained by the wheal identification module and the area obtained by the wheal measurement value calculation module;

[0047] The wind group analysis module is used to perform wind group analysis.

[0048] According to an embodiment of the wind group identification system of the present invention, the preprocessing step of the positioning point identification module is further configured as: converting the captured RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

[0049] According to one embodiment of the wind group identification system of the present invention, the identification of the positioning point by the positioning point identification module adopts the spot detection method, and after the positioning point identification, it also includes checking the positioning point identification results, and the inspection process includes: checking whether the number of detection results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal coordinate and the vertical coordinate of the center point of each positioning point meets the preset value.

[0050] According to an embodiment of the wheal identification system of the present invention, the wheal identification module is further configured to perform the following processing:

[0051] First, the skin prick image captured by the positioning point recognition module is input into the pre-trained wheal model to obtain the potential wheal or bulge area;

[0052] The potential wheal or raised areas identified in the previous step are then matched to the pre-set puncture sites.

[0053] According to an embodiment of the wheal identification system of the present invention, in the wheal identification module, the wheal model is a Mask-RCNN wheal model, and the model is trained in the following manner:

[0054] Step 1: Open the sample photo and determine the locations of all wind masses and bulges in the sample photo;

[0055] Step 2: Use the image annotation tool to mark a circle along the outer contours of all wheals and record the category as wheal;

[0056] Step 3: Mark a circle along the outer contours of all the ridges and record this category as ridges;

[0057] Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model. Then, select the sample files containing non-circular irregular shaped wind groups from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model. The second model is used to measure the irregular shaped wind groups that may appear in the samples. The deep learning model is the Mask-RCNN model, and the annotation result file type is json.

[0058] According to an embodiment of the wheal identification system of the present invention, in the wheal identification module, the method for matching the identified potential wheal or raised area with the preset pricking site is as follows:

[0059] First, the coordinates of the center point of each potential wheal or bulge area are obtained, and the puncture site closest to the center point of each potential wheal or bulge area is found, and the distance between the two is required to be less than a minimum distance threshold;

[0060] Then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

[0061] According to one embodiment of the wheal identification system of the present invention, the secondary screening is based on the relationship between any two or three of the wheal confidence, the wheal area or the puncture site attributes, wherein the puncture site attributes include the near-light side site, the negative site, the positive site and / or the specific allergen site.

[0062] According to an embodiment of the wheal identification system of the present invention, in the wheal measurement value calculation module, the measured lengths of the maximum diameter and the vertical diameter of each wheal are calculated based on the true value conversion ratio of the length, and the measured area of ​​the internal area of ​​the wheal contour is calculated based on the true value conversion ratio of the area.

[0063] According to an embodiment of the wheal identification system of the present invention, in the wheal measurement value calculation module, the maximum diameter and the true length of the vertical diameter and the true area of ​​each wheal are calculated based on the correction ratio.

[0064] According to an embodiment of the wheal identification system of the present invention, in the wheal secondary screening module, the wheals identified and screened by the wheal identification module are screened and supplemented for the second time according to the following rules, and the "retained result" means retaining the binary mask, area, maximum diameter and vertical diameter corresponding to the wheal for wheal analysis, wherein:

[0065] (1) For wheals whose first model confidence is greater than the first confidence preset value and whose area is greater than the first area preset value, the first model recognition result is retained;

[0066] (2) For wheals whose first model confidence is greater than the second confidence preset value and whose area is greater than the second area preset value, the first model recognition result is retained;

[0067] (3) For wheals whose first model confidence is greater than the third confidence preset value and whose area is greater than the third area preset value, the first model recognition result is retained;

[0068] (4) For the wheal corresponding to the puncture site close to the light source, if the first confidence preset value > the first model confidence > the fourth confidence preset value and the area > the third area preset value, the first model recognition result is retained;

[0069] (5) For the positive control wheal, regardless of the confidence of the first model wheal, the first model result is retained; if there is no wheal corresponding to the positive control prick solution in the first model, return to the wheal identification module, select the first model and select the output category as "protrusion", repeat the wheal identification module and the wheal measurement value calculation module, find the protrusion corresponding to the positive control prick solution, regard the protrusion as a wheal and retain the result;

[0070] (6) For negative control wheals whose first model confidence is greater than the fifth confidence preset value, the first model recognition results are retained, and all allergen wheals whose first model confidence is greater than the fifth confidence preset value are retained;

[0071] (7) For a house dust mite allergen wheal of which the first model confidence is greater than the sixth confidence preset value and the area is greater than the fourth area preset value, if the house dust mite allergen wheal confidence is greater than the first confidence preset value, the house dust mite allergen wheal identification result of the first model is retained;

[0072] (8) For allergen wheals whose first model confidence is greater than the first model confidence of the positive control wheals, their first model identification results are retained;

[0073] (9) For a wheal with a second model confidence level greater than the seventh confidence level preset value and an area greater than the fifth area preset value, or a wheal with a second model confidence level greater than the eighth confidence level preset value and an area greater than the sixth area preset value at the same point in the first model, the point will be retained based on the recognition result of the second model;

[0074] (10) If there is no potential allergen wheal, that is, the confidence is less than the minimum confidence threshold, then return to the wheal identification module, select the first model and select the output category as "protrusion", and repeat the wheal identification module and the wheal measurement value calculation module to calculate all protrusion results; if there is only one protrusion that reaches the minimum confidence except the positive control, and the protrusion confidence is greater than the ninth confidence preset value, then the protrusion at this point will be regarded as a wheal and the result will be retained;

[0075] (11) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

[0076] According to one embodiment of the wheal identification system of the present invention, the wheal analysis module further includes calculating corresponding analysis indicators for the wheal according to the type of pricking fluid, wherein the analysis indicators are selected from the ratio of the wheal area to the positive control area, the ratio of the average diameter of the wheal to the average diameter of the positive control, and the maximum diameter.

[0077] The present invention also discloses a computer device for wheal identification, comprising a memory, a processor, and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the wheal identification method described above.

[0078] The present invention also discloses a computer-readable storage medium for wheal identification, which stores program instructions executable by a processor to implement the steps of the wheal identification method as described above.

[0079] The present invention also discloses a prick reading system, including a prick reader, which includes a chassis, an arm lift pad, an image acquisition mechanism and an image processing mechanism, wherein the image acquisition mechanism includes a camera and a light source for acquiring the skin prick area, and the image processing mechanism is configured to run the wheal identification method as described above, so as to analyze the wheal and output the size and / or detection result of the wheal.

[0080] According to one embodiment of the prick reading system of the present invention, the image acquisition mechanism is arranged above the arm lifting pad, wherein the light source is located above the front of the palm and the irradiation surface is parallel to the front; the image processing mechanism is an industrial control computer, and a touch screen is arranged on the industrial control computer, which is configured to display the real-time picture and / or the image of the wind group and its associated data captured by the image acquisition mechanism to the user.

[0081] According to an embodiment of the dot prick reading system of the present invention, the case is lightproof and has an opening at the front end thereof for placing the arm into the case.

[0082] According to an embodiment of the prick reading system of the present invention, a light shielding device adapted to arms of different thicknesses is provided at the opening at the front end of the case.

[0083] According to an embodiment of the prick reading system of the present invention, the light blocking device is made of a light blocking cloth surrounded by soft cloth, with an opening in the middle, and the size of the opening is limited by an elastic rubber band.

[0084] According to one embodiment of the prick reading system of the present invention, the light blocking device is composed of a plurality of light blocking sheets which are interconnected. The light blocking sheets are installed at the lower part of the front door of the host through a slide rail. The handle passes through a slide groove in the middle of the front door of the host and is connected to the light blocking sheets. The opening and closing state of the light blocking sheets is controlled by the handle. The spring sheet buckle arranged on the front door of the host is used to clamp the light blocking sheets when the handle is pulled to the highest point. The handle is pulled downward to disengage the light blocking sheets from the spring sheet buckle to close the entrance.

[0085] According to an embodiment of the dot prick reading system of the present invention, a buckle-type door lock is installed on the rear side of the host front door for self-locking when the host front door is pushed closed.

[0086] According to an embodiment of the prick reading system of the present invention, the arm support pad is arranged in the front cavity of the chassis, and the arm support pad has an upward slope to fix the arm. The end of the arm support pad changes from a slope to a flat slope, which is used to place the wrist horizontally to increase comfort.

[0087] According to an embodiment of the prick reading system of the present invention, the arm support pad is detachably connected to the chassis.

[0088] According to an embodiment of the prick reading system of the present invention, a bumper structure is installed under the arm lift pad to fix the arm lift pad in the bumper structure on the bottom surface of the chassis to achieve rapid installation and removal of the arm lift pad.

[0089] According to an embodiment of the dot prick reading system of the present invention, a printer placement platform is further provided at the rear of the chassis to place an additionally configured printer.

[0090] Compared with the prior art, the present invention has the following beneficial effects: the present invention identifies wind groups from the captured images based on the Mask-RCNN wind group model, and automatically measures and calculates the parameters of the wind groups, and finally obtains the wind group identification detection results. Compared with the prior art, the present invention can transform the measurement and interpretation of wind groups from manual methods to computer automatic methods, which reduces working time, improves work efficiency, and also avoids the subjectivity of manual methods, further improving the accuracy of judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0092] Figures 1A to 1E FIG. 1 is a view of a dot prick reader in one embodiment of the dot prick reading system of the present invention.

[0093] Figure 2 yes Figures 1A to 1E The internal structure diagram of the prick reader is shown.

[0094] Figure 3 It is a flow chart of an embodiment of the wheal identification method of the present invention.

[0095] Figure 4 It is a schematic diagram of an embodiment of a wind group identification system of the present invention.

[0096] Figure 5 It is a diagram that labels the wheals and / or ridges.

[0097] Figure 6It is a schematic diagram for finding all wind groups that meet the regional range and calculating their confidence.

[0098] Figure 7 It is a schematic diagram for finding the maximum diameter L and vertical diameter D of all wind groups.

[0099] Figure 8 It is a schematic diagram for calculating the center distance and correction ratio of the left and right positioning points. DETAILED DESCRIPTION

[0100] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Note that the aspects described below in conjunction with the accompanying drawings and specific embodiments are only exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0101] Figures 1A to 1E FIG. 1 is a view of an embodiment of a prick reader in an embodiment of a prick reading system of the present invention. Figure 2 The internal structure of the prick reader embodiment is shown. Figures 1A to 1E as well as Figure 2 As shown, the prick reader of this embodiment includes: a chassis 1, an arm lifting pad 5, an image acquisition mechanism and an image processing mechanism. The image acquisition mechanism is arranged above the arm lifting pad 5, and includes a camera 8 and a light source 9 for acquiring the skin prick area. The light source 9 is located above the palm, and the irradiation surface is parallel to the front, so the arm close to the palm is the low beam side, the color temperature of the light source is selected from 2000-6000K, and the power is selected from 0.5-30w. Preferably, the color temperature is 4000K and the power is 6W / m. The image processing mechanism is an industrial control computer 2, which is configured to analyze the wind group and output the size and / or detection result of the wind group. The size of the wind group can be the maximum diameter of the wind group, the vertical diameter and / or the area of ​​the wind group. The image processing mechanism may include a graphical user interface, for example, a touch screen is set on the industrial control computer 2, and is configured to display the real-time picture captured by the image acquisition mechanism and / or the image of the wind group and its associated data to the user.

[0102] The industrial control computer 2 is located at the upper front part of the chassis 1, and the host front door 6 is located at the lower front part of the chassis 1. The industrial control computer 2 and the host front door 6 are directly mounted on the chassis 1. The chassis 1 is light-proof. Preferably, the inner surface of the chassis is black to prevent the internal components from affecting the photographing results. An opening is provided at the front end of the chassis 1 for placing the arm into the chassis. A light-blocking device that can adapt to arms of different thicknesses is provided at the opening to reduce the influence of external light on the photography inside the instrument. The light-blocking device can be made of a light-blocking cloth surrounded by soft cloth, with an opening in the middle, and the opening size is limited by an elastic rubber band. It can also be as follows Figure 1EThe form shown in the figure is composed of three mutually linked light shielding sheets 4. The light shielding sheets 4 are installed at the lower part of the mainframe front door 6 through a slide rail. The handle 3 passes through the slide groove in the middle of the mainframe front door 6 and is connected to the light shielding sheet 4. The handle 3 controls the opening and closing state of the light shielding sheet 4. When the handle 3 is pulled upward, the three mutually linked light shielding sheets 4 will move along the Figure 1E Slide in the direction of the middle arrow to open the opening (i.e., the entrance for the arm to extend into the instrument). When the handle 3 is pulled to the highest point, the opening size is the largest. Pull the handle downward to reduce the opening size until the three light shielding sheets 4 touch the arm. The buckle-type door lock 10 and the spring sheet buckle 11 are both installed on the rear side of the main machine front door 6. The arm lifting pad 5 is located at the front lower part of the interior of the chassis 1, the camera 8 is located at the front upper part of the interior of the chassis 1, and the light source 9 is located at the middle and rear part of the interior of the chassis 1. The arm lifting pad 5, the camera 8, and the light source 9 are all directly installed in the chassis 1.

[0103] The arm support pad 5 has an upward slope and is arranged in the front cavity of the chassis 1, and is used to fix the arm at a comfortable and favorable angle for taking pictures to ensure the imaging effect. The end of the arm support pad 5 changes from a slope to a flat slope, which is used to place the wrist horizontally to increase comfort.

[0104] When placing the arm, make a fist with the arm to be tested, with the palm side of the forearm facing upwards, and slowly extend it into the instrument along the slope of the arm lifting pad 5 facing the opening until the arm reaches a suitable position, and then place the arm on the arm lifting pad 5. Next, pull the handle 3 downward by hand to close the entrance until the three light shielding sheets 4 touch the arm, and then stop pulling the handle 3. When the handle 3 is at the lowest point, the opening size is the smallest.

[0105] The arm lifting pad 5 is connected to the chassis 1 in a detachable manner, such as a bumper buckle, a magnetic structure, etc. Preferably, a bumper structure is provided below the arm lifting pad 5 to fix the arm lifting pad 5 in the bumper structure on the bottom surface of the chassis 1, so as to achieve quick installation and removal of the arm lifting pad 5. The material of the arm lifting pad 5 is not limited, and can be metal, plastic or soft material. Preferably, the material of the arm lifting pad 5 is ABS.

[0106] The host front door 6 is arranged at the front side of the chassis, and is used to open the front cavity of the prick reader, and install, maintain or replace the arm support pad. A buckle-type door lock 10 is arranged on the host front door 6, which can be self-locked when the host front door 6 is pushed closed. A spring sheet buckle 11 is arranged on the host front door 6, which can clamp the light shielding sheet 4 when the handle 3 is pulled to the highest point, thereby preventing the subject from falling and causing damage to the hand when the subject extends the arm into the chassis. When the subject has placed the arm correctly, the handle 3 is pulled downward to disengage the light shielding sheet 4 from the spring sheet buckle 11, thereby closing the entrance.

[0107] A printer placement platform 7 is also provided at the top of the rear part of the chassis, on which an additionally configured printer can be placed to print the prick measurement results.

[0108] The method of using the prick reader is as follows.

[0109] Step 1: Mark the positioning points in the puncture area.

[0110] Select a suitable area on the palm side of the subject's forearm and mark the positioning point with a puncture template as a positioning mark for image acquisition. The puncture template can be a tattoo sticker with positioning points and puncture sites pre-printed on it, which is directly attached to the palm side of the subject's forearm, or it can be a positioning ruler with positioning holes and puncture site holes. Use a marker to draw the positioning point on the palm side of the subject's arm. The positioning point can be a blue dot.

[0111] Step 2: Prick test.

[0112] The standard prick test operation is performed in the positioning area according to the pre-set prick points on the prick template. In one embodiment, the prick points are evenly distributed between the positioning points. The allergen prick solution, positive control solution and negative control solution required for prick are inserted into the prick area corresponding to the prick template, and image acquisition is prepared 15 to 20 minutes after the prick.

[0113] Step 3: Image acquisition.

[0114] Pull the handle 3 located at the center of the front end of the instrument upwards, and then three interlocking light shielding sheets 4 will move along the Figure 1E Slide in the direction of the arrow in the figure to open the entrance. Open the opening to a suitable size for the arm to pass through. Make a fist with the arm to be tested, with the palm side of the forearm facing up, and slowly extend it into the instrument along the slope of the arm lifting pad 5 facing the opening until the arm reaches a suitable position, and then place the arm on the arm lifting pad 5. Pull the handle 3 downward by hand to close the entrance until the three light shielding sheets 4 touch the arm, and then stop pulling the handle 3. When the handle is at the lowest point, the opening size is the smallest.

[0115] The subject observes the red frame (preset arm placement area) on the monitor screen and adjusts the arm position so that the positioning marks all fall inside the red frame. The instrument will automatically determine whether the arm has been placed as required. The judgment is based on whether the positioning points on the arm can be clearly identified. For specific steps, please refer to the first step of the wind group identification method below, positioning point identification. If it meets the requirements, the image will be automatically collected and a prompt will be given in the software interface. After that, the subject will extend his arm from the opening of the device. Wait for the machine to analyze the image, and automatically print the result report after the analysis is completed.

[0116] Figure 3 This is a flow chart of an embodiment of the wind group identification method of the present invention. Figure 3, the following is a detailed description of each step of the method of this embodiment.

[0117] Step 1: Identify the positioning points: pre-process the real-time image, identify the positioning points in the pre-processed image, and capture the skin prick image based on the positioning points.

[0118] The preprocessing steps include: converting the RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

[0119] Specifically, the preprocessing steps are as follows: convert the RGB three-channel image with a resolution of 2160×3840 into a grayscale image; compress the grayscale image resolution to 1 / 8 in both length and width by interpolation to obtain a 270×480 image; perform median filtering on the compressed image for smoothing, where kernelsize (kernel size, representing the size of the filter window, that is, the neighborhood range considered for each pixel when performing median filtering) is set to 5.

[0120] The identification of the positioning point adopts the blob detection method. Specifically, the SimpleBlobDetector_create operator in OpenCV is used to perform blob detection on the median filtered image, and the minimum area of ​​the blob minArea is set to 30. The SimpleBlobDetector_create operator in OpenCV is a function used to create a blob detector (Blob Detector). This function belongs to the feature detection module in the OpenCV library and is used to detect small and bright areas in the image. These areas are usually called spots or plaques.

[0121] After the positioning points are identified, the positioning point identification results need to be checked, including: checking whether the number of test results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal and vertical coordinates of the center point of each positioning point meets the preset value.

[0122] Specifically, first check whether the Blob detection result is 4 points. If not, recapture the camera screen and return to the step of preprocessing the image. Then multiply the horizontal and vertical coordinates of the center points of the 4 positioning points detected by the Blob by 8 to obtain the coordinate values ​​in the 2160×3840 image. Divide the pre-given red frame area in the screen into four areas of upper left UL, lower left DL, upper right UR, and lower right DR according to the coordinates of the four corners. Check whether the center point of the positioning point detected by the Blob falls in one of the upper left UL, lower left DL, upper right UR, and lower right DR areas. If not, recapture the camera screen and return to the step of preprocessing the image. Finally, check whether the difference in the horizontal coordinates of the center points of the upper left and lower left positioning points, the difference in the horizontal coordinates of the center points of the upper right and lower right positioning points, the difference in the vertical coordinates of the center points of the upper left and upper right positioning points, and the difference in the vertical coordinates of the center points of the lower left and lower right positioning points are all less than 200. If they meet the requirements, take a photo and save the skin pricking image and the positions of the four center points. If not, recapture the camera screen and return to the step of preprocessing the image.

[0123] In a specific example, the detailed processing of step 1 is as follows:

[0124] Step S101: Capture the camera image in real time to obtain an RGB three-channel image with a resolution of 2160×3840;

[0125] Step S102: converting the RGB three-channel image with a resolution of 2160×3840 into a grayscale image;

[0126] Step S103: compressing the grayscale image resolution to 1 / 8 in both length and width by interpolation to obtain an image of 270×480;

[0127] Step S104: performing median filtering on the compressed image, wherein kernelsize is set to 5;

[0128] Step S105: Use the SimpleBlobDetector_create operator in OpenCV to perform blob detection on the median filtered image, and set the minimum area minArea to 30;

[0129] Step S106: Check whether the Blob detection result is 4 points. If not, recapture the camera image and return to step S101;

[0130] Step S107: multiply the horizontal and vertical coordinates of the center points of the four positioning points detected by the Blob by 8 to obtain the coordinate values ​​in the 2160×3840 image;

[0131] Step S108: Divide the pre-given red frame area in the screen into four areas of upper left, lower left, upper right, and lower right according to the coordinates of the four corners;

[0132] Step S109: Check whether the center point of the positioning point detected by the Blob falls in one of the upper left UL, lower left DL, upper right UR, and lower right DR areas. If not, recapture the camera image and return to step S101;

[0133] Step S110: Check whether the difference between the horizontal coordinates of the center points of the upper left and lower left positioning points, the difference between the horizontal coordinates of the center points of the upper right and lower right positioning points, the difference between the vertical coordinates of the center points of the upper left and upper right positioning points, and the difference between the vertical coordinates of the center points of the lower left and lower right positioning points are all less than 200. If so, take a photo and save the three-channel image and the center point positions of the four positioning points. If not, recapture the camera screen and return to step S101.

[0134] Step 2: Identify wheals: Based on the captured skin prick images, the wheals to be measured and their corresponding confidence levels are identified and screened based on the deep learning method.

[0135] Step 2 further includes the following processing steps.

[0136] First, the skin prick image captured by the positioning point recognition in step 1 is input into the first model and the second model of the pre-trained Mask-RCNN wheal to obtain potential wheal or bulge areas that meet the minimum wheal confidence threshold.

[0137] The Mask-RCNN wheal model is constructed in the following way. The collected images are used as sample photos, and the manually annotated data is added to train the wheal recognition model. By observing the sample photos, it can be seen that large and small bumps will appear on the arm after the prick test. These bumps may be caused by the wheals produced by the reaction of the skin and the solution, or they may be small bumps caused by the needle. Therefore, it is necessary to distinguish between the two in the stage of manually annotating data to improve the ability to accurately identify wheals. The training steps of the Mask-RCNN wheal model are as follows: Steps 1 to 4:

[0138] Step 1: Open a specific sample photo and determine the locations of all wheals and small ridges in the photo based on the process records of the prick test and the photo review;

[0139] Step 2: Use image annotation tools, such as labelme, to mark a circle along the outer contours of all wheals and record the category as wheal, for example Figure 5 The sites numbered 1, 3, 4, 5, 8, 9, and 10 in Figure 5In the figure, the gray protrusions, which are marked with numbers 1, 3, 4, 5, 8, 9, and 10, represent wind masses, the black dots, which are marked with numbers 2, 6, and 7, represent ridges, and the four unnumbered black circles represent positioning points);

[0140] Step 3: Mark a circle along the outer contours of all small ridges and record the category as ridges, for example Figure 5 The site numbered 2;

[0141] Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model; then select the sample files containing non-circular irregular shaped wind groups from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model, which is specifically used to measure the irregular shaped wind groups that may appear in the samples; the Mask-RCNN model is used in this embodiment, and the annotation result file type is json.

[0142] After completing the Mask-RCNN model training, a sample image is input into the first model and the second model respectively, and the result of outputting the classification as "wind group" is selected, and the results of all the wind groups identified by the model in this figure are obtained. Each wind group corresponds to a mask, where mask is a binary variable equivalent to the resolution of the photo, and the non-zero element represents the area where the wind group is located. At the same time, each wind group corresponds to a confidence score between 0 and 1, which is recorded as "wind group confidence". The larger the wind group confidence value, the higher the probability that it is a wind group. Similarly, the result of outputting the classification as "uplift" is selected, and the results of all the uplifts identified in the model of this figure are obtained. At the same time, each uplift also corresponds to a confidence score between 0 and 1, which is recorded as "uplift confidence".

[0143] Then, the potential wheal or raised area identified in the previous step is matched with the preset puncture sites. The specific matching method is: first, the coordinates of the center point of each potential wheal or raised area are obtained, and the puncture site closest to the center point of each potential wheal or raised area is found and the distance between the two is required to be less than the minimum distance threshold; then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

[0144] Combination Figure 6 As shown, in a specific example, the specific processing steps of wind group identification are as follows.

[0145] Step S201: Use the picture as input variable to input the trained Mask-RCNN wind group first model and second model respectively, select the output category as "wind group", and calculate the mask of each wind group in the picture.

[0146] Step S202: Set the minimum wheal confidence threshold of potential wheals (for example, 0.1) to remove redundant masks; based on the positioning point positions obtained in the positioning point identification in step 1 (upper left UL, lower left DL, upper right UR, lower right DR), calculate the theoretical position of the center of each acupuncture point according to the following formula (assuming that there are 2N points in two rows, w represents the corresponding number of each acupuncture point):

[0147] Number 1 to number N: ;

[0148] Number N+1 to number 2N: ;

[0149] Step S203: Setting the minimum vertical distance threshold , minimum horizontal distance threshold ; where [0] represents the vertical coordinate and [1] represents the horizontal coordinate.

[0150] Step S204: For the wheals that meet the confidence level greater than the minimum wheal confidence level threshold, calculate the center position of the wheal according to the mask, and calculate the distance from the center of the wheal to the theoretical center of the acupuncture sites with different numbers obtained in step S202. The one with the smallest distance is the corresponding acupuncture number w. If the longitudinal distance between the wheal and the theoretical center of the acupuncture site with number w is less than And the horizontal distance is less than , then it is determined that the wheal is the corresponding pricking fluid numbered w; otherwise, the wheal does not correspond to any pricking fluid.

[0151] Step S205: For a certain pricking fluid numbered w, if there are multiple wheals corresponding to it, only the wheal with the highest confidence is retained and the remaining wheals are deleted, thereby ensuring that each pricking fluid has only one wheal corresponding to it.

[0152] Step 3: Calculation of wind group measurement values: Calculate the area, maximum diameter and vertical diameter of the wind group identified and screened in step 2; the maximum diameter is the maximum distance between any two points of the wind group outline, and the vertical diameter is the maximum distance between two points of the wind group outline in the vertical direction of the maximum diameter.

[0153] In one embodiment, the measured length (in mm) of the maximum diameter and the vertical diameter of each wheal can be calculated based on the true value conversion ratio of the length, and the measured area (in mm²) of the inner region of the wheal contour can be calculated based on the true value conversion ratio of the area.

[0154] In one embodiment, the true length (in mm) of the maximum diameter and vertical diameter of each wind group and the true area (in mm²) can also be calculated based on the correction ratio.

[0155] Combination Figure 7 As shown, in a specific example, the specific method for calculating the wheal measurement value is as follows:

[0156] For the binary mask of each prick fluid wheal obtained in step 2 wheal identification, the measurement value of the corresponding wheal is calculated according to the following steps:

[0157] Step S301: directly count the non-zero elements in mask to obtain Figure 7 The number of pixels A on .

[0158] Step S302: Get the contour of the wheal based on the findContours function in OpenCV (in OpenCV, the findContours function is used to extract contours from binary images), traverse the points on the contour, find the farthest point pair, and record the distance between the farthest point pair as the number of pixels of the wheal. Figure 7 The maximum diameter L on the surface.

[0159] Step S303: Draw a perpendicular line in the vertical direction of the farthest point pair to obtain two intersection points with the contour, traverse different perpendicular line intersection points, find the point pair with the farthest perpendicular line distance, and record the distance pixel number of the point pair with the farthest perpendicular line distance as the wind group Figure 7 The vertical diameter D on it.

[0160] Step S304: Based on the previously identified positions of the anchor points UL, UR, DL, and DR on the arm, the number of pixels V in the area enclosed by the anchor points is calculated according to the following formula:

[0161] V = | (UL[0] - DR[0]) (UR[1] - DL[1]) - (UL[1] - DR[1]) (UR[0] -DL[0]) | / 2

[0162] Among them, [0] represents the vertical coordinate, and [1] represents the horizontal coordinate;

[0163] Step S305: Taking the actual size L×H (e.g., 102.2 mm×20 mm) of the area enclosed by the positioning point as a reference, the length real value conversion ratio and the area real value conversion ratio are calculated according to the following formula:

[0164] Length true value conversion ratio = ;

[0165] Area real value conversion ratio = ;

[0166] Step S306: According to the following formula, the actual length (in mm) of the maximum diameter and the actual length (in mm) of the vertical diameter of each wheal are calculated based on the length true value conversion ratio, and the actual area (in mm²) of the inner area of ​​the wheal contour is calculated based on the area true value conversion ratio:

[0167] Maximum diameter of wind mass = maximum diameter on the graph * conversion ratio of actual length,

[0168] The vertical diameter of the wind mass = the vertical diameter on the graph * the conversion ratio of the actual length,

[0169] Wheal area = number of pixels on the image * area true value conversion ratio.

[0170] Step S307: Considering that the armrest is not perpendicular to the camera lens, the distance between the arm and the camera is different at different positions, which may make the wind group close to the camera lens larger in the picture and the wind group far from the camera lens smaller in the picture. Therefore, the wind group measurement value obtained in the above steps needs to be corrected. Taking the positioning point as the reference point, a reasonable correction method should make the actual measurement value between the positioning points on both sides of the arm 20mm, that is, UL and UR are 20mm apart, and DL and DR are 20mm apart. Therefore, combined with Figure 8 As shown, the design correction steps are as follows:

[0171] Calculate the left and right positioning point measurement distances according to the following formula:

[0172] Left positioning point measurement distance = ||UL - UR|| * length true value conversion ratio,

[0173] Right positioning point measurement distance = ||DL - DR|| * length true value conversion ratio.

[0174] Step S308: Calculate the left positioning point correction ratio and the right positioning point correction ratio according to the following formula:

[0175] Left correction ratio = left positioning point spacing / 20mm

[0176] Right correction ratio = right positioning point spacing / 20mm.

[0177] Step S309: Calculate the correction ratio of each wind group. For the wind group numbered k, calculate according to the following formula:

[0178] If k≤N, the correction ratio is:

[0179] .

[0180] If N < k ≤ 2N, the correction ratio is:

[0181] .

[0182] Step S310: Correct the wheal measurement values corresponding to each number according to the correction ratio corresponding to each number:

[0183] Maximum wheal diameter (after correction) = Maximum wheal diameter * Correction ratio,

[0184] Vertical wheal diameter (after correction) = Vertical wheal diameter * Correction ratio,

[0185] Wheal area (after correction) = Wheal area * Correction ratio².

[0186] Step 4, secondary screening of wheals: Based on the wheal confidence obtained in Step 2 and the wheal area obtained in Step 3, perform secondary screening and supplementation on the wheals. The wheals that are screened out are subjected to subsequent wheal analysis, and the areas that are not screened out are considered to have no wheals. Specifically, the secondary screening is based on the relationship between any two or three of the wheal confidence, wheal area, or prick site attributes, where the prick site attributes include near-light-side sites, negative sites, positive sites, and / or specific allergen sites.

[0187] The area may be the measured area or the true area.

[0188] In a specific example, Step 4 further includes the following processing steps.

[0189] Perform secondary screening and supplementation on the wheals identified and screened in Step 2 according to the following rules. "Retaining the result" means retaining the binary mask, area, maximum diameter, and vertical diameter corresponding to the wheal for wheal analysis.

[0190] (1) For wheals with a first model confidence > first confidence preset value (e.g., 0.7) and area > first area preset value (e.g., 5 mm²), retain their first model recognition results;

[0191] (2) For wheals with a first model confidence > second confidence preset value (e.g., 0.65) and area > second area preset value (e.g., 6 mm²), retain their first model recognition results;

[0192] (3) For wheals with a first model confidence > third confidence preset value (e.g., 0.3) and area > third area preset value (e.g., 8 mm²), retain their first model recognition results;

[0193] (4) For the wheal corresponding to the puncture site close to the light source (number 1 or number N+1), if the first confidence preset value > the first model confidence > the fourth confidence preset value (e.g., 0.6) and the area > the third area preset value (e.g., 8 mm²), the first model recognition result is retained;

[0194] (5) For the positive control wheal, regardless of the confidence of the first model wheal, the first model result is retained; if there is no wheal corresponding to the positive control prick solution in the first model (i.e., the confidence is less than the minimum confidence threshold), return to step 2, select the first model and select the output category as "protrusion", repeat steps 2 and 3, find the protrusion corresponding to the positive control prick solution, regard the protrusion as a wheal and retain the result;

[0195] (6) For negative control wheals whose first model confidence is greater than the fifth confidence preset value (e.g., 0.6), the first model recognition results are retained, and all allergen wheals whose first model confidence is greater than the fifth confidence preset value are retained;

[0196] (7) For a house dust mite allergen wheal of which the first model confidence is greater than the sixth confidence preset value (e.g., 0.3) and the area measurement value is greater than the fourth area preset value (e.g., 5 mm²), if the dust mite allergen wheal confidence is greater than the first confidence preset value, the first model house dust mite allergen wheal identification result is retained;

[0197] (8) For allergen wheals whose first model confidence is greater than the first model confidence of the positive control wheals, their first model identification results are retained;

[0198] (9) For a wheal with a second model confidence > the seventh confidence preset value (e.g., 0.98) and an area > the fifth area preset value (e.g., 35 mm²), or a wheal with a second model confidence > the eighth confidence preset value (e.g., 0.9) and an area of ​​the same point in the first model > the sixth area preset value (e.g., 20 mm²), the point will be retained based on the recognition result of the second model;

[0199] (10) If there is no potential allergen wheal, that is, the confidence is less than the minimum wheal confidence threshold, then return to step 2, select the first model and select the output category as "uplift", and repeat steps 2 and 3 to calculate all uplift results; if there is only one uplift that reaches the minimum confidence except the positive control, and the uplift confidence is greater than the ninth confidence preset value (for example, 0.8), then the uplift at this point will be regarded as a wheal and the result will be retained;

[0200] (11) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

[0201] Among them, the seventh confidence preset value>the eighth confidence preset value, the first confidence preset value>the second confidence preset value>the third confidence preset value>the minimum confidence, and the first area preset value<the second area preset value<the third area preset value<the sixth area preset value / the fifth area preset value.

[0202] In another specific example, step 4 further includes the following processing steps.

[0203] According to the following rules, only the wheals identified and screened by the first model in step 2 are screened and supplemented for the second time. "Retaining the results" means retaining the binary mask, area, maximum diameter and vertical diameter corresponding to the wheal for wheal analysis.

[0204] (1) For wheals with a confidence level greater than the first preset confidence level (e.g., 0.8) and an area measurement value greater than the first preset area value (e.g., 8 mm²), the results are retained;

[0205] (2) For the wheals at the puncture site close to the light source (number 1 or number N+1), if the confidence level is greater than the fourth confidence level preset value (e.g., 0.6) and the area measurement value is greater than the third area preset value (e.g., 8 mm²), the results are retained;

[0206] (3) For positive control wheals, the measurement results are retained regardless of their confidence level;

[0207] (4) For negative wheals with a confidence level greater than the fifth confidence level preset value (e.g., 0.3), their measurement results are retained;

[0208] (5) For allergen wheals with a confidence level greater than the tenth confidence level preset value (e.g., 0.2) and an area measurement value greater than the seventh area preset value (e.g., 10 mm²) or the positive control area measurement value, their measurement results shall be retained;

[0209] (6) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

[0210] Step 5: Wheal analysis: Calculate the corresponding analysis index for the wheal according to the type of pricking solution. The analysis index is selected from:

[0211] (1) Ratio of wheal area to positive control area;

[0212] (2) The ratio of the average diameter of the wheal to the average diameter of the positive control, that is, (average diameter of the prick fluid – average diameter of the negative control) / (average diameter of the positive control – average diameter of the negative control), where average diameter = (maximum diameter + vertical diameter) / 2;

[0213] (3) Maximum diameter.

[0214] In a specific example, the specific method of wheal analysis is as follows: according to the type of pricking solution, the corresponding analysis index is calculated for the wheal, and the analysis index is selected from:

[0215] (1) For dust mite allergen wheals, calculate the ratio of the wheal area to the positive control area as the analysis indicator.

[0216] (2) For house dust mite allergen wheals, calculate the ratio of the average diameter of the wheals to the average diameter of the positive control as the analysis index: (average diameter of the prick fluid – average diameter of the negative control) / (average diameter of the positive control – average diameter of the negative control), where average diameter = (maximum diameter + vertical diameter) / 2.

[0217] (3) For other allergen wheals, the maximum diameter is directly used as the analysis indicator.

[0218] Figure 4 The principle of an embodiment of the wheal identification system of the present invention is shown. The wheal identification system of this embodiment includes: a positioning point identification module, a wheal identification module, a wheal measurement value calculation module, a wheal secondary screening module, and a wheal analysis module.

[0219] The positioning point recognition module is used to pre-process the captured image, identify the positioning points in the pre-processed image, and capture the skin pricking image based on the positioning points.

[0220] The preprocessing step of the positioning point recognition module is further configured as follows: converting the captured RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

[0221] The positioning point recognition module adopts the spot detection method to recognize the positioning points, and after the positioning point recognition, it also includes checking the positioning point recognition results. The inspection process includes: checking whether the number of detection results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal and vertical coordinates of the center point of each positioning point meets the preset value.

[0222] The wheal recognition module is used to identify and screen the wheals to be measured and their corresponding confidence levels based on the captured skin prick images and the deep learning method.

[0223] The wheal identification module is further configured to perform the following processing:

[0224] First, the skin prick image captured by the positioning point recognition module is input into the pre-trained wheal model to obtain the potential wheal or bulge area;

[0225] The potential wheal or raised areas identified in the previous step are then matched to the pre-set puncture sites.

[0226] The wind cluster model is the Mask-RCNN wind cluster model, which is trained in the following way:

[0227] Step 1: Open the sample photo and determine the locations of all wind masses and bulges in the sample photo;

[0228] Step 2: Use the image annotation tool to mark a circle along the outer contours of all wheals and record the category as wheal;

[0229] Step 3: Mark a circle along the outer contours of all the ridges and record this category as ridges;

[0230] Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model. Then, select the sample files containing non-circular irregular shaped wind groups from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model. The second model is used to measure the irregular shaped wind groups that may appear in the samples. The deep learning model is the Mask-RCNN model, and the annotation result file type is json.

[0231] In the wheal identification module, the identified potential wheal or raised area is matched with the preset puncture site in the following way:

[0232] First, the coordinates of the center point of each potential wheal or bulge area are obtained, and the puncture site closest to the center point of each potential wheal or bulge area is found, and the distance between the two is required to be less than a minimum distance threshold;

[0233] Then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

[0234] The wind mass measurement value calculation module is used to measure and calculate the wind mass to be measured that is identified and screened by the wind mass identification module, and obtain the area, maximum diameter and vertical diameter of the wind mass to be measured.

[0235] In the wheal measurement value calculation module, the measured lengths of the maximum diameter and vertical diameter of each wheal are calculated based on the true value conversion ratio of the length, and the measured area of ​​the inner region of the wheal contour is calculated based on the true value conversion ratio of the area.

[0236] In the wind mass measurement value calculation module, the maximum diameter and true length of the vertical diameter of each wind mass and the true area are also calculated based on the correction ratio.

[0237] The wheal secondary screening module is used to perform secondary screening and supplementation on the wheals based on the confidence obtained by the wheal identification module and the area obtained by the wheal measurement value calculation module. The secondary screening is based on the relationship between any two or three of the wheal confidence, wheal area or puncture site attributes, wherein the puncture site attributes include near-light side sites, negative sites, positive sites and / or specific allergen sites.

[0238] The wheal analysis module is used to perform wheal analysis. The wheal analysis module further includes calculating corresponding analysis indicators for the wheal according to the type of pricking solution, wherein the analysis indicators are selected from the ratio of the wheal area to the positive control area, the ratio of the average diameter of the wheal to the average diameter of the positive control, and the maximum diameter.

[0239] In addition, the present invention also discloses a computer device for wind group identification, including a memory, a processor and program instructions stored in the memory for the processor to run, wherein the processor executes the program instructions to implement the steps in the aforementioned wind group identification method embodiment.

[0240] In addition, the present invention also discloses a computer-readable storage medium for wind group identification, which stores program instructions executable by a processor to implement the steps in the aforementioned wind group identification method embodiment.

[0241] In addition, the present invention also discloses a prick reading system, which includes a prick reader. The structure of the prick reader is as described in the above-mentioned embodiment and will not be repeated here. The image processing mechanism therein is configured to run the wind group identification method of the above-mentioned embodiment to analyze the wind group and output the size and / or detection result of the wind group, which will not be repeated here.

[0242] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0243] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0244] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0245] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying a wheal, characterized in that: The following steps are involved: Step 1: Identify the positioning points: pre-process the captured image, identify the positioning points in the pre-processed image, and capture the skin prick image based on the positioning points; Step 2: Identify wheals: Based on the captured skin prick images, identify and screen the wheals to be measured and their corresponding confidence levels based on the deep learning method; Step 3: Calculation of wind group measurement values: Measure and calculate the wind groups identified and screened in step 2 to obtain the area, maximum diameter and vertical diameter of the wind group to be measured; Step 4: Secondary screening of wind groups: Based on the confidence obtained in step 2 and the area obtained in step 3, secondary screening and supplementation of wind groups are performed; Step 5: Wheal analysis; Wherein, step 2 further includes the following processing steps: First, the skin prick image captured by the positioning point recognition in step 1 is input into the pre-trained wheal model to obtain the potential wheal or raised area; Then, the potential wheal or raised area identified in the previous step is matched with the preset puncture site; Among them, the wind group model is the Mask-RCNN wind group model, and the model is trained in the following way: Step 1: Open the sample photo and determine the locations of all wind masses and bulges in the sample photo; Step 2: Use the image annotation tool to mark a circle along the outer contours of all wheals and record the category as wheal; Step 3: Mark a circle along the outer contours of all the ridges and record this category as ridges; Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model. Then select sample files containing non-circular irregular wheals from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model. The second model is used to measure the irregular wheals that may appear in the samples. The deep learning model is the Mask-RCNN model, and the annotation result file type is json. Among them, in step 3, the measured lengths of the maximum diameter and vertical diameter of each wind group are calculated based on the true value conversion ratio of the length, and the measured area of ​​the internal area of ​​the wind group contour is calculated based on the true value conversion ratio of the area; the true lengths of the maximum diameter and vertical diameter of each wind group and the true area are also calculated based on the correction ratio.

2. The wheal identification method according to claim 1, characterized in that: The preprocessing steps in step 1 include: converting the captured RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

3. The wheal identification method according to claim 1, characterized in that: The identification of the positioning points in step 1 adopts the spot detection method, and after the positioning points are identified, the positioning point identification results are also checked. The inspection process includes: checking whether the number of detection results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal and vertical coordinates of the center point of each positioning point meets the preset value.

4. The wheal identification method according to claim 1, characterized in that: The identified potential wheal or raised areas are matched to the preset puncture sites as follows: First, the coordinates of the center point of each potential wheal or bulge area are obtained, and the puncture site closest to the center point of each potential wheal or bulge area is found, and the distance between the two is required to be less than a minimum distance threshold; Then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

5. The wheal identification method according to claim 1, characterized in that: The secondary screening is based on the relationship between any two or three of wheal confidence, wheal area or puncture site attributes, where the puncture site attributes include near-light side site, negative site, positive site and / or specific allergen site.

6. The wheal identification method according to claim 1, characterized in that: In step 4, the wheals identified and screened in step 2 are screened and supplemented again according to the following rules. "Retained results" means retaining the binary mask, area, maximum diameter and vertical diameter corresponding to the wheal for wheal analysis, where: (1) For wheals whose first model confidence is greater than the first confidence preset value and whose area is greater than the first area preset value, the first model recognition result is retained; (2) For wheals whose first model confidence is greater than the second confidence preset value and whose area is greater than the second area preset value, the first model recognition result is retained; (3) For wheals whose first model confidence is greater than the third confidence preset value and whose area is greater than the third area preset value, the first model recognition result is retained; (4) For the wheal corresponding to the puncture site close to the light source, if the first confidence preset value > the first model confidence > the fourth confidence preset value and the area > the third area preset value, the first model recognition result is retained; (5) For the positive control wheal, regardless of the confidence of the first model wheal, the first model result is retained; if there is no wheal corresponding to the positive control prick solution in the first model, return to step 2, select the first model and select the output category as "protuberance", repeat steps 2 and 3, find the protuberance corresponding to the positive control prick solution, regard the protuberance as a wheal and retain the result; (6) For negative control wheals whose first model confidence is greater than the fifth confidence preset value, the first model recognition results are retained, and all allergen wheals whose first model confidence is greater than the fifth confidence preset value are retained; (7) For a house dust mite allergen wheal of which the first model confidence is greater than the sixth confidence preset value and the area is greater than the fourth area preset value, if the house dust mite allergen wheal confidence is greater than the first confidence preset value, the house dust mite allergen wheal identification result of the first model is retained; (8) For allergen wheals whose first model confidence is greater than the first model confidence of the positive control wheals, their first model identification results are retained; (9) For a wheal with a second model confidence level greater than the seventh confidence level preset value and an area greater than the fifth area preset value, or a wheal with a second model confidence level greater than the eighth confidence level preset value and an area greater than the sixth area preset value at the same point in the first model, the point will be retained based on the recognition result of the second model; (10) If there is no potential allergen wheal, that is, the confidence is less than the minimum confidence threshold, then return to step 2, select the first model and select the output category as "protrusion", and repeat steps 2 and 3 to calculate all protrusion results; if there is only one protrusion that reaches the minimum confidence except the positive control, and the protrusion confidence is greater than the ninth confidence preset value, then the protrusion at this point will be regarded as a wheal and the result will be retained; (11) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

7. The wheal identification method according to claim 1, characterized in that: Step 5 further includes calculating corresponding analysis indicators for the wheal according to the type of pricking solution, wherein the analysis indicators are selected from the ratio of the wheal area to the positive control area, the ratio of the average diameter of the wheal to the average diameter of the positive control, and the maximum diameter.

8. A wind mass identification system, characterized in that: The system includes: A positioning point recognition module is used to pre-process the captured image, identify the positioning points in the pre-processed image, and capture the skin prick image based on the positioning points; A wheal recognition module is used to identify and select the wheals to be measured and their corresponding confidence levels based on the captured skin prick images based on a deep learning method; The wind mass measurement value calculation module is used to measure and calculate the wind masses identified and screened by the wind mass identification module to obtain the area, maximum diameter and vertical diameter of the wind mass to be measured; A wheal secondary screening module is used to perform secondary screening and supplement of wheals based on the confidence obtained by the wheal identification module and the area obtained by the wheal measurement value calculation module; Wheal analysis module, used for wheal analysis; The wheal identification module is further configured to perform the following processing: First, the skin prick image captured by the positioning point recognition module is input into the pre-trained wheal model to obtain the potential wheal or bulge area; Then, the potential wheal or raised area identified in the previous step is matched with the preset puncture site; Among them, in the wind group recognition module, the wind group model is the Mask-RCNN wind group model, and the model is trained in the following way: Step 1: Open the sample photo and determine the locations of all wind masses and bulges in the sample photo; Step 2: Use the image annotation tool to mark a circle along the outer contours of all wheals and record the category as wheal; Step 3: Mark a circle along the outer contours of all the ridges and record this category as ridges; Step 4: Export all the annotation results in the file format required for model training, extract a certain number of sample files from all sample sets without bias, and use the deep learning model of multi-classification instance segmentation to train to obtain the first model. Then select sample files containing non-circular irregular wheals from the total sample set, and use the deep learning model of multi-classification instance segmentation to train to obtain the second model. The second model is used to measure the irregular wheals that may appear in the samples. The deep learning model is the Mask-RCNN model, and the annotation result file type is json. Among them, in the wind group measurement value calculation module, the measured length of the maximum diameter and vertical diameter of each wind group is calculated based on the true value conversion ratio of the length, and the measured area of ​​the internal area of ​​the wind group contour is calculated based on the true value conversion ratio of the area; the true length of the maximum diameter and vertical diameter of each wind group and the true area are also calculated based on the correction ratio.

9. The wheal identification system according to claim 8, characterized in that: The preprocessing step of the positioning point recognition module is further configured as follows: converting the captured RGB image into a grayscale image, compressing the grayscale image, and then smoothing the compressed image.

10. The wheal identification system according to claim 8, characterized in that: The positioning point recognition module adopts the spot detection method to recognize the positioning points, and after the positioning point recognition, it also includes checking the positioning point recognition results. The inspection process includes: checking whether the number of detection results is equal to the set value, checking whether the center point of the positioning point falls within the preset area, and checking whether the difference between the horizontal and vertical coordinates of the center point of each positioning point meets the preset value.

11. The wheal identification system according to claim 8, characterized in that: In the wheal identification module, the identified potential wheal or raised area is matched with the preset puncture site in the following way: First, the coordinates of the center point of each potential wheal or bulge area are obtained, and the puncture site closest to the center point of each potential wheal or bulge area is found, and the distance between the two is required to be less than a minimum distance threshold; Then, for each puncture site, the potential wheal or raised area with the highest confidence is selected to correspond to it.

12. The wheal identification system according to claim 8, characterized in that: The secondary screening is based on the relationship between any two or three of wheal confidence, wheal area or puncture site attributes, where the puncture site attributes include near-light side site, negative site, positive site and / or specific allergen site.

13. The wheal identification system according to claim 8, characterized in that: In the wheal secondary screening module, the wheals identified and screened by the wheal identification module are screened and supplemented according to the following rules. "Retained results" means retaining the binary mask, area, maximum diameter and vertical diameter corresponding to the wheal for wheal analysis, where: (1) For wheals whose first model confidence is greater than the first confidence preset value and whose area is greater than the first area preset value, the first model recognition result is retained; (2) For wheals whose first model confidence is greater than the second confidence preset value and whose area is greater than the second area preset value, the first model recognition result is retained; (3) For wheals whose first model confidence is greater than the third confidence preset value and whose area is greater than the third area preset value, the first model recognition result is retained; (4) For the wheal corresponding to the puncture site close to the light source, if the first confidence preset value > the first model confidence > the fourth confidence preset value and the area > the third area preset value, the first model recognition result is retained; (5) For the positive control wheal, regardless of the confidence of the first model wheal, the first model result is retained; if there is no wheal corresponding to the positive control prick solution in the first model, return to the wheal identification module, select the first model and select the output category as "protrusion", repeat the wheal identification module and the wheal measurement value calculation module, find the protrusion corresponding to the positive control prick solution, regard the protrusion as a wheal and retain the result; (6) For negative control wheals whose first model confidence is greater than the fifth confidence preset value, the first model recognition results are retained, and all allergen wheals whose first model confidence is greater than the fifth confidence preset value are retained; (7) For a house dust mite allergen wheal of which the first model confidence is greater than the sixth confidence preset value and the area is greater than the fourth area preset value, if the house dust mite allergen wheal confidence is greater than the first confidence preset value, the house dust mite allergen wheal identification result of the first model is retained; (8) For allergen wheals whose first model confidence is greater than the first model confidence of the positive control wheals, their first model identification results are retained; (9) For a wheal with a second model confidence level greater than the seventh confidence level preset value and an area greater than the fifth area preset value, or a wheal with a second model confidence level greater than the eighth confidence level preset value and an area greater than the sixth area preset value at the same point in the first model, the point will be retained based on the recognition result of the second model; (10) If there is no potential allergen wheal, that is, the confidence is less than the minimum confidence threshold, then return to the wheal identification module, select the first model and select the output category as "protrusion", repeat the wheal identification module and the wheal measurement value calculation module to calculate all protrusion results; if there is only one protrusion that reaches the minimum confidence except the positive control, and the protrusion confidence is greater than the ninth confidence preset value, then the protrusion at this point will be regarded as a wheal and the result will be retained; (11) For potential wheal areas that do not meet the above retention rules, it is considered that there is no wheal here and it is screened out, and the maximum diameter, vertical diameter, and area results are all recorded as 0.

14. The wheal identification system according to claim 8, characterized in that: The wheal analysis module further includes calculating corresponding analysis indicators for the wheal according to the type of pricking solution, wherein the analysis indicators are selected from the ratio of the wheal area to the positive control area, the ratio of the average diameter of the wheal to the average diameter of the positive control, and the maximum diameter.

15. A computer device for wind group identification, characterized in that: The method comprises a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the wheal identification method according to any one of claims 1 to 7.

16. A computer-readable storage medium for wheal identification, storing program instructions executable by a processor to implement the steps of the wheal identification method according to any one of claims 1 to 7.

17. A prick reading system, characterized in that: The device comprises a prick reader, which comprises a chassis, an arm lift pad, an image acquisition mechanism and an image processing mechanism, wherein the image acquisition mechanism comprises a camera and a light source for acquiring the skin prick area, and the image processing mechanism is configured to run the wheal identification method as described in any one of claims 1 to 7, so as to analyze the wheal and output the size and / or detection result of the wheal.

18. The prick reading system according to claim 17, characterized in that: The image acquisition mechanism is arranged above the arm lifting pad, wherein the light source is located above the front of the palm and the irradiation surface is parallel and forward; the image processing mechanism is an industrial control computer, and a touch screen is arranged on the industrial control computer, which is configured to display the real-time picture and / or the image of the wind group and its associated data captured by the image acquisition mechanism to the user.

19. The prick reading system according to claim 17, characterized in that: The case is lightproof and has an opening at the front end for placing the arm into the case.

20. The prick reading system according to claim 17, characterized in that: The opening at the front end of the chassis is equipped with a light-blocking device that is suitable for arms of different thicknesses.

21. The prick reading system according to claim 20, characterized in that: The light blocking device is made of a light blocking cloth which is surrounded by soft cloth and has an opening in the middle, and the size of the opening is limited by an elastic rubber band.

22. The prick reading system according to claim 20, characterized in that: The light blocking device consists of a plurality of light blocking sheets which are interconnected. The light blocking sheets are installed at the lower part of the front door of the main unit through a slide rail. The handle passes through the slide groove in the middle part of the front door of the main unit and is connected to the light blocking sheets. The opening and closing state of the light blocking sheets is controlled by the handle. The spring sheet buckle arranged on the front door of the main unit is used to clamp the light blocking sheets when the handle is pulled to the highest point. The handle is pulled downward to disengage the light blocking sheets from the spring sheet buckle to close the entrance.

23. The prick reading system according to claim 17, characterized in that: The hasp-type door lock is installed on the rear side of the host front door and is used to self-lock when the host front door is pushed closed.

24. The prick reading system according to claim 17, characterized in that: The arm lift pad is arranged in the front cavity of the chassis. The arm lift pad has an upward slope to fix the arm. The end of the arm lift pad changes from a slope to a flat slope, which is used to place the wrist horizontally to increase comfort.

25. The prick reading system according to claim 17, characterized in that: The arm lifting pad is connected to the chassis in a detachable manner.

26. The prick reading system according to claim 25, characterized in that: A bumper structure is arranged below the arm lift pad, which is used to fix the arm lift pad in the bumper structure on the bottom surface of the chassis, so as to realize quick installation and removal of the arm lift pad.

27. The prick reading system according to claim 17, characterized in that: A printer placement platform is also provided at the rear of the chassis to place additional configured printers.

Images