A hand, foot and mouth disease detection method and system based on machine vision

Abstract

The present invention relates to the field of image data processing, and more specifically, to a hand, foot and mouth disease detection method and system based on machine vision, the method comprising: collecting an oral cavity inner wall image according to an endoscope, setting a label for image segmentation processing, and obtaining a binary image of the oral cavity inner wall image; projecting the pixel points in the binary image to obtain a herpes area distribution map and distribution density, dividing the size of the image block according to the distribution density, and determining the window size of each image block according to the brightness, size and distribution density; extracting the binary number of the number of pixel points according to the local binary mode of each image block corresponding to the window and obtaining the number of changes to obtain the possible degree of the lesion; in response to the possible degree of the lesion being greater than an abnormal threshold, the herpes degree of the oral cavity inner wall is serious. The present invention improves the recognition ability of the degree of lesions in the oral cavity in the image by finely capturing the detailed features of the lesion herpes area.

Description

Technical Field

[0001] The present invention relates to the field of image data processing, and more specifically, to a hand, foot and mouth disease detection method and system based on machine vision. Background Art

[0002] Hand, foot and mouth disease is a common infectious disease that mainly affects infants and young children. After the virus invades the body through the digestive tract or respiratory tract, it mainly binds to the corresponding viral receptors on the surface of the pharyngeal and intestinal epithelial cells. It is usually possible to determine whether a child has hand, foot and mouth disease by observing the oral cavity and seeing whether herpes grows on the inner wall of the mouth. However, parents usually do not have medical knowledge and cannot accurately determine whether a child has oral herpes.

[0003] The existing Chinese patent application document with publication number CN111598865A discloses a hand, foot and mouth disease detection method, device and system based on thermal infrared and RGB dual cameras. The method includes: obtaining temperature through a thermal infrared camera; obtaining face and hand images through an RGB camera; spatially correlating temperature and face to obtain correlation results; using an improved YOLO-V3 lesion detection network to detect face and hand images; combining temperature, correlation results, face detection results and hand detection results to make a joint judgment to obtain a judgment result; triggering an alarm based on the judgment result.

[0004] In the above technical scheme, detection is performed only by acquiring images of the face and hands. The cause of hand, foot and mouth disease is mainly reflected in the binding of corresponding viral receptors on the surface of pharyngeal and intestinal epithelial cells. It is usually necessary to take an image of the child's mouth and upload it to a related platform for a doctor to determine whether the child has oral herpes, and then determine whether the child has hand, foot and mouth disease. However, the surface of the mouth is relatively smooth, and a certain amount of reflection will be generated when the image is collected, which affects the doctor's judgment on whether the child has oral herpes, and thus affects the judgment on whether the child has hand, foot and mouth disease. Summary of the invention

[0005] In order to solve the problem that the relatively smooth surface of the oral cavity will produce certain reflections when collecting images, which affects the doctor's judgment on whether the child's oral cavity has herpes, the present invention provides solutions in the following aspects.

[0006] In a first aspect, a hand, foot and mouth disease detection method based on machine vision comprises: collecting an image of the inner wall of the oral cavity through an endoscope, setting a label for the image of the inner wall of the oral cavity, and performing image segmentation processing on the image of the inner wall of the oral cavity to obtain a binary image of the image of the inner wall of the oral cavity; projecting pixels in the binary image onto the horizontal axis and the vertical axis respectively, and counting the number of pixels in the herpes area on each coordinate to obtain a distribution map of the herpes area on the horizontal axis and the vertical axis respectively, obtaining a distribution density of the herpes area according to the ratio of the number of pixels in the herpes area to the total number of pixels in the image of the inner wall of the oral cavity, and dividing the size of the image block according to the distribution density; dividing the image block into the image blocks of the oral cavity; and dividing the image blocks into the image blocks of the oral cavity. The inner wall image is converted into the HSV color space, and the size of the window corresponding to each image block is determined according to the brightness, size and distribution density of the herpes area of ​​the image block; the binary number of the number of pixels in each image block is extracted based on the local binary pattern window size of the window corresponding to each image block as the radius, and the possible degree of lesions in each image block is obtained according to the number of changes of the binary number; the possible degree of lesions of the entire oral inner wall image is obtained according to the possible degree of lesions of each image block; in response to the possible degree of lesions of the oral inner wall image being greater than the abnormal threshold, the herpes degree of the oral inner wall is serious, which can assist in determining whether the patient suffers from hand, foot and mouth disease.

[0007] The effect is as follows: the image of the inner wall of the oral cavity is collected through an endoscope, and then the image of the inner wall of the oral cavity is labeled and segmented to generate a binary image, and the number of pixels in the herpes area is counted by analyzing the projections of the pixels in the binary image on the horizontal and vertical axes, and then a distribution map of the herpes area is drawn, and the size of the image block is intelligently divided according to the ratio of the number of pixels in the herpes area to the total number of pixels, and the size of the window corresponding to each image block is determined by using the brightness, size and distribution density of the herpes area of ​​the image block, thereby obtaining the possible degree of lesions on the inner wall of the oral cavity, and significantly improving the diagnostic efficiency and accuracy of hand, foot and mouth disease.

[0008] Preferably, obtaining the herpes area distribution map on the horizontal axis and the vertical axis comprises the steps of:

[0009] The line from the lower left corner to the lower right corner of the binary image is set as the horizontal axis, and the line from the lower left corner to the upper left corner is set as the vertical axis, and the pixel points in the binary image are projected onto the horizontal axis and the vertical axis respectively;

[0010] The number of pixel points projected onto each herpes area on the horizontal axis and the vertical axis is counted, and the number of pixel points of each herpes area in the horizontal axis and the vertical axis are connected in coordinate order to obtain a horizontal axis herpes area distribution map and a vertical axis herpes area distribution map.

[0011] The effect is: through the herpes area distribution map on the horizontal and vertical axes, the accuracy and efficiency of hand, foot and mouth disease detection are improved, and at the same time, an intuitive herpes area distribution map is provided for doctors, which is convenient for rapid identification and understanding of the severity and scope of the lesions.

[0012] Preferably, the size of the image block satisfies the following polynomial:

[0013] ,

[0014] ,

[0015] In the formula, Indicates the horizontal size of the blocks of the oral cavity wall image collected by the endoscope. Indicates the longitudinal size of the blocks of the oral cavity wall image collected by the endoscope. Represents the number of pixels in the herpes area in the oral cavity wall image, Represents the total number of pixels in the image. Represents the mean of the peak widths of all peaks in the herpes area distribution graph on the horizontal axis, Represents the mean of the peak widths of all peaks in the longitudinal herpes area distribution graph, is the ceiling function.

[0016] Preferably, the size of the window satisfies the following polynomial:

[0017] ,

[0018] ,

[0019] In the formula, Indicates the first The window size corresponding to the image block is Indicates the adjustment benchmark of the image block corresponding to the window size, Indicates the first The number of pixels in the herpes area in an image block, Indicates the first The total number of pixels in an image block, For custom parameters, Indicates the first The average value of the pixel brightness of the herpes area of ​​the image block, Indicates the first The average brightness of pixels in the image block except the herpes area, represents the maximum value function, represents the normalization function, Indicates the first The number of sampling points corresponding to each image block is is the ceiling function.

[0020] The effect is that by calculating the window size of each image block, the window size can be automatically adjusted according to the number of pixels and brightness differences in the herpes area in the image block, adapting to the characteristics of different image blocks, and more accurately locating the herpes area, thereby improving the recognition accuracy of the lesion area.

[0021] Preferably, the possible degree of the lesion satisfies the following relationship:

[0022] ,

[0023] In the formula, Indicates The possible degree of lesion in each image block, Indicates the first The window size corresponding to the image block is Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 0 to 1, Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 1 to 0, Represents the normalization function.

[0024] The effect is that by calculating the number of jumps of the local binary image pattern features, the subtle changes in the herpes area within the image block can be accurately captured, thereby evaluating the possible degree of lesions in each image block.

[0025] Preferably, the possible degree of lesions of the entire oral cavity inner wall image satisfies the following relationship:

[0026] ,

[0027] in, Indicates the possible degree of lesions in the entire oral cavity wall image. Represents the number of image blocks in the oral cavity wall image, Indicates The possible degree of lesion in each image block.

[0028] Preferably, obtaining the binary image of the oral cavity inner wall image comprises the steps of:

[0029] Collecting an image of the inner wall of the oral cavity, setting labels for the image, wherein the labels are: herpes area and other areas, and the pixels in the herpes area are marked as 1, and the pixels in other areas are marked as 0, to construct a sample data set;

[0030] The sample data set is input into a preset semantic segmentation network model, wherein the structure of the preset semantic segmentation network model adopts a UNet structure, uses a cross entropy loss function, back-propagates an error signal according to a loss value, updates network parameters, and stops updating when the network loss value is less than a set loss value or reaches a certain number of training times, thereby obtaining a trained prediction semantic segmentation network model;

[0031] The oral cavity inner wall image is collected in real time and input into the trained prediction semantic segmentation network model to obtain an output result, wherein the output result is: a binary image of the oral cavity inner wall image, that is, the herpes area.

[0032] The effect is: by combining image processing technology and artificial intelligence algorithms, the effectiveness of hand, foot and mouth disease detection is improved, making it easier for doctors to observe and make judgments.

[0033] In the second aspect, a hand, foot and mouth disease detection system based on machine vision includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned hand, foot and mouth disease detection method based on machine vision is implemented.

[0034] The present invention has the following effects:

[0035] 1. The present invention divides the herpes area by the color characteristics of the pixel points in different image blocks in the image of the inner wall of the oral cavity, and then analyzes whether the herpes area is a pathological herpes area and a reflective herpes area. According to the possibility of different herpes areas in different image blocks, the window size and the number of sampling points when generating the local binary pattern features in each image block are adjusted, so that the local binary pattern can capture the characteristics of the reflective herpes area more widely and the detailed characteristics of the pathological herpes area more finely, thereby improving the recognition ability of the degree of lesions in the oral cavity in the image.

[0036] 2. The present invention processes the binary numbers obtained from the local binary pattern, obtains the characteristic value according to the relationship between 0 and 1 in the binary number, and combines the characteristics of the diseased herpes area and the reflective herpes area to more accurately extract the possible degree of the lesion in the oral cavity in the image block, thereby improving the accuracy of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0038] Figure 1 It is a method flow chart of steps S1 to S5 in a hand, foot and mouth disease detection method based on machine vision in an embodiment of the present invention.

[0039] Figure 2 It is a method flow chart of steps S10 to S12 in a hand, foot and mouth disease detection method based on machine vision in an embodiment of the present invention.

[0040] Figure 3 It is a method flow chart of step S20-step S21 in a hand, foot and mouth disease detection method based on machine vision in an embodiment of the present invention.

[0041] Figure 4 It is a schematic diagram of dividing an image of the inner wall of the oral cavity into image blocks in a hand, foot and mouth disease detection method based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0044] Reference Figure 1 A hand, foot and mouth disease detection method based on machine vision includes steps S1 to S5, which are as follows:

[0045] S1: Collecting an oral cavity inner wall image through an endoscope, setting a label for the oral cavity inner wall image, and performing image segmentation processing on the oral cavity inner wall image to obtain a binary image of the oral cavity inner wall image.

[0046] To further explain, the image of the inner wall of the oral cavity is collected through an endoscope. Due to the reflection, the color characteristics of the herpes area and the diseased herpes area are very similar. The semantic segmentation network model is first used to extract the herpes area in the image of the inner wall of the oral cavity for further analysis.

[0047] Reference Figure 2 , including steps S10 to S12:

[0048] S10: Collect an image of the inner wall of the oral cavity, set a label for the image, wherein the label is: herpes area and other areas, and the pixel points in the herpes area are marked as 1, and the pixel points in other areas are marked as 0, and a sample data set is constructed;

[0049] S11: Input the sample data set into the preset semantic segmentation network model. The structure of the preset semantic segmentation network model adopts the UNet structure. The cross entropy loss function is used to back-propagate the error signal according to the loss value, and the network parameters of the preset semantic segmentation network model are updated until the network loss value is less than the set loss value or a certain number of training times is reached. Then, the updating is stopped to obtain a trained prediction semantic segmentation network model.

[0050] Further explanation: a semantic segmentation model is used. The semantic segmentation network model is a type of neural network model used to assign each pixel in an image to a specific category. The network structure of the semantic segmentation network model in this embodiment is a UNet structure. UNet is a neural network structure commonly used in image segmentation tasks. The network structure is a well-known technology and will not be described in detail.

[0051] S12: Collect the oral cavity inner wall image in real time, input it into the trained prediction semantic segmentation network model, and obtain the output result, wherein the output result is: a binary image of the oral cavity inner wall image, that is, the herpes area.

[0052] It should be noted that the above-mentioned herpes area may be a herpes area caused by reflection and may be a diseased herpes area. Direct semantic segmentation cannot effectively distinguish the two. Therefore, in order to better analyze, the diseased herpes area in the herpes area is further obtained.

[0053] Further explanation: in the herpes area of ​​the oral cavity inner wall image, the color of the herpes area caused by reflection is usually uniform and bright white, and the outline of this area is relatively smooth, while the color of the herpes area caused by the lesion may be uneven and present white of varying shades, and the outline of this area usually presents an irregular shape. In order to better distinguish the features, the oral cavity inner wall image is divided into image blocks, and the two areas are distinguished according to the local binary image.

[0054] S2: Project the pixels in the binary image onto the horizontal and vertical axes respectively, and count the number of pixels in the herpes area at each coordinate to obtain the herpes area distribution maps on the horizontal and vertical axes respectively. According to the ratio of the number of pixels in the herpes area to the total number of pixels in the oral cavity inner wall image, the distribution density of the herpes area is obtained, and the size of the image block is divided according to the distribution density.

[0055] Reference Figure 3 , including step S20-step S21:

[0056] S20: setting the line from the lower left corner to the lower right corner of the binary image as the horizontal axis, setting the line from the lower left corner to the upper left corner as the vertical axis, and projecting the pixel points in the binary image onto the horizontal axis and the vertical axis respectively;

[0057] To further illustrate, the binary image is projected, the distribution of pixels on the horizontal and vertical axes is visualized, and the number of pixels on the horizontal and vertical axes is plotted in the horizontal and vertical directions, respectively. It can be visualized using a bar chart or a line chart, which helps to analyze the characteristics and structure of the image.

[0058] S21: Counting the number of pixel points projected onto each herpes area on the horizontal axis and the vertical axis, and connecting the number of pixel points of each herpes area on the horizontal axis and the vertical axis in the order of coordinates to obtain a horizontal axis herpes area distribution map and a vertical axis herpes area distribution map;

[0059] To further explain, the number of pixels in the herpes area refers to the number of pixels with a value of 1 in the binary image.

[0060] Specifically, the size of the image block satisfies the following polynomial:

[0061] ,

[0062] ,

[0063] In the formula, Indicates the horizontal size of the blocks of the oral cavity wall image collected by the endoscope. Indicates the longitudinal size of the blocks of the oral cavity wall image collected by the endoscope. Represents the number of pixels in the herpes area in the oral cavity wall image, Represents the total number of pixels in the image. Represents the mean of the peak widths of all peaks in the herpes area distribution graph on the horizontal axis, Represents the mean of the peak widths of all peaks in the longitudinal herpes area distribution graph, is the ceiling function.

[0064] Further explanation: Indicates the proportion of the herpes area in the oral cavity inner wall image. The larger the value, the more or larger the herpes area in the image. In this case, the size of the image block should be enlarged so that the image block can accommodate the entire herpes area as much as possible. The smaller the value, the fewer or smaller the herpes area in the image. In this case, the size of the image block should be reduced so that the image block can accommodate the herpes area while reducing the part of other areas in the image block, so that the area in the image block contains only the herpes area as much as possible or only contains other areas as much as possible.

[0065] For example, the peak width is determined by the difference between the peak value and the value around the peak value. For example, in the herpes area distribution map, the value at the position with coordinate 88 is the peak value. , if the coordinates are from 79 to 96, the corresponding values ​​are , where the value corresponding to the coordinates 80 to 95 is different from the peak value ( ) are all less than 0.08 (empirical value). At the same time, the difference between the value with coordinate 79 and the peak value is greater than 0.08 (empirical value), and the difference between the value with coordinate 96 and the peak value is greater than 0.08 (empirical value). Therefore, the values ​​with coordinates 79 and 96 are used as the cutoff conditions, and the peak value The corresponding peak width is .

[0066] It should be noted that the reference Figure 4 , the oral cavity wall image is divided into several image blocks of the sizes calculated above. If the size of part of the image on the image boundary is less than the size of an image block, the image less than the size of an image block is downward compatible with the size of the remaining image, such as Figure 4 As shown in the image block corresponding to the thin line, since the herpes area in the oral cavity inner wall image is distributed in blocks, the more pixels corresponding to a certain coordinate position in the herpes area distribution map, the length corresponding to the coordinate position of the segment is the size of the herpes area corresponding to the coordinate position. Therefore, the length and width of the herpes area in the oral cavity inner wall image can be estimated according to the peak width of the herpes area distribution map of the horizontal axis and the herpes area distribution map of the vertical axis. Therefore, the image block size can be determined according to the length and width of the herpes area and the distribution density of the herpes area, so that the image block containing a large herpes area contains as many herpes areas as possible, and the image block containing a large area of ​​other areas contains as many other areas as possible.

[0067] S3: Convert the oral cavity inner wall image into HSV color space, and determine the size of the window corresponding to each image block according to the brightness and size of the image block and the distribution density of the herpes area.

[0068] It should be noted that the color of the herpes area caused by reflection is usually uniform and bright white, and the outline of the area is smooth, while the color of the herpes area caused by lesions may be uneven and present different shades of white. The color features and local binary patterns in each image block are used to distinguish whether the herpes area in the image block is caused by reflection or oral lesions, and then the possibility of lesions in the oral area corresponding to each image block is obtained.

[0069] The oral wall image is converted from RGB (Red, Green, Blue) color space to HSV (Hue, Saturation, Value) color space. The size of the local binary pattern window in the image block is determined according to the brightness, size and distribution of the herpes area of ​​the image block. In order to reasonably use windows of different sizes, different numbers of sampling points should be used for different window sizes.

[0070] Specifically, the size of the window satisfies the following polynomial:

[0071] ,

[0072] ,

[0073] In the formula, Indicates the first The window size corresponding to the image block is Indicates the adjustment benchmark of the image block corresponding to the window size, Indicates the first The number of pixels in the herpes area in an image block, Indicates the first The total number of pixels in an image block, For custom parameters, Indicates the first The average value of the pixel brightness of the herpes area of ​​the image block, Indicates the first The average brightness of pixels in the image block except the herpes area, represents the maximum value function, represents the normalization function, Indicates the first The number of sampling points corresponding to each image block is is the ceiling function.

[0074] Further explanation, in this embodiment , the implementation personnel can make adjustments according to the actual situation, For custom parameters, , the implementation personnel can adjust according to the actual situation, where , or These are all empirical values. The size of the window is determined by the number of pixels in the window, so there is a floating upward situation. Among the empirical values, the maximum is ;

[0075] The larger the value is, the more likely the herpes area in the image block is a reflective herpes area. Since the reflective area may cover a large area, selecting a large window helps capture a wider range of reflective areas and extract the overall features of the reflective area. The smaller the window is, the more likely the herpes area in the image block is a lesion herpes area. The herpes area caused by the lesion may be small in size and may have complex texture or irregular shape. Selecting a smaller window can more finely capture the detailed features of the lesion herpes area and improve the sensitivity and resolution of the lesion area.

[0076] Indicates The average value of the pixel brightness in the herpes area of ​​the image block With the image The average brightness of pixels outside the herpes area of ​​the image block The difference of, the larger the difference is, the more prominent the brightness of the herpes area in the image is, the more likely the herpes area in the image block is a reflective herpes area, and a large window should be selected to help capture a wider range of reflective areas, the smaller the difference is, the less prominent the brightness of the herpes area in the image is, the more likely the herpes area in the image block is a lesion herpes area, and a smaller window should be selected to more finely capture the detailed features of the lesion herpes area. It should be noted that in order to facilitate the extraction of local binary pattern features, the formula is divided by 2, multiplied by 2, and then added to 1 to limit the window size corresponding to the image block to an odd number, where divided by 2, multiplied by 2, and then added to 7, the same window size is limited to an odd number, and divided by 2 and then multiplied by 2 to limit the number of sampling points to an even number.

[0077] S4: Taking the local binary pattern window size of the window corresponding to each image block as the radius, extract the binary number of the number of pixels in each image block, and obtain the possible degree of the lesion of each image block according to the number of changes of the binary number.

[0078] To further illustrate, a local binary pattern is used in this embodiment. The local binary pattern (LBP) is a feature descriptor used in image processing and computer vision. The method for calculating the local binary pattern is a well-known technology and will not be described in detail in this embodiment. The binary number is a local binary model feature. Generally speaking, the herpes area caused by reflection often presents a relatively uniform white color, and its binary number may be relatively average. The number of binary numbers changing from 0 to 1 and from 1 to 0 is small. For example, in 11110000, the number of times from 0 to 1 is 0, and the number of times from 1 to 0 is 1; in 10110101, the number of times from 0 to 1 is 3, and the number of times from 1 to 0 is 3; and for the diseased herpes area, the whitening caused by the lesion often presents a more irregular texture, and its spacing characteristics may be relatively uneven.

[0079] Specifically, the degree of lesion may satisfy the following relationship:

[0080] ,

[0081] In the formula, Indicates The possible degree of lesion in each image block, Indicates the first The window size corresponding to the image block is Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 0 to 1, Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 1 to 0, Represents the normalization function.

[0082] Further explanation: The larger the value, the more likely it is to be average. The herpes area in the pixel of the image block is the herpes area caused by reflection. The smaller the value, the more likely it is to be uneven. The herpes area in the pixel points in the image block is the herpes area caused by the lesion;

[0083] Since the window size is calculated based on the characteristics of the reflective herpes area and the diseased herpes area, the larger the window, the more likely the corresponding area is to be the reflective herpes area, and the smaller the window, the more likely the corresponding area is to be the diseased herpes area, and the larger the possibility of the disease in the corresponding image block.

[0084] S5: The possible degree of lesions of the entire oral cavity inner wall image is obtained according to the possible degree of lesions of each image block. When the possible degree of lesions of the oral cavity inner wall image is greater than the abnormal threshold, the degree of herpes on the oral cavity inner wall is serious, thereby assisting in determining whether the patient suffers from hand, foot and mouth disease.

[0085] Specifically, the possible degree of lesions in the entire oral cavity inner wall image satisfies the following relationship:

[0086] ,

[0087] in, Indicates the possible degree of lesions in the entire oral cavity wall image. Represents the number of image blocks in the oral cavity wall image, Indicates The possible degree of lesion in each image block.

[0088] To further illustrate, in this embodiment, the abnormal threshold is 0.6, which is set by personnel in this field and can be adjusted according to actual conditions. When the possible degree of lesions in the oral wall image is greater than the abnormal threshold, the degree of herpes on the oral wall is serious, which can assist in determining whether the patient has hand, foot and mouth disease.

[0089] At this point, the method of the present invention ends.

[0090] The present invention also provides a hand, foot and mouth disease detection system based on machine vision. The system includes a processor and a memory, the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a hand, foot and mouth disease detection method based on machine vision according to the first aspect of the present invention is implemented.

[0091] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and will not be described in detail here.

[0092] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0093] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0094] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A hand, foot and mouth disease detection method based on machine vision, characterized in that: include: Collecting an oral cavity inner wall image through an endoscope, setting a label for the oral cavity inner wall image, and performing image segmentation processing on the oral cavity inner wall image to obtain a binary image of the oral cavity inner wall image; Projecting the pixels in the binary image onto the horizontal axis and the vertical axis respectively, and counting the number of pixels in the herpes area at each coordinate, obtaining the herpes area distribution map on the horizontal axis and the vertical axis respectively, obtaining the distribution density of the herpes area according to the ratio of the number of pixels in the herpes area to the total number of pixels in the oral cavity inner wall image, and dividing the size of the image block according to the distribution density; The oral cavity wall image is converted into HSV color space, and the size of the window corresponding to each image block is determined according to the brightness and size of the image block and the distribution density of the herpes area. ; , , where Indicates the adjustment benchmark of the image block corresponding to the window size, Indicates the first The number of pixels in the herpes area in an image block, Indicates the first The total number of pixels in an image block, For custom parameters, Indicates the first The average value of the pixel brightness of the herpes area of ​​the image block, Indicates the first The average brightness of pixels in the image block except the herpes area, represents the maximum value function, represents the normalization function, Indicates the first The number of sampling points corresponding to each image block is is the ceiling rounding function; According to the local binary pattern window size of the window corresponding to each image block as the radius, the binary number of the number of pixels in each image block is extracted, and the possible degree of the lesion of each image block is obtained according to the number of changes of the binary number; According to the possible degree of lesions in each image block, the possible degree of lesions of the entire inner wall of the oral cavity is obtained. When the possible degree of lesions in the inner wall of the oral cavity is greater than the abnormal threshold, the degree of herpes on the inner wall of the oral cavity is serious, which further assists in determining whether the patient suffers from hand, foot and mouth disease.

2. The hand, foot and mouth disease detection method based on machine vision according to claim 1, characterized in that: Obtaining the herpes regional distribution map on the horizontal axis and the vertical axis comprises the steps of: The line from the lower left corner to the lower right corner of the binary image is set as the horizontal axis, and the line from the lower left corner to the upper left corner is set as the vertical axis, and the pixel points in the binary image are projected onto the horizontal axis and the vertical axis respectively; The number of pixel points projected onto each herpes area on the horizontal axis and the vertical axis is counted, and the number of pixel points of each herpes area in the horizontal axis and the vertical axis are connected in coordinate order to obtain a horizontal axis herpes area distribution map and a vertical axis herpes area distribution map.

3. The hand, foot and mouth disease detection method based on machine vision according to claim 1 is characterized in that: The size of the image block satisfies the following polynomial: ， ， In the formula, Indicates the horizontal size of the blocks of the oral cavity wall image collected by the endoscope. Indicates the longitudinal size of the blocks of the oral cavity wall image collected by the endoscope. Represents the number of pixels in the herpes area in the oral cavity wall image, Represents the total number of pixels in the image. Represents the mean of the peak widths of all peaks in the herpes area distribution graph on the horizontal axis, Represents the mean of the peak widths of all peaks in the longitudinal herpes area distribution graph, is the ceiling function.

4. The hand, foot and mouth disease detection method based on machine vision according to claim 1, characterized in that: The possible degree of the lesion satisfies the following relationship: ， In the formula, Indicates The possible degree of lesion in each image block, Indicates the first The window size corresponding to the image block is Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 0 to 1, Indicates The average number of times the pattern feature of the local binary image of the pixel in the image block jumps from 1 to 0, Represents the normalization function.

5. The hand, foot and mouth disease detection method based on machine vision according to claim 1 is characterized in that: The possible degree of lesions in the entire oral cavity inner wall image satisfies the following relationship: ， in, Indicates the possible degree of lesions in the entire oral cavity wall image. Represents the number of image blocks in the oral cavity wall image, Indicates The possible degree of lesion in each image block.

6. The hand, foot and mouth disease detection method based on machine vision according to claim 1 is characterized in that: Acquiring a binary image of the oral cavity inner wall image comprises the steps of: Collect an image of the inner wall of the oral cavity, set a label for the image, wherein the label is: herpes area and other areas, and the pixels in the herpes area are marked as 1, and the pixels in other areas are marked as 0, to construct a sample data set; input the sample data set into a preset semantic segmentation network model, the structure of the preset semantic segmentation network model adopts a UNet structure, uses a cross entropy loss function, back-propagates an error signal according to the loss value, updates the network parameters, and stops updating when the network loss value is less than the set loss value or reaches a certain number of training times, thereby obtaining a trained predictive semantic segmentation network model; collect an image of the inner wall of the oral cavity in real time and input it into the trained predictive semantic segmentation network model to obtain an output result, wherein the output result is: a binary image of the inner wall of the oral cavity, i.e., the herpes area.

7. A hand, foot and mouth disease detection system based on machine vision, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the hand, foot and mouth disease detection method based on machine vision according to any one of claims 1-6 is implemented.

Images