Hand-foot-and-mouth disease detection device based on image acquisition

Abstract

The invention provides a hand-foot-and-mouth disease detection device based on image acquisition. The hand-foot-and-mouth disease detection device comprises an image acquisition module, an image processing module, a model training module and a result detection module. And the image acquisition module acquires a palm image, the palm image is preprocessed by the processing module and then sent to the improved PCL-Net and CAS-PUN models for training, and a detection result is output. The PCL-Net optimizes the network structure, and outputs 0 or 1 to judge whether the disease is sick or not; the CAS-PUN uses a U-Net structure to predict a target position, and the maximum probability value category is a prediction target. If the finger is predicted and PCL-Net detection is non-disease, determining that the finger is false positive, and prompting continuous detection. According to the invention, efficient and accurate hand-foot-and-mouth disease detection of the palm part image can be realized, and the accuracy and efficiency of diagnosis are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a hand, foot and mouth disease detection device based on image acquisition. Background Art

[0002] Existing HFMD detection mainly relies on doctors' clinical experience and laboratory tests. These methods are not only time-consuming and costly, but also have certain limitations for early diagnosis and large-scale screening. In the context of the rapid development of image recognition technology, some studies have attempted to use image processing technology to assist in the diagnosis of HFMD. However, due to the complexity and diversity of HFMD symptoms, these technologies are still insufficient in accuracy and efficiency. In addition, traditional image processing methods often require complex preprocessing and feature extraction steps, which are difficult for non-professionals to operate and difficult to adapt to image acquisition under different equipment and environmental conditions.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing detection devices lack the ability to automatically and highly accurately identify hand, foot and mouth disease images, and in actual applications, there is a risk of misjudgment in the identification of hand, foot and mouth disease symptoms in different parts of the body, especially in the image acquisition and processing of the palm part, there is a lack of effective automated detection means. Summary of the invention

[0004] The present invention provides a hand, foot and mouth disease detection device based on image acquisition, comprising: Image acquisition module, image processing module, model training module and result detection module; wherein the image acquisition module is used to collect image information of the patient's palm and input it into the image processing module; the image processing module is used to perform preprocessing operations on the collected images and then send them to the first model and the second model in the model training module for training, and obtain the detection result information and the result detection module detects whether it belongs to hand, foot and mouth virus, and specifically performs the following operations: S1, uses the improved PCL-Net and CAS-PUN as the first model and the second model respectively; S2, the improved PCL-Net is based on the original PCL-Net network structure and uses the residual structure to optimize the network; S3, the trained first model predicts whether the input hand, foot and mouth disease palm image information belongs to hand, foot and mouth disease, that is, the output of the first model is 0 or 1, if it is 0, it does not belong to hand, foot and mouth disease, and if it is 1, it belongs to hand, foot and mouth disease; S4, CAS-PUN uses a parallel-branched U-Net structure to use the trained second model to predict the probability values ​​of the input image belonging to five types of target positions, including the thumb, index finger, middle finger, ring finger, and pinky finger. The category with the largest probability value is defined as the target category predicted to belong to the palm; S5, when the second model predicts that the target category of the image belonging to the palm is one of the 5 target categories of the finger area, the palm area image belonging to the finger category is input into the improved PCL-Net. If the improved PCL-Net model detects that it does not belong to hand, foot and mouth disease, the image is judged to be a false positive, and a prompt is given to continue to detect the next image; otherwise, the image is judged to be a real hand, foot and mouth disease patient.

[0005] Furthermore, the image acquisition module: the module is composed of a collector, a processor and a USB. The collector is used to collect hand images and is connected using a USB interface. The image of the patient's palm is obtained by querying it using a smart mobile terminal or a computer terminal, and the collected image is sent to the image processing module for processing; The image processing module receives the images transmitted by the image acquisition module through the network protocol, and then sends them to the improved PCL-Net model and the CAS-PUN model for different processing.

[0006] Furthermore, the improved PCL-Net adds a residual structure based on the network structure of the original PCL-Net, and the improved PCL-Net structure includes a 5×5×64 convolution layer, four downsampling modules, four feature fusion modules, four 3×3×64 small convolution kernels and two 1×1×1 convolution base layers; The input of the model is a palm area image, and the output of the improved PCL-Net is a single value, which determines whether the input palm area image belongs to hand, foot and mouth disease. If it does, the result is 1, and if it does not, the result is 0.

[0007] Furthermore, the four downsampling modules output feature maps with resolutions of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original image through the downsampling layer. Specifically, the image input into the improved PCL-Net model first passes through a 5×5×64 convolution layer and ReLU activation, and then passes through a convolution layer containing 64 channels before starting the processing of the first downsampling module. The first downsampling module consists of a maximum pooling layer, a corresponding region feature addition structure of the maximum pooling layer, a 3×3×64 convolutional layer, and a ReLU activation function; The feature map processed by the first layer downsampling module is processed by a feature fusion module and then input into the second layer downsampling module; The structure of the second-layer downsampling module is the same as that of the first-layer downsampling module, and the convolution channels are increased from 64 to 128; The structure of the third-layer downsampling module is the same as that of the second-layer downsampling module, and the convolution channels are changed from 128 to 256; The structure of the fourth-layer downsampling module is the same as that of the third-layer downsampling module, and the convolution channel is changed from 256 to 512; The feature maps processed by the second, third, and fourth downsampling modules are input into the second convolution layer after adding the feature maps of the corresponding areas and processing by the feature fusion module. Finally, the results are output through a convolution layer of size 1×1×1. The improved PCL-Net model outputs a value of 0-1, where 0 indicates that the image does not belong to hand, foot and mouth disease, and 1 indicates that the image belongs to hand, foot and mouth disease.

[0008] Furthermore, the feature maps of two different levels in the feature fusion module, one in the first downsampling module and the second downsampling module, and the other in the third downsampling module and the fourth downsampling module, are added at the pixel level to the feature map of the fourth downsampling module and the feature map of the second downsampling module, and then a 3×3×64 channel convolution operation is performed to change the number of channels from 64 to 64.

[0009] Furthermore, the palm area image of the second model has 5 categories, wherein the categories of thumb, index finger, middle finger, ring finger and little finger are all 1, and the category of background category is 0; The second model outputs 5 categories and one background category. Each category has a probability value. The larger the probability value, the greater the possibility of belonging to the palm category. The category with the largest probability value greater than 0.5 is the background category. The category with the maximum probability value of the other 4 categories is the target area of ​​the palm in the palm image.

[0010] Furthermore, the second model is based on a pathological data collection and labeling system, which uniformly labels the clinically collected pathological data, labels the thumb area as 1, the index finger area as 2, the middle finger area as 3, the ring finger area as 4, the little finger area as 5, and the background category area as 6. The labeled data set is sent to the second model as training data for training. The loss function of the second model is a cross entropy loss function, which is expressed as:

[0011] in, It is The true value of each category, The model predicts The probability value of each category, is the total number of categories.

[0012] Furthermore, the result detection module includes a palm fingerprint feature collector, a data processing module and a network processor; The data source of this module is the detection results output by the improved PCL-Net and the second model. According to the classification results output by the improved PCL-Net, there are two cases: When the improved PCL-Net outputs a detection result of hand, foot and mouth disease, the input palm area image is input into the second model again to obtain the probability values ​​of 5 categories, among which the value with the largest probability value represents that the palm area of ​​the input image is a 5-category finger area. Since the finger is detected to be a diseased hand, foot and mouth disease, at this time, the network processor outputs the image as a false positive; When the improved PCL-Net output detects whether it belongs to hand, foot and mouth disease, it is directly output as the result.

[0013] Furthermore, the processing steps of the improved PCL-Net are specifically as follows: The collected pictures are sent to the improved PCL-Net, which processes the pictures to determine whether the pictures are of hand, foot and mouth disease. If not, the results are directly output and the pictures are returned to the image acquisition module to collect new pictures. If yes, the image output by the improved PCL-Net is passed to the second model. The second model outputs the categories of five types of palm target areas according to the input image, including the thumb, index finger, middle finger, ring finger and little finger. If it is one of the five types of palm target areas, it is further determined which target area of ​​the five types of palm parts the input palm image belongs to. The image belonging to the finger category is input into the improved PCL-Net model again. The improved PCL-Net model determines the probability that the image does not belong to hand and foot disease. If yes, it outputs that the image is a false positive and returns it to the image acquisition module to acquire a new image. If not, the image is output as belonging to hand, foot and mouth disease, the network processor outputs the result, and returns the image to the image acquisition module to acquire a new image.

[0014] Further, if the input image of S3 is the palm area, in the second model of S4, the second model will output the probability values ​​of the five types of target areas, among which the value with the largest probability belongs to which of the five types of target areas, among which the probability of the thumb is 1, the probability of the index finger is 2, the probability of the middle finger is 3, the probability of the ring finger is 4, and the probability of the little finger is 5. If the palm area is in category 6, it is considered that the input image belongs to the background category; If it belongs to the target area of ​​the first five categories, then S4 is further used to determine whether the target area belongs to the palm area. If the probability value of the five categories is the largest, the target area of ​​the palm of the input image is input into the improved PCL-Net in S3. The improved PCL-Net model determines that the probability value of the input disease feature is less than 0.5, and the output result is a false positive. If the probability value of the first five categories is not the largest, the output result is returned to the image acquisition module to collect a new picture.

[0015] The above-mentioned embodiments of the present invention have at least the following beneficial effects: the hand, foot and mouth disease detection device based on image acquisition provided by the present invention can improve the detection efficiency and accuracy of hand, foot and mouth disease. By adopting the improved PCL-Net and CAS-PUN models, the device can automatically pre-process and analyze the collected palm images, quickly identify the characteristics of hand, foot and mouth disease, reduce the dependence on professional doctors' diagnosis, and make the detection process more convenient and faster. In addition, the design of the device takes into account image acquisition under different equipment and environmental conditions, which can enhance the adaptability and practicality of the detection system.

[0016] The device can also effectively reduce the risk of misdiagnosis and missed diagnosis. By combining the advantages of the two models, on the one hand, the improved PCL-Net model is used to perform preliminary screening for hand, foot and mouth disease, and on the other hand, the CAS-PUN model is used to accurately locate the target area on the palm, which can ensure the reliability of the test results. This dual verification mechanism can reduce false positive or false negative results caused by single model judgment errors, and can provide strong technical support for the early diagnosis and treatment of hand, foot and mouth disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which: Figure 1 A schematic diagram of the structure of a hand, foot and mouth disease detection device based on image acquisition provided by one embodiment of the present invention; Figure 2 A schematic diagram of a process flow of a hand, foot and mouth disease detection device based on image acquisition provided by an embodiment of the present invention; Figure 3 The schematic diagram schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0019] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0020] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0021] Reference below Figure 1 and Figure 2 , Figure 1 A schematic diagram of the structure of a hand, foot and mouth disease detection device based on image acquisition provided by one embodiment of the present invention; Figure 2 The following is a flow chart of a hand, foot and mouth disease detection device based on image acquisition provided by an embodiment. Figure 1 and Figure 2 As shown, a hand, foot and mouth disease detection device 100 and 200 based on image acquisition includes: Image acquisition module 101, image processing module 102, model training module 103 and result detection module 104; The image acquisition module is used to collect the image information of the patient's palm and input it into the image processing module; The image processing module is used to perform preprocessing operations on the collected images and then send them to the first model and the second model in the model training module for training, obtain the detection result information, and the result detection module detects whether it belongs to hand, foot and mouth disease virus. Specifically, the following operations are performed: S1, uses the improved PCL-Net and CAS-PUN as the first model and the second model respectively; S2, the improved PCL-Net is based on the original PCL-Net network structure and uses the residual structure to optimize the network; S3, the trained first model predicts whether the input hand, foot and mouth disease palm image information belongs to hand, foot and mouth disease, that is, the output of the first model is 0 or 1, if it is 0, it does not belong to hand, foot and mouth disease, and if it is 1, it belongs to hand, foot and mouth disease; S4, CAS-PUN uses a parallel-branched U-Net structure to use the trained second model to predict the probability values ​​of the input image belonging to five types of target positions, including the thumb, index finger, middle finger, ring finger, and pinky finger. The category with the largest probability value is defined as the target category predicted to belong to the palm; S5, when the second model predicts that the target category of the image belonging to the palm is one of the 5 target categories of the finger area, the palm area image belonging to the finger category is input into the improved PCL-Net. If the improved PCL-Net model detects that it does not belong to hand, foot and mouth disease, the image is judged to be a false positive, and a prompt is given to continue to detect the next image; otherwise, the image is judged to be a real hand, foot and mouth disease patient.

[0022] It should be noted that this embodiment describes a hand, foot and mouth disease detection device based on image acquisition, which includes an image acquisition module, an image processing module, a model training module and a result detection module. The image acquisition module refers to a device for acquiring images of the patient's palm area, while the image processing module is responsible for preprocessing the acquired images so that they can be input into the model training module for training and analysis. The model training module includes two models, namely the improved PCL-Net and CAS-PUN, which are used for the detection of hand, foot and mouth disease and the identification of the palm area, respectively.

[0023] Specifically, the image acquisition module can be composed of a collector, a processor and a USB interface. The collector is used to obtain the hand image and transmit the image to the smart mobile terminal or computer terminal through the USB interface. The image processing module receives the image transmitted by the image acquisition module through the network protocol and performs necessary preprocessing operations on it, such as adjusting the image size, enhancing the contrast, etc., to adapt to the input requirements of the model training module. The improved PCL-Net model and CAS-PUN model respectively receive the processed images and perform corresponding training and prediction.

[0024] Preferably, the improved PCL-Net model is optimized based on the original PCL-Net network structure, and a residual structure is added to improve the performance of the network. The model includes a 5×5×64 convolution layer, four downsampling modules, four feature fusion modules, four 3×3×64 small convolution kernels, and two 1×1×1 convolution base layers.

[0025] Furthermore, in practical applications, the specific parameter settings of these modules can be adjusted according to the characteristics of hand, foot and mouth disease images to achieve the best detection effect. For example, the downsampling module can be implemented through a combination of a maximum pooling layer and a convolution layer, while the feature fusion module can enhance the expressiveness of features through the addition and convolution operations of feature maps.

[0026] In some embodiments, the image acquisition module is composed of a collector, a processor and a USB. The collector is used to collect hand images and is connected using a USB interface. The image of the patient's palm is obtained by querying it using a smart mobile terminal or a computer terminal, and the collected image is sent to the image processing module for processing; The image processing module receives the images transmitted by the image acquisition module through the network protocol, and then sends them to the improved PCL-Net model and the CAS-PUN model for different processing.

[0027] It should be noted that this embodiment describes in detail the specific composition and functions of the image acquisition module. The image acquisition module consists of a collector, a processor and a USB interface, wherein the collector refers to a hardware device for capturing hand images, the processor is an electronic component responsible for processing the collected image data, and the USB interface is a universal serial bus interface used to connect the collector to a smart mobile terminal or a computer terminal to achieve data transmission. This module is designed to directly obtain images of the patient's palm through a physical device and digitize them for further processing.

[0028] Specifically, the collector in the image acquisition module can be a high-resolution camera or scanner that can capture details of the palm for accurate disease detection. The processor can be an embedded system that has sufficient computing power to process image data and convert it into a format suitable for model training. The USB interface ensures the convenience and compatibility of data transmission, allowing the acquisition module to be easily connected to a variety of devices. In terms of parameter setting, the resolution of the collector can be adjusted as needed to ensure the clarity of the image; the computing power of the processor also needs to match the complexity of processing the image.

[0029] Preferably, the collector in the image acquisition module can select a camera with automatic focus and exposure adjustment functions to adapt to different shooting environments and lighting conditions. The processor can use a high-performance microprocessor or a graphics processing unit (GPU) to improve the efficiency of image processing.

[0030] Furthermore, the USB interface can be a USB 3.0 or higher version interface to support high-speed data transmission. In practical applications, wireless transmission technology, such as Wi-Fi or Bluetooth, can also be considered as an alternative to the USB interface to achieve wireless image transmission and further improve the convenience of use.

[0031] In some embodiments, the improved PCL-Net adds a residual structure based on the network structure of the original PCL-Net, and the improved PCL-Net structure includes a 5×5×64 convolution layer, four downsampling modules, four feature fusion modules, four 3×3×64 small convolution kernels and two 1×1×1 convolution base layers; The input of the model is a palm area image, and the output of the improved PCL-Net is a single value, which determines whether the input palm area image belongs to hand, foot and mouth disease. If it does, the result is 1, and if it does not, the result is 0.

[0032] It should be noted that this implementation further elaborates on the structure and function of the improved PCL-Net model. The improved PCL-Net model is optimized based on the original PCL-Net network structure, and it improves the performance of the network by adding a residual structure. The residual structure is a technology in deep learning, which is used to solve the problem of gradient disappearance or gradient explosion in deep network training, and helps information flow in the network through jump connections. The model includes a 5×5×64 convolution layer, four downsampling modules, four feature fusion modules, four 3×3×64 small convolution kernels and two 1×1×1 convolution base layers. These components work together to improve the recognition ability of hand, foot and mouth disease palm images.

[0033] Specifically, the input of the improved PCL-Net model is a picture of the palm area, and the output is a single value to determine whether the input palm area picture belongs to hand, foot and mouth disease. The 5×5×64 convolution layer in the model refers to a convolution layer with a convolution kernel size of 5×5, 3 input channels (corresponding to RGB three colors), and 64 output channels. The downsampling module extracts features by reducing the spatial resolution of the image while reducing the amount of calculation. The feature fusion module is used to combine feature information at different levels to enhance the feature expression ability of the model. The small convolution kernel of 3×3×64 and the convolution base layer of 1×1×1 are used for further feature extraction and the final classification decision, respectively.

[0034] Preferably, the downsampling module in the improved PCL-Net model can adopt a combination of a maximum pooling layer and a convolution layer to achieve dimensionality reduction and extraction of features. The feature fusion module can be implemented by adding and convolution operations of feature maps, so that features can be fused at different levels to improve the recognition ability of the model. In practical applications, the depth and width of the model can be adjusted according to the characteristics and complexity of hand, foot and mouth disease images, such as increasing or decreasing the number of convolution layers, or adjusting the size of the convolution kernel and the number of channels.

[0035] Furthermore, you can also consider using different activation functions, such as Leaky ReLU or ELU, to further improve the performance of the model. During model training, you can also use data augmentation techniques, such as rotation, scaling, or color transformation, to improve the generalization ability of the model.

[0036] In some embodiments, the four downsampling modules output feature maps with resolutions of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original image through the downsampling layer, specifically: the image first input into the improved PCL-Net model first passes through a 5×5×64 convolution layer and ReLU activation, and then passes through a convolution layer containing 64 channels before starting the processing of the first layer of downsampling modules; The first downsampling module consists of a maximum pooling layer, a corresponding region feature addition structure of the maximum pooling layer, a 3×3×64 convolutional layer, and a ReLU activation function; The feature map processed by the first layer downsampling module is processed by a feature fusion module and then input into the second layer downsampling module; The structure of the second-layer downsampling module is the same as that of the first-layer downsampling module, and the convolution channels are increased from 64 to 128; The structure of the third-layer downsampling module is the same as that of the second-layer downsampling module, and the convolution channels are changed from 128 to 256; The structure of the fourth-layer downsampling module is the same as that of the third-layer downsampling module, and the convolution channel is changed from 256 to 512; The feature maps processed by the second, third, and fourth downsampling modules are input into the second convolution layer after adding the feature maps of the corresponding areas and processing by the feature fusion module. Finally, the results are output through a convolution layer of size 1×1×1. The improved PCL-Net model outputs a value of 0-1, where 0 indicates that the image does not belong to hand, foot and mouth disease, and 1 indicates that the image belongs to hand, foot and mouth disease.

[0037] It should be noted that this embodiment describes in detail the specific structure and function of the downsampling module. The downsampling module is a key component in the improved PCL-Net model. They extract deep features by reducing the spatial resolution of the image while reducing the amount of calculation. These modules output feature maps with resolutions of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original image to achieve multi-scale feature extraction. In the model, the input image first passes through a 5×5×64 convolution layer and a ReLU activation function, and then enters the first layer of downsampling module processing.

[0038] Specifically, the downsampling module consists of a maximum pooling layer, a corresponding region feature addition structure of the maximum pooling layer, a 3×3×64 convolution layer, and a ReLU activation function. The maximum pooling layer is a commonly used downsampling technique that reduces the spatial dimension of the image by selecting the maximum value in the region. The corresponding region feature addition structure is a residual connection that adds the pooled feature map to the original feature map to enhance the model's feature transfer capability. The 3×3×64 convolution layer is used to further extract features, where 64 represents the number of output channels. The ReLU activation function is used to introduce nonlinearity and enhance the model's expressiveness. The specific parameter settings of these components, such as the convolution kernel size, pooling window size, and step size, can be adjusted according to the actual image size and feature extraction requirements.

[0039] Preferably, the design of the downsampling module can be optimized according to the specific application scenario and image characteristics. For example, the maximum pooling layer can use windows of different sizes, such as 2×2 or 3×3, to adapt to different downsampling ratios. The corresponding regional feature addition structure can use different fusion methods, such as element-by-element addition or weighted addition, to improve the effect of feature fusion.

[0040] Furthermore, we can also consider introducing a batch normalization layer to reduce internal covariate shift and accelerate the convergence of the model. In practical applications, we can also further optimize the performance and computational efficiency of the model by adjusting the depth and width of the convolution layer or introducing techniques such as depthwise separable convolution.

[0041] In some embodiments, the feature maps of two different levels in the feature fusion module are one in the first layer downsampling module and the second layer downsampling module, and the other in the third layer downsampling module and the fourth layer downsampling module. The feature map of the fourth layer downsampling module and the feature map of the second layer downsampling module are added at the pixel level and then subjected to a 3×3×64 channel convolution operation to change the number of channels from 64 to 64.

[0042] It should be noted that this implementation method elaborates on the working principle and implementation details of the feature fusion module. The feature fusion module is a key part of the improved PCL-Net model for integrating features at different levels. It enhances the model's ability to capture image features by combining feature maps of different depths. In the model, the feature fusion module involves feature maps of two different levels, one from the first-layer downsampling module and the other from the third-layer downsampling module. The feature maps of these two levels are added at the pixel level, and a 3×3×64 channel convolution operation is performed to change the number of channels from 64 to 64 to achieve feature fusion and information transmission.

[0043] Specifically, the pixel-level addition operation in the feature fusion module refers to aligning two feature maps of different depths in the spatial dimension and then performing pixel-by-pixel addition operations, which can retain more spatial information. The convolution operation of 3×3×64 channels refers to the use of 3×3 convolution kernels, and 64 such convolution kernels act on the feature map at the same time to extract and fuse features. This convolution operation helps to further extract local features in the fused feature map and enhance the model's discriminative ability. In terms of parameter setting, the size, step size, and padding of the convolution kernel can be adjusted according to the actual image size and feature extraction requirements to achieve the best fusion effect.

[0044] Preferably, the design of the feature fusion module can be optimized according to the specific application scenario and image characteristics. For example, a nonlinear activation function such as ReLU or Leaky ReLU can be introduced after feature fusion to enhance the nonlinear expression ability of the model. In addition, it is also possible to consider introducing an attention mechanism such as the Squeeze-and-Excitation (SE) module in the feature fusion module to adaptively emphasize useful features and suppress unimportant features.

[0045] Furthermore, in practical applications, the performance and computational efficiency of the model can be further optimized by adjusting the depth and width of the convolution layer or introducing techniques such as depthwise separable convolution. These optimization measures help improve the model's recognition accuracy of hand, foot and mouth disease features, especially when processing palm images with complex backgrounds and diverse symptoms.

[0046] In some embodiments, the palm region image of the second model has 5 categories, wherein the categories of thumb, index finger, middle finger, ring finger and little finger are all 1, and the category of background category is 0; The second model outputs 5 categories and one background category. Each category has a probability value. The larger the probability value, the greater the possibility of belonging to the palm category. The category with the largest probability value greater than 0.5 is the background category. The category with the maximum probability value of the other 4 categories is the target area of ​​the palm in the palm image.

[0047] It should be noted that this embodiment describes the category setting of the second model and the judgment logic of the output probability value. The palm area image of the second model is divided into 5 finger categories and one background category, where the categories of the thumb, index finger, middle finger, ring finger and little finger are all marked as 1, and the background category is marked as 0. The model outputs the probability values ​​of these 6 categories and one background category. The larger the probability value, the higher the possibility that the image belongs to the category. In particular, if the category with the largest probability value exceeding 0.5 is the background category, the image is judged to be the background; if one of the other 4 finger categories has the largest probability value, the image is judged to be the target area of ​​the palm in the palm image.

[0048] Specifically, the output probability value of the second model is calculated by the model's softmax layer, which is an activation function commonly used in multi-classification problems. It can convert the model's output into a probability distribution. In this model, each category has a corresponding probability value, and the sum of these probability values ​​is 1. The probability value is calculated based on the model's predicted output, that is, the model's score for each category.

[0049] More specifically, in practical applications, the prediction output of the model can be optimized by the cross entropy loss function to improve the accuracy of classification. The category settings can be adjusted according to the actual pathological data and labeling system to ensure that the model can accurately identify and distinguish different finger areas and backgrounds.

[0050] Preferably, the category setting and probability value judgment logic of the second model can be adjusted according to specific application requirements. For example, more categories can be introduced or existing categories can be subdivided to adapt to different detection scenarios. In the judgment of probability values, different thresholds can be set, not just 0.5, to adapt to different false positive and false negative tolerances.

[0051] Furthermore, we can also consider introducing additional features, such as texture features or color features, to improve the model's ability to recognize the palm area. During model training, data enhancement techniques, such as rotation, scaling, or color transformation, can be used to improve the generalization and robustness of the model. These optimization measures help improve the accuracy and reliability of the model in practical applications.

[0052] In some embodiments, the second model is based on a pathological data collection and marking system, and the clinically collected pathological data is uniformly marked, the thumb area is marked as 1, the index finger area is marked as 2, the middle finger area is marked as 3, the ring finger area is marked as 4, the little finger area is marked as 5, and the background category area is marked as 6. The marked data set is sent to the second model as training data for training, and the loss function of the second model is a cross entropy loss function, which is expressed as:

[0053] in, It is The true value of each category, The model predicts The probability value of each category, is the total number of categories.

[0054] It should be noted that this embodiment describes in detail the training process of the second model and the selection of the loss function. The second model is constructed based on a pathological data collection and labeling system, which uniformly labels the clinically collected pathological data, where the areas of the thumb, index finger, middle finger, ring finger and little finger are marked as 1 to 5 respectively, and the area of ​​the background category is marked as 6. These labeled data sets are used as training data and input into the second model for training. The loss function of the model is selected as the cross entropy loss function, which is a loss function commonly used in multi-classification problems to measure the difference between the model's predicted probability distribution and the true label.

[0055] Specifically, the training of the second model involves inputting the labeled pathological data into the model and optimizing the model parameters through the back propagation algorithm. The calculation formula of the cross entropy loss function is:

[0056] in It is The true value of each category (0 or 1), The model predicts The probability value of each category. During the training process, the model parameters are adjusted according to the gradient of the loss function to minimize the loss value, thereby improving the classification accuracy of the model. Parameter settings include learning rate, batch size, number of iterations, etc. These parameters can be adjusted according to the convergence speed and overfitting of the model.

[0057] Preferably, the training process of the second model can be further optimized. For example, data enhancement techniques such as random cropping, rotation or color transformation can be used to increase the diversity of training data and improve the generalization ability of the model.

[0058] Furthermore, you can consider using regularization techniques, such as L1 or L2 regularization, to prevent the model from overfitting. In terms of the choice of loss function, in addition to the cross entropy loss, you can also consider introducing focal loss to solve the problem of class imbalance. These optimization measures help improve the robustness and accuracy of the model in practical applications.

[0059] In some embodiments, the result detection module includes a palm fingerprint feature collector, a data processing module and a network processor; The data source of this module is the detection results output by the improved PCL-Net and the second model. According to the classification results output by the improved PCL-Net, there are two cases: When the improved PCL-Net outputs a detection result of hand, foot and mouth disease, the input palm area image is input into the second model again to obtain the probability values ​​of 5 categories, among which the value with the largest probability value represents that the palm area of ​​the input image is a 5-category finger area. Since the finger is detected to be a diseased hand, foot and mouth disease, at this time, the network processor outputs the image as a false positive; When the improved PCL-Net output detects whether it belongs to hand, foot and mouth disease, it is directly output as the result.

[0060] It should be noted that this embodiment describes the composition and function of the result detection module. The result detection module includes a palm fingerprint feature collector, a data processing module and a network processor. The palm fingerprint feature collector refers to a device used to collect feature information of the palm area, which may include biometric features such as the texture and shape of the palm. The data processing module is responsible for analyzing and processing the collected feature information, while the network processor makes the final decision output based on the processing results. The design of this module aims to combine the output results of the improved PCL-Net with the second model to achieve accurate detection of hand, foot and mouth disease.

[0061] Specifically, the data source of the result detection module is the detection results output by the improved PCL-Net and the second model. When the improved PCL-Net outputs the detection result of hand, foot and mouth disease, the data processing module will input the input palm area image into the second model again to obtain the probability values ​​of 5 categories.

[0062] More specifically, among these probability values, the value with the largest probability value represents which of the five types of finger areas the palm area of ​​the input image belongs to. If the finger is detected to be suffering from hand, foot and mouth disease, the network processor will output the image as a false positive. If the improved PCL-Net outputs the detection result as not belonging to hand, foot and mouth disease, the data processing module will directly output the result. In terms of parameter setting, the threshold and classification criteria of the model can be adjusted according to the actual detection needs to optimize the accuracy and efficiency of the detection.

[0063] Preferably, the operation steps of the result detection module can be further refined or alternative solutions can be provided. For example, an additional image quality assessment step can be introduced to ensure that the quality of the input image meets the detection requirements. In the data processing module, a more complex algorithm can be used to analyze the features of the palm area, such as using deep learning technology to extract deeper feature information.

[0064] Furthermore, the decision logic of the network processor can be further optimized, for example by introducing machine learning algorithms to improve the ability to identify false positives and false negatives. These optimization measures help improve the overall performance and reliability of the result detection module.

[0065] In some embodiments, the processing steps of the improved PCL-Net are specifically as follows: The collected pictures are sent to the improved PCL-Net, which processes the pictures to determine whether the pictures are of hand, foot and mouth disease. If not, the results are directly output and the pictures are returned to the image acquisition module to collect new pictures. If yes, the image output by the improved PCL-Net is passed to the second model. The second model outputs the categories of five types of palm target areas according to the input image, including the thumb, index finger, middle finger, ring finger and little finger. If it is one of the five types of palm target areas, it is further determined which target area of ​​the five types of palm parts the input palm image belongs to. The image belonging to the finger category is input into the improved PCL-Net model again. The improved PCL-Net model determines the probability that the image does not belong to hand and foot disease. If yes, it outputs that the image is a false positive and returns it to the image acquisition module to acquire a new image. If not, the image is output as belonging to hand, foot and mouth disease, the network processor outputs the result, and returns the image to the image acquisition module to acquire a new image.

[0066] It should be noted that this embodiment describes in detail the processing steps and logic of the improved PCL-Net model. The improved PCL-Net model is a key component for determining whether the collected picture suffers from hand, foot and mouth disease. The model first receives the picture transmitted from the image acquisition module, and then processes the picture to determine whether the picture shows the characteristics of hand, foot and mouth disease. If the model determines that the picture does not show the characteristics of hand, foot and mouth disease, it directly outputs the result and returns the picture to the image acquisition module to collect a new picture; if the model determines that the picture shows the characteristics of hand, foot and mouth disease, the picture is further passed to the second model for classification of the palm target area.

[0067] Specifically, the processing steps of the improved PCL-Net model include inputting the collected pictures into the model, and the model extracts and classifies the pictures through its trained network structure. If the output of the model shows that the picture does not belong to hand, foot and mouth disease, the model will output the result as 0 and return the picture to the image acquisition module. If the output of the model shows that the picture belongs to hand, foot and mouth disease, the model will output the result as 1 and pass the picture to the second model for further processing.

[0068] More specifically, in terms of parameter setting, the model's threshold, classification criteria, and network structure can be adjusted according to actual detection requirements and dataset characteristics to optimize detection accuracy and efficiency.

[0069] Preferably, the processing steps of the improved PCL-Net model can be further refined or alternatives can be provided. For example, image enhancement steps such as contrast enhancement and noise reduction can be introduced before and after model processing to improve the model's ability to recognize hand, foot and mouth disease features. In addition, a multi-scale analysis method can be used, that is, different scale versions of the same image are processed to improve the model's ability to recognize hand, foot and mouth disease features of different sizes and shapes.

[0070] Furthermore, in the output stage of the model, a confidence scoring mechanism can be introduced, that is, in addition to the simple 0 or 1 output, the model's confidence in its classification results can also be output, which can help subsequent processing modules better judge and process the results. These optimization measures help improve the overall performance of the model and the accuracy of hand, foot and mouth disease detection.

[0071] In some embodiments, if the input image of S3 is a palm area, in the second model of S4, the second model will output probability values ​​of 5 types of target areas, where the value with the highest probability belongs to which of the five types of target areas, wherein the probability of the thumb is 1, the probability of the index finger is 2, the probability of the middle finger is 3, the probability of the ring finger is 4, and the probability of the little finger is 5. If the palm area is in category 6, it is considered that the input image belongs to the background category; If it belongs to the target area of ​​the first five categories, then S4 is further used to determine whether the target area belongs to the palm area. If the probability value of the five categories is the largest, the target area of ​​the palm of the input image is input into the improved PCL-Net in S3. The improved PCL-Net model determines that the probability value of the input disease feature is less than 0.5, and the output result is a false positive. If the probability value of the first five categories is not the largest, the output result is returned to the image acquisition module to collect a new picture.

[0072] It should be noted that this embodiment describes the specific operation process of the second model when processing palm area images. In this process, the second model outputs probability values ​​for 5 types of target areas, including the thumb, index finger, middle finger, ring finger and little finger, and each finger category corresponds to a probability value. If the palm area is classified as a background category (category 6), it is considered that the input image does not fall within the detection range of hand, foot and mouth disease. This step is a key step in determining whether the image belongs to hand, foot and mouth disease, which involves the interpretation of the model output and subsequent processing decisions.

[0073] Specifically, after receiving the palm area image, the second model will calculate the probability value of each category based on the trained classifier. These probability values ​​reflect the model's confidence that the image belongs to each finger category or background category.

[0074] More specifically, in terms of parameter settings, the classification threshold of the model can be set to 0.5, which means that only when the probability value of a finger category is higher than 0.5, the category will be considered as the true category of the image. In addition, the output of the model can include a softmax layer, which converts the original output of the model (i.e., logits) into probability values, ensuring that the sum of the probability values ​​of all categories is 1.

[0075] Preferably, the operation steps of the second model can be further refined or alternative solutions can be provided. For example, additional post-processing steps can be introduced, such as normalizing the probability values ​​output by the model to ensure the stability and reliability of the probability values.

[0076] Furthermore, a multi-model fusion strategy can be adopted, that is, combining the outputs of multiple models with different architectures or training data to improve the accuracy of classification. When processing the background category, a higher threshold can be set to reduce the misclassification of the background as hand, foot and mouth disease. These optimization measures help improve the robustness and accuracy of the model in practical applications.

[0077] The above-mentioned embodiments of the present invention have the following beneficial effects: The hand, foot and mouth disease detection device based on image acquisition described in the present invention can provide an efficient automated detection method, and can realize rapid diagnosis of hand, foot and mouth disease by integrating image acquisition module, image processing module, model training module and result detection module. The device uses the improved PCL-Net and CAS-PUN models to accurately identify the characteristics of hand, foot and mouth disease from palm images, thereby reducing dependence on laboratory testing, reducing diagnostic costs, and improving diagnostic speed.

[0078] In addition, the design of the device takes into account a variety of situations in practical applications, including image acquisition under different equipment and environmental conditions, and accurate identification of different areas of the palm. With this design, the device can adapt to changing clinical environments and improve the flexibility and accuracy of detection. Especially when dealing with false positive results, by combining the outputs of the two models, it is possible to more accurately determine whether the image belongs to hand, foot and mouth disease, thereby reducing the risk of misdiagnosis and providing clinicians with a reliable auxiliary diagnostic tool.

[0079] Reference below Figure 3, which shows a schematic diagram of a structure 300 of an electronic device suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0080] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0081] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0082] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0083] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A hand, foot and mouth disease detection device based on image acquisition, characterized in that include: Image acquisition module, image processing module, model training module and result detection module; wherein the image acquisition module is used to collect image information of the patient's palm and input it into the image processing module; The image processing module is used to perform preprocessing operations on the collected images and then send them to the first model and the second model in the model training module for training, obtain the detection result information, and the result detection module detects whether it belongs to hand, foot and mouth disease virus. Specifically, the following operations are performed: S1, uses the improved PCL-Net and CAS-PUN as the first model and the second model respectively; S2, the improved PCL-Net is based on the original PCL-Net network structure and uses the residual structure to optimize the network; S3, the trained first model predicts whether the input hand, foot and mouth disease palm image information belongs to hand, foot and mouth disease, that is, the output of the first model is 0 or 1, if it is 0, it does not belong to hand, foot and mouth disease, and if it is 1, it belongs to hand, foot and mouth disease; S4, CAS-PUN uses a parallel-branched U-Net structure to use the trained second model to predict the probability values ​​of the input image belonging to five types of target positions, including the thumb, index finger, middle finger, ring finger, and pinky finger. The category with the largest probability value is defined as the target category predicted to belong to the palm; S5, when the second model predicts that the target category of the image belonging to the palm is one of the 5 target categories of the finger area, the palm area image belonging to the finger category is input into the improved PCL-Net. If the improved PCL-Net model detects that it does not belong to hand, foot and mouth disease, the image is judged to be a false positive, and a prompt is given to continue to detect the next image; otherwise, the image is judged to be a real hand, foot and mouth disease patient.

2. The hand, foot and mouth disease detection device based on image acquisition according to claim 1 is characterized in that: The image acquisition module is composed of a collector, a processor and a USB. The collector is used to collect hand images and is connected to the USB interface. The image of the patient's palm is obtained by querying it using a smart mobile terminal or a computer terminal, and the collected image is sent to the image processing module for processing; The image processing module receives the images transmitted by the image acquisition module through the network protocol, and then sends them to the improved PCL-Net model and the CAS-PUN model for different processing.

3. The hand, foot and mouth disease detection device based on image acquisition according to claim 2 is characterized in that: The improved PCL-Net adds a residual structure based on the network structure of the original PCL-Net. The improved PCL-Net structure includes a 5×5×64 convolution layer, four downsampling modules, four feature fusion modules, four 3×3×64 small convolution kernels and two 1×1×1 convolution base layers. The input of the model is a palm area image, and the output of the improved PCL-Net is a single value, which determines whether the input palm area image belongs to hand, foot and mouth disease. If it does, the result is 1, and if it does not, the result is 0.

4. The hand, foot and mouth disease detection device based on image acquisition according to claim 3 is characterized in that: The four downsampling modules output feature maps with resolutions of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original image through the downsampling layer. Specifically, the image input into the improved PCL-Net model first passes through a 5×5×64 convolution layer and ReLU activation, and then passes through a convolution layer containing 64 channels before starting the processing of the first downsampling module. The first downsampling module consists of a maximum pooling layer, a corresponding region feature addition structure of the maximum pooling layer, a 3×3×64 convolutional layer, and a ReLU activation function; The feature map processed by the first layer downsampling module is processed by a feature fusion module and then input into the second layer downsampling module; The structure of the second-layer downsampling module is the same as that of the first-layer downsampling module, and the convolution channels are increased from 64 to 128; The structure of the third-layer downsampling module is the same as that of the second-layer downsampling module, and the convolution channels are changed from 128 to 256; The structure of the fourth-layer downsampling module is the same as that of the third-layer downsampling module, and the convolution channel is changed from 256 to 512; The feature maps processed by the second, third, and fourth downsampling modules are input into the second convolution layer after adding the feature maps of the corresponding areas and processing by the feature fusion module. Finally, the results are output through a convolution layer of size 1×1×1. The improved PCL-Net model outputs a value of 0-1, where 0 indicates that the image does not belong to hand, foot and mouth disease, and 1 indicates that the image belongs to hand, foot and mouth disease.

5. The hand, foot and mouth disease detection device based on image acquisition according to claim 4 is characterized in that: The feature fusion module has two feature maps of different levels, one in the first downsampling module and the second downsampling module, and the other in the third downsampling module and the fourth downsampling module. The feature map of the fourth downsampling module and the feature map of the second downsampling module are added at the pixel level and then subjected to a 3×3×64 channel convolution operation to change the number of channels from 64 to 64.

6. The hand, foot and mouth disease detection device based on image acquisition according to claim 5 is characterized in that: The palm area image of the second model has 5 categories, where the categories of thumb, index finger, middle finger, ring finger and little finger are all 1, and the category of background category is 0; The second model outputs 5 categories and one background category. Each category has a probability value. The larger the probability value, the greater the possibility of belonging to the palm category. The category with the largest probability value greater than 0.5 is the background category. The category with the maximum probability value of the other 4 categories is the target area of ​​the palm in the palm image.

7. The hand, foot and mouth disease detection device based on image acquisition according to claim 6 is characterized in that: The second model is based on the pathological data collection and marking system, which uniformly marks the clinically collected pathological data, marks the thumb area as 1, the index finger area as 2, the middle finger area as 3, the ring finger area as 4, the little finger area as 5, and the background category area as 6. The marked data set is sent to the second model as training data for training. The loss function of the second model is the cross entropy loss function, which is expressed as: in, It is The true value of each category, The model predicts The probability value of each category, is the total number of categories.

8. The hand, foot and mouth disease detection device based on image acquisition according to claim 7 is characterized in that: The result detection module includes a palm fingerprint feature collector, a data processing module and a network processor; The data source of this module is the detection results output by the improved PCL-Net and the second model. According to the classification results output by the improved PCL-Net, there are two cases: When the improved PCL-Net outputs a detection result of hand, foot and mouth disease, the input palm area image is input into the second model again to obtain the probability values ​​of 5 categories, among which the value with the largest probability value represents that the palm area of ​​the input image is a 5-category finger area. Since the finger is detected to be a diseased hand, foot and mouth disease, at this time, the network processor outputs the image as a false positive; When the improved PCL-Net output detects whether it belongs to hand, foot and mouth disease, it is directly output as the result.

9. The hand, foot and mouth disease detection device based on image acquisition according to claim 8, characterized in that: The processing steps of the improved PCL-Net are specifically as follows: The collected pictures are sent to the improved PCL-Net, which processes the pictures to determine whether the pictures are of hand, foot and mouth disease. If not, the results are directly output and the pictures are returned to the image acquisition module to collect new pictures. If yes, the image output by the improved PCL-Net is passed to the second model. The second model outputs the categories of five types of palm target areas according to the input image, including the thumb, index finger, middle finger, ring finger and little finger. If it is one of the five types of palm target areas, it is further determined which target area of ​​the five types of palm parts the input palm image belongs to. The image belonging to the finger category is input into the improved PCL-Net model again. The improved PCL-Net model determines the probability that the image does not belong to hand and foot disease. If yes, it outputs that the image is a false positive and returns it to the image acquisition module to acquire a new image. If not, the image is output as belonging to hand, foot and mouth disease, the network processor outputs the result, and returns the image to the image acquisition module to acquire a new image.

10. The hand, foot and mouth disease detection device based on image acquisition according to claim 9, characterized in that: If the input image of S3 is the palm area, in the second model of S4, the second model will output the probability values ​​of the five target areas, among which the value with the highest probability is which of the five target areas, among which the probability of the thumb is 1, the probability of the index finger is 2, the probability of the middle finger is 3, the probability of the ring finger is 4, and the probability of the little finger is 5. If the palm area is in category 6, it is considered that the input image belongs to the background category; If it belongs to the target area of ​​the first five categories, then S4 is further used to determine whether the target area belongs to the palm area. If the probability value of the five categories is the largest, the target area of ​​the palm of the input image is input into the improved PCL-Net in S3. The improved PCL-Net model determines that the probability value of the input disease feature is less than 0.5, and the output result is a false positive. If the probability value of the first five categories is not the largest, the output result is returned to the image acquisition module to collect a new picture.

Images