Method and system for identifying nail fungus abnormalities based on multiple models

Abstract

The present invention discloses a method and system for identifying nail fungus abnormalities based on multiple models, the method comprising: obtaining a limb image including at least two nail images of a patient; determining an abnormal toe image and a normal toe image corresponding to the limb image based on an image segmentation model; determining an onychomycosis identification result corresponding to the abnormal toe image based on a neural network model; and determining the nail fungus infection direction and abnormal identification result corresponding to the limb image based on the normal toe image and the onychomycosis identification result. It can be seen that the present invention can achieve more accurate and comprehensive identification of nail fungus abnormalities, so as to help research institutions judge the overall onychomycosis situation of a patient's limbs, and can also assist in improving the treatment effect of the determined patient treatment plan.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for identifying nail fungus anomalies based on multiple models. Background Art

[0002] The identification of the presence or development stage of nail fungus still basically relies on dermatoscope technology and manual visual identification. Although some research institutions have introduced artificial intelligence technology to identify nail fungus more intelligently and automatically, these technologies are still limited to the training and application of simple single models or single models with staged identification capabilities, and can only make a single judgment on a specific type of nail fungus. Some technical features of multiple different models are not fully applied to more accurately identify the location and infection direction of nail fungus. Therefore, both the accuracy and application prospects are limited. It can be seen that the existing technology has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for identifying onychomycosis abnormalities based on a multi-model, which can achieve more accurate and comprehensive onychomycosis abnormalities identification, so as to help research institutions judge the overall onychomycosis situation of patients' limbs, and also to assist in improving the therapeutic effect of the determined patient treatment plan.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a method for identifying nail fungus abnormalities based on multiple models, the method comprising:

[0005] acquiring an image of a limb including at least two nail images of a patient;

[0006] Based on the image segmentation model, determining an abnormal toe image and a normal toe image corresponding to the limb image;

[0007] Determining, based on a neural network model, an onychomycosis recognition result corresponding to the abnormal toe image;

[0008] According to the normal toe image and the onychomycosis recognition result, the onychomycosis infection direction and the abnormal recognition result corresponding to the limb image are determined.

[0009] As an optional embodiment, in the first aspect of the present invention, the limb image includes a combination of one or more of a negative image, an infrared imaging image, a positive image and a black-and-white image.

[0010] As an optional implementation, in the first aspect of the present invention, determining the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model includes:

[0011] Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; the toe region is a finger region or a toe region;

[0012] Based on a classifier algorithm, determining each of the toe regions as an abnormal toe or a normal toe;

[0013] determining all the toe regions belonging to the abnormal toe as abnormal toe images corresponding to the limb image;

[0014] All the toe regions belonging to normal toes are determined as normal toe images corresponding to the limb image.

[0015] As an optional embodiment, in the first aspect of the present invention, the determining, based on a classifier algorithm, that each of the toe regions is an abnormal toe or a normal toe comprises:

[0016] Based on a toenail region recognition algorithm, determining a toenail region corresponding to each of the toe regions;

[0017] For each of the toe regions, the toenail region corresponding to the toe region is input into the trained nail fungus classifier model to obtain a prediction result of whether the toe region is infected or not;

[0018] According to the infection prediction result, the toe area is determined to be an abnormal toe or a normal toe.

[0019] As an optional implementation, in the first aspect of the present invention, determining the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model includes:

[0020] Each toe area in the abnormal toe image is input into a trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe area in the abnormal toe image; the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including multiple training toe images and corresponding infection type annotations and infection stage annotations.

[0021] As an optional embodiment, in the first aspect of the present invention, determining the onychomycosis infection direction and the abnormality recognition result corresponding to the limb image according to the normal toe image and the onychomycosis recognition result includes:

[0022] Determine whether the toe area in the abnormal toe image and the first prediction result meet a preset complex and multiple disease rule to obtain a first judgment result; the complex and multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold;

[0023] If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result;

[0024] If the first judgment result is yes, input each toe region in the normal toe image into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; the second prediction result includes an infection type and an infection stage;

[0025] The nail fungus infection direction and the abnormality recognition result corresponding to the limb image are determined according to the first prediction result and the second prediction result.

[0026] As an optional embodiment, in the first aspect of the present invention, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result includes:

[0027] Counting the mode of infection types in the first prediction result corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image;

[0028] sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence;

[0029] Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set;

[0030] A line connecting the positions of all toe regions in the first infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0031] As an optional implementation, in the first aspect of the present invention, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes:

[0032] Counting the mode of infection types in all the first prediction results and all the second prediction results, and determining the mode as the abnormality recognition result corresponding to the limb image;

[0033] sorting all the toe regions from low to high according to the infection stages in all the first prediction results and all the second prediction results to obtain a second region sequence;

[0034] Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than the distance threshold as a second infection direction region set;

[0035] A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0036] A second aspect of an embodiment of the present invention discloses a multi-model-based nail fungus abnormality recognition system, the system comprising:

[0037] An acquisition module, used for acquiring a limb image including at least two nail images of a patient;

[0038] A segmentation module, used for determining an abnormal toe image and a normal toe image corresponding to the limb image based on an image segmentation model;

[0039] A recognition module, used for determining the onychomycosis recognition result corresponding to the abnormal toe image based on a neural network model;

[0040] The determination module is used to determine the onychomycosis infection direction and the abnormality recognition result corresponding to the limb image according to the normal toe image and the onychomycosis recognition result.

[0041] As an optional embodiment, in the second aspect of the present invention, the limb image includes a combination of one or more of a negative image, an infrared imaging image, a positive image and a black-and-white image.

[0042] As an optional implementation, in the second aspect of the present invention, the segmentation module determines the specific manner of the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model, including:

[0043] Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; the toe region is a finger region or a toe region;

[0044] Based on a classifier algorithm, determining each of the toe regions as an abnormal toe or a normal toe;

[0045] determining all the toe regions belonging to the abnormal toe as abnormal toe images corresponding to the limb image;

[0046] All the toe regions belonging to normal toes are determined as normal toe images corresponding to the limb image.

[0047] As an optional embodiment, in the second aspect of the present invention, the segmentation module determines the specific manner of each toe region being an abnormal toe or a normal toe based on a classifier algorithm, including:

[0048] Based on a toenail region recognition algorithm, determining a toenail region corresponding to each of the toe regions;

[0049] For each of the toe regions, the toenail region corresponding to the toe region is input into the trained nail fungus classifier model to obtain a prediction result of whether the toe region is infected or not;

[0050] According to the infection prediction result, the toe area is determined to be an abnormal toe or a normal toe.

[0051] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the recognition module determines the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model includes:

[0052] Each toe area in the abnormal toe image is input into a trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe area in the abnormal toe image; the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including multiple training toe images and corresponding infection type annotations and infection stage annotations.

[0053] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the determination module determines the onychomycosis infection direction and the abnormal identification result corresponding to the limb image according to the normal toe image and the onychomycosis identification result includes:

[0054] Determine whether the toe area in the abnormal toe image and the first prediction result meet a preset complex and multiple disease rule to obtain a first judgment result; the complex and multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold;

[0055] If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result;

[0056] If the first judgment result is yes, input each toe region in the normal toe image into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; the second prediction result includes an infection type and an infection stage;

[0057] The nail fungus infection direction and the abnormality recognition result corresponding to the limb image are determined according to the first prediction result and the second prediction result.

[0058] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the determination module determines the nail fungus infection direction and the abnormal recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result includes:

[0059] Counting the mode of infection types in the first prediction result corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image;

[0060] sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence;

[0061] Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set;

[0062] A line connecting the positions of all toe regions in the first infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0063] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes:

[0064] Counting the mode of infection types in all the first prediction results and all the second prediction results, and determining the mode as the abnormality recognition result corresponding to the limb image;

[0065] sorting all the toe regions from low to high according to the infection stages in all the first prediction results and all the second prediction results to obtain a second region sequence;

[0066] Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than the distance threshold as a second infection direction region set;

[0067] A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0068] The third aspect of the present invention discloses another multi-model-based nail fungus abnormality recognition system, the system comprising:

[0069] A memory storing executable program code;

[0070] a processor coupled to the memory;

[0071] The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for identifying nail fungus abnormalities based on multiple models disclosed in the first aspect of the present invention.

[0072] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the multi-model-based nail fungus abnormality identification method disclosed in the first aspect of the present invention.

[0073] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0074] The present invention can use the image segmentation model to determine the abnormal toe image and the normal toe image corresponding to the limb image, so as to preliminarily locate the onychomycosis area existing in the limb, and then comprehensively determine the onychomycosis infection direction and abnormal recognition result corresponding to the limb image by identifying different image parts, thereby realizing more accurate and comprehensive onychomycosis abnormality recognition, so as to help research institutions judge the overall onychomycosis situation of the patient's limb, and also to assist in improving the therapeutic effect of the determined patient treatment plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 It is a flow chart of a method for identifying nail fungus anomalies based on multiple models disclosed in an embodiment of the present invention.

[0077] Figure 2 It is a structural schematic diagram of a multi-model-based onychomycosis anomaly recognition system disclosed in an embodiment of the present invention.

[0078] Figure 3It is a structural schematic diagram of another onychomycosis anomaly recognition system based on multiple models disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0081] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0082] The present invention discloses a method and system for identifying onychomycosis abnormalities based on multiple models, which can use an image segmentation model to determine abnormal toe images and normal toe images corresponding to limb images, so as to preliminarily locate the onychomycosis area existing in the limbs, and then comprehensively determine the onychomycosis infection direction and abnormal identification results corresponding to the limb images by identifying different image parts, thereby achieving more accurate and comprehensive onychomycosis abnormality identification, helping research institutions to judge the overall onychomycosis situation of patients' limbs, and also assisting in improving the therapeutic effect of the determined patient treatment plan. The following are detailed descriptions.

[0083] Embodiment 1

[0084] See also Figure 1 , Figure 1 : is a flow chart of a method for identifying nail fungus abnormalities based on multiple models disclosed in an embodiment of the present invention. Figure 1The described multi-model-based nail fungus anomaly recognition method can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the multi-model-based onychomycosis abnormality recognition method may include the following operations:

[0085] 101. Acquire a limb image including at least two nail images of a patient.

[0086] 102. Based on the image segmentation model, determine the abnormal toe image and the normal toe image corresponding to the limb image.

[0087] 103. Based on the neural network model, determine the onychomycosis recognition result corresponding to the abnormal toe image.

[0088] 104. According to the normal toe image and the onychomycosis recognition result, determine the onychomycosis infection direction and abnormal recognition result corresponding to the limb image.

[0089] It can be seen that the above-mentioned embodiment of the invention can use the image segmentation model to determine the abnormal toe image and the normal toe image corresponding to the limb image, so as to preliminarily locate the onychomycosis area of ​​the limb, and then comprehensively determine the onychomycosis infection direction and abnormal recognition result corresponding to the limb image by identifying different image parts, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's limbs, and can also assist in improving the therapeutic effect of the determined patient treatment plan.

[0090] As an optional embodiment, in the above steps, the limb image includes a combination of one or more of a negative image, an infrared imaging image, a positive image and a black-and-white image.

[0091] It can be seen that through the above optional embodiments, the image type of the limb image is clarified, thereby assisting in achieving more accurate and comprehensive identification of onychomycosis abnormalities, helping research institutions to judge the overall onychomycosis condition of the patient's foot, and also assisting in improving the treatment effect of the determined patient treatment plan.

[0092] As an optional embodiment, in the above step, determining the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model includes:

[0093] Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; optionally, the toe region is a finger region or a toe region;

[0094] Based on the classifier algorithm, determining each toe area as an abnormal toe or a normal toe;

[0095] All toe regions belonging to the abnormal toe are determined as abnormal toe images corresponding to the limb image;

[0096] All toe regions belonging to normal toes are determined as normal toe images corresponding to the limb image.

[0097] It can be seen that through the above optional embodiments, the toe area can be determined by using the trained toe recognition and segmentation algorithm model, and then whether it is an abnormal toe or a normal toe can be determined according to the classifier algorithm, thereby assisting in achieving more accurate and comprehensive identification of nail fungus abnormalities, so as to help research institutions judge the overall onychomycosis of the patient's foot, and also assist in improving the therapeutic effect of the determined patient treatment plan.

[0098] As an optional embodiment, in the above steps, determining each toe region as an abnormal toe or a normal toe based on a classifier algorithm includes:

[0099] Based on the toenail region recognition algorithm, determine the toenail region corresponding to each toe region;

[0100] For each toe region, the toenail region corresponding to the toe region is input into the trained nail fungus classifier model to obtain a prediction result of whether the toe region is infected or not;

[0101] According to the infection prediction result, the toe area is determined to be an abnormal toe or a normal toe.

[0102] It can be seen that through the above optional embodiments, the toenail area recognition algorithm can be used to determine the toenail area of ​​the toe area and then determine whether there is an infection according to a simple classifier algorithm. This is to determine the abnormal area in advance through a simple classifier algorithm model and then perform more complex onychomycosis identification through a subsequent complex neural network algorithm, thereby reducing the amount of calculation and improving calculation efficiency, and assisting in achieving more accurate and comprehensive onychomycosis abnormality identification, so as to help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the treatment effect of the determined patient treatment plan.

[0103] As an optional embodiment, in the above step, determining the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model includes:

[0104] Each toe area in the abnormal toe image is input into the trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe area in the abnormal toe image; optionally, the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including multiple training toe images and corresponding infection type annotations and infection stage annotations.

[0105] It can be seen that through the above optional embodiments, the trained RNN neural network can be used to determine the onychomycosis infection prediction results corresponding to each toe area in the abnormal toe image, so as to facilitate the subsequent determination of the infection direction, thereby assisting in achieving more accurate and comprehensive identification of onychomycosis abnormalities, thereby helping research institutions to judge the overall onychomycosis situation of the patient's foot, and also assisting in improving the treatment effect of the determined patient treatment plan.

[0106] As an optional embodiment, in the above step, determining the nail fungus infection direction and abnormal identification result corresponding to the limb image according to the normal toe image and the onychomycosis identification result includes:

[0107] Determine whether the toe area in the abnormal toe image and the first prediction result meet the preset complex multiple disease rule to obtain a first judgment result; optionally, the complex multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold;

[0108] If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe region in the abnormal toe image and the first prediction result;

[0109] If the first judgment result is yes, each toe region in the normal toe image is input into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; optionally, the second prediction result includes an infection type and an infection stage;

[0110] The nail fungus infection direction and the abnormality recognition result corresponding to the limb image are determined according to the first prediction result and the second prediction result.

[0111] It can be seen that through the above optional embodiments, it is possible to first determine whether complex and multiple onychomycosis exists to determine whether it is necessary to further analyze the infection of the normal toes, so as to improve the calculation efficiency and analysis accuracy, thereby achieving more accurate and comprehensive identification of onychomycosis abnormalities, to help research institutions judge the overall onychomycosis of the patient's foot, and to assist in improving the treatment effect of the determined patient treatment plan.

[0112] As an optional embodiment, in the above step, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result includes:

[0113] Counting the mode of infection types in the first prediction results corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image;

[0114] sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence;

[0115] Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set;

[0116] A line connecting the positions of all toe regions in the first infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0117] It can be seen that through the above optional embodiments, based on the mode calculation and position connection calculation rules, the onychomycosis infection direction and abnormal recognition result corresponding to the limb image are determined, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the treatment effect of the determined patient treatment plan.

[0118] As an optional embodiment, in the above steps, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes:

[0119] Counting the modes of infection types in all first prediction results and all second prediction results, and determining them as abnormal recognition results corresponding to the limb image;

[0120] sorting all toe regions from low to high according to the infection stages in all first prediction results and all second prediction results to obtain a second region sequence;

[0121] Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a distance threshold as a second infection direction region set;

[0122] A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0123] It can be seen that through the above optional embodiments, based on the mode calculation and position connection calculation rules corresponding to all the first prediction results and all the second prediction results, the onychomycosis infection direction and abnormal recognition result corresponding to the limb image are determined, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the therapeutic effect of the determined patient treatment plan.

[0124] Embodiment 2

[0125] See also Figure 2 , Figure 2 Schematic diagram of a multi-model-based nail fungus anomaly recognition system disclosed in an embodiment of the present invention. Figure 2 The described multi-model-based nail fungus anomaly recognition system can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the multi-model-based nail fungus abnormality recognition system may include:

[0126] The acquisition module 201 is used to acquire a limb image including at least two nail images of a patient.

[0127] The segmentation module 202 is used to determine the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model.

[0128] The recognition module 203 is used to determine the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model.

[0129] The determination module 204 is used to determine the onychomycosis infection direction and the abnormality recognition result corresponding to the limb image according to the normal toe image and the onychomycosis recognition result.

[0130] It can be seen that the above-mentioned embodiment of the invention can use the image segmentation model to determine the abnormal toe image and the normal toe image corresponding to the limb image, so as to preliminarily locate the onychomycosis area of ​​the limb, and then comprehensively determine the onychomycosis infection direction and abnormal recognition result corresponding to the limb image by identifying different image parts, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's limbs, and can also assist in improving the therapeutic effect of the determined patient treatment plan.

[0131] As an optional embodiment, the limb image includes a combination of one or more of a negative image, an infrared imaging image, a positive image and a black-and-white image.

[0132] It can be seen that through the above optional embodiments, the image type of the limb image is clarified, thereby assisting in achieving more accurate and comprehensive identification of onychomycosis abnormalities, helping research institutions to judge the overall onychomycosis condition of the patient's foot, and also assisting in improving the treatment effect of the determined patient treatment plan.

[0133] As an optional embodiment, the segmentation module determines the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model in a specific manner, including:

[0134] Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; optionally, the toe region is a finger region or a toe region;

[0135] Based on the classifier algorithm, determining each toe area as an abnormal toe or a normal toe;

[0136] All toe regions belonging to the abnormal toe are determined as abnormal toe images corresponding to the limb image;

[0137] All toe regions belonging to normal toes are determined as normal toe images corresponding to the limb image.

[0138] It can be seen that through the above optional embodiments, the toe area can be determined by using the trained toe recognition and segmentation algorithm model, and then whether it is an abnormal toe or a normal toe can be determined according to the classifier algorithm, thereby assisting in achieving more accurate and comprehensive identification of nail fungus abnormalities, so as to help research institutions judge the overall onychomycosis of the patient's foot, and also assist in improving the therapeutic effect of the determined patient treatment plan.

[0139] As an optional embodiment, the segmentation module determines the specific manner of each toe region being an abnormal toe or a normal toe based on a classifier algorithm, including:

[0140] Based on the toenail region recognition algorithm, determine the toenail region corresponding to each toe region;

[0141] For each toe region, the toenail region corresponding to the toe region is input into the trained nail fungus classifier model to obtain a prediction result of whether the toe region is infected or not;

[0142] According to the infection prediction result, the toe area is determined to be an abnormal toe or a normal toe.

[0143] It can be seen that through the above optional embodiments, the toenail area recognition algorithm can be used to determine the toenail area of ​​the toe area and then determine whether there is an infection according to a simple classifier algorithm. This is to determine the abnormal area in advance through a simple classifier algorithm model and then perform more complex onychomycosis identification through a subsequent complex neural network algorithm, thereby reducing the amount of calculation and improving calculation efficiency, and assisting in achieving more accurate and comprehensive onychomycosis abnormality identification, so as to help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the treatment effect of the determined patient treatment plan.

[0144] As an optional embodiment, the recognition module determines the specific method of the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model, including:

[0145] Each toe area in the abnormal toe image is input into the trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe area in the abnormal toe image; optionally, the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including multiple training toe images and corresponding infection type annotations and infection stage annotations.

[0146] It can be seen that through the above optional embodiments, the trained RNN neural network can be used to determine the onychomycosis infection prediction results corresponding to each toe area in the abnormal toe image, so as to facilitate the subsequent determination of the infection direction, thereby assisting in achieving more accurate and comprehensive identification of onychomycosis abnormalities, thereby helping research institutions to judge the overall onychomycosis situation of the patient's foot, and also assisting in improving the treatment effect of the determined patient treatment plan.

[0147] As an optional embodiment, the specific manner in which the determination module determines the onychomycosis infection direction and the abnormal identification result corresponding to the limb image according to the normal toe image and the onychomycosis identification result includes:

[0148] Determine whether the toe area in the abnormal toe image and the first prediction result meet the preset complex multiple disease rule to obtain a first judgment result; optionally, the complex multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold;

[0149] If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe region in the abnormal toe image and the first prediction result;

[0150] If the first judgment result is yes, each toe region in the normal toe image is input into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; optionally, the second prediction result includes an infection type and an infection stage;

[0151] The nail fungus infection direction and the abnormality recognition result corresponding to the limb image are determined according to the first prediction result and the second prediction result.

[0152] It can be seen that through the above optional embodiments, it is possible to first determine whether complex and multiple onychomycosis exists to determine whether it is necessary to further analyze the infection of the normal toes, so as to improve the calculation efficiency and analysis accuracy, thereby achieving more accurate and comprehensive identification of onychomycosis abnormalities, to help research institutions judge the overall onychomycosis of the patient's foot, and to assist in improving the treatment effect of the determined patient treatment plan.

[0153] As an optional embodiment, the specific manner in which the determination module determines the nail fungus infection direction and the abnormal recognition result corresponding to the limb image according to the toe region in the abnormal toe image and the first prediction result includes:

[0154] Counting the mode of infection types in the first prediction results corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image;

[0155] sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence;

[0156] Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set;

[0157] A line connecting the positions of all toe regions in the first infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0158] It can be seen that through the above optional embodiments, based on the mode calculation and position connection calculation rules, the onychomycosis infection direction and abnormal recognition result corresponding to the limb image are determined, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the treatment effect of the determined patient treatment plan.

[0159] As an optional embodiment, the specific manner in which the determination module determines the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes:

[0160] Counting the modes of infection types in all first prediction results and all second prediction results, and determining them as abnormal recognition results corresponding to the limb image;

[0161] sorting all toe regions from low to high according to the infection stages in all first prediction results and all second prediction results to obtain a second region sequence;

[0162] Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a distance threshold as a second infection direction region set;

[0163] A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

[0164] It can be seen that through the above optional embodiments, based on the mode calculation and position connection calculation rules corresponding to all the first prediction results and all the second prediction results, the onychomycosis infection direction and abnormal recognition result corresponding to the limb image are determined, thereby achieving more accurate and comprehensive onychomycosis abnormality recognition, which can help research institutions judge the overall onychomycosis situation of the patient's foot, and can also assist in improving the therapeutic effect of the determined patient treatment plan.

[0165] Embodiment 3

[0166] See also Figure 3 , Figure 3 It is another onychomycosis abnormality recognition system based on multiple models disclosed in an embodiment of the present invention. Figure 3 The described multi-model-based nail fungus anomaly recognition system is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the multi-model-based nail fungus abnormality recognition system may include:

[0167] A memory 301 storing executable program codes;

[0168] a processor 302 coupled to the memory 301;

[0169] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the method for identifying nail fungus abnormalities based on multiple models described in the first embodiment.

[0170] Embodiment 4

[0171] The embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the method for identifying nail fungus anomalies based on multiple models described in the first embodiment.

[0172] Embodiment 5

[0173] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the method for identifying nail fungus anomalies based on multiple models described in the first embodiment.

[0174] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0177] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0178] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0182] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0183] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0185] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0186] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0187] Finally, it should be noted that the method and system for identifying nail fungus abnormalities based on multiple models disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical schemes described in the aforementioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical schemes from the spirit and scope of the technical schemes of the embodiments of the present invention.

Claims

1. A method for identifying nail fungus abnormalities based on multiple models, characterized in that: The method comprises: acquiring an image of a limb including at least two nail images of a patient; Determining an abnormal toe image and a normal toe image corresponding to the limb image based on an image segmentation model includes: Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; the toe region is a finger region or a toe region; Based on a classifier algorithm, determining each of the toe regions as an abnormal toe or a normal toe; determining all the toe regions belonging to the abnormal toe as abnormal toe images corresponding to the limb image; Determine all the toe regions belonging to normal toes as normal toe images corresponding to the limb image; Determining the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model includes: Inputting each toe region in the abnormal toe image into a trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe region in the abnormal toe image; the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including a plurality of training toe images and corresponding infection type annotations and infection stage annotations; Determining the onychomycosis infection direction and abnormality recognition result corresponding to the limb image according to the normal toe image and the onychomycosis recognition result includes: Determine whether the toe area in the abnormal toe image and the first prediction result meet a preset complex and multiple disease rule to obtain a first judgment result; the complex and multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold; If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result includes: Counting the mode of infection types in the first prediction result corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image; sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence; Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set; Determine a line connecting the positions of all toe regions in the first infection direction region set in the limb image as the nail fungus infection direction corresponding to the limb image; If the first judgment result is yes, input each toe region in the normal toe image into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; the second prediction result includes an infection type and an infection stage; Determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes: Counting the mode of infection types in all the first prediction results and all the second prediction results, and determining the mode as the abnormality recognition result corresponding to the limb image; sorting all the toe regions from low to high according to the infection stages in all the first prediction results and all the second prediction results to obtain a second region sequence; Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than the distance threshold as a second infection direction region set; A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

2. The method for identifying nail fungus abnormalities based on multiple models according to claim 1, characterized in that: The limb image includes a combination of one or more of a negative image, an infrared imaging image, a positive image and a black-and-white image.

3. The method for identifying nail fungus abnormalities based on multiple models according to claim 1, characterized in that: The step of determining each of the toe regions as an abnormal toe or a normal toe based on a classifier algorithm comprises: Based on a toenail region recognition algorithm, determining a toenail region corresponding to each of the toe regions; For each of the toe regions, the toenail region corresponding to the toe region is input into the trained nail fungus classifier model to obtain a prediction result of whether the toe region is infected or not; According to the infection prediction result, the toe area is determined to be an abnormal toe or a normal toe.

4. A multi-model based onychomycosis anomaly recognition system, characterized in that: The system comprises: An acquisition module, used for acquiring a limb image including at least two nail images of a patient; A segmentation module, used to determine the abnormal toe image and the normal toe image corresponding to the limb image based on the image segmentation model, comprising: Based on the trained toe recognition and segmentation algorithm model, determining at least one toe region corresponding to the limb image; the toe region is a finger region or a toe region; Based on a classifier algorithm, determining each of the toe regions as an abnormal toe or a normal toe; determining all the toe regions belonging to the abnormal toe as abnormal toe images corresponding to the limb image; Determine all the toe regions belonging to normal toes as normal toe images corresponding to the limb image; The recognition module is used to determine the onychomycosis recognition result corresponding to the abnormal toe image based on the neural network model, including: Inputting each toe region in the abnormal toe image into a trained RNN neural network to obtain a first prediction result of onychomycosis infection corresponding to each toe region in the abnormal toe image; the first prediction result includes an infection type and an infection stage; the RNN neural network is trained by a training data set including a plurality of training toe images and corresponding infection type annotations and infection stage annotations; A determination module, for determining the onychomycosis infection direction and abnormality recognition result corresponding to the limb image according to the normal toe image and the onychomycosis recognition result, comprises: Determine whether the toe area in the abnormal toe image and the first prediction result meet a preset complex and multiple disease rule to obtain a first judgment result; the complex and multiple disease rule is that the number of toe areas in the abnormal toe image is greater than a first threshold and the number of different infection types in all the first prediction results is greater than a second threshold; If the first judgment result is no, determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the toe area in the abnormal toe image and the first prediction result includes: Counting the mode of infection types in the first prediction result corresponding to all toe regions in the abnormal toe image, and determining it as the abnormal recognition result corresponding to the limb image; sorting all toe regions in the abnormal toe image from low to high according to the infection stage in the first prediction result to obtain a first region sequence; Determine at least two toe regions in the first region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than a preset distance threshold as a first infection direction region set; Determine a line connecting the positions of all toe regions in the first infection direction region set in the limb image as the nail fungus infection direction corresponding to the limb image; If the first judgment result is yes, input each toe region in the normal toe image into the RNN neural network to obtain a second prediction result corresponding to each toe region in the normal toe image; the second prediction result includes an infection type and an infection stage; Determining the nail fungus infection direction and the abnormality recognition result corresponding to the limb image according to the first prediction result and the second prediction result includes: Counting the mode of infection types in all the first prediction results and all the second prediction results, and determining the mode as the abnormality recognition result corresponding to the limb image; sorting all the toe regions from low to high according to the infection stages in all the first prediction results and all the second prediction results to obtain a second region sequence; Determine at least two toe regions in the second region sequence that are adjacent in position in the sequence and whose position distance in the limb image is less than the distance threshold as a second infection direction region set; A line connecting the positions of all toe regions in the second infection direction region set in the limb image is determined as the nail fungus infection direction corresponding to the limb image.

5. A multi-model based onychomycosis anomaly recognition system, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the multi-model-based nail fungus abnormality identification method according to any one of claims 1 to 3.

Images