Lesion recognition and malignancy risk assessment system and device using artificial intelligence based on ultrasound diagnosis

Abstract

To provide a lesion recognition and malignancy risk assessment system and device using artificial intelligence based on ultrasound diagnosis.SOLUTION: A lesion recognition and malignancy risk assessment system according to the present invention includes an image pre-processing module, a lesion diagnosis module, an attribute classification module, and a result feedback module. Utilizing deep learning techniques, the system adopts a fully data-driven method to learn Priori information in a deep layer inherent to images, highly simulates a thinking mode and determination rules of an image diagnostic doctor in clinical practice through cross-validation of mutually-different learning methods, realizes accurate and efficient recognition of nodular lesions and assessment of malignancy risk, and effectively avoids oversight and wrong diagnosis. The system features a high level of intelligence, accuracy, reliability, and efficiency. The system has the high recognition performance and malignancy risk assessment performance for early-stage nodular lesions of various cancers such as lung cancer, thyroid cancer, breast cancer, and liver cancer, thereby sufficiently satisfying actual clinical needs.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence diagnosis support devices, and specifically relates to a lesion recognition and malignant risk assessment system and device based on artificial intelligence using ultrasonic diagnosis.

Background Art

[0002] Early-stage lesions of various cancers such as lung cancer, thyroid cancer, breast cancer, and liver cancer may all appear as nodules (or lumps). However, many of these nodules are small in size, low in contrast, and highly heterogeneous in shape. Therefore, the recognition and diagnosis of benign and malignant nodules are usually performed manually by clinical experts with experience in the radiology department through reading films. However, the appearance of nodules in ultrasonic images is easily confused with several other tissues or sites, and completely depends on the personal experience and judgment of the radiologist. This not only requires a large amount of time and effort from the doctor, but also misses and misdiagnoses are very likely to occur. Therefore, in clinical work, there is a great need to apply a nodule lesion recognition and malignant risk assessment system and device with higher intelligence level, higher accuracy, higher reliability, and higher work efficiency.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In order to overcome the defects that the efficiency of ultrasonic diagnosis that completely depends on the manually read film results is low, the reliability is low, and misses and misdiagnoses are likely to occur, the inventor has designed and developed a lesion recognition and malignant risk assessment system and related device based on artificial intelligence using ultrasonic diagnosis. This system and device have high recognition and malignant risk assessment capabilities for early-stage nodule lesions of various cancers, can greatly avoid the occurrence of misses and misdiagnoses, and this system has characteristics such as high intelligence level, high accuracy, high reliability, and high work efficiency, and can fully meet the actual clinical needs.

Means for Solving the Problems

[0004] In the design process of this system, artificial intelligence technology (deep learning technology) is utilized to eliminate the need for prior information of image applications designed manually in the past. By using a completely data-driven approach, deep-level prior information specific to images is learned. Based on this, through the mutual verification of different learning methods, the thinking mode and judgment rules of image diagnosticians in clinical practice are highly simulated, thereby realizing accurate and efficient recognition of nodular lesions and malignant risk assessment, and effectively avoiding omissions and misdiagnoses.

[0005] Specifically, the lesion recognition and malignant risk assessment system based on ultrasonic diagnosis by artificial intelligence according to the first aspect of the present invention includes an image preprocessing module that preprocesses ultrasonic images that need to be read to obtain an image including only the parts suspected of having nodules, a lesion diagnosis module that determines the presence or absence of nodules in the diagnostic area of the image and determines the characteristics of the observed nodules, an attribute classification module that performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module and obtains a malignant risk assessment result of the nodules based on the obtained score value, and a result feedback module that summarizes the diagnosis and determination results and displays the recognition and malignant risk assessment results of the lesions.

[0006] Furthermore, according to the second aspect of the present invention, the image preprocessing module of the lesion recognition and malignant risk assessment system based on ultrasonic diagnosis by artificial intelligence according to the first aspect of the present invention preprocessing the ultrasonic images that need to be read includes trimming the images, scaling and normalizing the image sizes.

[0007] Furthermore, according to the third aspect of the present invention, the lesion diagnosis module of the lesion recognition and malignant risk assessment system based on ultrasonic diagnosis by artificial intelligence according to the first or second aspect of the present invention has performed deep learning on the determination of the benign or malignant nature of nodule characteristics in advance.

[0008] Furthermore, according to the fourth aspect of the present invention, the operation process of the lesion recognition and malignancy risk assessment system based on artificial intelligence according to any one of the first to third aspects of the present invention is as follows: A model 1 construction step (1) in which the lesion diagnosis module directly performs lesion recognition on the original ultrasonic image, determines the malignancy of the observed nodule's characteristics, and obtains the determination result of model 1; A model 2 construction step (2) in which the image preprocessing module performs image preprocessing on the original ultrasonic image to obtain an image including only the suspected nodule sites, the lesion diagnosis module performs lesion recognition on the image including only the suspected nodule sites, and determines the malignancy of the observed nodule's characteristics to obtain the determination result of model 2; A model 3 construction step (3) in which the image preprocessing module performs image preprocessing on the original ultrasonic image to obtain an image including only the suspected nodule sites, the lesion diagnosis module performs lesion recognition on the image including only the suspected nodule sites, the attribute classification module performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module, and determines the malignancy of the nodule's characteristics based on the obtained score value to obtain the determination result of model 3; It includes a comprehensive evaluation step (4) of comprehensively evaluating the determination results of the above models 1, 2, and 3, and the result feedback module displays the lesion recognition and malignancy risk assessment results.

[0009] Furthermore, according to the fifth aspect of the present invention, the comprehensive evaluation in step (4) of the operation process of the above-mentioned lesion recognition and malignancy risk assessment system based on artificial intelligence according to the fourth aspect is as follows: (a) If the determination results of models 1, 2, and 3 are all benign, it is determined to be benign. (b) If the determination results of models 1, 2, and 3 do not completely match, it is determined that there is a suspicion of malignancy, and the doctor is prompted to manually confirm. (c) If the determination results of models 1, 2, and 3 are all malignant, it is determined to be malignant, following the determination rule.

[0010] In the design process, the above three diagnostic modes (Model 1, Model 2, and Model 3) can be obtained through deep learning for ultrasonic images, but it has been found that each has its own drawbacks and advantages.

[0011] (1) Model 1 directly performs a learning determination on the benign or malignant nature of nodule characteristics for the original ultrasonic image. Its advantage is that the object of learning and determination is close to the actual ultrasonic image, does not require a complex cleaning process, and has more collation information. Its drawback is that there is too much interference information, which affects the efficiency and accuracy of learning and determination.

[0012] (2) Model 2 performs preprocessing on the original ultrasonic image, then obtains an image containing only the part suspected of having a nodule, and performs a learning determination on the benign or malignant nature of nodule characteristics for this image (that is, performs a learning determination on the benign or malignant nature of nodule characteristics only for the nodule position image). Its advantage is that there is no or little interference information in the object of learning and determination. Its drawback is that the workflow is complicated, there is no or little collation information, which affects the accuracy of learning and determination.

[0013] (3) Model 3 performs preprocessing on the original ultrasonic image, then obtains an image containing only the part suspected of having a nodule, performs attribute classification on the nodule in this image, then performs grading scoring, and further performs a learning determination on the benign or malignant nature of nodule characteristics based on the obtained score value. Its advantage is that the context is clear and the basis is reliable. Its drawback is that in addition to performing a cleaning process on the original ultrasonic image, doctors need to annotate each attribute, the workload is huge, and the number of ultrasonic images that need to be learned is huge.

[0014] The evaluation system according to the fifth aspect of the present invention fully integrates the above three diagnostic modes to construct an artificial intelligence comprehensive diagnostic mode based on deep learning, in order to realize efficient and accurate lesion recognition and malignant risk assessment as much as possible, overcome the defects and drawbacks existing in a single diagnostic mode, and achieve the purpose of maximizing the avoidance of omissions and misdiagnoses.

[0015] A sixth aspect of the present invention relates to an ultrasonic diagnostic apparatus or device including the above-described artificial intelligence-based lesion recognition and malignancy risk assessment system according to any one of the first to fifth aspects.

[0016] An artificial intelligence-based lesion recognition and malignancy risk assessment apparatus based on ultrasonic diagnosis according to a seventh aspect of the present invention includes an image preprocessing unit that preprocesses an ultrasonic image that needs to be interpreted to obtain an image including only a part suspected of having a nodule, a lesion diagnosis unit that determines the presence or absence of a nodule in the diagnostic region of the image and determines the characteristics of the observed nodule, an attribute classification unit that performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis unit and obtains a malignancy risk assessment result of the nodules based on the obtained score value, and a result feedback unit that summarizes the diagnosis and determination results and displays the lesion recognition and malignancy risk assessment results.

[0017] Furthermore, according to an eighth aspect of the present invention, the image preprocessing unit of the artificial intelligence-based lesion recognition and malignancy risk assessment apparatus based on ultrasonic diagnosis according to the seventh aspect of the present invention preprocessing the ultrasonic image that needs to be interpreted includes trimming the image and scaling and normalizing the image size.

[0018] Furthermore, according to a ninth aspect of the present invention, the lesion diagnosis unit of the artificial intelligence-based lesion recognition and malignancy risk assessment apparatus based on ultrasonic diagnosis according to the seventh or eighth aspect of the present invention has previously performed deep learning for determining the malignancy of nodule characteristics.

[0019] Furthermore, according to a tenth aspect of the present invention, the operation process of the artificial intelligence-based lesion recognition and malignancy risk assessment apparatus according to any one of the seventh to ninth aspects of the present invention is a model 1 construction step (1) in which the lesion diagnosis unit directly performs lesion recognition on the original ultrasonic image and determines the malignancy of the characteristics of the observed nodule to obtain the determination result of model 1, The model 2 construction step (2) in which the image preprocessing unit performs image preprocessing on the original ultrasonic image to obtain an image including only the part suspected of having a nodule, and the lesion diagnosis unit performs lesion recognition on the image including only the part suspected of having a nodule and determines the malignancy of the observed nodule to obtain the determination result of model 2. The model 3 construction step (3) in which the image preprocessing unit performs image preprocessing on the original ultrasonic image to obtain an image including only the part suspected of having a nodule, the lesion diagnosis unit performs lesion recognition on the image including only the part suspected of having a nodule, the attribute classification unit performs attribute classification and grading scoring on the nodule observed by the lesion diagnosis unit, and determines the malignancy of the nodule based on the obtained score value to obtain the determination result of model 3. Based on the determination results of the above model 1, model 2, and model 3, (a) When the determination results of model 1, model 2, and model 3 are all benign, it is determined to be benign. (b) When the determination results of model 1, model 2, and model 3 do not completely match, it is determined that there is a suspicion of malignancy, and the doctor is prompted to manually confirm. (c) When the determination results of model 1, model 2, and model 3 are all malignant, it is determined to be malignant, and a comprehensive evaluation is performed according to the following determination rule. The comprehensive evaluation step (4) in which the result feedback unit displays the recognition of the lesion and the evaluation result of the malignancy risk.

[0020] As described above, the lesion recognition and malignancy risk evaluation system and device based on artificial intelligence based on ultrasonic diagnosis of the present invention have the following advantages in the application process.

[0021] (1) The system of the first aspect and the device of the seventh aspect of the present invention have high recognition and malignancy risk evaluation capabilities for early nodular lesions of various cancers, and have characteristics such as high intelligence level, high accuracy, high reliability, and high working efficiency, and can fully meet the actual clinical needs.

[0022] (2) The system of the first aspect of the present invention and the device of the seventh aspect can significantly save the diagnosis time and labor costs, improve the diagnosis efficiency, and reduce human errors compared with the conventional manual fluoroscopy.

[0023] (3) The system of the third aspect of the present invention and the device that belongs to the tenth aspect of the invention and is the device of the ninth aspect are advantageous for comprehensive analysis and comprehensive evaluation by fusing a plurality of diagnostic modes based on deep learning. It enables general radiologists to reach the level of ultrasound specialists through artificial intelligence, minimizes the occurrence of oversights and misdiagnoses, and is advantageous for alleviating the problem of difficulty in seeing a doctor due to the shortage of high-quality medical resources.

[0024] (4) The system of the fifth aspect of the present invention and the device of the tenth aspect utilize the mutual verification of the determination results in three different ways to improve the accuracy of diagnosis, play a role in initial screening and diagnosis for lesions that are likely to be overlooked due to human factors (for example, lesions with a small volume), reduce the pain of patients, reduce the national medical expenses, and can reduce the incidence of medical disputes.

Brief Description of the Drawings

[0025] To more clearly explain the technical means of the embodiments of the present invention, the drawings necessary for the description of the embodiments are briefly described below. It is obvious that the drawings described below are only specific embodiments of the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0026]

Figure 1

Figure 2

Figure 3

Embodiments for Carrying Out the Invention

[0027] Hereinafter, embodiments of the present invention will be described with reference to specific specific examples. Those skilled in the art can easily understand other advantages and functions of the present invention from the content disclosed herein. The present invention may be implemented or applied in other different embodiments, and various modifications or changes can be made to each detail in this specification based on different viewpoints and applications without departing from the spirit of the present invention.

[0028] Before further explaining the specific embodiments of the present invention, it should be understood that the protection scope of the present invention is not limited to the following specific specific embodiments. Also, it should be understood that the terms used in the embodiments of the present invention are for explaining specific specific embodiments and are not for limiting the protection scope of the present invention.

[0029] Unless otherwise defined, all technical terms and scientific terms used in the present invention have the same meaning as the general understanding of those skilled in the art.

[0030] In the present invention, unless otherwise explained, all the methods mentioned are common methods in this field.

Examples

[0031] An artificial intelligence-based lesion recognition and malignancy risk assessment system based on ultrasonic diagnosis (FIG. 1, FIG. 2) preprocesses ultrasonic images that need to be interpreted to obtain images containing only suspected nodule sites. The image preprocessing includes an image preprocessing module that performs image trimming, scaling, and normalization of the ultrasonic images that need to be interpreted. Performs deep learning for pre - determination of the benign or malignant nature of nodule characteristics in advance, determines the presence or absence of nodules in the diagnostic region of the image, and a lesion diagnosis module for determining the characteristics of the observed nodules, An attribute classification module that performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module, and obtains a malignant risk assessment result of the nodules based on the obtained score value, A result feedback module that summarizes the diagnosis and determination results, and displays the recognition of the lesion and the malignant risk assessment result, is included.

[0032] The operation process of the above - mentioned lesion recognition and malignant risk assessment system by artificial intelligence is as follows. A model 1 construction step (1) in which the lesion diagnosis module directly performs lesion recognition on the original ultrasonic image, determines the benign or malignant nature of the characteristics of the observed nodules, and obtains the determination result of model 1. An image pre - processing module performs image pre - processing on the original ultrasonic image to obtain an image containing only the suspected nodule sites. The lesion diagnosis module performs lesion recognition on the image containing only the suspected nodule sites, determines the benign or malignant nature of the characteristics of the observed nodules, and obtains the determination result of model 2. An image pre - processing module performs image pre - processing on the original ultrasonic image to obtain an image containing only the suspected nodule sites. The lesion diagnosis module performs lesion recognition on the image containing only the suspected nodule sites. The attribute classification module performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module, determines the benign or malignant nature of the characteristics of the nodules based on the obtained score value, and obtains the determination result of model 3. Based on the determination results of the above - mentioned model 1, model 2, and model 3, (a) If the determination results of model 1, model 2, and model 3 are all benign, it is determined to be benign. (b) If the determination results of model 1, model 2, and model 3 do not completely match, it is determined that there is a suspicion of malignancy, and the doctor is urged to manually confirm. (c) If the determination results of Model 1, Model 2, and Model 3 are all malignant, then a comprehensive evaluation is performed according to the determination rule of determining malignancy, The result feedback module includes a comprehensive evaluation step (4) of displaying the recognition of the lesion and the result of the malignant risk assessment.

Example

[0033] Application of the artificial intelligence-based lesion recognition and malignant risk assessment system based on ultrasonic diagnosis in Example 1 in the ultrasonic diagnosis assistance of thyroid nodules (Figure 3) (1) Upload the original ultrasonic image to the system. The lesion diagnosis module performs lesion recognition and determination on the image, and Model 1 directly outputs the determination result (not displayed on the result interface).

[0034] (2) The image preprocessing module preprocesses the original ultrasonic image to obtain an image containing only the suspected nodule part. The lesion diagnosis module performs lesion recognition and determination on the image containing only the suspected nodule part, and Model 2 outputs the determination result (not displayed on the result interface).

[0035] (3) The lesion diagnosis module performs lesion recognition on the image containing only the suspected nodule part. The attribute classification module performs attribute classification and grading scoring on the observed nodules according to each attribute score of the TI-RADS classification of the nodules, and performs a determination on the benign or malignancy of the nodule properties based on the obtained score value. Model 3 outputs the determination result.

[0036] (4) Based on the determination results of the above Model 1, Model 2, and Model 3, (a) If the determination results output by Model 1, Model 2, and Model 3 are all benign, then it is determined to be benign, (b) If the determination results output by Model 1, Model 2, and Model 3 do not completely match, then it is determined that there is a suspicion of malignancy, and the doctor is urged to manually confirm, When the determination results output by Model 1, Model 2, and Model 3 are all malignant, perform comprehensive evaluation and verification according to the determination rule of determining it as malignant. The result feedback module displays the lesion recognition and malignant risk assessment results after comprehensive evaluation and verification.

[0037] As described above, the preferred specific embodiments and examples of the present invention have been described in detail. However, the present invention is not limited to the above embodiments and examples, and various modifications can be made without departing from the idea of the present invention within the scope of knowledge possessed by those skilled in the art.

Claims

1. An image preprocessing module that preprocesses ultrasonic images that need to be interpreted to obtain an image containing only parts suspected of nodules; A lesion diagnosis module that determines the presence or absence of nodules in the diagnostic region of the image and determines the characteristics of the observed nodules; An attribute classification module that performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module and obtains a malignant risk assessment result of the nodules based on the obtained score value; A result feedback module that summarizes the diagnosis and determination results and displays the recognition of the lesion and the malignant risk assessment result, characterized by including an artificial intelligence-based lesion recognition and malignant risk assessment system based on ultrasonic diagnosis.

2. The artificial intelligence-based lesion recognition and malignant risk assessment system according to claim 1, wherein the image preprocessing module preprocessing the ultrasonic image that needs to be interpreted includes image trimming, scaling, and normalization of the image size.

3. The artificial intelligence-based lesion recognition and malignant risk assessment system according to claim 1, wherein the lesion diagnosis module has previously performed deep learning for determining the malignancy of nodule characteristics.

4. A model 1 construction step (1) in which the lesion diagnosis module directly performs lesion recognition on the original ultrasonic image, determines the malignancy of the characteristics of the observed nodules, and obtains the determination result of model 1; A model 2 construction step (2) in which the image preprocessing module performs image preprocessing on the original ultrasonic image to obtain an image containing only parts suspected of nodules, the lesion diagnosis module performs lesion recognition on the image containing only parts suspected of nodules, and determines the malignancy of the characteristics of the observed nodules to obtain the determination result of model 2; The image preprocessing module performs image preprocessing on the original ultrasonic image to obtain an image containing only the suspected nodule sites, the lesion diagnosis module performs lesion recognition on the image containing only the suspected nodule sites, the attribute classification module performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis module, and determines the benign or malignant nature of the nodules based on the obtained score values to obtain the determination result of Model 3, which is the Model 3 construction step (3); Performing a comprehensive evaluation on the determination results of the above-mentioned Model 1, Model 2, and Model 3, and the comprehensive evaluation step (4) in which the result feedback module displays the recognition of the lesion and the malignant risk evaluation result. The artificial intelligence-based lesion recognition and malignant risk evaluation system according to claim 1 is characterized by including this.

5. The comprehensive evaluation in step (4) is (a) When the determination results of Model 1, Model 2, and Model 3 are all benign, it is determined to be benign. (b) When the determination results of Model 1, Model 2, and Model 3 do not completely match, it is determined that there is a suspicion of malignancy, and the doctor is prompted to manually confirm. (c) When the determination results of Model 1, Model 2, and Model 3 are all malignant, it is determined to be malignant. The artificial intelligence-based lesion recognition and malignant risk evaluation system according to claim 4 is characterized by following the determination rule.

6. An ultrasonic diagnostic apparatus or device equipped with the artificial intelligence-based lesion recognition and malignant risk evaluation system according to any one of claims 1 to 3.

7. An image preprocessing unit that preprocesses the ultrasonic image that needs to be read and obtains an image containing only the suspected nodule sites. A lesion diagnosis unit that determines the presence or absence of nodules in the diagnostic area of the image and determines the nature of the observed nodules. An attribute classification unit that performs attribute classification and grading scoring on the nodules observed by the lesion diagnosis unit and obtains the malignant risk evaluation result of the nodules based on the obtained score values. A lesion recognition and malignancy risk assessment apparatus based on ultrasonic diagnosis by artificial intelligence, comprising: a result feedback unit that summarizes diagnosis and determination results and displays lesion recognition and malignancy risk assessment results.

8. The apparatus for lesion recognition and malignancy risk assessment by artificial intelligence according to claim 7, wherein the image preprocessing unit preprocessing the ultrasonic image that needs to be interpreted includes image trimming, and scaling and normalization of the image size.

9. The apparatus for lesion recognition and malignancy risk assessment by artificial intelligence according to claim 7, wherein the lesion diagnosis unit has performed deep learning for determining the benignity or malignancy of nodule characteristics in advance.

10. A model 1 construction step (1) in which the lesion diagnosis unit directly performs lesion recognition on the original ultrasonic image and determines the benignity or malignancy of the observed nodule characteristics to obtain the determination result of model 1; A model 2 construction step (2) in which the image preprocessing unit performs image preprocessing on the original ultrasonic image to obtain an image including only the part suspected of having a nodule, the lesion diagnosis unit performs lesion recognition on the image including only the part suspected of having a nodule, and determines the benignity or malignancy of the observed nodule characteristics to obtain the determination result of model 2; A model 3 construction step (3) in which the image preprocessing unit performs image preprocessing on the original ultrasonic image to obtain an image including only the part suspected of having a nodule, the lesion diagnosis unit performs lesion recognition on the image including only the part suspected of having a nodule, the attribute classification unit performs attribute classification and grading scoring on the nodule observed by the lesion diagnosis unit, and determines the benignity or malignancy of the nodule characteristics based on the obtained score value to obtain the determination result of model 3; Based on the determination results of the above model 1, model 2, and model 3, (a) When the determination results of model 1, model 2, and model 3 are all benign, it is determined to be benign; (b) When the determination results of Model 1, Model 2, and Model 3 do not completely match, it is determined that there is a suspicion of malignancy, and the doctor is prompted to manually confirm. (c) When the determination results of Model 1, Model 2, and Model 3 are all malignant, a comprehensive evaluation is performed according to the determination rule of determining it as malignant. The result feedback unit includes a comprehensive evaluation step (4) of displaying the recognition of the lesion and the evaluation result of the malignancy risk, and the artificial intelligence-based lesion recognition and malignancy risk evaluation device according to claim 7 is characterized by including the above.

Images