Medical image diagnosis method based on computer vision

Abstract

The invention relates to the technical field of medical image diagnosis, and discloses a medical image diagnosis method based on computer vision, and the method comprises the steps: obtaining a medical diagnosis image of a patient through obtaining the inquiry information of the patient and taking a related medical image as a reference; identifying a key region characteristic spectrum of the medical diagnosis image based on the inquiry information, and analyzing the similarity between the related medical images according to a key region of the key region characteristic spectrum; identifying an abnormal area based on an analysis result of the similarity, generating different color identifiers for the abnormal area, superposing the different color identifiers on the medical diagnosis image, providing a multi-angle medical diagnosis image, generating an abnormal report, and uploading the abnormal report to an electronic case of a patient for reference by a doctor; a marking window is provided, a doctor can manually modify marking content, optimization iteration is carried out on the system according to final diagnosis information of the doctor, and the diagnosis efficiency and the diagnosis accuracy of the doctor are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image diagnosis, and in particular to a medical image diagnosis method based on computer vision. Background Art

[0002] As medical image diagnosis technology continues to develop, medical imaging technology will be combined with computer vision technology to innovate more diagnostic and treatment methods, providing more accurate and efficient methods for medical diagnosis and treatment. However, existing medical image diagnosis still provides doctors with a single medical image, relying on the doctor's professional knowledge and experience to detect abnormalities through naked eye observation and analysis of medical images. This method has problems such as strong subjectivity, easy to miss details, and low diagnostic efficiency. In addition, existing medical image diagnosis technology mostly stays at the simple image classification or target detection level, lacking in-depth analysis and precise labeling of medical image characteristics. Therefore, it is necessary to design a medical image diagnosis method based on computer vision to improve doctors' diagnostic efficiency and diagnostic accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide a medical image diagnosis method based on computer vision to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a medical image diagnosis system based on computer vision, which is executed in a medical image diagnosis method based on computer vision, and the operation steps of the method include:

[0005] Step S1: Obtain the patient's medical information, retrieve relevant medical images as reference, and obtain the patient's medical diagnosis image;

[0006] Step S2: identifying a key region feature map of the medical diagnostic image based on the medical inquiry information, and analyzing similarities between the related medical images according to key regions of the key region feature map;

[0007] Step S3: Identify abnormal areas based on the similarity analysis results, generate different color labels for the abnormal areas and superimpose them on the medical diagnostic image, provide multi-angle medical diagnostic images, and generate abnormality reports, which are uploaded to the patient's electronic medical records for the doctor's reference;

[0008] Step S4: Provide a marking window where the doctor can manually modify the marking content and optimize the system iteratively based on the doctor's final diagnosis information.

[0009] According to the above technical solution, step S1 further includes the following steps:

[0010] Step S11: The medical inquiry information includes: the patient's description of the symptoms, the doctor's preliminary diagnosis information, and the patient's historical medical records. The medical diagnostic images are images of the patient's body taken by different medical instruments;

[0011] Step S12: Based on the medical inquiry information, medical images related to the patient's condition are matched from the database as references, wherein the related medical images include: historical medical images of the patient, similar medical images, and normal standard medical images, wherein the historical medical images of the patient are used to longitudinally compare the changes in the patient's condition, the similar medical images are used as references for horizontal comparisons, and the normal standard medical images are used to assess whether the patient's current condition deviates from the normal standard. The historical medical images are retrieved according to the actual conditions of the patient, and if the patient has no historical medical images, the historical medical images of the patient will not be used as references;

[0012] Step S13: smoothing the medical diagnostic image using a Gaussian filtering algorithm, and performing contrast adjustment according to the grayscale distribution of the medical diagnostic image.

[0013] According to the above technical solution, step S2 further includes the following steps:

[0014] Step S21: parsing the medical inquiry information by natural language processing technology, converting the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and using the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of a convolutional neural network CNN;

[0015] Step S22: generating feature maps of multiple scales based on the medical diagnostic image by fusing the embedded vector of the inquiry information through the convolutional neural network CNN, adaptively allocating attention weights on image regions of different scales through an attention weight mechanism, and strengthening the features of key regions to construct a feature map of key regions;

[0016] Step S23: Based on the weight of the key area feature map and the set threshold, the area where the weight is greater than the threshold is the key area, and a multi-scale similarity analysis is performed on the related medical images between the key areas, and the similarity between the key areas and the corresponding areas in the related medical images is compared, wherein the similarity calculation formula is:

[0017]

[0018] S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B)

[0019] Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively. ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ AB represents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

[0020] According to the above technical solution, step S3 further includes the following steps:

[0021] Step S31: color-marking the related medical images step by step in the order of first analyzing the normal standard medical image, then analyzing the similar medical image, and then analyzing the patient's historical medical image;

[0022] Step S32: The presentation effect of the color mark in the medical diagnostic image is that the doctor can choose to view the color mark on the medical diagnostic image from one angle to provide a reference for the doctor, or can choose not to view the color mark on the medical diagnostic image from another angle, thereby retaining the original content of the medical diagnostic image;

[0023] Step S33: Perform lesion analysis on the abnormal diagnosis area by introducing the expert knowledge base, and generate an abnormality report based on the lesion analysis result. The abnormality report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

[0024] According to the above technical solution, the step S31 further includes the following steps:

[0025] Step S311: The key area feature map is preferentially subjected to the multi-scale similarity analysis with the normal standard medical image, and the calculation result of the similarity between the key area and the corresponding area in the normal standard image is compared with the set threshold value. When the similarity is greater than the threshold value, it is judged as a normal area and the corresponding key area identification is released. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal area and the preliminary diagnosis abnormal area is identified with a lighter color.

[0026] Step S312: performing further multi-scale similarity analysis based on the preliminary diagnosed abnormal area and the similar medical image, wherein the similar medical image is preferentially compared with the medical image similar to the patient's condition, age, and gender, and comparing the similarity calculation result between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image with a set threshold value, and when the similarity is greater than the threshold value, it is judged as a reference abnormal area, and the lighter color mark is retained; when the similarity is less than the threshold value, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area;

[0027] Step S313: When the patient has the patient's historical medical image, the abnormal diagnosis area is compared with the corresponding area in the patient's historical medical image, and the range with changes is shaded.

[0028] According to the above technical solution, step S4 further includes: the area marked with a lighter color is determined based on whether the doctor's final diagnosis information is adopted. When the area marked with a lighter color is judged to be adopted by the doctor, the relevant medical image is optimized and iterated; when the area marked with a lighter color is judged to be adopted by the doctor, the corresponding situation is added to the relevant medical image as the standard situation.

[0029] According to the above technical solution, the system includes a patient data acquisition module, a similarity analysis module, an abnormal area identification module, and a system optimization iteration module:

[0030] The patient data acquisition module is used to acquire the patient's medical information, retrieve relevant medical images as references, and acquire the patient's medical diagnosis images;

[0031] The similarity analysis module is used to identify the key area feature map of the medical diagnosis image based on the medical inquiry information, and analyze the similarity between the related medical images according to the key areas of the key area feature map;

[0032] The abnormal area identification module is used to identify the abnormal area based on the similarity analysis result, generate different color identifications for the abnormal area and superimpose them on the medical diagnosis image, provide multi-angle medical diagnosis images, and generate an abnormality report, which is uploaded to the patient's electronic medical record for reference by the doctor;

[0033] The system optimization and iteration module is used to provide a marking window, in which the doctor can manually modify the marking content and optimize and iterate the system according to the doctor's final diagnosis information.

[0034] According to the above technical solution, the similarity analysis module includes an image feature extraction module and a similarity calculation module:

[0035] The image feature extraction module is used to convert the medical inquiry information and the medical diagnosis image into feature data;

[0036] The similarity calculation module is used to perform multi-scale similarity analysis between the key areas of the related medical images based on the weight of the key area feature map and the set threshold, wherein the area where the weight is greater than the threshold is the key area, and compare the similarity between the key area and the corresponding area in the related medical image, wherein the similarity calculation formula is:

[0037]

[0038] S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B)

[0039] Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively. ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ ABrepresents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

[0040] According to the above technical solution, the image feature extraction module includes a medical information processing submodule and a feature map construction submodule:

[0041] The medical inquiry information processing submodule is used to parse the medical inquiry information by natural language processing technology, and convert the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and use the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of the convolutional neural network CNN;

[0042] The feature map construction submodule is used to generate feature maps of multiple scales based on the medical diagnostic image by fusing the question information embedding vector through the convolutional neural network CNN, adaptively allocate attention weights on image areas of different scales through the attention weight mechanism, and strengthen the features of key areas to construct a feature map of key areas.

[0043] According to the above technical solution, the abnormal area identification module includes an abnormal area refinement module, a multi-angle presentation module, and an abnormal report generation module:

[0044] The abnormal area refinement module is used for the key area feature map to perform the multi-scale similarity analysis with the normal standard medical image in priority, and compares the calculated similarity between the key area and the corresponding area in the normal standard image with the set threshold value. When the similarity is greater than the threshold value, it is judged as a normal area and the corresponding key area identification is removed. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal area and the preliminary diagnosis abnormal area is marked with a lighter color. Further multi-scale similarity analysis is performed based on the preliminary diagnosis abnormal area and the similar medical image, wherein the similar medical image is preferentially Compare the medical images similar to the patient's condition, age, and gender, and calculate the similarity between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image, and compare them with the set threshold value; when the similarity is greater than the threshold, it is judged as a reference abnormal area, and the lighter color mark is retained; when the similarity is less than the threshold, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area; when the patient has the patient's historical medical images, compare the diagnosed abnormal area with the corresponding area in the patient's historical medical images, and shade the range with changes;

[0045] The multi-angle presentation module is used for presenting the color mark in the medical diagnostic image so that the doctor can choose to view the color mark on the medical diagnostic image at one angle to provide a reference for the doctor, or choose not to view the color mark on the medical diagnostic image at another angle, thereby retaining the original content of the medical diagnostic image;

[0046] The abnormal report generation module is used to perform lesion analysis on the abnormal diagnosis area by introducing an expert knowledge base, and generate an abnormal report from the lesion analysis result. The abnormal report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

[0047] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention can analyze the patient's condition more comprehensively by integrating the patient's medical information, medical diagnostic images, and multiple reference images (such as the patient's historical medical images, similar medical images, and normal standard medical images). The combination of the embedded vector of the medical information and the convolutional neural network enables the system to more accurately identify the key areas in the medical diagnostic images, and further confirm the abnormal areas through multi-scale similarity analysis, thereby improving the accuracy of the diagnosis, providing the original content of the medical diagnostic images, and visually displaying the abnormal areas on the images through different color labels. Doctors can observe the images from different angles, and can view the images with color labels to obtain quick diagnostic references, or turn off the color labels to view the original image content. This flexibility provides doctors with richer diagnostic basis, thereby improving the doctor's diagnostic efficiency and diagnostic accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 A schematic diagram of the module composition of a medical image diagnosis method based on computer vision provided in the second embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the module composition of a medical image diagnosis system based on computer vision provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Embodiment 1: This embodiment can be applied to the scene of diagnosing and marking pictures taken by medical instruments. The method can be performed by a medical image diagnosis method based on computer vision provided by this embodiment. Figure 1 A schematic diagram of the module composition of a medical image diagnosis method based on computer vision provided in the second embodiment of the present invention, the method specifically comprises the following steps:

[0053] Step S1: Obtain the patient's medical information, retrieve relevant medical images as reference, and obtain the patient's medical diagnosis image;

[0054] Step S2: identifying a key region feature map of the medical diagnostic image based on the medical inquiry information, and analyzing similarities between the related medical images according to key regions of the key region feature map;

[0055] Step S3: Identify abnormal areas based on the similarity analysis results, generate different color labels for the abnormal areas and superimpose them on the medical diagnostic image, provide multi-angle medical diagnostic images, and generate abnormality reports, which are uploaded to the patient's electronic medical records for the doctor's reference;

[0056] Step S4: Provide a marking window where the doctor can manually modify the marking content and optimize the system iteratively based on the doctor's final diagnosis information.

[0057] In an embodiment of the present invention, the step S1 further includes the following steps:

[0058] Step S11: The medical inquiry information includes: the patient's description of the symptoms, the doctor's preliminary diagnosis information, and the patient's historical medical records. The medical diagnostic images are images of the patient's body taken by different medical instruments;

[0059] Step S12: Based on the medical inquiry information, medical images related to the patient's condition are matched from the database as references, wherein the related medical images include: historical medical images of the patient, similar medical images, and normal standard medical images, wherein the historical medical images of the patient are used to longitudinally compare the changes in the patient's condition, the similar medical images are used as references for horizontal comparisons, and the normal standard medical images are used to assess whether the patient's current condition deviates from the normal standard. The historical medical images are retrieved according to the actual conditions of the patient, and if the patient has no historical medical images, the historical medical images of the patient will not be used as references;

[0060] Step S13: smoothing the medical diagnostic image using a Gaussian filtering algorithm, and performing contrast adjustment according to the grayscale distribution of the medical diagnostic image.

[0061] In an embodiment of the present invention, the step S2 further includes the following steps:

[0062] Step S21: parsing the medical inquiry information by natural language processing technology, converting the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and using the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of a convolutional neural network CNN;

[0063] Step S22: generating feature maps of multiple scales based on the medical diagnostic image by fusing the embedded vector of the inquiry information through the convolutional neural network CNN, adaptively allocating attention weights on image regions of different scales through an attention weight mechanism, and strengthening the features of key regions to construct a feature map of key regions;

[0064] Step S23: Based on the weight of the key area feature map and the set threshold, the area where the weight is greater than the threshold is the key area, and a multi-scale similarity analysis is performed on the related medical images between the key areas, and the similarity between the key areas and the corresponding areas in the related medical images is compared, wherein the similarity calculation formula is:

[0065]

[0066] S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B)

[0067] Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively.ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ AB represents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

[0068] In an embodiment of the present invention, step S3 further includes the following steps:

[0069] Step S31: color-marking the related medical images step by step in the order of first analyzing the normal standard medical image, then analyzing the similar medical image, and then analyzing the patient's historical medical image;

[0070] Step S32: The presentation effect of the color mark in the medical diagnostic image is that the doctor can choose to view the color mark on the medical diagnostic image from one angle to provide a reference for the doctor, or can choose not to view the color mark on the medical diagnostic image from another angle, thereby retaining the original content of the medical diagnostic image;

[0071] Step S33: Perform lesion analysis on the abnormal diagnosis area by introducing the expert knowledge base, and generate an abnormality report based on the lesion analysis result. The abnormality report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

[0072] In the embodiment of the present invention, the step S31 further includes the following steps:

[0073] Step S311: The key area feature map is preferentially subjected to the multi-scale similarity analysis with the normal standard medical image, and the calculation result of the similarity between the key area and the corresponding area in the normal standard image is compared with the set threshold value. When the similarity is greater than the threshold value, it is judged as a normal area and the corresponding key area identification is released. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal area and the preliminary diagnosis abnormal area is identified with a lighter color.

[0074] Step S312: performing further multi-scale similarity analysis based on the preliminary diagnosed abnormal area and the similar medical image, wherein the similar medical image is preferentially compared with the medical image similar to the patient's condition, age, and gender, and comparing the similarity calculation result between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image with a set threshold value, and when the similarity is greater than the threshold value, it is judged as a reference abnormal area, and the lighter color mark is retained; when the similarity is less than the threshold value, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area;

[0075] Step S313: When the patient has the patient's historical medical image, the abnormal diagnosis area is compared with the corresponding area in the patient's historical medical image, and the range with changes is shaded.

[0076] In an embodiment of the present invention, step S4 further includes: the area marked with a lighter color is determined based on whether the doctor's final diagnosis information is adopted. When the area marked with a lighter color is judged to be adopted by the doctor, the relevant medical image is optimized and iterated; when the area marked with a lighter color is judged to be adopted by the doctor, the corresponding situation is added to the relevant medical image as the standard situation.

[0077] Embodiment 2: Embodiment 2 of the present invention provides a medical image diagnosis method based on computer vision. Figure 2 A schematic diagram of the module composition of a medical image diagnosis system based on computer vision provided in the second embodiment of the present invention is shown in FIG. Figure 2 As shown, the system includes a patient data acquisition module, a similarity analysis module, an abnormal area identification module, and a system optimization iteration module:

[0078] The patient data acquisition module is used to acquire the patient's medical information, retrieve relevant medical images as references, and acquire the patient's medical diagnosis images;

[0079] The similarity analysis module is used to identify the key area feature map of the medical diagnosis image based on the medical inquiry information, and analyze the similarity between the related medical images according to the key areas of the key area feature map;

[0080] The abnormal area identification module is used to identify the abnormal area based on the similarity analysis result, generate different color identifications for the abnormal area and superimpose them on the medical diagnosis image, provide multi-angle medical diagnosis images, and generate an abnormality report, which is uploaded to the patient's electronic medical record for reference by the doctor;

[0081] The system optimization and iteration module is used to provide a marking window, in which the doctor can manually modify the marking content and optimize and iterate the system according to the doctor's final diagnosis information.

[0082] In some embodiments of the present invention, the similarity analysis module includes an image feature extraction module and a similarity calculation module:

[0083] The image feature extraction module is used to convert the medical inquiry information and the medical diagnosis image into feature data;

[0084] The similarity calculation module is used to perform multi-scale similarity analysis between the key areas of the related medical images based on the weight of the key area feature map and the set threshold, wherein the area where the weight is greater than the threshold is the key area, and compare the similarity between the key area and the corresponding area in the related medical image, wherein the similarity calculation formula is:

[0085]

[0086] S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B)

[0087] Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively. ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ AB represents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

[0088] In some embodiments of the present invention, the image feature extraction module includes a medical inquiry information processing submodule and a feature map construction submodule:

[0089] The medical inquiry information processing submodule is used to parse the medical inquiry information by natural language processing technology, and convert the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and use the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of the convolutional neural network CNN;

[0090] The feature map construction submodule is used to generate feature maps of multiple scales based on the medical diagnostic image by fusing the question information embedding vector through the convolutional neural network CNN, adaptively allocate attention weights on image areas of different scales through the attention weight mechanism, and strengthen the features of key areas to construct a feature map of key areas.

[0091] In some embodiments of the present invention, the abnormal area identification module includes an abnormal area refinement module, a multi-angle presentation module, and an abnormal report generation module:

[0092] The abnormal area refinement module is used for the key area feature map to perform the multi-scale similarity analysis with the normal standard medical image in priority, and compares the calculated similarity between the key area and the corresponding area in the normal standard image with the set threshold value. When the similarity is greater than the threshold value, it is judged as a normal area and the corresponding key area identification is removed. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal area and the preliminary diagnosis abnormal area is marked with a lighter color. Further multi-scale similarity analysis is performed based on the preliminary diagnosis abnormal area and the similar medical image, wherein the similar medical image is preferentially Compare the medical images similar to the patient's condition, age, and gender, and calculate the similarity between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image, and compare them with the set threshold value; when the similarity is greater than the threshold, it is judged as a reference abnormal area, and the lighter color mark is retained; when the similarity is less than the threshold, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area; when the patient has the patient's historical medical images, compare the diagnosed abnormal area with the corresponding area in the patient's historical medical images, and shade the range with changes;

[0093] The multi-angle presentation module is used for presenting the color mark in the medical diagnostic image so that the doctor can choose to view the color mark on the medical diagnostic image at one angle to provide a reference for the doctor, or choose not to view the color mark on the medical diagnostic image at another angle, thereby retaining the original content of the medical diagnostic image;

[0094] The abnormal report generation module is used to perform lesion analysis on the abnormal diagnosis area by introducing an expert knowledge base, and generate an abnormal report from the lesion analysis result. The abnormal report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

[0095] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0096] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A medical image diagnosis method based on computer vision, The method comprises the following steps: Step S1: Obtain the patient's medical information, retrieve relevant medical images as reference, and obtain the patient's medical diagnosis image; Step S2: identifying a key region feature map of the medical diagnostic image based on the medical inquiry information, and analyzing similarities between the related medical images according to key regions of the key region feature map; Step S3: Identify abnormal areas based on the similarity analysis results, generate different color labels for the abnormal areas and superimpose them on the medical diagnostic image, provide multi-angle medical diagnostic images, and generate abnormality reports, which are uploaded to the patient's electronic medical records for the doctor's reference; Step S4: Provide a marking window where the doctor can manually modify the marking content and optimize the system iteratively based on the doctor's final diagnosis information.

2. The medical image diagnosis method based on computer vision according to claim 1, characterized in that: The step S1 further comprises the following steps: Step S11: The medical inquiry information includes: the patient's description of the symptoms, the doctor's preliminary diagnosis information, and the patient's historical medical records. The medical diagnostic images are images of the patient's body taken by different medical instruments; Step S12: Based on the medical inquiry information, medical images related to the patient's condition are matched from the database as references, wherein the related medical images include: historical medical images of the patient, similar medical images, and normal standard medical images, wherein the historical medical images of the patient are used to longitudinally compare the changes in the patient's condition, the similar medical images are used as references for horizontal comparisons, and the normal standard medical images are used to assess whether the patient's current condition deviates from the normal standard. The historical medical images are retrieved according to the actual conditions of the patient, and if the patient has no historical medical images, the historical medical images of the patient will not be used as references; Step S13: Smoothing the medical diagnostic image using a Gaussian filtering algorithm, and performing contrast adjustment according to the grayscale distribution of the medical diagnostic image.

3. The medical image diagnosis method based on computer vision according to claim 2, characterized in that: The step S2 further comprises the following steps: Step S21: parsing the medical inquiry information by natural language processing technology, converting the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and using the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of a convolutional neural network CNN; Step S22: generating feature maps of multiple scales based on the medical diagnostic image by fusing the embedded vector of the inquiry information through the convolutional neural network CNN, adaptively allocating attention weights on image regions of different scales through an attention weight mechanism, and strengthening the features of key regions to construct a feature map of key regions; Step S23: Based on the weight of the key area feature map and the set threshold, the area where the weight is greater than the threshold is the key area, and a multi-scale similarity analysis is performed on the related medical images between the key areas, and the similarity between the key areas and the corresponding areas in the related medical images is compared, wherein the similarity calculation formula is: S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B) Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively. ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ AB represents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

4. The medical image diagnosis method based on computer vision according to claim 3, characterized in that: The step S3 further comprises the following steps: Step S31: color-marking the related medical images step by step in the order of first analyzing the normal standard medical image, then analyzing the similar medical image, and then analyzing the patient's historical medical image; Step S32: The presentation effect of the color mark in the medical diagnostic image is that the doctor can choose to view the color mark on the medical diagnostic image from one angle to provide a reference for the doctor, or can choose not to view the color mark on the medical diagnostic image from another angle, thereby retaining the original content of the medical diagnostic image; Step S33: Perform lesion analysis on the abnormal diagnosis area by introducing the expert knowledge base, and generate an abnormality report based on the lesion analysis result. The abnormality report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

5. The method for medical image diagnosis based on computer vision according to claim 4, characterized in that: The step S31 further comprises the following steps: Step S311: The key area feature map is preferentially subjected to the multi-scale similarity analysis with the normal standard medical image, and the calculation result of the similarity between the key area and the corresponding area in the normal standard image is compared with the set threshold value. When the similarity is greater than the threshold value, it is judged as a normal area and the corresponding key area identification is released. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal area and the preliminary diagnosis abnormal area is identified with a lighter color. Step S312: performing further multi-scale similarity analysis based on the preliminary diagnosed abnormal area and the similar medical image, wherein the similar medical image is preferentially compared with the medical image similar to the patient's condition, age, and gender, and comparing the similarity calculation result between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image with a set threshold value, and when the similarity is greater than the threshold value, it is judged as a reference abnormal area, and the lighter color mark is retained; when the similarity is less than the threshold value, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area; Step S313: When the patient has the patient's historical medical image, the abnormal diagnosis area is compared with the corresponding area in the patient's historical medical image, and the range with changes is shaded.

6. The medical image diagnosis method based on computer vision according to claim 5, characterized in that: The step S4 further includes: the area marked with the lighter color is determined based on whether the doctor's final diagnosis information is adopted. When the doctor judges that the area marked with the lighter color is adopted, the relevant medical image is optimized and iterated; when the doctor judges that the area marked with the lighter color is adopted, the corresponding situation is added as the standard situation in the relevant medical image.

7. A medical image diagnosis system based on computer vision, characterized in that: The system includes a patient data acquisition module, a similarity analysis module, an abnormal area identification module, and a system optimization iteration module: The patient data acquisition module is used to acquire the patient's medical information, retrieve relevant medical images as references, and acquire the patient's medical diagnosis images; The similarity analysis module is used to identify the key area feature map of the medical diagnosis image based on the medical inquiry information, and analyze the similarity between the related medical images according to the key areas of the key area feature map; The abnormal area identification module is used to identify the abnormal area based on the similarity analysis result, generate different color identifications for the abnormal area and superimpose them on the medical diagnosis image, provide multi-angle medical diagnosis images, and generate an abnormality report, which is uploaded to the patient's electronic medical record for reference by the doctor; The system optimization and iteration module is used to provide a marking window, in which the doctor can manually modify the marking content and optimize and iterate the system according to the doctor's final diagnosis information.

8. The medical image diagnosis system based on computer vision according to claim 7, characterized in that: The similarity analysis module includes an image feature extraction module and a similarity calculation module: The image feature extraction module is used to convert the medical inquiry information and the medical diagnosis image into feature data; The similarity calculation module is used to perform multi-scale similarity analysis between the key areas of the related medical images based on the weight of the key area feature map and the set threshold, wherein the area where the weight is greater than the threshold is the key area, and compare the similarity between the key area and the corresponding area in the related medical image, wherein the similarity calculation formula is: S(A,B)=αS gray (A,B)+βS gray (A,B)+γS structure (A,B) Where A represents the key area, B represents the corresponding area in the relevant medical image, and S gray (A, B) represents the grayscale similarity, M and N represent the number of rows and columns of the corresponding region, respectively. ij and B ij Represents the grayscale value of region A and region B at position (i, j), S texture (A, B) represents texture similarity, L represents the grayscale level, P a (k) and P b (k) represent the grayscale histogram of area A and B respectively, S structure (A, B) represents the structural similarity, μ A and μ B Represent the mean of regions A and B, σ A and σ B denote the variance of regions A and B respectively, σ AB represents the covariance of regions A and B, C1 and C2 represent constants, S(A, B) represents the comprehensive similarity of the corresponding regions, and α, β, and γ represent weight coefficients.

9. The medical image diagnosis system based on computer vision according to claim 8, characterized in that: The image feature extraction module includes a medical information processing submodule and a feature map construction submodule: The medical inquiry information processing submodule is used to parse the medical inquiry information by natural language processing technology, and convert the medical inquiry information into a vector representation in a high-order space by constructing a medical inquiry information embedding vector, wherein the medical inquiry information embedding vector can capture the semantic features and contextual relationship of the medical inquiry information, and use the medical inquiry information embedding vector as prior knowledge to initialize the feature extraction layer of the convolutional neural network CNN; The feature map construction submodule is used to generate feature maps of multiple scales based on the medical diagnostic image by fusing the question information embedding vector through the convolutional neural network CNN, adaptively allocate attention weights on image areas of different scales through the attention weight mechanism, and strengthen the features of key areas to construct a feature map of key areas.

10. The medical image diagnosis system based on computer vision according to claim 9, characterized in that: The abnormal area identification module includes an abnormal area refinement module, a multi-angle presentation module, and an abnormal report generation module: The abnormal region refinement module is used to perform the multi-scale similarity analysis on the key region feature map in priority with the normal standard medical image, and compare the calculated similarity between the key region and the corresponding region in the normal standard image with a set threshold value. When the similarity is greater than the threshold value, it is judged as a normal region and the corresponding key region identification is removed. When the similarity is less than the threshold value, it is judged as a preliminary diagnosis abnormal region and the preliminary diagnosis abnormal region is identified with a lighter color. A further multi-scale similarity analysis is performed based on the preliminary diagnosed abnormal area and the similar medical image, wherein the similar medical image is preferentially compared with the medical image with similar condition, age and gender of the patient, and the similarity calculation result between the preliminary diagnosed abnormal area and the corresponding area in the similar medical image is compared with a set threshold value, when the similarity is greater than the threshold value, it is judged as a reference abnormal area, and the lighter color mark is retained, and when the similarity is less than the threshold value, it is judged as a diagnosed abnormal area, and a darker color mark is superimposed on the lighter color mark for the diagnosed abnormal area; when the patient has the patient's historical medical image, the diagnosed abnormal area is compared with the corresponding area in the patient's historical medical image, and the range with changes is shaded; The multi-angle presentation module is used for presenting the color mark in the medical diagnostic image so that the doctor can choose to view the color mark on the medical diagnostic image at one angle to provide a reference for the doctor, or choose not to view the color mark on the medical diagnostic image at another angle, thereby retaining the original content of the medical diagnostic image; The abnormal report generation module is used to perform lesion analysis on the abnormal diagnosis area by introducing an expert knowledge base, and generate an abnormal report from the lesion analysis result. The abnormal report includes: the medical diagnostic image, the enlarged image of the abnormal diagnosis area and the corresponding lesion analysis, the corresponding detection or treatment plan, and similar case examples.

Images