Digestive tract lesion analysis method and system based on image recognition

Abstract

The application discloses a digestive tract lesion analysis method and system based on image recognition, belongs to the technical field of image recognition analysis, and utilizes an image acquisition module to acquire images of the digestive tract of a current patient, transmits data processed by an image processing module, including segmentation, extraction, recognition and statistics, to the image acquisition module and a lesion calculation module, utilizes the lesion calculation module to sequentially calculate and output a preliminary evaluation value WP, a related part influence value G and a comprehensive lesion risk value BF, displays results based on the comprehensive lesion risk value BF by using a result display module to perform analysis, after achieving multi-dimensional comprehensive evaluation of lesion characteristics, the application incorporates related part information and comprehensively evaluates lesion influence, thereby comprehensively considering multiple factors, improving the accuracy and comprehensiveness of analysis, and finally integrates factors in spatial dimensions and time dimensions to comprehensively and accurately evaluate the comprehensive risk degree of the digestive tract lesion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition analysis, in particular to a digestive tract lesion analysis method and system based on image recognition. BACKGROUND

[0002] Digestive tract diseases are common health problems, and early and accurate diagnosis is crucial for disease treatment and prognosis. Traditional diagnosis of digestive tract lesions mainly relies on the experience of doctors, which is mainly through direct observation by endoscopy and analysis of pathological sections. However, this method has certain subjectivity and limitations, and the diagnosis results of different doctors are different, and early small lesions are easy to be missed. With the development of image processing and artificial intelligence technology, the digestive tract lesion analysis method and system based on image recognition has gradually become a research hotspot.

[0003] However, most of the existing technologies only analyze a certain feature of the lesion, and cannot comprehensively evaluate the lesion from different visual feature angles, thus missing important diagnostic clues and affecting the accuracy of diagnosis. Secondly, in analyzing the lesion, the existing method pays little attention to the relationship between the lesion site and other related sites in the digestive tract, and cannot accurately grasp the overall situation of the lesion in the entire digestive tract system. In addition, the existing image recognition technology is based on static image analysis, and ignores the dynamic changes of the lesion, making it difficult to accurately predict the potential risks of the lesion, which is not conducive to the development of a reasonable treatment plan. SUMMARY

[0004] The technical problem to be solved by the present application is that the existing technology lacks multi-dimensional comprehensive analysis, ignores the information of related sites, and does not fully consider the dynamic changes of the lesion. Therefore, we propose a digestive tract lesion analysis method and system based on image recognition.

[0005] The technical solution is as follows:

[0006] Step 1, use the image acquisition module to acquire the image of the patient's digestive tract and transmit it to the image processing module;

[0007] Step 2, use the image processing module to separate the lesion site, the related site of the lesion and the normal tissue site in the image, extract the color features of the lesion site and the related site, identify the area of the lesion site, the related site of the lesion and the normal tissue site, and count the number of related site lesions;

[0008] Step 3, transmit the data of the image processing module segmentation, extraction, identification and statistics to the image acquisition module and lesion calculation module for storage and calculation;

[0009] Step 4, calculating the preliminary evaluation value WP, the associated part influence value G and the comprehensive lesion risk value BF in turn by using the lesion calculation module;

[0010] Step 5, based on the comprehensive lesion risk value BF, and using the result display module to display the results for analysis;

[0011] Wherein, the lesion calculation module includes a lesion preliminary evaluation unit, an associated part influence unit, and a fusion space-time factor evaluation unit.

[0012] Preferably, the image processing module includes a region segmentation unit, a color analysis unit and a lesion recognition unit;

[0013] The region segmentation unit performs segmentation operation based on machine learning segmentation algorithm;

[0014] The color analysis unit performs color feature extraction operation based on color space conversion and feature extraction algorithm;

[0015] The lesion recognition unit performs recognition and statistics operation based on area calculation and quantity statistics.

[0016] Preferably, the calculation formula of the lesion preliminary evaluation unit is as follows:

[0017] ;

[0018] Wherein:

[0019] WP is the preliminary evaluation value, WP is the multiplication of the current lesion in the spatial dimension and the color dimension Y feature, to get a value that can reflect the overall characteristics of the lesion;

[0020] BM is the lesion area, BM reflects the size of the space occupied by the current lesion site in the digestive tract;

[0021] BM0 is the average lesion area, BM0 is calculated by image data statistics to reflect the average area size of the same part of the normal digestive tract and the lesion area BM of a large number of healthy people;

[0022] Y is the lesion color difference value, Y reflects the deviation degree of the current lesion site in the color feature from the normal tissue site in the digestive tract;

[0023] The greater the values of the lesion area BM and the lesion color difference value Y, the greater the value of the preliminary evaluation value WP;

[0024] The smaller the values of the lesion area BM and the lesion color difference value Y, the smaller the value of the preliminary evaluation value WP.

[0025] Preferably, the calculation formula of the associated site influence unit is as follows:

[0026] ;

[0027] Wherein:

[0028] G is the associated site influence value;

[0029] S is the number of associated site lesions, S is the number of lesions associated with the current lesion site and belonging to other digestive tract sites in the digestive tract, which is counted by image recognition;

[0030] S0 is the average number of lesions, S0 is the average number of lesions generated by a large number of sick people through data statistics;

[0031] d is the distance coefficient, d is in the range of [0, 1], and the specific value is as follows:

[0032] If the distance between other associated sites and the current lesion site is close, that is, close to the adjacent distance and belonging to the adjacent relationship, then d is close to 1 and far from 0;

[0033] If the distance between other associated sites and the current lesion site is far, that is, far from the adjacent relationship, then d is close to 0 and far from 1;

[0034] The preliminary evaluation value WP mainly focuses on the characteristics of the current lesion site itself, and the associated site disease condition is compared with the overall average situation, and the sum of the two is The calculation part is to measure the overall condition of the lesion by comprehensively considering the information of the main lesion site and the associated site.

[0035] Preferably, after the image is converted from the RGB color space to the HSV color space, the average hue value of the current lesion area and the normal tissue site is calculated, including the average hue value of hue, saturation and brightness, and then the Euclidean distance method is used to calculate the deviation degree of the average hue value of the current lesion area and the normal tissue site in the HSV color space, that is, the lesion color difference value Y.

[0036] Preferably, the calculation formula of the fusion space-time factor evaluation unit is as follows:

[0037] ;

[0038] Wherein:

[0039] BF is the comprehensive lesion risk value;

[0040] V is the lesion development speed, V reflects the development of the current and associated site lesions over time.

[0041] ​Preferably, the lesion development speed V is calculated by continuously collecting the identified image data, and the sum of the area change rates of the current and associated parts in the lesion observation time.

[0042] The image recognition-based digestive tract lesion analysis system comprises an image acquisition module, an image processing module, a lesion calculation module and a result display module.

[0043] The image acquisition module is responsible for acquiring the digestive tract image of the current patient, and storing the data processed by the image processing module and the lesion calculation module.

[0044] The image processing module is responsible for image segmentation, extraction, identification and statistics.

[0045] The lesion calculation module is responsible for calculating the preliminary evaluation value WP, the associated part influence value G and the comprehensive lesion risk value BF.

[0046] The result display module is responsible for displaying the result of the comprehensive lesion risk value BF.

[0047] The technical effects and advantages of the present application are as follows:

[0048] In the present application, the lesion preliminary evaluation unit combines the lesion area BM with the lesion average area BM0 and the lesion color difference value Y to preliminarily evaluate the lesion from two dimensions of spatial size and color characteristics. This multi-dimensional comprehensive analysis method can more comprehensively reflect the characteristics of the lesion, make up for the shortcomings of the prior art which only focuses on a single feature, and greatly improve the accuracy of the preliminary evaluation of the lesion.

[0049] In the present application, the associated part influence unit comprehensively evaluates the influence of the associated part on the current lesion by introducing the associated part lesion number S, the average lesion number S0 and the distance coefficient d based on the preliminary evaluation value WP. This correlation analysis method can understand the distribution and interaction of the lesion in the entire digestive tract system, avoid viewing the lesion in isolation, and thus more comprehensively and accurately judge the severity and development trend of the lesion.

[0050] In the present application, the spatio-temporal factor fusion evaluation unit combines the associated part influence value C with the lesion development speed V, balances the influence of spatio-temporal factors through root operation, and calculates the comprehensive lesion risk value BF. This innovation fully considers the dynamic changes of the lesion and can more accurately predict the potential risk of the lesion. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The method flowchart of the present digestive tract lesion analysis method is shown in the figure.

[0052] Figure 2 The overall structure schematic diagram of the present digestive tract lesion analysis system is shown in the figure.

[0053] Figure 3 For the segmentation of the digestive tract image in the present application. DETAILED DESCRIPTION

[0054] The present application will now be further described in detail in conjunction with the accompanying drawings and preferred embodiments.

[0055] Reference Figures 1 to 3 As shown, the present application provides a technical solution: an image recognition-based digestive tract lesion analysis method, which has the following specific implementation steps:

[0056] Step 1, using an image acquisition module to acquire images of the digestive tract of the current patient and transmit them to an image processing module;

[0057] Step 2, using the image processing module to segment the lesion site, the site associated with the lesion, and the normal tissue site in the image, extract the color features of the lesion site and the associated site, identify the areas of the lesion site, the site associated with the lesion, and the normal tissue site, and count the number of associated site lesions;

[0058] Step 3, transmitting the data segmented, extracted, identified, and counted by the image processing module to the image acquisition module and the lesion calculation module for storage and calculation;

[0059] Step 4, using the lesion calculation module to sequentially calculate the preliminary evaluation value WP, the associated site influence value G, and the comprehensive lesion risk value BF;

[0060] Step 5, based on the comprehensive lesion risk value BF, and using the result display module to display the results for analysis;

[0061] The lesion calculation module includes a lesion preliminary evaluation unit, an associated site influence unit, and a spatiotemporal factor fusion evaluation unit.

[0062] The image processing module includes a region segmentation unit, a color analysis unit, and a lesion identification unit.

[0063] The region segmentation unit performs segmentation operations based on machine learning segmentation algorithms.

[0064] The segmentation operation can use Mask-RCNN; another simpler way is to use the pixellib library under the TensorFlow framework for segmentation processing.

[0065] The color analysis unit performs color feature extraction operation based on color space conversion and feature extraction algorithm; in the present application, the output layer excitation function can be adjusted to a linear function to directly process the original pixel input of RGB three channels. Compared with the traditional Sigmoid function, this method can preserve the color information in the range of [-∞, +∞], automatically learn the color distribution law by using the convolution layer, and the reconstructed feature map of the output layer can reflect the color feature, so as to realize the comparison between the lesion area and the surrounding normal tissue, and thus perform the digestive tract lesion analysis.

[0066] In addition, in another implementation process, the color histogram method can be selected to realize color feature extraction, which is prior art and will not be repeated.

[0067] The lesion recognition unit performs recognition and statistics operation based on area calculation and quantity statistics.

[0068] Referring to FIGS. 1 to 3, in the present embodiment, the calculation formula of the lesion preliminary evaluation unit is as follows: Figure 1 Figure 3

[0069] ;

[0070] WP is the preliminary evaluation value, WP is the multiplication of the current lesion in the spatial dimension Y and the color dimension Y to obtain a value that can preliminarily reflect the overall characteristics of the lesion;

[0071] WP is the preliminary evaluation value, WP is the multiplication of the current lesion in the spatial dimension Y and the color dimension Y to obtain a value that can preliminarily reflect the overall characteristics of the lesion; BM is the lesion area, which reflects the size of the space occupied by the current lesion in the digestive tract;

[0072] BM0 is the average area of the lesion, which is calculated by image data statistics to reflect the average area size of the same part as the lesion area BM in the normal digestive tract of a large number of healthy people;

[0073] Y is the lesion color difference value, which reflects the deviation degree of the current lesion part from the normal tissue part in the color feature;

[0074] After the image is converted from the RGB color space to the HSV color space, the average hue value of the current lesion area and the normal tissue part is calculated, including the average hue value of the hue, saturation and brightness, and then the Euclidean distance method is used to calculate the deviation degree of the average hue value of the current lesion area and the normal tissue part in the HSV color space, i.e. the lesion color difference value Y;

[0075] The greater the values of the lesion area BM and the lesion color difference value Y, the greater the value of the preliminary evaluation value WP;

[0076] The greater the values of the lesion area BM and the lesion color difference value Y, the greater the value of the preliminary evaluation value WP;

[0077] ​​The smaller the value of the lesion area BM and the lesion color difference value Y, the smaller the value of the preliminary evaluation value WP.

[0078] In this embodiment, The calculation part can calculate the size of the lesion relative to the normal condition, which The result value of the calculation part is one of the important factors for preliminary evaluation of the lesion, which provides basic information of the lesion from the spatial dimension, and is an important part of the subsequent comprehensive evaluation. The lesion color difference value Y quantifies the difference between the lesion area and the surrounding normal tissue in color. Through image color analysis technology, the color characteristics of the lesion area and the surrounding normal tissue are compared, and the difference value between the two is calculated by using the algorithm. This value does not depend on other factors of the area, but specifically reflects the characteristics of the color. Therefore, the lesion color difference value Y, as an independent characteristic index, combined with the calculation part, supplements information from different visual feature angles, making the preliminary evaluation more comprehensive. The lesion preliminary evaluation unit preliminarily evaluates the condition of the digestive tract lesion from two aspects of the spatial size and color characteristics of the lesion, providing basic data for subsequent in-depth analysis. The reason for multiplication is that the two factors independently affect the preliminary evaluation of the lesion, and multiplication can consider the effects of both, obtaining a value that can preliminarily reflect the overall characteristics of the lesion.

[0079] The preliminary evaluation value WP provides a basic data for subsequent lesion analysis, and this value can help the system quickly understand the general situation of the lesion and determine whether the lesion has further value for in-depth analysis. The subsequent associated site influence unit and the spatio-temporal factor evaluation unit are based on this to carry out more complex analysis, to ensure the accuracy and reliability of the entire lesion analysis process.

[0080] Referring to Figure 1 and Figure 3 , in this embodiment, the calculation formula of the associated site influence unit is as follows:

[0081] ;

[0082] Wherein:

[0083] G is the associated site influence value;

[0084] S is the number of lesions in the associated site, S is the number of lesions in the associated site and the same digestive tract site related to the current lesion through image recognition statistics;

[0085] S0 is the average number of lesions, S0 is the average number of lesions in a large number of people with associated site lesions through data statistics;

[0086] ​d is a distance coefficient, d is in the range of [0, 1], and the specific value is as follows:

[0087] If the distance between the other associated parts and the current lesion site is close, that is, close to the adjacent distance and belongs to the adjacent relationship, then d is close to 1 and far from 0;

[0088] If the distance between the other associated parts and the current lesion site is far, that is, far from the adjacent relationship, then d is close to 0 and far from 1;

[0089] The preliminary evaluation value WP mainly focuses on the characteristics of the current lesion site itself, and reflects the comparison between the lesion condition of the associated part and the overall average, and the sum of the two is The calculation part integrates the information of the main lesion site and the associated part to measure the overall condition of the lesion.

[0090] In this embodiment, In the calculation part, The ratio of the calculation part is greater than 1, indicating that the associated part lesion number S is higher than the average level, and less than 1, indicating that it is lower than the average level. From the perspective of the associated part lesion number S, this ratio provides information for comparison with the average situation, helping to judge the abnormality degree of the lesion of the associated part, which is an important factor affecting the influence degree of the associated part on the current lesion, and The calculation part is added to the preliminary evaluation value WP because they describe the lesion-related information from different aspects, and the addition can comprehensively integrate these two parts of content, and then obtain a more comprehensive value to reflect the overall characteristics of the current lesion and the lesion condition of the associated part, and then the distance coefficient d of the associated part is multiplied by the value obtained by the calculation part The multiplication of the value obtained by the calculation part can take the distance factor into account, so that the final associated part influence value G can more accurately reflect the comprehensive influence of the associated part on the current lesion under the consideration of the distance. Such a calculation method can comprehensively and hierarchically consider multiple influencing factors, and obtain a value that can accurately reflect the comprehensive influence of the associated part on the current lesion through mathematical operation;

[0091] The associated part influence unit introduces the associated part lesion number S, the average lesion number S0 of the associated part and the distance coefficient d on the basis of the preliminary evaluation value of the lesion preliminary evaluation unit, can comprehensively evaluate the influence of the associated part on the current lesion, avoid isolated consideration of a single lesion, and more conform to the actual situation of mutual influence and transmission of the digestive tract disease, and by combining the preliminary evaluation value with the associated part related factors, the associated part influence unit realizes the comprehensive analysis from the main lesion site to the associated part. The fusion of multiple factors makes the evaluation of the lesion no longer limited to the single lesion itself, but considers the interaction between the lesions in the entire digestive tract system.

[0092] Referring to Figure 1 and Figure 3 , in the embodiment, the calculation formula of the spatio-temporal factor fusion evaluation unit is as follows:

[0093] ;

[0094] Wherein:

[0095] BF is the comprehensive lesion risk value;

[0096] V is the lesion development speed, which reflects the development of the current and associated parts lesions over time.

[0097] In the embodiment, The calculation part comprehensively considers the changes of the lesion in the spatial and temporal dimensions to obtain a value reflecting the spatio-temporal comprehensive change trend of the lesion. The multiplication is because the risk assessment of the lesion needs to consider the current situation in space and the change trend in time. The multiplication operation can combine the key factors of the two different dimensions to reflect the comprehensive dynamic characteristics of the lesion in space and time. Since a larger value will be obtained, the square root operation can adjust it to a more appropriate range, which is convenient for comprehensive calculation and comparison with other factors. At the same time, the influence of the lesion in the spatial and temporal dimensions is not simply linearly superimposed. The square root operation can more reasonably integrate the factors of the two dimensions to avoid the influence of some factors being overemphasized, so that the result can more accurately reflect the actual risk degree of the lesion. Then, the spatio-temporal comprehensive influence value adjusted by the square root is added to the associated part influence value G to comprehensively calculate the comprehensive risk value of the digestive tract lesion. Specifically, the associated part influence value G itself contains multiple aspects of information of the lesion in the spatial dimension, while the spatio-temporal comprehensive influence value adjusted by the square root more reasonably integrates the spatio-temporal factors. The addition of the two can integrate the basic information in the spatial dimension and the spatio-temporal comprehensive influence information to obtain a value that comprehensively reflects the risk of the digestive tract lesion.

[0098] The spatio-temporal factor fusion evaluation unit combines the associated part influence value G with the lesion development speed to comprehensively consider the changes of the lesion in the spatial and temporal dimensions, and then calculates the comprehensive lesion risk value BF. The comprehensive lesion risk value is a quantitative index, which is helpful to more accurately judge the severity and development trend of the disease.

[0099] Referring to Figure 1 and Figure 3 Figure 1 Figure 3 , in the embodiment, the lesion development speed V is calculated by continuously collecting and identifying the image data to calculate the sum of the area change rates of the current and associated parts within the lesion observation time.

[0100] In the embodiment, the sum of the area change rates can track the current lesion site and the lesion development of the associated site in real time. The gastrointestinal lesion is often a dynamic process. It is difficult to fully understand the development trend of the lesion by simply relying on single image analysis. By continuously collecting images and calculating the area change rate, the system can timely capture the slight changes of the lesion area. The subtle changes can be clearly presented by the area change rate, which is helpful for early detection of signs of lesion deterioration. Different gastrointestinal lesions and the same lesion in different parts have different development speeds. Therefore, the sum of the area change rates of the current site and the associated site can comprehensively reflect the differences.

[0101] There is a close physiological relationship between each part of the digestive tract, and the lesions also affect each other. The sum of the area change rates emphasizes the relevance between the current lesion site and the associated site, and by comprehensively considering the area change rates of both, the system can more comprehensively analyze the propagation and evolution rules of the lesion in the entire digestive system.

[0102] It should be noted that any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall also be within the protection scope of the present application.

Claims

1. An image recognition-based digestive tract lesion analysis method, characterized by: The specific implementation steps are as follows: Step 1: Use the image acquisition module to collect images of the current patient's digestive tract and transmit them to the image processing module; Step 2: Use the image processing module to segment the lesion site, associated site, and normal tissue site in the image, extract the color features of the lesion site and associated site, identify the area of the lesion site, associated site, and normal tissue site, and count the number of associated site lesions; Step 3: Transfer the data segmented, extracted, identified, and counted by the image processing module to the image acquisition module and lesion calculation module for storage and calculation; Step 4: Use the lesion calculation module to calculate the preliminary evaluation value WP, the associated site impact value G, and the comprehensive lesion risk value BF in sequence; Step 5: Based on the comprehensive lesion risk value BF, use the result display module to display the results for analysis; The lesion calculation module includes a lesion preliminary evaluation unit, an associated site impact unit, and a spatio-temporal factor fusion evaluation unit; The calculation formula of the lesion preliminary evaluation unit is as follows: ； Where: WP is a preliminary evaluation value, WP is the multiplication of the current lesion in the spatial dimension and color dimension Y, to obtain a value that can preliminarily reflect the overall characteristics of the lesion; BM is the lesion area, which reflects the spatial range of the current lesion site in the digestive tract; BM0 is the average lesion area, which is calculated by image data to reflect the average area of the same part of the normal digestive tract of a large number of healthy people; Y is the lesion color difference value, which reflects the deviation of the current lesion site from the normal tissue site in the digestive tract in terms of color characteristics; The larger the values of lesion area BM and lesion color difference value Y, the larger the preliminary evaluation value WP; The smaller the values of lesion area BM and lesion color difference value Y, the smaller the preliminary evaluation value WP. 2.The image recognition-based digestive tract lesion analysis method of claim 1, wherein: The image processing module includes a region segmentation unit, a color analysis unit, and a lesion recognition unit; The region segmentation unit performs segmentation operations based on machine learning segmentation algorithms; The color analysis unit performs color feature extraction operations based on color space conversion and feature extraction algorithms; The lesion recognition unit performs recognition and statistical operations based on area calculation and quantity statistics.

3. The image recognition-based digestive tract lesion analysis method according to claim 2, characterized by: The calculation formula of the associated site impact unit is as follows: ； Where: G is the associated site impact value; S is the number of associated site lesions, which is calculated by image recognition and statistics of the number of lesions associated with the current lesion site and other digestive tract sites in the digestive tract; S0 is the average number of lesions, which is calculated by data statistics of the average number of associated site lesions in a large number of patients; d is the distance coefficient, with a value range of [0, 1]; The preliminary assessment value WP focuses on the characteristics of the current lesion site itself, and embodies the comparison of the lesion condition of the associated site with the overall average condition, and the sum of both, i.e. .

4. The image recognition-based digestive tract lesion analysis method of claim 3, characterized by: After converting the image from the RGB color space to the HSV color space, the average hue value of the current lesion area and the normal tissue site is calculated, including the average hue value of hue, saturation, and brightness. Then, the Euclidean distance method is used to calculate the deviation of the average hue value of the current lesion area and the normal tissue site in the HSV color space, i.e., the lesion color difference value Y.

5. The image recognition-based digestive tract lesion analysis method of claim 4, characterized by: The calculation formula of the spatio-temporal factor fusion evaluation unit is as follows: ； Where: BF is the comprehensive lesion risk value; V is the lesion development speed, which reflects the development of the current and associated parts over time. 6.The image recognition-based digestive tract lesion analysis method of claim 5, wherein: The lesion development speed V is calculated by continuously collecting the identified image data to calculate the area change rate of the current and associated parts within the lesion observation time.

7. An image recognition-based digestive tract lesion analysis system that executes the image recognition-based digestive tract lesion analysis method according to any one of claims 1 to 6, characterized by: It includes an image acquisition module, an image processing module, a lesion calculation module, and a result display module. The image acquisition module is responsible for collecting the current patient's digestive tract images and storing the data processed by the image processing module and the lesion calculation module. The image processing module is responsible for image segmentation, extraction, identification, and statistics. The lesion calculation module is responsible for calculating the preliminary evaluation value WP, the associated part influence value G, and the comprehensive lesion risk value BF. The result display module is responsible for displaying the results of the comprehensive lesion risk value BF.

Images