Digestive tract lesion analysis method and system based on image recognition

Abstract

The invention discloses an alimentary canal lesion analysis method and system based on image recognition, and belongs to the technical field of image recognition analys.The alimentary canal lesion analysis method comprises the steps that an image collection module is used for collecting images of an alimentary canal of a current patient, and data segmented, extracted, recognized and counted by an image processing module is transmitted to the image collection module and a lesion calculation module; a lesion calculation module is used for sequentially calculating and outputting a preliminary evaluation value WP, an associated part influence value G and a comprehensive lesion risk value BF, and based on the comprehensive lesion risk value BF, a result display module is used for performing result display and analysis. According to the method, multiple factors are integrated, the accuracy and comprehensiveness of analysis are improved, finally, the factors of the lesion in the space dimension and the time dimension are integrated, and the comprehensive risk degree of the lesion of the digestive tract is comprehensively and accurately evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and analysis, and particularly to a method and system for analyzing digestive tract lesions based on image recognition. Background Art

[0002] Digestive tract diseases are common health problems. Early and accurate diagnosis is crucial for the treatment and prognosis of diseases. Traditional diagnosis of digestive tract lesions mainly relies on doctors' experience, mainly through direct observation by endoscopy and analysis with the aid of pathological sections. However, this method has certain subjectivity and limitations. Diagnostic results vary among different doctors, and early and minor lesions are likely to be missed. With the development of image processing and artificial intelligence technologies, methods and systems for analyzing digestive tract lesions based on image recognition have gradually become a research hotspot.

[0003] However, most of the existing technologies only analyze a certain feature of the lesion and fail to comprehensively evaluate the lesion from different visual feature perspectives. Therefore, important diagnostic clues will be missed, affecting the diagnostic accuracy. Secondly, when analyzing the lesion, the existing methods rarely pay attention to the connection between the location of the lesion and other related parts of the digestive tract, and cannot accurately grasp the overall situation of the lesion in the entire digestive tract system. In addition, most of the existing image recognition technologies analyze based on static images, ignoring the dynamic changes of the lesion, and thus it is difficult to accurately predict the potential risks of the lesion, which is not conducive to formulating a reasonable treatment plan. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that there are disadvantages in the prior art such as the lack of multi-dimensional comprehensive analysis, ignoring the information of related parts, and not fully considering the dynamic changes of the lesion. For this reason, we propose a method and system for analyzing digestive tract lesions based on image recognition.

[0005] The technical solution is mainly as follows: A method for analyzing digestive tract lesions based on image recognition, and the specific implementation steps are as follows: Step 1: Use an image acquisition module to acquire images of the digestive tract of the current patient and transmit them to an image processing module; Step 2: Use the image processing module to separate the lesion part, the part associated with the lesion, and the normal tissue part in the image, extract the color features of the lesion part and the associated part, identify the areas of the lesion part, the part associated with the lesion, and the normal tissue part, and count the number of lesions in the associated part; Step 3: Transmit the data segmented, extracted, identified, and counted by the image processing module to the image acquisition module and a lesion calculation module for storage and calculation; Step 4: Use the lesion calculation module to sequentially calculate and output a preliminary evaluation value WP, an associated part influence value G, and a comprehensive lesion risk value BF; Step 5: Based on the comprehensive lesion risk value BF, use the result display module to display the results for analysis; Among them, the lesion calculation module includes a preliminary lesion assessment unit, an associated site influence unit, and a spatio-temporal factor integration and assessment unit.

[0006] Preferably, the image processing module includes a region segmentation unit, a color analysis unit, and a lesion recognition unit; The region segmentation unit performs segmentation operations based on a segmentation algorithm of machine learning; The color analysis unit performs extraction operations on color features based on color space conversion and feature extraction algorithms; The lesion measurement unit performs recognition and statistical operations based on area calculation and quantity statistics.

[0007] Preferably, the calculation formula of the preliminary lesion assessment unit is as follows: ; Where: WP is the preliminary assessment value. WP multiplies the features of the current lesion in the spatial dimension and the color dimension Y to obtain a value that can preliminarily reflect the overall characteristics of the lesion; BM is the lesion area, which reflects the size of the space occupied by the current lesion site in the digestive tract; BM0 is the average lesion area. BM0 is statistically obtained from image data to reflect the average area of the same site as the lesion area BM in the normal digestive tract of a large number of non-diseased people; Y is the lesion color difference value, which reflects the degree of deviation of the current lesion site from the normal tissue site in the digestive tract in terms of color features; The larger the values of the lesion area BM and the lesion color difference value Y, the larger the value of the preliminary assessment value WP; The smaller the values of the lesion area BM and the lesion color difference value Y, the smaller the value of the preliminary assessment value WP.

[0008] Preferably, the calculation formula of the associated site influence unit is as follows: ; Where: G is the associated site influence value; S is the number of lesions at associated sites. S is the number of lesions in other digestive tract sites that are associated with and belong to the same digestive tract as the current lesion site, statistically obtained through image recognition; S0 is the average number of lesions. S0 is statistically obtained from data to represent the average number of lesions at associated sites in a large number of diseased people; d is the distance coefficient, and the value range of d is {0 - 1}. The specific values are as follows: If the distance between other associated parts and the current lesion site is close, that is, close to or belonging to the adjacent relationship, then d is close to 1 and far from 0; If the distance between other associated parts and the current lesion site is far, that is, far from the adjacent relationship, then d is close to 0 and far from 1; The preliminary evaluation value WP mainly focuses on the characteristics of the current lesion site itself, while reflects the comparison between the lesion conditions of the associated parts and the overall average situation. The sum of the two, that is in the calculation part, to integrate the information of the main lesion site and the associated parts to measure the overall condition of the lesion.

[0009] Preferably, after the image is converted from the RGB color space to the HSV color space, the average hue values of the current lesion area and the normal tissue site are calculated respectively, including the average hue values of hue, saturation, and lightness. Then, using the Euclidean distance method, the deviation degree of the average hue values of the current lesion area and the normal tissue site in the HSV color space is calculated, that is, the lesion color difference value Y.

[0010] Preferably, the calculation formula of the fusion spatio-temporal factor evaluation unit is as follows: ; Where: BF is the comprehensive lesion risk value; V is the lesion development speed, and V reflects the development of the lesions in the current and associated parts over time.

[0011] Preferably, the lesion development speed V is calculated by continuously collecting and identifying image data to calculate the sum of the area change rates of the current and associated parts during the lesion observation time.

[0012] The digestive tract lesion analysis system based on image recognition includes an image acquisition module, an image processing module, a lesion calculation module, and a result display module; The image acquisition module is responsible for acquiring the digestive tract images of the current patient and storing the data processed by the image processing module and the lesion calculation module; The image processing module is responsible for segmenting, extracting, identifying, and counting the images; The lesion calculation module is responsible for calculating and outputting the preliminary evaluation value WP, the associated part influence value G, and the comprehensive lesion risk value BF; The result display module is responsible for displaying the result of the comprehensive lesion risk value BF.

[0013] The technical effects and advantages of the present invention: In the present invention, through the preliminary lesion assessment unit, the lesion area BM, the average lesion area BM0, and the lesion color difference value Y are combined to preliminarily evaluate the lesion from two dimensions of spatial size and color characteristics. This multi-dimensional comprehensive analysis method can more comprehensively reflect the characteristics of the lesion, make up for the deficiency of the prior art that only focuses on a single feature, and greatly improve the accuracy of the preliminary lesion assessment.

[0014] In the present invention, based on the preliminary evaluation value WP, the associated part influence unit comprehensively evaluates the influence of the associated part on the current lesion by introducing the number of lesions S in the associated part, the average number of lesions S0, and the distance coefficient d. This association analysis method can understand the distribution and interaction of lesions in the entire digestive tract system, avoid viewing lesions in isolation, and thus more comprehensively and accurately judge the severity and development trend of the lesion.

[0015] In the present invention, the spatio-temporal factor fusion evaluation unit combines the associated part influence value C and the lesion development speed V, and balances the influence of spatio-temporal factors through square root operation to calculate the comprehensive lesion risk value BF. This innovation fully considers the dynamic changes of the lesion and can more accurately predict the potential risk of the lesion. Description of the Drawings

[0016] Figure 1 is the method flow chart of this digestive tract lesion analysis method; Figure 2 is the overall structure schematic diagram of this digestive tract lesion analysis system; Figure 3 is the segmentation schematic diagram of the digestive tract image in the present invention. Detailed Embodiment

[0017] Now, the present invention will be further described in detail with reference to the accompanying drawings and preferred embodiments.

[0018] Refer to Figures 1 to 3 As shown, the present invention provides a technical solution: a digestive tract lesion analysis method based on image recognition, and the specific implementation steps are as follows: Step 1: Use the image acquisition module to acquire images of the digestive tract of the current patient and transmit them to the image processing module; Step 2: Use the image processing module to separate the lesion part, the part associated with the lesion, and the normal tissue part in the image, extract the color characteristics of the lesion part and the associated part, identify the areas of the lesion part, the part associated with the lesion, and the normal tissue part, and count the number of lesions in the associated part; Step 3: Transmit the data segmented, extracted, identified, and counted by the image processing module to the image acquisition module and the lesion calculation module for storage and calculation; Step 4: Use the lesion calculation module to sequentially calculate and output a preliminary evaluation value WP, an associated part impact value G, and a comprehensive lesion risk value BF; Step 5: Based on the comprehensive lesion risk value BF, use the result display module to display the results for analysis; Among them, the lesion calculation module includes a lesion preliminary evaluation unit, an associated part impact unit, and a fusion spatio-temporal factor evaluation unit; The image processing module includes a region segmentation unit, a color analysis unit, and a lesion recognition unit; The region segmentation unit performs a segmentation operation based on a machine learning-based segmentation algorithm; The segmentation operation can use Mask-RCNN; another simpler method is to use the pixellib library for segmentation processing under the TensorFlow framework.

[0019] The color analysis unit performs an operation of extracting color features based on color space conversion and feature extraction algorithms; in this application, by adjusting the output layer activation function to a linear function, the original pixel inputs of the RGB three channels can be directly processed. Compared with the traditional Sigmoid function, this method can retain color information in the range of [-∞, +∞], automatically learn the color distribution law using the convolutional layer, and the feature map reconstructed by the output layer can reflect the color features. Based on the color features, the diseased area and the surrounding normal tissues are compared to perform digestive tract lesion analysis.

[0020] In addition, in another implementation process, the color histogram method can be selected to achieve color feature extraction, which is an existing technology and will not be elaborated here.

[0021] The lesion measurement unit performs identification and statistical operations based on area calculation and quantity statistics.

[0022] Refer to Figure 1 and Figure 3 As shown, in this implementation plan: the calculation formula of the lesion preliminary evaluation unit is as follows: ; Where: WP is the preliminary evaluation value. WP multiplies the features of the current lesion in the spatial dimension and the color dimension Y to obtain a value that can preliminarily reflect the overall characteristics of the lesion; BM is the lesion area, which reflects the size of the space occupied by the current lesion site in the digestive tract; BM0 is the average lesion area, and BM0 is statistically calculated from the image data to reflect the average area of the same part where the normal digestive tract of a large number of non-diseased people is located as the lesion area BM; Let Y be the color difference value of the lesion. Y reflects the degree of deviation of the current lesion site from the normal tissue site within the digestive tract in terms of color characteristics. After the image is converted from the RGB color space to the HSV color space, the average hue values of the current lesion area and the normal tissue site are calculated respectively, including the average hue values of hue, saturation, and lightness. Then, using the Euclidean distance method, the degree of deviation of the average hue values of the current lesion area and the normal tissue site in the HSV color space is calculated, which is the lesion color difference value Y. The larger the values of the lesion area BM and the lesion color difference value Y, the larger the value of the preliminary evaluation value WP. The smaller the values of the lesion area BM and the lesion color difference value Y, the smaller the value of the preliminary evaluation value WP.

[0023] In this embodiment, The calculation part can calculate and obtain a value that intuitively reflects the size of the lesion relative to the normal situation. This The result value of the calculation part is one of the important factors for the preliminary evaluation of the lesion. It provides basic information about the lesion from the spatial dimension and is an important part of the subsequent comprehensive evaluation. The lesion color difference value Y quantifies the color difference degree between the lesion area and the surrounding normal tissue. By using image color analysis technology, the color characteristics of the lesion area and the surrounding normal tissue are compared, and then the difference value between the two is calculated using an algorithm. This value does not depend on other factors related to the area and specifically reflects the characteristics in terms of color. Thus, the lesion color difference degree Y, as an independent characteristic index, combines with the calculation part to supplement information from different visual characteristic perspectives, making the preliminary evaluation more comprehensive. The lesion preliminary evaluation unit preliminarily evaluates the situation of digestive tract lesions from two aspects: the spatial size and color characteristics of the lesion, providing basic data for subsequent more in-depth analysis. The reason for the multiplication operation is that these two factors are independent of each other and jointly affect the preliminary evaluation of the lesion. Multiplying them can comprehensively consider the effects of both and obtain a value that can preliminarily reflect the overall characteristics of the lesion. The preliminary evaluation value WP provides a basic data for the subsequent lesion analysis, and this value can help the system quickly understand the general situation of the lesion and determine whether the lesion has the value for further in-depth analysis. The subsequent associated part influence unit and the fusion spatio-temporal factor evaluation unit carry out more complex analyses based on this to ensure the accuracy and reliability of the entire lesion analysis process.

[0024] Refer to Figure 1 and Figure 3 As shown, in this implementation plan: The calculation formula of the associated part influence unit is as follows: ; Where: G is the associated part influence value; S is the number of lesions in associated parts. S is obtained by image recognition to count the number of lesions in other digestive tract parts that are associated with the location of the current lesion and belong to the digestive tract; S0 is the average number of lesions. S0 is obtained through data statistics as the average number of lesions in associated parts that have occurred in a large number of diseased individuals; d is the distance coefficient. The value range of d is {0 - 1}, and the specific values are as follows: If the distance between other associated parts and the current lesion site is close, that is, close to or belonging to the adjacent relationship, then d is close to 1 and far from 0; If the distance between other associated parts and the current lesion site is far, that is, far from the adjacent relationship, then d is close to 0 and far from 1; The preliminary evaluation value WP mainly focuses on the characteristics of the current lesion site itself, while reflects the comparison between the lesion conditions of the associated parts and the overall average situation. The two are added together, that is, in the calculation part, to comprehensively integrate the information of the main lesion site and the associated parts to measure the overall condition of the lesion.

[0025] In this embodiment, in the calculation part The ratio obtained by dividing in the calculation part being greater than 1 indicates that the number of lesions S in the associated parts is higher than the average level, and less than 1 indicates that it is lower than the average level. This ratio provides information for comparing with the average situation from the perspective of the number of lesions S in the associated parts, helping to judge the abnormal degree of the lesions in the associated parts. It is an important factor affecting the degree of influence of the associated parts on the current lesion. While the calculation part is added to the preliminary evaluation value WP because they describe the lesion-related information from different aspects. Adding them can comprehensively integrate these two parts of content, and then obtain a more comprehensive value to reflect the overall characteristics of the current lesion and the lesion conditions of the associated parts. After that, the distance coefficient d of the associated parts is multiplied by the value obtained by comprehensively integrating the previous calculation part, which can take the distance factor into account, making the final influence value G of the associated parts more accurately reflect the comprehensive influence degree of the associated parts on the current lesion under the consideration of distance. Such a calculation method can comprehensively and hierarchically consider multiple influencing factors, and obtain a value through mathematical operations that can accurately reflect the comprehensive influence of the associated parts on the current lesion; Based on the preliminary evaluation value of the preliminary evaluation unit for the lesion, the associated part influence unit introduces the number of lesions S in the associated part, the average number of lesions S0 in the associated part, and the distance coefficient d, which can comprehensively evaluate the influence of the associated part on the current lesion, avoid viewing a single lesion in isolation, and is more in line with the actual situation of the mutual influence and spread of digestive tract diseases. Moreover, by combining the preliminary evaluation value with the relevant factors of the associated part, the associated part influence unit realizes the comprehensive analysis from the main lesion site to the associated part. This integration of multiple factors makes the evaluation of the lesion no longer limited to the single lesion itself, but takes into account the interaction between lesions within the entire digestive tract system.

[0026] Referring to Figure 1 and Figure 3 As shown, in this implementation plan: the calculation formula of the integrated spatio-temporal factor evaluation unit is as follows: ; Where: BF is the comprehensive lesion risk value; V is the lesion development speed, and V reflects the development of the current and associated part lesions over time.

[0027] In this embodiment, The calculation part comprehensively considers the changes of the lesion in the spatial dimension and the time dimension, and obtains a value reflecting the comprehensive spatio-temporal change trend of the lesion. The two are multiplied because the risk assessment of the lesion needs to consider both the current situation in space and the change trend in time simultaneously. The multiplication operation can combine these two key factors in different dimensions to reflect the comprehensive dynamic characteristics of the lesion in space and time. And because a relatively large value will be obtained, so the square root operation can adjust it to a more appropriate range, which is convenient for comprehensive calculation and comparison with other factors. At the same time, the influence of the lesion in the spatial and time dimensions is not a simple linear superposition. The square root operation can more reasonably integrate the factors of these two dimensions, avoid the influence of certain factors being over-amplified, and make the result more accurately reflect the actual risk degree of the lesion. Then, the spatio-temporal comprehensive influence value after square root adjustment is added to the associated part influence value G to comprehensively calculate the comprehensive risk value of the digestive tract lesion. Specifically, the associated part influence value G itself contains various information about the lesion in the spatial dimension, while the spatio-temporal comprehensive influence value after square root adjustment more reasonably integrates the spatio-temporal factors. The addition of the two can integrate the basic information in the spatial dimension and the spatio-temporal comprehensive influence information to obtain a value that comprehensively reflects the comprehensive risk of the digestive tract lesion; The integrated spatio-temporal factor evaluation unit combines the associated part influence value G with the lesion development speed, comprehensively considers the changes of the lesion in the spatial and time dimensions, and then calculates the comprehensive lesion risk value BF. The comprehensive lesion risk value is a quantitative index, which helps to more accurately judge the severity and development trend of the disease.

[0028] Referring to Figure 1 and Figure 3 As shown, in this embodiment: the lesion development speed V calculates the sum of the area change rates of the current and associated parts during the lesion observation time by continuously collecting the identified image data.

[0029] In this embodiment, calculating the sum of the area change rates can track the lesion development of the current lesion site and the associated site in real time. Gastrointestinal lesions are often a dynamic process. It is difficult to comprehensively understand the development trend of lesions simply relying on single-image analysis. By continuously collecting images and calculating the area change rates, the system can timely capture the minute changes in the lesion area. Such subtle changes can be clearly presented through the area change rates, which helps to detect the signs of lesion deterioration at an early stage. Moreover, different gastrointestinal lesions and the same lesion have different development speeds at different sites. Therefore, the sum of the area change rates of the current site and the associated site can comprehensively reflect this difference. There is a close physiological connection between different parts of the gastrointestinal tract, and lesions also affect each other. Calculating the sum of the area change rates emphasizes the correlation between the current lesion site and the associated site. And by comprehensively considering the area change rates of both, the system can more comprehensively analyze the spread and evolution laws of lesions in the entire gastrointestinal system.

[0030] It should be noted that any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall also be within the protection scope of the present invention.

Claims

1. A method for analyzing digestive tract lesions based on image recognition, characterized in that: The specific implementation steps are as follows: Step 1: Use an image acquisition module to acquire images of the digestive tract of the current patient and transmit them to the image processing module; Step 2: Use the image processing module to separate the lesion sites, associated sites with lesions, and normal tissue sites in the image, extract the color features of the lesion sites and associated sites, identify the areas of the lesion sites, associated sites with lesions, and normal tissue sites, and count the number of lesions in the associated sites; Step 3: Transmit the data segmented, extracted, identified, and counted by the image processing module to the image acquisition module and the lesion calculation module for storage and calculation; Step 4: Use the lesion calculation module to calculate and output the preliminary evaluation value WP, the influence value G of the associated site, and the comprehensive lesion risk value BF in sequence; Step 5: Based on the comprehensive lesion risk value BF, use the result display module to display the results for analysis; Among them, the lesion calculation module includes a lesion preliminary evaluation unit, an associated site influence unit, and a fusion spatio-temporal factor evaluation unit.

2. The method for analyzing digestive tract lesions based on image recognition according to claim 1, characterized in that: The image processing module includes a region segmentation unit, a color analysis unit, and a lesion identification unit; The region segmentation unit performs segmentation operations based on a segmentation algorithm of machine learning; The color analysis unit performs extraction operations of color features based on color space conversion and feature extraction algorithms; The lesion identification unit performs identification and statistical operations based on area calculation and quantity statistics.

3. The method for analyzing digestive tract lesions based on image recognition according to claim 2, wherein: The calculation formula of the lesion preliminary evaluation unit is as follows: ； Where: WP is the preliminary evaluation value. WP is obtained by multiplying the features of the current lesion in the spatial dimension and the color dimension Y to get a value that can preliminarily reflect the overall characteristics of the lesion; BM is the lesion area, and BM reflects the size of the space occupied by the current lesion site in the digestive tract; BM0 is the average lesion area, and BM0 is statistically obtained from image data to reflect the average area of the same site where the normal digestive tract and the lesion area BM of a large number of non-diseased people are located; Y is the lesion color difference value, and Y reflects the degree of deviation of the current lesion site from the normal tissue site in the digestive tract in terms of color features; The larger the values of the lesion area BM and the lesion color difference value Y, the larger the value of the preliminary evaluation value WP; The smaller the values of the lesion area BM and the lesion color difference value Y, the smaller the value of the preliminary evaluation value WP.

4. The method for analyzing digestive tract lesions based on image recognition according to claim 3, characterized in that: The calculation formula of the associated site influence unit is as follows: ； Where: G is the influence value of the associated site; S is the number of lesions in the associated site, and S is the number of lesions in other digestive tract sites that are associated with the current lesion site and belong to the digestive tract through image recognition statistics; S0 is the average number of lesions, and S0 is obtained through data statistics as the average number of lesions in the associated sites of a large number of diseased people; d is the distance coefficient, and the value range of d is {0 - 1}, and the specific value is as follows: If the distance between other associated sites and the current lesion site is close, that is, close to the adjacent and adjacent relationship distances, then d is close to 1 and far from 0; If the distance between other associated sites and the current lesion site is far, that is, far from the adjacent relationship distance, then d is close to 0 and far from 1; The preliminary evaluation value WP mainly focuses on the characteristics of the current lesion site itself, while reflects the comparison between the lesion conditions of the associated sites and the overall average. The sum of the two, that is the calculation part, combines the information of the main lesion site and the associated sites to measure the overall condition of the lesion.

5. The method for analyzing digestive tract lesions based on image recognition according to claim 3, characterized in that: After the image is converted from the RGB color space to the HSV color space, the average hue values of the current lesion area and the normal tissue site are calculated respectively, including the average hue values of hue, saturation, and lightness. Then, using the Euclidean distance method, the deviation degree of the average hue values of the current lesion area and the normal tissue site in the HSV color space is calculated, that is, the lesion color difference value Y.

6. The method for analyzing digestive tract lesions based on image recognition according to claim 4, wherein: The calculation formula of the fusion spatio-temporal factor evaluation unit is as follows: ； Where: BF is the comprehensive lesion risk value; V is the lesion development speed, and V reflects the development of the current and associated site lesions over time.

7. The method for analyzing digestive tract lesions based on image recognition according to claim 6, characterized in that: The lesion development speed V is calculated by continuously collecting and identifying image data and calculating the sum of the area change rates of the current and associated sites during the lesion observation time.

8. An image recognition-based digestive tract lesion analysis system for performing the image recognition-based digestive tract lesion analysis method according to any one of claims 1-7, characterized in that: It includes an image acquisition module, an image processing module, a lesion calculation module, and a result display module; The image acquisition module is responsible for acquiring the digestive tract image of the current patient and storing the data processed by the image processing module and the lesion calculation module; The image processing module is responsible for segmenting, extracting, identifying, and counting the image; The lesion calculation module is responsible for calculating and outputting the preliminary evaluation value WP, the associated site influence value G, and the comprehensive lesion risk value BF; The result display module is responsible for displaying the result of the comprehensive lesion risk value BF.

Images