Intelligent image recognition system based on digestive system department images

Abstract

The invention relates to the technical field of image processing, in particular to an intelligent image recognition system based on a digestive system department image. According to the technical scheme, the system comprises an acquisition terminal and a processing terminal, wherein the acquisition terminal is used for acquiring a first image of the intestinal tract of a user in a white light mode and a second image corresponding to a narrow-band imaging mode under different light source incidence angles; the processing terminal identifies a three-dimensional convex-concave structure of the inner wall of the intestinal tract according to the gray difference of the first image at different light source incidence angles so as to position a three-dimensional change area; screening out pixel points with blood vessel features in the second image, and marking blood vessel pixel points at corresponding positions of the first image; performing texture analysis on the blood vessel pixel points of the three-dimensional change areas to obtain a texture distribution coefficient of each three-dimensional change area; and according to the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional change area in the adjacent image frames, carrying out area division on the first image so as to improve the accuracy of image area intelligent identification.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an intelligent image recognition system based on gastroenterology images. Background Technology

[0002] Intelligent image recognition based on gastroenterology images involves the integration of medical imaging, computer vision, deep learning, and artificial intelligence technologies, including imaging techniques such as X-rays, CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and endoscopy (such as gastroscopy and colonoscopy). These images provide detailed information about the digestive system. In recent years, deep learning, especially convolutional neural networks (CNNs), has made significant progress in medical image analysis. By training on large amounts of labeled image data, deep learning convolutional neural network models can learn image features and automatically identify and classify images.

[0003] To achieve intelligent region recognition based on gastroenterological images, it is necessary to acquire endoscopic images of patients and identify different regions within them. However, some protrusions and depressions may not be obvious in the images, and due to the complex internal environment of the intestine, normal blood vessels on the intestinal surface and attachments in the intestine can affect the region segmentation effect of the intestinal surface image, resulting in poor reliability of image region segmentation. Summary of the Invention

[0004] To address the technical problem of poor reliability in image region segmentation in related technologies, the present invention aims to provide an intelligent image recognition system based on gastroenterology images. The specific technical solution adopted is as follows: This invention provides an intelligent image recognition system based on gastroenterology images, the system comprising: The acquisition terminal is used to acquire a first image of the user's intestine under different incident angles of light sources in white light mode, and a second image in the corresponding narrowband imaging mode, and output them externally; The processing terminal is connected to the acquisition terminal. The processing terminal is used to obtain the three-dimensional variation region of the intestinal wall as a convex and concave structure based on the gray scale difference of the first image at different incident angles of light sources. Pixels in the second image that contain vascular features are selected, and the corresponding positions in the first image are marked as vascular pixels. Texture feature analysis is performed on the blood vessel pixels in the three-dimensional change region to obtain the texture distribution coefficient of each three-dimensional change region. The first image is divided into regions based on the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional change region in adjacent image frames.

[0005] In one optional embodiment, based on the grayscale differences of the first image at different incident angles of light sources, a three-dimensional variation region in which the intestinal wall exhibits a convex-concave structure is obtained, including: Multiple first images of the same intestinal region under different incident angles of light sources are aligned and morphological opening operations are performed to obtain image region groups of the corresponding intestinal region. Three-dimensional convex-concave feature analysis is performed based on the pixel grayscale changes of each first image in the image region group to obtain the three-dimensional variation region of the intestinal wall as a convex-concave structure.

[0006] In one optional embodiment, three-dimensional convex-concave feature analysis is performed based on the pixel grayscale changes of each first image in the image region group to obtain a three-dimensional variation region in which the intestinal wall presents a convex-concave structure, including: The difference and absolute value of the gray values ​​of the pixels in the same position of each first image in the image region group are calculated to obtain the absolute value of the difference between the pixels. Perform Otsu thresholding on all absolute differences and identify pixels with values ​​greater than the corresponding threshold as changed pixels; Pixels whose pixel difference in each first image is greater than a preset value are marked as first-class pixels, and pixels whose pixel difference is less than or equal to the preset value are marked as second-class pixels. Based on the distribution characteristics represented by the first and second types of pixels, a three-dimensional variation region in which the intestinal wall presents a convex-concave structure is obtained.

[0007] In one optional embodiment, based on the distribution characteristics represented by the first type of pixels and the second type of pixels, a three-dimensional variation region in which the intestinal wall exhibits a convex-concave structure is obtained, including: Morphological opening and morphological closing operations are performed on the first type of pixels and the second type of pixels respectively to obtain the first cluster formed by the first type of pixels and the second cluster formed by the second type of pixels; The first cluster and the second cluster are compared in terms of their proximity. If the first cluster and the second cluster are the closest clusters to each other, the image area covered by the first cluster and the second cluster is determined to be a three-dimensional change region.

[0008] In one optional embodiment, pixels in the second image that contain vascular features are selected, and the corresponding positions in the first image are marked as vascular pixels, including: The first and second images of the same intestinal region are aligned in position, and the second image is input into the HSV model to obtain the hue value of each pixel in the second image; Based on the hue value and color saturation of each pixel in the second image, vascular feature analysis is performed to determine the target pixels in the second image that represent vascular features. The pixel in the first image corresponding to the target pixel is identified as the blood vessel pixel.

[0009] In one optional embodiment, vascular feature analysis is performed based on the hue value and color saturation of each pixel in the second image to determine the target pixels in the second image that represent vascular features, including: Pixels in the second image whose hue values ​​are within a preset range are identified as first-selection pixels, and pixels in the second image whose color saturation is greater than a saturation threshold are identified as second-selection pixels. Perform morphological closing operations on the pixels in the second image that are determined by both the first and second selected pixels, and determine the result of the operation as the target pixel.

[0010] In one optional embodiment, texture feature analysis is performed on the blood vessel pixels in the three-dimensional variation region to obtain the texture distribution coefficient of each three-dimensional variation region, including: The blood vessel density of the corresponding three-dimensional change region is obtained based on the first number of blood vessel pixels and the second number of all pixels in each three-dimensional change region. The three-dimensional variation region with blood vessel density greater than a preset density threshold is defined as the target variation region; A vascular skeleton morphology analysis is performed on the target change region to obtain the mean curvature value that characterizes the tortuosity of blood vessels in the target change region, as well as the feature value that characterizes the gradient change characteristics of each pixel in the target change region with its surrounding pixels. Based on the mean curvature of the target change region and all feature values, the texture distribution coefficient of the corresponding three-dimensional change region is obtained.

[0011] In one optional embodiment, vascular skeleton morphology analysis is performed on the target change region to obtain the mean curvature value characterizing the vascular curvature characteristics within the target change region, and feature values ​​characterizing the gradient change characteristics of each pixel within the target change region relative to surrounding pixels, including: Morphological skeleton features are extracted from the target change region to obtain skeleton line segments that represent the blood vessel extension path; The curvature of the corresponding skeleton line segment is obtained based on the line segment length of each skeleton line segment and the straight line lengths of its two endpoints. The mean curvature of the corresponding target change region is obtained based on the mean curvature of all skeleton line segments in each target change region. Based on the gradient comparison results between each pixel in each target change region and its surrounding pixels, a result sequence represented as binary is obtained; The binary result sequence is converted into decimal numbers to obtain the feature value of each pixel within the target variation area.

[0012] In one optional embodiment, the texture distribution coefficient of the corresponding three-dimensional variation region is obtained based on the mean curvature of the target variation region and all feature values, including: The distribution of feature values ​​of each pixel within the target change region is statistically processed, and the statistical results are used as the texture parameters of the corresponding target change region. The texture distribution coefficient of the corresponding three-dimensional variation region is determined by the ratio of the mean curvature of each target variation region to the texture parameter.

[0013] In one optional embodiment, the first image is divided into regions based on the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional variation region in adjacent image frames, including: Based on the optical flow characteristics generated by each three-dimensional change region in adjacent image frames, the variance of the optical flow vector of the current three-dimensional change region and the difference in optical flow vector between the current three-dimensional change region and the surrounding three-dimensional change regions are obtained. The regional rigidity of the corresponding three-dimensional change region is determined based on the texture distribution coefficient, optical flow vector variance, and optical flow vector difference of each three-dimensional change region. Based on the numerical value of the region's rigidity, the three-dimensional changing region is divided using the Otsu method to determine different recognition regions.

[0014] The present invention has the following beneficial effects: The technical solution of this invention includes an intelligent recognition system composed of an acquisition terminal and a processing terminal. The acquisition terminal acquires a first image of the user's intestine under different incident angles of light in white light mode and a second image in a corresponding narrowband imaging mode, obtaining multimodal image input. The processing terminal identifies the three-dimensional convex and concave structure of the intestinal wall based on the grayscale differences of the first image under different incident angles of light to locate three-dimensional change regions. It further filters out pixels with vascular features in the second image and maps them to the corresponding positions in the first image, marking them as vascular pixels. Texture feature analysis is performed on the vascular pixels in the three-dimensional change regions to calculate the texture distribution coefficient of each three-dimensional change region. Based on the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional change region in adjacent image frames, the first image is divided into regions. This invention provides a more comprehensive segmentation of image feature regions through the above scheme and reduces the influence of substances attached to the intestinal wall, thereby improving the accuracy of intelligent recognition of gastroenterology imaging regions. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the structure of an intelligent image recognition system based on gastroenterology images provided in one embodiment of the present invention; Figure 2 This is a first image of the intestinal wall at a first light source incident angle, provided in an embodiment of the present invention. Figure 3 This is a first image of the intestinal wall at a second light source incident angle, provided in an embodiment of the present invention. Figure 4 This is a second image of the intestinal wall at a first light source incident angle, provided as an embodiment of the present invention. Figure 5 This is a second image of the intestinal wall at a second incident angle of a second light source, provided as an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an image intelligent recognition system based on gastroenterology images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific solution for an intelligent image recognition system based on gastroenterology images provided by this invention.

[0020] Please see Figure 1 , Figure 1This is a schematic diagram of an intelligent image recognition system based on gastroenterology images, provided as an embodiment of the present invention. The intelligent recognition system includes an acquisition terminal 1 and a processing terminal 2. The acquisition terminal 1 is connected to the processing terminal 2 via a cable to transmit the acquired image data. The acquisition terminal 1 can be configured as a memory for storing image data statistically obtained from relevant data systems. The processing terminal 2 can be configured as a computer device or a server device, as long as it is capable of performing image data processing; no specific limitations are imposed here.

[0021] Terminal 1 acquires a first image of the user's intestine under different light source incident angles in white light mode and a second image in the corresponding narrowband imaging mode from a relevant medical database, and outputs them to processing terminal 2. Please refer to [link to relevant documentation]. Figures 2 to 3 , Figure 2 and Figure 3 These are the first images taken at different incident angles from different light sources. Because the intestines contain mucus and air bubbles, specular reflection occurs in some areas when the intestinal wall is illuminated by different light sources at different angles, resulting in highlights in the first images. Please refer to [link / reference]. Figures 4 to 5 , Figure 4 For the corresponding Figure 2 The second image at the first light source incident angle. Figure 5 For the corresponding Figure 3 In the second image at the second light source incident angle, in narrowband imaging mode, the broadband spectrum of the endoscopic light source is filtered by a filter, retaining only narrowband light of specific wavelengths, such as 415nm blue light and 540nm green light, to enhance the visualization of blood vessels and structures on the surface of the digestive tract mucosa, thereby improving the diagnostic accuracy of early lesions.

[0022] When acquiring the first and second images via terminal 1, informed consent from the patient is required. While maintaining the lens at a stable distance of 2-5 mm from the mucosal surface and the optimal focal length, the system sequentially switches between White Light (WL) mode and narrowband imaging mode to acquire spectrally paired images from the same viewing angle. Narrowband imaging can be either NBI (Narrow Band Imaging) or BLI (Blue Light Imaging). The incident angle of the light source is then adjusted to generate lateral grazing light, and the mucosal topological shadow response sequence is acquired. Subsequently, the magnifying endoscopy function is activated for 80-100x macro imaging to capture the pit pattern texture of the intestinal surface. Finally, all image data is embedded into a structured metadata package containing the patient's anonymous ID, anatomical coordinates, and device optical parameters, and transmitted in real-time to processing terminal 2 via an encrypted network to construct a multimodal feature fusion database.

[0023] The following will elaborate on how the processing terminal 2 divides the first image into regions based on the features of the first and second images, specifically including: S11. Based on the grayscale difference of the first image at different incident angles of light sources, obtain the three-dimensional variation region of the intestinal wall as a convex-concave structure.

[0024] Specifically, the intestinal wall can exhibit morphological changes, such as slight protrusions or depressions. These areas represent three-dimensional variations. The first image represents the intestinal endoscopy image under different light source angles. To highlight the state of different regions in the image, a three-dimensional morphological analysis of the acquired patient intestinal endoscopy images is necessary. Because mucus bubbles exist in the patient's digestive tract, some areas exhibit specular reflection. Therefore, grayscale difference analysis can be performed on multiple first images taken at the same intestinal wall location with different light source incident angles to determine the three-dimensional variations of the intestinal wall.

[0025] For example, step S11 includes sub-steps S11-1 to S11-2, which are described in detail below: S11-1. Perform position alignment and morphological opening operations on multiple first images of the same intestinal region under different incident angles of light sources to obtain image region groups for the corresponding intestinal regions. The white light image at a normal angle from the currently acquired patient's intestinal endoscopy is configured as the first image, and the first image is denoted as the patient's image. Imaging; multiple first images are available for each intestinal region being examined, and these images are obtained after adjusting the angle of the light source at the front of the endoscope and comparing them with the current image. Images acquired simultaneously from different light source directions are denoted as the patient's images. The image is captured and its position is aligned. For the acquired... and The images are subjected to morphological opening operations to eliminate specular reflection noise. Multiple first images of each intestinal region are combined to form an image region group for that region.

[0026] S11-2. Perform three-dimensional convex-concave feature analysis based on the pixel grayscale changes of each first image in the image region group to obtain the three-dimensional variation region of the intestinal wall as a convex-concave structure. For the noise-reduced... and The image is processed by acquiring the grayscale values ​​of pixels at different locations. Each grayscale value describes the brightness change of a pixel from pure black (0) to pure white (255), with a total of 256 levels. After grayscale processing of each first image, three-dimensional convex-concave feature analysis can be performed based on the grayscale values ​​of each pixel, and the three-dimensional change region can be derived from the analysis results. For example, a lightweight convolutional neural network model can be used for recognition and analysis, and the three-dimensional change region can be derived from the recognition and analysis results. In practical applications, identifying 3D changing regions based on convolutional neural network models requires significant computing power and is time-consuming. Therefore, in one specific implementation, sub-step S11-2 includes: The first step involves performing grayscale value difference and absolute value operations on pixels at the same position in each of the first images within the image region group to obtain the absolute value of the difference between each pixel. This grayscale value difference and absolute value operation can be performed based on a preset calculation model, for example, by obtaining the positional alignment of the same pixel within each of the first images. With the first image grayscale values ​​in the image and ; to pixel and The difference is denoted as the grayscale change of the pixel. ; Calculate the grayscale change of a pixel. absolute value This allows you to obtain the absolute value of the difference between each pixel.

[0027] The second step involves performing Otsu thresholding on all absolute differences, identifying pixels with values ​​greater than the corresponding threshold as changed pixels. Otsu thresholding is an automatic image binarization algorithm based on maximum inter-class variance. Its function is to find the optimal threshold to divide the image into foreground and background parts, maximizing the inter-class variance and thus separating the target from the background. Since changes in the lighting on the endoscope affect the tissue state of the intestinal surface, different regions in the image will exhibit different changes. The absolute difference of pixel values ​​is used to address this. The patient's endoscopic image was segmented using the Otsu threshold method, and the pixels above the threshold were identified and recorded as the variable pixels.

[0028] The third step involves marking pixels whose pixel differences in the corresponding first images are greater than a preset value as first-class pixels, and pixels whose pixel differences are less than or equal to the preset value as second-class pixels. Due to the depressions and bulges on the intestinal surface, the three-dimensional structure of the intestinal wall changes, resulting in one side being darker than the other. Similarly, with changes in light, one side may appear brighter than the other. The preset value can be set to 0, and the pixel difference between the two first images represents the grayscale change. The amount of grayscale change corresponding to the changed pixel. Pixels with a value greater than 0 are designated as first-class pixels. Pixels less than or equal to 0 are designated as second-class pixels; first-class and second-class pixels are marked in the first image respectively.

[0029] The fourth step involves obtaining the three-dimensional variation region of the intestinal wall exhibiting a convex-concave structure based on the distribution characteristics represented by the first and second types of pixels. Since the concave and convex regions will result in clusters of the first and second types of pixels on both sides due to changes in the angle of light, the three-dimensional variation region can be determined based on the cluster distribution characteristics of the first and second types of pixels.

[0030] For example, morphological opening and closing operations are performed on the first and second types of pixels respectively to obtain a first cluster formed by the first type of pixels and a second cluster formed by the second type of pixels. The morphological opening operation can eliminate some discrete marker points; the morphological closing operation merges pixels that should belong to the same cluster; thus, several clusters of the first and second types of pixels in the first image are obtained. The first and second clusters are then compared for proximity. If the current first cluster and the current second cluster are the closest clusters to each other, the image area covered by the current first cluster and the current second cluster is determined to be a three-dimensional change region. That is, the three-dimensional change region includes the image areas where the current first cluster and the current second cluster are located, as well as the image area between the current first cluster and the current second cluster.

[0031] It is understandable that when performing 3D change region analysis, we can compare and analyze a pixel in the first cluster with another pixel in the second cluster. For any first type of pixel cluster, we need to obtain the nearest second type of pixel cluster, and for any second type of pixel cluster, we need to obtain the nearest first type of pixel cluster. If the nearest second type of pixel cluster to the first cluster formed by the first type of pixels is the first cluster formed by the first type of pixels, then we obtain this pair of pixel clusters and grow the pixel clusters into the same pixel cluster. This indicates that there is a contrast feature between light and dark in this region, and this region is recorded as a 3D change region.

[0032] At this point, the three-dimensional variation region of the intestinal wall, which presents a convex and concave structure, has been obtained, and we proceed to step S12.

[0033] S12. Filter out the pixels in the second image that have blood vessel features, and mark the corresponding positions in the first image as blood vessel pixels.

[0034] Specifically, since the normal intestinal wall may also contain some normal protrusions and depressions, directly dividing and identifying regions based on the three-dimensional changes obtained in the above steps may lead to identification errors. Therefore, it is necessary to perform vascular structure analysis based on the spectral characteristics in the endoscopic images. To analyze the three-dimensional vascular structure, it is necessary to obtain partial vascular images from the three-dimensional images. The protrusions and depressions in the intestine are essentially abnormal microvascular structures. The narrow-band imaging mode in endoscopy is highly sensitive to microvascular patterns. Pixels with vascular features can be identified based on the second image and marked at the corresponding positions in the first image to determine the vascular pixels in the first image.

[0035] For example, step S12 includes sub-steps S12-1 to S12-3, which are described in detail below: S12-1. Align the first and second images of the same intestinal region and input the second image into the HSV model to obtain the hue value of each pixel in the second image. The first image is denoted as the WL image, and the corresponding second image is denoted as the NBI image. Register and align the WL image and the NBI image. Because specific wavelengths of light in narrow-band imaging technology are strongly absorbed by hemoglobin, abnormal vascular regions will exhibit specific color values ​​(e.g., dark green or brown) in the hue channel. Project the acquired patient WL and NBI images into the HSV space; the hue value H and color saturation value S of each pixel in the NBI image can then be obtained.

[0036] S12-2. Perform vascular feature analysis based on the hue and saturation of each pixel in the second image to identify target pixels representing vascular features. In the second image, hemoglobin-rich blood vessels absorb 415nm blue light, thus deviating from blue in hue and tending towards a specific brownish hue range. Consequently, the hue range of the blood vessel portion in the NBI image is typically within the brownish hue range of 20°-40°. Therefore, pixels with hue values ​​within a preset range in the second image can be identified as first-selection pixels, and pixels with saturation greater than a saturation threshold can be identified as second-selection pixels. Morphological closing operations are then performed on the pixels identified by both the first and second-selection pixels, and the result is determined as the target pixels.

[0037] Specifically, for the hue value H of the pixels in the second image, threshold segmentation is performed using dual thresholds of 20° and 40° to obtain pixels located between the two thresholds, which are denoted as the first selected pixels in the NBI image. Furthermore, since blood vessels in the intestine form a strong color contrast with the surrounding mucosal tissue, this means that the color purity of the edge region is very high, i.e., the color saturation is very high. Threshold segmentation can be performed on the image using the color saturation value S of the pixels in the first image, and the portion exceeding the threshold is denoted as the second selected pixels in the image. The overlapping pixels between the first and second selected pixels in the second image are then obtained, and morphological closing operations are applied to the overlapping pixels to eliminate internal small holes and connect broken parts. The resulting pixel portion is denoted as the target pixel in the second image.

[0038] S12-3. Determine the target pixel as the corresponding pixel in the first image as the blood vessel pixel. That is, mark the blood vessel pixels in the NBI image in the WL image to obtain the blood vessel pixels in the patient's WL image.

[0039] At this point, the vascular pixels with vascular features have been identified in the first image, and the process proceeds to step S13.

[0040] S13. Perform texture feature analysis on the blood vessel pixels in the three-dimensional change region to obtain the texture distribution coefficient of each three-dimensional change region.

[0041] Specifically, since concave-convex structures typically require more blood supply, it is necessary to analyze the distribution of blood vessels in a three-dimensional changing region based on blood vessel pixels. This can be achieved through quantitative analysis based on the skeletal feature distribution formed by the blood vessel pixels, with the analysis results defined as the texture distribution coefficient. The texture distribution coefficient characterizes the dimensionality of blood vessel distribution in a three-dimensional changing region; a larger coefficient indicates a more complex blood vessel texture distribution in the corresponding three-dimensional changing region.

[0042] In one specific implementation, texture feature analysis is performed based on the blood vessel density and tortuosity of the three-dimensional variation region, specifically including: S13-1. Based on the first number of blood vessel pixels and the second number of all pixels in each three-dimensional change region, obtain the blood vessel density of the corresponding three-dimensional change region. For any three-dimensional change region, obtain the number of pixels within it. This is denoted as the second quantity; and the number of blood vessel pixels within it. This is denoted as the first quantity; from this, the blood vessel density of the three-dimensional variation region can be obtained. Specifically, by the number of blood vessel pixels With the number of pixels The ratio of the two values ​​is normalized to obtain the vascular density. Among them, the vascular density... The larger the value, the greater the proportion of blood vessel pixels in the three-dimensional change region, thus indicating a higher blood vessel density in the three-dimensional change region; conversely, the smaller the value, the lower the blood vessel density in the three-dimensional change region.

[0043] S13-2. The three-dimensional variation region where the blood vessel density exceeds a preset density threshold is defined as the target variation region. The preset density threshold can be set based on the experience of technical personnel, or it can be determined by calibrating experimental facilities to accurately identify the target variation region; no specific restrictions are imposed here. When the blood vessel density exceeds the preset density threshold, it indicates that the three-dimensional variation region contains relatively dense blood vessels and has a complex texture distribution; therefore, it is defined as the target variation region.

[0044] S13-3. Perform vascular skeleton morphology analysis on the target change region to obtain the average curvature value representing the tortuosity of blood vessels within the target change region, and the feature value representing the gradient change characteristics of each pixel within the target change region relative to its surrounding pixels. Since there may be some vascular plexuses on the surface of the intestinal wall, directly judging texture features by blood vessel density may lead to misjudgment. In this embodiment of the invention, vascular skeleton morphology analysis is performed to quantify the curvature of blood vessels and the gradient change characteristics of pixels, so as to obtain a more accurate texture distribution coefficient.

[0045] Furthermore, sub-step S13-3 includes: The first step is to extract morphological skeleton features from the target area of ​​change to obtain skeleton segments representing the extension paths of blood vessels. The blood vessel portion of the 3D area of ​​change is obtained, and its morphological skeleton is extracted using morphological extraction. For the morphological skeleton within the blood vessel, several line segments are extracted and denoted as skeleton segments. Skeleton segments are mostly curves; however, they can also be straight lines, and no specific restriction is placed here.

[0046] The second step is to obtain the curvature of each skeleton line segment based on its segment length and the straight line lengths of its two endpoints (which do not coincide). For each skeleton line segment, its segment length is obtained. And for each line segment, obtain the length of the line connecting its two endpoints (straight line). Therefore, the curvature w of the skeleton line segment can be obtained, specifically through the formula: The calculation and normalization yield the following result: for the curvature w, the larger the value, the longer the line segment length is relative to the length between its two endpoints, indicating that the line segment is more severely curved between its two endpoints, and therefore the curvature is greater; conversely, the smaller the value, the smaller the curvature.

[0047] The third step is to obtain the average curvature of the corresponding target change region based on the average curvature of all skeleton line segments in each target change region. This can be achieved by calculating the average curvature of each target change region, denoted as _____. .

[0048] The fourth step involves comparing the gradient of each pixel in each target variation region with its surrounding pixels to obtain a binary sequence of results. For patients... In the three-dimensional changing region of the image, the gradient values ​​of the pixels are obtained. The gradients of the pixels in the eight-neighborhood are arranged in a certain order (e.g., clockwise). The gradient values ​​of the eight-neighborhood pixels are compared one by one with the gradient value of the center pixel. If the gradient value is greater than that of the center pixel, it is recorded as 0, and if it is less than that of the center pixel, it is recorded as 1. The comparison result is represented as the number of pixel labels in the eight-neighborhood. The pixel labels in the eight-neighborhood are arranged in their order to obtain a binary result sequence for that pixel.

[0049] The fifth step is to convert the binary result sequence into decimal numbers to obtain the feature value of each pixel within the target variation area. The result sequence for each pixel is then converted into a decimal number and recorded as the feature value of the corresponding pixel, denoted as z.

[0050] S13-4. Based on the mean curvature of the target changing region and all feature values, obtain the texture distribution coefficient of the corresponding three-dimensional changing region. A calculation model can be constructed based on the correspondence between the mean curvature, feature values, and texture distribution coefficients. The texture distribution coefficient of the corresponding three-dimensional changing region can then be calculated using this model.

[0051] For example, sub-step S13-4 specifically includes: The first step is to statistically process the distribution of feature values ​​for each pixel within the target change region, and then determine the statistical results as the texture parameters for that region. Pixels within the 3D change region can be labeled with their feature values ​​to obtain the distribution of different types of feature values ​​in the image, thus acquiring the texture parameters for the 3D change region. For example, feature values ​​z can be calculated for all pixels within the 3D change region; statistically analyzing the distribution of all feature values ​​z will generate a histogram or frequency distribution. The more dispersed the distribution, the more complex the local gray-level changes and the less uniform the texture. The variance of all feature values ​​can be calculated; a larger variance indicates a more complex texture. Alternatively, the entropy of the feature value distribution can be calculated; a higher entropy indicates stronger texture randomness. The texture parameters essentially characterize the complexity of gray-level gradient changes within the target change region.

[0052] The second step involves determining the texture distribution coefficient for each target variation region based on the ratio of the mean curvature to the texture parameter. Specifically, this involves calculating the ratio of the mean curvature to the texture parameter for each target variation region, followed by normalization to obtain the texture distribution coefficient. It should be noted that when the texture parameter value is 0, it is adjusted to 0.1 to make the calculation results meaningful. The error generated during this adjustment is negligible. Specifically, regarding the texture distribution coefficient... The larger the value, the more severe the tortuosity of the blood vessels in the three-dimensional variation region, and the more irregular the texture distribution of the pixels in the three-dimensional variation region.

[0053] At this point, the texture distribution coefficient of the three-dimensional variation region has been obtained based on the above method, and we proceed to step S14.

[0054] S14. Divide the first image into regions based on the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional change region in adjacent image frames.

[0055] Specifically, the presence of mucus and impurities in the intestines can lead to false positives, affecting the recognition accuracy. Therefore, for the three-dimensional change region in the current image, it is necessary to acquire the next frame image; and key point matching is used to obtain the three-dimensional change region in the next frame, and region segmentation is performed based on the optical flow field between adjacent image frames.

[0056] For example, step S14 includes sub-steps S14-1 to S14-3, which are described in detail below: S14-1. Based on the optical flow characteristics generated by each 3D change region in adjacent image frames, obtain the optical flow vector variance of the current 3D change region and the optical flow vector difference between the current 3D change region and its surrounding 3D change regions. Since the same region in consecutive image frames will maintain a consistent morphology if it belongs to the same state, false positives such as bubbles and mucus will quickly disappear with illumination or flow. The Lucas-Kanade algorithm can be used to calculate the optical flow vector of each pixel in two image frames, and the optical flow vector variance f and optical flow vector difference s can be calculated based on the optical flow vectors of the pixels in the 3D change region.

[0057] S14-2. Based on the texture distribution coefficient, optical flow vector variance, and optical flow vector difference of each three-dimensional variation region, determine the region rigidity of the corresponding three-dimensional variation region. This allows us to obtain the region rigidity within the three-dimensional variation region. It can be done through the formula: The region stiffness is obtained after calculation and normalization. The larger the value of the ratio of the optical flow vector difference s to the optical flow vector variance f, the more consistent the optical loss in the three-dimensional change region of the image is, and the greater the difference between the optical flow vector and the optical flow vector outside the three-dimensional change region. Therefore, it indicates that the three-dimensional change region has good integrity and a large difference from its surroundings, and thus its regional rigidity is large. Conversely, the calculated three-dimensional change region has smaller regional rigidity.

[0058] S14-3. Based on the numerical value of the region rigidity, the three-dimensional changing region is divided using the Otsu method to determine different recognition regions.

[0059] Otsu thresholding can be used to assess the regional rigidity of three-dimensional variable regions. The image is segmented to divide it into regions, and the corresponding locations are marked in the image.

[0060] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. An intelligent image recognition system based on gastroenterology images, characterized in that, The system includes: The acquisition terminal is used to acquire a first image of the user's intestine under different incident angles of light sources in white light mode, and a second image in the corresponding narrowband imaging mode, and output them externally; A processing terminal is connected to the acquisition terminal. The processing terminal is used to obtain a three-dimensional variation region in which the intestinal wall presents a convex and concave structure based on the grayscale difference of the first image at different incident angles of light sources. Pixels in the second image that contain vascular features are selected, and the corresponding positions in the first image are marked as vascular pixels; Texture feature analysis is performed on the blood vessel pixels in the three-dimensional change region to obtain the texture distribution coefficient of each three-dimensional change region; The first image is divided into regions based on the texture distribution coefficient and the optical flow characteristics generated by each three-dimensional change region in adjacent image frames.

2. The image intelligent recognition system based on gastroenterology images according to claim 1, characterized in that, Based on the grayscale differences in the first image at different incident angles of light sources, a three-dimensional variation region of the intestinal wall exhibiting a convex-concave structure is obtained, including: Multiple first images of the same intestinal region under different incident angles of light sources are aligned and morphological opening operations are performed to obtain image region groups of the corresponding intestinal region. Three-dimensional convex-concave feature analysis is performed based on the pixel grayscale changes of each first image in the image region group to obtain the three-dimensional variation region of the intestinal wall as a convex-concave structure.

3. The image intelligent recognition system based on gastroenterology images according to claim 2, characterized in that, The step of performing three-dimensional convex-concave feature analysis based on the pixel grayscale changes of each first image in the image region group to obtain a three-dimensional variation region in which the intestinal wall presents a convex-concave structure includes: The difference and absolute value of the gray values ​​of the pixels in the same position of each first image in the image region group are calculated to obtain the absolute value of the difference between the pixels. Perform Otsu thresholding on all absolute differences and identify pixels with values ​​greater than the corresponding threshold as changed pixels; Pixels whose pixel difference in each first image is greater than a preset value are marked as first-class pixels, and pixels whose pixel difference is less than or equal to the preset value are marked as second-class pixels. Based on the distribution characteristics represented by the first type of pixels and the second type of pixels, a three-dimensional variation region in which the intestinal wall presents a convex-concave structure is obtained.

4. The image intelligent recognition system based on gastroenterology images according to claim 3, characterized in that, The step of obtaining a three-dimensional variation region in which the intestinal wall exhibits a convex-concave structure based on the distribution characteristics represented by the first type of pixels and the second type of pixels includes: Morphological opening and morphological closing operations are performed on the first type of pixels and the second type of pixels respectively to obtain a first cluster formed by the first type of pixels and a second cluster formed by the second type of pixels; The first cluster and the second cluster are compared in terms of their proximity. If the first cluster and the second cluster are the closest clusters to each other, the image area covered by the first cluster and the second cluster is determined to be the three-dimensional change area.

5. The image intelligent recognition system based on gastroenterology images according to claim 1, characterized in that, The step of filtering out pixels in the second image that contain vascular features and marking the corresponding positions in the first image as vascular pixels includes: The first and second images of the same intestinal region are aligned in position, and the second image is input into the HSV model to obtain the hue value of each pixel in the second image; Based on the hue value and color saturation of each pixel in the second image, vascular feature analysis is performed to determine the target pixels in the second image that represent vascular features; The target pixel is identified as the blood vessel pixel corresponding to the pixel in the first image.

6. The image intelligent recognition system based on gastroenterology images according to claim 5, characterized in that, The step of performing vascular feature analysis based on the hue value and color saturation of each pixel in the second image to determine the target pixels in the second image that represent vascular features includes: Pixels in the second image whose hue values ​​are within a preset range are identified as first-selection pixels, and pixels in the second image whose color saturation is greater than a saturation threshold are identified as second-selection pixels. A morphological closing operation is performed on the pixels in the second image that are determined by both the first and second selected pixels, and the result of the operation is determined as the target pixel.

7. The image intelligent recognition system based on gastroenterology images according to claim 1, characterized in that, The process of performing texture feature analysis on the blood vessel pixels in the three-dimensional change region to obtain the texture distribution coefficient of each three-dimensional change region includes: The blood vessel density of the corresponding three-dimensional change region is obtained based on the first number of blood vessel pixels and the second number of all pixels in each three-dimensional change region. The ratio of the first number to the second number is calculated and normalized to obtain the blood vessel density. The three-dimensional variation region where the blood vessel density is greater than a preset density threshold is defined as the target variation region; Vascular skeleton morphology analysis is performed on the target change region to obtain the mean curvature value characterizing the vascular curvature characteristics within the target change region, and the feature value characterizing the gradient change characteristics between each pixel and its surrounding pixels within the target change region. Based on the mean curvature of the target change region and all feature values, the texture distribution coefficient of the corresponding three-dimensional change region is obtained; Methods for determining the mean and characteristic values ​​of curvature include: Morphological skeleton features are extracted from the target change region to obtain skeleton line segments that represent blood vessel extension paths; The curvature of each skeleton line segment is obtained based on its segment length and the straight line lengths at its two endpoints. Specifically, the segment length of each skeleton line segment is determined. For each line segment, obtain the length of the line connecting its two endpoints (straight line). The formula for the curvature w of the skeleton line segment is: ; The mean curvature of the corresponding target change region is obtained based on the mean curvature of all skeleton line segments in each target change region. Based on the gradient comparison results between each pixel in each target change region and its surrounding pixels, a result sequence represented as binary is obtained; The binary result sequence is converted into decimal numbers to obtain the feature value of each pixel within the target change area; Methods for determining texture distribution coefficients include: The distribution of feature values ​​of each pixel within the target change region is statistically processed, and the statistical results are determined as the texture parameters of the corresponding target change region. The variance of all feature values ​​is calculated as the texture parameters. The texture distribution coefficient of the corresponding three-dimensional variation region is determined by the ratio of the mean curvature of each target variation region to the texture parameter.

8. The image intelligent recognition system based on gastroenterology images according to claim 1, characterized in that, The step of dividing the first image into regions based on the texture distribution coefficient and the optical flow features generated by each three-dimensional change region in adjacent image frames includes: Based on the optical flow characteristics generated by each three-dimensional change region in adjacent image frames, the variance of the optical flow vector of the current three-dimensional change region and the difference in optical flow vector between the current three-dimensional change region and the surrounding three-dimensional change regions are obtained. The regional rigidity of the corresponding three-dimensional change region is determined based on the texture distribution coefficient, optical flow vector variance, and optical flow vector difference of each three-dimensional change region. Based on the numerical value of the region's rigidity, the three-dimensional changing region is divided using the Otsu method to determine different recognition regions.

Images