Colon image processing method and apparatus and related devices

Abstract

The application provides a colon image processing method and device and related equipment thereof. The method obtains a first segmentation image of a target object in an endoscopic image, boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmentation image. Based on a mean value of a first color feature vector set of the first segmentation image and a mean value of a second color feature vector set of the background mucosa image, a quantization coefficient of a color of the target object relative to the background mucosa is determined. A target mean value of a mean value set of a third color feature vector set of all target blood vessels in an overlay image is obtained, the overlay image being obtained by overlaying a second segmentation image and the endoscopic image. Based on the quantization coefficient, the target mean value, and the surface structure category features, an abnormal type of the target object is determined. The above scheme realizes accurate and detailed identification and division of the abnormal type of the target object on the colon.

Description

Technical Field

[0001] This application relates to the field of assistive medical technology, specifically to a colon image processing method, apparatus, and related equipment. Background Technology

[0002] Most colorectal cancers originate from adenomas, and nearly 15% to 30% of tumors develop via the serrated pathway. Stalkless serrated adenomas / polyps (SSA / P) and conventional serrated adenomas (TSA) are precursor lesions.

[0003] In order to effectively prevent and treat colonic abnormalities, it is necessary to accurately identify and classify the types of colonic abnormalities. However, traditional methods cannot accurately and thoroughly identify and classify them.

[0004] Therefore, how to accurately and thoroughly identify and classify the abnormal types of the colon is a technical problem that urgently needs to be solved in the field of assistive medical technology. Summary of the Invention

[0005] This application provides a colon image processing method, apparatus, and related equipment, aiming to solve the technical problem of how to accurately and thoroughly identify and classify abnormal types of the colon.

[0006] On one hand, this application provides a colon image processing method, the method comprising:

[0007] The first segmented image of the target object in the endoscopic image, the boundary coordinates of the target object, the background mucosa image near the target object, and the surface structure category features of the first segmented image are obtained. The endoscopic image is an image taken in advance for the target site of the patient, and the target site is the colon.

[0008] Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, the quantization coefficient of the target object relative to the background mucosa color is determined, wherein the first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image.

[0009] The target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image is obtained. The superimposed image is formed by superimposing the second segmentation image and the endoscopic image. The second segmentation image is obtained by segmenting the initial blood vessels in the first segmentation image. The target blood vessels are the blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels.

[0010] Based on the quantization coefficient, target mean, and surface structure category characteristics, the anomaly type of the target object is determined.

[0011] In one possible implementation of this application, obtaining the target mean of the mean set of the third color feature vectors of all target blood vessels in the overlay image includes:

[0012] The blood vessels in the first segmented image are segmented to obtain the second segmented image;

[0013] Based on the connected components, all blood vessels in the second segmented image are extracted to obtain a blood vessel segmentation map for each blood vessel. The blood vessel segmentation map of each blood vessel is then superimposed on the endoscopic image to obtain a superimposed image.

[0014] Calculate the mean of the fourth color feature vector set of each blood vessel in the superimposed image to obtain the mean set, and calculate the mean and variance of the mean set;

[0015] Based on the mean and variance of the mean set, all target blood vessels are selected from all blood vessels;

[0016] Calculate the target mean of the mean set of the third color feature vectors of all target blood vessels.

[0017] In one possible implementation of this application, determining the quantization coefficient of the target object's color relative to the background mucosa based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image includes:

[0018] Obtain the mean of the first color feature vector set of the first segmented image;

[0019] Obtain the mean value of the second color feature vector set of the background mucosal image;

[0020] The mean of the first color feature vector set is compared with the mean of the second color feature vector set to obtain the mean ratio, and the mean ratio is used as the quantization coefficient of the target object relative to the background mucosa color.

[0021] In one possible implementation of this application, acquiring the background mucosal image near the target object includes:

[0022] Based on the boundary coordinates, determine the minimum bounding horizontal rectangle of the target object;

[0023] The image cropping range is determined based on the minimum circumscribed horizontal rectangle.

[0024] Based on the image cropping range, the endoscopic image is cropped to obtain a cropped image;

[0025] Remove the target object from the cropped image to obtain the background mucosa image near the target object.

[0026] In one possible implementation of this application, obtaining the surface structure category features of the first segmented image includes:

[0027] Based on a pre-trained surface structure category recognition model, the surface structure category of the first segmented image is identified, and the surface structure category features of the first segmented image are obtained.

[0028] In one possible implementation of this application, after determining the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category features, the method further includes:

[0029] The surface clarity features, boundary clarity features, regularity features, and preset attribute features of the target area of ​​the target object are obtained, wherein the preset attribute features are the presence or absence of black spots.

[0030] Based on the surface clarity features, boundary clarity features, regularity features, and preset attribute features, the degree of anomaly of the target object is determined.

[0031] In one possible implementation of this application, determining the anomaly degree of the target object based on the surface sharpness features, boundary sharpness features, regularity features, and preset attribute features includes:

[0032] The surface clarity features, boundary clarity features, regularity features, and preset attribute features are weighted and fitted to obtain the anomaly coefficient of the target object;

[0033] The degree of abnormality of the target object is determined based on the abnormality coefficient and the preset abnormality threshold.

[0034] On the other hand, this application provides a colon image processing apparatus, the apparatus comprising:

[0035] The first acquisition unit is used to acquire a first segmented image of a target object in an endoscopic image, the boundary coordinates of the target object, a background mucosal image near the target object, and surface structure category features of the first segmented image. The endoscopic image is an image taken in advance for a target part of the patient, and the target part is the colon.

[0036] The first determining unit is configured to determine the quantization coefficient of the target object relative to the background mucosa color based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, wherein the first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image.

[0037] The second acquisition unit is used to acquire the target mean of the mean set of the third color feature vector set of all target blood vessels in the overlay image. The overlay image is formed by overlaying the second segmentation image and the endoscopic image. The second segmentation image is an image obtained by segmenting the initial blood vessels in the first segmentation image. The target blood vessels are blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels.

[0038] The second determining unit is used to determine the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category characteristics.

[0039] In one possible implementation of this application, the second obtaining unit is specifically used for:

[0040] The blood vessels in the first segmented image are segmented to obtain the second segmented image;

[0041] Based on the connected components, all blood vessels in the second segmented image are extracted to obtain a blood vessel segmentation map for each blood vessel. The blood vessel segmentation map of each blood vessel is then superimposed on the endoscopic image to obtain a superimposed image.

[0042] Calculate the mean of the fourth color feature vector set of each blood vessel in the superimposed image to obtain the mean set, and calculate the mean and variance of the mean set;

[0043] Based on the mean and variance of the mean set, all target blood vessels are selected from all blood vessels;

[0044] Calculate the target mean of the mean set of the third color feature vectors of all target blood vessels.

[0045] In one possible implementation of this application, the first determining unit is specifically used for:

[0046] Obtain the mean of the first color feature vector set of the first segmented image;

[0047] Obtain the mean value of the second color feature vector set of the background mucosal image;

[0048] The mean of the first color feature vector set is compared with the mean of the second color feature vector set to obtain the mean ratio, and the mean ratio is used as the quantization coefficient of the target object relative to the background mucosa color.

[0049] In one possible implementation of this application, acquiring the background mucosal image near the target object specifically includes:

[0050] Based on the boundary coordinates, determine the minimum bounding horizontal rectangle of the target object;

[0051] The image cropping range is determined based on the minimum circumscribed horizontal rectangle.

[0052] Based on the image cropping range, the endoscopic image is cropped to obtain a cropped image;

[0053] Remove the target object from the cropped image to obtain the background mucosa image near the target object.

[0054] In one possible implementation of this application, obtaining the surface structure category features of the first segmented image specifically includes:

[0055] Based on a pre-trained surface structure category recognition model, the surface structure category of the first segmented image is identified, and the surface structure category features of the first segmented image are obtained.

[0056] In one possible implementation of this application, after determining the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category characteristics, the apparatus further includes:

[0057] The third acquisition unit is used to acquire the surface clarity features, boundary clarity features, regularity features, and preset attribute features of the target area of ​​the target object, wherein the preset attribute features are the presence or absence of black spots.

[0058] The third determining unit is used to determine the degree of anomaly of the target object based on the surface clarity features, boundary clarity features, regularity features, and preset attribute features.

[0059] In one possible implementation of this application, the third determining unit is specifically used for:

[0060] The surface clarity features, boundary clarity features, regularity features, and preset attribute features are weighted and fitted to obtain the anomaly coefficient of the target object;

[0061] The degree of abnormality of the target object is determined based on the abnormality coefficient and the preset abnormality threshold.

[0062] On the other hand, this application also provides a computer device, the computer device comprising:

[0063] One or more processors;

[0064] Memory; and

[0065] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the colon image processing method.

[0066] On the other hand, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the colon image processing method.

[0067] The colon image processing method provided in this application acquires a first segmented image of the target object in an endoscopic image, the boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmented image. The endoscopic image is an image pre-captured for the target site of the patient, and the target site is the colon. Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, a quantization coefficient of the target object's color relative to the background mucosa color is determined. The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image. The target mean of the mean set of the third color feature vector set of all target blood vessels in the overlaid image is obtained. The overlaid image is formed by overlaying the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels. Based on the quantization coefficient, the target mean, and the surface structure category features, the abnormal type of the target object is determined. Compared to traditional methods, when it is impossible to accurately determine the abnormal type of the colon, this application can accurately and thoroughly identify and classify the abnormal type of the target object on the colon by finding multiple effective indicators, specifically the surface structure category features of the first segmented image, the quantization coefficient of the target object relative to the background mucosa color, and the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic diagram of a scenario for the colon image processing system provided in an embodiment of this application;

[0070] Figure 2 This is a schematic flowchart of an embodiment of the colon image processing method provided in this application.

[0071] Figure 3 This is a schematic flowchart of an embodiment of determining the degree of abnormality of a target object provided in this application;

[0072] Figure 4 This is a schematic diagram illustrating the quantification of the irregularity of the target object provided in the embodiments of this application;

[0073] Figure 5 This is a schematic diagram of an embodiment of the colon image processing device provided in this application.

[0074] Figure 6 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0077] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0078] This application provides a colon image processing method, apparatus, and related equipment, which will be described in detail below.

[0079] like Figure 1 As shown, Figure 1 This is a schematic diagram of a colon image processing system provided in an embodiment of this application. The colon image processing system may include a computer device 100, which integrates a colon image processing unit, such as... Figure 1 Computer equipment 100.

[0080] In this embodiment, the computer device 100 is mainly used to acquire a first segmented image of a target object in an endoscopic image, the boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmented image. The endoscopic image is an image taken in advance for the target site of the patient, and the target site is the colon. Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, the quantization coefficient of the target object's color relative to the background mucosa color is determined. The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image. The target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image is acquired. The superimposed image is formed by superimposing the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels. Based on the quantization coefficient, the target mean, and the surface structure category features, the abnormal type of the target object is determined.

[0081] In this embodiment, the computer device 100 can be a terminal or a server. When the computer device 100 is a server, it can be a standalone server or a server network or server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, computers, network hosts, single network servers, multiple sets of network servers, or cloud servers constructed from multiple servers. The cloud server is constructed from a large number of computers or network servers based on cloud computing.

[0082] It is understood that when the computer device 100 in this embodiment is a terminal, the terminal used can be a device that includes both receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices, having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the computer device 100 may be a desktop terminal or a mobile terminal, and may also be a mobile phone, tablet computer, laptop computer, medical auxiliary instrument, etc.

[0083] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario for the solution in this application and is not intended to limit the application scenario of the solution in this application. Other application environments may include more than one. Figure 1 The number of computer devices shown is more or less, for example Figure 1Only one computer device is shown in the diagram. It is understood that the colon image processing system may also include one or more other computer devices, which are not specifically limited here.

[0084] In addition, such as Figure 1 As shown, the colon image processing system may also include a memory 200 for storing data, such as endoscopic images of the patient's target site and colon image processing data, such as colon image processing data during the operation of the colon image processing system.

[0085] It should be noted that, Figure 1 The schematic diagram of the colon image processing system shown is merely an example. The colon image processing system and scenario described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of colon image processing systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0086] Next, we will introduce the colon image processing method provided in the embodiments of this application.

[0087] In this embodiment of the colon image processing method, a colon image processing device is used as the execution subject. For simplicity and ease of description, this execution subject will be omitted in subsequent method embodiments. The colon image processing device is applied to a computer device. The method includes: acquiring a first segmented image of a target object in an endoscopic image, the boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmented image. The endoscopic image is an image pre-captured for a target area of ​​the patient, and the target area is the colon. Based on the mean of a first color feature vector set of the first segmented image and the mean of a second color feature vector set of the background mucosa image, the amount of color of the target object relative to the background mucosa is determined. The quantization coefficient is used to determine the abnormality type of the target object. The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosal image. The target mean of the mean set of the third color feature vector set of all target blood vessels in the overlaid image is obtained. The overlaid image is formed by overlaying the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are the blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels. Based on the quantization coefficient, the target mean, and the surface structure category features, the abnormality type of the target object is determined.

[0088] Please see Figures 2 to 6 , Figure 2This is a schematic flowchart of an embodiment of the colon image processing method provided in this application. The colon image processing method includes:

[0089] 201. Obtain the first segmented image of the target object in the endoscopic image, the boundary coordinates of the target object, the background mucosa image near the target object, and the surface structure category features of the first segmented image.

[0090] The endoscopic images are images taken in advance for a target area of ​​the patient, the target area being the colon, and the target objects including polyps and adenomas. For clarity, this application uses polyps as an example.

[0091] In this embodiment of the application, by way of example, an endoscope image can be identified by a pre-trained target object segmentation model to obtain the first segmentation image of the target object in the endoscope image and the boundary coordinates of the target object. Specifically, the target object segmentation model in this application is preferably Unet++, and the labels used for its training can be marked by professional endoscopists using rectangular boxes to mark the target objects on the patient's target site, such as marking polyps on the patient's colon with rectangular boxes.

[0092] In this embodiment of the application, by way of example, obtaining the background mucosal image near the target object includes steps A1 to A4:

[0093] A1. Based on the boundary coordinates, determine the minimum bounding horizontal rectangle of the target object;

[0094] Specifically, the boundary of the target object is composed of many points, each with its own coordinates. Find the x-coordinate of each point and compare them together to find the minimum and maximum values ​​of x. Then, draw two lines parallel to the x-axis through the corresponding two points. The y-coordinate is handled in the same way. Draw two lines parallel to the y-axis at the two points corresponding to the minimum and maximum values ​​of the y-coordinate. In this way, the four lines form a rectangle.

[0095] A2. Determine the image cropping range based on the minimum bounding horizontal rectangle;

[0096] Specifically, by expanding the smallest circumscribed horizontal rectangle by a preset numerical multiple, the area of ​​the rectangle of that numerical multiple is used as the image cropping range. Preferably, the numerical multiple is 1.5 times.

[0097] A3. Based on the image cropping range, crop the endoscopic image to obtain the cropped image;

[0098] The captured image includes the target object and the background mucosa near the target object.

[0099] A4. Remove the target object from the cropped image to obtain the background mucosa image near the target object.

[0100] Here, removing the target object from the cropped image can be understood as extracting the target object, thereby obtaining the background mucosal image near the target object.

[0101] In this embodiment of the application, obtaining the surface structure category features of the first segmented image can be specifically achieved by identifying the surface structure category of the first segmented image based on a pre-trained surface structure category recognition model, thereby obtaining the surface structure category features of the first segmented image, as shown below:

[0102]

[0103] The surface structure category recognition model in this application embodiment is preferably ResNet50, and the labels can be oval / tubular / branched, uniform surface structures, and the dataset is a segmentation map of colon polyps.

[0104] 202. Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, determine the quantization coefficient of the target object's color relative to the background mucosa.

[0105] The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image. Each color feature vector includes the values ​​of three sub-pixels: RGB, for example (r...). 11 g 11 b 11 ).

[0106] In this embodiment of the application, determining the quantization coefficient of the target object relative to the background mucosa color based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image includes steps B1 to B3:

[0107] B1. Obtain the mean of the first color feature vector set of the first segmented image;

[0108] Specifically, the first color feature vector set of the first segmented image can be obtained through the PIL's built-in getcolors() method. This first color feature vector set is specifically a list of color features, for example:

[0109] color1 = [(r 11 ,g 11 ,b 11 ),(r 12 ,g12 ,b 12 )…(r 1n ,g 1n ,b 1n )).

[0110] The mean 1 of its first color feature vector set is calculated as follows:

[0111]

[0112] B2. Obtain the mean value of the second color feature vector set of the background mucosal image;

[0113] Specifically, the mean of the second color feature vector set of the background mucosa image is obtained in the same way as the mean of the first color feature vector set of the first segmented image obtained in step B1 above. The details are not elaborated here, but can be found in the calculation process in step B1.

[0114] B3. Compare the mean of the first color feature vector set with the mean of the second color feature vector set to obtain the mean ratio, and use the mean ratio as the quantization coefficient of the target object relative to the background mucosa color.

[0115] Specifically, the quantization coefficient of the target object relative to the background mucosal color can be calculated using the following formula:

[0116]

[0117] 203. Obtain the target mean of the mean set of the third color feature vectors of all target blood vessels in the overlay image.

[0118] The superimposed image is formed by superimposing the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are the blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all the target blood vessels.

[0119] In this embodiment of the application, obtaining the target mean of the mean set of the third color feature vectors of all target blood vessels in the overlay image includes steps C1 to C5:

[0120] C1. Segment the blood vessels in the first segmented image to obtain the second segmented image;

[0121] Specifically, in this embodiment, the blood vessels in the first segmented image can be segmented using a target object surface blood vessel segmentation model to obtain a second segmented image. In this embodiment, the target object surface blood vessel segmentation model is preferably Unet++, and the labels used for its training can be marked by professional endoscopists using rectangular boxes on the surface blood vessels of the target object at the target location.

[0122] C2. Based on the connected components, extract all blood vessels in the second segmented image to obtain a blood vessel segmentation map for each blood vessel, and then overlay the blood vessel segmentation map of each blood vessel with the endoscopic image to obtain an overlaid image.

[0123] C3. Calculate the mean of the fourth color feature vector set of each blood vessel in the superimposed image to obtain the mean set, and calculate the mean and variance of the mean set;

[0124] Specifically, the fourth color feature vector set for each blood vessel can be a list of color features in the following format:

[0125] color vi =[(r vi0 ,g vi0 ,b vi0 ),(r vi1 ,g vi1 ,b vi1 )…(r vin ,g vin ,b vin )];

[0126] Following the same method as obtaining the mean of the first color feature vector set of the first segmented image in step B1 above, obtain the mean of the fourth color feature vector set of each blood vessel. 3i Then, we can obtain the set of means composed of the mean values ​​of the fourth color feature vector set of each blood vessel, as shown in the table below:

[0127] list mean =[mean 30 ,mean 31 …mean 3i …mean Mi ];

[0128] Then calculate the mean set list. mean mean As shown in the formula below:

[0129]

[0130] Calculate the mean of the set list mean Variance list mean_std, as shown in the following formula:

[0131]

[0132] C4. Based on the mean and variance of the mean set, select all target blood vessels from all blood vessels;

[0133] Specifically, by using the mean and variance of the mean set, blood vessels that meet the preset removal criteria are removed from all blood vessels to obtain the target blood vessels. The specific removal criteria are shown in the following formula:

[0134] or

[0135] C5. Calculate the target mean of the mean set of the third color feature vectors of all target blood vessels.

[0136] Specifically, the method for calculating the target mean can be found in section C3 above. The resulting target mean... The following formula:

[0137]

[0138] 204. Based on the quantization coefficient, target mean, and surface structure category characteristics, determine the anomaly type of the target object.

[0139] In this embodiment of the application, the above-mentioned multiple parameters can be comprehensively analyzed to determine the anomaly type of the target object, for example, a weighted fitting method can be used.

[0140] For example, the quantization coefficient, the target mean, and the surface structure category features are weighted according to the following formula:

[0141]

[0142] Specifically, the weighted results are analyzed. If λ = 0, it is a type I polyp; otherwise, if λ ≥ 1, it is a type II polyp.

[0143] In this embodiment of the application, when it is impossible to accurately determine the abnormal type of the colon, by finding multiple effective indicators, specifically the surface structure category features of the first segmented image, the quantization coefficient of the target object relative to the background mucosa color, and the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image, the abnormal type of the target object on the colon can be accurately and in detail identified and classified.

[0144] In another embodiment of this application, after determining the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category features, the method further includes:

[0145] 301. Obtain the surface clarity features, boundary clarity features, regularity features, and preset attribute features of the target area of ​​the target object. The preset attribute features are the presence or absence of black dots.

[0146] The target area is the region within the crypt.

[0147] Specifically, the surface sharpness features of the target object can be identified using a preset target object surface sharpness recognition model, a target object boundary sharpness recognition model, and a target object crypt black dot recognition model, respectively.

[0148]

[0149] Boundary sharpness features:

[0150]

[0151] and preset attribute features of the target area

[0152] And based on the connected components, find the geometric centroid of the segmentation mask of the target object, such as... Figure 4 As shown. Calculate the distance from the target object's boundary to its geometric centroid:

[0153] list d =[d1,d2,…d i …d n ];

[0154] Calculate distance list d variance:

[0155]

[0156] 302. Determine the degree of anomaly of the target object based on surface clarity features, boundary clarity features, regularity features, and preset attribute features.

[0157] In this embodiment of the application, determining the degree of anomaly of the target object based on the surface clarity features, boundary clarity features, regularity features, and preset attribute features includes steps D1 and D2:

[0158] D1. The surface clarity features, boundary clarity features, regularity features, and preset attribute features are weighted and fitted to obtain the anomaly coefficient of the target object;

[0159] Specifically, when the abnormality type of the target object is type I polyp, its abnormality coefficient is... The calculation method is as follows:

[0160]

[0161] λ1, λ2, λ3, and λ4 are obtained by training machine learning models such as decision trees and random forests.

[0162] When the abnormality type of the target object is type II polyp, its abnormality coefficient ψ is calculated as follows:

[0163] ψ=λ1·label4+λ2·label5+λ3·label6+λ4·label7;

[0164] λ1, λ2, λ3, and λ4 are obtained by training machine learning models such as decision trees and random forests.

[0165] D2. Based on the anomaly coefficient and the preset anomaly threshold, determine the anomaly degree of the target object.

[0166] It should be noted that when the anomaly type of the target object is a type I polyp, the anomaly degree of the target object is determined as follows:

[0167]

[0168] When the anomaly type of the target object is type II polyp, the degree of anomaly of the target object is determined as follows:

[0169]

[0170] According to the embodiments of this application, the degree of abnormality of the target object can be accurately determined through the above-disclosed solution.

[0171] To better implement the colon image processing method in the embodiments of this application, based on the colon image processing method, the embodiments of this application also provide a colon image processing device, such as... Figure 5 As shown, the colon image processing device 500 includes:

[0172] The first acquisition unit 501 is used to acquire a first segmented image of a target object in an endoscopic image, the boundary coordinates of the target object, a background mucosal image near the target object, and the surface structure category features of the first segmented image. The endoscopic image is an image taken in advance for a target part of the patient, and the target part is the colon.

[0173] The first determining unit 502 is used to determine the quantization coefficient of the target object relative to the background mucosa color based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosa image, wherein the first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosa image.

[0174] The second acquisition unit 503 is used to acquire the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image. The superimposed image is formed by superimposing the second segmentation image and the endoscope image. The second segmentation image is an image obtained by segmenting the initial blood vessels in the first segmentation image. The target blood vessels are blood vessels after screening the initial blood vessels. The mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels.

[0175] The second determining unit 504 is used to determine the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category characteristics.

[0176] In one possible implementation of this application, the second acquisition unit 503 is specifically used for:

[0177] The blood vessels in the first segmented image are segmented to obtain the second segmented image;

[0178] Based on the connected components, all blood vessels in the second segmented image are extracted to obtain a blood vessel segmentation map for each blood vessel. The blood vessel segmentation map of each blood vessel is then superimposed on the endoscopic image to obtain a superimposed image.

[0179] Calculate the mean of the fourth color feature vector set of each blood vessel in the superimposed image to obtain the mean set, and calculate the mean and variance of the mean set;

[0180] Based on the mean and variance of the mean set, all target blood vessels are selected from all blood vessels;

[0181] Calculate the target mean of the mean set of the third color feature vectors of all target blood vessels.

[0182] In one possible implementation of this application, the first determining unit 502 is specifically used for:

[0183] Obtain the mean of the first color feature vector set of the first segmented image;

[0184] Obtain the mean value of the second color feature vector set of the background mucosal image;

[0185] The mean of the first color feature vector set is compared with the mean of the second color feature vector set to obtain the mean ratio, and the mean ratio is used as the quantization coefficient of the target object relative to the background mucosa color.

[0186] In one possible implementation of this application, acquiring the background mucosal image near the target object specifically includes:

[0187] Based on the boundary coordinates, determine the minimum bounding horizontal rectangle of the target object;

[0188] The image cropping range is determined based on the minimum circumscribed horizontal rectangle.

[0189] Based on the image cropping range, the endoscopic image is cropped to obtain a cropped image;

[0190] Remove the target object from the cropped image to obtain the background mucosa image near the target object.

[0191] In one possible implementation of this application, obtaining the surface structure category features of the first segmented image specifically includes:

[0192] Based on a pre-trained surface structure category recognition model, the surface structure category of the first segmented image is identified, and the surface structure category features of the first segmented image are obtained.

[0193] In one possible implementation of this application, after determining the anomaly type of the target object based on the quantization coefficient, the target mean, and the surface structure category characteristics, the apparatus further includes:

[0194] The third acquisition unit is used to acquire the surface clarity features, boundary clarity features, regularity features, and preset attribute features of the target area of ​​the target object, wherein the preset attribute features are the presence or absence of black spots.

[0195] The third determining unit is used to determine the degree of anomaly of the target object based on the surface clarity features, boundary clarity features, regularity features, and preset attribute features.

[0196] In one possible implementation of this application, the third determining unit is specifically used for:

[0197] The surface clarity features, boundary clarity features, regularity features, and preset attribute features are weighted and fitted to obtain the anomaly coefficient of the target object;

[0198] The degree of abnormality of the target object is determined based on the abnormality coefficient and the preset abnormality threshold.

[0199] The colon image processing method provided in this application includes a first acquisition unit 501, used to acquire a first segmented image of a target object in an endoscopic image, the boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmented image. The endoscopic image is an image pre-captured for a target area of ​​the patient, and the target area is the colon. A first determination unit 502 is used to determine a quantization coefficient of the target object's color relative to the background mucosa based on the mean of a first color feature vector set of the first segmented image and the mean of a second color feature vector set of the background mucosa image. The first color feature vector set includes the color features of all pixels in the first segmented image. The second color feature vector set includes the color feature vectors of all pixels in the background mucosal image; the second acquisition unit 503 is used to acquire the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image, the superimposed image is formed by superimposing the second segmented image and the endoscopic image, the second segmented image is obtained by segmenting the initial blood vessels in the first segmented image, the target blood vessels are the blood vessels after screening the initial blood vessels, and the mean set includes the mean of the third color feature vector set of each blood vessel in all target blood vessels; the second determination unit 504 is used to determine the abnormal type of the target object based on the quantization coefficient, the target mean, and the surface structure category features. Compared with traditional methods, when it is impossible to accurately determine the abnormal type of the colon, this application can accurately and thoroughly identify and classify the abnormal type of the target object on the colon by comprehensively analyzing multiple effective indicators, specifically the surface structure category features of the first segmented image, the quantization coefficient of the target object relative to the background mucosal color, and the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image.

[0200] In addition to the colon image processing methods and apparatus described above, embodiments of this application also provide a computer device that integrates any of the colon image processing apparatuses provided in the embodiments of this application. The computer device includes:

[0201] One or more processors;

[0202] Memory; and

[0203] One or more applications, wherein the one or more applications are stored in the memory and configured by the processor to perform operations of any of the methods described in any of the embodiments of the colon image processing method described above.

[0204] This application also provides a computer device that integrates any of the colon image processing devices provided in this application. For example... Figure 6 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0205] The computer device may include components such as a processor 601 with one or more processing cores, a storage unit 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 6 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0206] The processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the storage unit 602, and by calling data stored in the storage unit 602, thereby providing overall monitoring of the computer device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.

[0207] Storage unit 602 can be used to store software programs and modules. Processor 601 executes various functional applications and data processing by running the software programs and modules stored in storage unit 602. Storage unit 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, storage unit 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, storage unit 602 may also include a memory controller to provide processor 601 with access to storage unit 602.

[0208] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0209] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0210] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in the embodiments of this application, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more applications into the storage unit 602 according to the following instructions, and the processor 601 runs the applications stored in the storage unit 602 to realize various functions, as follows:

[0211] The process involves acquiring a first segmented image of the target object from an endoscopic image, the boundary coordinates of the target object, a background mucosal image near the target object, and surface structure category features of the first segmented image. The endoscopic image is a pre-captured image of the target site on the patient, specifically the colon. Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosal image, a quantization coefficient is determined for the target object's color relative to the background mucosa. The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosal image. The process also involves acquiring the target mean of the mean set of the third color feature vector sets of all target blood vessels in a superimposed image. The superimposed image is formed by overlaying the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are those after filtering the initial blood vessels. The mean set includes the mean of the third color feature vector sets of all target blood vessels. Based on the quantization coefficient, the target mean, and the surface structure category features, the abnormality type of the target object is determined.

[0212] This application provides a colon image processing method. Compared with traditional methods, when it is impossible to accurately determine the abnormal type of the colon, this application finds multiple effective indicators, specifically the surface structure category features of the first segmented image, the quantization coefficient of the target object relative to the background mucosa color, and the target mean of the mean set of the third color feature vector set of all target blood vessels in the superimposed image. By comprehensively analyzing these indicators, the abnormal type of the target object on the colon can be accurately and in detail identified and classified.

[0213] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The computer-readable storage medium stores multiple instructions, which can be loaded by a processor to execute the steps in any of the colon image processing methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0214] The process involves acquiring a first segmented image of the target object from an endoscopic image, the boundary coordinates of the target object, a background mucosal image near the target object, and surface structure category features of the first segmented image. The endoscopic image is a pre-captured image of the target site on the patient, specifically the colon. Based on the mean of the first color feature vector set of the first segmented image and the mean of the second color feature vector set of the background mucosal image, a quantization coefficient is determined for the target object's color relative to the background mucosa. The first color feature vector set includes the color feature vectors of all pixels in the first segmented image, and the second color feature vector set includes the color feature vectors of all pixels in the background mucosal image. The process also involves acquiring the target mean of the mean set of the third color feature vector sets of all target blood vessels in a superimposed image. The superimposed image is formed by overlaying the second segmented image and the endoscopic image. The second segmented image is obtained by segmenting the initial blood vessels in the first segmented image. The target blood vessels are those after filtering the initial blood vessels. The mean set includes the mean of the third color feature vector sets of all target blood vessels. Based on the quantization coefficient, the target mean, and the surface structure category features, the abnormality type of the target object is determined.

[0215] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0216] The above provides a detailed description of a colon image processing method, apparatus, and related equipment provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of colon image processing, characterized by, The method comprises: obtaining a first segmentation image of a target object in an endoscopic image, boundary coordinates of the target object, a background mucosa image near the target object, and surface structure category features of the first segmentation image, the endoscopic image being an image pre-shot for a target site of a patient, the target site being a colon; determining a quantization coefficient of the target object relative to a background mucosa color based on a mean of a first color feature vector set of the first segmentation image and a mean of a second color feature vector set of the background mucosa image, wherein the first color feature vector set comprises color feature vectors of all pixel points in the first segmentation image, and the second color feature vector set comprises color feature vectors of all pixel points in the background mucosa image; obtaining a target mean of a mean set of a third color feature vector set of all target blood vessels in an overlay image, the overlay image being obtained by overlaying a second segmentation image and the endoscopic image, the second segmentation image being an image obtained by segmenting initial blood vessels in the first segmentation image, the target blood vessels being blood vessels obtained by screening the initial blood vessels, and the mean set comprising a mean of a third color feature vector set of each blood vessel in the all target blood vessels; determining an abnormal type of the target object based on the quantization coefficient, the target mean, and the surface structure category features; wherein the determining the quantization coefficient of the target object relative to the background mucosa color based on the mean of the first color feature vector set of the first segmentation image and the mean of the second color feature vector set of the background mucosa image comprises: obtaining the mean of the first color feature vector set of the first segmentation image; obtaining the mean of the second color feature vector set of the background mucosa image; comparing the mean of the first color feature vector set with the mean of the second color feature vector set to obtain a mean ratio, and taking the mean ratio as the quantization coefficient of the target object relative to the background mucosa color.

2. The colon image processing method of claim 1, wherein, The obtaining the target mean of the mean set of the third color feature vector set of all target blood vessels in the overlay image comprises: segmenting blood vessels in the first segmentation image to obtain a second segmentation image; extracting all blood vessels in the second segmentation image based on connected domains to obtain a blood vessel segmentation image of each blood vessel, and overlaying each blood vessel segmentation image with the endoscopic image to obtain an overlay image; calculating a mean of a fourth color feature vector set of each blood vessel in the overlay image to obtain a mean set, and calculating a mean and a variance of the mean set; screening all target blood vessels from the all blood vessels based on the mean and the variance of the mean set; calculating the target mean of the mean set of the third color feature vector set of the all target blood vessels.

3. The colon image processing method of claim 1, wherein, The obtaining the background mucosa image near the target object comprises: determining a minimum circumscribed horizontal rectangular frame of the target object based on the boundary coordinates; determining an image cropping range based on the minimum circumscribed horizontal rectangular frame; cropping the endoscopic image based on the image cropping range to obtain a cropped image; The target object in the cropped image is removed to obtain a background mucosa image near the target object.

4. The colon image processing method of claim 1, wherein, The surface structure category feature of the first segmented image comprises: A surface structure category of the first segmented image is identified based on a pre-trained surface structure category identification model to obtain a surface structure category feature of the first segmented image.

5. The colon image processing method of claim 1, wherein, After determining the abnormal type of the target object based on the quantization coefficient, the target mean value and the surface structure category feature, the method further comprises: A surface definition feature, a boundary definition feature, a regularity degree feature of the target object and a preset attribute feature of a target region of the target object are obtained, the preset attribute feature being a feature of whether a black point exists; An abnormal degree of the target object is determined based on the surface definition feature, the boundary definition feature, the regularity degree feature and the preset attribute feature.

6. The colon image processing method of claim 5, wherein, The determination of the abnormal degree of the target object based on the surface definition feature, the boundary definition feature, the regularity degree feature and the preset attribute feature comprises: The surface definition feature, the boundary definition feature, the regularity degree feature and the preset attribute feature are weighted and fitted to obtain an abnormal degree coefficient of the target object; The abnormal degree of the target object is determined based on the abnormal degree coefficient and a preset abnormal degree threshold.

7. A colon image processing apparatus characterized by comprising: The device comprises: A first acquisition unit is configured to acquire a first segmented image of a target object in an endoscopic image, boundary coordinates of the target object, a background mucosa image near the target object and a surface structure category feature of the first segmented image, the endoscopic image being an image pre-acquired for a target site of a patient, the target site being a colon. A first determination unit is configured to determine a quantization coefficient of the target object relative to a background mucosa color based on a mean value of a first color feature vector set of the first segmented image and a mean value of a second color feature vector set of the background mucosa image, wherein the first color feature vector set comprises color feature vectors of all pixel points in the first segmented image, and the second color feature vector set comprises color feature vectors of all pixel points in the background mucosa image. A second acquisition unit is configured to acquire a target mean value of a mean value set of a third color feature vector set of all target blood vessels in a superimposed image, the superimposed image being obtained by superimposing a second segmented image and the endoscopic image, the second segmented image being an image obtained by segmenting an initial blood vessel in the first segmented image, the target blood vessel being a blood vessel obtained by screening the initial blood vessel, and the mean value set comprising mean values of third color feature vector sets of the target blood vessels. A second determination unit is configured to determine an abnormal type of the target object based on the quantization coefficient, the target mean value and the surface structure category feature. The first determining unit is further configured to obtain a mean value of a first color feature vector set of the first segmented image; obtain a mean value of a second color feature vector set of the background mucosa image; compare the mean value of the first color feature vector set with the mean value of the second color feature vector set to obtain a mean value ratio; and use the mean value ratio as a quantization coefficient of the target object relative to the color of the background mucosa.

8. A computer device, comprising: The computer device comprises: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the colon image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the colon image processing method according to any one of claims 1 to 6.

Images