Method and apparatus for recognizing esophageal endoscopic video anomalies

Abstract

This application provides a method and apparatus for identifying abnormalities in esophageal endoscopy videos. The method includes: acquiring the associated address of a target entity and a preset high-incidence area of ​​entity abnormalities; determining whether the associated address belongs to the preset high-incidence area of ​​entity abnormalities; if the associated address does not belong to the preset high-incidence area of ​​entity abnormalities, acquiring multiple preset type features of the target entity; if the multiple preset type features meet preset abnormality conditions, acquiring multiple initial endoscopic images; obtaining multiple lesion segmentation regions; three-dimensional reconstruction to obtain a three-dimensional model of the esophagus; calculating the average cross-sectional area of ​​multiple cross sections of the three-dimensional model of the esophagus and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; if a first ratio of the average cross-sectional area to the maximum lesion area value is greater than a first preset value, then an esophageal endoscopy video abnormality is determined. This application can improve the accuracy of identifying abnormalities in esophageal endoscopy videos.

Description

Technical Field

[0001] This application mainly relates to the field of image processing technology, specifically to a method and apparatus for identifying abnormalities in esophageal endoscopy videos. Background Technology

[0002] Esophageal cancer screening and early diagnosis and treatment have always been emphasized by national health authorities. Endoscopy allows for direct visualization of changes in the esophageal mucosa, assessment of esophageal lesions, and the capture or recording of lesion images. Furthermore, methods such as staining and magnification can be used to evaluate the nature, location, boundaries, and extent of lesions, enabling screening and early diagnosis in one step. Current technology primarily relies on doctors observing esophageal endoscopy videos to identify abnormalities. However, manual observation may introduce errors, leading to inaccurate identification of abnormal esophageal endoscopy videos.

[0003] In other words, the accuracy of identifying abnormalities in esophageal endoscopy videos using existing technologies is not high. Summary of the Invention

[0004] This application provides a method and apparatus for identifying abnormalities in esophageal endoscopy videos, aiming to solve the problem of low accuracy in identifying abnormalities in esophageal endoscopy videos in the prior art.

[0005] Firstly, this application provides a method for identifying abnormalities in esophageal endoscopy videos, the method comprising:

[0006] Obtain the associated addresses of the target entity and preset high-incidence areas of entity anomalies, wherein the associated addresses include at least one of the birth address and the residence address;

[0007] Determine whether the associated address belongs to a preset high-incidence area of ​​entity anomalies;

[0008] If the associated address does not belong to the preset high-incidence area of ​​entity anomalies, then obtain multiple preset type features of the target entity;

[0009] If multiple preset type features meet preset abnormal conditions, then the target esophageal endoscopy video of the target entity is obtained;

[0010] Decode the endoscopic video of the target esophagus to obtain multiple initial endoscopic images;

[0011] Multiple initial endoscopic images were segmented to obtain multiple lesion segmentation regions;

[0012] A three-dimensional model of the esophagus was obtained by reconstructing multiple initial endoscopic images.

[0013] Calculate the average cross-sectional area of ​​multiple cross sections of the esophageal three-dimensional model, and the maximum lesion area of ​​the largest lesion segmentation region among the multiple lesion segmentation regions;

[0014] If the first ratio of the average cross-sectional area to the maximum lesion area is greater than the first preset value, then the esophageal endoscopy video is determined to be abnormal.

[0015] Optionally, before performing three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus, the following steps are included:

[0016] The target endoscopic image is obtained from multiple initial endoscopic images. The target endoscopic image is an initial endoscopic image with lesion segmentation regions.

[0017] The target endoscopic image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image;

[0018] For each hypovascular segmentation region, multiple rays originating from the centroid of the hypovascular segmentation region are used to divide the hypovascular segmentation region into multiple sub-segments, with the same angle between adjacent rays.

[0019] For each hypovascular segmentation region, calculate the ratio of the area difference between the largest and smallest sub-segmentation regions in the hypovascular segmentation region to the area of ​​the hypovascular segmentation region.

[0020] Calculate the area ratio variance of each hypovascular segmentation region and the sum of the areas of multiple hypovascular segmentation regions;

[0021] Based on the area ratio variance and the area sum, abnormal parameters of the hypovascular segmentation region are determined;

[0022] The first target abnormal parameter is determined based on the abnormal parameters of the segmented region with poor blood vessels;

[0023] If the first target abnormal parameter is greater than the second preset value, then three-dimensional reconstruction is performed based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

[0024] Optionally, determining the first target abnormal parameter based on the abnormal parameters of the hypovascular segmentation region includes:

[0025] The target endoscopic image is input into the blood vessel segmentation model to obtain the blood vessel segmentation region on the target endoscopic image;

[0026] The Zhang-Suen thinning algorithm is used to process the segmented blood vessel region to obtain the centerline of the blood vessel corresponding to the segmented blood vessel region.

[0027] Corner point detection is performed on the blood vessel centerline, and the detected corner points are used to divide the blood vessel centerline into multiple blood vessel sub-centerlines;

[0028] By performing circular arc fitting on the centerline of each sub-vessel, the vessel curve corresponding to the centerline of each sub-vessel is obtained;

[0029] Calculate the maximum curvature and average curvature of multiple blood vessel curves;

[0030] Obtain the proportion of distorted blood vessel curves among the multiple blood vessel curves, wherein a distorted blood vessel curve is a blood vessel curve among the multiple blood vessel curves whose ratio of the maximum curvature to the average curvature is greater than a third preset value;

[0031] The abnormal vascular parameters are determined based on the proportion of distorted vascular curves.

[0032] The second target abnormal parameter is determined based on the abnormal parameters of the hypovascular segmentation region and the abnormal parameters of the blood vessels.

[0033] Optionally, determining the abnormal vascular parameters based on the proportion of the tortuous vascular curve includes:

[0034] Obtain each sub-region of a blood vessel corresponding to the center line of each sub-vessel;

[0035] Obtain multiple vessel diameters corresponding to multiple vessel sub-regions;

[0036] Calculate the first average value of multiple blood vessel diameters that are greater than the fourth preset value and the second average value of multiple blood vessel diameters that are not greater than the fourth preset value;

[0037] Calculate the ratio of the diameter average of the first average and the second average;

[0038] Vascular abnormality parameters are determined based on the ratio of the proportion of tortuous blood vessel curves to the average diameter.

[0039] Optionally, obtaining multiple vessel diameters corresponding to multiple vessel sub-regions includes:

[0040] Obtain multiple normals to the centerline of the blood vessel sub-line;

[0041] Obtain the distance between the two intersection points of the normal and the blood vessel sub-region corresponding to the blood vessel sub-centerline;

[0042] The average distance between the intersection points corresponding to multiple normal lines is determined as the blood vessel diameter of the blood vessel sub-region corresponding to the blood vessel sub-center line.

[0043] Optionally, before inputting the target endoscopic image into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image, the following steps are included:

[0044] Convert the target endoscopic image into an HSV mode image;

[0045] Obtain the first pixel value and corresponding first occurrence frequency of the HSV mode image with the highest frequency in the H channel, the second pixel value and corresponding second occurrence frequency of the HSV mode image with the highest frequency in the S channel, and the third pixel value and corresponding third occurrence frequency of the HSV mode image with the highest frequency in the V channel.

[0046] The frequency quantification parameters are determined based on the average and maximum frequencies of the first, second, and third occurrence frequencies.

[0047] The pixel quantization parameters are determined based on the maximum pixel value of the first pixel value, the second pixel value, and the third pixel value.

[0048] The color feature quantization parameters are determined based on the pixel quantization parameters and the frequency quantization parameters.

[0049] The anomaly parameters of the second target are determined based on the quantization parameters of color features;

[0050] If the second target abnormal parameter is greater than the fifth preset value, the target endoscopic image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image.

[0051] Optionally, determining the second target anomaly parameter based on the color feature quantization parameter includes:

[0052] Obtain the image pixel values ​​of each pixel point on the target endoscope image;

[0053] The surface roughness feature quantization parameters and surface texture feature quantization parameters are determined based on the pixel values ​​of each image.

[0054] The second target anomaly parameters are determined based on the color feature quantization parameters, surface roughness feature quantization parameters, and surface texture feature quantization parameters.

[0055] Secondly, this application provides a device for identifying abnormalities in esophageal endoscopy videos, the device comprising:

[0056] The first acquisition unit is used to acquire the associated address of the target entity and the preset high-incidence area of ​​entity anomalies, wherein the associated address includes at least one of the birth address and the residence address;

[0057] The judgment unit is used to determine whether the associated address belongs to a preset high-incidence area of ​​entity anomalies;

[0058] The second acquisition unit is used to acquire multiple preset type features of the target entity if the associated address does not belong to a preset high-incidence area of ​​entity anomalies.

[0059] The third acquisition unit is used to acquire the target esophageal endoscopy video of the target entity if multiple preset type features meet preset abnormal conditions.

[0060] The decoding unit is used to decode the target esophageal endoscopy video to obtain multiple initial endoscopic images;

[0061] The segmentation unit is used to segment lesions from multiple initial endoscopic images to obtain multiple lesion segmentation regions;

[0062] The reconstruction unit is used to perform three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

[0063] The calculation unit is used to calculate the average cross-sectional area of ​​multiple cross-sections of the esophageal three-dimensional model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions;

[0064] The determination unit is used to determine that the esophageal endoscopy video is abnormal if the first ratio of the average cross-sectional area to the maximum lesion area is greater than a first preset value.

[0065] Thirdly, this application provides a computer device, the computer device comprising:

[0066] One or more processors;

[0067] Memory; and

[0068] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for identifying esophageal endoscopic video abnormalities as described in any one aspect.

[0069] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the method for identifying abnormalities in esophageal endoscopy video as described in any one of the first aspects.

[0070] This application provides a method and apparatus for identifying abnormalities in esophageal endoscopy videos. The method includes: acquiring the associated address of a target entity and a preset high-incidence area for entity abnormalities, wherein the associated address includes at least one of a birth address and a residential address; determining whether the associated address belongs to the preset high-incidence area for entity abnormalities; if the associated address does not belong to the preset high-incidence area for entity abnormalities, acquiring multiple preset type features of the target entity; if the multiple preset type features meet preset abnormality conditions, acquiring the target esophageal endoscopy video of the target entity; decoding the target esophageal endoscopy video to obtain multiple initial endoscopic images; segmenting the multiple initial endoscopic images to obtain multiple lesion segmentation regions; performing three-dimensional reconstruction based on the multiple initial endoscopic images to obtain a three-dimensional model of the esophagus; calculating the average cross-sectional area of ​​multiple cross sections of the three-dimensional esophageal model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; if a first ratio of the average cross-sectional area to the maximum lesion area value is greater than a first preset value, then an esophageal endoscopy video abnormality is determined. This application can improve the accuracy of identifying abnormalities in esophageal endoscopy videos. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This is a schematic diagram of a scenario for the esophageal endoscopy video abnormality recognition system provided in this application embodiment;

[0073] Figure 2 This is a flowchart illustrating an embodiment of the method for identifying abnormalities in esophageal endoscopy video provided in this application.

[0074] Figure 3 This is a schematic diagram of a hypovascular segmentation region being divided into multiple sub-segmentation regions in one embodiment of the method for identifying abnormalities in esophageal endoscopy video provided in this application.

[0075] Figure 4 This is a schematic diagram of an embodiment of the esophageal endoscopy video abnormality identification device provided in this application.

[0076] Figure 5 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0077] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0078] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0079] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0080] This application provides a method and apparatus for identifying abnormalities in esophageal endoscopy video, which will be described in detail below.

[0081] Please see Figure 1 , Figure 1 This is a schematic diagram of a scenario for an esophageal endoscopy video abnormality recognition system provided in an embodiment of this application. The esophageal endoscopy video abnormality recognition system may include a computer device 100, which integrates an esophageal endoscopy video abnormality recognition device.

[0082] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0083] In this embodiment, the computer device 100 described above can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device 100 can be a desktop computer, a portable computer, a network server, a handheld computer (Personal Digital Assistant, PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the type of computer device 100.

[0084] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the image. It is understood that the system for identifying abnormalities in esophageal endoscopy video may also include one or more other computer devices capable of processing data, which are not specifically limited here.

[0085] In addition, such as Figure 1 As shown, the esophageal endoscopy video abnormality recognition system may also include a memory 200 for storing data.

[0086] It should be noted that, Figure 1 The schematic diagram of the esophageal endoscopy video abnormality recognition system shown is merely an example. The esophageal endoscopy video abnormality recognition system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of the esophageal endoscopy video abnormality recognition system and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0087] First, this application provides a method for identifying abnormalities in esophageal endoscopy videos. The method includes: acquiring the associated address of a target entity and a preset high-incidence area for entity abnormalities, wherein the associated address includes at least one of a birth address and a residential address; determining whether the associated address belongs to a preset high-incidence area for entity abnormalities; if the associated address does not belong to a preset high-incidence area for entity abnormalities, acquiring multiple preset type features of the target entity; if the multiple preset type features meet preset abnormality conditions, acquiring the target esophageal endoscopy video of the target entity; decoding the target esophageal endoscopy video to obtain multiple initial endoscopic images; segmenting the multiple initial endoscopic images to obtain multiple lesion segmentation regions; performing three-dimensional reconstruction based on the multiple initial endoscopic images to obtain a three-dimensional model of the esophagus; calculating the average cross-sectional area of ​​multiple cross sections of the three-dimensional esophageal model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; if a first ratio of the average cross-sectional area to the maximum lesion area value is greater than a first preset value, then an esophageal endoscopy video abnormality is determined.

[0088] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the method for identifying abnormalities in esophageal endoscopy video in this application. The method for identifying abnormalities in esophageal endoscopy video includes the following steps S201~S209:

[0089] S201. Obtain the associated address of the target entity and the preset high-incidence area of ​​entity anomalies.

[0090] The associated address includes at least one of the birth address and the residence address. The target entity is any patient. The preset high-incidence areas for entity anomalies refer to multiple map regions that are prone to esophageal cancer.

[0091] In one specific embodiment, the preset high-incidence areas of entity anomalies include multiple high-incidence map regions. The centroid coordinates of the multiple high-incidence map regions are then... The data is stored in a format that changes over time, allowing for updates to the centroid coordinates of multiple high-incidence map areas.

[0092] S202. Determine whether the associated address belongs to the preset high-incidence area of ​​entity anomalies.

[0093] In one specific embodiment, the coordinates of the associated address are: ,calculate The Euclidean distance between the associated address and the centroid coordinates of each high-incidence map area. If the associated address satisfies the Euclidean distance between the associated address and the centroid coordinates of any high-incidence map area... If the associated address does not belong to a pre-defined high-incidence area for entity anomalies, the target entity is considered to be a person from a high-incidence area. Conversely, if the associated address does not belong to a pre-defined high-incidence area for entity anomalies, the target entity is considered not to be a person from a high-incidence area. It can also be determined whether the coordinates of the associated address fall within any of multiple high-incidence map areas. If the coordinates of the associated address fall within any of multiple high-incidence map areas, the associated address is determined to belong to a pre-defined high-incidence area for entity anomalies.

[0094] S203. If the associated address does not belong to the preset high-incidence area of ​​entity anomalies, then obtain multiple preset type features of the target entity.

[0095] In this application embodiment, several preset type features include whether the age is greater than 40 years old, whether there is a family history of esophageal cancer, whether the individual has esophageal precancerous disease, whether there is precancerous disease, whether there is a history of head and neck tumors, whether there are high-risk factors, whether the individual eats hot food, whether the individual drinks alcohol for a long time, whether the individual smokes for a long time, whether the individual eats too fast, whether the individual has a high-salt diet, and whether the individual has missing at least 4 teeth.

[0096] age, ;

[0097] Does your family have a history of esophageal cancer? ;

[0098] Whether an individual has precancerous esophageal lesions can be determined by loading a pre-trained precancerous esophageal lesion recognition model and identifying whether precancerous esophageal lesions are present in the initial endoscopic images. ;

[0099] To determine if a lesion is precancerous, a pre-trained precancerous lesion recognition model is loaded to identify whether precancerous lesions are present in the initial endoscopic images. ;

[0100] Does the patient have a history of head and neck tumors? ;

[0101] Are there any high-risk factors? .

[0102] S204. If multiple preset type features meet preset abnormal conditions, then obtain the target esophageal endoscopy video of the target entity.

[0103] In a specific embodiment, if any one of the preset type features of the target entity has a eigenvalue of 'yes', then the multiple preset type features are determined to satisfy a preset anomaly condition. Specifically, the eigenvalues ​​of the multiple preset type features are summed to obtain the sum of the eigenvalues. ,

[0104] ;

[0105] like If multiple preset type features do not meet the preset exception conditions, then... If multiple preset type features satisfy the preset exception conditions, then...

[0106] S205. Decode the endoscopic video of the target esophagus to obtain multiple initial endoscopic images.

[0107] Among them, multiple initial endoscopic images are multiple frame images obtained by decoding the target esophageal endoscopic video.

[0108] S206. Perform lesion segmentation on multiple initial endoscopic images to obtain multiple lesion segmentation regions.

[0109] S207. Based on multiple initial endoscopic images, perform three-dimensional reconstruction to obtain a three-dimensional model of the esophagus.

[0110] In one specific embodiment, a three-dimensional reconstruction is performed based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus. Prior to this, the following steps are included:

[0111] (1) Obtain the target endoscopic image from multiple initial endoscopic images. The target endoscopic image is an initial endoscopic image with a lesion segmentation region.

[0112] (2) Input the target endoscopic image into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image.

[0113] The avascular region, or AVA region, is an area lacking blood supply. Unet++ is the preferred choice for training the avascular region segmentation model, with professional physicians delineating the boundaries of the avascular region in esophageal images.

[0114] (3) For each hypovascular segmentation region, multiple rays starting from the centroid of the hypovascular segmentation region are used to divide the hypovascular segmentation region into multiple sub-segmentation regions, with the angle between adjacent rays being the same.

[0115] Each hypovascular segmentation region The area is A hypovascular segmentation region A schematic diagram of the resulting sub-regions is shown below. Figure 3 As shown.

[0116] (4) For each hypovascular segmentation region, calculate the ratio of the area difference between the largest and smallest sub-segmentation regions in the hypovascular segmentation region to the area of ​​the hypovascular segmentation region.

[0117] For each hypovascular segmentation region, calculate the maximum sub-segmentation region. and minimum sub-segmentation region The area difference between them accounts for the area of ​​the non-vascular segmentation region area ratio .

[0118] Among them, the area ratio As shown in the formula below,

[0119] ,

[0120] (5) Calculate the variance of the area ratio of each hypovascular segmentation region and the sum of the areas of multiple hypovascular segmentation regions.

[0121] Area ratio variance As shown in the formula below,

[0122] .

[0123] area and N represents a hypovascular segmentation region. The number of neutron-segmented regions.

[0124] (6) Determine the abnormal parameters of the hypovascular segmentation region based on the area ratio variance and area sum.

[0125] In one specific embodiment, the sum of the area ratio variance and the area sum is determined as the abnormal parameter of the hypovascular segmentation region. Specifically, abnormal parameters in the hypovascular segmentation region. .

[0126] (7) Determine the first target abnormal parameters based on the abnormal parameters of the segmented region with poor blood vessels.

[0127] In one specific embodiment, abnormal parameters of the hypovascular segmentation region are... The first target anomaly parameter P.

[0128] (8) If the first target abnormal parameter is greater than the second preset value, then three-dimensional reconstruction is performed based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

[0129] If the first target abnormal parameter P is greater than the second preset value, then three-dimensional reconstruction is performed based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

[0130] Furthermore, the target endoscopic image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image. This can include, beforehand, the following:

[0131] (1) Convert the target endoscopic image into an HSV mode image.

[0132] (2) Obtain the value of the first pixel with the highest frequency of occurrence in the H channel of the HSV mode image. and the corresponding first occurrence frequency The second pixel value with the highest frequency of occurrence in the S channel of an HSV mode image. and the corresponding second frequency The third pixel value with the highest frequency of occurrence in the V channel of an HSV mode image. and the corresponding third frequency .

[0133] (3) Determine the pixel quantization parameters based on the average and maximum frequency of the first occurrence frequency, the second occurrence frequency, and the third occurrence frequency.

[0134] Specifically, the ratio of the maximum frequency to the average frequency is defined as a frequency quantification parameter.

[0135] The frequency quantification parameters are shown in the following formula.

[0136]

[0137] (4) Determine the frequency quantification parameters based on the maximum pixel value of the first pixel value, the second pixel value, and the third pixel value.

[0138] The pixel quantization parameters are shown in the following formula.

[0139]

[0140] (5) Determine the color feature quantization parameters based on the pixel quantization parameters and the frequency quantization parameters.

[0141] Specifically, the pixel quantization parameters and frequency quantization parameters are weighted and summed according to preset weight coefficients to obtain the color feature quantization parameters. .

[0142] Among them, color feature quantization parameters The weighting coefficient for the frequency quantization parameter is 0.7, and the weighting coefficient for the pixel quantization parameter is 0.3.

[0143]

[0144] (6) Determine the second target anomaly parameters based on the color feature quantization parameters.

[0145] (7) If the second target abnormal parameter is greater than the fifth preset value, the target endoscope image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscope image.

[0146] If the second target anomaly parameter is greater than the fifth preset value, it indicates that the target entity may be abnormal.

[0147] Furthermore, based on the color feature quantization parameters, the second target anomaly parameters are determined, including:

[0148] (1) Obtain the image pixel values ​​of each pixel point on the target endoscope image.

[0149] (2) Determine the surface roughness feature quantization parameter and the surface texture feature quantization parameter based on the pixel values ​​of each image.

[0150] In this embodiment, image pixel values ​​not less than a pixel threshold are obtained. The first image pixel average value, and the image pixel values ​​less than the pixel threshold. The average pixel value of the first image is used as the second image pixel value. The ratio of the average pixel value of the first image to the average pixel value of the second image is determined as the surface roughness feature quantization parameter.

[0151] Surface roughness characteristic quantization parameters The calculation formula is as follows:

[0152] ,

[0153] Where W and H are the width and height of the image, respectively.

[0154] In this embodiment, the surface texture feature quantization parameter The calculation formula is as follows:

[0155]

[0156] (3) Determine the second target anomaly parameters based on the color feature quantization parameters, surface roughness feature quantization parameters, and surface texture feature quantization parameters.

[0157] In this embodiment, the summation of the color feature quantization parameter, the surface roughness feature quantization parameter, and the surface texture feature quantization parameter is determined as the second target anomaly parameter. ,

[0158] Second target abnormal parameters The calculation method is as follows:

[0159]

[0160] Furthermore, determining the first target abnormal parameter based on the abnormal parameters of the hypovascular segmentation region may include:

[0161] (1) Input the target endoscopic image into the blood vessel segmentation model to obtain the blood vessel segmentation region on the target endoscopic image.

[0162] For training the blood vessel segmentation model, Unet++ is preferred, and professional physicians delineate the blood vessel boundaries in the esophageal images.

[0163] (2) The Zhang-Suen thinning algorithm is used to process the segmented blood vessel region to obtain the center line of the blood vessel corresponding to the segmented blood vessel region.

[0164] (3) Detect the corner points of the vascular centerline and divide the vascular centerline into multiple vascular sub-centerlines based on the detected corner points.

[0165] Since the segmented vascular region is composed of multiple blood vessels, that is, multiple vascular sub-regions, the vascular centerline can be divided into multiple vascular sub-centerlines by intersection detection. These multiple vascular sub-centerlines correspond to multiple blood vessels, that is, multiple vascular sub-regions.

[0166] (4) Perform circular arc fitting on each sub-center line of the blood vessel to obtain the blood vessel curve corresponding to each sub-center line of the blood vessel.

[0167] (5) Calculate the maximum curvature and average curvature of multiple blood vessel curves.

[0168] Specifically, calculate the curvature of the y-curve for each blood vessel. Obtain the maximum curvature among the curvatures of each blood vessel curve. and average curvature .

[0169] (6) Obtain the proportion of tortuous blood vessel curves among multiple blood vessel curves, wherein a tortuous blood vessel curve is a blood vessel curve among multiple blood vessel curves whose ratio of the maximum curvature to the average curvature is greater than a third preset value.

[0170] Specifically, if the ratio of the maximum curvature to the average curvature of the blood vessel curve is... A value of 0 indicates high tortuosity of the blood vessel, i.e., a tortuous blood vessel curve; conversely, a value of 0 indicates low tortuosity. The number of tortuous blood vessel curves, m, is obtained, and the proportion K of tortuous blood vessel curves is calculated.

[0171] , where M is the number of multiple blood vessel curves.

[0172] (7) Determine the abnormal vascular parameters based on the proportion of the tortuous vascular curve.

[0173] In one specific embodiment, the proportion of distorted vascular curves is determined as a vascular abnormality parameter. .

[0174] In another specific embodiment, each sub-region of a blood vessel corresponding to the centerline of each sub-vessel is obtained; multiple blood vessel diameters corresponding to multiple sub-regions of a blood vessel are obtained; and the first average value of multiple blood vessel diameters that are greater than a fourth preset value is calculated respectively. The second average value of multiple blood vessel diameters not greater than the fourth preset value ; Calculate the first average value Second average The ratio of the average diameter D to the proportion of the tortuous blood vessel curve K and the average diameter D is used to determine the abnormal vascular parameters.

[0175] Specifically, multiple normals to the vessel sub-centerline are obtained; the distance between the intersection points of the normals and the vessel sub-regions corresponding to the vessel sub-centerline is obtained; and the average of the intersection points of the multiple normals is determined as the vessel diameter of the vessel sub-region corresponding to the vessel sub-centerline. Alternatively, the diameter measurement toolkit in Halcon can be used to measure the diameters of multiple vessels corresponding to multiple vessel sub-regions. .

[0176] Specifically, the fourth preset value is Abnormal vascular parameters .

[0177] (8) Determine the second target abnormal parameters based on the abnormal parameters of the segmented region with poor blood vessels and the abnormal parameters of blood vessels.

[0178] Specifically, abnormal parameters in the segmented region of hypovascularity are analyzed. and abnormal vascular parameters The sum of these values ​​is determined as the second target anomaly parameter P.

[0179] S208. Calculate the average cross-sectional area of ​​multiple cross sections of the esophageal three-dimensional model, and the maximum lesion area of ​​the largest lesion segmentation region among multiple lesion segmentation regions.

[0180] In this embodiment of the application, the average cross-sectional area of ​​multiple cross-sections of the esophagus three-dimensional model is calculated. Obtain the lesion area of ​​each segmented lesion region. The maximum lesion area value of the largest lesion segmentation region among multiple lesion segmentation regions is obtained. .

[0181] S209. If the first ratio of the average cross-sectional area to the maximum lesion area is greater than the first preset value, then the esophageal endoscopy video is determined to be abnormal.

[0182] Average cross-sectional area The first ratio to the largest lesion area value First ratio As shown in the formula below,

[0183] ,

[0184] If the first ratio of the average cross-sectional area to the maximum lesion area is... If the value exceeds the preset value, the esophageal endoscopy video is considered abnormal. Further, if the esophageal endoscopy video is abnormal, treatment will be initiated; otherwise, a gastroscopy follow-up examination will be performed every three years.

[0185] To better implement the method for identifying abnormalities in esophageal endoscopy video in the embodiments of this application, this application also provides a device for identifying abnormalities in esophageal endoscopy video, such as... Figure 4 As shown, the esophageal endoscopy video abnormality recognition device 300 includes:

[0186] The first acquisition unit 301 is used to acquire the associated address of the target entity and the preset abnormal high-incidence area of ​​the entity, wherein the associated address includes at least one of the birth address and the residence address;

[0187] Judgment unit 302 is used to determine whether the associated address belongs to a preset high-incidence area of ​​entity anomalies;

[0188] The second acquisition unit 303 is used to acquire multiple preset type features of the target entity if the associated address does not belong to a preset high-incidence area of ​​entity anomalies.

[0189] The third acquisition unit 304 is used to acquire the target esophageal endoscopy video of the target entity if multiple preset type features meet preset abnormal conditions.

[0190] Decoding unit 305 is used to decode the target esophageal endoscopy video to obtain multiple initial endoscopic images;

[0191] The segmentation unit 306 is used to segment lesions from multiple initial endoscopic images to obtain multiple lesion segmentation regions.

[0192] Reconstruction unit 307 is used to perform three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus;

[0193] The calculation unit 308 is used to calculate the average cross-sectional area of ​​multiple cross-sections of the esophageal three-dimensional model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions;

[0194] The determination unit 309 is used to determine that the esophageal endoscopy video is abnormal if the first ratio of the average cross-sectional area to the maximum lesion area is greater than a first preset value.

[0195] This application also provides a computer device that integrates any of the esophageal endoscopy video abnormality recognition devices provided in this application. The computer device includes:

[0196] One or more processors;

[0197] Memory; and

[0198] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor in the steps of the method for identifying esophageal endoscopy video abnormalities in any of the embodiments described above.

[0199] like Figure 5 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0200] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0201] Processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory 402, and by calling data stored in memory 402, thereby providing overall monitoring of the computer device. Optionally, processor 401 may include one or more processing cores; processor 401 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 401.

[0202] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0203] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0204] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0205] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0206] The process involves: acquiring the associated address of the target entity and a preset high-incidence area for entity anomalies, wherein the associated address includes at least one of the birth address and the residence address; determining whether the associated address belongs to a preset high-incidence area for entity anomalies; if the associated address does not belong to a preset high-incidence area for entity anomalies, acquiring multiple preset type features of the target entity; if the multiple preset type features meet preset anomaly conditions, acquiring the target esophageal endoscopy video of the target entity; decoding the target esophageal endoscopy video to obtain multiple initial endoscopic images; segmenting the multiple initial endoscopic images to obtain multiple lesion segmentation regions; performing three-dimensional reconstruction based on the multiple initial endoscopic images to obtain a three-dimensional model of the esophagus; calculating the average cross-sectional area of ​​multiple cross sections of the three-dimensional esophageal model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; if a first ratio of the average cross-sectional area to the maximum lesion area value is greater than a first preset value, then an anomaly is determined in the esophageal endoscopy video.

[0207] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0208] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the methods for identifying abnormalities in esophageal endoscopy videos provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0209] The process involves: acquiring the associated address of the target entity and a preset high-incidence area for entity anomalies, wherein the associated address includes at least one of the birth address and the residence address; determining whether the associated address belongs to a preset high-incidence area for entity anomalies; if the associated address does not belong to a preset high-incidence area for entity anomalies, acquiring multiple preset type features of the target entity; if the multiple preset type features meet preset anomaly conditions, acquiring the target esophageal endoscopy video of the target entity; decoding the target esophageal endoscopy video to obtain multiple initial endoscopic images; segmenting the multiple initial endoscopic images to obtain multiple lesion segmentation regions; performing three-dimensional reconstruction based on the multiple initial endoscopic images to obtain a three-dimensional model of the esophagus; calculating the average cross-sectional area of ​​multiple cross sections of the three-dimensional esophageal model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; if a first ratio of the average cross-sectional area to the maximum lesion area value is greater than a first preset value, then an anomaly is determined in the esophageal endoscopy video.

[0210] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0211] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0212] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0213] The above provides a detailed description of the method and apparatus for identifying abnormalities in esophageal endoscopy video provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying abnormalities in esophageal endoscopy video, characterized in that, The method for identifying abnormalities in esophageal endoscopy video includes: Obtain the associated addresses of the target entity and preset high-incidence areas of entity anomalies, wherein the associated addresses include at least one of the birth address and the residence address; Determine whether the associated address belongs to a preset high-incidence area of ​​entity anomalies; If the associated address does not belong to the preset high-incidence area of ​​entity anomalies, then obtain multiple preset type features of the target entity; If multiple preset type features meet preset abnormal conditions, then the target esophageal endoscopy video of the target entity is obtained; Decode the endoscopic video of the target esophagus to obtain multiple initial endoscopic images; Multiple initial endoscopic images were segmented to obtain multiple lesion segmentation regions; A three-dimensional model of the esophagus was obtained by reconstructing multiple initial endoscopic images. Calculate the average cross-sectional area of ​​multiple cross sections of the esophageal three-dimensional model, and the maximum lesion area of ​​the largest lesion segmentation region among the multiple lesion segmentation regions; If the first ratio of the average cross-sectional area to the maximum lesion area is greater than the first preset value, then the esophageal endoscopy video is determined to be abnormal. The step of performing three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus includes, prior to: The target endoscopic image is obtained from multiple initial endoscopic images. The target endoscopic image is an initial endoscopic image with lesion segmentation regions. The target endoscopic image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image; For each hypovascular segmentation region, multiple rays originating from the centroid of the hypovascular segmentation region are used to divide the hypovascular segmentation region into multiple sub-segments, with the same angle between adjacent rays. For each hypovascular segmentation region, calculate the ratio of the area difference between the largest and smallest sub-segmentation regions in the hypovascular segmentation region to the area of ​​the hypovascular segmentation region. Calculate the area ratio variance of each hypovascular segmentation region and the sum of the areas of multiple hypovascular segmentation regions; Based on the area ratio variance and the area sum, abnormal parameters of the hypovascular segmentation region are determined; The first target abnormal parameter is determined based on the abnormal parameters of the segmented region with poor blood vessels; If the first target abnormal parameter is greater than the second preset value, then three-dimensional reconstruction is performed based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

2. The method for identifying abnormalities in esophageal endoscopy video according to claim 1, characterized in that, The step of determining the first target abnormal parameter based on the abnormal parameters of the hypovascular segmentation region includes: The target endoscopic image is input into the blood vessel segmentation model to obtain the blood vessel segmentation region on the target endoscopic image; The Zhang-Suen thinning algorithm is used to process the segmented blood vessel region to obtain the centerline of the blood vessel corresponding to the segmented blood vessel region. Corner point detection is performed on the blood vessel centerline, and the detected corner points are used to divide the blood vessel centerline into multiple blood vessel sub-centerlines; By performing circular arc fitting on the centerline of each sub-vessel, the vessel curve corresponding to the centerline of each sub-vessel is obtained; Calculate the maximum curvature and average curvature of multiple blood vessel curves; Obtain the proportion of distorted blood vessel curves among the multiple blood vessel curves, wherein a distorted blood vessel curve is a blood vessel curve among the multiple blood vessel curves whose ratio of the maximum curvature to the average curvature is greater than a third preset value; The abnormal vascular parameters are determined based on the proportion of distorted vascular curves. The second target abnormal parameter is determined based on the abnormal parameters of the hypovascular segmentation region and the abnormal parameters of the blood vessels.

3. The method for identifying abnormalities in esophageal endoscopy video according to claim 2, characterized in that, The determination of abnormal vascular parameters based on the proportion of tortuous vascular curves includes: Obtain each sub-region of a blood vessel corresponding to the center line of each sub-vessel; Obtain multiple vessel diameters corresponding to multiple vessel sub-regions; Calculate the first average value of multiple blood vessel diameters that are greater than the fourth preset value and the second average value of multiple blood vessel diameters that are not greater than the fourth preset value; Calculate the ratio of the diameter average of the first average and the second average; Vascular abnormality parameters are determined based on the ratio of the proportion of tortuous blood vessel curves to the average diameter.

4. The method for identifying abnormalities in esophageal endoscopy video according to claim 3, characterized in that, The process of obtaining multiple blood vessel diameters corresponding to multiple blood vessel sub-regions includes: Obtain multiple normals to the centerline of the blood vessel sub-line; Obtain the distance between the two intersection points of the normal and the blood vessel sub-region corresponding to the blood vessel sub-centerline; The average distance between the intersection points corresponding to multiple normal lines is determined as the blood vessel diameter of the blood vessel sub-region corresponding to the blood vessel sub-center line.

5. The method for identifying abnormalities in esophageal endoscopy video according to claim 1, characterized in that, Before inputting the target endoscopic image into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image, the process includes: Convert the target endoscopic image into an HSV mode image; Obtain the first pixel value and corresponding first occurrence frequency of the HSV mode image with the highest frequency in the H channel, the second pixel value and corresponding second occurrence frequency of the HSV mode image with the highest frequency in the S channel, and the third pixel value and corresponding third occurrence frequency of the HSV mode image with the highest frequency in the V channel. The frequency quantification parameters are determined based on the average and maximum frequencies of the first, second, and third occurrence frequencies. The pixel quantization parameters are determined based on the maximum pixel value of the first pixel value, the second pixel value, and the third pixel value. The color feature quantization parameters are determined based on the pixel quantization parameters and the frequency quantization parameters. The anomaly parameters of the second target are determined based on the quantization parameters of color features; If the second target abnormal parameter is greater than the fifth preset value, the target endoscopic image is input into the hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image.

6. The method for identifying abnormalities in esophageal endoscopy video according to claim 5, characterized in that, The step of determining the second target anomaly parameter based on the color feature quantization parameter includes: Obtain the image pixel values ​​of each pixel point on the target endoscope image; The surface roughness feature quantization parameters and surface texture feature quantization parameters are determined based on the pixel values ​​of each image. The second target anomaly parameters are determined based on the color feature quantization parameters, surface roughness feature quantization parameters, and surface texture feature quantization parameters.

7. A device for identifying abnormalities in esophageal endoscopy video, characterized in that, The device for identifying abnormalities in esophageal endoscopy video includes: The first acquisition unit is used to acquire the associated address of the target entity and the preset high-incidence area of ​​entity anomalies, wherein the associated address includes at least one of the birth address and the residence address; The judgment unit is used to determine whether the associated address belongs to a preset high-incidence area of ​​entity anomalies; The second acquisition unit is used to acquire multiple preset type features of the target entity if the associated address does not belong to a preset high-incidence area of ​​entity anomalies. The third acquisition unit is used to acquire the target esophageal endoscopy video of the target entity if multiple preset type features meet preset abnormal conditions. The decoding unit is used to decode the target esophageal endoscopy video to obtain multiple initial endoscopic images; The segmentation unit is used to segment lesions from multiple initial endoscopic images to obtain multiple lesion segmentation regions; The reconstruction unit is used to perform three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus. The calculation unit is used to calculate the average cross-sectional area of ​​multiple cross-sections of the esophageal three-dimensional model and the maximum lesion area value of the largest lesion segmentation region among the multiple lesion segmentation regions; The determination unit is used to determine that the esophageal endoscopy video is abnormal if the first ratio of the average cross-sectional area to the maximum lesion area is greater than a first preset value. The device for identifying abnormalities in esophageal endoscopy videos is further configured to: acquire a target endoscopic image from multiple initial endoscopic images, wherein the target endoscopic image is an initial endoscopic image with lesion segmentation regions; input the target endoscopic image into a hypovascular region segmentation model to obtain multiple hypovascular segmentation regions on the target endoscopic image; for each hypovascular segmentation region, divide the hypovascular segmentation region into multiple sub-segmentation regions using multiple rays originating from the centroid of the hypovascular segmentation region, wherein the angle between adjacent two rays is the same; for each hypovascular segmentation region, calculate the area difference between the largest and smallest sub-segmentation regions in the hypovascular segmentation region as a percentage of the area of ​​the hypovascular segmentation region; calculate the area ratio variance of each hypovascular segmentation region and the sum of the areas of multiple hypovascular segmentation regions; determine the hypovascular segmentation region abnormality parameters based on the area ratio variance and the sum of the areas; determine a first target abnormality parameter based on the hypovascular segmentation region abnormality parameters; if the first target abnormality parameter is greater than a second preset value, perform three-dimensional reconstruction based on multiple initial endoscopic images to obtain a three-dimensional model of the esophagus.

8. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for identifying abnormalities in esophageal endoscopy video as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps in the method for identifying abnormalities in esophageal endoscopy video according to any one of claims 1 to 6.

Images