Gastroscope image processing method and device, electronic equipment and storage medium

Abstract

A gastroscopic image processing method and device, electronic equipment and storage medium are disclosed. The method comprises: acquiring an endoscopic image set of a target user collected through gastroscopic detection; identifying the endoscopic image set to determine first images belonging to a target object in the endoscopic image set; performing anomaly detection on the first images, and determining images with anomalies in the first images as second images; acquiring position information of target image features in the second images; performing clustering processing on the target image features according to the position information to obtain clustering results; and determining a gastroscopic detection result of the target user according to the clustering results. The embodiment can improve the detection accuracy of the target image features, thereby improving the processing efficiency of the gastroscopic images.

Description

Technical Field

[0001] This invention relates to the field of medical auxiliary technology, and in particular to a gastroscopy image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Esophageal and gastric varices are common diseases of the digestive system. In severe cases, rupture and bleeding of esophageal or gastric varices can directly endanger the patient's life. Therefore, accurate assessment of the severity of varices is crucial. Currently, gastroscopy is the simplest and most effective method for diagnosing esophageal and gastric varices. Furthermore, the grading of esophageal and gastric varices helps to understand the severity of the condition, enabling clinicians to take appropriate treatment measures.

[0003] However, gastroscopy is a demanding procedure that can easily lead to missed or misdiagnosed cases. It places high demands on the physician's energy, skills, and experience. The results of gastroscopy examinations vary depending on the physician's experience, resulting in strong subjectivity and unstable results, and also reducing the efficiency of gastroscopy image processing. Summary of the Invention

[0004] The purpose of this invention is to provide a gastroscopy image processing method, apparatus, electronic device, and storage medium to solve the technical problem of low detection efficiency in manual inspection.

[0005] In a first aspect, to achieve the above objectives, embodiments of the present invention provide a gastroscopy image processing method, comprising:

[0006] Acquire the set of endoscopic images of the target user obtained through gastroscopy;

[0007] The endoscopic image set is identified to determine the first image in the endoscopic image set that belongs to the target object;

[0008] Anomaly detection is performed on the first image, and the image containing anomalies in the first image is identified as the second image;

[0009] Obtain the location information of the target image features in the second image;

[0010] The target image features are clustered based on the location information to obtain clustering results;

[0011] Based on the clustering results, the gastroscopy results of the target user are determined.

[0012] Furthermore, acquiring the set of endoscopic images of the target user obtained through gastroscopy includes:

[0013] Acquire gastroscopy video captured during gastroscopy, wherein the gastroscopy video includes multiple frames of gastroscopy images;

[0014] Determine the effective region of each frame of the gastroscopy image, and determine the horizontally circumscribed rectangle of the effective region;

[0015] The gastroscopy image is cropped according to the horizontally circumscribed rectangle to obtain a qualified image;

[0016] The sharpness of each frame of the qualified image is calculated, and the qualified images with a sharpness greater than a preset threshold are determined as the endoscope image set.

[0017] Furthermore, the target objects include the esophagus and the fundus of the stomach;

[0018] The step of identifying the endoscopic image set and determining the first image in the endoscopic image set that belongs to the target object includes:

[0019] The trained target object classification model is invoked to classify the endoscopic image set to determine the first image in the endoscopic image set that belongs to the esophagus and the gastric fundus, respectively.

[0020] The trained target object classification model is trained based on endoscopic images acquired during gastroscopy and labeled data containing endoscopic images of the esophagus and gastric fundus.

[0021] Furthermore, the abnormality includes varicose veins;

[0022] The step of performing anomaly detection on the first image and identifying images containing anomalies in the first image as the second image includes:

[0023] The trained anomaly classification model is invoked to classify the first image to determine whether a second image containing varicose veins exists in the first image.

[0024] The trained anomaly classification model is trained based on the labeled data of the first image belonging to the esophagus or gastric fundus and the first image containing varicose veins.

[0025] Furthermore, the target image features include red features, and the location information includes a coordinate sequence;

[0026] The step of obtaining the location information of the target image features in the second image includes:

[0027] The trained image segmentation model is invoked to perform recognition processing on the second image to determine the red sign regions in the second image; wherein, the trained image segmentation model is trained based on the second image containing varicose veins and the labeled data of the second image with the red sign regions marked.

[0028] Based on the second image, determine the coordinate sequence of the boundary points of the red feature region in the second image.

[0029] Furthermore, the step of clustering the target image features based on the location information to obtain clustering results includes:

[0030] The coordinate sequence is sparsified to obtain a set of points corresponding to the coordinate sequence representing the red feature region;

[0031] Iterate through all points in the point set and the first distance between them and a preset first target point, and classify the first points whose first distance is less than a preset distance threshold into a first group, where the first target point is any point in the point set;

[0032] If there exists a second point whose first distance is not less than the preset distance threshold, then traverse all second points whose first distance is not less than the preset distance threshold and the preset second target point, and classify the second points whose second distance is less than the preset distance threshold into the second group, and the second target point is any one of the second points;

[0033] If there exists a third point where both the first distance and the second distance are not less than the preset distance threshold, then the third point is classified into the third group.

[0034] The number of all categorized groups is used as the clustering result.

[0035] Furthermore, determining the gastroscopy results of the target user based on the clustering results includes:

[0036] If the number of groups already classified is 1, then the target user's varicose vein redness level is determined to be Level 1;

[0037] If the number of groups that have been classified is 2, then the target user's varicose vein redness level is determined to be Level 2;

[0038] If the number of groups already classified is 3, then the target user's varicose vein redness level is determined to be level three;

[0039] The diffusion degree of the red sign corresponding to the third level is greater than that of the red sign corresponding to the second level, and the diffusion degree of the red sign corresponding to the second level is greater than that of the red sign corresponding to the first level.

[0040] In a second aspect, to address the same technical problem, embodiments of the present invention provide a gastroscopy image processing apparatus, comprising:

[0041] The first acquisition module is used to acquire the set of endoscopic images of the target user collected through gastroscopy.

[0042] The recognition module is used to recognize the endoscopic image set and determine the first image in the endoscopic image set that belongs to the target object;

[0043] The detection module is used to perform anomaly detection on the first image and identify the image containing anomalies in the first image as the second image;

[0044] The second acquisition module is used to acquire the location information of the target image features in the second image.

[0045] The clustering module is used to perform clustering processing on the target image features based on the location information to obtain clustering results;

[0046] The determination module is used to determine the gastroscopy test results of the target user based on the clustering results.

[0047] In a third aspect, in order to solve the same technical problem, embodiments of the present invention provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the memory being coupled to the processor, and the processor executing the computer program to implement the steps in the gastroscopy image processing method described in any of the above claims.

[0048] In a fourth aspect, in order to solve the same technical problem, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the steps in the gastroscopy image processing method described above.

[0049] This invention provides a gastroscopy image processing method, apparatus, electronic device, and storage medium. The method obtains the location information of target image features in the second image and performs clustering processing on the target image features based on the location information, which can improve the detection accuracy of target image features and thus improve the processing efficiency of gastroscopy images. Attached Figure Description

[0050] Figure 1 This is a schematic flowchart of a gastroscopy image processing method provided in an embodiment of the present invention;

[0051] Figures 2a-2d These are the first images belonging to the esophagus and the fundus of the stomach, respectively, provided in the embodiments of the present invention;

[0052] Figures 3a-3b This is a second image of varicose veins provided in an embodiment of the present invention;

[0053] Figures 4a-4c This is a second image provided in an embodiment of the present invention, showing the boundary points of the region marked with red symbols;

[0054] Figures 5a-5c Is it separate from Figures 4a-4c A schematic diagram of the coordinate sequence of the boundary points of the corresponding red feature area in the corresponding second image;

[0055] Figure 6 This is a schematic diagram of a gastroscopy image processing device provided in an embodiment of the present invention;

[0056] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0057] Figure 8 This is another structural schematic diagram of the electronic device provided in the embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0060] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0061] Among related technologies, gastroscopy is the simplest and most effective method for diagnosing esophageal and gastric varices. Furthermore, the grading of esophageal and gastric varices helps to understand the severity of the condition, facilitating appropriate treatment by clinicians. Clinically, physicians make a comprehensive diagnosis based on the image characteristics of the patient's gastroscopy examination. These image characteristics include the morphology, size, color, and redness of the veins. The identification and judgment of the redness significantly impacts the final assessment of the severity of the varices. However, the accurate grading of the redness through manual judgment involves subjective factors, thus affecting the accuracy of the assessment.

[0062] To resolve the technical problems existing in the related technologies, please refer to Figure 1 , Figure 1 This is a schematic flowchart of a gastroscopy image processing method provided in an embodiment of the present invention, such as... Figure 1 As shown, the gastroscopy image processing method provided in this embodiment of the invention includes steps 101 to 106;

[0063] Step 101: Obtain the set of endoscopic images of the target user collected through gastroscopy.

[0064] An endoscopic image set refers to a collection of multiple endoscopic images. These images are acquired through gastroscopy and reflect information about the digestive tract. Each target user corresponds to one endoscopic image set. Specifically, a video containing digestive tract information can be acquired using the probe of an electronic gastroscope. The video is then processed frame by frame and filtered. The filtering rules can be based on the clarity of each frame or the digestive tract information reflected in each frame to generate an endoscopic image set.

[0065] In this embodiment, the step of acquiring the endoscopy image set of the target user collected through gastroscopy detection specifically includes: acquiring the gastroscopy video collected through gastroscopy detection, the gastroscopy video including multiple frames of gastroscopy images; determining the effective region of each frame of the gastroscopy image, and determining the horizontal circumscribed rectangle of the effective region; cropping the gastroscopy image according to the horizontal circumscribed rectangle to obtain qualified images; calculating the sharpness of each frame of the qualified images, and determining the qualified images with a sharpness greater than a preset threshold as the endoscopy image set.

[0066] The effective region refers to the area containing the target object, such as the esophagus or the fundus of the stomach. Specifically, this can be achieved by pre-training an image segmentation model, for example, using a UNet++ network model. The endoscopy image is input into the image segmentation model, and the effective region is obtained based on the model's output. The corresponding horizontal circumscribed rectangle is then determined based on the boundary coordinates of the effective region. This horizontal circumscribed rectangle is used to crop the endoscopy image, resulting in a qualified image. Since the qualified image includes both the effective region and the circumscribed rectangle, the integrity of the effective region is ensured. Furthermore, the qualified image is a regular rectangle, facilitating subsequent image processing calculations. Image sharpness is a crucial indicator of image quality. The sharpness of a qualified image can be determined by calculating the Laplacian gradient function, variance function, and energy gradient function of the image. Then, an endoscopy image set is selected based on the sharpness of the qualified images; images with a sharpness greater than a preset threshold are identified as endoscopy images. Understandably, in this embodiment, by segmenting the effective region, redundant processing of the invalid region is avoided. At the same time, by selecting qualified images with higher clarity, the quality of the endoscopic image set is guaranteed, which is beneficial to improving the processing efficiency of subsequent endoscopic images.

[0067] Optionally, after confirming the effective region, this embodiment also needs to identify ME-NBI (magnifying endoscopy with narrow-band imaging) images and non-ME-NBI images within the effective region, so that the region belonging to the ME-NBI image is taken as the effective region. In this way, by using the region of the ME-NBI image as the effective region, the image features in the image can be magnified, thereby effectively improving the processing efficiency of subsequent endoscopic images.

[0068] Step 102: Identify the endoscopic image set and determine the first image in the endoscopic image set that belongs to the target object.

[0069] In this embodiment, the target objects include the esophagus and the fundus of the stomach. Specifically, the step of identifying the first image in the endoscopic image set that belongs to the target object involves: calling a trained target object classification model to classify the endoscopic image set to determine the first image in the endoscopic image set that belongs to the esophagus and the fundus of the stomach, respectively.

[0070] The trained target object classification model is trained based on endoscopic images acquired during gastroscopy, as well as labeled data of endoscopic images containing the esophagus and gastric fundus. Specifically, the trained target object classification model can be a model with high recognition accuracy, such as the ResNet50 image classification neural network model, which will not be listed here.

[0071] In this embodiment, the first image belonging to the esophagus refers to an endoscopic image containing information about the esophageal region within the digestive tract, and the first image belonging to the gastric fundus refers to an endoscopic image containing information about the gastric fundus within the digestive tract. Since the digestive tract comprises multiple regions, different regions reflect different image information.

[0072] It should be noted that the esophagus has a larger area than the fundus of the stomach. Typically, during the acquisition process of an electronic gastroscopy, the probe moves from the esophagus to the fundus of the stomach according to a preset path and speed. Because the area of ​​the esophagus is larger than that of the fundus, the number of first images belonging to the esophagus in the endoscopic image set is usually greater than the number of first images belonging to the fundus of the stomach. For details, please refer to [link to relevant documentation]. Figures 2a-2d , Figures 2a-2d These are first images belonging to the esophagus and the fundus of the stomach, respectively, provided in an embodiment of the present invention. Figure 2a and Figure 2b The first image belongs to the esophagus. Figure 2c and Figure 2d This is the first image belonging to the fundus of the stomach.

[0073] Step 103: Perform anomaly detection on the first image and identify the image containing anomalies in the first image as the second image.

[0074] In this embodiment, the abnormality includes varicose veins. Specifically, the step of performing anomaly detection on the first image and identifying the image containing the abnormality in the first image as the second image involves: calling a trained anomaly classification model to classify the first image to identify a second image containing varicose veins in the first image.

[0075] The trained anomaly classification model is trained based on labeled data from a first image belonging to the esophagus or gastric fundus, and a first image showing varicose veins. Specifically, the trained anomaly classification model can be two ResNet50 image classification neural network models. These two models are used to identify whether varicose veins are present in endoscopic images of the esophagus and gastric fundus, respectively, thereby effectively improving the accuracy of image recognition. For details, please refer to [link to relevant documentation]. Figures 3a-3b , Figures 3a-3b This is a second image of varicose veins provided in an embodiment of the present invention. Figure 3a The second image shows esophageal varices. Figure 3b The second image shows varicose veins in the fundus of the stomach.

[0076] Step 104: Obtain the location information of the target image features in the second image.

[0077] In this embodiment, the target image features include red features, and the location information includes a coordinate sequence. Specifically, the step of obtaining the location information of the target image features in the second image includes: calling a trained image segmentation model to perform recognition processing on the second image to determine the red feature region in the second image; and determining the coordinate sequence of the boundary points of the red feature region in the second image based on the second image.

[0078] The trained image segmentation model is trained based on a second image containing varicose veins and labeled data of the second image with red sign regions marked. The trained image segmentation model can be a UNet++ image segmentation neural network model or similar model capable of accurately identifying red sign regions in the second image and marking the boundary points of each red sign region in the second image.

[0079] As an optional embodiment, please refer to Figures 4a-4c and Figures 5a-5c , Figures 4a-4c This is a second image provided in an embodiment of the present invention, showing the boundary points of the region marked in red. Figures 5a-5c Is it separate from Figures 4a-4c A schematic diagram showing the coordinate sequence of the boundary points of the corresponding red feature region in the corresponding second image. Wherein, Figures 4a-4c These are the second images, output by the trained image segmentation model, marked with the boundary points of the red-labeled regions. Figure 5a Then it is by Figure 4a The diagram illustrates the coordinate sequence generated using the boundary points of the red-colored region in the image, with the corresponding second image as the plane coordinate system. Figure 5b Then it is by Figure 4b The diagram illustrates the coordinate sequence generated using the boundary points of the red-colored region in the image, with the corresponding second image as the plane coordinate system. Figure 5c Then it is by Figure 4c The diagram shows the coordinate sequence generated using the corresponding second image as a planar coordinate system, based on the boundary points of the red feature area in the image.

[0080] According to the above embodiments Figures 5a-5c This allows us to determine the coordinate sequence of the boundary points of each red feature region in the second image.

[0081] Step 105: Perform clustering processing on the target image features based on the location information to obtain clustering results.

[0082] In this embodiment, the clustering process mainly merges similar data into a group. Specifically, this embodiment mainly classifies points corresponding to adjacent coordinate sequences into the same group, so that the number of groups obtained by classification can represent the degree of diffusion of red signs in the second image. In turn, the degree of diffusion of red signs can be used to determine the level of red signs of varicose veins in the target user, thereby obtaining accurate gastroscopy results. This is beneficial for clinicians to take the most appropriate treatment methods and measures for the target user based on accurate gastroscopy results.

[0083] Specifically, the clustering algorithm used in the clustering process can be a hierarchical clustering algorithm or a partitioning-based method. As long as it is a clustering algorithm that can merge similar data into a group, it is within the protection scope of this invention.

[0084] As an optional embodiment, the step of clustering the target image features based on the location information to obtain clustering results specifically includes: performing sparsification processing on the coordinate sequence to obtain a set of points corresponding to the coordinate sequence representing the red feature region; traversing the first distance between all points in the point set and a preset first target point, and classifying the first points whose first distance is less than a preset distance threshold into a first group, where the first target point is any point in the point set; if there exists a second point whose first distance is not less than the preset distance threshold, then traversing the second distance between all second points whose first distance is not less than the preset distance threshold and a preset second target point, and classifying the second points whose second distance is less than the preset distance threshold into a second group, where the second target point is any point in the second group; if there exists a third point whose first distance and second distance are both not less than the preset distance threshold, then classifying the third point into a third group; and using the number of all classified groups as the clustering result.

[0085] To obtain a coordinate sequence that accurately represents the red feature regions, the coordinate sequence of the boundary points of each red feature region in the second image needs to be sparsified. Furthermore, the coordinate sequence of the boundary points of each red feature region in the second image can also be uniformly processed simultaneously, thereby improving the accuracy of representing the red feature regions.

[0086] After obtaining the coordinate sequence representing the red characteristic region, the points corresponding to these coordinate sequences can be acquired, thus obtaining a point set of all points corresponding to the coordinate sequences representing the red characteristic region. At this point, all points in the point set are not yet grouped; therefore, the following classification and grouping steps are performed:

[0087] Step 1: Pre-determine an ungrouped point as a target point, and classify all points within a preset range of this target point into the same group. Specifically, by traversing the first distance between the ungrouped points in the point set and the pre-determined first target point, points with a first distance less than a preset distance threshold are classified into the same group, i.e., the first group.

[0088] Step 2: If there are still ungrouped points, determine another target point among these ungrouped points. That is, pre-determine a second target point from all the second points, and classify all points within a preset range of the second target point into the same group. Specifically, by traversing the second distance between the ungrouped points in the point set and the pre-determined second target point, points with a second distance less than a preset distance threshold are classified into the same group, i.e., the second group.

[0089] Step 3: Repeat steps 1 and 2 above until all points have been classified and grouped.

[0090] Thus, through the above classification and grouping steps, the clustering process of the points corresponding to the coordinate sequences representing each red feature in the second image can be completed, resulting in at least one group after classification, and the number of groups after classification is taken as the clustering result.

[0091] As an optional embodiment, after obtaining the categorized groups, it is also necessary to iterate through the groups to see if the distance between any two points is within a preset range, and delete the points that exceed the preset range from the categorized groups, so as to redefine the deleted points as ungrouped points.

[0092] Please continue to refer to Figures 5a-5c ,according to Figure 5a It can be seen that, Figure 5a The points in the array are distributed across at least three different preset ranges, which can be 300*300 pixels. Figure 5a After the above classification and grouping steps, the corresponding coordinate sequence diagram can yield at least three or more groups; according to Figure 5b It can be seen that, Figure 5b The points are distributed across two different preset ranges, which can be 600*600. Therefore... Figure 5b After the above classification and grouping steps, the corresponding coordinate sequence diagrams can be divided into two groups; according to Figure 5c It can be seen that, Figure 5c The points in the image are distributed within a preset range, which can be 600*600. Therefore... Figure 5c After the above classification and grouping steps, the corresponding coordinate sequence diagram can be divided into one group. Specifically, the preset range can be set according to the number of points corresponding to the coordinate sequence, or according to actual technical requirements; no limitation is imposed here.

[0093] Step 106: Determine the gastroscopy results of the target user based on the clustering results.

[0094] In this embodiment, the step of determining the gastroscopy results of the target user based on the clustering results is as follows: if the number of groups that have been classified is 1, then the target user's varicose vein redness level is determined to be level 1; if the number of groups that have been classified is 2, then the target user's varicose vein redness level is determined to be level 2; if the number of groups that have been classified is 3, then the target user's varicose vein redness level is determined to be level 3.

[0095] The diffusion degree of the red sign corresponding to the third level is greater than that of the red sign corresponding to the second level, and the diffusion degree of the red sign corresponding to the second level is greater than that of the red sign corresponding to the first level.

[0096] It should be noted that when the number of categorized groups is greater than 3, the target user's varicose vein redness level is also determined to be level three.

[0097] For details, please continue to see Figures 5a-5c According to the above embodiments, Figure 5a If there are at least 3 corresponding groups, then it is possible to determine the relationship with... Figure 5a corresponding Figure 4a The varicose vein redness sign is grade three, that is... Figure 4a The degree of diffusion of the redness of the varicose veins is severe; Figure 5b If there are two corresponding groups, then it is possible to determine the relationship with... Figure 5b corresponding Figure 4b The varicose vein redness sign is grade two, that is... Figure 4b The degree of diffusion of the redness of the varicose veins is moderate; Figure 5c If there is one corresponding group, then it is possible to determine the relationship with... Figure 5c corresponding Figure 4c The redness of the varicose veins is grade one, that is... Figure 4c The redness of the varicose veins was diffuse to a mild degree.

[0098] Using the diffusion degree of the varicose vein red sign determined in the above embodiments as the gastroscopy results for the target user can improve the detection accuracy of the varicose vein red sign, thereby improving the processing efficiency of gastroscopy images. Furthermore, clinicians can accurately provide the most appropriate treatment methods and measures for the target user based on the gastroscopy results.

[0099] In summary, the gastroscopy image processing method provided by this invention includes acquiring a set of endoscopic images of a target user collected through gastroscopy, identifying the endoscopic image set, determining a first image belonging to the target object in the endoscopic image set, performing anomaly detection on the first image, identifying the image with anomalies in the first image as a second image, acquiring the position information of the target image features in the second image, performing clustering processing on the target image features based on the position information to obtain a clustering result, and determining the gastroscopy detection result of the target user based on the clustering result. Using this invention, the detection accuracy of target image features can be improved, thereby improving the processing efficiency of gastroscopy images.

[0100] Based on the method described in the above embodiments, this embodiment will be further described from the perspective of a gastroscopy image processing device. The gastroscopy image processing device can be implemented as an independent entity or integrated into an electronic device, such as a terminal, which may include a mobile phone, a tablet computer, etc.

[0101] Please see Figure 6 , Figure 6 This is a schematic diagram of a gastroscopy image processing device provided in an embodiment of the present invention, such as... Figure 6 As shown, the gastroscopy image processing device 600 provided in this embodiment of the invention includes:

[0102] The first acquisition module 601 is used to acquire the set of endoscopic images of the target user collected through gastroscopy.

[0103] In this embodiment, the first acquisition module 601 is specifically used for: acquiring gastroscopy video collected by gastroscopy detection, the gastroscopy video including multiple frames of gastroscopy images; determining the effective region of each frame of the gastroscopy image, and determining the horizontal circumscribed rectangle of the effective region; cropping the gastroscopy image according to the horizontal circumscribed rectangle to obtain qualified images; calculating the clarity of each frame of the qualified images, and determining the qualified images with a clarity greater than a preset threshold as the endoscopy image set.

[0104] The recognition module 602 is used to recognize the endoscopic image set and determine the first image in the endoscopic image set that belongs to the target object.

[0105] In this embodiment, the target objects include the esophagus and the fundus of the stomach. The identification module 602 is specifically used to: invoke a trained target object classification model to classify the endoscopic image set, thereby determining the first images in the endoscopic image set that belong to the esophagus and the fundus of the stomach, respectively.

[0106] The trained target object classification model is trained based on endoscopic images acquired during gastroscopy and labeled data containing endoscopic images of the esophagus and gastric fundus.

[0107] The detection module 603 is used to perform anomaly detection on the first image and identify the image containing anomalies in the first image as the second image.

[0108] In this embodiment, the abnormality includes varicose veins. The detection module 603 is specifically used to: invoke a trained abnormality classification model to classify the first image, thereby determining a second image containing varicose veins within the first image.

[0109] The trained anomaly classification model is trained based on the labeled data of the first image belonging to the esophagus or gastric fundus and the first image containing varicose veins.

[0110] The second acquisition module 604 is used to acquire the location information of the target image features in the second image.

[0111] In this embodiment, the target image features include red features, and the location information includes a coordinate sequence. The second acquisition module 604 is specifically used to: invoke a trained image segmentation model to perform recognition processing on the second image to determine the red feature region in the second image; and, based on the second image, determine the coordinate sequence of the boundary points of the red feature region in the second image.

[0112] The trained image segmentation model is trained based on the second image containing varicose veins and the labeled data of the second image with the red sign region marked.

[0113] Clustering module 605 is used to perform clustering processing on the target image features based on the location information to obtain clustering results.

[0114] In this embodiment, the clustering module 605 is specifically used for: performing sparsification processing on the coordinate sequence to obtain a set of points corresponding to the coordinate sequence representing the red feature region; traversing the first distance between all points in the set and a preset first target point, and classifying the first points whose first distance is less than a preset distance threshold into a first group, where the first target point is any point in the set; if there exists a second point whose first distance is not less than the preset distance threshold, then traversing the second distance between all second points whose first distance is not less than the preset distance threshold and a preset second target point, and classifying the second points whose second distance is less than the preset distance threshold into a second group, where the second target point is any point in the second group; if there exists a third point whose first distance and second distance are both not less than the preset distance threshold, then classifying the third point into a third group; and using the number of all classified groups as the clustering result.

[0115] The determination module 606 is used to determine the gastroscopy test results of the target user based on the clustering results.

[0116] In this embodiment, the determining module 606 is specifically used to: if the number of categorized groups is 1, then determine the target user's varicose vein redness level as level 1; if the number of categorized groups is 2, then determine the target user's varicose vein redness level as level 2; if the number of categorized groups is 3, then determine the target user's varicose vein redness level as level 3.

[0117] The diffusion degree of the red sign corresponding to the third level is greater than that of the red sign corresponding to the second level, and the diffusion degree of the red sign corresponding to the second level is greater than that of the red sign corresponding to the first level.

[0118] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0119] Additionally, please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device can be a mobile terminal such as a smartphone, tablet computer, or other similar device. Figure 7 As shown, the electronic device 700 includes a processor 701 and a memory 702. The processor 701 and the memory 702 are electrically connected.

[0120] The processor 701 is the control center of the electronic device 700. It connects various parts of the electronic device through various interfaces and lines. By running or loading the application program stored in the memory 702 and calling the data stored in the memory 702, it executes various functions of the electronic device 700 and processes data, thereby performing overall monitoring of the electronic device 700.

[0121] In this embodiment, the processor 701 in the electronic device 700 loads the instructions corresponding to the processes of one or more application programs into the memory 702 according to the following steps, and the processor 701 runs the application programs stored in the memory 702 to realize various functions:

[0122] Acquire the set of endoscopic images of the target user obtained through gastroscopy;

[0123] The endoscopic image set is identified to determine the first image in the endoscopic image set that belongs to the target object;

[0124] Anomaly detection is performed on the first image, and the image containing anomalies in the first image is identified as the second image;

[0125] Obtain the location information of the target image features in the second image;

[0126] The target image features are clustered based on the location information to obtain clustering results;

[0127] Based on the clustering results, the gastroscopy results of the target user are determined.

[0128] The electronic device 700 can implement the steps of any embodiment of the gastroscopy image processing method provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that any gastroscopy image processing method provided in the embodiments of the present invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0129] Please see Figure 8 , Figure 8 This is another structural schematic diagram of the electronic device provided in the embodiments of the present invention, such as... Figure 8 As shown, Figure 8 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the gastroscopy image processing method provided in the above embodiments. The electronic device 800 can be a mobile terminal such as a smartphone or a laptop computer.

[0130] RF circuit 810 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby enabling communication with communication networks or other devices. RF circuit 810 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity modules (SIM cards), memory, etc. RF circuit 810 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0131] The memory 820 can be used to store software programs and modules, such as the program instructions / modules corresponding to the gastroscopy image processing method in the above embodiment. The processor 880 executes various functional applications and gastroscopy image processing by running the software programs and modules stored in the memory 820, that is, it realizes the following functions:

[0132] Acquire the set of endoscopic images of the target user obtained through gastroscopy;

[0133] The endoscopic image set is identified to determine the first image in the endoscopic image set that belongs to the target object;

[0134] Anomaly detection is performed on the first image, and the image containing anomalies in the first image is identified as the second image;

[0135] Obtain the location information of the target image features in the second image;

[0136] The target image features are clustered based on the location information to obtain clustering results;

[0137] Based on the clustering results, the gastroscopy results of the target user are determined.

[0138] Memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 820 may further include memory remotely located relative to processor 880, which can be connected to electronic device 800 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0139] The input unit 830 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 830 may include a touch-sensitive surface 831 and other input devices 832. The touch-sensitive surface 831, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 831), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 831 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 880, and can receive and execute commands sent by the processor 880. In addition, the touch-sensitive surface 831 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 831, the input unit 830 may also include other input devices 832. Specifically, other input devices 832 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0140] Display unit 840 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of electronic device 800. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 840 may include display panel 841, which may optionally be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar display panel. Further, touch-sensitive surface 831 may cover display panel 841. When touch-sensitive surface 831 detects a touch operation on or near it, it transmits the information to processor 880 to determine the type of touch event. Subsequently, processor 880 provides corresponding visual output on display panel 841 according to the type of touch event. Although in the figures, touch-sensitive surface 831 and display panel 841 are implemented as two separate components to achieve input and output functions, in some embodiments, touch-sensitive surface 831 and display panel 841 can be integrated to achieve input and output functions.

[0141] The electronic device 800 may also include at least one sensor 850, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 841 according to the ambient light level, and the proximity sensor can generate an interruption when the flip is closed or shut down. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the electronic device 800, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0142] Audio circuitry 860, speaker 861, and microphone 862 provide an audio interface between the user and electronic device 800. Audio circuitry 860 converts received audio data into electrical signals, which are then transmitted to speaker 861, where they are converted into sound signals for output. Conversely, microphone 862 collects sound signals, converts them into electrical signals, which are received by audio circuitry 860, converted back into audio data, and then processed by processor 880 before being transmitted via RF circuitry 810 to, for example, another terminal, or output to memory 820 for further processing. Audio circuitry 860 may also include an earphone jack to facilitate communication between peripheral headphones and electronic device 800.

[0143] Electronic device 800, through transmission module 870 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 870 is shown in the figure, it is understood that it is not an essential component of electronic device 800 and can be omitted as needed without changing the essence of the invention.

[0144] The processor 880 is the control center of the electronic device 800. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 800 by running or executing software programs and / or modules stored in the memory 820, and by calling data stored in the memory 820, thereby providing overall monitoring of the electronic device. Optionally, the processor 880 may include one or more processing cores; in some embodiments, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 880.

[0145] The electronic device 800 also includes a power supply 890 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to the processor 880 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. The power supply 890 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0146] Although not shown, the electronic device 800 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0147] Acquire the set of endoscopic images of the target user obtained through gastroscopy;

[0148] The endoscopic image set is identified to determine the first image in the endoscopic image set that belongs to the target object;

[0149] Anomaly detection is performed on the first image, and the image containing anomalies in the first image is identified as the second image;

[0150] Obtain the location information of the target image features in the second image;

[0151] The target image features are clustered based on the location information to obtain clustering results;

[0152] Based on the clustering results, the gastroscopy results of the target user are determined.

[0153] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0154] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the gastroscopy image processing method provided by the present invention.

[0155] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0156] Since the instructions stored in the storage medium can execute the steps in any embodiment of the gastroscopy image processing method provided in the embodiments of the present invention, the beneficial effects that any gastroscopy image processing method provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0157] The foregoing has provided a detailed description of a gastroscopy image processing method, apparatus, electronic device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application. Moreover, those skilled in the art can make several improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered to be within the scope of protection of this invention.

Claims

1. A gastroscopic image processing method, characterized by, The method comprises the following steps: obtaining an endoscopic image set of a target user collected through gastroscopy detection; performing recognition on the endoscopic image set to determine first images belonging to target objects in the endoscopic image set, wherein the target objects include an esophagus and a gastric fundus; performing anomaly detection on the first images and determining images with anomalies in the first images as second images, wherein the anomalies include varicosity; obtaining position information of a target image feature in the second images, wherein the target image feature includes a red sign, and the position information includes a coordinate sequence, and the process of obtaining the position information of the target image feature in the second images comprises the following steps: calling a trained image segmentation model to perform recognition processing on the second images to determine a red sign region in the second images, wherein the trained image segmentation model is trained according to a second image with varicosity and annotation data of the second image in which the red sign region is annotated, and the coordinate sequence of boundary points of the red sign region in the second image is determined according to the second image; performing clustering processing on the target image feature according to the position information to obtain a clustering result; determining a gastroscopy detection result of the target user according to the clustering result.

2. The gastroscopic image processing method of claim 1, wherein, The method comprises the following steps: obtaining a gastroscopy video collected through gastroscopy detection, wherein the gastroscopy video comprises a plurality of gastroscopy images; determining an effective region of each gastroscopy image and a horizontal circumscribed rectangle of the effective region; cropping the gastroscopy images according to the horizontal circumscribed rectangle to obtain qualified images; calculating the definition of each qualified image, and determining a qualified image with a definition greater than a preset threshold as the endoscopic image set.

3. The gastroscopic image processing method of claim 1, wherein, The method comprises the following steps: calling a trained target object classification model to perform classification processing on the endoscopic image set to determine first images belonging to an esophagus and a gastric fundus in the endoscopic image set; wherein the trained target object classification model is trained according to endoscopic images collected through gastroscopy detection and annotation data of endoscopic images respectively containing an esophagus and a gastric fundus.

4. The gastroscopic image processing method of claim 3, wherein, The method comprises the following steps: calling a trained anomaly classification model to perform classification processing on the first images to determine second images with varicosity in the first images; wherein the trained anomaly classification model is trained according to first images belonging to an esophagus or a gastric fundus and annotation data of first images with varicosity.

5. The gastroscopic image processing method of claim 1, wherein, The method comprises the following steps: performing sparse processing on the coordinate sequence to obtain a point set corresponding to the coordinate sequence representing the red sign region; Traverse a first distance between all points in the point set and a preset first target point, and classify a first point with a first distance less than a preset distance threshold as a first group, the first target point being any point in the point set; If there is a second point with a first distance not less than the preset distance threshold, traverse a second distance between all the second points with a first distance not less than the preset distance threshold and a preset second target point, and classify a second point with a second distance less than the preset distance threshold as a second group, the second target point being any point in the second point; If there is a third point with both the first distance and the second distance not less than the preset distance threshold, classify the third point as a third group; The number of all classified groups is taken as a clustering result.

6. The gastroscopic image processing method of claim 5, wherein, The determination of the gastroscopy detection result of the target user according to the clustering result comprises: If the number of the classified groups is 1, the varicose vein red sign level of the target user is determined as a first level; If the number of the classified groups is 2, the varicose vein red sign level of the target user is determined as a second level; If the number of the classified groups is 3, the varicose vein red sign level of the target user is determined as a third level; The dispersion degree of the red sign corresponding to the third level is greater than the dispersion degree of the red sign corresponding to the second level, and the dispersion degree of the red sign corresponding to the second level is greater than the dispersion degree of the red sign corresponding to the first level.

7. An apparatus for processing a gastroscopic image, characterized by comprising: It comprises: A first acquisition module is configured to acquire an endoscopic image set of a target user collected through gastroscopy detection; An identification module is configured to identify the endoscopic image set to determine a first image belonging to a target object in the endoscopic image set, the target object including an esophagus and a gastric fundus; An detection module is configured to perform abnormality detection on the first image and determine an image with an abnormality in the first image as a second image, the abnormality including varicose veins; A second acquisition module is configured to acquire position information of a target image feature in the second image, wherein the target image feature includes a red sign, and the position information includes a coordinate sequence. The process of acquiring the position information of the target image feature in the second image is as follows: a trained image segmentation model is called to identify and process the second image to determine a red sign region in the second image. The trained image segmentation model is trained according to a second image with varicose veins and annotation data of a second image with an annotated red sign region. According to the second image, the coordinate sequence of the boundary point of the red sign region in the second image is determined. A clustering module is configured to perform clustering processing on the target image feature according to the position information to obtain a clustering result. A determination module is configured to determine a gastroscopy detection result of the target user according to the clustering result.

8. An electronic device, comprising: A computer readable storage medium storing a computer program, wherein the computer program, when executed, controls a device in which the computer readable storage medium is located to perform the steps of the gastroscopic image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, ​

Images