A multi-scale detection method and device for tumors in digestive tract endoscopic images

Abstract

The application relates to the technical field of medical detection, in particular to a multi-scale detection method and device for tumors in digestive tract endoscope images, which comprises the following steps: acquiring endoscope tumor initial section, middle section and terminal section image graphs in a target region of the digestive tract, pre-processing the image graphs to obtain first, second and third image graph sets; performing stripping analysis on the image graph sets to obtain contour image graph sets; performing gray scale processing on the contour image graph sets to obtain gray scale graph sets; inputting the gray scale graph sets into a plane coordinate system, performing edge cutting on each gray scale graph set to obtain the coordinates of each graph set after cutting; inputting the coordinates of each graph set into three-dimensional modeling software to perform reconstruction to obtain the image of the endoscope tumor in the target region after reconstruction, and grading the image according to a preset tumor grade. The application obtains a visual three-dimensional tumor state by combining the endoscope tumor initial section, middle section and terminal section image graphs in the target region of the digestive tract, the graph set coordinates and the three-dimensional software, determines the tumor grade, and realizes multi-scale detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical detection, in particular to a multi-scale detection method for tumors in digestive tract endoscopic images, and also relates to a device. BACKGROUND

[0002] At present, computer technology has been widely applied in the medical field. A digestive endoscopic image often contains a large amount of information. At present, most of the digestive endoscopic image analysis algorithms based on graph theory are to regard each pixel of a digestive endoscopic image as a node of a graph, and to connect any adjacent pair of nodes to establish the edges of the graph. Therefore, the edges express the adjacency relationship between the pixels in the digestive endoscopic image. After the adjacency relationship of the graph is established, the weight of each edge can be established by calculating the information of the pixels represented by the adjacent two nodes. The weight expresses the similarity degree between the pixels in the image. The existing improvement for the recognition of digestive tract endoscopic images is usually to improve the extraction of feature points of the digestive endoscopic image and the establishment of the corresponding internal structure of the image based on graph theory.

[0003] In the prior art, a Chinese invention patent application with the publication date of June 21, 2019 and the publication number of CN109919212A and the name of Multi-scale detection method and device for tumors in digestive tract endoscopic images discloses that each digestive tract endoscopic image sample is marked with tumor images, and according to a preset tumor property grade, the digestive tract endoscopic image sample marked with the tumor images is marked with the tumor property grade. Based on the digestive tract endoscopic image sample marked with the tumor property grade, a digestive tract endoscopic image detection model is trained. A to-be-detected endoscopic image is obtained. According to the digestive tract endoscopic image detection model, a detection result of the to-be-detected endoscopic image is obtained. The detection result includes one or more of a first classification result, a second classification result and a multi-scale tumor region detection result.

[0004] The above technical solution can reduce the marking cost and realize accurate positioning of the tumor region in the digestive tract endoscopic image, but it cannot realize multi-scale detection of the digestive tract tumor. SUMMARY

[0005] The present application relates to the technical field of medical detection, in particular to a multi-scale detection method for tumors in digestive tract endoscopic images, and also relates to a device.

[0006] The present application relates to the technical field of medical detection, in particular to a multi-scale detection method for tumors in digestive tract endoscopic images, and also relates to a device. The present application aims to provide a multi-scale detection method and device for tumors in digestive tract endoscopic images, which solves the technical problem that the prior art cannot provide a method for multi-scale detection of tumors and accurate determination of tumor grades by combining the initial, middle and terminal image graphs of endoscopic tumors.

[0007] According to an aspect of the present application, a multi-scale detection method for tumors in digestive tract endoscopic images is provided, which is executed by a controller and comprises:

[0008] Obtaining endoscopic tumor initial, middle and terminal image graphs of a target region in the digestive tract, and pre-processing the initial, middle and terminal endoscopic tumor image graphs to obtain a first image graph set, a second image graph set and a third image graph set;

[0009] Stripping and analyzing the first image graph set, the second image graph set and the third image graph set to obtain a first contour image graph set, a second contour image graph set and a third contour image graph set;

[0010] Performing gray scale processing on the first contour image graph set, the second contour image graph set and the third contour image graph set to obtain a first gray scale graph set, a second gray scale graph set and a third gray scale graph set;

[0011] Inputting the first gray scale graph set, the second gray scale graph set and the third gray scale graph set into a plane coordinate system, performing edge clipping on each gray scale graph set to obtain the coordinates (x i , y i ) of each graph set after clipping, wherein i represents the i-th gray scale graph set;

[0012] Inputting the coordinates (x i , y i ) of each graph set into a three-dimensional modeling software to perform reconstruction, obtaining a reconstructed image of the endoscopic tumor of the target region, and grading the image according to a preset tumor grade.

[0013] In some embodiments, the endoscopic tumor initial, middle and terminal image graphs of the target region are specifically: the endoscopic tumor initial image graph is an image when the endoscope approaches the tumor in the digestive tract, the endoscopic tumor middle image graph is an image when the endoscope is located at the tumor in the digestive tract, and the endoscopic tumor terminal image graph is an image when the endoscope is away from the tumor in the digestive tract, wherein the initial image graph and the terminal image graph are respectively located on opposite sides of the tumor position in the digestive tract.

[0014] In some embodiments, the pre-processing of the initial, middle and terminal endoscopic tumor image graphs is specifically: at least performing normalization, gray scale and pixelization processing on the initial, middle and terminal endoscopic tumor image graphs.

[0015] In some embodiments, the stripping and analyzing of the first image graph set, the second image graph set and the third image graph set is specifically:

[0016] Obtaining each pixel point in the first image graph set, the second image graph set and the third image graph set, and recording the pixel point as an anchor point;

[0017] The anchor pixel values between adjacent anchor points are obtained, when the difference between the anchor pixel values is less than the standard deviation value, the corresponding anchor points corresponding to the relevant difference value are merged, and are recorded as a first anchor point group, when the difference between the anchor pixel values is greater than the standard deviation value, the corresponding anchor points corresponding to the relevant difference value are merged, and are recorded as a second anchor point group; when the difference between the anchor pixel values is equal to the standard deviation value, the corresponding anchor points corresponding to the relevant difference value are merged, and are recorded as a third anchor point group;

[0018] The image set corresponding to the first anchor point group is regarded as a first contour image set, the image set corresponding to the second anchor point group is regarded as a second contour image set, and the image set corresponding to the third anchor point group is regarded as a third contour image set.

[0019] In some embodiments, the edge clipping is performed on each gray scale set, and the coordinates (x i , y i ) of each set after clipping are obtained, specifically:

[0020] The standard rectangular view frame is adjusted, and the first gray scale set, the second gray scale set and the third gray scale set are edge-normalized and clipped to obtain a first rectangular gray scale set, a second rectangular gray scale set and a third rectangular gray scale set.

[0021] The first rectangular gray scale set, the second rectangular gray scale set and the third rectangular gray scale set are put into a plane coordinate system, and the corresponding coordinates (x i , y i ) are obtained.

[0022] In some embodiments, the coordinates (x i , y i ) of each set are input into a three-dimensional modeling software to perform reconstruction, and an endoscopic tumor image in the target region after reconstruction is obtained, and the image is graded according to a preset tumor grade, specifically:

[0023] The coordinates (x i , y i ) corresponding to the first rectangular gray scale set, the second rectangular gray scale set and the third rectangular gray scale set are respectively input into a three-dimensional modeling software, and are restored by spline sweeping and surface modeling to obtain an endoscopic tumor pattern in the target region;

[0024] According to the comparison between the endoscopic tumor pattern in the target region and the preset tumor grade, the tumor grade of the target region is determined, and marking is performed.

[0025] According to another aspect of the present application, a multi-scale detection device for tumors in a digestive tract endoscope image is provided, which includes a device body, and the device body includes functional components for performing detection, and the functional components include:

[0026] The acquisition module is used to acquire endoscopic tumor initial section, middle section and terminal section image graphs in a target digestive tract region, and preprocess the endoscopic tumor image graphs of the initial section, the middle section and the terminal section to obtain a first image graph set, a second image graph set and a third image graph set.

[0027] The first preprocessing module is used to peel and analyze the first image graph set, the second image graph set and the third image graph set to obtain a first contour image graph set, a second contour image graph set and a third contour image graph set.

[0028] The second preprocessing module is used to perform gray scale processing on the first contour image graph set, the second contour image graph set and the third contour image graph set to obtain a first gray scale graph set, a second gray scale graph set and a third gray scale graph set.

[0029] The edge processing module is used to input the first gray scale graph set, the second gray scale graph set and the third gray scale graph set into a plane coordinate system, perform edge clipping on each gray scale graph set, and acquire coordinates (x i , y i ) of each graph set after clipping, where i represents the i-th gray scale graph set.

[0030] The reconstruction module is used to input the coordinates (x i , y i ) of each graph set into three-dimensional modeling software to perform reconstruction, obtain an endoscopic tumor image of the target region after reconstruction, and grade the image according to a preset tumor grade.

[0031] In some embodiments, the acquisition module, the first preprocessing module, the second preprocessing module, the edge processing module and the reconstruction module are all in data connection with the device body.

[0032] Compared with the prior art, the present application has the following beneficial effects: the present application obtains endoscopic tumor initial section, middle section and terminal section image graphs in a target digestive tract region, combines the coordinates of each graph set, and restores the digestive tract tumor state in a three-dimensional software through three-dimensional restoration, to obtain a visual three-dimensional tumor state, quickly and accurately detect and determine the tumor grade, and realize multi-scale detection of the tumor in the digestive tract endoscopic image. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0034] Fig. 1 is a flowchart of the multiscale detection method of the present application;

[0035] Fig. 2 is a structural schematic diagram of the multiscale detection device of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0037] The present embodiment provides a multiscale detection method and device for tumors in digestive tract endoscopic images. The multiscale detection method and device have been in the testing and using stage. The multiscale detection method and device will be described below in combination with the accompanying drawings. Figs. 1-2

[0038] SUMMARY

[0039] Computer technology has been widely applied in the medical field. Digestive endoscopic images often contain a large amount of information. For the detection of tumors in the digestive tract, a gastroenteroscope needs to be used, but only the position of the tumor can be determined through imaging, and the tumor cannot be visually evaluated, so that the grade of the tumor cannot be quickly determined, which cannot lay a foundation for the next medical development. Therefore, only biopsy resection can be performed. For benign tumors, this method has high feasibility. However, for malignant tumors, whether they can be directly resected without secondary injury can only depend on the experience accumulation of doctors. Therefore, for doctors with little experience, direct resection is often performed, which increases the risk of secondary injury.

[0040] Based on the foregoing considerations, the present application restores the visual tumor state by preprocessing the initial segment, middle segment and terminal segment endoscopic tumor images of the target region of the digestive tract and three-dimensional modeling through three-dimensional software, realizes rapid determination of the tumor grade, and can perform more diverse evaluation and analysis on the tumor after three-dimensional restoration.

[0041] EMBODIMENT

[0042] EXEMPLARY METHOD

[0043] ​To solve the foregoing problems and achieve the corresponding effects, the embodiment provides a multi-scale detection method for tumors in digestive tract endoscopic images. The method is executed by a controller and includes: acquiring endoscopic tumor initial section, middle section and terminal section image graphs of a target region in the digestive tract; preprocessing the endoscopic tumor image graphs of the initial section, the middle section and the terminal section to obtain a first image graph set, a second image graph set and a third image graph set; performing stripping analysis on the first image graph set, the second image graph set and the third image graph set to obtain a first contour image graph set, a second contour image graph set and a third contour image graph set; performing gray processing on the first contour image graph set, the second contour image graph set and the third contour image graph set to obtain a first gray graph set, a second gray graph set and a third gray graph set; inputting the first gray graph set, the second gray graph set and the third gray graph set into a plane coordinate system, performing edge clipping on each gray graph set, and acquiring coordinates (x i , y i ) of each graph set after clipping, where i represents the i-th gray graph set; inputting the coordinates (x i , y i ) of each graph set into three-dimensional modeling software to perform reconstruction, obtaining a reconstructed endoscopic tumor image of the target region, and grading the image according to a preset tumor grade.

[0044] The multi-scale detection method for tumors in digestive tract endoscopic images solves the problem that the prior art can only obtain a single gastroscope image and cannot effectively obtain the three-dimensional shape of the entire tumor and accurately locate the tumor grade.

[0045] In the following, each step will be described in detail.

[0046] In some embodiments, the target region endoscopic tumor initial section image graph is an image obtained when the endoscope approaches the digestive tract tumor, the target region endoscopic tumor middle section image graph is an image obtained when the endoscope is located at the digestive tract tumor, and the target region endoscopic tumor terminal section image graph is an image obtained when the endoscope is away from the digestive tract tumor, where the initial section image graph and the terminal section image graph are respectively located on opposite sides of the digestive tract tumor. It can be understood that in the present embodiment, the images of the three positions are obtained by the built-in camera of the gastroscope, that is, the internal digestive tract tumor can be comprehensively restored from three angles, and multi-angle detection can be achieved.

[0047] In some embodiments, the endoscopic tumor image graphs of the initial section, the middle section and the terminal section are at least subjected to normalization, graying and pixelization processing. It can be understood that the preprocessing in the present embodiment is a common method, and the purpose is to clean the images. In other equivalent embodiments, other processing methods can also be used in addition to the above, such as image flipping and image size adjustment.

[0048] In some embodiments, the first image set, the second image set and the third image set are analyzed by peeling. In this embodiment, the peeling method is to place the pixelated and gray-scaled first, second and third image sets in a planar rectangular coordinate system, and then use the peeling analysis method on the first, second and third image sets to obtain a plurality of graphic contours.

[0049] In detail, each pixel point in the first image set, the second image set and the third image set is obtained, and the pixel point is recorded as an anchor point. The anchor pixel values between adjacent anchor points are obtained. When the difference between the anchor pixel values is less than a standard deviation value, the corresponding anchor points corresponding to the related difference value are merged and recorded as a first anchor point group. When the difference between the anchor pixel values is greater than the standard deviation value, the corresponding anchor points corresponding to the related difference value are merged and recorded as a second anchor point group. When the difference between the anchor pixel values is equal to the standard deviation value, the corresponding anchor points corresponding to the related difference value are merged and recorded as a third anchor point group. The image set corresponding to the first anchor point group is regarded as a first contour image set, the image set corresponding to the second anchor point group is regarded as a second contour image set, and the image set corresponding to the third anchor point group is regarded as a third contour image set.

[0050] In some embodiments, a standard rectangular view frame is selected, and the edges of the first gray-scale image set, the second gray-scale image set and the third gray-scale image set are standardized and cropped to obtain a first rectangular gray-scale image set, a second rectangular gray-scale image set and a third rectangular gray-scale image set. The first rectangular gray-scale image set, the second rectangular gray-scale image set and the third rectangular gray-scale image set are placed in a planar coordinate system to obtain corresponding coordinates (x i , y i ). It can be understood that the standard rectangular view frame selected in this embodiment can be understood as a selected window. After the image passes through the window, window cropping is performed, and the image region after cropping is determined by the window shape. After obtaining the cropped image, it is placed in the coordinate system to determine the horizontal and vertical coordinate values, i.e., to obtain the specific values of the coordinates (x i , y i ).

[0051] Further, the obtained coordinates (x i , y i ) of the first rectangular gray-scale image set, the second rectangular gray-scale image set and the third rectangular gray-scale image set are input into a three-dimensional modeling software, and restoration is performed through spline sweeping and surface modeling to obtain an endoscopic tumor pattern in the target region. According to the comparison between the endoscopic tumor pattern in the target region and the preset tumor grade, the tumor grade of the target region is determined, and marking is performed. It can be understood that after the coordinate system is input into the three-dimensional modeling software, the positions of the three images can be determined according to the coordinate values. After combining them into a closed size chain, three-dimensional rough patterns can be restored through related operations such as stretching, spline sweeping and surface modeling, and then subsequent tumor grade determination and possible pathological predictive analysis can be performed.

[0052] For better understanding of the method of the present application, the embodiment also provides a device, comprising a device body, the device body comprising functional components for performing detection, the functional components comprising:

[0053] The acquisition module is configured to acquire endoscopic tumor image graphs of initial, middle and terminal segments in a target region of a digestive tract, preprocess the endoscopic tumor image graphs of the initial, middle and terminal segments, and obtain a first image graph set, a second image graph set and a third image graph set.

[0054] The first preprocessing module is configured to perform stripping analysis on the first image graph set, the second image graph set and the third image graph set, and obtain a first contour image graph set, a second contour image graph set and a third contour image graph set.

[0055] The second preprocessing module is configured to perform gray scale processing on the first contour image graph set, the second contour image graph set and the third contour image graph set, and obtain a first gray scale graph set, a second gray scale graph set and a third gray scale graph set.

[0056] The edge processing module is configured to input the first gray scale graph set, the second gray scale graph set and the third gray scale graph set into a plane coordinate system, perform edge clipping on each gray scale graph, and obtain coordinates (x i , y i ) of each graph after clipping, where i represents the i-th gray scale graph.

[0057] The reconstruction module is configured to input the coordinates (x i , y i ) of each graph into a three-dimensional modeling software to perform reconstruction, obtain an endoscopic tumor image of the target region after reconstruction, and grade the image according to a preset tumor grade.

[0058] It should be noted that in some embodiments, the acquisition module, the first preprocessing module, the second preprocessing module, the edge processing module and the reconstruction module are all connected with the device body data to form a functional closed loop and perform multi-scale detection.

[0059] The device provided in the embodiment of the present application comprises a processor, a memory and a computer program stored in the memory and executable on the processor, such as a multi-scale detection program. The processor executes the computer program to implement each step in the multi-scale detection method embodiment described above. Alternatively, the processor executes the computer program to implement the functions of each module in the device embodiment described above.

[0060] For example, the computer program can be divided into one or more modules, one or more modules are stored in the memory and executed by the processor to complete the present application. One or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the multi-scale detection device. For example, the computer program can be divided into an acquisition module, a first preprocessing module, a second preprocessing module, an edge processing module, and a reconstruction module, and the specific functions of each module are as described above.

[0061] In addition, the aforementioned processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the multi-scale detection device, and connects various parts of the multi-scale detection device through various interfaces and lines.

[0062] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-scale tumor detection method in digestive tract endoscopic images, the method being executed by a controller, characterized in that: The method comprises the following steps: acquire endoscopic tumor initial section, middle section and terminal section image graphs in a target region of a digestive tract, preprocess the endoscopic tumor initial section, middle section and terminal section image graphs to obtain a first image graph set, a second image graph set and a third image graph set; strip analysis is performed on the first image graph set, the second image graph set and the third image graph set to obtain a first contour image graph set, a second contour image graph set and a third contour image graph set; perform gray processing on the first contour image graph set, the second contour image graph set and the third contour image graph set to obtain a first gray graph set, a second gray graph set and a third gray graph set; The first gray scale atlas, the second gray scale atlas and the third gray scale atlas are input into a plane coordinate system, edge clipping is performed on each gray scale atlas, and coordinates (x i , y i ) of each atlas after clipping are obtained, wherein i represents the i-th gray scale atlas; Inputting each atlas coordinate (x i , y i ) into a three-dimensional modeling software to perform reconstruction, obtaining an endoscopic image of the target region after reconstruction, and grading the image according to a preset tumor grade; wherein the strip analysis of the first image graph set, the second image graph set and the third image graph set comprises: acquire each pixel point in the first image graph set, the second image graph set and the third image graph set, and record the pixel point as an anchor point; acquire anchor pixel values between adjacent anchor points, and when the difference between the anchor pixel values is less than a standard deviation value, the anchor points corresponding to the related difference value are merged and recorded as a first anchor point group, when the difference between the anchor pixel values is greater than the standard deviation value, the anchor points corresponding to the related difference value are merged and recorded as a second anchor point group, and when the difference between the anchor pixel values is equal to the standard deviation value, the anchor points corresponding to the related difference value are merged and recorded as a third anchor point group; the image graph set corresponding to the first anchor point group is regarded as the first contour image graph set, the image graph set corresponding to the second anchor point group is regarded as the second contour image graph set, and the image graph set corresponding to the third anchor point group is regarded as the third contour image graph set.

2. The method of claim 1, wherein, The endoscopic tumor initial section image graph in the target region is an image when the endoscope approaches the tumor in the digestive tract, the endoscopic tumor middle section image graph is an image when the endoscope is located at the tumor in the digestive tract, and the endoscopic tumor terminal section image graph is an image when the endoscope is away from the tumor in the digestive tract, wherein the initial section image graph and the terminal section image graph are respectively located on opposite sides of the tumor position in the digestive tract.

3. The method of claim 1, wherein, The preprocessing of the initial section, middle section and terminal section endoscopic tumor image graphs comprises at least normalization, graying and pixelization processing.

4. The method of claim 1, wherein, The edge clipping is performed on each gray scale atlas to obtain the coordinates (x i , y i ) of each atlas after clipping, and specifically comprises the following steps. adjust a standard rectangular view frame, perform edge standardization cropping on the first gray graph set, the second gray graph set and the third gray graph set to obtain a first rectangular gray graph set, a second rectangular gray graph set and a third rectangular gray graph set; The first, second and third rectangular grayscale image sets are put into a plane coordinate system to obtain corresponding coordinates (x i , y i ).

5. The method of claim 1, wherein, The endoscopic image of the target region is obtained by inputting the atlas coordinates (x i , y i ) into three-dimensional modeling software for reconstruction. i , y i ) into three-dimensional modeling software for reconstruction, obtaining the endoscopic image of the target region after reconstruction, and grading the image according to a preset tumor grade. The coordinates (x i , y i ) corresponding to the first rectangular gray scale atlas, the second rectangular gray scale atlas and the third rectangular gray scale atlas are respectively input into a three-dimensional modeling software, and a mirror tumor sample in a target region is obtained through spline sweeping and surface modeling. determine the tumor grade of the target region according to the comparison between the endoscopic tumor state and the preset tumor grade, and perform marking.

6. A multi-scale detection device for tumors in digestive endoscopic images, comprising a device body, the device body comprising functional components for performing detection, characterized in that, The functional components comprise: an acquisition module, configured to acquire endoscopic tumor initial section, middle section and terminal section image graphs in a target region of a digestive tract, preprocess the endoscopic tumor initial section, middle section and terminal section image graphs to obtain a first image graph set, a second image graph set and a third image graph set; a first preprocessing module, configured to perform strip analysis on the first image graph set, the second image graph set and the third image graph set to obtain a first contour image graph set, a second contour image graph set and a third contour image graph set; A second preprocessing module is configured to perform gray scale processing on the first profile image set, the second profile image set and the third profile image set to obtain a first gray scale image set, a second gray scale image set and a third gray scale image set. an edge processing module, configured to input the first gray scale atlas, the second gray scale atlas and the third gray scale atlas into a plane coordinate system, perform edge clipping on each gray scale atlas, and obtain the coordinates (x i , y i ) of each clipped atlas, where i represents the i-th gray scale atlas; a reconstruction module for inputting each atlas coordinate (x i , y i ) into a three-dimensional modeling software to perform reconstruction, obtaining an endoscopic image of a tumor in a target region after reconstruction, and grading the image according to a preset tumor grade.

7. The apparatus of claim 6, wherein, The acquisition module, the first preprocessing module, the second preprocessing module, the edge processing module and the reconstruction module are all connected with the device body.

Images