Dimensional evaluation system and dimensional evaluation method

Abstract

This invention provides a dimensional evaluation system and method that can obtain mathematical and statistical results for the dimensions of abnormal features. [Solution] The three-dimensional feature prediction model calculates the dimensions of anomaly features based on real-time images and location information. The feature tracking prediction model performs mathematical statistics on all dimensions of the same anomaly feature when the anomaly feature detected in the current frame of the real-time image is the same as the anomaly feature detected in the previous frame. Additionally, when the computing device receives a screen still signal, it generates mathematical statistical results for the dimensions corresponding to the anomaly feature.

Description

[Technical Field]

[0001] The present application relates to a dimension evaluation system and a dimension evaluation method capable of obtaining mathematical and statistical results of dimensions of abnormal features. [Background Art]

[0002] An endoscopy apparatus is an apparatus that uses an endoscope to examine organs and structures of a human body, enters the human body through various routes, and observes the internal state of the human body to determine the state of a disease.

[0003] Taking general colonoscopy as an example, a flexible fiber endoscope is used to enter the large intestine for direct observation and examination.

[0004] Generally, a colonoscopy apparatus is composed of a special elongated flexible tube and a small camera attached to the distal end of the tube, enters the intestinal tract through the anus, and observes whether there are diseases such as polyps and tumors along the inner wall of the intestine. In addition, during the examination, the colonoscopy apparatus is connected to a monitor, and real-time images of the internal structure of the intestine captured are displayed on the monitor, so that a doctor can be provided with the ability to examine or diagnose the internal health condition of the patient's large intestine. [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] However, in colonoscopy, when a polyp is imaged with an endoscope, the size of the polyp is usually calculated from a single frame image. Therefore, when measuring the size of a polyp, errors may occur in inspection data due to differences in the imaging angle of the polyp or slight differences in image frames. As a result, the size results of the same polyp measured in different frames of an image may become inconsistent, and it may become impossible to determine which result is correct. [Means for Solving the Problem]

[0006] In view of the above-mentioned problems, the present invention has the following configuration.

[0007] That is, a dimensional evaluation system electrically connected to a test apparatus, the test apparatus inspects an object and generates a real-time image, the dimensional evaluation system includes a computing device and a display device, The computing device is signal-connected to the test apparatus, and the test apparatus internally includes an anomaly detection model, a three-dimensional feature prediction model, and a feature tracking prediction model. The computing device receives the real-time image, and the anomaly detection model detects anomaly features on the real-time image, marks a selection box around the anomaly feature, and obtains location information. The three-dimensional feature prediction model calculates the dimensions of the anomaly feature based on the real-time image and the position information. The feature tracking and prediction model, when the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in the previous frame, performs mathematical statistics on all dimensions of the same anomaly feature, and when the computing device receives a screen still signal, generates mathematical statistical results for the dimensions corresponding to the anomaly feature. The display device is electrically connected to the computing device and is used to display the real-time image, the selection box, and the mathematical statistics results.

[0008] Furthermore, the test apparatus is a dimensional evaluation method suitable for inspecting an object and generating real-time images, and the dimensional evaluation method is implemented using a computing device and a display device. The computing device detects anomaly features in the real-time image, marks a selection box around the anomaly feature, and obtains location information. Based on the real-time image and the position information, the dimensions of the anomaly feature are calculated. When the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in the previous frame, mathematical statistics are performed on all dimensions of the same anomaly feature. When a screen freeze signal is received, the abnormal feature generates a mathematical statistical result for the corresponding dimension. The display device displays the real-time image, the selection box, and the mathematical statistics results. [Effects of the Invention]

[0009] As described above, the dimensional evaluation system and method of the present invention acquire real-time images, use a built-in artificial intelligence (AI) model to analyze and evaluate the mathematical statistical results of the dimensions of abnormal features in the real-time images (abnormal feature dimensions), and directly display the real-time images showing the abnormal feature dimensions and the mathematical statistical results on a display device, thereby improving the stability of the measurement results of abnormal feature dimensions.

[0010] Therefore, this invention can provide stable and accurate abnormality feature dimensions to enable physicians to make more accurate diagnoses, thus effectively helping to improve the diagnostic results of physicians. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram of a dimensional evaluation system and a test apparatus connected thereto, based on one embodiment of the present invention. [Figure 2] This is a flowchart of a dimensional evaluation method based on one embodiment of the present invention. [Figure 3] This is a schematic diagram showing real-time images, indicated by a selection box, on a timeline, based on one embodiment of the present invention. [Figure 4] This is a schematic diagram showing a dimensional evaluation system according to one embodiment of the present invention and a real-time image displaying mathematical and statistical results. [Figure 5] This is a flowchart of tracking prediction using a feature tracking prediction model in a dimensional evaluation system according to one embodiment of the present invention. MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings of the embodiments, part of the configuration or structure may be omitted to clearly illustrate the technical features of the present invention. In the drawings, the same reference numerals represent the same or related configurations or circuits.

[0013] FIG. 1 is a block diagram of a dimension evaluation system according to an embodiment of the present invention and a test apparatus connected thereto.

[0014] Referring to FIG. 1, the dimension evaluation system 10 is suitable for being signal-connectably or electrically connected to a test apparatus 22.

[0015] The test apparatus 22 inspects and images an object 24, generates a corresponding real-time image 26, and transmits the real-time image 26 to the dimension evaluation system 10.

[0016] The dimension evaluation system 10 includes a computing device 12 and a display device 14, and the computing device 12 is signal-connected to the test apparatus 22.

[0017] The computing device 12 is connected to the test apparatus 22 via a high-definition multimedia interface (HDMI (registered trademark)), a universal serial bus (USB) interface, a serial digital interface (SDI), or the like, but the present invention is not limited thereto.

[0018] The computing device 12 is electrically connected to the display device 14. For example, the computing device 12 is connected to the display device 14 via a high-definition multimedia interface (HDMI (registered trademark)), DisplayPort (DP), a serial digital interface (SDI), or the like, but the present invention is not limited thereto.

[0019] The computing device 12 incorporates an anomaly detection model 16, a three-dimensional feature prediction model 18, and a feature tracking prediction model 20.

[0020] After the computing device 12 receives the real-time image 26 from the test device 22, the computing device 12 can perform computing processing on the real-time image 26.

[0021] To detect abnormal features using the anomaly detection model 16, the three-dimensional feature prediction model 18 is used to predict the dimensions of the abnormal features, the feature tracking prediction model 20 is used to track and predict the abnormal features, and mathematical statistics are performed; the display device 14 is configured to display the real-time image 26 processed by the computing device 12.

[0022] In one embodiment, the test device 22 is an endoscope system such as a colonoscopy device, and in the present invention, the object 24 is the intestine.

[0023] In one embodiment, the abnormal features include proliferative tissue or diseased (affected) tissue of the object 24, such as polyps, tumors, or other formations generated on the tissue of the object.

[0024] In one embodiment, when the object 24 is the intestine, the abnormal feature is a colorectal polyp.

[0025] In one embodiment, the computing device 12 is a computer host or other electronic device that can operate independently for use with the display device 14, but the present invention is not limited thereto.

[0026] In another embodiment, a notebook computer can be directly used to replace the functions of the computing device 12 and the display device 14. The notebook computer is used to simultaneously process the operations of the computing device 12 and the display device 14.

[0027] Next, using the configuration shown in Figure 1, we will explain each step in executing the dimensional evaluation method with the dimensional evaluation system 10 of the present invention.

[0028] The explanation will be given with reference to Figures 1 and 2 simultaneously. After the test device 22 inspects the object 24 and generates a real-time image 26, the computing device 12 receives the real-time image 26 from the test device 22, as shown in step S10.

[0029] At this time, the computing device 12 transmits the real-time image 26 to the display device 14 so that the real-time image 26 is displayed on the display device 14.

[0030] As shown in step S12, the computing device 12 uses the anomaly detection model 16 to detect anomaly features 28 on the real-time image 26, as shown in Figure 3, which will be described later.

[0031] In addition to Figures 1 and 2, Figure 3 will also be used for explanation. Here, Figure 3 is a schematic diagram showing a real-time image 26 indicated by a selection box 30 based on one embodiment of the present invention on a timeline. The selection box 30 is shown (marked) around the anomaly feature 28, and positional information corresponding to the anomaly feature 28 is acquired.

[0032] As shown in step S14, the computing device 12 runs a three-dimensional feature prediction model 18, which calculates one dimension of an anomaly feature 28 based on the real-time image 26 and location information.

[0033] As shown in step S16, the computing device 12 runs a feature tracking and prediction model 20, which verifies that the anomaly feature 28 detected in each current frame 262 in the real-time image 26 is the same as the anomaly feature 28 detected in the previous frame 261, and then performs mathematical statistics on all dimensions of the same anomaly feature 28.

[0034] As shown in step S18, when the test device 22 is triggered and generates a screen still signal, the signal is sent to the computing device 12. Upon receiving the screen still signal, the computing device 12 generates mathematical statistical results 32 for all dimensions corresponding to the anomaly feature 28.

[0035] Finally, as shown in step S20, the process will be described with reference to Figures 3 and 4. Here, Figure 4 is a schematic diagram showing a real-time image of a dimensional evaluation system and mathematical statistical results according to one embodiment of the present invention. The computing device 12 adds the mathematical statistical results 32 to the real-time image 26 and outputs it to the display device 14, which displays the real-time image 26, the selection box 30, and the mathematical statistical results 32.

[0036] In one embodiment, as shown in Figures 1 and 3, the three-dimensional feature prediction model 18 further includes a depth prediction model 181 and a dimension prediction model 182.

[0037] The depth prediction model 181 estimates the depth from the anomaly feature 28 based on the real-time image 26 and location information. This depth is the distance between the lens of the test device 22 and the anomaly feature 28 on the object 24.

[0038] After obtaining the depth of the anomaly feature 28, the dimension prediction model 182 calculates the dimensions of the corresponding anomaly feature 28 based on the location information and depth.

[0039] In one embodiment, the mathematical statistical methods used by the computing device 12 include, but are not limited to, the mean, arithmetic mean, geometric mean, harmonic mean, weighted mean, trimmed mean, median, mode, or percentiles.

[0040] One embodiment will be described with reference to Figures 1, 3, and 5. Here, Figure 5 is a flowchart of tracking prediction using a feature tracking prediction model 20 in a dimensional evaluation system 10 according to one embodiment of the present invention. The computing device 12 further includes the following steps when performing tracking prediction using the feature tracking prediction model 20.

[0041] First, as shown in step S30 of Figure 5, an identification code is created corresponding to each anomalous feature 28 on the real-time image 26 of Figure 3. Then, as shown in step S32, the anomalous feature 28 is predicted within a predicted frame (not shown) of the current frame 262 using a Kalman filter.

[0042] As shown in step S34, the selection box 30 in the previous frame 261 of Figure 3 is acquired via the anomaly detection model 16 of Figure 1.

[0043] As shown in step S36, the Intersection Over Union (IOU) between the selection box 30 and the prediction box is calculated.

[0044] The Hungarian algorithm is used to match intersection overunions, and when the intersection overunion match is successful, the match between selection box 30 and prediction box is displayed as successful.

[0045] In step S38, mathematical statistics are performed on all dimensions of the anomaly feature 28, and the process returns to step S32.

[0046] Then, if the intersection overunion match is unsuccessful, it is displayed that the anomaly feature 28 match has not yet been successful or the selection box 30 match has not yet been successful, and the process returns to step S30 to wait for the detection of a new anomaly feature 28.

[0047] In one embodiment, the anomaly detection model 16, the three-dimensional feature prediction model 18 (including the depth prediction model 181 and the dimension prediction model 182), and the feature tracking prediction model 20 are each trained neural network models.

[0048] As described above, the dimensional evaluation system and method of the present invention can improve the stability of measurement results for abnormal feature dimensions by acquiring real-time images, analyzing and evaluating the mathematical statistical results of abnormal feature dimensions on the real-time images via a built-in artificial intelligence (AI) model, and directly displaying the real-time images and marks of the mathematical statistical results of abnormal feature dimensions on a display device.

[0049] Therefore, this invention contributes to improving physicians' diagnostic results by providing stable and accurate abnormality feature dimensions to enable them to make more accurate diagnoses.

[0050] The embodiments described above are merely for illustrating the technical idea and features of the present application, and are intended to enable those skilled in the art to understand the contents of these embodiments and implement the present invention accordingly. The embodiments described in this application are not to be used as a basis for limiting the interpretation of the technical scope of the patent. In other words, all equivalent changes or modifications made based on the spirit of the present invention are still included within the technical scope of this application. [Explanation of Symbols]

[0051] 10 Dimensional Evaluation System 12 Computing Devices 14 Display device 16 Anomaly Detection Models 18. Three-dimensional feature prediction models 181 Depth Prediction Model 182 Dimensional Prediction Model 20 Feature Tracking Prediction Models 22 Test equipment 24 Object 26 Real-time images 261 Previous frame 262 Current Frame 28 Abnormal Characteristics 30 selection boxes 32 Mathematics statistics results S10~S20 Step S30~S38 Step

Claims

1. A dimensional evaluation system is electrically connected to a test apparatus, the test apparatus inspects an object and generates a real-time image, and the dimensional evaluation system includes a computing device and a display device. The computing device is signal-connected to the test apparatus, and the test apparatus internally includes an anomaly detection model, a three-dimensional feature prediction model, and a feature tracking prediction model. The computing device receives the real-time image, and the anomaly detection model detects anomaly features on the real-time image, marks a selection box around the anomaly feature, and obtains location information. The three-dimensional feature prediction model calculates the dimensions of the anomaly feature based on the real-time image and the position information. The feature tracking and prediction model, when the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in the previous frame, performs mathematical statistics on all dimensions of the same anomaly feature, and when the computing device receives a screen still signal, generates mathematical statistical results for the dimensions corresponding to the anomaly feature. The display device is electrically connected to the computing device and is used to display the real-time image, the selection box, and the mathematical statistics results. A dimensional evaluation system characterized by the following features.

2. The aforementioned three-dimensional feature prediction model further includes a depth prediction model and a dimension prediction model. The depth prediction model estimates the depth of one of the anomaly features based on the real-time image and the position information, and the dimension prediction model calculates the dimension of the anomaly feature based on the position information and the depth. The dimensional evaluation system according to feature 1.

3. The aforementioned test apparatus is an endoscope system. The dimensional evaluation system according to feature 1.

4. The aforementioned mathematical statistics include the mean, arithmetic mean, geometric mean, harmonic mean, weighted mean, trimmed mean, median, mode, or percentile. The dimensional evaluation system according to feature 1.

5. For the aforementioned feature tracking and prediction model to perform tracking and prediction, it further requires: The feature tracking and prediction model generates an identification code corresponding to each of the anomaly features on the real-time image. The anomalous features of the prediction box of the current frame are predicted via a Kalman filter. The selection box of the previous frame is obtained via the anomaly detection model. The intersection over union of the selection box and the prediction box is calculated. This includes using a Hungarian algorithm to match the intersection overunion, indicating that the match between the selection box and the prediction box was successful when the intersection overunion was successfully matched, and performing mathematical statistics on all the dimensions of the anomaly feature. The dimensional evaluation system according to feature 1.

6. If the intersection overunion match is unsuccessful, the abnormal feature match has not yet been successful, or the selection box match has not yet been successful. The dimensional evaluation system according to claim 5.

7. The aforementioned abnormal characteristics do not include proliferative tissue or lesional tissue of the object. The dimensional evaluation system according to feature 1.

8. The aforementioned screen still signal is generated by being triggered by the test device. The dimensional evaluation system according to feature 1.

9. The test apparatus is a dimensional evaluation method suitable for inspecting an object and generating real-time images, and the dimensional evaluation method is implemented using a computing device and a display device. The computing device detects anomaly features in the real-time image, marks a selection box around the anomaly feature, and obtains location information. Based on the real-time image and the position information, the dimensions of the anomaly feature are calculated. When the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in the previous frame, mathematical statistics are performed on all dimensions of the same anomaly feature. When a screen freeze signal is received, the abnormal feature generates a mathematical statistical result for the corresponding dimension. The display device displays the real-time image, the selection box, and the mathematical statistics results. A dimensional evaluation method characterized by the following.

10. The aforementioned test apparatus is an endoscope system. The dimensional evaluation method according to feature 9.

11. The aforementioned anomaly features are detected by the anomaly detection model, and the selection box is displayed. The dimensional evaluation method according to feature 9.

12. The aforementioned dimensions are generated by a three-dimensional feature prediction model. The dimensional evaluation method according to feature 9.

13. The aforementioned three-dimensional feature prediction model further includes a depth prediction model and a dimension prediction model. The depth prediction model estimates the depth of one of the anomaly features based on the real-time image and the position information, and the dimension prediction model calculates the dimension of the anomaly feature based on the position information and the depth. The dimensional evaluation method according to feature 12.

14. The aforementioned mathematical statistics include the mean, arithmetic mean, geometric mean, harmonic mean, weighted mean, trimmed mean, median, mode, or percentile. The dimensional evaluation method according to feature 9.

15. The step of confirming whether the aforementioned anomaly features are identical is performed by a feature tracking and prediction model, and mathematical statistics are performed on all of the dimensions of the identical anomaly features. The dimensional evaluation method according to feature 9.

16. The step of performing tracking predictions on the aforementioned feature tracking prediction model further involves: The feature tracking and prediction model generates an identification code corresponding to each of the anomaly features on the real-time image. The anomalous features of the prediction box of the current frame are predicted via a Kalman filter. The selection box from the previous frame is obtained by detecting an anomaly. The intersection over union of the selection box and the prediction box is calculated. The Hungarian algorithm is used to match the intersection overunion, and when the intersection overunion is successfully matched, the match between the selection box and the prediction box is indicated as successful, and mathematical statistics are performed on all the dimensions of the anomaly feature. The dimensional evaluation method according to feature 15.

17. If the intersection overunion match is unsuccessful, the abnormal feature match has not yet been successful, or the selection box match has not yet been successful. The dimensional evaluation method according to feature 16.

18. The aforementioned abnormal characteristics do not include proliferative tissue or lesional tissue of the object. The dimensional evaluation method according to feature 9.

19. The aforementioned screen still signal is generated by being triggered by the test device. The dimensional evaluation method according to feature 9.

Images