Dimension estimation system and method thereof

Abstract

The disclosure provides a dimension estimation system and method thereof. The dimension estimation system includes a computing device and a display device connected thereto. The computing device has built-in an abnormality detection model, a three-dimensional feature prediction model and a feature tracking prediction model. The abnormality detection model detects an abnormal feature on a real-time image, marks a selection box around the abnormal features, and obtains location information. The three-dimensional feature prediction model calculates a size of the abnormal feature based on the real-time image and the location information. When the feature tracking prediction model confirms that the abnormal features detected in two consecutive frames in the real-time image are the same, mathematical statistics are performed on all sizes of the abnormal feature. When the computing device receives an image freeze signal, a mathematical statistical result of the sizes of the abnormal feature is generated. The display device displays the real-time image, the selection box and the mathematical statistical result.

Description

[Technical Field]

[0001] This case relates to a size evaluation system and method that can obtain mathematical statistical results of abnormal characteristic dimensions. [Previous Technology]

[0002] An endoscopic examination instrument is a device that uses an endoscope to examine internal organs or structures. It enters the human body through various tubes to observe the internal condition and determine if any lesions are present. Taking the common colonoscope as an example, it uses a flexible fiber optic endoscope to directly observe and examine the large intestine. A typical colonoscope examination instrument includes a specially designed thin, flexible tube and a small camera at the tip of the tube. It is inserted into the intestine through the anus and observes along the tube wall for lesions such as polyps or tumors. During the examination, the colonoscope examination instrument is connected to a monitor to display real-time images of the internal intestinal structure, providing doctors with information to examine or diagnose the health status of the patient's large intestine.

[0003] During a colonoscopy, when a polyp is captured by the endoscope, the size of the polyp is usually calculated using a single frame of the image. However, when measuring the size of the polyp, errors in the measurement data may occur due to different viewing angles of the polyp or slight differences in the image. This can lead to inconsistent size results for the same polyp measured in different frames, which can easily cause disputes. [Summary of the Invention]

[0004] This application provides a size assessment system suitable for electrical connection with a detection instrument. The detection instrument inspects a target object and generates a real-time image. The size assessment system includes a computing device and a display device. The computing device is signal-connected to the detection instrument and has a built-in anomaly detection model, a three-dimensional feature prediction model, and a feature tracking prediction model. The computing device receives the real-time image. The anomaly detection model detects an anomaly feature on the real-time image, marks a selection box around the anomaly feature, and obtains position information. The three-dimensional feature prediction model calculates the size of the anomaly feature based on the real-time image and the position information. The feature tracking prediction model confirms that when the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in a previous frame, it performs mathematical statistics on all sizes of the same anomaly feature and generates a mathematical statistical result of the size corresponding to the anomaly feature when the computing device receives a static frame signal. The display device is electrically connected to the computing device and is used to display the real-time image, the selection box, and the mathematical statistical result.

[0005] This case further provides a size assessment method suitable for a real-time image generated by an inspection instrument inspecting a target object. This size assessment method includes: receiving the real-time image and detecting an abnormal feature on the real-time image, so as to mark a selection box around the abnormal feature and obtain position information; calculating the size of the abnormal feature based on the real-time image and the position information; when confirming that the abnormal feature detected in each current frame of the real-time image is the same as the abnormal feature detected in a previous frame, performing mathematical statistics on all sizes of the same abnormal feature; generating a mathematical statistical result of the size corresponding to the abnormal feature when a frame still signal is received; and displaying the real-time image, the selection box and the mathematical statistical result.

[0006] In summary, the dimensional assessment system and method of this case, after acquiring real-time images, can analyze and evaluate the mathematical statistical results of abnormal feature dimensions on the real-time images through a built-in artificial intelligence (AI) model, and directly display the real-time images and the mathematical statistical results of the abnormal feature dimensions on them on the display device, thereby improving the stability of the measurement results of abnormal feature dimensions. Therefore, this case can effectively assist physicians in providing stable and accurate abnormal feature dimensions for physicians to make more accurate diagnoses and improve the physician's user experience.

Implementation Method

[0007] The embodiments of this invention will be described below with reference to the relevant drawings. Furthermore, some components or structures are omitted in the drawings of the embodiments to clearly show the technical features of this invention. In these drawings, the same reference numerals denote the same or similar components or circuits.

[0008] Figure 1 is a block diagram of a size assessment system and its connected testing instrument according to an embodiment of this invention. Referring to Figure 1, a size assessment system 10 is adapted to be electrically connected to a testing instrument 22. The testing instrument 22 examines and captures a target object 24 and generates a real-time image 26, which is then transmitted to the size assessment system 10. The size assessment system 10 includes a computing device 12 and a display device 14. The computing device 12 is signal-connected to the testing instrument 22, for example, via a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, or a serial digital interface (SDI), but this invention is not limited thereto. The computing device 12 is electrically connected to the display device 14, for example, via a high-definition multimedia interface (HDMI), a display port (DisplayPort, DP) interface, or a serial digital interface (SDI), but this invention is not limited thereto. The computing device 12 has an anomaly detection model 16, a three-dimensional feature prediction model 18, and a feature tracking prediction model 20 built into it. After the computing device 12 receives the real-time image 26 from the detection instrument 22, it can perform calculations on the real-time image 26 to detect abnormal features using the anomaly detection model 16, predict the size of the abnormal features using the three-dimensional feature prediction model 18, and track and predict the abnormal features and perform mathematical statistics using the feature tracking prediction model 20. The display device 14 is used to display the real-time image 26 processed by the computing device 12.

[0009] In one embodiment, the detection instrument 22 is an endoscope system, such as a colonoscope, and the target object 24 is the intestine. In one embodiment, the abnormal feature includes proliferating or diseased tissue of the target object 24, such as polyps, tumors, or other formations on the target object tissue. In one embodiment, when the target object 24 is the intestine, the abnormal feature is a colonic polyp.

[0010] In one embodiment, the computing device 12 is a computer host or other electronic device capable of independent operation, used in conjunction with the display device 14, but this invention is not limited thereto. In another embodiment, a laptop computer can be used directly to replace the functions of the computing device 12 and the display device 14, so that the laptop computer can simultaneously handle the work of the computing device 12 and the display device 14.

[0011] The flow of each step in the size evaluation system 10 of this case in performing the size evaluation method will continue to be described with reference to the architecture shown in Figure 1. Please refer to Figures 1 and 2 simultaneously. After the detection instrument 22 inspects the target object 24 and generates a real-time image 26, as shown in step S10, the computing device 12 receives the real-time image 26 from the detection instrument 22. At this time, the computing device 12 transmits the real-time image 26 to the display device 14 so that the real-time image 26 is displayed on the display device 14. As shown in step S12, the computing device 12 uses the anomaly detection model 16 to detect an abnormal feature 28 on the real-time image 26, as shown in Figure 3, to mark a selection box 30 around the abnormal feature 28 and obtain the position information corresponding to this abnormal feature 28. As shown in step S14, the computing device 12 executes the three-dimensional feature prediction model 18. The three-dimensional feature prediction model 18 calculates the size of this abnormal feature 28 based on the real-time image 26 and the position information. As shown in step S16, the computing device 12 executes the feature tracking prediction model 20. This feature tracking prediction model 20 confirms that when the abnormal feature 28 detected in each current frame 262 of the real-time image 26 is the same as the abnormal feature 28 detected in a previous frame 261, it performs mathematical statistics on all dimensions of the same abnormal feature 28. As shown in step S18, when the detection instrument 22 is triggered to generate a frame freeze signal, it is transmitted to the computing device 12. When the computing device 12 receives this frame freeze signal, it generates a mathematical statistical result 32 for all dimensions corresponding to this abnormal feature 28. Finally, as shown in step S20, please refer to Figures 3 and 4, the computing device 12 adds the mathematical statistical result 32 to the real-time image 26 and outputs it to the display device 14, so that the display device 14 displays the real-time image 26, the selection box 30, and the mathematical statistical result 32.

[0012] In one embodiment, as shown in Figures 1 and 3, the three-dimensional feature prediction model 18 further includes a depth prediction model 181 and a size prediction model 182. The depth prediction model 181 estimates the depth of one of the abnormal features 28 based on the real-time image 26 and position information. This depth is the distance between the lens of the detection instrument 22 and the abnormal feature 28 on the target object 24. After obtaining the depth of the abnormal feature 28, the size prediction model 182 calculates the size of the corresponding abnormal feature 28 based on the position information and the depth.

[0013] In one embodiment, the mathematical statistics method used by the computing device 12 may be a mean, an arithmetic mean, a geometric mean, a harmonic mean, a weighted mean, a truncated mean, a median, a mode, or a percentile, etc., but this application is not limited to these.

[0014] In one embodiment, as shown in Figures 1, 3, and 5, the computing device 12 further includes the following steps in the tracking and prediction process using the feature tracking prediction model 20. First, as shown in step S30, an identification code is created for each anomalous feature 28 on the real-time image 26. As shown in step S32, a prediction box (not shown) for this anomalous feature 28 in the current frame 262 is predicted using Kalman filtering. As shown in step S34, the selection box 30 of the previous frame 261 is obtained through the anomaly detection model 16. As shown in step S36, the intersection over union (IOU) ratio between the selection box 30 and the prediction box is calculated, and a Hungarian algorithm is used to match the IOU. When the IOU is successfully matched, it indicates that the selection box 30 and the prediction box are successfully matched. As shown in step S38, all dimensions of this anomalous feature 28 are mathematically statistically analyzed, and the process returns to step S32. If the crosslinking ratio matching fails, it means that the abnormal feature 28 or the selection box 30 has not been matched successfully. In this case, the process will return to step S30 and wait for the detection of a new abnormal feature 28.

[0015] In one embodiment, the anomaly detection model 16, the three-dimensional feature prediction model 18 (including the depth prediction model 181 and the size prediction model 182) and the feature tracking prediction model 20 are each a trained neural network model.

[0016] In summary, the dimensional assessment system and method of this case, after acquiring real-time images, can analyze and evaluate the mathematical statistical results of abnormal feature dimensions on the real-time images through a built-in artificial intelligence (AI) model, and directly display the real-time images and the mathematical statistical results of abnormal feature dimensions on them on the display device, thereby improving the stability of the measurement results of abnormal feature dimensions. Therefore, this case can effectively assist physicians in providing stable and accurate abnormal feature dimensions for physicians to make more accurate diagnoses and improve the physician's user experience.

[0017] The above-described embodiments are only for illustrating the technical ideas and features of this case. Their purpose is to enable those skilled in this technology to understand the content of this case and implement it accordingly. They should not be used to limit the scope of the patent in this case. That is, all equivalent changes or modifications made in accordance with the spirit disclosed in this case should still be covered within the scope of the patent application in this case. [Simplified Explanation of the Diagram]

[0018] Figure 1 is a block diagram of a size assessment system and its connected testing instrument according to an embodiment of the present invention. Figure 2 is a flowchart of a size assessment method according to an embodiment of the present invention. Figure 3 is a schematic diagram of an architecture displaying a real-time image with a selection box marked on a time axis according to an embodiment of the present invention. Figure 4 is a schematic diagram of an architecture of a size assessment system and its real-time image displaying mathematical statistical results according to an embodiment of the present invention. Figure 5 is a flowchart of a size assessment system using a feature tracking prediction model to perform tracking prediction according to an embodiment of the present invention.

Claims

1. A size assessment system adapted to be electrically connected to an inspection instrument that inspects a target object and generates a real-time image, the size assessment system comprising: a computing unit, signal-connected to the inspection instrument, and having built-in anomaly detection model, a three-dimensional feature prediction model, and a feature tracking prediction model; the computing unit receiving the real-time image; the anomaly detection model detecting an anomaly feature in the real-time image, marking a selection box around the anomaly feature, and obtaining position information; the three-dimensional feature prediction model calculating a size of the anomaly feature based on the real-time image and the position information; the feature tracking prediction model confirming that when the anomaly feature detected in each current frame of the real-time image is the same as the anomaly feature detected in a previous frame, performing mathematical statistics on all sizes of the same anomaly feature, and generating a mathematical statistical result of the size corresponding to the anomaly feature when the computing unit receives a frame stillness signal; and a display device, electrically connected to the computing unit, the display device for displaying the real-time image, the selection box, and the mathematical statistical result. The static signal in the image was generated by the detection instrument being triggered.

2. The size assessment system as described in claim 1, wherein the three-dimensional feature prediction model further includes a depth prediction model and a size prediction model, wherein the depth prediction model estimates the depth of the anomalous feature based on the real-time image and the location information, and the size prediction model calculates the size of the anomalous feature based on the location information and the depth.

3. The size assessment system as described in claim 1, wherein the measuring instrument is an endoscope system.

4. The size evaluation system as described in claim 1, wherein the mathematical statistics system is a mean, an arithmetic mean, a geometric mean, a harmonic mean, a weighted mean, a truncated mean, a median, a mode, or a percentile.

5. The size assessment system as described in claim 1, wherein the feature tracking prediction model further comprises, in the step of performing tracking prediction: creating an identification code for each of the anomalous features on the real-time image; predicting a prediction box of the anomalous feature in the current frame using Kalman filtering; obtaining the selection box of the previous frame using the anomaly detection model; calculating the intersection over union (IOU) ratio of the selection box and the prediction box, matching the IOU using a Hungarian algorithm, and indicating that the selection box and the prediction box are successfully matched when the IOU is successfully matched, and performing mathematical statistics on all the dimensions of the anomalous feature.

6. The size assessment system as described in claim 5, wherein if the crosslinking ratio matching fails, the abnormal feature fails to match or the selection box fails to match.

7. The size assessment system as described in claim 1, wherein the abnormal feature includes proliferative or lesion tissue of the target object.

8. A size assessment method suitable for a real-time image generated by an inspection instrument inspecting a target object, the size assessment method comprising: receiving the real-time image and detecting an anomalous feature on the real-time image, thereby marking a selection box around the anomalous feature and obtaining position information; calculating a size of the anomalous feature based on the real-time image and the position information; when confirming that the anomalous feature detected in each current frame of the real-time image is the same as the anomalous feature detected in a previous frame, performing mathematical statistics on all the sizes of the same anomalous feature; generating a mathematical statistical result of the size corresponding to the anomalous feature upon receiving a frame stilling signal; and displaying the real-time image, the selection box, and the mathematical statistical result, wherein the frame stilling signal is generated by the inspection instrument being triggered.

9. The size assessment method as described in claim 8, wherein the testing instrument is an endoscope system.

10. The size evaluation method as described in claim 8, wherein the anomalous feature is detected and marked by a selection box by an anomaly detection model.

11. The size evaluation method as described in claim 8, wherein the size is generated by a three-dimensional feature prediction model.

12. The size assessment method as described in claim 11, wherein the three-dimensional feature prediction model further includes a depth prediction model and a size prediction model, the depth prediction model estimating a depth of the anomalous feature based on the real-time image and the location information, and the size prediction model calculating the size of the anomalous feature based on the location information and the depth.

13. The size evaluation method as described in claim 8, wherein the mathematical statistics are a mean, an arithmetic mean, a geometric mean, a harmonic mean, a weighted mean, a truncated mean, a median, a mode, or a percentile.

14. The size assessment method as described in claim 8, wherein the step of confirming whether the anomalous features are the same is performed by a feature tracking prediction model, and mathematical statistics are performed on all the dimensions of the same anomalous feature.

15. The size evaluation method as described in claim 14, wherein the feature tracking prediction model further comprises, in the step of performing tracking prediction: creating an identification code for each of the anomalous features on the real-time image; predicting a prediction box of the anomalous feature in the current frame using Kalman filtering; obtaining the selection box of the previous frame through anomaly detection; and calculating the intersection over union (IOU) ratio of the selection box and the prediction box, matching the IOU using a Hungarian algorithm, and indicating that the selection box and the prediction box are successfully matched when the IOU is successfully matched, and performing mathematical statistics on all the dimensions of the anomalous feature.

16. The size assessment method as described in claim 15, wherein if the crosslinking ratio matching is unsuccessful, the abnormal feature is not matched successfully or the selection box is not matched successfully.

17. The size assessment method as described in claim 8, wherein the abnormal feature comprises proliferative or lesion tissue of the target object.

Images