Abnormal feature evaluation system and method thereof

Abstract

The present application provides an abnormal feature evaluation system and method thereof. An operation device is built with an abnormality detection model and an attribute feature estimation model. After the operation device receives a real-time image, the abnormality detection model detects an abnormal feature on the real-time image, marks a selection box around the abnormal feature, and acquires a position information. The attribute feature estimation model calculates a lesion index of the abnormal feature according to the real-time image and the position information. A display device is used to display the real-time image and the selection box. When the operation device analyzes and judges that the real-time image is a still picture, the lesion index is outputted, so that the display device displays the real-time image and the lesion index. When the operation device analyzes and judges that the real-time image is a non-still picture, the display of the lesion index is turned off.

Description

Technical Field

[0001] This application relates to an abnormal feature assessment system and method that can display lesion indicators in real time based on image analysis when the image is judged to be a still image. Background Technology

[0002] An endoscope is a device that uses an endoscope to examine internal organs or structures. It enters the body through various tubes to observe the internal condition and determine the presence of any lesions. Endoscopes include esophagoscopy, gastroscopy, and colonoscopy. Taking a common colonoscope as an example, it uses a flexible fiber optic endoscope to directly observe the large intestine and check for lesions such as polyps or tumors. During the examination, the colonoscope is connected to a monitor to display real-time images of the internal intestinal structure, allowing physicians to examine or diagnose the health of the patient's large intestine.

[0003] During endoscopic examinations, a special type of endoscope requires buttons to perform dimensional measurement functions. When a polyp is detected by the endoscope, an external button is pressed to capture the image and calculate the polyp's size, which is then displayed on the screen. This method of examination, requiring buttons on a special type of endoscope, is inconvenient due to its specific operation and cannot be widely used in examinations. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide an abnormal feature evaluation system, which is suitable for electrical connection with a detection instrument. The detection instrument examines a target object and generates a real-time image. The abnormal feature evaluation system includes a computing device and a display device. The computing device is signal-connected to the detection instrument and has a built-in abnormal detection model and an attribute feature estimation model. After receiving the real-time image, the abnormal detection model detects an abnormal feature on the real-time image, marks a selection box around the abnormal feature, and obtains a position information. The attribute feature estimation model calculates a lesion index of the abnormal feature based on the real-time image and the position information. When the computing device analyzes and determines that the real-time image is a still image, it outputs the lesion index. The display device is electrically connected to the computing device and is used to display the real-time image and the selection box. When the computing device determines that the real-time image is a still image, the display device displays the real-time image and the lesion index; when the computing device determines that the real-time image is not a still image, the display of the lesion index is turned off.

[0005] This application also provides an abnormal feature assessment method suitable for a real-time image generated by a detection instrument examining a target object. This abnormal feature assessment method includes: receiving the real-time image to detect an abnormal feature on the real-time image and obtain location information; calculating a lesion index corresponding to the abnormal feature based on the real-time image and location information; outputting the lesion index upon activation of a still image function; displaying the real-time image and the lesion index; and deactivating the display of the lesion index upon deactivation of the still image function.

[0006] In summary, the abnormal feature assessment system and method for real-time display of lesion indicators of this application, after acquiring real-time images, does not require additional hardware buttons to perform measurement functions when displaying lesion indicators. It automatically determines when the image is still using the existing image stillness function of the detection instrument, triggering the display of the lesion indicators that need to be displayed. When the image stillness function is released, the display of lesion indicators is turned off, reducing the inconvenience caused by physicians needing to use too many buttons during examination. Therefore, this application can effectively assist physicians by providing an accurate and easy-to-use abnormal feature assessment system and method, achieving real-time display of specific functions using existing detection instrument functions, providing intuitive and simple use, and thus improving the physician's user experience.

[0007] The other effects and embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a block diagram of an anomaly feature assessment system and its connected detection instrument according to an embodiment of this application;

[0010] Figure 2 This is a flowchart illustrating an anomaly feature evaluation method according to an embodiment of this application;

[0011] Figure 3 This is a real-time image diagram of an abnormal feature evaluation system according to an embodiment of the present application before, during and after the screen freeze function is activated;

[0012] Figure 4 This is a schematic diagram of the architecture of an abnormal feature assessment system according to an embodiment of the present application, which displays real-time images and lesion indicators when the screen is in a static state.

[0013] Symbol Explanation Detailed Implementation

[0014] The embodiments of this application will be described below with reference to relevant drawings. Furthermore, some components or structures may be omitted in the drawings of the embodiments to clearly show the technical features of this application. In these drawings, the same reference numerals denote the same or similar components or circuits.

[0015] Figure 1 For a block diagram of an anomaly feature evaluation system and its connected detection instrument according to an embodiment of this application, please refer to [link / reference]. Figure 1 As shown, an anomaly feature evaluation system 10 is adapted to be electrically connected to a detection instrument 20. The detection instrument 20 examines and captures a target object 22 and generates a real-time image 24, which is then transmitted to the anomaly feature evaluation system 10. The anomaly feature evaluation system 10 mainly includes a computing device 12 and a display device 14. The computing device 12 is signal-connected to the detection instrument 20. In one embodiment, the computing device 12 is electrically connected to the detection instrument 20 via a Universal Serial Bus (USB) interface, a High Definition Multimedia Interface (HDMI) interface, or a Serial Digital Interface (SDI) interface, but this application is not limited thereto. The computing device 12 is electrically connected to the display device 14. In one embodiment, the computing device 12 can be electrically connected to the display device 14 via a High Definition Multimedia Interface (HDMI), a DisplayPort (DP) interface, or a Serial Digital Interface (SDI) interface, but this application is not limited thereto. The computing device 12 has a built-in anomaly detection model 16 and an attribute feature estimation model 18. The computing device 12 also has a built-in algorithm. After receiving the real-time image 24 from the self-detection instrument 20, the computing device 12 can process the real-time image 24 to detect abnormal features 26 using the anomaly detection model 16, such as... Figure 3 As shown, the attribute feature estimation model 18 is used to calculate a lesion index 30 corresponding to the abnormal feature 26. When the computing device 12 determines through algorithm analysis that the real-time image 24 is a still image, it can output the lesion index 30 in real time. The display device 14 is used to display the real-time image 24 processed by the computing device 12. When the still image function is activated, the real-time image 24 and the lesion index 30 are displayed in real time.

[0016] In one embodiment, the detection instrument 20 is an endoscope system, such as an esophagoscopy endoscope, a gastroscopy endoscope, or a colonoscopy endoscope. When the detection instrument 20 is an esophagoscopy endoscope, the target object 22 is the esophagus; when the detection instrument 20 is a gastroscopy endoscope, the target object 22 is the cardia, pylorus, fundus, or body of the stomach; when the detection instrument 20 is a colonoscopy endoscope, the target object 22 is the large intestine, cecum, ascending colon, transverse colon, descending colon, sigmoid colon, rectum, etc. In one embodiment, abnormal features include proliferating or diseased tissue of the target object 22, such as polyps, tumors, or other formations on the target object tissue. In this embodiment, when the target object 22 is the intestine, the abnormal feature is a colonic polyp.

[0017] In one embodiment, the computing device 12 is a computer host or other electronic device capable of independent operation, used in conjunction with the display device 14, but this application is not limited thereto. In another embodiment, this application can also directly use a laptop computer to simultaneously replace the functions of the computing device 12 and the display device 14, so that the laptop computer can simultaneously handle the work of the computing device 12 and the display device 14.

[0018] Continue Figure 1 The architecture shown illustrates the detailed process of each step in the anomaly feature evaluation system 10 of this application when performing the anomaly feature evaluation method. Please also refer to... Figure 1 , Figure 2 and Figure 3 As shown, after the detection instrument 20 inspects the target object 22 and generates a real-time image 24, as shown in step S10, the computing device 12 receives the real-time image 24 from the detection instrument 20. At this time, the computing device 12 transmits the real-time image 24 to the display device 14, so that the real-time image 24 is displayed on the display device 14. As shown in step S12, the computing device 12 detects an abnormal feature 26 on the real-time image 24 through the anomaly detection model 16, such as... Figure 3 As shown, a selection box 28 is marked around the abnormal feature 26, and the location information corresponding to the abnormal feature 26 is obtained. As shown in step S14, the computing device 12 executes the attribute feature estimation model 18, which calculates a lesion index 30 corresponding to the abnormal feature 26 based on the real-time image 24 and the location information. As shown in step S16, when the detection instrument 20 is triggered to activate a still image function, the computing device 12 will automatically analyze whether the real-time image 24 is a still image through an algorithm. When the computing device 12 determines that the real-time image 24 is a still image, the computing device 12 will output the lesion index 30 corresponding to the abnormal feature 26 based on the still image. As shown in step S18, the computing device 12 will add the lesion index 30 to the real-time image 24 and output it to the display device 14, so that the display device 14 displays the real-time image 24 and the lesion index 30 in real time. Figure 3 and Figure 4As shown, the lesion indicator 30 in this embodiment is based on the lesion size, but this application is not limited thereto. Finally, as shown in step S20, when the detection instrument 20 is triggered to release the screen freeze function, the computing device 12 will automatically analyze whether the real-time image 24 is a non-static image through an algorithm. When the computing device 12 determines that the real-time image 24 is a non-static image, the computing device 12 will turn off the display of the lesion indicator 30 according to this non-static image. The computing device 12 will only output the real-time image 24 and the selection box 28 to the display device 14, so that the display device 14 will restore only displaying the real-time image 24 and the selection box 28, and not displaying the lesion indicator 30.

[0019] In one embodiment, the attribute feature estimation model 18 is either a size feature estimation model or a degree feature estimation model. When the attribute feature estimation model 18 is a size feature estimation model, the lesion index 30 is a lesion size of the abnormal feature 26. For example, the lesion size can be the longest diameter of the esophageal cancer area, the size of a polyp (including the large intestine, cecum, ascending colon, transverse colon, descending colon, sigmoid colon, rectum, cardia, pylorus, fundus, or body of the stomach, etc.), or the proportion of the longest diameter of the intestinal metaplasia area. When the attribute feature estimation model 18 is a degree feature estimation model, the lesion index 30 is a degree of lesion invasion or a degree of lesion inflammation of the abnormal feature 26, such as the degree of esophageal cancer invasion or the degree of atrophic gastritis.

[0020] In one embodiment, please also refer to Figure 1 , Figure 3 and Figure 4 As shown, the detection instrument 20 is also equipped with a function key 32. This function key 32 can be a built-in key of the detection instrument 20 or a key on the keyboard (not shown) connected to the computing device 12. Taking the built-in key of the detection instrument 20 as an example, the physician can press the function key 32 to activate the screen freeze function of the detection instrument 20 and generate a still image, so that the display device 14 displays the real-time image 24 and the lesion index 30. After the screen freeze function is activated, pressing the function key 32 again will deactivate the screen freeze function and generate a non-still image, so that the display device 14 displays the real-time image 24 and the selection box 28 but does not display the lesion index 30.

[0021] In one embodiment, regarding the algorithm built into the computing device 12, this application does not require additional hardware buttons to perform measurement functions. By comparing the correlation between preceding and following images in the real-time image 24 pixel-by-pixel, the similarity between the preceding and following images is confirmed. A sampling method is used to compare the images, ensuring the algorithm meets the clinical needs of displaying the real-time image 24. Furthermore, the algorithm can automatically analyze the similarity between preceding and following images and quickly determine if any changes exist. When the algorithm detects that the similarity between preceding and following images reaches a certain standard, it indicates a static image and automatically triggers the display of lesion indicators. When a non-static image is detected, the display of lesion indicators is turned off, reducing the inconvenience caused by physicians needing to use too many buttons during examination.

[0022] In one embodiment, the anomaly detection model 16 and the attribute feature estimation model 18 (including the size feature estimation model and the degree feature estimation model) are each a trained neural network model.

[0023] In summary, the abnormal feature assessment system and method for real-time display of lesion indicators of this application, after acquiring real-time images, does not require additional hardware buttons to perform measurement functions when displaying lesion indicators. It automatically determines when the image is still using the existing image stillness function of the detection instrument, triggering the display of the lesion indicators that need to be displayed. When the image stillness function is released, the display of lesion indicators is turned off, reducing the inconvenience caused by physicians needing to use too many buttons during examination. Therefore, this application can effectively assist physicians by providing an accurate and easy-to-use abnormal feature assessment system and method, achieving real-time display of specific functions using existing detection instrument functions, providing intuitive and simple use, and thus improving the physician's user experience.

[0024] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of this application, and are not intended to limit the implementation methods of the technology of this application in any way. Any person skilled in the art may make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in this application, but these should still be regarded as the technology or embodiments that are substantially the same as those of this application.

Claims

1. An anomaly feature assessment system, suitable for electrical connection with a detection instrument, the detection instrument examining a target object and generating real-time images, characterized in that, The anomaly feature evaluation system includes: A computing device, signal-connected to the detection instrument, and having a built-in anomaly detection model and attribute feature estimation model, receives the real-time image. The anomaly detection model detects abnormal features on the real-time image, marks selection boxes around the abnormal features, and obtains location information. The attribute feature estimation model calculates lesion indicators of the abnormal features based on the real-time image and the location information. When the computing device analyzes and determines that the real-time image is a still image, it outputs the lesion indicators. A display device is electrically connected to the computing device. The display device is used to display the real-time image and the selection box. When the computing device determines that the real-time image is a still image, the display device displays the real-time image and the lesion indicators. When the computing device determines that the real-time image is not a still image, the display of the lesion indicators is turned off.

2. The anomaly feature evaluation system according to claim 1, characterized in that, The testing instrument is an endoscope system.

3. The anomaly feature evaluation system according to claim 2, characterized in that, The endoscopic system is an esophageal endoscope, gastroscope, or colon endoscope.

4. The anomaly feature evaluation system according to claim 1, characterized in that, The attribute feature estimation model is either a size feature estimation model or a degree feature estimation model.

5. The anomaly feature evaluation system according to claim 4, characterized in that, When the attribute feature estimation model is the size feature estimation model, the lesion index is the lesion size of the abnormal feature.

6. The anomaly feature evaluation system according to claim 4, characterized in that, When the attribute feature estimation model is the degree feature estimation model, the lesion index is the degree of lesion invasion or the degree of lesion inflammation of the abnormal feature.

7. The anomaly feature evaluation system according to claim 1, characterized in that, The abnormal features include proliferating or diseased tissue of the target object.

8. The anomaly feature evaluation system according to claim 1, characterized in that, The computing device also has a built-in algorithm to analyze and determine whether the real-time image is a still image or a non-still image.

9. The anomaly feature evaluation system according to claim 8, characterized in that, The detection instrument is also equipped with a function key, which can be pressed to activate the image stilling function of the detection instrument. When the computing device analyzes and determines that the real-time image is the still image through the algorithm, it causes the display device to display the real-time image and the lesion indicators.

10. The anomaly feature evaluation system according to claim 9, characterized in that, After the image freeze function is activated, the image freeze function can be deactivated by pressing the function key again. When the computing device analyzes and determines that the real-time image is a non-still image through the algorithm, the display device displays the real-time image and the selection box but does not display the lesion indicators.

11. An abnormal feature assessment method, suitable for detecting real-time images generated by an instrument inspecting a target object, characterized in that, The abnormal feature evaluation method includes: Receive the real-time imagery to detect abnormal features on the real-time imagery and obtain location information; The lesion index of the abnormal features is calculated based on the real-time images and the location information; The lesion indicators are output based on the activation of the still image function; Display the real-time images and the lesion indicators; and Once the screen freeze function is deactivated, the display of the lesion indicators will be turned off.

12. The abnormal feature evaluation method according to claim 11, characterized in that, The testing instrument is an endoscope system.

13. The abnormal feature evaluation method according to claim 12, characterized in that, The endoscopic system is an esophageal endoscope, gastroscope, or colon endoscope.

14. The abnormal feature evaluation method according to claim 11, characterized in that, The abnormal features are detected by the abnormality detection model, and a selection box can be displayed around the abnormal features.

15. The abnormal feature evaluation method according to claim 11, characterized in that, The lesion indicators are generated by the attribute feature estimation model.

16. The abnormal feature evaluation method according to claim 15, characterized in that, The attribute feature estimation model is either a size feature estimation model or a degree feature estimation model.

17. The abnormal feature evaluation method according to claim 16, characterized in that, When the attribute feature estimation model is the size feature estimation model, the lesion index is the lesion size of the abnormal feature.

18. The abnormal feature evaluation method according to claim 16, characterized in that, When the attribute feature estimation model is the degree feature estimation model, the lesion index is the degree of lesion invasion or the degree of lesion inflammation of the abnormal feature.

19. The abnormal feature evaluation method according to claim 11, characterized in that, The abnormal features include proliferating or diseased tissue of the target object.

20. The abnormal feature evaluation method according to claim 11, characterized in that, When the image freeze function is activated, the algorithm analyzes and determines whether the real-time image is a still image or a non-still image.

21. The abnormal feature evaluation method according to claim 20, characterized in that, The detection instrument is also equipped with a function key. By pressing the function key, the image stilling function of the detection instrument is activated. When the algorithm analyzes and determines that the real-time image is a still image, the real-time image and the lesion indicators are displayed.

22. The abnormal feature evaluation method according to claim 21, characterized in that, After the image freeze function is activated, the image freeze function can be deactivated by pressing the function key again. When the algorithm analyzes and determines that the real-time image is a non-static image, the real-time image is displayed but the lesion indicators are not displayed.

Images