Methods, apparatuses, and computer readable storage media for reducing false positive detections in video signal images
By calculating information content and characteristics in video signal images, and using extended entropy measurement and similarity characteristics to set thresholds, false positive detection is suppressed, improving the specificity of object detection algorithms and saving computational resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HOYA CORPORATION
- Filing Date
- 2021-11-11
- Publication Date
- 2026-06-26
AI Technical Summary
Artificial intelligence-based object detection algorithms can produce false positives for non-domain objects (such as artifacts and blurred images), interfering with the actual information and reducing the specificity of the detection.
By calculating the information content and characteristics of video signal images, and utilizing extended entropy measurement and similarity characteristics, a variable threshold is set to suppress false positive detection. This includes feature analysis of image segments and comparison of characteristics of adjacent images, and processing is performed using circuit devices.
This improves the detection specificity of object detection algorithms, reduces the consumption of computing resources, and effectively reduces false positive detections.
Smart Images
Figure CN116249471B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, apparatus, and computer-readable storage medium for reducing false positive detection in video signal images. Background Technology
[0002] Artificial intelligence-based object detection algorithms are particularly sensitive to non-domain objects, which have not been trained on, such as artifacts and blurred images. False positives resulting from this can interfere with general perception and further obscure the actual information. Summary of the Invention
[0003] One objective of this invention is to reduce false positive detections through object detection algorithms in video signal images.
[0004] According to the present invention, this objective is achieved as specified in the appended claims.
[0005] According to a first embodiment of the invention, the calculation and indication (e.g., display) of potential false positive detection is suppressed in images of video signals without informational content. According to a second embodiment of the invention, the calculation and indication (e.g., display) of potential false positive detection is further suppressed in multiple image series of video signals without content-related connections.
[0006] According to the present invention, such suppression increases the detection specificity of object detection algorithms, particularly artificial intelligence-based object detection algorithms. Furthermore, computational power is saved because the object detection algorithm only needs to perform detection calculations and indications for images of video signals containing specific informational content. Attached Figure Description
[0007] In the following description, embodiments of the present invention will be given in more detail with reference to the accompanying drawings. The following shows:
[0008] Figure 1 A flowchart of a method is shown according to a first embodiment of the present invention, which is used for the calculation and indication of object detection in an image of a video signal.
[0009] Figure 2 A schematic diagram is shown according to an embodiment of the present invention to explain the calculations performed on an image of a video signal.
[0010] Figure 3 A flowchart of a method is shown according to a second embodiment of the present invention, which is used for the calculation and indication of object detection in an image of a suppressed video signal.
[0011] Figure 4 A schematic block diagram of a circuit device is shown, in which embodiments of the present invention can be implemented. Detailed Implementation
[0012] In the following text, the first embodiment refers to Figure 1 Describe it.
[0013] Figure 1 A flowchart of a method is shown according to a first embodiment, which is used for the calculation and indication of object detection in an image of a suppressed video signal.
[0014] The video signal is, for example, an endoscopic video signal, which is detected by endoscopic examination (e.g., of the stomach and / or intestines) using an endoscopic device, and the video signal is output from the endoscopic device. The object is, for example, a diagnostically relevant structure, such as a lesion, polyp, etc. According to one application example of the invention, object detection includes the detection of lesions, polyps, etc.
[0015] Endoscopic devices include rigid endoscopes, flexible endoscopes, and capsule endoscopes.
[0016] The video signal has passed Figure 1 After processing by the method shown, the data is fed into an algorithm that calculates and indicates the detection of objects in the image of the processed video signal. For example, the algorithm may be based on artificial intelligence or utilize machine learning techniques.
[0017] exist Figure 1 In process S101, information content of a segment of the current image from a series of images of the video signal is calculated. This information content is calculated based on, for example, an extended entropy metric. Then, process S103 continues.
[0018] The entropy level of an image, image detail, or image segment is based on the probability distribution of information present in the image (e.g., the intensity distribution within an 8-bit grayscale image). To determine the information content associated with an object, the entropy level described by Shannon is extended by including additional components in the extended entropy metric, such as spatial information (e.g., the first or second derivative of image intensity) or texture information (e.g., Haralick features or local binary patterns).
[0019] In process S103, the calculated information content of the current image segment is compared with the threshold. Then, process S105 continues.
[0020] The thresholds described above are determined, for example, for a specific domain or relative to an object, and are verified using independent test data. Therefore, the thresholds are preferably variable thresholds.
[0021] If the calculated information content is determined to be less than a threshold, then the calculation and indication of object detection for the current image segment are suppressed in process S105. According to one embodiment, the calculation and indication of object detection are suppressed not only for the current image segment but also for corresponding segments in subsequent images. The number of these subsequent images includes, for example, a selection from 1 to 1000 images.
[0022] For example, processes S101, S103, and S105 repeat for all images in this series. This series of images may include partial or complete video signals.
[0023] For example, after process S105 for the current image N, process S101 for the current image N+1 begins. Furthermore, parallel processing of processes S101, S103, and S105 (current image N+2 in S101, current image N+1 in S130, and current image N in S105) is also possible.
[0024] For example, if it is determined in process S105 that the calculated information content I is greater than or equal to the threshold t, then the calculation and indication of object detection for a segment of the current image N or for other images can be recognized under certain conditions.
[0025] This section includes, for example, complete details of the current image. Alternatively, this section includes portions of the current image that have been finely analyzed as needed. For example, this section includes portions of 2×2, 3×3, etc., segments of the current image.
[0026] Figure 1 The method shown is, for example, through Figure 4 The circuit device 40 shown is used to implement this, and the circuit device 40 will be described in more detail below.
[0027] For reference Figure 2 , Figure 2 A schematic diagram is shown according to an embodiment of the present invention to explain the calculations performed on an image of a video signal.
[0028] exist Figure 1 In the method shown, information content I is calculated for the current image N of a series of images of the video signal and compared with a threshold.
[0029] According to the second embodiment, in addition to the information content I of the current image N, the characteristic C of the current image N is also calculated. Furthermore, the similarity characteristic S is calculated via adjacent (e.g., previous) images N-1, N-2, N-3, etc., of the series of images in the video signal. It should be noted that "adjacent images" are not limited to "previous images." For example, in the case of non-real-time processing of the video signal, images after image N can also be considered for calculation.
[0030] Figure 3 A flowchart of a method is shown according to a second embodiment, which is used for the calculation and indication of object detection in an image of a suppressed video signal.
[0031] In process S301, the current image N of the series of images of the input video signal is selected. Then, process S303 continues, where, as in process 101, the information content I of a segment of the current image N is calculated. The information content I is calculated, for example, based on the extended entropy metric described above.
[0032] In subsequent process S305, as in process S103, it is determined whether the calculated information content I of a segment of the current image N is greater than or equal to a threshold t. That is, the information content I is compared with the threshold t. As described above, the threshold t is determined, for example, for a specific domain or relative to an object and verified using independent test data. Preferably, the threshold t is variable.
[0033] If it is determined in process S305 that the calculated information content I is not greater than or equal to the threshold t, then this ultimately leads to the suppression of the calculation and indication of the segment of the current image N in the series of images or the object detection of the current image N and other images in process S311, similar to process S105.
[0034] In the second embodiment, when the information content I is less than the threshold t, process S305 is followed by process S307.
[0035] In process S307, the characteristic C of a segment of the current image N is calculated. In subsequent process S309, the similarity characteristic S is updated using the calculated characteristic C of the segment of the current image N. The similarity characteristic S is calculated using neighboring images N-1, N-2, N-3, etc., of the series of images in the video signal. These neighboring images are adjacent to the current image N, for example, preceding it. Finally, process S309 is followed by process S311.
[0036] If it is determined in process S305 that the calculated information content I is greater than or equal to the threshold t, then process S313, corresponding to process S307, continues, in which the characteristic C of the segment of the current image N is calculated. Subsequently, process S315 corresponds to process S309, in which the similarity characteristic S is updated with the calculated characteristic C of the segment of the current image N.
[0037] It should be noted that processes S307 or S313 and S309 or S315 may also execute before process S305. In this configuration of the second embodiment, process S303 may also execute after process S307 or S313, or after process S309 or S315.
[0038] In process S317, which follows process S315, it is determined whether the updated similarity feature S is similar to at least one known interference feature.
[0039] If it is determined in process S317 that the updated similarity feature S is similar to at least one known interference feature, then process S311 continues for the calculation and indication of object detection for the segment of the current image N.
[0040] If it is determined in process S305 that the calculated information content I is greater than or equal to the threshold t, and if it is determined in process S317 that the updated similarity characteristic S is not similar to the known interference characteristic, then process S319 is executed, which allows the calculation and indication of object detection for segments of the current image N.
[0041] Figure 3 The process shown is, for example, repeated for all images in the series. The series of images may include a portion or all of the video signal.
[0042] For example, after process S311 or S319 of the current image N, process S301 of the current image N+1 of the video signal in this series of images begins. However, it is not necessary to wait for the end of process S311 or S319 of the current image N before inputting the current image N+1.
[0043] Figure 3 The processes shown can also be executed in parallel.
[0044] The suppression of detection calculations and indications in processes S105 and S311 includes, for example, at least image N not being fed to the algorithm, which calculates and indicates the detection of an object and removes it, for example, from the processed video signal input to the algorithm. Alternatively, at least image N is marked as an image in processes S105 and S311 that has not been edited in the processed video signal fed to the algorithm.
[0045] The calculations and indications allowed for detection in process S319 include, for example, holding an image N in a processed video signal, which is fed to an algorithm for calculating and indicating the detection of an object.
[0046] In process S307 or S313, the characteristic C of the current image N is calculated using typical features of the image, such as those of a video signal. Typical features of the image are identified, for example, depending on the type of object. Such features (similar to extended entropy measures) are based on spatial information (e.g., the first or second derivative of image intensity) or texture information (e.g., Haralick features or local binary patterns). According to one implementation, characteristic C is a vector with intervals. According to another implementation, characteristic C is a decision tree.
[0047] In process S309 or S315, the similarity feature S is updated, for example, based on the similarity measure between segments of the current image N and segments of neighboring images N-1, N-2, N-3, etc. The similarity feature S includes the feature C of neighboring images N-1, N-2, N-3, etc. It depends on the type of feature where independent features are linked.
[0048] The number of neighboring images N-1, N-2, N-3, etc. (based on which similarity characteristics are calculated) ranges from, for example, one image to one hundred images.
[0049] In process S317, it is determined whether the updated similarity feature S is similar to at least one known interference feature.
[0050] Interference characteristics are expressions of the characteristics described above. Interference characteristics are, for example, vectors or decision trees with intervals, where at each node a decision is made as to whether a computed feature (e.g., a Haralick feature) is within the interval. Known predefined interference characteristics include, for example: rinse water and rinse artifacts; blurred images caused when the endoscope lens rests directly on the mucosa and therefore everything visible is outside the lens's focusing range; blurred images caused by rapid movement during endoscopic examination or by dirt on the endoscope lens.
[0051] According to the implementation, to determine whether an updated similarity feature S is similar to at least one known interfering feature, an interval vector or decision tree is compared and a decision is made to determine whether they are sufficiently similar. For example, in both cases, the interval is used to determine whether a known interfering feature is included in the similarity feature S, or vice versa.
[0052] For example, the known interference property is a specific expression, such as a vector of features as described above, and the similarity property S includes intervals. If the independent values of the vectors are within the vector intervals of the similarity property S, then the known interference property and the similarity property S are similar.
[0053] On the other hand, if the similarity characteristic S is a specific expression and the known interference characteristics include an interval (which constitutes a preferred embodiment), then the characteristics are similar when the value of the specific expression is included within the interval. That is, if the similarity characteristic S is included in at least one of the known interference characteristics, then the similarity characteristic is similar to it.
[0054] According to the configuration of the second embodiment, the calculated characteristics C of segments of the current image N are weighted, and the similarity characteristic S is updated with the weighted calculated characteristics of the segments of the current image N. Therefore, the influence of the image on the similarity characteristic can be controlled. For example, a weighted method is applied that has a greater influence on the current image compared to earlier images. According to one embodiment, this relationship is linear. According to another embodiment, this relationship is related to other parameters.
[0055] Figure 3 The method shown is, for example, through Figure 4 The circuit device 40 shown is used to implement this.
[0056] The circuit device 40 includes a processing device (e.g., processing circuitry) 41 (such as one or more processors, such as a CPU), a storage device (e.g., storage circuitry) 42 (such as one or more read-only memories (ROMs), one or more random access memories (RAMs), etc.), and an interface (e.g., interface circuitry) 43.
[0057] According to one embodiment of the present invention, the memory device 42 stores a program that is executed when executed by the processor device 41. Figure 1 The method shown or Figure 3 The method shown.
[0058] According to another implementation example, Figure 1 or Figure 3 The method shown is implemented using a dedicated circuit structure that utilizes a processing device 41, a storage device 42, and an interface 43. For example, according to one embodiment of the invention, the processing device 41 and the storage device implement a calculating device, a determining device, a suppressing device, or an acknowledging device of the device.
[0059] Through interface 43, circuit device 40 receives video signals from an endoscope device in one aspect, and outputs the processed video signal to an algorithm for calculating and indicating the detection of objects in the processed video signal in another aspect.
[0060] According to the present invention, a video signal is filtered based on its image content, and the thus processed video signal is fed to an algorithm for calculating and indicating the detection of objects in the processed video signal. Compared to filtering based on the characteristics and distribution of detection, image content-based filtering has the advantage that the detection indication in the algorithm does not experience any additional delay.
Claims
1. A method for reducing false positive detection in video signal images, comprising the following steps: Calculate the information content of a segment of the current image from a series of images of the video signal (S101, S303). The video signal must be fed to the algorithm for calculating and indicating the detection of objects in the video signal. Determine (S103, S305) whether the calculated information content of the segment of the current image is greater than or equal to a threshold, and When it is determined that the calculated information content is not greater than or equal to the threshold, the calculation and indication of object detection in at least the segment of the current image or the current image and other images in the series of images are suppressed (S105, S311). The method for reducing false positive detections in video signal images further includes the following steps: Calculate (S307, S313) the characteristics of the segment of the current image and update the similarity characteristics based on the calculated characteristics of the segment of the current image, the similarity characteristics having been calculated using neighboring images of the series of images of the video signal, the neighboring images being adjacent to the current image. When it is determined that the calculated information content is greater than or equal to the threshold. Determine (S317) whether the updated similarity property is similar to at least one known interference property, and When it is determined that the updated similarity characteristic is similar to at least one known interference characteristic, the calculation and indication of object detection of at least the segment of the current image or the current image and other images in the series of images are suppressed (S311).
2. The method for reducing false positive detection in video signal images according to claim 1, further comprising: When it is determined that the calculated information content is greater than or equal to the threshold, and when it is determined that the updated similarity characteristic is not similar to the known interference characteristic, the calculation and indication steps of the segment of the current image or the object detection of the current image and the other images in the series of images are acknowledged (S319).
3. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein the similarity characteristic is updated based on a similarity metric between the segment of the current image and the segment of the adjacent image.
4. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein... The number of adjacent images includes a selection from a series of 1 to 1000 images, and / or The number of other images includes a selection from a range of 1 to 1000 images.
5. The method for reducing false positive detection in video signal images according to claim 1 or 2, further comprising: The step of weighting the calculated characteristics of the segment of the current image. The similarity characteristic is updated by weighting the characteristic of the segment of the current image.
6. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein the information content is calculated based on an extended entropy metric.
7. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein the segment includes complete details of the current image or a portion of the current image, the complete details being finely parsed as needed.
8. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein... The video signal is an endoscopic video signal, and / or The object is a diagnostic-related structure, and / or The algorithm is based on artificial intelligence or utilizes machine learning techniques, and / or The threshold is variable.
9. The method for reducing false positive detection in video signal images according to claim 1 or 2, wherein the method for reducing false positive detection in video signal images is repeated for the series of images.
10. A computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform a method for reducing false positive detection in video signal images according to any one of claims 1 to 9.
11. An apparatus for reducing false positive detections in video signal images, comprising: A calculation device is used to calculate information content of a segment of a current image from a series of images of a video signal, the video signal being fed to an algorithm for calculating and indicating the detection of objects in the video signal. A determining device, the determining device being used to determine whether the calculated information content of the segment of the current image is greater than or equal to a threshold, and A suppression device, configured to suppress the calculation and indication of object detection in at least the segment of the current image or between the current image and other images in a series of images when it is determined that the calculated information content is not greater than or equal to the threshold. The calculator is configured to calculate similarity characteristics of adjacent images of the series of images of the video signal, and to calculate characteristics of the segment of the current image and to update the similarity characteristics based on the calculated characteristics of the segment of the current image, wherein the adjacent images are adjacent to the current image. The determining device, when it has been determined that the calculated information content is greater than or equal to the threshold, is configured to determine whether the updated similarity characteristic is similar to at least one known interference characteristic, and When the determining device determines that the updated similarity characteristic is similar to at least one known interference characteristic, the suppressing device is configured to suppress the calculation and indication of object detection in at least the segment of the current image or the current image and other images in the series of images.
12. The apparatus for reducing false positive detection in video signal images according to claim 11, further comprising an acknowledgment device configured to acknowledge the calculation and indication of object detection of the segment of the current image or the current image and the other images when the determining device determines that the calculated information content is greater than or equal to the threshold and the updated similarity characteristic is not similar to a known interference characteristic.
13. The apparatus for reducing false positive detection in video signal images according to claim 11 or 12, wherein... The calculator is configured to weight the calculated characteristics of the segments of the current image and update the similarity characteristics based on the weighted calculated characteristics of the segments of the current image.
14. The apparatus for reducing false positive detections in video signal images according to claim 11 or 12, wherein the video signal is an endoscopic video signal, and the object is a diagnostically relevant structure, the diagnostically relevant structure including lesions and / or polyps.