Devices, methods, and computer-readable storage media for detecting objects in video signals based on visual evidence derived from the output of a machine learning model.

Abstract

Detections in video frames of the video signal are correlated to generate a detection chain (S307), the video frames being output from a machine learning model (S305). The display of the detections in the video signal is caused by the position of the detections in the detection chain, the confidence value of the detections, and the position of the detections (S309).

Description

Technical Field

[0001] The present invention relates to apparatus, method and computer-readable storage medium for detecting objects in video signals based on visual evidence utilizing the output of a machine learning model. Background Technology

[0002] Traditional machine learning helps to find a decision function that maps features relevant to the target being detected onto an image to classify it into a label. Machine learning algorithms must go through a training phase, during which the decision function is modified to minimize the error on the training data. After the training phase is complete, the decision function is fixed and used to predict previously unseen data.

[0003] Deep learning, a technique that can automatically discover suitable features, is employed to provide machine learning algorithms with appropriate features related to the detection target (e.g., color distribution, gradient histogram, etc.).

[0004] Deep learning typically utilizes deep convolutional neural networks. Compared to traditional neural networks, the first layer has been replaced by convolutional operations. This allows convolutional neural networks to learn image filtering techniques capable of extracting features. Because the filter coefficients are now part of the decision function, the training process can also optimize feature extraction. Therefore, convolutional neural networks can automatically discover useful features.

[0005] A distinction must be made between classification and object class detection. With classification, the input is an image and the output is a class label. Therefore, classification answers questions such as "Does this image contain a detected object, such as a polyp? (Yes / No)". In contrast, object class detection provides not only class labels but also the location of the object in the form of a bounding box. It's possible to think of an object detector as a classifier applied to many different small patches of an image.

[0006] The well-known approach for object detection is the Single-Step Multi-Frame Detector (SSD), which was disclosed by W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, CY. Fu, and ACBerg in “SSD: Single-Step Multi-Frame Detector”, European Conference on Computer Vision 2016. Summary of the Invention

[0007] The basic principle of SSD is to place an imaginary grid of so-called anchor boxes across the image. At each location, there are multiple anchor boxes with different scales and aspect ratios. Regarding the detection of a certain target (e.g., a polyp), the question is "Does this anchor box contain the target, such as a polyp (yes / no)?". Therefore, a neural network with two output neurons is needed for each anchor box. The anchor box is classified as positive or negative based on which of the two output neurons is more strongly activated.

[0008] Detectors (such as SSD) provide a framework for object detection in still images.

[0009] The purpose of this invention is to provide an object detection and display mechanism that can detect the temporal consistency of video signals based on the output of a machine learning model.

[0010] This objective is achieved by means of apparatus, methods, and computer-readable storage media as defined in the appended claims.

[0011] According to one aspect of the present invention, an apparatus is provided, the apparatus comprising:

[0012] An apparatus for obtaining one or more current detections of at least one current video frame of a series of consecutive video frames of a video signal output from a machine learning model, the at least one current video frame being input to the machine learning model, wherein the current detection of the one or more current detections includes a confidence value and the position of a detected target in the at least one current video frame, the confidence value indicating the probability that the current detection includes the detected target to be detected by the machine learning model;

[0013] An apparatus for generating a detection chain by associating detections output from the machine learning model, wherein the current detection of one or more current detections is associated with a previous detection of one or more previous detections of at least one previous video frame obtained from the machine learning model for the series of video frames, the at least one previous video frame being preceding the at least one current video frame and having been input to the machine learning model, wherein the previous detection of one or more previous detections includes a confidence value and the position of the detection target in the at least one previous video frame, the confidence value indicating the probability that the previous detection includes the detection target, wherein the current detection is associated with the previous detection based on the positions of the current and previous detections;

[0014] An apparatus for causing the display of the at least one current detection in the video signal based on the position of the current detection in the detection chain, the confidence value of the current detection, and the position of the current detection; and

[0015] For at least one next video frame in the series that is the at least one current video frame, the apparatus for obtaining, generating and causing display is repeated.

[0016] According to one embodiment of the present invention, the current detection is associated with the previous detection such that if the overlap between the positions of the current and previous detections satisfies a predetermined condition, the current detection and the previous detection belong to the same detection chain.

[0017] According to one embodiment of the present invention, when the current detection belongs to N+M detections in the detection chain, the current detection is displayed, where N and M are positive integers equal to or greater than 1, and where N indicates the first detection at time N in the detection chain, and where the current detection is not displayed when it belongs to the first detection at time N in the detection chain.

[0018] According to one embodiment of the present invention, the display of the current detection is triggered when the confidence value of the current detection is equal to or greater than a first threshold.

[0019] According to one embodiment of the present invention, the display of the current detection is caused when the confidence value of the current detection is equal to or greater than a second threshold and when the confidence value of a previous detection belonging to the same detection chain as the current detection is equal to or greater than a first threshold, wherein the second threshold is less than the first threshold.

[0020] According to one embodiment of the invention, the device further includes an apparatus for performing smoothing processing on the position of the detection chain.

[0021] According to one embodiment of the invention, video signals are captured by an endoscope during the examination process.

[0022] According to one embodiment of the present invention, the target for detection is a polyp.

[0023] According to one embodiment of the present invention, an object detection and display mechanism is provided, which utilizes a machine learning model to achieve temporal consistency detection of a video signal based on visual evidence of video frames.

[0024] According to one implementation, the object detection and display mechanism processes video signals of moving images by utilizing the output of a machine learning model. This object detection and display mechanism is able to suppress artifacts, such as false detections, missing detections, and unstable localizations as described below, while also reducing the training load of the machine learning model.

[0025] According to one embodiment of the present invention, a heuristic approach is employed to perform object detection in video signals of moving images by utilizing the output of a machine learning model, thereby visually improving the quality of the detection.

[0026] Using this invention, for example, when performing an endoscopic examination (such as a colonoscopy screening), it can help a physician focus his / her attention on relevant image areas that include tissue consistent with the appearance of a polyp.

[0027] In the following description, the present invention will be described by way of embodiments thereof with reference to the accompanying drawings. Attached Figure Description

[0028] Figure 1 A schematic diagram illustrating the cross-union ratio (IoU) standard is shown according to an embodiment of the present invention, which is used to group detection components into detection chains.

[0029] Figure 2 An illustration is provided according to an embodiment of the present invention to schematically illustrate an "ideal" solution for performing object detection in a video signal and a heuristic approach to performing object detection.

[0030] Figure 3 A flowchart illustrating the object detection and display process is shown according to an embodiment of the present invention.

[0031] Figure 4 A schematic block diagram is shown illustrating the configuration of the control unit, wherein an example of an embodiment of the present invention is implemented.

[0032] Figure 5 An illustration is provided according to an embodiment of the present invention, illustrating the PR curve when flicker suppression is applied to the object detection and display process.

[0033] Figure 6 An illustration is provided according to an embodiment of the present invention to illustrate hysteresis thresholding to be applied to the object detection and display process.

[0034] Figure 7 An illustration is provided according to an embodiment of the present invention to illustrate the PR curve when hysteresis thresholding is applied to the object detection and display process. Detailed Implementation

[0035] According to the present invention, the output of a machine learning model is used. The machine learning model outputs one or more detections for each video frame of a video signal input to the machine learning model. For example, the video signal is captured by an endoscope during an examination.

[0036] Specifically, the machine learning model outputs a confidence value and the location of each detection. The confidence value indicates the probability that the detection includes the target to be detected by the machine learning model, and the location indicates the region of the detected target in the video frame. For example, the detected target is a polyp.

[0037] For example, as a machine learning model, a neural network is employed, which has two output neurons for each anchor box as described above. The anchor box is classified as positive or negative based on the stronger activation of the two output neurons. The location of the detected target is based on the location of the anchor box. The outputs of the two neurons form a confidence value.

[0038] The machine learning model has been trained using training data for object detection, i.e., training data for detecting targets (such as polyps), which includes images and annotated objects (e.g., in the form of bounding boxes).

[0039] To objectively evaluate the performance improvement of machine learning models on datasets and using filtering techniques, standard metrics are used for object detection tasks. The relevant metrics used are precision, recall, and average precision (AP). Precision is defined as the score of correctly detected instances relative to the total number of detections returned by the machine learning model. Recall is defined as the score of correctly detected instances relative to the total number of instances to be detected.

[0040] Therefore, precision and recall can be formally defined as...

[0041] Precision = TP / (TP+FP) Recall = TP / (TP+FN)

[0042] TP represents the number of true positives (correctly detected), FP represents the number of false positives (incorrectly detected), and FN represents the number of false negatives (missed detections).

[0043] To classify detection as "real" or "false," the quality of the positioning needs to be measured. The intersection-over-union (IoU) metric is used to measure positioning quality.

[0044] IoU(A,B)=|A∩B| / |A∪B|

[0045] The intersection-union ratio (IoU) will be 1 only in the case of perfect localization. Both underdetection and overdetection are penalized. A detection is considered correctly classified if the IoU between the detection and the annotation is ≥ 0.5. Figure 1 Examples of poor positioning, minimum acceptable positioning, and perfect positioning are given.

[0046] While precision and recall are available tools for evaluating algorithm performance, they have a significant drawback: the classifier outputs a confidence value, which measures the probability that an image region contains the detected target (such as a polyp). A threshold must be applied to make the final decision on whether to display the detection. However, the values ​​of precision and recall depend on this threshold. For example, by increasing the threshold, it is always possible to increase precision at the expense of recall.

[0047] Therefore, as will be described later Figure 5 and Figure 7 In this calculation, precision (P) and recall (R) are evaluated against all possible thresholds to plot a precision-recall curve. The area under the curve is called the average precision (AP) and is used as a metric to indicate how well different classifiers perform overall. This value can be used to compare different classifiers with each other.

[0048] In the following text, it is assumed that the machine learning model (whose output is used in this invention) has been trained to achieve good performance when generating detections based on video frames of a video signal. However, information from previous video frames may be able to, or even further improve, performance.

[0049] Generating detection based solely on the current video frame can lead to the following artifacts:

[0050] — False positives: These often appear in a single frame of the video signal and disappear in the next frame.

[0051] —Missing Detection: When a machine learning model has detected a target (e.g., a polyp), the detection is usually very stable across multiple consecutive frames of the video signal. However, the confidence level of the detection may sometimes briefly drop below the detection threshold, causing detection flicker.

[0052] —Unstable localization: The machine learning model estimates bounding boxes to localize each detection. If the input image changes slightly, the localization will also change. However, this change may appear non-smooth to the user.

[0053] Detectors that can take into account past video frames have a good chance of reducing these artifacts. However, training such detectors would require collecting video sequences as a dataset. This would place a huge burden on doctors, as they would need to label every single frame in the video signal.

[0054] To avoid using video sequences to train machine learning models, this invention employs a heuristic solution to visually improve detection quality. To this end, a filtering heuristic is introduced to address the artifacts described above. Figure 2 The illustrations show schematic depictions of the "ideal" solution and the solution according to the present invention.

[0055] The "ideal" solution is shown in Figure 2 The left-hand side. For example, a deep convolutional neural network (DCNN) long short-term memory (LSTM) architecture takes multiple video frames as input and outputs detections based on visual evidence from multiple frames.

[0056] The solution according to the invention is shown in Figure 2 The right-hand side. Prediction is based on independent frames and filtered heuristically.

[0057] The difference between these two solutions lies in the fact that a true multi-frame detector will be able to rely on visual evidence from multiple video frames. The heuristic solution according to the invention relies on visual evidence from the current frame to generate a detection. As described above, the detection includes position and confidence value. Therefore, the heuristic approach can operate on these values.

[0058] According to one embodiment of the invention, before applying a filtering heuristic, the detections are correlated with each other across multiple video frames. According to one embodiment of the invention, it is assumed that the detections generally do not tend to move rapidly across consecutive video frames, and the positions of the detections are used to correlate them. According to one implementation example, the previously described Intersection over Union (IoU) criterion is used to correlate the detections. For example, detections with IoU ≥ 0.3 in consecutive video frames are considered part of the same detection chain. Consecutive detections with IoU < 0.3 are considered part of different detection chains. The filtering stages, described below, each operate on these detection chains.

[0059] Before describing the filtering stage, refer to Figure 3 , Figure 3 An embodiment of the present invention illustrates the process of object detection and display.

[0060] exist Figure 3 In step S305, one or more current detections are obtained regarding at least one current video frame input to the machine learning model, as output from the machine learning model. The at least one current video frame belongs to a series of consecutive video frames of the video signal. According to one example implementation, the video signal is acquired from an endoscopic device that captures the video signal. For example, the video signal includes moving images.

[0061] One or more current detections include a confidence value and the position of the detected target in at least one current video frame. The confidence value indicates the probability that the current detection includes the detected target that will be detected by the machine learning model. In step S305, one or more current detections are obtained for at least one current video frame.

[0062] In step S307, the detection chain is generated by associating detections output from the machine learning model. One or more current detections are associated with one or more previous detections of at least one previous video frame obtained from the machine learning model for the series of video frames. This at least one previous video frame precedes at least one current video frame and has been input into the machine learning model. Each of the one or more previous detections includes a confidence value and the position of the detected target in the at least one previous video frame, the confidence value indicating the probability that the previous detection includes the detected target. According to one embodiment of the invention, the current detection is associated with the previous detection based on the positions of the current and previous detections. According to an alternative embodiment or further, the current detection is associated with the previous detection based on at least one of the speed and orientation of the detection in consecutive video frames.

[0063] In step S309, at least one current detection is displayed in the video signal based on the current detection's position in the detection chain, the current detection's confidence value, and the current detection's position.

[0064] In step S311, it is checked whether the termination condition is met. If the termination condition is met, the process ends. If the termination condition is not met, the process returns to step S305 to process at least one next video frame in the series, which is at least one current video frame.

[0065] For example, the termination condition is met if there is no next video frame in the series.

[0066] According to one example implementation, in step S307, the current detection is associated with the previous detection such that if the overlap between the positions of the current and previous detections satisfies a predetermined condition (e.g., IoU ≥ 0.3), the current detection and the previous detection belong to the same detection chain.

[0067] In addition, according to one embodiment, in step S309, if the confidence value of the current detection is equal to or greater than a first threshold, the current detection is displayed.

[0068] For reference Figure 4 , Figure 4 A control unit 40 is shown, wherein an embodiment of the present invention is implemented. For example, the control unit 40 implements... Figure 3 The object detection and display process.

[0069] The control unit 40 includes processing resources (e.g., processing circuitry) 41, memory resources (e.g., memory circuitry) 42, and interfaces (e.g., interface circuitry) 43, which are connected via links (e.g., bus, wired connection, wireless connection, etc.) 44.

[0070] According to one embodiment, memory resource 42 stores a program that, when executed by processing resource 41, causes control unit 40 to operate according to at least some embodiments of the invention.

[0071] Generally speaking, exemplary embodiments of the present invention can be implemented by computer software stored in memory resource 42 and executable by processing resource 41 or by hardware or by a combination of software and / or firmware and hardware.

[0072] The filtering stages that operate on the detection chain obtained as described above will be described below.

[0073] Filtering Stage 1: Flicker Suppression

[0074] Flicker suppression is designed to address the problem of false detections. Because false detections only appear for a few frames and then disappear again, the solution to circumvent this problem is to suppress the first occurrence of a detection in an image. For example, if only a target (e.g., a polyp) has been detected individually at the same location in multiple subsequent video frames, then the detection corresponding to that location is displayed in S309.

[0075] There are two different ways to achieve this type of flicker suppression. One way is unpredictable suppression, which always suppresses the first N occurrences of the detection. The other way is predictable suppression, which only suppresses the first N occurrences of the detection if the detection has disappeared in the (N+1)th frame.

[0076] Both methods increase the accuracy of object detection and display mechanisms. However, because detection is intentionally suppressed, recall is inevitably affected. This decrease in recall is greater when using unpredictable flicker suppression compared to when using predicted flicker suppression. Furthermore, there will be a delay of N+1 frames until the knowledge of whether to display the detection is acquired. Since such delay is generally unacceptable, unpredictable flicker suppression is preferred.

[0077] according to Figure 3 In one example implementation of the object detection and display process, in step S309, if the current detection belongs to the N+M detection chain, the current detection is displayed, where N and M are positive integers equal to or greater than 1, and where N indicates the first detection at time N of the detection chain. Conversely, if the current detection belongs to the first detection at time N of the detection chain, the current detection is not displayed.

[0078] exist Figure 5 In the meantime, precision and recall are evaluated for all possible thresholds to plot precision-recall (PR) curves for: (1) the original dataset (i.e., flicker suppression not applied). Figure 3 (2) has the function of object detection and display process), and has the function of object detection and display process. Figure 3 The dataset for unpredictable (wof) flicker suppression in object detection and display processes, and (3) with applications for Figure 3 A dataset of predictive (wf) flicker suppression for object detection and display processes.

[0079] As described earlier, the area under the curve is called the average accuracy (AP) and is used as... Figure 3 The indicator of how well the object detection and display process works in the following situations: (1) without flicker suppression, (2) without known flicker suppression, and (3) with known flicker suppression.

[0080] Figure 5 The effects of applying predicted and unpredictable flicker suppression are illustrated. A reduction in maximum recall is achieved while accuracy in the high-precision portion of the detector's (e.g., machine learning model) characteristics is improved. With predicted flicker suppression, neither effect is as pronounced. The increase in accuracy is highly visible to the user, while the decrease in recall is not, because for most application scenarios, the detector (e.g., machine learning model) will have its operating point in the high-precision region of the PR curve.

[0081] While applying unpredictable flicker suppression implies a more significant decrease in recall, lost recall is difficult for users to detect. Once a polyp enters the field of view, several lost detections are far less noticeable than false positives that pop up on the image and immediately disappear.

[0082] Filtering Stage 2: Lag

[0083] Sometimes, the opposite occurs with flicker detection. Detection is briefly missed for a single frame and then quickly detected again in the next frame. This can happen, for example, when motion blur occurs.

[0084] To compensate for these missed detections, a hysteresis thresholding process is introduced, such as... Figure 6 As shown.

[0085] Hysteresis thresholding utilizes two thresholds: the first threshold is called the high threshold ( Figure 6 The middle threshold is called the "high" threshold, and the second threshold is called the "low" threshold. Figure 6 (This is referred to as "low" in Chinese). Initially, the confidence value of the detection must exceed a high threshold for display. In other words, initially, over multiple frames (e.g., over time, such as...),... Figure 6As shown in the diagram, if a detection is performed with high confidence at a similar location, the detection is displayed. If a detection has already been displayed at a similar location for several frames, then the detection is allowed to drop below a high threshold and still be displayed. It will only stop being displayed if the detection drops below a low threshold. Figure 6 In this context, the confidence value is indicated as "points".

[0086] According to one example implementation, in Figure 3 In step S309, the current detection is displayed if the confidence value of the current detection is equal to or greater than a second threshold and if the confidence value of a previous detection belonging to the same detection chain as the current detection is equal to or greater than a first threshold, wherein the second threshold is less than the first threshold.

[0087] Figure 7 The hysteresis thresholding process is shown in Figure 3 This demonstrates the typical effect of applications in object detection and display processes. For a given accuracy, recall can be improved. In practice, the potential reduction in accuracy is unobservable.

[0088] It should be noted that Figure 5 and Figure 7 The PR curves shown have been obtained based on different datasets.

[0089] Because lag thresholding results in more detected polyps, it can increase recall. Potentially, it can also lead to a decrease in precision, as one of these detections may prove incorrect. However, this problem is not encountered because neural networks are generally good at assigning high confidence values ​​when polyps are present and very low confidence values ​​when polyps are absent. In such cases, the network's confidence values ​​often do not even exceed the low threshold.

[0090] Filtering Stage 3: Position Smoothing Processing

[0091] In filtering stage 3, smoothing is performed on the detected location.

[0092] According to one example implementation, in Figure 3 In step S309, when at least one current detection is displayed, its position is smoothed based on the detection position of the detection chain to which the current detection belongs, wherein the current detection is prior to the current detection.

[0093] For example, smoothing is performed by applying a weighted average of the detected coordinates. This creates a phenomenon where the location is more stable compared to the initial condition. Alternatively, smoothing can be performed using more complex filtering structures, such as applying a Kalman filter to the detected position in the video signal.

[0094] Effect

[0095] The combined effect of applying the heuristic filtering stages 1 through 3 described above has been evaluated on a large test dataset of 6000 images. On average, a 62% reduction in false positive detections was observed compared to the case without heuristic filtering. Similarly, a 16% increase in false negatives was observed. It should also be noted that the reduction in false positives is highly visible, while the reduction in false negatives is difficult to see. Several frames in which polyps were not detected when entering the field of view are technically measured as false negatives. However, this is difficult for human users to see. False positives that pop up in the video, however, are very easily perceived by the user.

[0096] At this point, it's important to note that the 16% increase in false negatives does not mean that 16% of polyps were missed during the colonoscopy. It means there are 16% more video frames where polyps are present but not detected. However, multiple video frames often depict the same polyp. If the network is good at detecting polyps, it's almost certain that at least one video frame will be encountered where a specific polyp is detected. In practice, heuristic filtering does not affect the number of polyps detected at least once.

[0097] The object detection and display mechanism described above enables reliable real-time detection of polyps during colonoscopy.

[0098] The three-stage heuristic filtering approach allows for frame-by-frame filtering of the video signal, i.e., over time. Therefore, while object detection and display mechanisms operate on video frames (e.g., individual video frames), individual detection appears more stable. This heuristic filtering visually improves the results and allows for temporal consistency detection during training without requiring video data (and corresponding annotations).

[0099] It should be understood that the foregoing description illustrates the invention and should not be construed as limiting it. Various modifications and applications can be made by those skilled in the art without departing from the true spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for detecting objects in a video signal, comprising, Obtain one or more current detections of at least one current video frame from a series of consecutive video frames of a video signal output from a machine learning model, the at least one current video frame being input to the machine learning model, wherein the current detection of the one or more current detections includes a confidence value and the position of the detected target in the at least one current video frame, the confidence value indicating the probability that the current detection includes the detected target to be detected by the machine learning model; A detection chain is generated by associating detections output from the machine learning model, wherein the current detection of one or more current detections is associated with a previous detection of one or more previous video frames of at least one previous video frame obtained from the machine learning model for the series of video frames, the at least one previous video frame being preceding the at least one current video frame and having been input into the machine learning model, wherein the previous detection of one or more previous detections includes a confidence value and the position of the detection target in the at least one previous video frame, the confidence value indicating the probability that the previous detection includes the detection target, wherein the current detection is associated with the previous detection based on the positions of the current and previous detections; and The display of at least one of the current detections in the video signal is caused based on the position of the current detection in the detection chain, the confidence value of the current detection, and the position of the current detection. The acquisition, generation, and induction of display are repeated for at least one next video frame in the series, which is the at least one current video frame, and The triggering of the display is performed when the confidence value of the current detection is at the following condition: Equal to or greater than the first threshold, and The confidence value of the previous detection, which is equal to or greater than the second threshold and belongs to the same detection chain as the current detection, is equal to or greater than the first threshold, and the second threshold is less than the first threshold, so that hysteresis thresholding is performed and flickering is avoided.

2. The method of claim 1, wherein the current detection is associated with the previous detection such that the current detection and the previous detection belong to the same detection chain if the overlap of the positions of the current and previous detections satisfies a predetermined condition.

3. The method according to claim 1 or 2, wherein if the current detection belongs to N+M detections in the detection chain, the current detection is displayed, wherein N and M are positive integers equal to or greater than 1, and wherein N indicates the first detection at time N in the detection chain, and wherein if the current detection belongs to the first detection at time N in the detection chain, the current detection is not displayed.

4. The method according to claim 1, further comprising: Smoothing is performed on the detection positions of the detection chain.

5. The method of claim 1, wherein the video signal is captured by an endoscope during the examination process.

6. The method according to claim 1, wherein the detection target is a polyp.

7. A computer-readable non-transitory storage medium storing a program that, when executed by a computer, causes the computer to perform the method according to any one of claims 1 to 6.

8. An apparatus for detecting an object in a video signal, the apparatus comprising at least one processor and at least one memory, the at least one memory including computer program code, the at least one memory and the computer program code configured together with the at least one processor to cause the apparatus to perform at least the following: Obtain one or more current detections of at least one current video frame from a series of consecutive video frames of a video signal output from a machine learning model, the at least one current video frame being input to the machine learning model, wherein the current detection of the one or more current detections includes a confidence value and the position of the detected target in the at least one current video frame, the confidence value indicating the probability that the current detection includes the detected target to be detected by the machine learning model; A detection chain is generated by associating detections output from the machine learning model, wherein the current detection of one or more current detections is associated with a previous detection of one or more previous video frames of at least one previous video frame obtained from the machine learning model for the series of video frames, the at least one previous video frame being preceding the at least one current video frame and having been input into the machine learning model, wherein the previous detection of one or more previous detections includes a confidence value and the position of the detection target in the at least one previous video frame, the confidence value indicating the probability that the previous detection includes the detection target, wherein the current detection is associated with the previous detection based on the positions of the current and previous detections; The display of at least one of the current detections in the video signal is caused based on the position of the current detection in the detection chain, the confidence value of the current detection, and the position of the current detection. and For at least one next video frame in the series that is the at least one current video frame, the process of obtaining, generating, and causing display is repeated, and The triggering of the display is performed when the confidence value of the current detection is at the following condition: Equal to or greater than the first threshold, and The confidence value of the previous detection, which is equal to or greater than the second threshold and belongs to the same detection chain as the current detection, is equal to or greater than the first threshold, and the second threshold is less than the first threshold, so that hysteresis thresholding is performed and flickering is avoided.

Images