Correction Device and Correction Method for Instant Images

Abstract

The present invention provides an instant image correction device and a correction method. The correction method includes: continuously receiving instant images from an image input device; analyzing the image features of the instant images; calculating the motion vector of each instant image according to the feature difference of the same image features of any two adjacent instant images in time; calculating the predicted value of the motion trajectory of the current instant image based on the accumulated motion vectors; when it is determined that the difference between the predicted value of the motion trajectory of the current instant image and the motion vector falls within the correction allowable range, calculating a compensation value based on the predicted value of the motion trajectory and the motion vector of the current instant image, and correcting the current instant image based on the compensation value; when it is determined that the difference between the predicted value of the motion trajectory of the current instant image and the motion vector exceeds the correction allowable range, resetting the image input device and obtaining a preset image as the output image.

Description

Technical Field

[0001] The present invention relates to a correction device and a correction method, and more particularly to a correction device and a correction method for instant images. Background Art

[0002] Although the acquisition and display technologies of instant images can be applied in many different technical fields, there are problems to be overcome in each technical field.

[0003] For example, in a remote monitoring system, the device is usually built in a specific area, and maintenance personnel are not allowed to enter or leave at will, or are too far away from the installation location of the device. When the device has a problem, the maintenance personnel cannot repair the device immediately to restore the image output. For another example, when using an image acquisition device (such as an acquisition card) on an electronic device to acquire and record images, if the input image is unstable, it is easily misjudged by the image processing software as the image has been interrupted, and thus the acquisition and recording are stopped. In this way, the preservation of important images will be missed.

[0004] For another example, in the medical field, a photographic lens (such as an endoscope) often extends into the human body together with an electrocautery knife or other equipment that emits high-frequency signals for examination, and these high-frequency signals easily affect the image obtained by the photographic lens, thereby causing misjudgment by doctors. Moreover, the photographic lens may also be affected by factors such as temperature and unstable power supply, resulting in unstable images.

[0005] Currently, there are already some image processing systems on the market that can, when detecting an abnormal input image, allow the user to manually or the system to automatically send an instruction from a remote location to control the entire system to restart, thereby solving the problem of abnormal input images. However, in some application fields (especially the medical field), the user may not be able to restart the system manually during the image viewing process (such as during a surgery). Moreover, restarting the entire system until it resumes a usable state usually takes several minutes, and in some application fields, the user may not have time to wait for the system to restart.

[0006] To solve the above problems, the present invention proposes a correction device and a correction method for instant images, which can automatically judge the condition of instant images and automatically perform corrections to enable the system to maintain the quality and stability of the output images, thereby avoiding causing troubles to users. Summary of the Invention

[0007] The main object of the present invention is to provide a correction device and a correction method for instant images, which can automatically detect the state of the input image and perform automatic corrections immediately to achieve long-term stable image output.

[0008] To achieve the above object, the correction device of the present invention includes:

[0009] An image detection module is connected to an image input device to continuously receive a plurality of instant images;

[0010] An image comparison module is connected to the image detection module, executes an analysis algorithm to analyze an image feature of each of the instant images, calculates a feature difference of the image features of any two adjacent instant images in time, and calculates a movement vector of the image feature of each of the instant images based on the feature difference;

[0011] A control module is connected to the image input device, the image detection module, and the image comparison module, records the movement vector of each of the instant images, and calculates a movement trajectory prediction value of a current instant image among the plurality of instant images based on the accumulated movement vectors; and

[0012] An image correction module is connected to the image comparison module and the control module for providing an output image;

[0013] Wherein, when a first difference between the movement trajectory prediction value of the current instant image and the movement vector of the current instant image falls within a correction allowable range, the control module calculates a compensation value based on the movement trajectory prediction value and the movement vector of the current instant image, and controls the image correction module to correct the current instant image based on the compensation value to generate the output image and simultaneously provide a first warning message;

[0014] Wherein, when the first difference exceeds the correction allowable range, the control module resets the image input device, and controls the image correction module to use a preset image as the output image and simultaneously provide a second warning message.

[0015] As described above, wherein the analysis algorithm is a movement vector algorithm.

[0016] As described above, wherein the feature difference is a brightness offset, a chromaticity offset, a discrete wavelet transform (DWT), or a continuous wavelet transform (CWT) of the same area in any two adjacent instant images in time.

[0017] As described above, the image input device is an image sensor, a High Definition Multimedia Interface (HDMI), a Serial Digital Interface (SDI), an Electronic Data Processing (EDP) interface, or a Mobile Industry Processor Interface (MIPI).

[0018] As described above, when the first difference is greater than or equal to a first preset value and less than a second preset value, the control module determines that the first difference falls within the correction allowable range, and when the first difference is greater than the second preset value, determines that the first difference exceeds the correction allowable range, where the first preset value is less than the second preset value.

[0019] As described above, the first preset value is plus or minus 5%, and the second preset value is plus or minus 30%.

[0020] As described above, the preset image is the previous image adjacent to the current instantaneous image in time.

[0021] As described above, when the first difference falls within the correction allowable range, the control module calculates an average value of the predicted value of the movement trajectory of the current instantaneous image and an interpolation value of the movement vector, and uses the average value as the compensation value.

[0022] As described above, when the first difference is less than the first preset value, the control module sets the compensation value to 0, and controls the image correction module to correct the current instantaneous image based on the compensation value to generate the output image.

[0023] As described above, the image detection module detects basic image information of each instantaneous image, the control module calculates a second difference between the accumulated basic image information of several instantaneous images and the basic image information of the current instantaneous image, sets the compensation value to 0 when the first difference and the second difference are less than the first preset value, calculates the compensation value based on the predicted value of the movement trajectory of the current instantaneous image and the movement vector when the first difference and the second difference fall within the correction allowable range, and resets the image input device when the first difference and the second difference exceed the correction allowable range.

[0024] As described above, the basic image information is at least one of the exposure intensity, frequency, frames per second, number of effective image vertical lines, and number of effective image horizontal pixels of each instantaneous image.

[0025] To achieve the above object, the calibration method of the present invention includes the following steps:

[0026] a) Continuously receive a plurality of live images from an image input device;

[0027] b) Execute an analysis algorithm to analyze an image feature of each of the live images;

[0028] c) Calculate a feature difference of the image features of any two temporally adjacent live images, and calculate a motion vector of the image feature of each of the live images based on the feature difference;

[0029] d) Calculate a predicted motion trajectory value of a current live image among the plurality of live images based on the accumulated motion vectors;

[0030] e) When it is determined that a first difference between the predicted motion trajectory value of the current live image and the motion vector falls within a calibration allowable range, calculate a compensation value based on the predicted motion trajectory value and the motion vector of the current live image;

[0031] e1) After step e, correct the current live image based on the compensation value to generate an output image, and simultaneously provide a first warning message;

[0032] f) When it is determined that the first difference exceeds the calibration allowable range, control the image input device to be reset; and

[0033] f1) Obtain a preset image as the output image, and simultaneously provide a second warning message.

[0034] As described above, step c) includes: calculating a luminance offset, a chrominance offset, a discrete wavelet transform (DWT), or a continuous wavelet transform (CWT) of the same region in any two temporally adjacent live images to generate the feature difference.

[0035] As described above, step e) includes: when the first difference is greater than or equal to a first preset value and less than a second preset value, determining that the first difference falls within the calibration allowable range, where the first preset value is less than the second preset value; step f) includes: when the first difference is greater than the second preset value, determining that the first difference exceeds the calibration allowable range.

[0036] As described above, the first preset value is plus or minus 5%, and the second preset value is plus or minus 30%.

[0037] As described above, the step f1) includes: obtaining the previous image temporally adjacent to the current live image as the preset image.

[0038] As described above, the step e) includes: calculating an average value of the predicted value of the movement trajectory of the current live image and an interpolation value of the movement vector, and using the average value as the compensation value.

[0039] As described above, it further includes:

[0040] g) When it is determined that the first difference is less than the first preset value, setting the compensation value to 0; and

[0041] g1) After the step g), correcting the current live image based on the compensation value to generate the output image.

[0042] As described above, the step a) includes: detecting basic image information of each live image, and the correction method further includes a step a1): calculating a second difference between the accumulated basic image information of a plurality of the live images and the basic image information of the current live image; wherein in the step g), when the first difference and the second difference are less than the first preset value, the compensation value is set to 0, in the step e), when the first difference and the second difference fall within the correction allowable range, the compensation value is calculated, and in the step f), when the first difference and the second difference exceed the correction allowable range, the image input device is reset.

[0043] As described above, the basic image information is at least one of the exposure intensity, frequency, frames per second, effective number of vertical image lines, and effective number of horizontal image pixels of each live image.

[0044] By means of the technical solution of the present invention, the correction device can automatically analyze after receiving the input image. When the input image is abnormal and can be corrected, the correction device can immediately correct the input image. When the input image is too abnormal to be corrected, the correction device can reset the front-end image input device and continuously output images during the reset process. Thereby, images can be continuously and stably provided to the user to reduce misjudgment behaviors caused by abnormal or missing images for the user, and at the same time improve the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The first specific embodiment of the block diagram of the correction device of the present invention;

[0046] Figure 2 Schematic diagram of the input image;

[0047] Figure 3 The first specific embodiment of the flowchart of the correction method of the present invention;

[0048] Figure 4 The first specific embodiment of the schematic diagram of the motion vector;

[0049] Figure 5A The second specific embodiment of the schematic diagram of the motion vector;

[0050] Figure 5B The third specific embodiment of the schematic diagram of the motion vector;

[0051] Figure 6 The first specific embodiment of the flowchart for generating the output image of the present invention;

[0052] Figure 7 The second specific embodiment of the flowchart of the correction method of the present invention.

[0053] Wherein, reference numerals:

[0054] 1... Correction device;

[0055] 11... Control module;

[0056] 12... Image detection module;

[0057] 13... Image comparison module;

[0058] 14... Image correction module;

[0059] 15... Storage device;

[0060] 151... Previous image;

[0061] 152... Motion vector;

[0062] 153... Basic image information;

[0063] 2... Image input device;

[0064] 3... Display;

[0065] 4... Input image;

[0066] 41... Valid image;

[0067] 42... Vertical blank line;

[0068] 43... Horizontal blank pixel;

[0069] 51... Previous image;

[0070] 511... Image feature;

[0071] 52, 53... Current image;

[0072] 521, 531... Image feature;

[0073] S10~S34, S40~S62... calibration steps;

[0074] S220~S230... output image generation steps. Detailed implementation manners

[0075] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited do not limit the present invention.

[0076] First, please refer to Figure 1 , which is the first specific embodiment of the block diagram of the calibration device of the present invention. As Figure 1 shown, the present invention discloses a calibration device 1 for real-time images (hereinafter simply referred to as calibration device 1 in the specification). The calibration device 1 is mainly applied to various image systems (not shown in the figure). Specifically, one end of the calibration device 1 of the present invention is connected to the image input device 2 of the image system to receive the input of real-time images, and the other end is connected to the display 3 of the image system to continuously and stably provide output images.

[0077] The main technical effect of the present invention is that the calibration device 1 continuously judges the state of the received real-time images, directly outputs through the display 3 when the real-time images do not need to be calibrated; when the real-time images need to be calibrated, the real-time images are first calibrated and then output; and when the real-time images are abnormal and cannot be calibrated, the image input device 2 is reset, and preset images are used for output.

[0078] In other words, when the interference received by the image input device 2 is not strong, the calibration device 1 can directly output the real-time images or output the calibrated images after calibrating the real-time images; and when the image input device 2 is severely interfered and causes the real-time images to be severely abnormal or even unable to output images, the calibration device 1 can timely and automatically reset the front-end image input device 2 and maintain the rear-end display 3 to continuously and stably display images, thereby avoiding the trouble of users.

[0079] As Figure 1 shown, the calibration device 1 mainly includes a control module 11, an image detection module 12, an image comparison module 13, an image calibration module 14 and a storage device 15. Among them, the image comparison module 13 is connected to the image detection module 12, the image calibration module 14 is connected to the image comparison module 13, and the control module 11 is connected to the image detection module 12, the image comparison module 13 and the image calibration module 14.

[0080] In one embodiment, the control module 11 is a hardware module implemented by a micro control unit (MCU) or a graphics processing unit (GPU) in the calibration device 1, and the image processing module 12, the image comparison module 13, and the image calibration module 14 are software modules implemented by a field programmable gate array (FPGA) combined with code in the calibration device 1. The storage device 15 is a memory or a hard disk, but is not limited thereto.

[0081] The calibration device 1 is connected to the image input device 2 through the image detection module 12 to continuously receive real-time images from the image input device 2. In one embodiment, the image input device 2 may be an image sensor (such as a camera, an infrared sensor, a laser sensor, etc.) for instantaneously sensing external images and importing them into the calibration device 1 for analysis and processing. For example, the image input device 2 may be a medical endoscope.

[0082] In another embodiment, the image input device 2 may be a high definition multimedia interface (HDMI), a serial digital interface (SDI), an electronic data processing (EDP) interface, a mobile industry processor interface (MIPI), etc., for directly inputting digital images into the calibration device 1 for analysis and processing. For example, the image input device 2 may be an image capture card.

[0083] It is worth mentioning that the control module 11 in the calibration device 1 can also be connected to the image input device 2 through a communication interface, such as a serial peripheral interface (SPI), an I 2 C interface, etc. In this embodiment, the calibration device 1 can set parameters for the image input device 2 through the control module 11, and when the input image of the image input device 2 is abnormal due to external interference, the control module 11 can directly reset the image input device through the above communication interface.

[0084] In one embodiment, after receiving the live image, the image detection module 12 converts the image format of the live image into an image format supported by the calibration device 1. At the same time, the image detection module 12 can also detect the basic image information 153 of the live image, such as exposure intensity, frequency, frames per second (FPS), the number of effective vertical lines of the image, and the number of effective horizontal pixels of the image, etc. The control module 11 receives the basic image information 153 of the live image from the image detection module 12 and stores it in the storage device 15 of the calibration device 1. Through the basic image information 153, the calibration device 1 can confirm the state of each received live image, and then determine whether it is necessary to calibrate the live image.

[0085] Please also refer to Figure 1 and Figure 2 where Figure 2 is a schematic diagram of the input image. As shown in the figure, based on the resolution adopted by the image input device 2, each input image 4 will form a specific-sized effective image 41 (composed of the number of effective lines and the number of effective pixels in each line), invalid vertical blank lines 42, and invalid horizontal blank pixels 43 in each line based on a fixed horizontal scanning frequency (Hsync) and vertical scanning frequency (Vsync). In the above embodiment, the calibration device 1 can compare the information such as the effective image 41, vertical blank lines 42, and horizontal blank pixels 43 of two or more adjacent live images in time. When the difference in the basic image information 153 of these adjacent live images is too large, the calibration device 1 can determine that the image input device 2 has an abnormality.

[0086] As described above, the calibration device 1 of the present invention can determine whether it is necessary to calibrate the live image through the basic image information 153. However, in another embodiment, the calibration device 1 can also determine whether it is necessary to calibrate the live image through the feature difference of the image features in at least two adjacent live images in time (details will be described later).

[0087] Please also refer to Figure 1 and Figure 3 where Figure 3 is the first specific embodiment of the flowchart of the calibration method of the present invention. Figure 3 Combined with Figure 1 the calibration device 1 shown, the flowchart illustrates how the calibration device 1 calibrates the live image input by the image input device 2 through the internal control module 11, image detection module 12, image comparison module 13, and image calibration module 14.

[0088] AsFigure 3 As shown, the calibration device 1 is connected to the image input device 2 through the image detection module 12, and continuously receives real-time images from the image input device 2 (step S10). Then, the calibration device 1 executes an analysis algorithm through the image comparison module 13 to collect and analyze the image features in the real-time images (step S12).

[0089] In one embodiment, the analysis algorithm is based on a pixel matrix of a preset size (such as a 4x4 matrix, an 8x8 matrix, etc.) to collect the image features of one or more specific partitions in the real-time image, and determines the state of the entire real-time image by analyzing these image features. In this embodiment, the calibration device 1 does not need to store and analyze the entire real-time image, thereby reducing the hardware requirements and improving the processing speed at the same time.

[0090] In one embodiment, the analysis algorithm is a motion vector algorithm. In this embodiment, the calibration device 1 calculates the feature difference of the same image features in any two adjacent real-time images in time through the image comparison module 13, and calculates the motion vector of the image features of each real-time image relative to the previous image based on this feature difference (step S14).

[0091] More specifically, since the hardware resources of the calibration device 1 are limited, the calibration device 1 can only store the previous image 151 (i.e., the real-time image received at time point t-1) through the storage device 15, and calculate the feature difference between the image features in the current real-time image (i.e., the real-time image received at time point t) and the same image features in the previous image 151, and then calculate the motion vector 152 of the image features in the current real-time image compared to the same image features in the previous image 151. And, the control module 11 obtains the motion vector 152 from the image comparison module 13 and records it in the storage device 15.

[0092] Please also refer to Figure 4 , which is the first specific embodiment of the schematic diagram of the motion vector. As Figure 4 shown, through the execution of the analysis algorithm, the image comparison module 13 of the present invention collects specific image features 511 in the previous image 51, and collects the same image features 521 in the current image 52. The analysis algorithm calculates the offset between the same image features 511 and 521 in the two images 51 and 52 through interpolation (Interpolation) to calculate the motion vector of the image features 521 of the current image 52 relative to the image features 511 of the previous image 51.

[0093] Taking the current image 52 as an example, since the image feature 521 is a neighboring partition that has moved to the position of the image feature 511, after the analysis algorithm calculates the positions of the two image features 511 and 521, a movement vector with an integer relationship can be obtained.

[0094] For another example Figure 4 As shown, the analysis algorithm can collect a specific image feature 511 from the previous image 51 and the same image feature 531 from the current image 53. Taking the current image 53 as an example, since the image feature 531 is not a neighboring partition that has directly moved to the position of the image feature 511 (but is located in a sub - partition), the analysis algorithm needs to perform interpolation calculations on the positions of the two image features 511 and 531 to obtain a movement vector with a decimal relationship.

[0095] Please also refer to Figure 5A for the second specific embodiment of the schematic diagram of the movement vector. In the embodiment of Figure 5A , the analysis algorithm can use a Finite Impulse Response Filter (FIR Filter) with six weights (including ). The finite impulse response filter can perform interpolation calculations based on integer position relationships, thereby obtaining the position of the target pixel point and then calculating the movement component. In the embodiment of Figure 5A , the analysis algorithm can mainly calculate the positions of the target pixel point b, pixel point s, pixel point h, and pixel point m in an image according to the following formulas:

[0096] ;

[0097] ;

[0098] ;

[0099] .

[0100] Please also refer to Figure 5B for the third specific embodiment of the schematic diagram of the movement vector. In the embodiment of Figure 5B , the analysis algorithm can mainly perform interpolation calculations based on decimal position relationships to obtain the position of the target pixel point and then calculate the movement component. In the embodiment of Figure 5B , the analysis algorithm can mainly calculate the position of the target pixel point j according to the following formula:

[0101]

[0102] The above - mentioned interpolation calculations are common technical means in this technical field and will not be elaborated here.

[0103] In the above embodiments, the image comparison module 13 of the calibration device 1 mainly calculates the motion vector through the motion vector algorithm, thereby judging the state of the live image. For example, the image comparison module 13 can calculate the luminance offset, chrominance offset, discrete wavelet transform (DWT), or continuous wavelet transform (CWT) of the same region in any two adjacent live images as the feature difference, thereby calculating the motion vector. However, the above are only some specific embodiments of the present invention and are not limited thereto.

[0104] It is worth mentioning that in other embodiments, the image comparison module 13 can also adopt an AI-based algorithm (such as an edge computing algorithm) to calculate the difference between the same image features in two consecutive live images, thereby judging the change state of the live image.

[0105] Back to Figure 3 . The calibration device 1 records the motion vectors 152 of each live image calculated by the image comparison module 13 in the storage device 15 through the control module 11. And the control module 11 calculates the predicted value of the motion trajectory of the current live image based on multiple accumulated motion vectors 152 that are continuous in time (step S16).

[0106] Specifically, the calibration device 1 continuously receives live images from the image input device 1, so the multiple received live images are continuous in time. Therefore, based on the motion vectors 152 of one or more specific image features in the previously received multiple images (taking three images as an example, namely the images received at time points t-3, t-2, and t-1), the control module 11 can predict the motion vector 152 of the same image feature in the next live image before receiving the next live image as the predicted value of the motion trajectory. In other words, if the calibration device 1 can receive the current live image at time point t, the control module 11 can predict the motion trajectory of the current live image based on the motion vectors 152 of the previously received multiple images before time point t.

[0107] If the image input device 1 operates normally, the motion vector 152 of the current live image calculated by the image comparison module 13 should be equal to the predicted value of the motion trajectory predicted by the control module 11 in advance, or the difference between the two should be less than the threshold.

[0108] In the present invention, the control module 11 receives the motion vector 152 of the current real-time image from the image comparison module 13, and compares the motion vector 152 of the current real-time image (which is an actual value) with the predicted motion trajectory value (which is a predicted value) (step S18). Specifically, in step S18, the control module 11 calculates the difference between the motion vector 152 of the current real-time image and the predicted motion trajectory value (hereinafter referred to as the first difference), and determines whether the first difference is less than a preset correction allowable range, within the correction allowable range, or exceeds the correction allowable range.

[0109] Continuing from the above, if the first difference is less than the correction allowable range, it means that the current real-time image (and the image input device 2) has no abnormality or the abnormality is not serious. At this time, the correction device 1 does not need to correct the current real-time image. Therefore, the control module 11 can send a first instruction to the image correction module 14 to control the image correction module 14 to directly output the current real-time image to the display 3. If the first difference falls within the correction allowable range, it means that the current real-time image (or the image input device 2) is abnormal, but this abnormal situation is still within the range that can be corrected. At this time, the control module 11 can send a second instruction to the image correction module 14 to control the image correction module 14 to first correct the current real-time image and then output the corrected image to the display 3.

[0110] If the first difference exceeds the correction allowable range, it means that the current real-time image (or the image input device 2) is abnormal and has become so serious that it cannot be corrected. At this time, the control module can send a third instruction to the image correction module 14 to control the image correction module 14 to output a preset image instead of the current real-time image. In this way, even if the current real-time image or the image input device 2 has an abnormality, the correction device 1 can still enable the display 3 to maintain a stable image output.

[0111] It is worth mentioning that in the present invention, the control module 11, the image detection module 12, the image comparison module 13, and the image correction module operate in a pipeline manner, so that the correction device 1 can quickly correct a large number of consecutive images to maintain a stable image output.

[0112] For example, when the image detection module 12 receives the first image (corresponding to time point t - 3), the image comparison module 13 calculates the motion vector of the second image (corresponding to time point t - 2), the control module 11 calculates the compensation value required for the third image (corresponding to time point t - 1), and the image correction module 14 corrects the fourth image (corresponding to time point t) and outputs the corrected image externally.

[0113] The above technical means of the pipeline are common technical means in the technical field and will not be elaborated herein.

[0114] If it is determined in step S18 that the first difference is less than the calibration allowable range (i.e., less than the lower limit value of the calibration allowable range), the control module 11 may output a compensation value of 0 for the current live image (step S20), and control the image calibration module 14 to generate an output image based on this compensation value (step S22). Specifically, a compensation value of 0 means that the current live image does not need to be calibrated. In other words, the output image is equal to the current live image.

[0115] If it is determined in step S18 that the first difference falls within the calibration allowable range (i.e., between the lower limit value and the upper limit value of the calibration allowable range), the control module 11 calculates a compensation value based on the predicted movement trajectory value of the current live image and the movement vector 152 (step S24). And, the control module outputs the calculated compensation value to the image calibration module 14 (step S26), and controls the image calibration module 14 to generate an output image based on this compensation value (step S22). Specifically, a non-zero compensation value means that the current live image needs to be calibrated. In other words, the output image is not equal to the current live image.

[0116] In an embodiment, the control module 11 calculates the average value of the interpolation of the predicted movement trajectory value of the current live image and the movement vector 152 in step S24, and uses this average value as the compensation value. For example, if the predicted movement trajectory value of one of the image features in the current live image is -0.9 and the movement vector 152 is -0.5, the control module 11 can calculate and determine that the better position of this image feature should be at the position of -0.7 (i.e., ), so the compensation value of this image feature on the current live image can be calculated as -0.2. The above is only one specific embodiment of the present invention, but it is not limited thereto.

[0117] It is worth mentioning that if the image calibration module 14 corrects the current live image based on the compensation value and then outputs it, it means that the image displayed on the display 3 is different from the image input by the image input device 2. To prevent the user from being affected by the corrected image and causing misjudgment, the control module 11 may generate a corresponding warning message (for example, "The image is interfered, and the currently displayed image is the corrected image") while generating the compensation value. In this embodiment, the image calibration module 14 can output the generated output image and the warning message to the display 3 at the same time to remind the user to pay attention.

[0118] If it is determined in step S18 that the first difference exceeds the calibration allowable range, the control module 11 may send an instruction to the image input device 2 to reset the image input device 1 (step S28), thereby solving the problem that the input image has been severely abnormal and cannot be calibrated.

[0119] Generally speaking, it only takes a few seconds to reset the image input device 1 at the front end. Compared with restarting the entire image system (such as a remote monitoring system, a medical endoscope system, etc.), the user will not be greatly affected. Taking the medical field as an example, only resetting the endoscope at the front end of the endoscope system without restarting the entire endoscope system can effectively gain the golden medical time limit.

[0120] Moreover, in order to prevent the user from being unable to obtain relevant information during the reset of the image input device 1, or causing the relevant software to misjudge that the image input program has ended and automatically close, during the period when the calibration device 1 controls the reset of the image input device 1, the control module 11 will continuously provide a preset image to the image calibration module 14 (step S30). Thus, the control image calibration module 14 can directly use the preset image as the output image (step S22). Directly replacing the current real-time image with the preset image means that the current real-time image has been severely abnormal to the extent that it cannot be corrected. Therefore, during the reset of the image input device 2, the calibration device 1 first continuously outputs the preset image so that the user can continuously see a stable picture output. And, when the control module 11 senses that the reset of the image input device 2 is completed, it then controls the image calibration module 14 to resume outputting the real-time image provided by the image input device 2.

[0121] In one embodiment, the control module 11 records the previous image 151 in the storage device 15 after each processing procedure, and continuously updates the previous image 151 as time passes. Since the previous image 151 is the closest to the current real-time image in time, the image difference is the smallest. In one embodiment, the control module 11 obtains the previous image 151 stored in the storage device 15 in step S30 and provides it to the image calibration module 14 to use the previous image 151 as the preset image. And the above is only one implementation aspect of the present invention, but not limited thereto.

[0122] In Figure 1 the embodiment, it is taken as an example that only a single image is stored in the storage device 15. However, when there are sufficient hardware resources, the number of images stored in the storage device 15 is not limited to one.

[0123] It is worth mentioning that if the image calibration module 14 uses the preset image as the output image, it means that the image displayed on the display 3 is different from the image input by the image input device 2. In order to prevent the user from being affected by the output image and making a misjudgment, the control module 11 can generate a corresponding warning message (for example, "The image input device is being reset, and the currently displayed image is not the real-time image") while providing the preset image to the image calibration module 14. In this embodiment, the image calibration module 14 can output both the output image and the warning message to the display 3 to avoid the user's misjudgment.

[0124] Through the comparison program, the judgment program, the correction program, and the replacement program, the correction device 1 can continuously provide an output image through the image correction module 14 (step S32). Moreover, the control module 11 can continuously determine whether the entire image system is shut down (step S34), and continuously execute steps S10 to S32 before the image system is shut down. Thereby, regardless of whether the image input device 2 is abnormal, the user can continuously obtain a stable image on the display 3.

[0125] In one embodiment, the correction device 1 can define the correction allowable range through a first preset value and a second preset value, where the second preset value is greater than the first preset value. In step S18, when the first difference between the motion vector of the current live image and the predicted value of the motion trajectory is less than the first preset value, the control module 11 determines that the first difference is less than the correction allowable range and does not need to correct the live image; when the first difference is greater than or equal to the first preset value and less than the second preset value, the control module 11 determines that the first difference falls within the correction allowable range and needs to correct the live image; when the first difference is greater than the second preset value, the control module 11 determines that the first difference exceeds the correction allowable range and the live image cannot be corrected.

[0126] In one embodiment, the first preset value is ±5%, and the second preset value is ±30%. In other words, if the difference percentage between the motion vector of the current live image and the predicted value of the motion trajectory is less than ±5%, the current live image does not need to be corrected; if the difference percentage between the motion vector of the current live image and the predicted value of the motion trajectory falls between ±5% and ±30%, the current live image needs to be corrected; if the difference percentage between the motion vector of the current live image and the predicted value of the motion trajectory is greater than ±30%, the current live image cannot be corrected.

[0127] The above is only one specific implementation example of the present invention, but it is not limited thereto.

[0128] Please also refer to Figure 1 、 Figure 3 and Figure 6 where Figure 6 is the first specific embodiment of the flowchart for generating the output image of the present invention. Figure 6 for further explaining Figure 3 step S22 in

[0129] to illustrate how the image correction module 14 in the correction device 1 generates corresponding output images in various different situations.

[0129] As shown in Figure 6 and Figure 3In step S22, first, the image correction module 14 determines whether it is necessary to correct the current live image (step S220). In the present invention, if the compensation value generated by the control module 11 is 0, the image correction module 14 can determine that there is no need to correct the current live image. At this time, the image correction module 14 directly outputs the current live image (step S222) to directly use the current live image as the output image. More specifically, in this embodiment, the image correction module 14 adds the compensation value (which is 0) to the current live image to generate the output image. Since the compensation value is 0, the output image is equal to the current live image.

[0130] If the compensation value generated by the control module 11 is not 0, the image correction module 14 can determine that it is necessary to correct the current live image. At this time, the image correction module 14 adds the corresponding compensation value to one or more image features in the current live image respectively (step S224), and adds the corresponding warning information (step S226) to generate the output image.

[0131] If the preset image is received from the control module 11, or an instruction to read the preset image is received, the image correction module 14 can determine that the current live image is seriously abnormal and cannot be corrected. At this time, the image correction module 14 obtains the preset image (step S228), and adds the corresponding warning information (step S230) to generate the output image. In one embodiment, the preset image is the previous image adjacent to the current live image in time (such as Figure 1 the previous image 151 in

[0132] After step S222, step S226 or step S230, the image correction module 14 can further execute Figure 3 step S32 to continuously output the output image generated by the image correction module 145. Thus, regardless of the state of the current live image, the correction device 1 of the present invention can continuously and stably provide an image for the user to view through the display 3.

[0133] In the above Figure 3 embodiment, the correction device 1 mainly determines whether it is necessary to correct the current live image based on the difference between the motion vector of the current live image and the predicted motion trajectory value. However, as described above, the image detection module 12 can detect the basic image information of each received live image while receiving the live image. In other embodiments, the correction device 1 can also determine whether it is necessary to correct the current live image based on the difference between the basic image information of the current live image and the basic image information of one or more previous images.

[0134] Please also refer to Figure 1 and Figure 7, wherein Figure 7 This is the second specific embodiment of the flowchart of the calibration method of the present invention. In this embodiment, the calibration device 1 is connected to the image input device 2 through the image detection module 12 to continuously receive real-time images (step S40), and the image detection module 12 continuously detects and records the basic image information 153 of the received real-time images (step S42).

[0135] In one embodiment, the basic image information 153 may be, for example, the exposure intensity, frequency, frames per second, number of effective image vertical lines, number of effective image horizontal pixels, etc. of the real-time image, but is not limited thereto.

[0136] After step S42, the calibration device 1 analyzes the image features of each real-time image through the image comparison module 13, and calculates the movement vector 152 and the movement trajectory prediction value of the current real-time image through the control module 11 and the image comparison module 13 (step S44). The calculation methods of the movement vector 152 and the movement trajectory prediction value are the same as those Figure 3 shown, and will not be elaborated herein.

[0137] In this embodiment, the control module 11 of the calibration device 1 can calculate the difference between the movement vector 152 and the movement trajectory prediction value of the current real-time image (hereinafter referred to as the first difference), and calculate the difference between the basic image information 153 of the current real-time image and the basic image information 153 of one or more accumulated previous real-time images (hereinafter referred to as the second difference). Thereby, the control module 11 can determine whether to correct the current real-time image based on the values of the first difference and the second difference (step S46).

[0138] For example, if the basic image information 153 of one or more previous real-time images indicates that the number of vertical lines of the effective image in each of the real-time images is 1920 lines, but the basic image information 153 of the current real-time image indicates that the number of vertical lines of the effective image in the current real-time image is 1900 lines. At this time, the control module 11 can determine that although there is the second difference between the basic image information 153 of the current real-time image and the basic image information 153 of the previous real-time image, the value of the second difference is less than the calibration allowable range, which belongs to a situation that is not serious and does not need to be corrected.

[0139] For another example, if the basic image information 153 of one or more previous real-time images indicates that the number of vertical lines of the effective image in each of the real-time images is 1920 lines, but the basic image information 153 of the current real-time image indicates that the number of vertical lines of the effective image in the current real-time image is 1800 lines. At this time, the control module 11 can determine that the current real-time image needs to be corrected based on the value of the second difference.

[0140] For another example, if the basic image information 153 of one or more prior instant images indicates that the number of vertical rows of effective images in each of the instant images is 1920 rows, but the basic image information 153 of the current instant image indicates that the number of vertical rows of effective images in the current instant image is only 150 rows. At this time, the control module 11 can determine that the current instant image is seriously abnormal based on the value of the second difference and cannot be corrected.

[0141] As Figure 7 shown, if both the first difference and the second difference are less than a preset correction allowable range (for example, less than a first threshold value), it indicates that the current instant image does not need to be corrected. Therefore, the control module 11 outputs a compensation value of 0 (step S48), and controls the image correction module 14 to generate an output image based on the compensation value output in step S48 (step S50). In this embodiment, the image correction module 14 adds the compensation value to the current instant image to generate the output image. Since the compensation value is 0, the output image is equal to the current instant image.

[0142] If both the first difference and the second difference fall within the correction allowable range (for example, greater than or equal to the first threshold value and less than the second threshold value), it indicates that the current instant image needs to be corrected. Therefore, the control module 11 calculates a corresponding compensation value based on the motion vector 152 of the current instant image and the motion trajectory prediction value (step S52). And, the control module 11 outputs the compensation value to the image correction module 14 (step S54), thereby controlling the image correction module 14 to generate an output image based on the compensation value output in step S54 (step S50).

[0143] In step S52, the control module 11 mainly calculates the average value of the interpolation of the motion vector 152 of the current instant image and the motion trajectory prediction value, and uses this average value as the compensation value. In step S54, the image correction module 14 adds the corresponding compensation values to one or more image features in the current instant image to generate the output image. Since the compensation value is not 0, the output image is different from the current instant image. It is worth mentioning that in this embodiment, the image correction module 14 also adds the warning information to the output image to remind the user that the image currently displayed on the display 3 is a corrected image, which is different from the image actually input by the image input device 2.

[0144] If both the first difference and the second difference exceed the calibration allowable range (for example, greater than the second threshold value), it indicates that the current live image is severely abnormal and cannot be calibrated. Therefore, the control module 11 directly issues a control command to the image input device 2 to reset the image input device 2 (step S56). Moreover, the control module 11 outputs a preset image (for example, the previous image 151 stored in the storage device 15) to the image calibration module 14 (step S58), whereby the control image calibration module 14 can directly use the preset image obtained in step S58 as the output image (step S50).

[0145] In this embodiment, the image calibration module 14 will also add the warning information to the output image at the same time to remind the user that the image currently displayed on the display 3 is not the live image currently actually input by the image input device 2.

[0146] After step S50, the calibration device 1 can continuously provide the generated output image through the image calibration module 14 (step S60). Moreover, the calibration device 1 continuously determines whether the imaging system is shut down through the control module 11 (step S62), and continuously executes steps S40 to S60 before the imaging system is shut down. Thereby, regardless of whether the image input device 2 is interfered or the current live image is abnormal, the calibration device 1 can continuously and stably output images.

[0147] The above-described embodiments are only preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention is subject to the claims.

Claims

1. An apparatus for correcting instant images, characterized in that, comprising: an image detection module connected to an image input device to continuously receive a plurality of instant images; an image comparison module connected to the image detection module, performing a motion vector algorithm to collect the same image feature in one or more specific partitions of each of the instant images based on a pixel matrix of a preset size, calculating a feature difference of the image feature of any two temporally adjacent instant images, and calculating a motion vector of the image feature of each of the instant images based on the feature difference; a control module connected to the image input device, the image detection module and the image comparison module, recording the motion vector of each of the instant images, and calculating a motion trajectory prediction value of a current instant image among the plurality of instant images based on the accumulated motion vectors; and an image correction module connected to the image comparison module and the control module for providing an output image; wherein, when a first difference between the motion trajectory prediction value of the current instant image and the motion vector of the current instant image falls within a correction allowable range, the control module calculates a compensation value based on the motion trajectory prediction value and the motion vector of the current instant image, and controls the image correction module to correct the current instant image based on the compensation value to generate the output image and simultaneously provide a first warning message; wherein, when the first difference exceeds the correction allowable range, the control module resets the image input device, and controls the image correction module to use a preset image as the output image and simultaneously provide a second warning message, wherein the preset image is the previous image temporally adjacent to the current instant image.

2. The apparatus for correcting instant images according to claim 1, characterized in that, the feature difference is a brightness offset, a chromaticity offset, a discrete wavelet transform amount or a continuous wavelet transform amount of the same region in any two temporally adjacent instant images.

3. The apparatus for correcting instant images according to claim 1, characterized in that, the image input device is an image sensor, a high-definition multimedia interface, a serial digital interface, an electronic data processing interface or a mobile industry processing interface.

4. The apparatus for correcting instant images according to claim 1, characterized in that, when the first difference is greater than or equal to a first preset value and less than a second preset value, the control module determines that the first difference falls within the correction allowable range, and when the first difference is greater than the second preset value, the control module determines that the first difference exceeds the correction allowable range, wherein the first preset value is less than the second preset value.

5. The apparatus for correcting instant images according to claim 4, characterized in that, the first preset value is plus or minus 5%, and the second preset value is plus or minus 30%.

6. The apparatus for correcting instant images according to claim 4, characterized in that, when the first difference falls within the correction allowable range, the control module calculates an average value of an interpolation value of the motion trajectory prediction value and the motion vector of the current instant image, and uses the average value as the compensation value.

7. The correction device for instant images according to claim 4, wherein, when the first difference is less than the first preset value, the control module sets the compensation value to 0, and controls the image correction module to correct the current instant image based on the compensation value to generate the output image.

8. The correction device for instant images according to claim 7, wherein, the image detection module detects basic image information of each instant image, the control module calculates a second difference between the basic image information of a plurality of accumulated instant images and the basic image information of the current instant image, sets the compensation value to 0 when the first difference and the second difference are less than the first preset value, calculates the compensation value based on the predicted movement trajectory value and the movement vector of the current instant image when the first difference and the second difference fall within the correction allowable range, and resets the image input device when the first difference and the second difference exceed the correction allowable range, wherein the basic image information is at least one of the exposure intensity, frequency, frames per second, number of effective image vertical lines, and number of effective image horizontal pixels of each instant image.

9. A method for correcting instant images, wherein, comprising: step a) continuously receiving a plurality of instant images from an image input device; step b) performing a movement vector algorithm to collect the same image feature in one or more specific partitions of each instant image based on a pixel matrix of a preset size; step c) calculating a feature difference between the image features of any two temporally adjacent instant images, and calculating a movement vector of the image feature of each instant image based on the feature difference; step d) calculating a predicted movement trajectory value of a current instant image among the plurality of instant images based on the accumulated movement vectors; step e) when it is determined that a first difference between the predicted movement trajectory value and the movement vector of the current instant image falls within a correction allowable range, calculating a compensation value based on the predicted movement trajectory value and the movement vector of the current instant image; step e1) after step e, correcting the current instant image based on the compensation value to generate an output image, and simultaneously providing a first warning message; step f) when it is determined that the first difference exceeds the correction allowable range, controlling the image input device to be reset; and step f1) obtaining a preset image as the output image, and simultaneously providing a second warning message, wherein the preset image is the previous image temporally adjacent to the current instant image.

10. The method for correcting instant images according to claim 9, wherein, step c) includes: calculating a brightness offset, a chromaticity offset, a discrete wavelet transform amount, or a continuous wavelet transform amount in the same area of any two temporally adjacent instant images to generate the feature difference.

11. The method for correcting instant images according to claim 9, wherein, Step e) includes: when the first difference is greater than or equal to a first preset value and less than a second preset value, determining that the first difference falls within the correction allowable range, where the first preset value is less than the second preset value; Step f) includes: when the first difference is greater than the second preset value, determining that the first difference exceeds the correction allowable range.

12. The correction method for instant images according to claim 11, wherein, the first preset value is plus or minus 5%, and the second preset value is plus or minus 30%.

13. The correction method for instant images according to claim 11, wherein, Step e) includes: calculating an average value of a predicted value of the movement trajectory of the current instant image and an interpolation value of the movement vector, and using the average value as the compensation value.

14. The correction method for instant images according to claim 11, wherein, further includes: g) when it is determined that the first difference is less than the first preset value, setting the compensation value to 0; and g1) after step g), correcting the current instant image based on the compensation value to generate the output image.

15. The correction method for instant images according to claim 14, wherein, Step a) includes: detecting basic image information of each instant image, and the correction method further includes a step a1): calculating a second difference between the basic image information of a plurality of accumulated instant images and the basic image information of the current instant image; wherein step g) sets the compensation value to 0 when the first difference and the second difference are less than the first preset value, step e) calculates the compensation value when the first difference and the second difference fall within the correction allowable range, and step f) resets the image input device when the first difference and the second difference exceed the correction allowable range; wherein the basic image information is at least one of the exposure intensity, frequency, frames per second, number of effective image vertical rows, and number of effective image horizontal pixels of each instant image.

Images