Method and device for improving image frame rate

Abstract

The invention discloses a method and a device for improving an image frame rate. The method comprises the following steps: carrying out image acquisition and exposure according to a set pulse time sequence; in a set pulse time sequence, part of image exposure and collection frames are filtered out according to a preset interval so as to obtain image frames needing to be reserved; and compensating the filtered partial image exposure and acquisition frame to obtain a compensated image frame.

Description

Technical Field

[0001] The present invention relates to the field of image imaging technology, and in particular to a method and device for improving image frame rate. Background Art

[0002] Existing X-ray imaging equipment usually adopts the pulse exposure imaging mode, but the video sequence images generated by this exposure mode often have low frame rate and image freeze. On the one hand, it is due to the limitations of technical conditions, such as the signal readout time of the detector, the time for the X-ray generator to generate stable X-rays, timing control logic, etc. On the other hand, it is due to user needs. Using a lower exposure frame rate can significantly reduce the radiation dose of X-rays.

[0003] Therefore, how to improve the stuttering phenomenon of video sequence images under the premise of equal technical conditions and reduced X-ray radiation dose is of great significance for doctors to observe the surgical site in real time and improve the visual experience. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0005] To this end, the present invention proposes a method and device for improving the image frame rate, which can generate a high frame rate image video sequence under the same technical conditions and under the premise of reducing the X-ray radiation dose, which is beneficial to improving the stuttering phenomenon of the video sequence image and improving the visual effect.

[0006] According to a first aspect of the present application, a method for improving an image frame rate is provided, comprising the following steps:

[0007] Image acquisition and exposure are performed according to the set pulse timing;

[0008] In the set pulse timing, part of the image exposure and acquisition frames are filtered out according to a preset interval to obtain the image frames to be retained;

[0009] The filtered portion of the image exposure and acquisition frame is compensated to obtain a compensated image frame.

[0010] In the above method, the step of compensating the filtered part of the image exposure and acquisition frame to obtain a compensated image frame includes: using a MEMC method to compensate the filtered part of the image exposure and acquisition frame.

[0011] In the above method, the step of using the MEMC method to compensate for the filtered part of the image exposure and acquisition frames includes: using the pyramid LK optical flow algorithm model for compensation.

[0012] In the above method, the step of using the pyramid LK optical flow algorithm model for compensation includes: obtaining the displacement vector of the pixels in two adjacent frames of the image frame to be retained; and obtaining the compensated image frame between the two adjacent frames of the image based on the displacement vector.

[0013] In the above method, the step of obtaining the compensation image frame between two adjacent frames of images based on the displacement vector includes: taking half of the vector in the displacement vector, taking the first frame of the two adjacent frames of images as a reference frame, and combining half of the vector in the displacement vector to obtain the compensation image frame.

[0014] In the above method, the step of taking the first frame of two adjacent frames of images as a reference frame and combining half of the vector in the displacement vector to obtain the compensated image frame includes: performing sub-pixel interpolation on the reference frame based on half of the vector in the displacement vector to obtain the compensated image frame.

[0015] According to a second aspect of the present application, a device for improving an image frame rate is provided, the device comprising:

[0016] An image acquisition and exposure module, used for performing image acquisition and exposure according to a set pulse timing;

[0017] An image acquisition and exposure frame filtering module, used to filter out part of the image exposure and acquisition frames according to a preset interval in the set pulse timing to obtain image frames to be retained;

[0018] The compensation image frame acquisition module is used to compensate for the filtered part of the image exposure and acquisition frames to obtain a compensation image frame.

[0019] In the above device, the compensation image frame acquisition module includes:

[0020] A displacement vector acquisition unit, used for acquiring displacement vectors of pixels in two adjacent image frames in the image frames to be retained;

[0021] The compensation image frame acquisition unit is used to acquire the compensation image frame between two adjacent image frames according to the displacement vector.

[0022] According to a third aspect of the present application, a terminal is provided, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes any one of the above-described methods for improving the image frame rate when running the computer program.

[0023] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the terminal where the storage medium is located is controlled to execute any of the above-mentioned methods for improving the image frame rate.

[0024] According to the technical solution provided by the present application, at least the following beneficial effects are achieved: by performing image acquisition and exposure according to the set pulse timing, the imaging quality of the initial image can be guaranteed; in the set pulse timing, part of the image exposure and acquisition frames are filtered out according to the preset interval to obtain the image frames to be retained, and part of the image exposure and acquisition frames are filtered out to reduce the exposure to reduce the radiation dose of X-rays; the filtered part of the image exposure and acquisition frames are compensated to obtain the compensation image frames, and the filtered part of the image exposure and acquisition frames are compensated to avoid the problem of video sequence image freeze. The above technical solution can generate a high frame rate image video sequence while reducing the radiation dose of X-rays, and can effectively prevent the video sequence image freeze, which is conducive to improving the visual effect.

[0025] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0027] Figure 1 An application scenario diagram provided for an embodiment of the present application;

[0028] Figure 2 A flowchart of a method for improving an image frame rate provided in an embodiment of the present application;

[0029] Figure 3 A timing diagram for compensation of the MEMC module provided in an embodiment of the present application;

[0030] Figure 4 A process diagram of sub-pixel interpolation is provided for the embodiment of the present application;

[0031] Figure 5 A structural block diagram of a device for improving image frame rate provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, not to describe a specific order or sequence.

[0034] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0035] First, a specific application scenario of an embodiment of the present application is described.

[0036] Figure 1 The figure shows an X-ray machine with a dual-path image chain, including an anteroposterior tube, an anteroposterior detector, a lateral tube and a lateral detector. The connecting axis of the anteroposterior tube and the anteroposterior detector is arranged orthogonally to the connecting axis of the lateral tube and the lateral detector. The X-rays generated by the anteroposterior tube pass through the anteroposterior position of the human body and are received by the anteroposterior detector to form an anteroposterior image chain. The X-rays generated by the lateral tube pass through the lateral position of the human body and are received by the lateral detector to form a lateral image chain. The anteroposterior detector and the lateral detector capture and collect the corresponding X-rays and perform data conversion, and transmit the converted digital signals to the computer. The computer converts the data into content that can be displayed on the screen through reconstruction software.

[0037] exist Figure 1 Based on the application scenario shown, the first aspect of the present application provides a method for improving the image frame rate, which can generate a high frame rate image video sequence under the same technical conditions and under the premise of reducing the X-ray radiation dose, which is conducive to improving the stuttering phenomenon of the video sequence image and improving the visual effect. Figure 2 As shown, the method includes steps S110 to S130.

[0038] Step S110: performing image acquisition and exposure according to the set pulse timing.

[0039] In this step, a pulse sequence of 30 pps (pulse per second) is used in advance for image acquisition and exposure. One pulse corresponds to one frame of image. With a pulse sequence of 30 pps, the device can capture 30 frames of image per second. In this application, the pulse sequence can be set according to the actual application to ensure the imaging quality of the initial image. The size of the pulse sequence is not specifically limited in this application.

[0040] Step S120: In the set pulse timing, part of the image exposure and acquisition frames are filtered out according to a preset interval to obtain image frames to be retained.

[0041] In this step, if Figure 3 As shown, during the image acquisition and exposure process at a pulse timing of 30 pps, the exposure and acquisition frames are filtered out every other frame, and some exposure and acquisition frames are filtered out to reduce exposure, which can effectively reduce the radiation dose of X-rays.

[0042] Step S130: compensating the filtered partial image exposure and acquisition frames to obtain compensated image frames.

[0043] In step S120, every other frame is filtered out for exposure and acquisition. Although this can effectively reduce the radiation dose of X-rays, the video sequence image may be stuck. To solve this problem, Figure 3 As shown, the MEMC method (motion estimation and compensation technology) is used to compensate for the filtered image exposure and acquisition frames, so that under the condition of reducing the exposure dose by half, the image resolution effect of 30pps pulse timing can still be presented to ensure image quality. In the image presentation process, it is equivalent to inserting one or more compensation image frames between two adjacent image frames acquired at a low frame rate of 15pps.

[0044] It should be noted that in step S130, the method of inserting a compensating image frame between two adjacent image frames can be implemented by the optical flow algorithm in the MEMC method, specifically using the pyramid LK optical flow algorithm model for compensation. The pyramid LK optical flow algorithm model uses a method of identifying the motion of an object and then performing compensating interpolation, which has good robustness for scenes with strong motion and strong noise.

[0045] In some specific embodiments of the present invention, the method for compensating using the pyramid LK optical flow algorithm model includes steps S131 and S132.

[0046] Step S131: obtaining displacement vectors of pixels in two adjacent image frames to be retained.

[0047] In this step, two adjacent frames of images are represented by I and J respectively, and the specific process of obtaining the displacement vector is as follows:

[0048] (a) Create images I and J as {I L} L＝0,……Lm and {J L} L＝0,……Lm ;

[0049] (b) Initialize the optical flow of the highest layer to g Lm =[0 0] T ;

[0050] (c) Iterate from L = Lm:

[0051] Calculate image I L The position of pixel u in:

[0052] Calculate image I L Gradient in the x direction:

[0053]

[0054] Calculate image I L Gradient in the y direction:

[0055]

[0056] Calculate the matrix G:

[0057] Initialize the initial value of iteration k: d 0 =[0 0] T ;

[0058] Perform k iterations

[0059] Compute image differences:

[0060]

[0061] Calculate vector b k :

[0062]

[0063] Calculate the optical flow η k :η k =G -1 b k ;

[0064] Calculate the initial value for the next iteration: d k =d k-1 +η k ;

[0065] End the iteration on k and get the optical flow on the L layer image: d L =d k ;

[0066] Provide the initial value of optical flow for the L-1 layer: g L-1 =2(g L +d L );

[0067] (d) End the iterative calculation on layer L, and the final optical flow d = g 0 +d 0 , find the corresponding position of pixel u in image I in image J: v=u+d, d represents the displacement vector.

[0068] Step S132: obtaining a compensation image frame between two adjacent image frames according to the displacement vector.

[0069] In this step, half of the displacement vector is taken, and the first frame of two adjacent frames of images is taken as a reference frame, and half of the displacement vector is combined to obtain a compensated image frame.

[0070] Specifically, sub-pixel interpolation is performed on the reference frame based on half of the displacement vectors to obtain a compensated image frame.

[0071] In this step, if Figure 4 As shown, assuming that the pixel coordinates of the compensated image are v = (10, 13), and the displacement vector is d = (0.5, 0.3), according to the formula u = vd = (10, 13) - (0.5, 0.3) = (9.5, 12.7) = (x, y), because the coordinates of the pixel points are integers, the pixel point with coordinates (9.5, 12.7) (i.e. Figure 4 The black square in the center) cannot be accurately located on a certain pixel; x = 9.5 is in the interval [9,10], and its distance from 9 is 0.5, and its distance from 10 is 0.5; y = 12.7 is 0.7 away from 12, and 0.3 away from 13. Therefore, the coordinates of the four pixels associated with the pixel point at coordinates (9.5, 12.7) are (9, 12), (10, 12), (9, 13), and (10, 13). The grayscale value of the pixel point at coordinates (9.5, 12.7) is determined by these four pixels. The pixel point determines that the interpolated grayscale value of the pixel with coordinates (9.5, 12.7) = [grayscale value of (9, 13) * 0.5 + grayscale value of (10, 13) * 0.5] * 0.3 + [grayscale value of (9, 12) * 0.5 + grayscale value of (10, 12) * 0.5] * 0.7, and then the calculated grayscale value of the pixel with coordinates (9.5, 12.7) is assigned to the pixel point (10, 13) of the compensated image. This is how the grayscale value of (10, 13) is calculated through interpolation.

[0072] By adopting the method of improving image frame rate in the present application, through the insertion of compensatory image frames, the user is allowed to acquire patient images under low frame rate exposure conditions, while presenting high frame rate image effects. In high radiation dose application scenarios such as PKP (percutanous kyphoplasty) or angiography that require real-time observation, the radiation dose of patients and doctors is greatly reduced.

[0073] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present invention.

[0074] like Figure 5 As shown, an embodiment of the present application provides a device for improving an image frame rate, the device comprising an image acquisition and exposure module 10, an image acquisition and exposure frame filtering module 20 and a compensation image frame acquisition module 30.

[0075] in,

[0076] An image acquisition and exposure module 10, used for performing image acquisition and exposure according to a set pulse timing;

[0077] The image acquisition and exposure frame filtering module 20 is used to filter out part of the image exposure and acquisition frames according to a preset interval in the set pulse timing to obtain the image frames to be retained;

[0078] The compensation image frame acquisition module 30 is used to compensate for the filtered partial image exposure and acquisition frames to acquire compensation image frames.

[0079] In some specific embodiments of the present invention, the compensated image frame acquisition module 30 adopts the MEMC method, specifically, using the pyramid LK optical flow algorithm model to compensate for the filtered partial image exposure and acquisition frames.

[0080] In some specific embodiments of the present invention, the compensated image frame acquisition module 30 includes a displacement vector acquisition unit 31 and a compensated image frame acquisition unit 32 .

[0081] The displacement vector acquisition unit 31 is used to acquire the displacement vectors of pixels in two adjacent image frames to be retained;

[0082] The compensation image frame acquisition unit 32 is used to acquire a compensation image frame between two adjacent image frames according to the displacement vector.

[0083] In some specific embodiments of the present invention, the compensation image frame acquisition unit 32 is further used to obtain half of the displacement vector, take the first frame of two adjacent frames as the reference frame, and combine the half of the displacement vector to obtain the compensation image frame. Specifically, the reference frame is sub-pixel interpolated based on the half of the displacement vector to obtain the compensation image frame.

[0084] An embodiment of the present application further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the above-mentioned Figure 2 The method of increasing the image frame rate is shown.

[0085] Specifically, the processor may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0086] Specifically, the processor is connected to the memory via a bus, and the bus may include a path for transmitting information. The bus may be a PCI bus or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0087] The memory can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, CD-ROM or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0088] Optionally, the memory is used to store the code of the computer program for executing the solution of the present application, and the execution is controlled by the processor. The processor is used to execute the application program code stored in the memory to implement Figure 5 The operation of the device for increasing the image frame rate is shown.

[0089] An embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein when the computer program is executed by a processor, a terminal where the storage medium is located is controlled to execute the above Figure 2 The method of increasing the image frame rate is shown.

[0090] The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0092] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present application. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for improving an image frame rate, characterized in that: The following steps are involved: Image acquisition and exposure are performed according to the set pulse timing; In the set pulse timing, part of the image exposure and acquisition frames are filtered out according to a preset interval to obtain the image frames to be retained; The filtered portion of the image exposure and acquisition frame is compensated to obtain a compensated image frame.

2. The method for improving the image frame rate according to claim 1, characterized in that: The step of compensating the filtered part of the image exposure and acquisition frame to obtain a compensated image frame includes: using a MEMC method to compensate the filtered part of the image exposure and acquisition frame.

3. The method for improving the image frame rate according to claim 2, characterized in that: The step of using the MEMC method to compensate for the filtered part of the image exposure and acquisition frames includes: The pyramid LK optical flow algorithm model is used for compensation.

4. The method for improving the image frame rate according to claim 3, characterized in that: The step of using the pyramid LK optical flow algorithm model for compensation includes: Obtaining displacement vectors of pixels in two adjacent image frames in the image frames to be retained; The compensated image frame between two adjacent image frames is obtained according to the displacement vector.

5. The method for improving the image frame rate according to claim 4, characterized in that: The step of acquiring the compensated image frame between two adjacent image frames according to the displacement vector comprises: A half of the displacement vector is taken, and the first frame of two adjacent frames of images is taken as a reference frame, and the half of the displacement vector is combined to obtain the compensated image frame.

6. The method for improving the image frame rate according to claim 5, characterized in that: The step of taking the first frame of two adjacent frames of images as a reference frame and combining half of the displacement vectors to obtain the compensated image frame comprises: Sub-pixel interpolation is performed on the reference frame based on half of the displacement vectors to obtain the compensated image frame.

7. A device for improving an image frame rate, characterized in that: include: An image acquisition and exposure module, used for performing image acquisition and exposure according to a set pulse timing; An image acquisition and exposure frame filtering module, used to filter out part of the image exposure and acquisition frames according to a preset interval in the set pulse timing to obtain image frames to be retained; The compensation image frame acquisition module is used to compensate for the filtered part of the image exposure and acquisition frames to obtain a compensation image frame.

8. The device for improving image frame rate according to claim 7, characterized in that: The compensation image frame acquisition module comprises: A displacement vector acquisition unit, used for acquiring displacement vectors of pixels in two adjacent image frames in the image frames to be retained; The compensation image frame acquisition unit is used to acquire the compensation image frame between two adjacent image frames according to the displacement vector.

9. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the method for improving the image frame rate according to any one of claims 1 to 6 is performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the terminal where the storage medium is located is controlled to execute the method for improving the image frame rate according to any one of claims 1 to 6.

Images