TOF Camera Motion Blur Detection Method, System, Device and Storage Medium

Abstract

The present invention provides a method, system, device and storage medium for detecting motion blur of a TOF camera, including: collecting at least a first electrical signal, a second electrical signal, a third electrical signal and a fourth electrical signal at a pixel point according to a preset equal-interval phase difference; determining a scaling factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal; determining whether the scaling factor is within a preset threshold range, and when the scaling factor is within the preset ratio range, determining that the pixel point is motion blurred. In the present invention, four electrical signals are collected at equal-interval phase differences at a pixel point, the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal is used to determine the scaling factor, and whether there is motion blur at the pixel point is determined according to the scaling factor, so as to realize the rapid detection of motion blur within a single-frequency frame and between double-frequency frames, facilitate the removal of motion blur, and improve the display effect of the depth image.

Description

Background Art

[0002] In recent years, 3D vision has been widely applied in industries such as robotics, industrial production, intelligent logistics, medical treatment, autonomous driving, and secure payment. Among them, as a device capable of outputting depth maps, TOF cameras have also received extensive research. The principle of TOF cameras is to calculate the distance between the photographed object and the camera by calculating the phase difference between the emitted infrared light and the received infrared light. To improve the accuracy, four sine waves with different phases are usually used in the prior art to perform correlation with the received light for calculating depth values. Further, in order to double the range of the camera ranging, two infrared lights with different frequencies are usually used to calculate two depth maps, and a depth map with a larger range is obtained through a dual-frequency fusion method.

[0003] However, in practical applications, due to the movement of the object or the movement of the camera itself, the problem of mismatch of the correlation maps of four different phases at the same frequency and the mismatch of the depth maps between two frequencies usually occurs, resulting in incorrect depth values due to movement when calculating depth using four phases and during dual-frequency fusion, and motion blur is generated.

[0004] For the elimination of motion blur, due to the inconsistent imaging methods of TOF cameras and traditional RGB cameras, the methods used by traditional cameras to eliminate motion blur cannot be directly applied to TOF cameras. Currently, the methods for detecting and removing motion blur for TOF cameras all have drawbacks, cannot be applied generally to most scenarios, and there is currently no detection and removal of motion blur generated during dual-frequency fusion. Summary of the Invention

[0005] Aiming at the defects in the prior art, the purpose of the present invention is to provide a method, system, device, and storage medium for detecting motion blur of a TOF camera.

[0006] The method for detecting motion blur of a TOF camera according to the present invention includes the following steps:

[0007] Step S1: At least collect a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal-interval phase difference;

[0008] Step S2: Determine a proportionality factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal;

[0009] Step S3: Determine whether the proportionality factor is within a preset threshold range. When the proportionality factor is within the preset proportionality range, it is determined that the pixel point is motion blurred.

[0010] Preferably, the step S1 includes the following steps:

[0011] Step S101: Project a light beam onto the target through the light projector of the TOF camera, where the light beam is a sinusoidal light beam formed by modulation;

[0012] Step S102: Receive the light beam reflected by the target through the imaging array of the TOF camera, and generate an electrical signal according to the light signal received by each detector in the imaging array;

[0013] Step S103: Collect the first electrical signal, the second electrical signal, the third electrical signal, and the fourth electrical signal at an equal interval phase of 90°.

[0014] Preferably, step S1 includes the following steps:

[0015] Step S101: Project a first light beam with a first frequency and a second light beam with a second frequency onto the target through the light projector of the TOF camera, where the first light beam and the second light beam are sinusoidal light beams formed by modulation;

[0016] Step S102: Receive the first light beam and the second light beam reflected by the target through the imaging array of the TOF camera, and generate an electrical signal according to the light signal received by each detector in the imaging array;

[0017] Step S103: Collect the first electrical signal A, the second electrical signal A, the third electrical signal A, and the fourth electrical signal A at an equal interval phase of 90° for the electrical signal formed by receiving the first light beam;

[0018] Step S104: Collect the first electrical signal B, the second electrical signal B, the third electrical signal B, and the fourth electrical signal B at an equal interval phase of 90° for the electrical signal formed by receiving the second light beam.

[0019] Preferably, step S2 includes the following steps:

[0020] Step S201: Calculate the sum of the values of the first electrical signal and the third electrical signal, that is, Sum0 + Sum 180 , where the first electrical signal Sum0 represents the electrical signal with a phase angle of 0, and the third electrical signal Sum 180 represents the electrical signal with a phase angle of 180;

[0021] Step S202: Calculate the sum of the values of the second electrical signal and the fourth electrical signal, that is, Sum 90 + Sum 270 , where the second electrical signal Sum 90 represents the electrical signal with a phase angle of 90, and the third electrical signal Sum 270 represents the electrical signal with a phase angle of 270;

[0022] Step S203: Generate a scale factor ratio, where ratio = (Sum o + Sum 180 ) / (Sum 90 + Sum 270 ).

[0023] Preferably, step S2 includes the following steps:

[0024] Step S201: Calculate the sum of the values of the first electrical signal A, the first electrical signal B, the third electrical signal A, and the third electrical signal B, that is The first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the first light beam, and the third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the first light beam. The first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the second light beam, and the third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the second light beam. 75Mhz represents the frequency of the first light beam, and 100Mhz represents the frequency of the second light beam;

[0025] Step S202: Calculate the sum of the values of the second electrical signal A, the fourth electrical signal A, the second electrical signal B, and the fourth electrical signal B, that is The first electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the first light beam, and the third electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the first light beam. The first electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the second light beam, and the third electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the second light beam;

[0026] Step S203: Generate a scale factor ratio, where

[0027] Preferably, it further includes the following steps:

[0028] Step M1: Repeatedly execute steps S1 to S3 to determine whether there is motion blur in multiple pixel points. When a pixel point has motion blur, set the pixel value of this pixel point to the first value, otherwise set it to the second value, and generate a first binary image;

[0029] Step M2: Perform a dilation operation on the first binary image to remove isolated pixel points and generate a second binary image;

[0030] Step M3: Obtain a depth image, which is pixel - level aligned with the second binarized image. Set the depth values at the corresponding positions of the pixels with the first numerical value in the second binarized image to 0 in the depth image to remove motion blur.

[0031] Preferably, each of the electrical signals is the sum of electrical signal tapA and electrical signal tapB;

[0032] The electrical signal tapA is collected through a first capacitor, and the electrical signal tapB is collected through a second capacitor; the phase difference between the signal reception time windows of the first capacitor and the second capacitor is 180°.

[0033] According to the TOF camera motion blur detection system provided by the present invention, it is characterized by including the following modules:

[0034] An electrical signal acquisition module, configured to collect at least a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal - interval phase difference;

[0035] A scale factor calculation module, configured to determine a scale factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal;

[0036] A motion blur determination module, configured to determine whether the scale factor is within a preset threshold range. When the scale factor is within the preset ratio range, it is determined that the pixel point is motion - blurred.

[0037] According to the TOF camera motion blur detection device provided by the present invention, it includes:

[0038] A processor;

[0039] A memory, in which executable instructions of the processor are stored;

[0040] Wherein, the processor is configured to execute the steps of the TOF camera motion blur detection method by executing the executable instructions.

[0041] According to the computer - readable storage medium provided by the present invention, it is used to store a program, and when the program is executed, the steps of the TOF camera motion blur detection method are implemented.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] In the present invention, four electrical signals are collected at equal intervals of phase difference at a pixel point. A proportionality factor is determined by the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal. Whether there is motion blur at the pixel point is judged according to the proportionality factor, realizing the fast detection of single-frequency intra-frame and dual-frequency inter-frame motion blur, facilitating the removal of motion blur, and improving the display effect of the depth image;

[0044] In the present invention, each of the electrical signals is the sum of the electrical signal tapA and the electrical signal tapB. The electrical signal tapA and the electrical signal tapB are electrical signals collected at the same moment through two different capacitors. Compared with only using tapA or tapB, it can improve the accuracy of motion blur, enhance the robustness of the present invention to the time noise of a single capacitor, thereby reducing the possibility of misjudgment and missed judgment, and can ensure the effective identification of the motion blur area in more scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages of the present invention will become more obvious:

[0046] Figure 1 It is a flowchart of the steps of the TOF camera motion blur detection method in the embodiment of the present invention;

[0047] Figure 2 It is a flowchart of the steps of collecting electrical signals at a pixel point in the embodiment of the present invention;

[0048] Figure 3 It is a flowchart of the steps of collecting electrical signals at a pixel point in the modified example of the present invention;

[0049] Figure 4 It is a flowchart of the steps of calculating the proportionality factor in the embodiment of the present invention;

[0050] Figure 5 It is a flowchart of the steps of calculating the proportionality factor in the modified example of the present invention;

[0051] Figure 6 It is a flowchart of the steps of removing motion blur in the embodiment of the present invention;

[0052] Figure 7 It is a schematic diagram of the modules of the TOF camera motion blur detection system in the embodiment of the present invention;

[0053] Figure 8 It is a schematic structural diagram of the TOF camera motion blur detection device in an embodiment of the present invention; and

[0054] Figure 9 It is a schematic structural diagram of the computer-readable storage medium in an embodiment of the present invention. Specific embodiments

[0055] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made. These all fall within the protection scope of the present invention.

[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here, for example, can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0057] The technical solution of the present invention will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0058] The TOF camera motion blur detection method provided by the present invention aims to solve the problems existing in the prior art.

[0059] The technical solution of the present invention and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below with reference to the drawings.

[0060] Figure 1 It is a step flowchart of the TOF camera motion blur detection method in an embodiment of the present invention. As Figure 1 shown, the TOF camera motion blur detection method provided by the present invention includes the following steps:

[0061] Step S1: At least collect a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal interval phase difference;

[0062] Step S2: Determine a proportionality factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal;

[0063] Step S3: Determine whether the proportionality factor is within a preset threshold interval. When the proportionality factor is within the preset proportionality interval, it is determined that the pixel point is motion blurred.

[0064] In the embodiment of the present invention, the present invention can be applied to a 2 - tap, 4 - phase TOF camera. Each of the electrical signals is the sum of electrical signal tapA and electrical signal tapB;

[0065] The electrical signal tapA is collected through a first capacitor, and the electrical signal tapB is collected through a second capacitor; the phase difference between the signal reception time windows of the first capacitor and the second capacitor is 180°.

[0066] In the embodiment of the present invention, the electrical signal tapB is the anti - phase signal of the electrical signal tapA. The electrical signal tapB and the electrical signal tapA can be voltage signals, or can be electric charge amounts or pixel values. In the present invention, each of the electrical signals is the sum of the electrical signal tapA and the electrical signal tapB. The electrical signals of tapA and tapB are collected at the same moment through two different capacitors. Compared with only using tapA or tapB, it can improve the accuracy of motion blur, enhance the robustness of the present invention to the time noise of a single capacitor, thereby reducing the possibility of misjudgment and missed judgment, and can ensure the effective identification of the motion - blurred area in more scenarios.

[0067] In the embodiment of the present invention, the threshold interval can be set to [0, 0.98) U (1.02, +∞].

[0068] Figure 2 This is the flowchart of the steps for collecting electrical signals at a pixel point in the embodiment of the present invention. As Figure 2 shown, the step S1 includes the following steps:

[0069] Step S101: Project a light beam towards the target through the light projector of the TOF camera. The light beam is a sinusoidal light beam formed by modulation;

[0070] Step S102: Receive the light beam reflected by the target through the imaging array of the TOF camera, and generate electrical signals according to the light signals received by each detector in the imaging array;

[0071] Step S103: Collect the first electrical signal, the second electrical signal, the third electrical signal, and the fourth electrical signal at equal interval phases of 90°.

[0072] In an embodiment of the present invention, an optical projector is configured to project a light beam onto a target object in a scene; an imaging array is configured to receive, through at least four receiving windows, the light beam reflected by the target object to form an optical signal, and then generate four acquisition electrical signals.

[0073] In an embodiment of the present invention, the optical projector includes a light source, a light source driver, and an optical modulator.

[0074] The light source driver is connected to the light source and is configured to drive the light source to emit light.

[0075] The optical modulator is connected to the light source and is configured to modulate the light projected by the light source and then project the light beam onto the object to be measured.

[0076] In an embodiment of the present invention, the imaging array includes a lens, a filter, and an image sensor arranged along the optical path, and at least four of the receiving windows are provided on the image sensor.

[0077] The image sensor is configured to receive at least the optical signal through at least four receiving windows; the at least four receiving windows are arranged sequentially in time series, and then each of the electrical signals is generated according to the optical signal received by each of the receiving windows.

[0078] In an embodiment of the present invention, the first electrical signal is an electrical signal with a phase angle of 0, the second electrical signal is an electrical signal with a phase angle of 90, the third electrical signal is an electrical signal with a phase angle of 180, and the fourth electrical signal is an electrical signal with a phase angle of 270.

[0079] Figure 3 This is a flowchart of the steps for collecting electrical signals at pixel points in a modified example of the present invention. As Figure 3 shown, the step S1 includes the following steps:

[0080] Step S101: Project a first light beam with a first frequency and a second light beam with a second frequency onto a target object through the optical projector of the TOF camera, and the first light beam and the second light beam are sinusoidal light beams formed by modulation.

[0081] Step S102: Receive the first light beam and the second light beam reflected by the target object through the imaging array of the TOF camera, and generate electrical signals according to the optical signals received by each detector in the imaging array.

[0082] Step S103: Collect a first electrical signal A, a second electrical signal A, a third electrical signal A, and a fourth electrical signal A at equal interval phases of 90° for the electrical signals formed by receiving the first light beam.

[0083] Step S104: Collect the first electrical signal B, the second electrical signal B, the third electrical signal B, and the fourth electrical signal B at equal interval phases of 90° for the electrical signal formed by receiving the second light beam.

[0084] In the embodiment of the present invention, the first electrical signal A is an electrical signal with a phase angle of 0, the second electrical signal A is an electrical signal with a phase angle of 90, the third electrical signal A is an electrical signal with a phase angle of 180, and the fourth electrical signal A is an electrical signal with a phase angle of 270. The first electrical signal B is an electrical signal with a phase angle of 0, the second electrical signal B is an electrical signal with a phase angle of 90, the third electrical signal B is an electrical signal with a phase angle of 180, and the fourth electrical signal B is an electrical signal with a phase angle of 270.

[0085] Figure 4 It is a flowchart of the steps for calculating the scale factor in the embodiment of the present invention, as Figure 4 shown, the step S2 includes the following steps:

[0086] Step S201: Calculate the numerical sum of the first electrical signal and the third electrical signal, that is, Sum0 + Sum 180 , where the first electrical signal Sum0 represents the electrical signal with a phase angle of 0, and the third electrical signal Sum 180 represents the electrical signal with a phase angle of 180;

[0087] Step S202: Calculate the numerical sum of the second electrical signal and the fourth electrical signal, that is, Sum 90 + Sum 270 , where the second electrical signal Sum 90 represents the electrical signal with a phase angle of 90, and the third electrical signal Sum 270 represents the electrical signal with a phase angle of 270;

[0088] Step S203: Generate a scale factor ratio, where ratio = (Sum o + Sum 180 ) / (Sum 90 + Sum 270 ).

[0089] Figure 5 It is a flowchart of the steps for calculating the scale factor in the variant example of the present invention, as Figure 5 shown, the step S2 includes the following steps:

[0090] Step S201: Calculate the numerical sum of the first electrical signal A, the first electrical signal B, the third electrical signal A, and the third electrical signal B, that is The first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the first light beam, and the third electrical signal Represents an electrical signal with a phase angle of 180 formed by receiving the first light beam, and the first electrical signal Represents an electrical signal with a phase angle of 0 formed by receiving the second light beam, and the third electrical signal Represents an electrical signal with a phase angle of 180 formed by receiving the second light beam. 75Mhz represents the frequency of the first light beam, and 100Mhz represents the frequency of the second light beam;

[0091] Step S202: Calculate the sum of the values of the second electrical signal A, the fourth electrical signal A, the second electrical signal B, and the fourth electrical signal B, that is The first electrical signal Represents an electrical signal with a phase angle of 90 formed by receiving the first light beam, and the third electrical signal Represents an electrical signal with a phase angle of 270 formed by receiving the first light beam, and the first electrical signal Represents an electrical signal with a phase angle of 90 formed by receiving the second light beam, and the third electrical signal Represents an electrical signal with a phase angle of 270 formed by receiving the second light beam;

[0092] Step S203: Generate a scale factor ratio, where

[0093]

[0094] In an embodiment of the present invention, it is also possible to judge motion blur according to |1 - ratio| > ε1, and the value of ε1 can be set to 0.02.

[0095] Figure 6 This is the flowchart of the steps for removing motion blur in an embodiment of the present invention. As Figure 6 shown, the TOF camera motion blur detection method provided by the present invention further includes the following steps:

[0096] Step M1: Repeatedly execute Step S1 to Step S3. When judging whether there is motion blur in multiple pixel points, when a pixel point has motion blur, set the pixel value of this pixel point to the first value, otherwise set it to the second value, and generate a first binary image;

[0097] Step M2: Perform a dilation operation on the first binary image to remove isolated pixel points and generate a second binary image;

[0098] Step M3: Obtain a depth image. The depth image is pixel - level aligned with the second binary image. Set the depth value at the corresponding position of the depth image for the pixel points with the pixel value of the first value in the second binary image to 0 to remove motion blur.

[0099] In an embodiment of the present invention, the first value may be set to 255; the second value may be set to 0; the depth image is generated from an infrared image generated by the TOF camera while collecting the first electrical signal, the second electrical signal, the third electrical signal, and the fourth electrical signal. The dilation operation specifically enlarges each pixel point so that adjacent pixel points are connected.

[0100] Figure 7 It is a schematic diagram of the modules of the TOF camera motion blur detection system in an embodiment of the present invention. As Figure 7 shown, the TOF camera motion blur detection system provided by the present invention includes the following modules:

[0101] An electrical signal acquisition module, configured to collect at least a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal-interval phase difference;

[0102] A scale factor calculation module, configured to determine a scale factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal;

[0103] A motion blur judgment module, configured to judge whether the scale factor is within a preset threshold range. When the scale factor is within the preset ratio range, it is determined that the pixel point is motion blurred.

[0104] An embodiment of the present invention also provides a TOF camera motion blur detection device, including a processor. A memory, which stores executable instructions of the processor. Wherein, the processor is configured to execute the steps of the TOF camera motion blur detection method by executing the executable instructions.

[0105] As described above, this embodiment can determine a scale factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal through four electrical signals collected at equal-interval phase differences at a pixel point, and determine whether there is motion blur at the pixel point according to the scale factor, realizing fast detection of motion blur within a single frequency frame and between double frequency frames, facilitating motion blur removal, and improving the display effect of the depth image.

[0106] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0107] Figure 8 It is a schematic diagram of the structure of the TOF camera motion blur detection device in an embodiment of the present invention. The following refers toFigure 8 Describe the electronic device 600 according to this embodiment of the present invention. Figure 8 The displayed electronic device 600 is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0108] As Figure 8 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0109] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above TOF camera motion blur detection method part of this specification. For example, the processing unit 610 can execute the steps as Figure 1 shown.

[0110] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0111] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0112] The bus 630 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any bus structure in a variety of bus structures.

[0113] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Figure 8 is not shown, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0114] An embodiment of the present invention also provides a computer-readable storage medium for storing a program, and when the program is executed, the steps of the TOF camera motion blur detection method are implemented. In some possible implementation manners, various aspects of the present invention can also be implemented in the form of a program product, which includes program code, and when the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above-mentioned TOF camera motion blur detection method part of this specification.

[0115] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, in the present invention, four electrical signals collected at equal interval phase differences at a pixel point are used to determine a proportionality factor by the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal, and it is determined whether there is motion blur at this pixel point according to the proportionality factor, so as to realize the quick detection of motion blur within a single frequency frame and between double frequency frames, facilitate the removal of motion blur, and improve the display effect of the depth image.

[0116] Figure 9 is a schematic structural diagram of the computer-readable storage medium in the embodiment of the present invention. Refer to Figure 9 shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device.

[0117] The program product may employ any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable storage medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0119] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0120] In an embodiment of the present invention, in the present invention, four electrical signals are collected at equal interval phase differences at a pixel point. A proportionality factor is determined by the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal. Whether there is motion blur at the pixel point is determined according to the proportionality factor, realizing fast detection of single-frequency intra-frame and double-frequency inter-frame motion blur, facilitating motion blur removal, and improving the display effect of the depth image; in the present invention, each of the electrical signals is the sum of electrical signal tapA and electrical signal tapB. Electrical signal tapA and electrical signal tapB are electrical signals collected by two different capacitors at the same moment. Compared with only using tapA or tapB, it can improve the accuracy of motion blur, enhance the robustness of the present invention to the time noise of a single capacitor, thereby reducing the possibility of misjudgment and missed judgment, and can ensure effective identification of the motion blur area in more scenarios.

[0121] In this specification, the various embodiments are described in a progressive manner. The key point of each embodiment is the difference from other embodiments. The same or similar parts among the various embodiments can be referred to each other. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0122] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which does not affect the essence of the present invention.

Claims

1. A method for detecting motion blur of a TOF camera, characterized in that, It includes the following steps: Step S1: At least collect a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal-interval phase difference; Step S2: Determine a scaling factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal; Step S3: Determine whether the scaling factor is within a preset threshold interval. When the scaling factor is within the preset ratio interval, it is determined that the pixel point is motion blurred; Step S2 includes the following steps: Step S201: Calculate the sum of the values of the first electrical signal A, the first electrical signal B, the third electrical signal A, and the third electrical signal B, that is The first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the first light beam. The third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the first light beam. The first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the second light beam. The third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the second light beam. 75Mhz represents the frequency of the first light beam, and 100Mhz represents the frequency of the second light beam; Step S202: Calculate the sum of the values of the second electrical signal A, the fourth electrical signal A, the second electrical signal B, and the fourth electrical signal B, that is The second electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the first light beam, and the fourth electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the first light beam, and the second electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the second light beam, and the fourth electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the second light beam; Step S203: Generate a scale factor ratio, where, 2. The TOF camera motion blur detection method according to claim 1, wherein Step S1 includes the following steps: Step S101: Project a first light beam with a first frequency and a second light beam with a second frequency onto the target object through the light projector of the TOF camera. The first light beam and the second light beam are sinusoidal light beams formed by modulation; Step S102: Receive the first light beam and the second light beam reflected by the target object through the imaging array of the TOF camera, and generate electrical signals according to the light signals received by each detector in the imaging array; Step S103: Collect a first electrical signal A, a second electrical signal A, a third electrical signal A, and a fourth electrical signal A at equal-interval phases of 90° for the electrical signals formed by receiving the first light beam; Step S104: Collect a first electrical signal B, a second electrical signal B, a third electrical signal B, and a fourth electrical signal B at equal-interval phases of 90° for the electrical signals formed by receiving the second light beam.

3. The method for detecting motion blur of a TOF camera according to claim 1, characterized in that, It also includes the following steps: Step M1: Repeat steps S1 to S3. When determining whether there is motion blur in multiple pixel points, when a pixel point has motion blur, set the pixel value of this pixel point to a first value, otherwise set it to a second value, and generate a first binary image; Step M2: Perform a dilation operation on the first binary image to remove isolated pixel points and generate a second binary image; Step M3: Obtain a depth image. The depth image is pixel-level aligned with the second binary image. Set the depth value at the corresponding position of the pixel points with the first value in the second binary image on the depth image to 0 to remove motion blur.

4. The method for detecting motion blur of a TOF camera according to claim 1, characterized in that, Each of the electrical signals is the sum of an electrical signal tapA and an electrical signal tapB; The electrical signal tapA is collected through a first capacitor, and the electrical signal tapB is collected through a second capacitor; the phase difference between the signal reception time windows of the first capacitor and the second capacitor is 180°.

5. A TOF camera motion blur detection system, characterized in that, It includes the following modules: An electrical signal acquisition module for collecting at least a first electrical signal, a second electrical signal, a third electrical signal, and a fourth electrical signal at a pixel point according to a preset equal-interval phase difference; A scaling factor calculation module for determining a scaling factor according to the ratio of the sum of the values of the first electrical signal and the third electrical signal to the sum of the values of the second electrical signal and the fourth electrical signal; A motion blur determination module for determining whether the scaling factor is within a preset threshold interval. When the scaling factor is within the preset ratio interval, it is determined that the pixel point is motion blurred; When the scaling factor calculation module is processing, it includes the following steps: Step S201: Calculate the sum of the values of the first electrical signal A, the first electrical signal B, the third electrical signal A, and the third electrical signal B, that is the first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the first light beam, and the third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the first light beam, and the first electrical signal represents the electrical signal with a phase angle of 0 formed by receiving the second light beam, and the third electrical signal represents the electrical signal with a phase angle of 180 formed by receiving the second light beam, 75Mhz represents the frequency of the first light beam, and 100Mhz represents the frequency of the second light beam; Step S202: Calculate the sum of the values of the second electrical signal A, the fourth electrical signal A, the second electrical signal B, and the fourth electrical signal B, that is The second electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the first light beam. The fourth electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the first light beam. The second electrical signal represents the electrical signal with a phase angle of 90 formed by receiving the second light beam. The fourth electrical signal represents the electrical signal with a phase angle of 270 formed by receiving the second light beam; Step S203: Generate a scale factor ratio, where, 6. A TOF camera motion blur detection device, characterized in that, It includes: A processor; A memory in which executable instructions of the processor are stored; Wherein, the processor is configured to execute the steps of the TOF camera motion blur detection method according to any one of claims 1 to 4 by executing the executable instructions.

7. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, the steps of the TOF camera motion blur detection method according to any one of claims 1 to 4 are implemented.

Images