Detection device and detection method

Abstract

A detection device is used for detecting an object and comprises a camera module, an image processing module and a radar module. The camera module can acquire an image of an object. The image processing module can analyze the image to define a target sensing area and generate a radar set value. The radar module can be controlled by the image processing module, wherein the radar module can be selectively operated in a first analysis mode or a second analysis mode. The radar module can be initially operated in a first analysis mode. In the second analytic mode, the radar module can detect a specific portion of an object located in the target sensing area according to the radar setpoint.

Description

Technical Field

[0001] The present invention relates to a detection device, and in particular to a detection device and a detection method. Background Art

[0002] Radar devices are common positioning components. However, when used in the fields of virtual reality (VR) or augmented reality (AR), traditional radar devices are prone to target locking errors, which will reduce the overall detection accuracy. In view of this, it is necessary to propose a new solution to overcome the difficulties faced by previous technologies. Summary of the invention

[0003] In a preferred embodiment, the present invention proposes a detection device for detecting an object, and includes: a camera module for acquiring an image of the object; an image processing module for analyzing the image to define a target sensing area and generate a radar setting value; and a radar module controlled by the image processing module, wherein the radar module selectively operates in a first analysis mode or a second analysis mode; wherein the radar module initially operates in the first analysis mode; wherein in the second analysis mode, the radar module detects a specific portion of the object located in the target sensing area according to the radar setting value.

[0004] In some embodiments, the detection device is implemented via a head mounted display.

[0005] In some embodiments, the object is a human body, and the specific part of the object is a hand of the human body.

[0006] In some embodiments, in the first analysis mode, if the radar module detects that anything is approaching, the radar module will notify the camera module so that the camera module is activated.

[0007] In some embodiments, a frame rate of the camera module is lower than or equal to a predetermined value.

[0008] In some embodiments, in the first analysis mode, a pulse repetition time of the radar module is greater than or equal to a critical value.

[0009] In some embodiments, in the second analysis mode, the pulse repetition time of the radar module is less than the critical value.

[0010] In some embodiments, after the image processing module finishes analyzing the image, the image processing module controls the radar module to leave the first analysis mode and enter the second analysis mode.

[0011] In some embodiments, the target sensing area corresponds to a rectangular parallelepiped area.

[0012] In some embodiments, the radar setting includes an antenna angle setting and a pulse repetition time setting.

[0013] In some embodiments, the radar module includes: an antenna array including a plurality of antenna elements; and a radar controller for controlling a deflection angle of the antenna array according to the radar setting value.

[0014] In another preferred embodiment, the present invention proposes a detection method, comprising the following steps: operating a radar module in a first analysis mode; obtaining an image of an object through a camera module; analyzing the image through an image processing module to define a target sensing area and generate a radar setting value; operating the radar module in a second analysis mode; and detecting a specific portion of the object located in the target sensing area according to the radar setting value through the radar module.

[0015] In some embodiments, the detection method further includes: in the first analysis mode, if anything is detected approaching, the camera module is notified through the radar module so that the camera module is activated.

[0016] In some embodiments, the detection method further includes: after the image is analyzed, controlling the radar module through the image processing module to leave the first analysis mode and enter the second analysis mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram showing a detection device according to an embodiment of the present invention.

[0018] Figure 2 The figure shows a pulse waveform of a radar module according to an embodiment of the present invention.

[0019] Figure 3 is a schematic diagram showing an object according to an embodiment of the present invention.

[0020] Figure 4 is a schematic diagram showing a detection device according to an embodiment of the present invention.

[0021] Figure 5 is a schematic diagram showing a radar module according to an embodiment of the present invention.

[0022] Figure 6 is a flow chart showing a detection method according to an embodiment of the present invention.

[0023] Explanation of symbols:

[0024] 100,400: Detection Device

[0025] 110: Camera module

[0026] 120: Image processing module

[0027] 130,530: Radar module

[0028] 190,390,590: Objects

[0029] 192,392: Dashed box

[0030] 395: Specific parts of objects

[0031] 497: Cuboid area

[0032] 540: Antenna array

[0033] 550-1,550-2,550-N: Antenna element

[0034] 554: Radar Plane

[0035] 555: Normal

[0036] 556: Connection

[0037] 560: Radar Controller

[0038] H1: Height

[0039] IG:Image

[0040] L1: Length

[0041] MD1: First parsing mode

[0042] MD2: Second parsing mode

[0043] RN: Frame rate

[0044] RS: Radar setting value

[0045] TD: Pulse repetition time

[0046] W1: Width

[0047] X: X-axis

[0048] Y: Y axis

[0049] Z: Z axis

[0050] ZG: Target sensing area

[0051] θ: deflection angle DETAILED DESCRIPTION

[0052] In order to make the purpose, features and advantages of the present invention more clearly understood, specific embodiments of the present invention are given below and described in detail with reference to the accompanying drawings.

[0053] Certain words are used in the specification and claims to refer to specific components. It should be understood by those skilled in the art that hardware manufacturers may use different terms to refer to the same component. This specification and claims do not use differences in names as a way to distinguish components, but use differences in the functions of components as the criteria for distinction. The words "include" and "comprise" mentioned throughout the specification and claims are open-ended terms and should be interpreted as "include but not limited to". The word "substantially" means that within an acceptable error range, those skilled in the art can solve the technical problem and achieve the basic technical effect within a certain error range. In addition, the word "coupled" in this specification includes any direct and indirect electrical connection means. Therefore, if the text describes a first device coupled to a second device, it means that the first device can be directly electrically connected to the second device, or indirectly electrically connected to the second device via other devices or connection means.

[0054] The following disclosure provides many different embodiments or examples to implement the different features of the present invention. The following disclosure describes specific examples of various components and their arrangements to simplify the description. Of course, these specific examples are not intended to be limiting. For example, if the present disclosure describes a first feature formed on or above a second feature, it means that it may include an embodiment in which the first feature and the second feature are in direct contact, and may also include an embodiment in which an additional feature is formed between the first feature and the second feature, so that the first feature and the second feature may not be in direct contact. In addition, the same reference symbols or (and) marks may be reused in different examples of the following disclosure. These repetitions are for the purpose of simplification and clarity, and are not intended to limit the specific relationship between the different embodiments or (and) structures discussed.

[0055] In addition, spatially related terms such as "below", "below", "lower", "above", "higher" and similar terms are used to facilitate the description of the relationship between one element or feature and another element or features in the figure. In addition to the orientation shown in the drawings, these spatially related terms are intended to include different orientations of the device in use or operation. The device may be turned to different orientations (rotated 90 degrees or other orientations), and the spatially related terms used herein may also be interpreted in the same manner.

[0056] Figure 11 is a schematic diagram showing a detection device 100 according to an embodiment of the present invention. The detection device 100 can be applied to a head mounted display (HMD) or a mobile device, such as an augmented reality glasses (AR Glasses), a smart phone (Smart Phone), a tablet computer (Tablet Computer), or a notebook computer (Notebook Computer), but is not limited thereto. Figure 1 In the embodiment of the present invention, the detection device 100 at least includes: a camera module 110, an image processing module 120, and a radar module 130. It should be understood that although not shown in FIG. Figure 1 However, the detection device 100 may further include other components, such as: a housing, a speaker, or (and) a power supply module.

[0057] In some embodiments, the detection device 100 can be used to detect an object 190. The type of the object 190 is not particularly limited in the present invention. For example, the object 190 can be a human body or a conductor. Alternatively, the object 190 can also be an insulator. It should be noted that the object 190 is not part of the detection device 100.

[0058] The shape and type of the camera module 110 are not particularly limited in the present invention. The camera module 110 can be used to obtain an image IG of the object 190. In order to reduce power consumption, a frame rate RN of the camera module 110 can be lower than or equal to a predetermined value. For example, the predetermined value can be approximately equal to 10 Hz, but is not limited thereto.

[0059] The image processing module 120 is coupled to the camera module 110. In detail, the image processing module 120 can be used to receive and analyze the image IG to define a target sensing zone ZG and generate a radar setting value RS. After the image IG is analyzed, the image processing module 120 can determine an important part of the image IG, which can correspond to the target sensing zone ZG. In this way, in an image scene mode (Image Scene Mode) defined according to the content of the image IG, the aforementioned important part can refer to a representative part of the image IG, such as: a flower part of a flower pot, or a gesture part of an arm, but it is not limited to this. Then, the target sensing area ZG can be used to indicate a specific part of the object 190. Assuming that an upper part of the image IG is more important than the rest, the target sensing area ZG generated by the image processing module 120 will be able to correspondingly indicate an upper part of the object 190 (which can be located at Figure 1 For example, the relevant information of the target sensing area ZG can be transmitted to the radar module 130 in the form of a wired signal or a wireless signal. On the other hand, in some embodiments, the radar setting value RS defined by the image scene mode described in the present invention may include an antenna angle setting value and a pulse repetition time (PRT) setting value, but is not limited thereto.

[0060] The radar module 130 is coupled to the image processing module 120, wherein the radar module 130 can be controlled by the image processing module 120. In detail, the radar module 130 can selectively operate in a first analysis mode MD1 or a second analysis mode MD2. For example, the first analysis mode MD1 can provide a rougher detection result, but the power consumption of the radar module 130 is relatively low; on the contrary, the second analysis mode MD2 can provide a more accurate detection result, but the power consumption of the radar module 130 is relatively high. The radar module 130 can initially operate in the first analysis mode MD1. If a specific condition is met, the radar module 130 will leave the first analysis mode MD1 and enter the second analysis mode MD2. In the second analysis mode MD2, the radar module 130 can detect a specific part of the object 190 located in the target sensing area ZG according to the radar setting value RS, wherein the relevant information of the target sensing area ZG is provided to the radar module 130 by the image processing module 120.

[0061] In some embodiments, the camera module 110 may be in a closed or dormant state at the beginning to save power. In the first analysis mode MD1, if the radar module 130 detects that anything is approaching (not limited to the object 190), the radar module 130 will notify the camera module 110 so that the camera module 110 is activated. For example, if something falls into a detectable range of the radar module 130, the radar module 130 will determine that a proximity event has occurred, thereby activating the camera module 110. Alternatively, the radar module 130 may first notify the image processing module 120, and the image processing module 120 then activates the camera module 110, but it is not limited to this. In other embodiments, if the detection device 100 is picked up from the table by the user, the camera module 110 may also be activated or awakened.

[0062] In some embodiments, the radar module 130 continues to operate in the first analysis mode MD1 until receiving a notification from the image processing module 120. For example, after the image processing module 120 has finished analyzing the image IG, the target sensing area ZG has been defined and the radar setting value RS has been generated. At this time, the image processing module 120 can control the radar module 130 to leave the first analysis mode MD1 and enter the second analysis mode MD2, but is not limited thereto.

[0063] Under the design of the present invention, the proposed detection device 100 can first use the camera module 110 and the image processing module 120 to find the target sensing area ZG, and then use the radar module 130 to detect the specific part of the object 190 in the target sensing area ZG. It must be understood that the image processing module 120 usually consumes more power than the radar module 130. Therefore, the proposed detection device 100 uses the radar module 130 as a final detection tool, which can greatly reduce the overall power consumption of the detection device 100. On the other hand, because the target sensing area ZG has been defined by the image processing module 120, the design of the present invention also helps to reduce the probability of the radar module 130 making a target locking error.

[0064] The following embodiments will introduce various configurations and detailed structural features of the detection device 100. It should be understood that these drawings and descriptions are only for illustrative purposes and are not intended to limit the present invention.

[0065] Figure 21 is a pulse waveform diagram of the radar module 130 according to an embodiment of the present invention, wherein the horizontal axis represents time (s) and the vertical axis represents potential level (V). The radar module 130 can continuously generate multiple pulses, wherein an interval between the emission time points of two adjacent pulses can be regarded as a pulse repetition time TD of the radar module 130. In the first analysis mode MD1, the pulse repetition time TD of the radar module 130 can be greater than or equal to a critical value, and in addition, a distance resolution (Distance Resolution) and a sampling rate (Sampling Rate) of the radar module 130 can also be reduced. Conversely, in the second analysis mode MD2, the pulse repetition time TD of the radar module 130 can be less than the critical value, and the distance resolution and sampling rate of the radar module 130 are also higher than their corresponding values ​​in the first analysis mode MD1. For example, the aforementioned critical value can be approximately equal to 1 second or 2 seconds, but is not limited thereto.

[0066] Figure 3 FIG. 3 is a schematic diagram showing an object 390 according to an embodiment of the present invention. Figure 3 In the embodiment of the present invention, the object 390 is a human body, wherein a specific portion 395 of the object 390 is located within a dotted frame 392 indicated by the target sensing area ZG. For example, the specific portion 395 of the object 390 may be a hand of a human body (a carrier, or another user), but is not limited thereto. On the other hand, the portion outside the dotted frame 392 may be considered as a non-detection zone. In some embodiments, the radar device 130 does not perform any detection procedures for the aforementioned non-detection zone, thereby saving its computing resources.

[0067] In some embodiments for detecting hand gestures, the radar module 130 may be operated as follows. An intermediate frequency operating bandwidth of the radar module 130 may be 6 GHz, and a distance resolution that the radar module 130 can detect may be approximately 2.5 cm. Assuming that a center frequency of the radar module 130 is located at 60 GHz, a wavelength of the radar module 130 may be approximately equal to 5 mm. In addition, if the pulse repetition time TD of the radar module 130 is set to 0.5 ms, a maximum sensing speed of the radar module 130 may be 2.5 m / s. Based on a 64-point discrete Fourier sampling operation, a speed resolution of the radar module 130 may be approximately 0.08 m / s. If the number of sampling points for each chirp signal of the radar module 130 is 64, a maximum effective distance (Maximum Unambiguous Range) of the radar module 130 may be approximately 0.8 m, which may be approximately equal to an arm length of the user. In detail, the aforementioned operations and calculation methods can be described as follows:

[0068]

[0069]

[0070]

[0071]

[0072] Among them, "QA" represents a radar wave resolution of the radar module 130, "QB" represents the maximum sensing speed of the radar module 130, "QC" represents the speed resolution of the radar module 130, "QD" represents the maximum effective distance of the radar module 130, "c" represents the speed of light, "BW" represents the intermediate frequency operating bandwidth of the radar module 130, "WL" represents the wavelength of the radar module 130, "TD" represents the pulse repetition time TD of the radar module 130, and "SN" represents the number of sampling points of the radar module 130 (for example: 64).

[0073] Figure 4 FIG. 4 is a schematic diagram showing a detection device 400 according to an embodiment of the present invention. Figure 4 In the embodiment of FIG. 1 , the target sensing area ZG defined by the image processing module 120 corresponds to a rectangular area 497 in the coordinate space. That is, the radar module 130 can be used to detect a specific portion (not shown) of an object in the rectangular area 497. In addition, the length L1, width W1, height H1, X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the rectangular area 497 can all be regarded as relevant information of the target sensing area ZG, wherein all relevant information of the target sensing area ZG can be transmitted from the image processing module 120 to the radar module 130.

[0074] Figure 5 is a schematic diagram showing a radar module 530 according to an embodiment of the present invention. Figure 5In the embodiment of the present invention, the radar module 530 includes an antenna array 540 and a radar controller 560. In detail, the antenna array 540 includes a plurality of antenna elements 550-1, 550-2, ..., 550-N, wherein "N" is any positive integer greater than or equal to 2. For example, the aforementioned antenna elements 550-1, 550-2, ..., 550-N can all be arranged on the same radar plane 554. The radar plane 554 has a normal line 555, and a connection line 556 is provided between the radar plane 554 and an object 590, wherein the aforementioned normal line 555 and connection line 556 can be used to define a tilt angle θ of the antenna array 540. In addition, the radar controller 560 can also control and adjust the deflection angle θ of the antenna array 540 according to the radar setting value RS, so that a main beam (Main Beam) of a radiation pattern (Radiation Pattern) of the antenna array 540 can be directed exactly toward a specific portion of the object 590 corresponding to the target sensing area ZG. Furthermore, the radar controller 560 can also selectively activate or deactivate at least one of the aforementioned antenna elements 550-1, 550-2, ..., 550-N in the antenna array 540 according to the radar setting value RS, so as to obtain a better detection effect. It must be understood that although not shown in Figure 5 However, the radar controller 560 may further include other elements, such as: a pulse generator (Pulse Generator), a signal feeding element (Signal Feeding Element), or (and) a phase shifter (Phase Shifter).

[0075] Figure 6 is a flow chart showing the detection method described in accordance with an embodiment of the present invention. First, in step S610, a radar module is operated in a first analysis mode. In step S620, an image of an object is acquired through a camera module. In step S630, the image is analyzed through an image processing module to define a target sensing area and generate a radar setting value. In some embodiments, if the image processing module or the radar module determines that the aforementioned target sensing area and the radar setting value are not yet perfect, the program may also return to step 620 to reacquire the image of the object for subsequent analysis. In step S640, the radar module is operated in a second analysis mode. Finally, in step S650, a specific part of the object located in the target sensing area is detected through the radar module according to the radar setting value. It must be understood that the above steps do not need to be performed in sequence, but Figure 1-5 Each feature of the embodiment of the present invention can be applied to Figure 6 of detection methods.

[0076] The present invention provides a novel detection device and detection method. Compared with the conventional design, the present invention has at least the advantages of improving detection accuracy and reducing overall power consumption, so it is very suitable for application in various devices.

[0077] It is worth noting that the above-mentioned component parameters are not limiting conditions of the present invention. Designers can adjust these setting values ​​according to different needs. The detection device and detection method of the present invention are not limited to Figure 1-6 The present invention may only include Figure 1-6 In other words, not all the features shown in the figures need to be implemented in the detection device and the detection method of the present invention at the same time.

[0078] The method of the present invention, or a specific form or part thereof, may exist in the form of program code. The program code may be contained in a physical medium, such as a floppy disk, a CD, a hard disk, or any other machine-readable (such as computer-readable) storage medium, or a computer program product that is not limited to an external form, wherein when the program code is loaded and executed by a machine, such as a computer, the machine becomes a device for participating in the present invention. The program code may also be transmitted through some transmission medium, such as wires or cables, optical fibers, or any transmission mode, wherein when the program code is received, loaded and executed by a machine, such as a computer, the machine becomes a device for participating in the present invention. When actually operated on a general-purpose processing unit, the program code combines with the processing unit to provide a unique device that operates similarly to an application-specific logic circuit.

[0079] In the present specification and claims, ordinal numbers, such as "first", "second", "third", etc., have no sequential relationship with each other, and are only used to distinguish two different components with the same name.

[0080] Although the present invention is disclosed as above with preferred embodiments, it is not intended to limit the scope of the present invention. Anyone skilled in the art can make some improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A detection device for detecting an object, comprising: a camera module to obtain an image of the object; an image processing module that analyzes the image to define a target sensing area and generate a radar setting value; and a radar module controlled by the image processing module, wherein the radar module selectively operates in a first analysis mode or a second analysis mode; The radar module is initially operated in the first analysis mode; In the second analysis mode, the radar module detects a specific portion of the object located in the target sensing area according to the radar setting value.

2. The detection device as claimed in claim 1, wherein the detection device is implemented by a head mounted display.

3. The detection device as claimed in claim 1, wherein the object is a human body, and the specific part of the object is a hand of the human body.

4. The detection device as claimed in claim 1, wherein in the first analysis mode, if the radar module detects that anything is approaching, the radar module will notify the camera module so that the camera module is activated. 5 . The detection device as claimed in claim 1 , wherein a frame rate of the camera module is lower than or equal to a predetermined value. 6 . The detection device as claimed in claim 1 , wherein in the first analysis mode, a pulse repetition time of the radar module is greater than or equal to a critical value. 7 . The detection device as claimed in claim 6 , wherein in the second analysis mode, the pulse repetition time of the radar module is less than the critical value.

8. The detection device as claimed in claim 1, wherein after the image processing module finishes analyzing the image, the image processing module controls the radar module to leave the first analysis mode and enter the second analysis mode.

9. The detection device as claimed in claim 1, wherein the target sensing area corresponds to a rectangular parallelepiped area.

10. The detection device as claimed in claim 1, wherein the radar setting value comprises an antenna angle setting value and a pulse repetition time setting value.

11. The detection device as claimed in claim 1, wherein the radar module comprises: an antenna array including a plurality of antenna elements; as well as A radar controller controls a deflection angle of the antenna array according to the radar setting value.

12. A detection method comprising the following steps: operating a radar module in a first analysis mode; Acquire an image of an object through a camera module; Analyzing the image through an image processing module to define a target sensing area and generate a radar setting value; operating the radar module in a second analysis mode; and A specific portion of the object located in the target sensing area is detected by the radar module according to the radar setting value. 13 . The detection method as claimed in claim 12 , wherein the object is a human body, and the specific part of the object is a hand of the human body.

14. The detection method of claim 12, further comprising: In the first analysis mode, if anything is detected approaching, the radar module will notify the camera module so that the camera module is activated. 15 . The detection method as claimed in claim 12 , wherein a frame rate of the camera module is lower than or equal to a predetermined value. 16 . The detection method as claimed in claim 12 , wherein in the first analysis mode, a pulse repetition time of the radar module is greater than or equal to a critical value. 17 . The detection method as claimed in claim 16 , wherein in the second analysis mode, the pulse repetition time of the radar module is less than the critical value.

18. The detection method of claim 12, further comprising: After the image is analyzed, the radar module is controlled by the image processing module to leave the first analysis mode and enter the second analysis mode.

19. The detection method as claimed in claim 12, wherein the target sensing area corresponds to a rectangular parallelepiped area.

20. The detection method of claim 12, wherein the radar setting value comprises an antenna angle setting value and a pulse repetition time setting value.

Images