A target tracking method and apparatus

Abstract

The present application provides a target tracking method and device, the target tracking device includes radar unit and camera unit. The method first acquires the first position information of at least one dynamic target being human body through the radar unit, then acquires the display partition image including the dynamic target through the camera unit according to the first position information, and judges whether the initial position of the face of the dynamic target in the display partition image is located at the center position of the display partition image. When it is judged that the initial position is not located at the center position of the display partition image, the compensation information of the radar unit is acquired according to the initial position and the center position of the display partition image, so as to compensate the radar unit, and the radar unit and the camera unit are controlled to perform face tracking on the dynamic target. The present application can achieve the effects of person tracking and face tracking.

Description

Technical Field

[0001] This invention relates to image processing technology, and more particularly to a target tracking method and apparatus for person tracking and face centering. Background Technology

[0002] In existing target tracking systems, directional microphones are typically used to detect the location of speakers in a meeting. However, directional microphones are often affected by echo and distance, and can only be used in small meetings. When multiple people participate in the meeting, the actual screen may not be able to focus precisely on the speaker, making it impossible for other participants to see the speaker's gestures and expressions clearly in real time, thus reducing the interactivity and experience of the video conference. Summary of the Invention

[0003] In view of this, the present invention provides a target tracking system that first uses the reflected waves of a radar unit to detect dynamic targets, such as speakers in a video conference, and then uses the distance between the radar unit and the camera unit to calculate the position of the speaker, adjusts the size of the speaker on the display screen and places the speaker in the center of the display screen, thereby achieving the effect of person tracking and face centering.

[0004] This invention provides a target tracking method executed in a target tracking device, which includes a radar unit and a camera unit. The method includes the following steps: acquiring first position information of at least one dynamic target through the radar unit; acquiring a display partition image including the dynamic target through the camera unit based on the first position information; determining whether the dynamic target is a human body based on the display partition image; when the dynamic target is determined to be a human body, further acquiring the initial position of the dynamic target's face in the display partition image; determining whether the initial position is located at the center of the display partition image; when the initial position is determined not to be located at the center of the display partition image, acquiring compensation information of the radar unit based on the initial position and the center position of the display partition image; and controlling the radar unit and the camera unit to perform face tracking on the dynamic target based on the compensation information.

[0005] This invention also provides a target tracking device, comprising: a radar unit, a camera unit, a processor, and a memory. The memory stores at least one computer program, wherein the computer program includes instructions executed by the processor, causing the processor to perform the following steps: acquiring first position information of at least one dynamic target through the radar unit; acquiring a display partition image including the dynamic target through the camera unit based on the first position information; determining whether the dynamic target is a human body based on the display partition image; when the dynamic target is determined to be a human body, further acquiring the initial position of the dynamic target's face in the display partition image; determining whether the initial position is located at the center of the display partition image; when the initial position is determined not to be located at the center of the display partition image, acquiring compensation information for the radar unit based on the initial position and the center position of the display partition image; and controlling the radar unit and the camera unit to perform face tracking on the dynamic target based on the compensation information.

[0006] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the present invention. Attached Figure Description

[0007] Figure 1 This diagram shows a flowchart of the target tracking method described in an embodiment of the present invention.

[0008] Figure 2 This diagram illustrates the operation of the target tracking device described in an embodiment of the present invention.

[0009] Figures 3A-3C This diagram illustrates the target tracking method described in an embodiment of the present invention.

[0010] Figure 4 This diagram shows a flowchart of the target tracking method for obtaining compensation information according to an embodiment of the present invention.

[0011] Figure 5 This diagram illustrates the calculation of compensation information in the target tracking method according to an embodiment of the present invention.

[0012] Figure 6 This shows a block diagram of the target tracking device described in an embodiment of the present invention.

[0013] Explanation of main component symbols

[0014] Process Steps S101~S108, S401~S406 Target tracking device 200、600 radar unit 201、601 Camera unit 202、602 Dynamic goals 210 Field of view partitioning image 301 Display partition image 302 Center position of the field of view partition image 303 First location information 304 initial position 305 processor 603 memory 604

[0015] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0016] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0017] Figure 1 This diagram shows a flowchart of the target tracking method according to an embodiment of the present invention. The method can be executed in a target tracking device, which includes a radar unit and a camera unit. In this embodiment, the radar unit is electrically connected to the camera unit and integrated into the target tracking device. In other embodiments, the radar unit or camera unit can be a separate device independent of the target tracking device. The radar unit can communicate with the camera unit via various communication links, such as wired or wireless communication links or fiber optic cables. The target tracking device can be any electronic device with camera and radar detection capabilities, such as a video conferencing device or a monitoring device.

[0018] Step S101: Detect dynamic targets through the radar unit and obtain the first position information of at least one dynamic target.

[0019] First, the radar unit is used to detect dynamic targets in the environment. When multiple dynamic targets are detected, the position information of the dynamic target with the largest movement is used as the first position information.

[0020] Step S102: Based on the first position information, acquire a display partition image including the dynamic target through the camera unit.

[0021] The focal length of the camera unit is adjusted based on the first position information to obtain a display zone image that includes dynamic targets and is sufficiently clear.

[0022] Step S103: Determine whether the dynamic target is a human body based on the displayed partition image.

[0023] In order to eliminate non-human dynamic targets in the environment, image analysis technology is used to determine whether the dynamic target captured by the camera unit is a human body. If it is determined that the dynamic target is not a human body, return to step S101 and use the radar unit to continuously detect dynamic targets until the detected dynamic target is a human body. If it is determined that the dynamic target is a human body, execute step S104.

[0024] Step S104: Obtain the face of the dynamic target at the initial position of the display partition image.

[0025] The facial recognition technology is used to determine the position of the dynamic target's face, and the position of the dynamic target's face is set as the initial position.

[0026] Step S105: Determine whether the initial position is located at the center of the displayed partition image.

[0027] If it is determined that the initial position is not located at the center of the display area, step S106 is executed; if it is determined that the initial position is located at the center of the display area, step S108 is executed.

[0028] Step S106: Obtain the compensation information of the radar unit based on the initial position and the center position of the displayed partition image.

[0029] Step S107: Based on the compensation information, control the radar unit and camera unit to perform face tracking on the dynamic target.

[0030] Step S108: Control the radar unit and camera unit to perform face tracking on dynamic targets.

[0031] Figure 2 This diagram illustrates the operation of the target tracking device according to an embodiment of the present invention. Figure 2 As shown, the target tracking device 200 includes: a radar unit 201 and a camera unit 202.

[0032] The distance between radar unit 201 and camera unit 202 is x, and the direction directly in front of radar unit 201 and camera unit 202 is used as the reference direction for angles. The radar unit 201 detects a moving target 210 and obtains first position information of the moving target 210, which includes the distance d1 of the moving target 210 relative to the radar unit and its azimuth relative to the direction directly in front of the radar unit 201 θ1. The horizontal distance x1 and vertical distance y1 between the moving target 210 and radar unit 201 can be calculated based on d1 and θ1. The horizontal distance between camera unit 202 and moving target 210 is x2, where x2 = x1 - x, and the vertical distance is y2. Since y1 = y2, the distance d2 and azimuth relationship θ2 between the moving target 210 and camera unit 202 can be calculated based on x2 and y2. In this embodiment, the focal length of camera unit 202 is adjusted according to the distance d2 and azimuth relationship θ2 between the moving target 210 and camera unit 202.

[0033] Taking a distance of x = 0.1 meters between radar unit 201 and camera unit 202 as an example, at this time, radar unit 201 measures the position information d1 = 4.2 meters and θ1 = 30° relative to the dynamic target 210.

[0034] First, calculate the horizontal distance x1 between radar unit 201 and moving target 210: x1 = d1 * sinθ1 = 4.2 * sin30° = 2.1 meters.

[0035] Next, calculate the vertical distance y1 between radar unit 201 and moving target 210: y1 = d1 * cosθ1 = 4.2 * cos30° = 3.637 meters.

[0036] The vertical distance y2 between camera unit 202 and moving target 210 is equal to the vertical distance y1 between radar unit 201 and moving target 210, y2 = y1 = 3.64 meters.

[0037] The horizontal distance x2 between camera unit 202 and moving target 210 is equal to the horizontal distance x1 between radar unit 201 and moving target 210 minus the interval x between radar unit 201 and camera unit, x2 = x1 - x = 2.1 - 0.1 = 2 meters.

[0038] Based on the vertical distance y2 and horizontal distance x2 between the camera unit 202 and the dynamic target 210, the distance d2 and the azimuth relationship θ2 between the dynamic target 210 and the camera unit 202 can be calculated. tanθ2=x2 / y2=2 / 3.64=0.55, therefore, θ2=28.81°, d2=y2 / cosθ2=3.64 / cos28.81°=4.19 meters. Therefore, based on the first position information of the dynamic target, the focal length of the camera unit 202 is set at a distance of 4.19 meters and an azimuth relationship of 28.81° relative to the camera unit 202.

[0039] Figures 3A-3C This diagram illustrates the target tracking method described in an embodiment of the present invention.

[0040] The image captured by the camera unit includes a field-of-view image 301 and a display image 302. The field-of-view image 301 represents the imaging range of the camera unit's photosensitive component, while the display image 302 is the image output by the camera unit to other devices. In this embodiment, the other devices include a display device. For example, the resolution of the field-of-view image 301 of a commonly used 12-megapixel camera is 4000*3000p, while the resolution of the display image 302 is typically 1920*1080p. As can be seen from the above data, the resolution of the field-of-view image 301 is much greater than that of the display image 302. Therefore, by adjusting the position of the display image 302 within the field-of-view image 301, the dynamic target can be centered within the display image 302. This method allows for centering the dynamic target within the display image without moving the lens to align with or track the target.

[0041] like Figure 3A As shown, in step S101, the display partition image 302 is centered at the center position 303 of the field of view partition image.

[0042] In step S102, the first position information 304 of the moving target 210 is obtained through the radar unit 201. Therefore, as Figure 3BAs shown, the center position of the display partition image 302 is adjusted from the original center position 303 of the field of view partition image to the first position information 304 according to the first position information 304.

[0043] In step S103, the camera unit 202 determines whether the dynamic target 210 in the display partition image 302 is a human body.

[0044] If the dynamic target 210 is a human body, the face position of the dynamic target 210 is obtained as the initial position 305. When the initial position 305 is inconsistent with the first position information 304, that is, when the initial position 305 is not at the center of the display partition image 302, the horizontal offset distance Δx and the vertical offset distance Δy between the initial position 305 and the first position information 304 are obtained. In this embodiment, the horizontal offset distance Δx and the vertical offset distance Δy are used as compensation information for the radar unit 201, and the center position of the display image area 302 is adjusted according to the compensation information. Figure 3C As shown, the initial position 305 is taken as the center of the image display area 302, and the radar unit 201 and the camera unit 202 are controlled to perform face tracking on the dynamic target 210.

[0045] Figure 4 This diagram shows a flowchart of the target tracking method for obtaining compensation information according to an embodiment of the present invention.

[0046] Step S401: Obtain the horizontal field of view (HFOV) and vertical field of view (VFOV) of the camera unit.

[0047] Step S402: Obtain the default resolution of the field of view area.

[0048] Step S403: Obtain the first position information of the dynamic target through the radar unit, and obtain the distance d2 and azimuth relationship θ2 between the camera unit and the dynamic target based on the first position information.

[0049] Step S404: Calculate the field of view range size based on the distance d2, orientation relationship θ2, HFOV, and VFOV. Obtain the size of each horizontal pixel and each vertical pixel in the field of view range based on the field of view range size and the default resolution of the field of view range.

[0050] Step S405: When the face of the dynamic target is not located at the center of the display area, obtain the horizontal offset and vertical offset of the face of the dynamic target relative to the center of the display area, obtain the horizontal offset distance based on the horizontal pixel size and the horizontal offset, and obtain the vertical offset distance based on the vertical pixel size and the vertical offset.

[0051] Step S406: Feedback the horizontal offset distance Δx and the vertical offset distance Δy to the radar unit as compensation information for the radar unit.

[0052] In this embodiment, the accuracy of radar unit detection is improved through continuous detection and constant compensation iteration.

[0053] Figure 5 This section illustrates an example of compensation information calculation in the target tracking method described in the embodiments of the present invention.

[0054] by Figure 5 For example, the horizontal and vertical field of view of the camera unit are known. In this example, the HFOV is 104°, ∠ABC is 1 / 2HFOV = 52°, VFOV is 96°, ∠ABD is 1 / 2VFOV = 48°, and the field of view partition image 301 in the display device has a resolution of 12 megapixels (4000*3000). Among them, the image transmitted through the radar unit ( Figure 5 (Not displayed) After obtaining the first position information 304 of the dynamic target, it can be known that the distance d2 between the camera unit 202 and the dynamic target is 4.19 meters and the azimuth relationship θ2 is 28.81°. It can be calculated that the vertical distance from the camera unit 202 to the dynamic target is line segment AB, line segment AB = d2 * cosθ2 = 4.19 meters * cos28.81° = 3.64 meters.

[0055] In this example, ∠ABC is 1 / 2HFOV = 52°, and line segment AB is 3.64 meters. Therefore, the length of line segment AC can be calculated from triangle ABC: line segment AC = line segment AB * tan52° = 3.64 meters * 1.28 = 4.66 meters = 2000 pixels. Thus, each pixel in the x-direction is 0.002 meters. Through image processing technology, the horizontal offset distance △x in the displayed image partition can be obtained. In this example, △x is 100 pixels, so we can know that △x in this example is 100 pixels = 0.2 meters.

[0056] Similarly, ∠ABD is 1 / 2VFOV = 48°, and line segment AB is 3.64 meters. Therefore, the length of line segment AD can be calculated using triangle ABD: line segment AD = line segment AB * tan48° = 3.64 meters * 1.11 = 4.04 meters = 1500 pixels. Thus, each pixel in the y-direction is 0.003 meters. Through image processing technology, the vertical offset distance △y in the displayed image partition can be obtained. In this example, △y is 20 pixels, so △y in this example is 20 pixels = 20 * 0.003 = 0.06 meters.

[0057] Figure 6 This shows a block diagram of the target tracking device according to an embodiment of the present invention. Figure 6 As shown, the target tracking device 600 includes: a radar unit 601, a camera unit 602, a processor 603, and a memory 604.

[0058] Radar unit 601 is used to detect moving targets, and camera unit 602 is used to acquire display partition images including the moving targets. Memory 604 (computer) is used to store at least one computer program, which includes instructions executed by processor 603, causing processor 603 to perform... Figures 1 to 5 Examples of implementations.

[0059] This invention can quickly detect moving targets using a radar unit and capture images of the moving targets using a camera unit when using target tracking. The position of the face in the image is used as compensation for the radar unit, thereby improving the accuracy of the radar unit. The image is centered by adjusting the display area of ​​the camera unit, which can eliminate the need to move the lens to align with or track moving targets. This technology is also applicable to fixed camera lenses.

[0060] In summary, this invention meets the requirements for an invention patent, and therefore a patent application is filed in accordance with the law. However, the above description is only a preferred embodiment of the invention, and the scope of the invention is not limited to the described embodiments. All equivalent modifications or variations made by those skilled in the art in accordance with the spirit of the invention should be covered within the scope of the following patent application.

Claims

1. A target tracking method, executed in a target tracking device, characterized in that, The target tracking device includes a radar unit and a camera unit, and the method includes the following steps: The radar unit described above acquires the first position information of at least one dynamic target. Based on the first position information, a display partition image including the dynamic target is acquired through the camera unit; Based on the displayed partitioned image, determine whether the dynamic target is a human body; When the dynamic target is determined to be a human body, the initial position of the face of the dynamic target in the display partition image is further obtained. Determine whether the initial position is located at the center of the displayed partition image; When it is determined that the initial position is not located at the center of the display partition image, compensation information for the radar unit is obtained based on the initial position and the center position of the display partition image. The process of obtaining the compensation information for the radar unit further includes: Obtain the horizontal and vertical viewing range of the aforementioned camera unit; Obtain the default resolution of the field of view of the aforementioned camera unit; Based on the first position information mentioned above, the distance d2 and the orientation relationship θ2 between the camera unit and the dynamic target are obtained; The size of the field of view area is calculated based on the distance d2, the orientation relationship θ2, the horizontal visible range, and the vertical visible range. The size of each horizontal pixel and each vertical pixel in the field of view area is obtained based on the size of the field of view area and the default resolution of the field of view area. The horizontal and vertical offsets of the face of the dynamic target relative to the center position of the display partition image are obtained. The horizontal offset distance is obtained based on the horizontal pixel size and the horizontal offset, and the vertical offset distance is obtained based on the vertical pixel size and the vertical offset. The aforementioned horizontal offset distance and the aforementioned vertical offset distance are used as the aforementioned compensation information; and Based on the aforementioned compensation information, the aforementioned radar unit and camera unit are controlled to perform face tracking on the aforementioned dynamic target.

2. The target tracking method as described in claim 1, characterized in that, The step of acquiring a display partition image including the dynamic target through the camera unit based on the first position information further includes: Acquire the field-of-view partition images of the aforementioned camera unit; and The display partition image is obtained from the above-mentioned field of view partition image based on the first position information mentioned above.

3. The target tracking method as described in claim 2, characterized in that, The step of obtaining the display partition image from the field of view partition image based on the first position information further includes: Obtain the distance between the aforementioned camera unit and the aforementioned radar unit; and The above-mentioned display partition image is obtained based on the above-mentioned interval distance and the above-mentioned first position information.

4. The target tracking method as described in claim 1, characterized in that, The method also includes the following steps: Adjust the display partition image according to the horizontal offset distance and the vertical offset distance, so that the initial position is located at the center of the display partition image.

5. A target tracking device, characterized in that, The target tracking device includes: Radar unit; Camera unit; Processor; and A memory for storing at least one computer program, wherein the computer program includes instructions executable by the processor, causing the processor to perform the following steps: The radar unit described above acquires the first position information of at least one moving target. Based on the first position information, a display partition image including the dynamic target is acquired through the camera unit; Based on the displayed partitioned image, determine whether the dynamic target is a human body; When the dynamic target is determined to be a human body, the initial position of the face of the dynamic target in the display partition image is further obtained. Determine whether the initial position is located at the center of the displayed partition image; When it is determined that the initial position is not located at the center of the display partition image, compensation information for the radar unit is obtained based on the initial position and the center position of the display partition image. The process of obtaining the compensation information for the radar unit further includes: Obtain the horizontal and vertical viewing range of the aforementioned camera unit; Obtain the default resolution of the field of view of the aforementioned camera unit; Based on the first position information mentioned above, the distance d2 and the orientation relationship θ2 between the camera unit and the dynamic target are obtained; The size of the field of view area is calculated based on the distance d2, the orientation relationship θ2, the horizontal visible range, and the vertical visible range. The size of each horizontal pixel and each vertical pixel in the field of view area is obtained based on the size of the field of view area and the default resolution of the field of view area. The horizontal and vertical offsets of the face of the dynamic target relative to the center position of the display partition image are obtained. The horizontal offset distance is obtained based on the horizontal pixel size and the horizontal offset, and the vertical offset distance is obtained based on the vertical pixel size and the vertical offset. The aforementioned horizontal offset distance and the aforementioned vertical offset distance are used as the aforementioned compensation information; and Based on the aforementioned compensation information, the aforementioned radar unit and camera unit are controlled to perform face tracking on the aforementioned dynamic target.

6. The target tracking device as described in claim 5, characterized in that, The step of acquiring a display partition image including the dynamic target through the camera unit based on the first position information further includes: Acquire the field-of-view partition images of the aforementioned camera unit; and The display partition image is obtained from the above-mentioned field of view partition image based on the first position information mentioned above.

7. The target tracking device as described in claim 6, characterized in that, The step of obtaining the display partition image from the field of view partition image based on the first position information further includes: Obtain the distance between the aforementioned camera unit and the aforementioned radar unit; and The above-mentioned display partition image is obtained based on the above-mentioned interval distance and the above-mentioned first position information.

8. The target tracking device as described in claim 5, characterized in that, The processor also performs the following steps: Adjust the display partition image according to the horizontal offset distance and the vertical offset distance, so that the initial position is located at the center of the display partition image.

Images