Method, system, device and equipment for controlling camera to track moving object

Abstract

The invention provides a method, a system, a device and equipment for controlling a camera to track a moving object. The method comprises the following steps: determining a tth first space coordinate of the moving object based on ultrasonic waves and a microphone array; determining a tth direction angle and a tth pitching angle of the camera based on the tth first space coordinate; the tth second space coordinate is the coordinate of the camera; the tth second space coordinate is a coordinate in a preset coordinate system; the tth second connecting line is connected with the tth first space coordinate and the tth third space coordinate; and controlling the camera to track the moving object based on the tth direction angle and the tth pitching angle. According to the technical scheme, the moving object at any position of the camera can be monitored and tracked.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a method, system, device and equipment for controlling a camera to track a moving object. Background Art

[0002] At present, due to the growing demand for home security, commercial monitoring and intelligent management, cameras have been widely used in various places. And with the popularization of wireless technologies such as Bluetooth, cameras can be easily connected to the Internet to achieve remote monitoring and data transmission. Therefore, users can check the situation of the monitored area anytime and anywhere, which greatly facilitates users' lives.

[0003] However, in some scenarios, some objects are moving, so the camera is not only required to monitor static objects, but also to track moving objects. However, the current camera tracking range is limited, and generally can only track moving objects below the camera or objects above the moving camera, which greatly reduces the application range of the camera. Summary of the invention

[0004] The embodiments of the present application provide a method, system, device and apparatus for controlling a camera to track a moving object, which can monitor and track a moving object at any position of the camera, thereby increasing the application scope of the camera. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present application, a method for controlling a camera to track a moving object is provided, the method comprising:

[0006] Determine the t-th first spatial coordinate of the moving object based on ultrasound and the microphone array; the first spatial coordinate is a coordinate in a preset coordinate system; t is an integer greater than or equal to 2; the microphone array includes a position microphone; the position microphone is used to determine the positional relationship between the camera and the moving object;

[0007] Based on the tth first spatial coordinate, determine the tth direction angle and the tth elevation angle of the camera; the tth elevation angle is the angle between the tth first connecting line and the tth second connecting line; the tth direction angle is the angle between the tth second connecting line and the x-axis of the preset coordinate system;

[0008] Based on the t-th direction angle and the t-th elevation angle, control the camera to track the moving object.

[0009] In a possible implementation, the method further includes:

[0010] generating ultrasonic waves and playing the ultrasonic waves;

[0011] Receive a valid echo; the valid echo is the ultrasonic wave reflected by the moving object.

[0012] In a possible implementation, the method further includes:

[0013] Acquire an initial digital signal corresponding to the effective echo based on the microphone array;

[0014] The initial digital signal is filtered to obtain a target digital signal.

[0015] In a possible implementation, the tth first line connects the tth first space coordinate and the tth second space coordinate; the tth second space coordinate is the coordinate of the camera; the tth second space coordinate is the coordinate within the preset coordinate system; the tth second line connects the tth first space coordinate and the tth third space coordinate; the tth third space coordinate indicates the foot of the tth first space coordinate on the second plane.

[0016] In a possible implementation, the microphone array further includes a non-positional microphone; the non-positional microphone further includes a first microphone, a second microphone, and a third microphone;

[0017] The coordinates of the first microphone are fourth spatial coordinates; the fourth spatial coordinates are the same as the second spatial coordinates;

[0018] The second microphone is arranged on the x-axis;

[0019] The third microphone is arranged on the y-axis of the preset coordinate system;

[0020] The position microphone is arranged on the z-axis of the preset coordinate system and is located above the camera; the position microphone is arranged on a first plane; the camera and the non-position microphone are arranged on a second plane; the first plane and the second plane are planes within the preset coordinate system.

[0021] In a possible implementation manner, determining the t-th first spatial coordinate of the moving object based on the ultrasonic wave includes:

[0022] Obtaining a play time and a receiving time, and calculating the length of the t-th first connecting line based on the play time, the receiving time and the wave speed; the play time is the play time of the ultrasonic wave; the receiving time is the time of receiving the effective echo; the wave speed is the propagation speed of the ultrasonic wave in the air;

[0023] Based on the length of the tth first connecting line, the length of the tth third connecting line, the length of the tth fourth connecting line and the length of the tth fifth connecting line are calculated; the tth third connecting line connects the tth first spatial coordinate and the tth fifth spatial coordinate; the tth fifth spatial coordinate is the coordinate of the second microphone in the preset coordinate system; the tth fourth connecting line connects the tth first spatial coordinate and the tth sixth spatial coordinate; the tth sixth spatial coordinate is the coordinate of the third microphone in the preset coordinate system; the tth fifth connecting line connects the tth first spatial coordinate and the tth seventh spatial coordinate; the seventh spatial coordinate is the coordinate of the position microphone in the preset coordinate system;

[0024] Based on the length of the t-th first connecting line, the length of the t-th third connecting line, the length of the t-th fourth connecting line and the length of the t-th fifth connecting line, the t-th first angle, the t-th second angle and the t-th third angle are calculated; the t-th first angle is the angle between the t-th third connecting line and the t-th first connecting line; the t-th second angle is the angle between the t-th fourth connecting line and the t-th first connecting line; the t-th third angle is the angle between the t-th first connecting line and the z-axis;

[0025] Based on the tth first angle, the tth second angle and the tth third angle, the tth first space coordinate is calculated.

[0026] In a possible implementation, the calculating the tth first angle, the tth second angle, and the tth third angle based on the length of the tth first line, the length of the tth third line, the length of the tth fourth line, and the length of the tth fifth line includes:

[0027] The tth first angle is calculated based on the length of the tth first connecting line, the length of the tth third connecting line, and a first distance; the first distance is the distance between the first microphone and the second microphone;

[0028] The t-th second angle is calculated based on the length of the t-th first connecting line, the length of the t-th fourth connecting line and a second distance; the second distance is the distance between the first microphone and the third microphone;

[0029] The tth third angle is calculated based on the length of the tth first connecting line, the length of the tth fifth connecting line and a third distance; the third distance is the distance between the first microphone and the position microphone.

[0030] In one possible implementation, when the third angle is an acute angle, the positional relationship indicates that the moving object is above the camera; or when the third angle is a right angle, the positional relationship indicates that the moving object and the camera are in the same horizontal plane; or when the third angle is an obtuse angle, the positional relationship indicates that the moving object is below the camera.

[0031] In a possible implementation manner, calculating the tth first space coordinate based on the tth first angle, the tth second angle, and the tth third angle includes:

[0032] Based on the tth first angle and the first connecting line, a tth x value is obtained; the tth x value is the distance from the tth first spatial coordinate to the yoz plane; the yoz plane is the plane in the preset coordinate system;

[0033] Based on the tth second angle and the first connecting line, a tth y value is obtained; the tth y value is the distance from the tth first spatial coordinate to the xoz plane; the xoz plane is the plane in the preset coordinate system;

[0034] Based on the tth third angle and the first connecting line, the tth z value is obtained; the tth z value is the distance from the tth first spatial coordinate to the xoy plane; the xoy plane is the plane in the preset coordinate system.

[0035] In a possible implementation manner, determining the tth direction angle and the tth elevation angle of the camera based on the tth first space coordinate includes:

[0036] Calculate the tth pitch angle based on the length of the tth first connecting line and the tth z value;

[0037] Based on the tth y value and the tth second connecting line, the tth direction angle is calculated.

[0038] In a possible implementation manner, controlling the camera to track the moving object based on the t-th direction angle and the t-th elevation angle includes:

[0039] Obtaining the t-1th first difference and the t-1th second difference; the t-1th first difference is the difference between the tth direction angle and the t-1th direction angle; the t-1th second difference is the difference between the tth elevation angle and the t-1th elevation angle;

[0040] According to the t-1th first difference and the t-1th second difference, the camera is controlled to rotate to the tth target state; the target state indicates that the moving object falls into the shooting area of ​​the camera, and the center of the shooting area is in a straight line with the center of the moving object.

[0041] In a possible implementation, controlling the camera to rotate to the t-th target state according to the t-1th first difference and the t-1th second difference includes:

[0042] If the tth first difference is greater than or equal to a preset threshold, and the tth second difference is greater than or equal to the preset threshold, then based on the tth first difference and the tth second difference, the camera is controlled to rotate to the tth target state; the tth first difference is the difference between the t+1th direction angle and the tth direction angle; the tth second difference is the difference between the t+1th pitch angle and the tth pitch angle; or

[0043] If the t-th first difference is greater than or equal to the preset threshold, and the t-th second difference is less than the preset threshold, then controlling the camera to rotate to the t-th target state based on the t-th first difference; or

[0044] If the t-th first difference is less than the preset threshold, and the t-th second difference is greater than or equal to the preset threshold, controlling the camera to rotate to the t-th target state based on the t-th second difference; or

[0045] If the t-th first difference value is smaller than the preset threshold value, and the t-th second difference value is smaller than the preset threshold value, the camera is not rotated.

[0046] In a possible implementation, the method further includes:

[0047] If a plurality of the moving objects enter the surrounding area of ​​the camera at the same time, obtaining the t-th direction angle and the t-th elevation angle of each of the moving objects;

[0048] Obtaining the t-th first difference value and the t-th second difference value of each of the moving objects;

[0049] According to the tth first difference value and the tth second difference value of the kth moving object, control the camera to rotate and photograph the kth moving object for a preset time period; k is an integer greater than or equal to 1;

[0050] According to the tth first difference and the tth second difference of the k+1th moving object, the camera is controlled to rotate and photograph the k+1th moving object for the preset time period.

[0051] According to a second aspect of an embodiment of the present application, a system for controlling a camera to track a moving object is provided, comprising:

[0052] Includes a camera, a sound wave generator, and a microphone array;

[0053] The sound wave generator and the microphone array are used to determine the spatial coordinates of the moving object; the spatial coordinates are coordinates of a preset coordinate system;

[0054] The microphone array includes a position microphone; the position microphone is used to determine the position relationship between the camera and the moving object.

[0055] In a possible implementation, the microphone array also includes a non-positional microphone; the positional microphone is arranged on a first plane; the camera and the non-positional microphone are arranged on a second plane; and the first plane and the second plane are planes within the preset coordinate system.

[0056] According to a third aspect of an embodiment of the present application, a device for controlling a camera to track a moving object is provided, comprising:

[0057] A first determination module is used to determine the t-th first spatial coordinate of the mobile object based on ultrasound and a microphone array; the first spatial coordinate is a coordinate in a preset coordinate system; t is an integer greater than or equal to 2; the microphone array includes a position microphone; the position microphone is used to determine the positional relationship between the camera and the mobile object;

[0058] A second determination module is used to determine the tth direction angle and the tth elevation angle of the camera based on the tth first spatial coordinate; the tth elevation angle is the angle between the tth first connecting line and the tth second connecting line; the tth direction angle is the angle between the tth second connecting line and the x-axis of the preset coordinate system;

[0059] A control module is used to control the camera to track the moving object based on the tth direction angle and the tth elevation angle.

[0060] According to a fourth aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executed to control a camera to track a moving object.

[0061] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement a method of controlling a camera to track a moving object.

[0062] In an embodiment of the present application, a method for controlling a camera to track a moving object is provided. The moving object in the area around the camera is detected by ultrasonic waves, and the direction angle and pitch angle of the moving object are determined to avoid the blind spot problem of the camera. The positional relationship between the moving object and the camera is determined by a microphone array, and the moving object can be tracked at any position of the camera in combination with the direction angle and pitch angle of the moving object. Therefore, the camera of the embodiment of the present application can be applied to any place, thereby greatly expanding the application scope of the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0065] Figure 2 It is a flowchart of a method for controlling a camera to track a moving object provided in an embodiment of the present application;

[0066] Figure 3 is a schematic structural diagram of a preset coordinate system provided according to an embodiment of the present application;

[0067] Figure 4 is a flow chart of step 201 provided according to an embodiment of the present application;

[0068] Figure 5 is a flow chart of step 2013 provided according to an embodiment of the present application;

[0069] Figure 6 is a flow chart of step 2014 provided according to an embodiment of the present application;

[0070] Figure 7 is a flow chart of step 202 provided according to an embodiment of the present application;

[0071] Figure 8 is a flow chart of step 203 provided according to an embodiment of the present application;

[0072] Fig. 9 is a structural diagram of a system 900 for controlling a camera to track a moving object provided according to an embodiment of the present application;

[0073] Fig.10 is a structural schematic diagram of a device for controlling a camera to track a moving object provided according to an embodiment of the present application;

[0074] Fig.11 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0075] Fig.12 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0077] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0078] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0079] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0080] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0081] In the related art, cameras have been widely used. Generally, the shooting area of ​​a camera is limited. The shooting area of ​​the camera can be expanded by rotating the camera. However, the camera still has blind spots. To address the problem of blind spots, in the related art, when an object makes a sound, the camera can be controlled to rotate, so as to shoot objects that were not in the shooting area before. However, for some objects that do not make a sound and are not in the shooting area, the camera still cannot shoot, that is, the camera still has blind spots.

[0082] In addition, since most of the objects captured by the camera are moving, it is necessary to track the moving objects. However, the current camera tracking range is limited, and generally can only track moving objects below the camera or objects above the moving camera, which greatly reduces the application range of the camera.

[0083] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0084] The terminal 101 is provided with a camera, such as a camera provided on a computer or a camera component provided on a mobile phone.

[0085] The terminal 101 may be a smart phone with a camera, a wearable device, a personal computer, a laptop, a tablet computer, a smart TV, a car terminal, etc.

[0086] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0087] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0088] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.

[0089] Figure 2 is a flow chart of a method for controlling a camera to track a moving object provided in an embodiment of the present application, such as Figure 2 As shown, in the embodiment of the present application, the application is described by taking the application on a terminal with a camera as an example. The method comprises the following steps:

[0090] In step 201, the terminal determines the tth first spatial coordinate of the moving object based on ultrasound and a microphone array.

[0091] Among them, the first spatial coordinate is a coordinate in a preset coordinate system; and t is an integer greater than or equal to 2.

[0092] In some embodiments, the microphone array includes a positional microphone and a non-positional microphone; the positional microphone is arranged on a first plane; the camera and the non-positional microphone are arranged on a second plane; the first plane and the second plane are planes within a preset coordinate system; the positional microphone is used to determine the positional relationship between the camera and the moving object.

[0093] The position relationship indicates that the moving object is above or below the camera or that the moving object and the camera are in the same horizontal plane.

[0094] In some embodiments, ultrasonic waves are generated and broadcast; and valid echoes are received; and valid echoes are ultrasonic waves reflected by a moving object.

[0095] The ultrasonic wave may be an ultrasonic wave of a fixed frequency. It should be noted that the ultrasonic wave frequency is not limited here.

[0096] In one example, the terminal plays ultrasonic waves based on a speaker; the coordinates of the speaker and the sound wave generator in the preset coordinates are the same as the coordinates of the camera. That is, the speaker and the sound wave generator are set at the same position as the camera to facilitate subsequent calculations. Optionally, the terminal plays periodic ultrasonic waves based on the speaker.

[0097] In one example, the terminal estimates the ambient noise based on the noise estimation technology when there are no moving objects in the environment. For example, the noise estimation technology relies on measuring equipment such as acoustic sensors and sound level meters, and analysis methods such as spectrum analysis to estimate the ambient noise.

[0098] In one example, after receiving the ambient noise, the microphone array converts the ambient noise into an electrical signal, and converts the electrical signal into an ambient digital signal based on an analog-to-digital conversion module. The terminal stores the ambient digital signal.

[0099] It should be noted that the analog-to-digital conversion module can be obtained from the relevant technology and will not be described in detail in the embodiments of the present application.

[0100] In some embodiments, even if there are no obvious moving objects in the environment, ultrasound waves may be reflected and scattered by tiny particles in the air, temperature gradient changes, or medium inhomogeneities, forming reference echoes, which may be captured by the microphone array. Therefore, in order to determine whether the echo received by the microphone is reflected by a moving object, the reference echo needs to be determined.

[0101] In one example, after receiving the reference echo, the microphone converts the reference echo into an electrical signal, and then converts the electrical signal into a reference digital signal based on an analog-to-digital conversion module.

[0102] In some embodiments, an initial digital signal corresponding to a valid echo is acquired based on a microphone array; and the initial digital signal is filtered to obtain a target digital signal.

[0103] In one example, after the microphone receives the echo, a digital signal corresponding to the echo is obtained based on the above method. The digital signal is filtered based on the reference digital signal and the ambient digital signal. That is, the ambient digital signal and the reference digital signal are removed from the digital signal to obtain the remaining part of the digital signal. If there is no remaining part of the digital signal after removing the ambient digital signal and the reference digital signal, or the remaining part is 0. That is, the sum of the ambient digital signal and the reference digital signal is the digital signal. Then the echo is not an echo reflected by a moving object. That is, the echo is not a valid echo, but an invalid echo.

[0104] Figure 3 It is a structural schematic diagram of a preset coordinate system provided according to an embodiment of the present application.

[0105] Combine the following Figure 3 The preset coordinate system of the present application is exemplified.

[0106] In some embodiments, the non-positional microphone includes a first microphone, a second microphone, and a third microphone;

[0107] The coordinates of the first microphone are the fourth spatial coordinates; the fourth spatial coordinates are the same as the second spatial coordinates;

[0108] The second microphone is set on the x-axis;

[0109] The third microphone is arranged on the y-axis of the preset coordinate system;

[0110] The position microphone is set on the z-axis of the preset coordinate system and is located above the camera. The x-axis, y-axis and z-axis are the coordinate axes of the preset coordinate system. For example, in the preset coordinate system, the first microphone is represented by A; the second microphone is represented by B; the third microphone is represented by C; and the position microphone is represented by D.

[0111] In one example, the position microphone may also be located below the camera.

[0112] From the above analysis, it can be known that the reason why the position microphone is set in the embodiment of the present application is to determine whether the moving object is above or below the camera through the position microphone.

[0113] Figure 4 It is a flowchart of step 201 provided according to an embodiment of the present application.

[0114] Combine the following Figure 4 Step 201 is exemplarily described.

[0115] In some embodiments, the above step 201 includes the following steps 2011 to 2014:

[0116] In step 2011, the terminal obtains the playback time and the receiving time, and calculates the length of the tth first connection line based on the playback time, the receiving time and the wave speed; the playback time is the playback time of the ultrasonic wave; the receiving time is the time for receiving the effective echo; the wave speed is the propagation speed of the ultrasonic wave in the air.

[0117] In one example, since the coordinates of the sound wave generator, the speaker, and the camera are the same, the coordinates of the first microphone are also the same as the camera. That is, the ultrasonic wave reaches the moving object from the position of the first microphone, and then reaches the first microphone from the moving object. That is, the path of the ultrasonic wave between the first microphone and the moving object is the same. It can also be understood that the time it takes for the first microphone to receive a valid echo is the length of the two first connecting lines.

[0118] For example, the length of the first line can be calculated using the following formula:

[0119] The length of the first connection line = (the receiving time of the first microphone - the playing time of the first microphone) × the playing speed / 2. Similarly, the length of the t-th first connection line can be obtained.

[0120] In step 2012, the terminal calculates the length of the tth third line, the length of the tth fourth line, and the length of the tth fifth line based on the length of the tth first line.

[0121] In some embodiments, the tth third line connects the tth first spatial coordinate and the tth fifth spatial coordinate; the tth fifth spatial coordinate is the coordinate of the second microphone in the preset coordinate system; the tth fourth line connects the tth first spatial coordinate and the tth sixth spatial coordinate; the tth sixth spatial coordinate is the coordinate of the third microphone in the preset coordinate system; the tth fifth line connects the tth first spatial coordinate and the tth seventh spatial coordinate; the seventh spatial coordinate is the coordinate of the position microphone in the preset coordinate system.

[0122] In one example, since the coordinates of the second microphone, the third microphone, and the position microphone are different from those of the first microphone, that is, the path of the ultrasonic wave from the first microphone to the moving object and then from the moving object to the second microphone are different. Therefore, the length of the first line is different from the length of the third line. Similarly, the length of the first line is different from the length of the fourth line. The length of the first line is also different from the length of the fifth line. In the above embodiment, the length of the first line has been obtained. Therefore, the length of the third line, the length of the fourth line, and the length of the fifth line can be calculated respectively through the length of the first line.

[0123] For example, the length of the third line can be obtained by the following formula:

[0124] The length of the third connection line = (the receiving time of the second microphone - the playing time of the first microphone) × the playing speed - the length of the first connection line. Similarly, the length of the t-th third connection line can be obtained.

[0125] The length of the fourth line can be obtained by the following formula:

[0126] The length of the fourth connection line = (the receiving time of the third microphone - the playing time of the first microphone) × the playing speed - the length of the first connection line. Similarly, the length of the t-th fourth connection line can be obtained.

[0127] The length of the fifth line can be obtained by the following formula:

[0128] The length of the fifth line = (receiving time of the position microphone - playing time of the first microphone) × playing speed - length of the first line. Similarly, the length of the tth fifth line can be obtained.

[0129] In step 2013, the terminal calculates the tth first angle, the tth second angle and the tth third angle based on the length of the tth first line, the length of the tth third line, the length of the tth fourth line and the length of the tth fifth line.

[0130] In some embodiments, the tth first angle is the angle between the tth third line and the tth first line; the tth second angle is the angle between the tth fourth line and the tth first line; and the tth third angle is the angle between the tth first line and the z-axis.

[0131] Figure 5 It is a flowchart of step 2013 provided according to an embodiment of the present application.

[0132] Combine the following Figure 5 Step 2013 is exemplarily described.

[0133] In some embodiments, the above step 2013 includes the following steps 20131 to 20133:

[0134] In step 20131, the terminal calculates the tth first angle based on the length of the tth first line, the length of the tth third line, and the first distance; the first distance is the distance between the first microphone and the second microphone.

[0135] In one example, the terminal calculates the cosine value of the tth first angle based on the length of the tth first line, the length of the tth third line, and the first distance.

[0136] For example: the first angle is represented by ∠BAM. The second angle is represented by ∠CAM, and the third angle is represented by ∠DAM. The first line is represented by MA. The third line is represented by MB. The fourth line is represented by MC. The fifth line is represented by MD. The first distance is represented by AB.

[0137] For example, the cosine value of the first angle can be calculated using the following formula:

[0138] cos∠BAM=(MA 2 +AB 2 -MB 2 ) / (2×MA×AB). Similarly, the cosine value of the tth first angle can be obtained.

[0139] In step 20132, the terminal calculates the tth second angle based on the length of the tth first line, the length of the tth fourth line, and the second distance; the second distance is the distance between the first microphone and the third microphone.

[0140] In one example, the terminal calculates the cosine value of the tth second angle based on the length of the tth first line, the length of the tth fourth line, and the second distance. For example, the second distance is represented by AC. For example, the cosine value of the second angle can be calculated by the following formula:

[0141] cos∠CAM=(MA 2 +AC 2 -MC 2 ) / (2×MA×AC). Similarly, the cosine value of the t-th second angle can be obtained.

[0142] In step 20133, the terminal calculates the tth third angle based on the length of the tth first line, the length of the tth fifth line, and the third distance; the third distance is the distance between the first microphone and the position microphone.

[0143] In one example, the terminal calculates the cosine value of the tth third angle based on the length of the tth first line, the length of the tth fifth line, and the third distance. For example, the third distance is represented by AD. For example, the cosine value of the third angle can be calculated by the following formula:

[0144] cos∠DAM=(MA 2 +AD 2 -MD 2 ) / (2×MA×AD). Similarly, the cosine value of the t-th third angle can be obtained.

[0145] In some embodiments, when the third angle is an acute angle, the positional relationship indicates that the moving object is above the camera; or when the third angle is a right angle, the positional relationship indicates that the moving object and the camera are on the same horizontal plane; or when the third angle is an obtuse angle, the positional relationship indicates that the moving object is below the camera.

[0146] In step 2014, the terminal calculates the tth first space coordinate based on the tth first angle, the tth second angle, and the tth third angle.

[0147] Figure 6 It is a flowchart of step 2014 provided according to an embodiment of the present application.

[0148] Combine the following Figure 6 Step 2014 is exemplarily described.

[0149] In some embodiments, the above step 2014 includes the following steps 20141 to 20143:

[0150] In step 20141, the terminal obtains the tth x value based on the tth first angle and the first connecting line; the tth x value is the distance from the tth first spatial coordinate to the yoz plane; the yoz plane is a plane in the preset coordinate system.

[0151] In one example, the terminal calculates the tth first angle based on the cosine value of the tth first angle; the terminal uses the product of the cosine value of the first angle and the first line as the tth x value.

[0152] For example, the x value can be obtained by the following formula:

[0153] x value = MA × cos ∠BAM.

[0154] Where ∠BAM=arccos((MA 2 +AB 2 -MB 2 ) / (2×MA×AB)).

[0155] Similarly, the tth x value can be obtained.

[0156] In step 20142, the terminal obtains the tth y value based on the tth second angle and the first connecting line; the tth y value is the distance from the tth first spatial coordinate to the xoz plane; the xoz plane is a plane in the preset coordinate system.

[0157] In one example, the terminal calculates the tth second angle based on the cosine value of the tth second angle; the terminal uses the product of the cosine value of the second angle and the first line as the tth y value.

[0158] For example, the y value can be obtained by the following formula:

[0159] y value = MA × cos ∠CAM.

[0160] Where, ∠CAM = arccos((MA 2 +AC 2 -MC 2 ) / (2×MA×AC)).

[0161] Similarly, the tth y value can be obtained.

[0162] In step 20143, the terminal obtains the tth z value based on the tth third angle and the first connecting line; the tth z value is the distance from the tth first spatial coordinate to the xoy plane; the xoy plane is a plane in the preset coordinate system.

[0163] In one example, the terminal calculates the tth third angle based on the cosine value of the tth third angle; the terminal uses the product of the cosine value of the third angle and the first line as the tth z value.

[0164] For example, the z value can be obtained by the following formula:

[0165] z value = MA × cos∠DAM.

[0166] Where, ∠DAM=arccos((MA 2 +AD 2 -MD 2 ) / (2×MA×AD)).

[0167] Similarly, the tth z value can be obtained.

[0168] That is, the first spatial coordinate can be expressed as: (x value, y value, z value).

[0169] In one example, when ∠DAM is an acute angle, the z value is a positive value, that is, the moving object is above the camera. When ∠DAM is an obtuse angle, the z value is a negative value, that is, the moving object is below the camera. When ∠DAM is a right angle, the z value is 0, that is, the moving object and the camera are in the same horizontal plane. Optionally, in the preset coordinate system of the present application, when the z value is 0, it means that the moving object and the camera are both in the xoy plane.

[0170] Through the analysis of step 201, it can be known that the embodiment of the present application determines the first spatial coordinates of the moving object through ultrasound. That is, even if the moving object does not make a sound, it will be captured by the camera after entering the shooting area of ​​the camera, thereby solving the problem of blind spots in the camera in the related art. And according to ∠DAM, it can be determined whether the moving object is above, below or on the same horizontal plane as the camera, so when controlling the rotation of the camera, it can be determined whether to rotate the camera upward or downward, and then the camera can be used to track moving objects on the ground and moving objects in the air. That is, in the embodiment of the present application, the camera can be installed at any position to track moving objects at any position.

[0171] In step 202, the terminal determines the tth direction angle and the tth elevation angle of the camera based on the tth first space coordinate.

[0172] In some embodiments, the tth pitch angle is the angle between the tth first connecting line and the tth second connecting line; the tth first connecting line connects the tth first spatial coordinate and the tth second spatial coordinate; the tth second spatial coordinate is the coordinate of the camera; the tth second spatial coordinate is the coordinate in the preset coordinate system; the tth second connecting line connects the tth first spatial coordinate and the tth third spatial coordinate; the tth third spatial coordinate indicates the foot of the perpendicular of the tth first spatial coordinate on the second plane; the tth direction angle is the angle between the tth second connecting line and the x-axis of the preset coordinate system; for example: the second plane is the xoy plane. o is the origin of the preset coordinate system. For another example, for the convenience of calculation, the camera is set at the origin of the preset coordinate system, that is, the second spatial coordinate is (0, 0, 0). For another example: when the center of the camera is facing the positive direction of the z-axis, the pitch angle is 90° and the direction angle is 0°.

[0173] Figure 7 It is a flowchart of step 202 provided according to an embodiment of the present application.

[0174] Combine the following Figure 7 Step 202 is exemplarily described.

[0175] In some embodiments, the above step 202 includes the following steps 2021 to 2023:

[0176] In step 2021, the terminal calculates the tth pitch angle based on the length of the tth first connecting line and the tth z value.

[0177] In step 2022, the terminal calculates the tth direction angle based on the tth y value and the tth second connecting line.

[0178] For example, the foot of the perpendicular is represented by N. Then the second line is represented by AN. The pitch angle is represented by ∠NAM. The azimuth angle is represented by ∠NAB. The pitch angle can be calculated by the following formula:

[0179] ∠NAM=arcsin(MN / AM). The length of MN is also the z value.

[0180] The direction angle can be calculated using the following formula:

[0181] ∠NAB=arcsin(NE / AE). The length of NE is also the y value, and the length of AE is also the x value.

[0182] In one example, the terminal determines a first direction angle and a first elevation angle of the camera based on a first first spatial coordinate of the moving object. Based on the first direction angle and the first elevation angle, the camera is controlled to rotate. The shooting area of ​​the adjusted camera faces the moving object, that is, the moving object falls into the shooting area of ​​the camera. And the center of the shooting area is on the same straight line as the center of the moving object.

[0183] In step 203, the terminal controls the camera to track the moving object based on the tth direction angle and the tth elevation angle.

[0184] In some embodiments, when the angle between the center of the shooting area of ​​the camera and the center of the moving object is large, the moving object cannot be accurately tracked. To solve this problem, the embodiments of the present application provide the following technical solutions.

[0185] Figure 8 It is a flowchart of step 203 provided according to an embodiment of the present application.

[0186] Combine the following Figure 8 Step 203 is exemplarily described.

[0187] In some embodiments, the above step 203 includes the following steps 2031 and 2032:

[0188] In step 2031: the terminal obtains the t-1th first difference and the t-1th second difference; the t-1th first difference is the difference between the tth direction angle and the t-1th direction angle; the t-1th second difference is the difference between the tth elevation angle and the t-1th elevation angle;

[0189] In step 2032, the terminal controls the camera to rotate to the tth target state according to the t-1th first difference value and the t-1th second difference value.

[0190] In some embodiments, the target state indicates that the moving object falls into a shooting area of ​​the camera, and the center of the shooting area and the center of the moving object are in a straight line.

[0191] In some embodiments, since the moving object is constantly moving, it is necessary to continuously determine the position of the moving object after it moves, that is, the t-th first spatial coordinate. In order to realize the tracking of the moving object by the camera, it is necessary to control the rotation of the camera according to the position of the moving object after it moves, so that the center of the shooting area of ​​the rotated camera and the center of the moving object after it moves are still in a straight line, thereby realizing the tracking of the moving object.

[0192] In some embodiments, the above step 2032 may be implemented in the following ways:

[0193] The first implementation method: if the tth first difference is greater than or equal to the preset threshold, and the tth second difference is greater than or equal to the preset threshold, the camera is controlled to rotate to the tth target state based on the tth first difference and the tth second difference. The tth first difference is the difference between the t+1th direction angle and the tth direction angle; the tth second difference is the difference between the t+1th elevation angle and the tth elevation angle.

[0194] In one example, since the terminal updates the direction angle and the elevation angle in real time. Each time the terminal obtains a direction angle and an elevation angle, it obtains the difference between the direction angle and the elevation angle and the previous direction angle and elevation angle, that is, the first difference and the second difference, so as to determine the positional relationship between the moving object and the camera. If the first difference is greater than or equal to the preset threshold, and the t-th second difference is greater than or equal to the preset threshold, it indicates that the direction angle has a large change relative to the previous direction angle, and the elevation angle has a large change relative to the previous elevation angle. That is, the change in the direction angle and the change in the elevation angle make the angle between the center of the camera's shooting area and the center of the moving object larger. Therefore, it is necessary to adjust the elevation angle and the direction angle of the camera to accurately track the moving object. That is, adjust the previous elevation angle to the elevation angle, and adjust the previous direction angle to the direction angle. That is, the moving object falls into the shooting area of ​​the camera, and the center of the shooting area and the center of the moving object are on a straight line.

[0195] For example: the preset threshold is 20°.

[0196] The second implementation method: if the tth first difference is greater than or equal to the preset threshold, and the tth second difference is less than the preset threshold, the camera is controlled to rotate to the tth target state based on the tth first difference.

[0197] In one example, if the t-th first difference is greater than or equal to a preset threshold, and the t-th second difference is less than a preset threshold, it indicates that the change in the direction angle relative to the previous direction angle is large, and the change in the elevation angle relative to the previous elevation angle is small. That is, the change in the direction angle makes the angle between the center of the camera's shooting area and the center of the moving object larger. Therefore, the moving object can be accurately tracked by only adjusting the direction angle of the camera. That is, there is no need to adjust the elevation angle of the camera, only the previous direction angle needs to be adjusted to this direction angle. That is, the moving object falls into the shooting area of ​​the camera, and the center of the shooting area and the center of the moving object are on a straight line.

[0198] The third implementation method: if the tth first difference is less than the preset threshold and the tth second difference is greater than or equal to the preset threshold, the camera is controlled to rotate to the tth target state based on the tth second difference.

[0199] In one example, if the t-th first difference is less than a preset threshold, and the t-th second difference is greater than or equal to a preset threshold, it indicates that the change in the direction angle relative to the previous direction angle is small, and the change in the elevation angle relative to the previous elevation angle is large. That is, the change in the elevation angle makes the angle between the center of the camera's shooting area and the center of the moving object larger, so only adjusting the elevation angle of the camera can accurately track the moving object. That is, there is no need to adjust the direction angle of the camera, and the previous elevation angle is adjusted to this elevation angle. That is, the moving object falls into the shooting area of ​​the camera, and the center of the shooting area and the center of the moving object are on a straight line.

[0200] A fourth implementation method: if the t-th first difference value is smaller than a preset threshold value, and the t-th second difference value is smaller than a preset threshold value, the camera is not rotated.

[0201] In one example, if the t-th first difference is less than a preset threshold, and the t-th second difference is greater than or equal to the preset threshold, it indicates that the change of the direction angle relative to the previous direction angle is small, and the change of the elevation angle relative to the previous elevation angle is also small. That is, the angle between the center of the camera's shooting area and the center of the moving object is small. Therefore, there is no need to adjust the camera's direction angle and elevation angle.

[0202] It can be seen from the above analysis that, therefore, in the process of the camera tracking a moving object, the center of the shooting area and the center of the moving object are on a straight line or the angle between them does not exceed the preset threshold.

[0203] In some embodiments, a single moving object is tracked.

[0204] In some embodiments, if multiple moving objects enter the surrounding area of ​​the camera at the same time, the t-th direction angle and the t-th elevation angle of each moving object are obtained;

[0205] Obtain the t-th first difference value and the t-th second difference value of each moving object;

[0206] According to the tth first difference value and the tth second difference value of the kth moving object, control the camera to rotate and shoot the kth moving object for a preset time; k is an integer greater than or equal to 1;

[0207] According to the t-th first difference value and the t-th second difference value of the k+1-th moving object, the camera is controlled to rotate and shoot the k+1-th moving object for a preset time.

[0208] In one example, if multiple moving objects enter the surrounding area of ​​the camera at the same time, the terminal records the t-th direction angle and the t-th pitch angle of each moving object in real time, obtains the t-th first difference and the t-th second difference of each moving object; rotates the camera according to the t-th first difference and the t-th second difference of each moving object. Optionally, the terminal controls the camera to shoot each moving object in a cycle. If the direction angle and the pitch angle of a moving object are the same as those of the previous shooting, skip the moving object and continue to shoot and track other moving objects. For example: the preset duration is 1-2 seconds.

[0209] For example, three moving objects enter the area around the camera at the same time. The azimuth and elevation angles of the first moving object are 20° and 30° respectively; the azimuth and elevation angles of the second moving object are 35° and 45° respectively; and the azimuth and elevation angles of the third moving object are 45° and 55° respectively.

[0210] In the first cycle of shooting, the first moving object is shot continuously for 1-2 seconds, the second moving object is shot for 1-2 seconds, and the third moving object is shot for 1-2 seconds. In the second cycle of shooting, the updated direction angle and pitch angle of each moving object are obtained. If the t-th first difference value and the t-th second difference value of each moving object obtained by the second moving object are both 0, the camera is controlled to rotate directly from the first moving object to the center of the shooting area facing the center of the third moving object, and the third moving object is shot continuously for 1-2 seconds until the direction angles and pitch angles of all moving objects no longer change, thereby achieving the shooting and tracking of each moving object.

[0211] In some embodiments, the camera is controlled to rotate by a guidance controller. Optionally, after receiving the rotation signal, the guidance controller controls the camera to rotate based on the rotation signal. The rotation signal is obtained based on the t-th first difference and the t-th second difference. It should be noted that the guidance controller can be obtained from the relevant technology, and the embodiments of the present application will not be repeated.

[0212] The embodiment of the present application detects moving objects in the area around the camera through ultrasonic waves, determines the direction angle and pitch angle of the moving object, and avoids the blind spot problem of the camera. The positional relationship between the moving object and the camera is determined by the microphone array, and combined with the direction angle and pitch angle of the moving object, the moving object at any position of the camera can be tracked. Therefore, the camera of the embodiment of the present application can be used in any place, thereby greatly expanding the application range of the camera.

[0213] Fig. 9 is a structural diagram of a system 900 for controlling a camera to track a moving object provided in an embodiment of the present application, the system comprising a camera, a sound wave generator, and a microphone array;

[0214] The sound wave generator and the microphone array are used to determine the spatial coordinates of the moving object; the spatial coordinates are the coordinates of the preset coordinate system;

[0215] The microphone array includes a position microphone; the position microphone is used to determine the position relationship between the camera and the moving object.

[0216] In some embodiments, the microphone array further includes a non-positional microphone; the positional microphone is arranged on a first plane; the camera and the non-positional microphone are arranged on a second plane; the first plane and the second plane are planes within a preset coordinate system.

[0217] It should be noted that the system for controlling a camera to track a moving object and the method for controlling a camera to track a moving object provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0218] The embodiment of the present application detects moving objects in the area around the camera through ultrasonic waves, determines the direction angle and pitch angle of the moving object, and avoids the blind spot problem of the camera. The positional relationship between the moving object and the camera is determined by the microphone array, and combined with the direction angle and pitch angle of the moving object, the moving object at any position of the camera can be tracked. Therefore, the camera of the embodiment of the present application can be used in any place, thereby greatly expanding the application range of the camera.

[0219] Fig.10 1 is a schematic diagram of a structure of a device 1000 for controlling a camera to track a moving object according to an embodiment of the present application, the device comprising:

[0220] The first determination module 1001 is used to determine the t-th first spatial coordinate of the moving object based on the ultrasonic wave and the microphone array; the first spatial coordinate is a coordinate in a preset coordinate system; t is an integer greater than or equal to 2; the microphone array includes a position microphone; the position microphone is used to determine the position relationship between the camera and the moving object;

[0221] The second determining module 1002 is used to determine the t-th direction angle and the t-th elevation angle of the camera based on the t-th first spatial coordinate; the t-th elevation angle is the angle between the t-th first connecting line and the t-th second connecting line; the t-th direction angle is the angle between the t-th second connecting line and the x-axis of the preset coordinate system;

[0222] The control module 1003 is used to control the camera to track the moving object based on the t-th direction angle and the t-th elevation angle.

[0223] In some embodiments, the apparatus is further configured to:

[0224] Generate ultrasonic waves and play ultrasonic waves;

[0225] Receive effective echo; effective echo is the ultrasonic wave reflected by the moving object.

[0226] In some embodiments, the apparatus is further configured to:

[0227] Acquire an initial digital signal corresponding to a valid echo based on a microphone array;

[0228] The initial digital signal is filtered to obtain the target digital signal.

[0229] In some embodiments, the tth first line connects the tth first spatial coordinate and the tth second spatial coordinate; the tth second spatial coordinate is the coordinate of the camera; the tth second spatial coordinate is the coordinate within a preset coordinate system; the tth second line connects the tth first spatial coordinate and the tth third spatial coordinate; the tth third spatial coordinate indicates the foot of the tth first spatial coordinate on the second plane.

[0230] In some embodiments, the microphone array further includes a non-positional microphone; the non-positional microphone further includes a first microphone, a second microphone, and a third microphone;

[0231] The coordinates of the first microphone are the fourth spatial coordinates; the fourth spatial coordinates are the same as the second spatial coordinates;

[0232] The second microphone is set on the x-axis;

[0233] The third microphone is arranged on the y-axis of the preset coordinate system;

[0234] The position microphone is arranged on the z-axis of the preset coordinate system and is located above the camera; the position microphone is arranged on the first plane; the camera and the non-position microphone are arranged on the second plane; the first plane and the second plane are planes within the preset coordinate system.

[0235] In some embodiments, determining the t-th first spatial coordinate of the moving object based on the ultrasonic wave includes:

[0236] Obtain the play time and the receiving time, and calculate the length of the t-th first connection line based on the play time, the receiving time and the wave speed; the play time is the play time of the ultrasonic wave; the receiving time is the time of receiving the effective echo; the wave speed is the propagation speed of the ultrasonic wave in the air;

[0237] Based on the length of the t-th first line, the length of the t-th third line, the length of the t-th fourth line and the length of the t-th fifth line are calculated; the t-th third line connects the t-th first spatial coordinate and the t-th fifth spatial coordinate; the t-th fifth spatial coordinate is the coordinate of the second microphone in the preset coordinate system; the t-th fourth line connects the t-th first spatial coordinate and the t-th sixth spatial coordinate; the t-th sixth spatial coordinate is the coordinate of the third microphone in the preset coordinate system; the t-th fifth line connects the t-th first spatial coordinate and the t-th seventh spatial coordinate; the seventh spatial coordinate is the coordinate of the position microphone in the preset coordinate system;

[0238] Based on the length of the t-th first connecting line, the length of the t-th third connecting line, the length of the t-th fourth connecting line and the length of the t-th fifth connecting line, the t-th first angle, the t-th second angle and the t-th third angle are calculated; the t-th first angle is the angle between the t-th third connecting line and the t-th first connecting line; the t-th second angle is the angle between the t-th fourth connecting line and the t-th first connecting line; the t-th third angle is the angle between the t-th first connecting line and the z-axis;

[0239] Based on the tth first angle, the tth second angle, and the tth third angle, the tth first space coordinate is calculated.

[0240] In some embodiments, based on the length of the t-th first line, the length of the t-th third line, the length of the t-th fourth line, and the length of the t-th fifth line, the t-th first angle, the t-th second angle, and the t-th third angle are calculated, including:

[0241] The tth first angle is calculated based on the length of the tth first connecting line, the length of the tth third connecting line and the first distance; the first distance is the distance between the first microphone and the second microphone;

[0242] The tth second angle is calculated based on the length of the tth first connecting line, the length of the tth fourth connecting line and the second distance; the second distance is the distance between the first microphone and the third microphone;

[0243] The tth third angle is calculated based on the length of the tth first connecting line, the length of the tth fifth connecting line and the third distance; the third distance is the distance between the first microphone and the position microphone.

[0244] In some embodiments, when the third angle is an acute angle, the positional relationship indicates that the moving object is above the camera; or when the third angle is a right angle, the positional relationship indicates that the moving object and the camera are on the same horizontal plane; or when the third angle is an obtuse angle, the positional relationship indicates that the moving object is below the camera.

[0245] In some embodiments, the apparatus is further configured to:

[0246] Based on the tth first angle and the first connecting line, the tth x value is obtained; the tth x value is the distance from the tth first spatial coordinate to the yoz plane; the yoz plane is a plane in the preset coordinate system;

[0247] Based on the t-th second angle and the first connecting line, the t-th y value is obtained; the t-th y value is the distance from the t-th first spatial coordinate to the xoz plane; the xoz plane is a plane in the preset coordinate system;

[0248] Based on the tth third angle and the first connecting line, the tth z value is obtained; the tth z value is the distance from the tth first spatial coordinate to the xoy plane; the xoy plane is a plane in the preset coordinate system.

[0249] In some embodiments, the apparatus is further configured to:

[0250] Based on the length of the t-th first connecting line and the t-th z value, the t-th pitch angle is calculated;

[0251] Based on the tth y value and the tth second connecting line, the tth direction angle is calculated.

[0252] In a possible implementation, based on the t-th direction angle and the t-th elevation angle, controlling the camera to track the moving object includes:

[0253] Obtain the t-1th first difference and the t-1th second difference; the t-1th first difference is the difference between the tth direction angle and the t-1th direction angle; the t-1th second difference is the difference between the tth elevation angle and the t-1th elevation angle;

[0254] According to the t-1th first difference and the t-1th second difference, the camera is controlled to rotate to the tth target state; the target state indicates that the moving object falls into the shooting area of ​​the camera, and the center of the shooting area is in a straight line with the center of the moving object.

[0255] In some embodiments, the apparatus is further configured to:

[0256] If the tth first difference is greater than or equal to a preset threshold, and the tth second difference is greater than or equal to the preset threshold, the camera is controlled to rotate to the tth target state based on the tth first difference and the tth second difference; the tth first difference is the difference between the t+1th direction angle and the tth direction angle; the tth second difference is the difference between the t+1th elevation angle and the tth elevation angle; or

[0257] If the t-th first difference is greater than or equal to the preset threshold, and the t-th second difference is less than the preset threshold, the camera is controlled to rotate to the t-th target state based on the t-th first difference; or

[0258] If the t-th first difference is less than the preset threshold, and the t-th second difference is greater than or equal to the preset threshold, the camera is controlled to rotate to the t-th target state based on the t-th second difference; or

[0259] If the t-th first difference value is smaller than the preset threshold value, and the t-th second difference value is smaller than the preset threshold value, the camera is not rotated.

[0260] In some embodiments, the apparatus is further configured to:

[0261] If multiple moving objects enter the area around the camera at the same time, obtain the tth direction angle and tth elevation angle of each moving object;

[0262] Obtain the t-th first difference value and the t-th second difference value of each moving object;

[0263] According to the tth first difference value and the tth second difference value of the kth moving object, control the camera to rotate and shoot the kth moving object for a preset time; k is an integer greater than or equal to 1;

[0264] According to the t-th first difference value and the t-th second difference value of the k+1-th moving object, the camera is controlled to rotate and shoot the k+1-th moving object for a preset time.

[0265] It should be noted that: when the device for controlling a camera to track a moving object provided in the above embodiment performs the corresponding steps, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device for controlling a camera to track a moving object provided in the above embodiment and the method embodiment for controlling a camera to track a moving object belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0266] The embodiment of the present application detects moving objects in the area around the camera through ultrasonic waves, determines the direction angle and pitch angle of the moving object, and avoids the blind spot problem of the camera. The positional relationship between the moving object and the camera is determined by the microphone array, and combined with the direction angle and pitch angle of the moving object, the moving object at any position of the camera can be tracked. Therefore, the camera of the embodiment of the present application can be used in any place, thereby greatly expanding the application range of the camera.

[0267] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.

[0268] Taking computer equipment as the terminal as an example, Fig.11 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Fig.11 The terminal 1100 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The terminal 1100 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0269] Typically, the terminal 1100 includes a processor 1101 and a memory 1102 .

[0270] The processor 1101 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0271] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one program code, which is used to be executed by the processor 1101 to implement the process provided by the method embodiment of the present application for the terminal execution in the above method.

[0272] In some embodiments, the terminal 1100 may further optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, the memory 1102 and the peripheral device interface 1103 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1103 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 1104, a camera assembly 1105, an audio circuit 1106 and a power supply 1107.

[0273] The peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0274] The display screen 1104 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1104 is a touch display screen, the display screen 1104 also has the ability to collect touch signals on the surface or above the surface of the display screen 1104. The touch signal can be input to the processor 1101 as a control signal for processing. At this time, the display screen 1104 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1104 can be one, set on the front panel of the terminal 1100; in other embodiments, the display screen 1104 can be at least two, respectively set on different surfaces of the terminal 1100 or in a folding design; in other embodiments, the display screen 1104 can be a flexible display screen, set on the curved surface or folding surface of the terminal 1100. Even, the display screen 1104 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1104 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0275] The camera assembly 1105 is used to capture images or videos. In some embodiments, the camera assembly 1105 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1105 may also include a flash. The flash may be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0276] The audio circuit 1106 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1101 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1100. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1101 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1106 may also include a headphone jack.

[0277] The power supply 1107 is used to power various components in the terminal 1100. The power supply 1107 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1107 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0278] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the terminal 1100, and the terminal 1100 may include more or fewer components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0279] Take the computer device as a server as an example. Fig.12 1 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1200 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1201 and one or more memories 1202, wherein the one or more memories 1202 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 1201 to implement the above-mentioned background removal method. Of course, the server 1200 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 1200 may also include other components for implementing device functions, which will not be repeated here.

[0280] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above method. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0281] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0282] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for controlling a camera to track a moving object, characterized in that: include: Determine the tth first spatial coordinate of the moving object based on the ultrasonic wave and the microphone array; the first spatial coordinate is a coordinate in a preset coordinate system; t is an integer greater than or equal to 2; the microphone array includes a position microphone; the position microphone is used to determine the position relationship between the camera and the moving object; Based on the tth first spatial coordinate, determine the tth direction angle and the tth elevation angle of the camera; the tth elevation angle is the angle between the tth first connecting line and the tth second connecting line; the tth direction angle is the angle between the tth second connecting line and the x-axis of the preset coordinate system; Based on the t-th direction angle and the t-th elevation angle, control the camera to track the moving object.

2. The method according to claim 1, characterized in that The method further comprises: generating ultrasonic waves and playing the ultrasonic waves; Receive a valid echo; the valid echo is the ultrasonic wave reflected by the moving object.

3. The method according to claim 2, characterized in that The method further comprises: Acquire an initial digital signal corresponding to the effective echo based on the microphone array; The initial digital signal is filtered to obtain a target digital signal.

4. The method according to claim 3, characterized in that The tth first line connects the tth first space coordinate and the tth second space coordinate; the tth second space coordinate is the coordinate of the camera; the tth second space coordinate is the coordinate within the preset coordinate system; the tth second line connects the tth first space coordinate and the tth third space coordinate; the tth third space coordinate indicates the foot of the tth first space coordinate on the second plane.

5. The method according to claim 4, characterized in that The microphone array further includes a non-positional microphone; the non-positional microphone further includes a first microphone, a second microphone and a third microphone; The coordinates of the first microphone are fourth spatial coordinates; the fourth spatial coordinates are the same as the second spatial coordinates; The second microphone is arranged on the x-axis; The third microphone is arranged on the y-axis of the preset coordinate system; The position microphone is arranged on the z-axis of the preset coordinate system and is located above the camera; The positional microphone is arranged on a first plane; the camera and the non-positional microphone are arranged on a second plane; the first plane and the second plane are planes within the preset coordinate system.

6. The method according to claim 5, characterized in that The method of determining the t-th first spatial coordinate of the moving object based on the ultrasonic wave comprises: Obtaining a play time and a receiving time, and calculating the length of the t-th first connecting line based on the play time, the receiving time and the wave speed; the play time is the play time of the ultrasonic wave; the receiving time is the time of receiving the effective echo; the wave speed is the propagation speed of the ultrasonic wave in the air; Based on the length of the tth first connecting line, the length of the tth third connecting line, the length of the tth fourth connecting line and the length of the tth fifth connecting line are calculated; the tth third connecting line connects the tth first spatial coordinate and the tth fifth spatial coordinate; the tth fifth spatial coordinate is the coordinate of the second microphone in the preset coordinate system; the tth fourth connecting line connects the tth first spatial coordinate and the tth sixth spatial coordinate; the tth sixth spatial coordinate is the coordinate of the third microphone in the preset coordinate system; the tth fifth connecting line connects the tth first spatial coordinate and the tth seventh spatial coordinate; the seventh spatial coordinate is the coordinate of the position microphone in the preset coordinate system; Based on the length of the t-th first connecting line, the length of the t-th third connecting line, the length of the t-th fourth connecting line and the length of the t-th fifth connecting line, the t-th first angle, the t-th second angle and the t-th third angle are calculated; the t-th first angle is the angle between the t-th third connecting line and the t-th first connecting line; the t-th second angle is the angle between the t-th fourth connecting line and the t-th first connecting line; the t-th third angle is the angle between the t-th first connecting line and the z-axis; Based on the tth first angle, the tth second angle and the tth third angle, the tth first space coordinate is calculated.

7. The method according to claim 6, characterized in that The calculating of the tth first angle, the tth second angle and the tth third angle based on the length of the tth first line, the length of the tth third line, the length of the tth fourth line and the length of the tth fifth line comprises: The tth first angle is calculated based on the length of the tth first connecting line, the length of the tth third connecting line, and a first distance; the first distance is the distance between the first microphone and the second microphone; The t-th second angle is calculated based on the length of the t-th first connecting line, the length of the t-th fourth connecting line and a second distance; the second distance is the distance between the first microphone and the third microphone; The tth third angle is calculated based on the length of the tth first connecting line, the length of the tth fifth connecting line and a third distance; the third distance is the distance between the first microphone and the position microphone.

8. The method according to claim 7, characterized in that When the third angle is an acute angle, the positional relationship indicates that the moving object is above the camera; or when the third angle is a right angle, the positional relationship indicates that the moving object and the camera are in the same horizontal plane; or when the third angle is an obtuse angle, the positional relationship indicates that the moving object is below the camera.

9. The method according to claim 7, characterized in that: The calculating the tth first space coordinate based on the tth first angle, the tth second angle and the tth third angle comprises: Based on the tth first angle and the first connecting line, a tth x value is obtained; the tth x value is the distance from the tth first spatial coordinate to the yoz plane; the yoz plane is the plane in the preset coordinate system; Based on the tth second angle and the first connecting line, a tth y value is obtained; the tth y value is the distance from the tth first spatial coordinate to the xoz plane; the xoz plane is the plane in the preset coordinate system; Based on the tth third angle and the first connecting line, the tth z value is obtained; the tth z value is the distance from the tth first spatial coordinate to the xoy plane; the xoy plane is the plane in the preset coordinate system.

10. The method according to claim 9, characterized in that The determining, based on the tth first space coordinate, the tth direction angle and the tth elevation angle of the camera comprises: Calculate the tth pitch angle based on the length of the tth first connecting line and the tth z value; Based on the tth y value and the tth second connecting line, the tth direction angle is calculated.

11. The method according to claim 1, characterized in that: The controlling the camera to track the moving object based on the t-th direction angle and the t-th elevation angle includes: Obtaining the t-1th first difference and the t-1th second difference; the t-1th first difference is the difference between the tth direction angle and the t-1th direction angle; the t-1th second difference is the difference between the tth elevation angle and the t-1th elevation angle; According to the t-1th first difference and the t-1th second difference, the camera is controlled to rotate to the tth target state; the target state indicates that the moving object falls into the shooting area of ​​the camera, and the center of the shooting area is in a straight line with the center of the moving object.

12. The method according to claim 11, characterized in that The controlling the camera to rotate to the t-th target state according to the t-1th first difference and the t-1th second difference includes: If the tth first difference is greater than or equal to a preset threshold, and the tth second difference is greater than or equal to the preset threshold, then based on the tth first difference and the tth second difference, the camera is controlled to rotate to the tth target state; the tth first difference is the difference between the t+1th direction angle and the tth direction angle; the tth second difference is the difference between the t+1th pitch angle and the tth pitch angle; or If the t-th first difference is greater than or equal to the preset threshold, and the t-th second difference is less than the preset threshold, then controlling the camera to rotate to the t-th target state based on the t-th first difference; or If the t-th first difference is less than the preset threshold, and the t-th second difference is greater than or equal to the preset threshold, controlling the camera to rotate to the t-th target state based on the t-th second difference; or If the t-th first difference value is smaller than the preset threshold value, and the t-th second difference value is smaller than the preset threshold value, the camera is not rotated.

13. The method according to claim 12, characterized in that The method further comprises: If a plurality of the moving objects enter the surrounding area of ​​the camera at the same time, obtaining the t-th direction angle and the t-th elevation angle of each of the moving objects; Obtaining the t-th first difference value and the t-th second difference value of each of the moving objects; According to the tth first difference value and the tth second difference value of the kth moving object, control the camera to rotate and photograph the kth moving object for a preset time period; k is an integer greater than or equal to 1; According to the tth first difference and the tth second difference of the k+1th moving object, the camera is controlled to rotate and photograph the k+1th moving object for the preset time period.

14. A system for controlling a camera to track a moving object, characterized in that: include: Includes a camera, a sound wave generator, and a microphone array; The sound wave generator and the microphone array are used to determine the spatial coordinates of the moving object; The spatial coordinates are coordinates of a preset coordinate system; The microphone array includes a position microphone; the position microphone is used to determine the position relationship between the camera and the moving object.

15. The system according to claim 14, characterized in that The microphone array also includes a non-positional microphone; the positional microphone is arranged on a first plane; the camera and the non-positional microphone are arranged on a second plane; the first plane and the second plane are planes within the preset coordinate system.

16. A device for controlling a camera to track a moving object, characterized in that: include: A first determination module, used to determine the tth first spatial coordinate of the moving object based on ultrasonic waves and a microphone array; The first spatial coordinate is a coordinate in a preset coordinate system; t is an integer greater than or equal to 2; the microphone array includes a position microphone; The position microphone is used to determine the position relationship between the camera and the moving object; A second determination module is used to determine the tth direction angle and the tth elevation angle of the camera based on the tth first spatial coordinate; the tth elevation angle is the angle between the tth first connecting line and the tth second connecting line; the tth direction angle is the angle between the tth second connecting line and the x-axis of the preset coordinate system; A control module is used to control the camera to track the moving object based on the tth direction angle and the tth elevation angle.

17. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the method for controlling a camera to track a moving object as described in any one of claims 1 to 13.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the method for controlling a camera to track a moving object according to any one of claims 1 to 13.

Images