Camera control method and related equipment thereof

Abstract

The invention provides a camera control method and related equipment thereof, and the method comprises the steps: detecting an observation region through a radar, and judging whether a target person exists in the observation region or not; if yes, radar point cloud information in the observation area is acquired, and position information of the target person is determined; according to the position information, designated feature information of the target person is obtained, the motion state of the target person is recognized, and the designated feature information comprises at least one of Doppler velocity information, micro-Doppler frequency information and position information; and activating a camera according to the motion state of the target person, and outputting position information of the target person. Through fusion of the radar and the camera, the radar serves as an activation switch for working of the camera, so that the camera is started to work when needed, and the power consumption of the camera is reduced. In addition, the information of the target position can be acquired in real time through the radar, and the target person can be accurately positioned.

Description

Technical Field

[0001] The present invention relates to the field of security, and in particular to a camera control method and related equipment. Background Art

[0002] Camera, also known as "IPC (Internet Protocol Camera)", is a video device used for security protection and monitoring management. Its main uses include real-time monitoring, video storage and security warning. It helps users observe, analyze and process the situation in the monitoring area by capturing and recording images to ensure the safety of people, property and the environment. Combined with modern technologies such as AI, the Internet of Things and big data, the application scope and functions of cameras are constantly expanding and optimizing.

[0003] Since the camera module consumes a lot of power when working normally, the standby time of the wireless battery-powered IPC is strongly related to the start-up time of its camera module. Therefore, how to correctly trigger the low-power camera and locate the target with high precision is an urgent problem to be solved. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a camera control method and related equipment, aiming to solve the technical problem of how to achieve correct triggering of a low-power camera and high-precision positioning of a target.

[0005] In a first aspect, an embodiment of the present invention provides a camera control method, which is applied to a camera, wherein a radar communicates with the camera, and includes:

[0006] Use radar to detect the observation area and determine whether there is a target person in the observation area;

[0007] If yes, obtain the radar point cloud information in the observation area to confirm the location information of the target person;

[0008] According to the position information, the designated characteristic information of the target person is obtained to identify the motion state of the target person, wherein the designated characteristic information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information;

[0009] According to the movement status of the target person, the camera is activated and the location information of the target person is output.

[0010] Preferably, the step of using radar to detect the observation area and determining whether there is a target person in the observation area includes:

[0011] The target distance is calculated by detecting the echo delay, where the formula for calculating the target distance is:

[0012]

[0013] Wherein, fB represents the beat frequency between TX and RX signals, fD represents the Doppler frequency shift caused by the target speed, f1 represents the Doppler frequency shift caused by the target distance, K represents the frequency modulated continuous wave sweep slope, τ represents the echo delay, c represents the speed of light, and R represents the target distance;

[0014] The target speed is calculated by detecting the Doppler frequency shift generated by the target person, where the formula for calculating the target speed is:

[0015]

[0016] Where fD represents the Doppler frequency shift caused by the target speed, λ represents the wavelength, and v represents the target speed;

[0017] According to the intermediate frequency signal, target speed and target distance, it is determined whether there is a micro-moving target or a moving target in the detection area.

[0018] Preferably, if yes, the step of obtaining radar point cloud information within the observation area and confirming the location information of the target person includes:

[0019] According to the distribution of radar point cloud information in two-dimensional space, point cloud feature information is counted, wherein the point cloud feature information includes at least one of radar point count, point cloud density, and valid point count;

[0020] Determine whether the radar point cloud is within the alarm area;

[0021] If yes, the number of point cloud regions is determined, and interference points are removed based on the point cloud density.

[0022] Preferably, the step of determining whether the radar point cloud is within the warning area includes:

[0023] Get the radar detection area and camera monitoring area respectively;

[0024] According to the distribution of radar point cloud in the camera monitoring area, the radar point cloud is divided into invalid area, observation area and alarm area respectively;

[0025] Eliminate the radar point cloud information in the invalid area;

[0026] The radar point cloud information in the observation area is retained, and the radar point cloud information in the observation area is not used to identify the movement state of the target person;

[0027] The radar point cloud information in the observation area is retained, and the radar point cloud information in the observation area is used to identify the motion state of the target person.

[0028] Preferably, the step of acquiring the designated characteristic information of the target person according to the position information and identifying the motion state of the target person includes:

[0029] According to the scene corresponding to the location information, determine whether the target person is a false target;

[0030] If not, the moving direction of the target person is obtained according to the position information, Doppler velocity information, micro-Doppler frequency information, and the trajectory association between multiple frames;

[0031] or,

[0032] If not, the moving direction of the target person is obtained according to the Doppler velocity, where the calculation formula of the Doppler velocity is:

[0033] θ=arctan(Vy / Vx)

[0034] Among them, θ is the angle of the moving direction, Vy is the Doppler velocity of the target, and Vx is the velocity of the target along the X axis.

[0035] Preferably, if not, the step of obtaining the moving direction of the target person according to the position information, the Doppler velocity information, the micro-Doppler frequency information, and the trajectory association between multiple frames includes:

[0036] The detection results of the same target at different time points in multiple frames of data are associated with each other through a specified method to form a continuous trajectory, wherein the specified method includes at least one of reducing feature dimension, reducing computational complexity, precise matching of matching algorithms, and model prediction.

[0037] Preferably, according to the motion state of the target person, the step of activating the camera and outputting the position information of the target person includes:

[0038] The state of the camera is controlled according to the position information and the motion state of the target person, wherein the state of the camera includes at least one of direct activation, delayed activation and inactivation.

[0039] In a second aspect, an embodiment of the present invention provides a camera control system, including:

[0040] The initial inspection module is used to detect the observation area using radar and determine whether there is a target person in the observation area;

[0041] A location confirmation module is used to obtain radar point cloud information within the observation area and confirm the location information of the target person;

[0042] an identification module, used to obtain designated characteristic information of a target person according to the position information, and identify the motion state of the target person, wherein the designated characteristic information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information;

[0043] The activation module is used to activate the camera according to the movement state of the target person and output the position information of the target person.

[0044] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, a method as described in any one of the first aspects is implemented.

[0045] In a fourth aspect, an embodiment of the present invention provides a storage medium having computer program instructions stored thereon, which implements any method of the first aspect when the computer program instructions are executed by a processor.

[0046] In summary, the beneficial effects of the present invention are as follows:

[0047] Through the integration of radar and camera, radar acts as an activation switch for the camera, so that the camera starts working only when needed, reducing the power consumption of the camera. In addition, through radar, the target position information can be obtained in real time, and the target person can be accurately located. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0049] Figure 1 The figure is a flow chart of a camera control method embodiment of the present invention.

[0050] Figure 2 for Figure 1 Schematic diagram of camera installation in the embodiment.

[0051] Figure 3 Schematic diagram of the law wave and echo signal according to an embodiment of the present invention.

[0052] Figure 4 Schematic diagram of the structure of a camera control system according to an embodiment of the present invention.

[0053] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0056] refer to Figure 1 and 2 The present invention provides a camera control method, which is applied to a camera, a radar and a camera communication, comprising:

[0057] S1: Use radar to detect the observation area and determine whether there is a target person in the observation area;

[0058] S2: If yes, obtain the radar point cloud information in the observation area to confirm the location information of the target person;

[0059] S3: acquiring designated characteristic information of the target person according to the position information, and identifying the motion state of the target person, wherein the designated characteristic information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information;

[0060] S4: According to the movement state of the target person, activate the camera and output the location information of the target person.

[0061] In an embodiment of the present invention, the radar is installed in the camera device, and the radar and the camera lens are located in the same plane, facing the observation area. The communication methods between the radar and the camera include but are not limited to the following two: first, the radar is directly connected to the camera through a TTL cable for communication, which is used to wake up the camera to take pictures; second, it can be linked to the camera DSP module through a serial cable for communication, which is used to transmit the target's location information and motion trajectory to the camera DSP module and record it on the memory card. The present invention uses a consumer 60GHz millimeter wave radar SOC chip platform. Taking the AT60LF1T2RS32 chip of a certain technology company as an example, the number of antenna channels is 1Tx and 2Rx, the operating frequency is 59-64GHz, the operating bandwidth is 5GHz, the frame period is 100ms, the 1mw ultra-low power consumption, the maximum detection distance is 30m, the chip supports GPIO and serial port output, and the signal line can be directly connected to the camera, providing a cost-effective radar detection solution. The radar obtains the radar point cloud information in the observation area and confirms the location information of the target person. According to the position information of the target person, the designated feature information of the target person is obtained, and the motion state of the target person is identified, wherein the designated feature information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information. The motion state of the target person is judged from a one-dimensional or multi-dimensional angle, and the radar is used as the activation switch of the camera. According to the motion state and position of the target person, the radar communicates with the camera, and the camera performs inactivation, delayed activation or activation. The system integrates the millimeter wave radar (hereinafter referred to as "radar") with the camera, and the radar is used as the low-power activation switch of the camera. After the radar detects a valid target, it sends a signal through the communication bus to activate the camera to take photos or record videos, thereby achieving high-precision detection and photography of the target. When there is no valid target, the camera enters a dormant state, and power consumption is greatly reduced. The application of millimeter wave radar modules in the field of security, the high-frequency band and high-bandwidth millimeter wave characteristics bring high-precision measurement and good environmental adaptability, which largely solves the market pain points of existing PIR and 24GHz radar module solutions. This solution uses an advanced 60GHz millimeter wave radar module. Compared with PIR and 24GHz radar technology, the 60GHz millimeter wave frequency band radar has high sensitivity, high measurement accuracy, good anti-interference performance, and can provide more target information (distance, speed, azimuth / pitch angle), and realize micro-motion, motion detection and other functions. In addition, the millimeter wave radar has a wide detection range FOV, which can cover the camera detection area. Based on micro-motion and motion characteristics, the radar is not affected by the posture and motion state of the human body, and realizes human body detection in the camera detection area. At the same time, it can filter interference to avoid false alarms or missed alarms. After the radar recognizes that someone has entered the camera's shooting area, it sends a trigger signal to the camera through the communication bus to wake up the camera to take photos or record videos. At the same time, the radar can feedback the target's location information and motion trajectory to the camera in real time for target tracking and record storage, which is convenient for subsequent analysis.In summary, through the integration of radar and camera, radar acts as an activation switch for the camera, so that the camera starts working only when needed, reducing the power consumption of the camera. In addition, through radar, the target position information can be obtained in real time, and the target person can be accurately located.

[0062] Reference Figure 3 , using radar to detect the observation area, and determining whether there is a target person in the observation area, step S2, includes:

[0063] S21: Calculate the target distance by detecting the echo delay, wherein the formula for calculating the target distance is:

[0064]

[0065] Wherein, fB represents the beat frequency between TX and RX signals, fD represents the Doppler frequency shift caused by the target speed, f1 represents the Doppler frequency shift caused by the target distance, K represents the frequency modulated continuous wave sweep slope, τ represents the echo delay, c represents the speed of light, and R represents the target distance;

[0066] S22: Calculate the target speed by detecting the Doppler frequency shift generated by the target person, wherein the formula for calculating the target speed is:

[0067]

[0068] Wherein, fD represents the Doppler frequency shift caused by the target speed, λ represents the wavelength, and v represents the target speed;

[0069] S23: judging whether there is a micro-moving target or a moving target in the detection area according to the intermediate frequency signal, the target speed and the target distance.

[0070] In an embodiment of the present invention, after the radar enters the detection mode, it will periodically (the frame period is usually 100ms) emit millimeter wave band radio waves and receive its reflected signals to measure the position, speed and angle of suspicious targets. The reflected signals received by the radar are not completely still due to the micro-movements of the human body such as breathing and heartbeat, so even if the person is sitting still, it is impossible for the person to be in a completely still state. Therefore, only moving or micro-moving targets need to be detected. If there are no moving or micro-moving targets in the detection area, it is considered that there are no people in the current environment; otherwise, it is considered that there are people in the detection area. The method for detecting human motion or micro-motion characteristics is as follows: millimeter wave radar ranging is mainly to calculate the target distance by detecting the echo delay, and speed measurement is to calculate the target speed by detecting the Doppler frequency shift caused by the target motion. When using the LFCMW (linear frequency modulation continuous wave) signal system, such as Figure 3 As shown, the algorithm principle:

[0071] 1) The echo signal (RX chirp) is a delayed replica of the transmitted signal (TX chirp), and the echo delay τ is linearly related to the beat frequency f1. The echo delay τ can be calculated by the beat frequency f1, and thus the target distance R can be calculated. The calculation formula is as follows:

[0072]

[0073] Wherein, fB represents the beat frequency between TX and RX signals, fD represents the Doppler shift caused by the target speed, f1 represents the Doppler shift caused by the target distance, K represents the frequency modulated continuous wave sweep slope, τ represents the echo delay, c represents the speed of light, and R represents the target distance.

[0074] 2) When the target is moving, the echo signal is not only a time-delayed replica of the transmitted signal, but also has a certain Doppler frequency shift (Doppler effect). By extracting this frequency shift, the target velocity v (also called micromotion) can be calculated.

[0075]

[0076] Where fD represents the Doppler frequency shift caused by the target velocity, λ represents the wavelength, and v represents the target velocity.

[0077] 3) Chest displacement caused by human breathing ("micromotion")

[0078] For linear frequency modulation millimeter wave radar, if the object in front of the radar changes its position by 1mm (for 60GHz radar 1mm≈λ / 5), the phase change of the intermediate frequency signal is 144°. It can be seen that the phase of the intermediate frequency signal is very sensitive to slight changes in the range of the object. The slight movement of the object will change the phase of the intermediate frequency signal, and the micro-motion characteristics of the target can be identified by the changed phase. The chest displacement caused by human breathing is usually 5 to 20mm, which is much larger than the minimum sensitivity of the radar. Therefore, the radar can accurately measure the micro-motion characteristics caused by the chest fluctuation. In summary, this scheme can accurately determine whether there is a micro-motion target or a moving target in the detection area based on the intermediate frequency signal, target speed and target distance.

[0079] Further, if yes, then the step S3 of obtaining radar point cloud information within the observation area and confirming the location information of the target person includes:

[0080] S31: Counting point cloud feature information according to the distribution of radar point cloud information in the two-dimensional space, wherein the point cloud feature information includes at least one of the number of radar points, point cloud density, and the number of valid points;

[0081] S32: Determine whether the radar point cloud is within the alarm area;

[0082] S33: If yes, determine the number of point cloud regions, and remove interference points according to the point cloud density.

[0083] In an embodiment of the present invention, point cloud feature information is counted according to the distribution of radar point cloud information in two-dimensional space, wherein the point cloud feature information includes at least one of radar point count, point cloud density and valid point count. Specifically, the specific statistical method of radar point count is to traverse and count the point cloud to count the number of all radar points. The point cloud density statistical method is to calculate the point cloud density by radar point count / space area in the XY space area detected by the radar. The valid point count statistical method is to count the number of all radar points in the designated alarm area (usually the alarm area is smaller than the radar detection range, set by the user or preset default value). First, it is determined whether the radar point cloud is within the alarm area. If not, the radar point cloud outside the alarm area is proposed because it is an invalid radar point cloud. If the radar point cloud is within the alarm area, the number of point cloud areas is determined, and interference points are proposed according to the point cloud density. Specifically, the number of point clouds needs to be greater than the preset threshold to be valid. For discrete points, if the number of points does not meet the threshold, it is considered to be an invalid point and can be eliminated. If the point cloud is scattered throughout the detection area, it is considered that the micro-motion information detected at this time is due to interference caused by the shaking of debris in the environment or multipath reflection, and there is actually no person. Through the above measures, interference items can be eliminated and the accuracy of radar recognition of target persons can be increased.

[0084] Further, step S32 of determining whether the radar point cloud is within the warning area includes:

[0085] S321: Obtain the radar detection area and the camera monitoring area respectively;

[0086] S322: Divide the radar point cloud into an invalid area, an observation area, and an alarm area according to the distribution of the radar point cloud in the camera monitoring area;

[0087] S223: Eliminate radar point cloud information in the invalid area;

[0088] S224: retaining the radar point cloud information in the observation area, and temporarily not using the radar point cloud information in the observation area to identify the motion state of the target person;

[0089] S325: retaining the radar point cloud information in the observation area, and using the radar point cloud information in the observation area to identify the motion state of the target person.

[0090] In an embodiment of the present invention, the radar has a large FOV detection angle (usually more than 120°), while the camera FOV angle is 90° to 110°, and the FOV of the radar is larger than the camera FOV. The radar used in this solution has multiple Rx channels, and the angle of the target and the distance of the target are obtained by angle measurement. When a target enters the camera FOV, in order to ensure correct activation, the radar detection area is divided into multiple sub-areas, and each sub-area can independently set trigger conditions and alarm strategies. Sub-areas include but are not limited to: 1) Invalid area: The angle of the detection point exceeds the camera FOV area, and the camera does not need to be activated, and the point cloud is directly eliminated; 2) Observation area: The angle of the detection point is within the camera FOV area, but the distance is far (such as the camera focal length is 8 meters, and the current detection point distance is greater than 8 meters). The camera is not activated temporarily, but the detection point is retained; 3) Alarm area: The angle and distance of the detection point are both within the camera FOV area, the detection point is retained, and the next processing flow is directly entered.

[0091] Through the above measures, interference items in radar detection can be eliminated and the accuracy of radar recognition can be further improved.

[0092] Furthermore, step S4 of acquiring designated feature information of the target person according to the position information and identifying the motion state of the target person includes:

[0093] S41: judging whether the target person is a false target according to the scene corresponding to the position information;

[0094] S42: If not, the moving direction of the target person is obtained according to the position information, the Doppler velocity information, the micro-Doppler frequency information, and the trajectory association between multiple frames;

[0095] or,

[0096] S43: If not, the moving direction of the target person is obtained according to the Doppler velocity, wherein the calculation formula of the Doppler velocity is:

[0097] θ=arctan(Vy / Vx)

[0098] Among them, θ is the angle of the moving direction, Vy is the Doppler velocity of the target, and Vx is the velocity of the target along the X axis.

[0099] In an embodiment of the present invention, first, for suspicious targets that do not conform to the scene where people exist, it is determined that they are invalid target types. For example, if a suspicious target appears in an area that is impossible to exist, such as inside an obstacle or outside a wall, it is considered that the target may be a false target caused by the multipath of the interference object, and the target is deleted. Therefore, based on the fusion of multiple features, non-human interference targets can be distinguished to avoid false triggering. Specifically, the method for identifying false targets is as follows: the position information corresponding to the target person is obtained before, combined with the radar installation position and the known camera and the corresponding spatial position of the ground (XY coordinates, which are manually input by the user or self-calibrated when the device is installed). If the target spatial position overlaps with the corresponding spatial position of obstacles, walls, etc., the target can be considered to be an interference target, that is, a false target, and can be deleted. Among them, the XY space and installation method are described: In the present invention, the radar measures the target distance and angle information, and then converts it into an XY two-dimensional coordinate system space with the radar as the center of the coordinate system. False targets are eliminated according to the installation method. The installation method of the security camera is mainly side-mounted, see attached. Figure 3 . When mounted on the side, the radar's XY plane is approximately parallel to the ground. In this installation mode, since there is no target height information, the radar cannot distinguish below the ground. However, obstacles such as walls, trees, and houses in the observation area can be reflected in different coordinate positions of the XY two-dimensional coordinate system. Therefore, when mounted on the side, suspicious targets, that is, false targets, that appear at the locations of obstacles such as walls, trees, and houses can be directly eliminated. For the remaining targets, the target's direction of movement is obtained based on the previously obtained position information, Doppler velocity, micro-Doppler frequency and other characteristics corresponding to the target person, as well as the trajectory correlation between multiple frames. If the target Doppler velocity is zero and there is no displacement within multiple frames, the target can be judged to be a stationary person. If the target has a Doppler velocity and there is displacement within multiple frames, the target can be judged to be a moving person.

[0100] Specifically, the detection results of millimeter wave radars are usually expressed as point targets. For surface targets, it is impossible to directly obtain the contour and movement direction information of the target. However, by accumulating multi-frame point cloud data and performing linear fitting on historical trajectory points, the movement direction of the target can be estimated. The estimation of trajectory point association between multiple frames is a key step in radar target tracking. Its purpose is to associate the detection results of different time points of the same target in multiple frames of data to form a continuous trajectory. In order to take into account the limitations of computing resources and real-time performance, the embodiment of the present invention uses multi-dimensional features (including distance, angle, acceleration, average speed, RCS, etc.) to measure the matching degree between the detection point of the current frame and the trajectory point of the previous frame. In addition, in order to eliminate measurement noise, jitter and abnormal changes and ensure the smoothness and accuracy of the trajectory, trajectory smoothing is also introduced. Since there are certain errors in radar detection data, directly using these data may cause unstable trajectories, thereby affecting the prediction and control of target motion. By smoothing the trajectory points, position fluctuations can be reduced, the accuracy of motion direction estimation can be improved, and the trajectory can be made more consistent with the actual motion trajectory of the target.

[0101] In another embodiment of the present invention, the moving direction of the target person is obtained by the Doppler velocity, and the calculation formula of the Doppler velocity is:

[0102] θ=arctan(Vy / Vx)

[0103] Among them, θ is the angle of the moving direction, Vy is the Doppler velocity of the target, and Vx is the velocity of the target along the X axis.

[0104] Further, if not, step S42 of obtaining the moving direction of the target person according to the position information, the Doppler velocity information, the micro-Doppler frequency information, and the trajectory association between multiple frames includes:

[0105] S421: Associating detection results of the same target at different time points in multiple frames of data through a specified method to form a continuous trajectory, wherein the specified method includes at least one of reducing feature dimension, reducing computational complexity, precise matching of matching algorithms, and model prediction.

[0106] In the embodiment of the present invention, in radar trajectory tracking, a multidimensional feature matching algorithm is a method for efficiently associating radar detection targets with existing trajectories. This algorithm is based on the multidimensional features of the target (such as distance, angle, speed, direction, acceleration, etc.). The newly detected target features are compared with the features of the existing trajectory, and the matching degree is calculated to determine whether they belong to the same target, thereby achieving continuous tracking of the target trajectory. However, in order to balance computing resources and real-time performance, this solution focuses on optimizing the following aspects: 1) Reducing feature dimensions: giving priority to features that contribute the most to trajectory matching (such as distance, angle, and speed), and reducing the use of low-correlation features; 2) Reducing computational complexity: dividing the radar detection area into grids (such as a detection area of ​​10m*10m, divided into 25 sub-grids according to the minimum grid of 2m*2m), and only calculating the sub-grid where the target is located, reducing the participation of irrelevant targets. Set matching range constraints to only calculate targets close to the trajectory prediction position and ignore targets far away from the prediction range; 3) Matching algorithms from simple to complex: first use low-dimensional features (such as distance, angle) for coarse matching to narrow the candidate set, and then use high-dimensional features (such as acceleration, average speed, RCS) for precise matching; 4) Combination of prediction and matching: Use lightweight prediction models for short-term predictions to reduce the computational burden of complex models, and dynamically adjust the prediction complexity according to the target motion state (uniform speed, acceleration, turning, etc.).

[0107] Furthermore, according to the motion state of the target person, the camera is activated to output the position information of the target person in step S5, including:

[0108] S51: Control the state of the camera according to the position information and movement state of the target person, wherein the state of the camera includes at least one of direct activation, delayed activation and inactivation.

[0109] In an embodiment of the present invention, when a person is detected in the detection area, according to a preset camera activation strategy, the radar IO port is controlled to output a trigger signal (for example, when there is a target, a high level is output, and when there is no target, a low level is output), and the camera is activated through a TTL cable; when a person continues to move in the observation area, the radar serial port outputs the target's position information (distance, angle) and motion trajectory data (position information and movement direction of each frame) in real time, which are transmitted to the camera DSP through a cable and recorded in a storage card for subsequent analysis.

[0110] When the radar detects a person in the detection area, the camera activation timing is controlled based on the position information (distance, speed, azimuth angle) and motion trajectory data (position information and motion direction of each frame) measured by the radar, combined with the preset alarm area. The strategies include but are not limited to: 1) Direct activation: If the effective distance of the camera focal length is 10 meters, the radar alarm distance is set to be valid within 10 meters. When a person suddenly appears (such as entering from a door) within 10 meters, the camera is directly activated; 2) Delayed activation: If the effective distance of the camera focal length is 10 meters, the radar alarm distance is set to be valid within 10 meters. When a person enters from 15 meters and is detected by the radar, the camera is temporarily not activated. The radar continuously tracks the person's trajectory. If the person's movement direction is approaching, the camera is activated when the person moves to a distance of 10 meters; 3) Inactivation: If the effective distance of the camera focal length is 10 meters, the radar alarm distance is set to be valid within 10 meters. When a person enters from 15 meters and is detected by the radar, the radar tracks the person's trajectory. If the person's movement direction is moving away, the target is directly eliminated and the camera is not activated; the intelligent activation of the camera is realized through the information of people in different positions and motion states.

[0111] refer to Figure 4 The present invention provides a camera control system, comprising:

[0112] The initial inspection module 1 is used to detect the observation area using radar and determine whether there is a target person in the observation area;

[0113] Position confirmation module 2, for obtaining radar point cloud information within the observation area to confirm the position information of the target person;

[0114] The identification module 3 is used to obtain the designated characteristic information of the target person according to the position information, and identify the motion state of the target person, wherein the designated characteristic information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information;

[0115] The activation module 4 is used to activate the camera according to the movement state of the target person and output the position information of the target person.

[0116] In an embodiment of the present invention, the radar is installed in the camera device, and the radar and the camera lens are located in the same plane, facing the observation area. The communication methods between the radar and the camera include but are not limited to the following two: first, the radar is directly connected to the camera through a TTL cable for communication, which is used to wake up the camera to take pictures; second, it can be linked to the camera DSP module through a serial cable for communication, which is used to transmit the target's location information and motion trajectory to the camera DSP module and record it on the memory card. The present invention uses a consumer 60GHz millimeter wave radar SOC chip platform. Taking the AT60LF1T2RS32 chip of a certain technology company as an example, the number of antenna channels is 1Tx and 2Rx, the operating frequency is 59-64GHz, the operating bandwidth is 5GHz, the frame period is 100ms, the 1mw ultra-low power consumption, the maximum detection distance is 30m, the chip supports GPIO and serial port output, and the signal line can be directly connected to the camera, providing a cost-effective radar detection solution. The radar obtains the radar point cloud information in the observation area and confirms the location information of the target person. According to the position information of the target person, the designated feature information of the target person is obtained, and the motion state of the target person is identified, wherein the designated feature information includes at least one of Doppler velocity information, micro-Doppler frequency information and position information. The motion state of the target person is judged from a one-dimensional or multi-dimensional angle, and the radar is used as the activation switch of the camera. According to the motion state and position of the target person, the radar communicates with the camera, and the camera performs inactivation, delayed activation or activation. The system integrates the millimeter wave radar (hereinafter referred to as "radar") with the camera, and the radar is used as the low-power activation switch of the camera. After the radar detects a valid target, it sends a signal through the communication bus to activate the camera to take photos or record videos, thereby achieving high-precision detection and photography of the target. When there is no valid target, the camera enters a dormant state, and power consumption is greatly reduced. The application of millimeter wave radar modules in the field of security, the high-frequency band and high-bandwidth millimeter wave characteristics bring high-precision measurement and good environmental adaptability, which largely solves the market pain points of existing PIR and 24GHz radar module solutions. This solution uses an advanced 60GHz millimeter wave radar module. Compared with PIR and 24GHz radar technology, the 60GHz millimeter wave frequency band radar has high sensitivity, high measurement accuracy, good anti-interference performance, and can provide more target information (distance, speed, azimuth / pitch angle), and realize micro-motion, motion detection and other functions. In addition, the millimeter wave radar has a wide detection range FOV, which can cover the camera detection area. Based on micro-motion and motion characteristics, the radar is not affected by the posture and motion state of the human body, and realizes human body detection in the camera detection area. At the same time, it can filter interference to avoid false alarms or missed alarms. After the radar recognizes that someone has entered the camera's shooting area, it sends a trigger signal to the camera through the communication bus to wake up the camera to take photos or record videos. At the same time, the radar can feedback the target's location information and motion trajectory to the camera in real time for target tracking and record storage, which is convenient for subsequent analysis.In summary, through the integration of radar and camera, radar acts as an activation switch for the camera, so that the camera starts working only when needed, reducing the power consumption of the camera. In addition, through radar, the target position information can be obtained in real time, and the target person can be accurately located.

[0117] In addition, combined Figure 5 The camera control method according to the embodiment of the present invention described above may be implemented by an electronic device. Figure 5 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown.

[0118] The electronic device may include a processor and a memory storing computer program instructions.

[0119] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0120] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0121] The processor implements any one of the camera control methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0122] In one example, the electronic device may further include a communication interface and a bus. Figure 5 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0123] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0124] Bus includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0125] In addition, in combination with the camera control method in the above embodiment, the embodiment of the present invention can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any camera control method in the above embodiment is implemented.

[0126] In summary, the embodiments of the present invention provide a camera control method and related devices.

[0127] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0128] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0129] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0130] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A camera control method, applied to a camera, a radar communicating with the camera, characterized in that: include: Use radar to detect the observation area and determine whether there is a target person in the observation area; If yes, obtain the radar point cloud information within the observation area to confirm the location information of the target person; According to the position information, obtaining designated feature information of the target person, and identifying the motion state of the target person, wherein the designated feature information includes at least one of Doppler velocity information, micro-Doppler frequency information and the position information; According to the movement state of the target person, the camera is activated to output the position information of the target person.

2. The camera control method according to claim 1, characterized in that: The step of using radar to detect the observation area and determining whether there is a target person in the observation area includes: The target distance is calculated by detecting the echo delay, where the formula for calculating the target distance is: Wherein, fB represents the beat frequency between TX and RX signals, fD represents the Doppler frequency shift caused by the target speed, f1 represents the Doppler frequency shift caused by the target distance, K represents the frequency modulated continuous wave sweep slope, τ represents the echo delay, c represents the speed of light, and R represents the target distance; The target speed is calculated by detecting the Doppler frequency shift generated by the target person, wherein the formula for calculating the target speed is: Wherein, fD represents the Doppler frequency shift caused by the target speed, λ represents the wavelength, and v represents the target speed; According to the intermediate frequency signal, the target speed and the target distance, it is determined whether there is a micro-moving target or a moving target in the detection area.

3. The camera control method according to claim 1 or 2, characterized in that: If so, the step of obtaining radar point cloud information within the observation area and confirming the location information of the target person includes: According to the distribution of the radar point cloud information in the two-dimensional space, point cloud feature information is counted, wherein the point cloud feature information includes at least one of the number of radar points, point cloud density, and the number of valid points; Determine whether the radar point cloud is within the alarm area; If so, the number of point cloud regions is determined, and interference points are removed according to the point cloud density.

4. The camera control method according to claim 3, characterized in that: The step of determining whether the radar point cloud is within the alarm area includes: Get the radar detection area and camera monitoring area respectively; According to the distribution of the radar point cloud in the camera monitoring area, the radar point cloud is divided into an invalid area, an observation area and an alarm area respectively; Eliminating the radar point cloud information in the invalid area; The radar point cloud information in the observation area is retained, and the radar point cloud information in the observation area is not used to identify the motion state of the target person; The radar point cloud information within the observation area is retained, and the radar point cloud information within the observation area is used to identify the motion state of the target person.

5. The camera control method according to claim 4, characterized in that: The step of acquiring the designated characteristic information of the target person according to the position information and identifying the motion state of the target person comprises: Determining whether the target person is a false target according to the scene corresponding to the location information; If not, the moving direction of the target person is obtained according to the position information, the Doppler velocity information, the micro-Doppler frequency information, and the trajectory association between multiple frames; or, If not, the moving direction of the target person is obtained according to the Doppler velocity, wherein the calculation formula of the Doppler velocity is: θ=arctan(Vy / Vx) Among them, θ is the angle of the moving direction, Vy is the Doppler velocity of the target, and Vx is the velocity of the target along the X axis.

6. The camera control method according to claim 5, characterized in that: If not, the step of obtaining the moving direction of the target person according to the position information, the Doppler velocity information, the micro-Doppler frequency information, and the trajectory association between multiple frames includes: The detection results of the same target at different time points in multiple frames of data are associated with each other through a specified method to form a continuous trajectory, wherein the specified method includes at least one of reducing feature dimensions, reducing computational complexity, accurate matching of matching algorithms, and model prediction.

7. The camera control method according to claim 6, characterized in that: The step of activating the camera and outputting the position information of the target person according to the motion state of the target person includes: The state of the camera is controlled according to the position information and the motion state of the target person, wherein the state of the camera includes at least one of direct activation, delayed activation and inactivation.

8. A camera control system, characterized in that: include: The initial inspection module is used to detect the observation area using radar and determine whether there is a target person in the observation area; A location confirmation module, for obtaining radar point cloud information within the observation area to confirm the location information of the target person; an identification module, configured to obtain, according to the position information, designated characteristic information of the target person, and identify the motion state of the target person, wherein the designated characteristic information includes at least one of Doppler velocity information, micro-Doppler frequency information and the position information; The activation module is used to activate the camera according to the motion state of the target person and output the position information of the target person.

9. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, which implement the method according to any one of claims 1 to 7 when the computer program instructions are executed by the processor.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images