Parameter calibration method, target object tracking method, device, and system

Abstract

The present application provides a parameter calibration method, a target object tracking method, a device, and a system. The parameter calibration method includes: receiving a parameter calibration start notification; in response to the parameter calibration start notification, acquiring a first picture captured by a camera; the first picture includes an image of a first object; detecting a first image from the first picture, the first image being an image of the first object; calculating a T value and a P value corresponding to a target pixel in the first image; acquiring a first target object that matches the first object according to the P value; the first target object is an object detected by a radar; finding a first grid where the radar coordinates of the first target object are located; the radar coordinates are coordinates in a radar coordinate system; updating a reference value corresponding to the first grid according to the T value. The present application can improve the situation where only local features of the target object or no target object at all are present in the camera picture when the radar is linked to the camera for target object tracking, and improve the tracking and shooting effect of the camera on the target object.

Description

Technical Field

[0001] This application relates to the technical field of intelligent terminals, and in particular, to a parameter calibration method, a target object tracking method, a device, and a system. Background Art

[0002] The radar-camera linkage system can be used for intrusion monitoring. Its main functions are: monitoring and managing key boundaries or areas to prevent illegal intrusion or accidental entry, and providing real-time preview and tracking of the intrusion behavior of illegal intrusion targets. Since the coordinate systems of the radar (hereinafter referred to as the radar coordinate system) and the camera (hereinafter referred to as the camera coordinate system) are established differently, in order to realize the radar-camera linkage, it is necessary to perform parameter calibration to obtain the conversion relationship between the coordinates in the radar coordinate system and the coordinates in the camera coordinate system (hereinafter referred to as camera coordinates). Thus, after detecting a target object in the radar according to the radar coordinate system, the coordinates of the target object in the radar coordinate system can be converted into camera coordinates in the camera coordinate system, so that the camera can adjust the shooting state based on the converted camera coordinates and accurately track and shoot the image of the target object.

[0003] Currently, common parameter calibration methods include single-point calibration, multi-point calibration, and Global Positioning System (GPS) calibration, etc. However, after calibration by existing calibration methods, when the radar-camera linkage system is used to track a target object, there are situations where only partial feature images of the target object or no target object images at all are present in the images captured by the camera, resulting in poor tracking and shooting effects of the camera for the target object. Summary of the Invention

[0004] This application provides a parameter calibration method, a device, and a system, which can improve the situation where only partial feature images of the target object or no target object images at all are present in the images captured by the camera when the radar-camera linkage system is used to track a target object, and improve the tracking and shooting effects of the camera for the target object.

[0005] This application also provides a target object tracking method, a device, and a system, which can set the camera coordinates of the camera based on the parameter calibration results obtained by the parameter calibration method of this application, and improve the tracking and shooting effects of the camera for the target object when the radar-camera linkage system is used.

[0006] In a first aspect, an embodiment of this application provides a parameter calibration method, including:

[0007] Receiving a parameter calibration start notification;

[0008] In response to the parameter calibration start notification, obtaining a first picture captured by the camera; the first picture includes images of at least one object;

[0009] Detect a first image from the first picture, where the first image is an image of a first object; the first object is one of the at least one object;

[0010] Calculate the T value and the P value corresponding to the target pixel in the first image, where the T value is the T value when the camera captures a second picture, and the P value is the P value when the camera captures the second picture. The second picture is a picture in which the target pixel is located at a specified position in the picture;

[0011] Obtain a first target object that matches the first object according to the P value; the first target object is an object detected by the radar;

[0012] Find the first grid where the radar coordinates of the first target object are located; the radar coordinates are coordinates in the radar coordinate system;

[0013] Update the reference value corresponding to the first grid according to the T value. The reference value is a reference value of the T value of the camera when the radar and the camera are linked.

[0014] In a possible implementation, the updating the reference value corresponding to the first grid according to the T value includes:

[0015] Calculate the average value of the historical T value and the T value corresponding to the target pixel, and use the average value as the reference value. The historical T value is the T value used to update the reference value before updating the reference value according to the T value corresponding to the target pixel.

[0016] In a possible implementation, the obtaining a first target object that matches the first object according to the P value includes:

[0017] Obtain the radar coordinates of the target object detected by the radar;

[0018] Calculate the P value of the target object according to the radar coordinates of the target object;

[0019] Find a target object from the target objects whose difference between the P value and the P value of the first object is less than a preset first threshold, and determine the first target object according to the found target object.

[0020] In a possible implementation, the determining the first target object according to the found target object includes:

[0021] If the number of the found target objects is 1, use the found target object as the first target object;

[0022] If the number of target objects found is greater than 1, search for object images in the object images included in the first picture whose distance from the first image is less than a preset second threshold. If the number of object images found is a first value, where the first value is the number of target objects found minus 1, sort the found target objects in descending order according to the radar coordinates, sort the found object images and the first image in ascending order, and obtain the target object with the same sorting position as the first image as the first target object.

[0023] In a possible implementation, the first grid is a grid in a preset grid terrain map; the preset grid terrain map is obtained by dividing the monitoring area of the radar into grids;

[0024] The method further includes: determining that the proportion of calibrated grids in a specified area of the grid terrain map reaches a preset third threshold, and ending the parameter calibration, where the calibrated grids are grids with the parameter values.

[0025] In a second aspect, an embodiment of the present application provides a target object tracking method, including:

[0026] When a target object is detected, obtain the radar coordinates of the target object; the radar coordinates are coordinates in a radar coordinate system;

[0027] According to the preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system, calculate the PTZ value corresponding to the radar coordinates;

[0028] If a reference value corresponding to the radar coordinates is found, correct the T value in the PTZ value according to the reference value; the reference value is a reference value of the T value of the camera when the radar and the camera are linked.

[0029] Send the corrected PTZ value to the camera, and the corrected PTZ value is used to make the camera take pictures according to the corrected PTZ value.

[0030] In a possible implementation, the correcting the T value in the PTZ value according to the reference value includes:

[0031] Replace the T value in the PTZ value with the reference value.

[0032] In a possible implementation, the finding the reference value corresponding to the radar coordinates includes:

[0033] Find the second grid corresponding to the radar coordinates;

[0034] If the second grid has a corresponding reference value, use the reference value corresponding to the second grid as the reference value corresponding to the radar coordinate;

[0035] If the second grid does not have a corresponding reference value, find the third grid closest to the second grid among the grids with corresponding reference values. If the distance between the third grid and the second grid is not greater than a preset fourth threshold, use the reference value corresponding to the third grid as the reference value corresponding to the radar coordinate.

[0036] In a third aspect, an embodiment of the present application provides a parameter calibration system, including:

[0037] A camera for tracking and photographing an object;

[0038] A radar for generating a radar coordinate of a target object, where the radar coordinate is a coordinate in a radar coordinate system;

[0039] A processing device for receiving a parameter calibration start notification; in response to the parameter calibration start notification, obtaining a first picture captured by the camera; the first picture includes images of at least one object; detecting a first image from the first picture, where the first image is an image of a first object; the first object is one of the at least one object; calculating a T value and a P value corresponding to a target pixel in the first image, where the T value is the T value when the camera captures a second picture, and the P value is the P value when the camera captures a second picture, and the second picture is a picture in which the target pixel is located at a specified position in the picture; obtaining a first target object matching the first object according to the P value; the first target object is an object monitored by the radar; finding a first grid where the radar coordinate of the first target object is located; the radar coordinate is a coordinate in a radar coordinate system; updating the reference value corresponding to the first grid according to the T value, where the reference value is a reference value of the T value of the camera when the radar and the camera are linked.

[0040] In a possible implementation manner, it further includes:

[0041] A client device for detecting a parameter calibration start operation of the user and sending the parameter calibration start notification to the processing device.

[0042] In a fourth aspect, an embodiment of the present application provides a target object tracking system, including:

[0043] A camera for tracking and photographing a target object;

[0044] A radar for generating a radar coordinate of the target object, where the radar coordinate is a coordinate in a radar coordinate system;

[0045] A processing device is configured to obtain the radar coordinates of a target object when the target object is detected; the radar coordinates are coordinates in a radar coordinate system; according to a preset coordinate conversion relationship between the radar coordinate system and a camera coordinate system, calculate the PTZ value corresponding to the radar coordinates; if a reference value corresponding to the radar coordinates is found, correct the T value in the PTZ value according to the reference value; the reference value is a reference value of the T value of the camera when the radar and the camera are linked; send the corrected PTZ value to the camera, and the corrected PTZ value is used to make the camera perform image capture according to the corrected PTZ value.

[0046] In a fifth aspect, an embodiment of the present application provides a processing device, including:

[0047] One or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the method according to any one of the first aspect or the second aspect.

[0048] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when it runs on a computer, it causes the computer to execute the method according to any one of the first aspect or the second aspect.

[0049] In the parameter calibration method of the embodiment of the present application, upon receiving a parameter calibration start notification, in response to the parameter calibration start notification, a first picture captured by the camera is obtained. The first picture includes images of at least one object. A first image is detected from the first picture, and the first image is an image of a first object. The first object is one of the at least one object. Calculate the T value and P value corresponding to the target pixel in the first image. The T value is the T value when the camera captures a second picture, and the P value is the P value when the camera captures a second picture. The second picture is a picture in which the target pixel is located at a specified position in the picture. According to the P value, a first target object matching the first object is obtained. The first target object is an object detected by the radar. Search for the first grid where the radar coordinates of the first target object are located. The radar coordinates are coordinates in the radar coordinate system. Update the reference value corresponding to the first grid according to the T value. The reference value is the reference value of the T value of the camera when the radar and the camera are linked. Thus, the reference values corresponding to several grids in the radar monitoring area are obtained. This reference value is the reference value of the T value of the camera. As long as, during the calibration process, as many grids included in the uneven terrain area in the radar monitoring area as possible have reference values, then when the camera tracks and captures a target object at this grid, the T value of the camera can be corrected based on this reference value, so that the target object image in the image captured by the camera is located at a suitable position in the camera's captured picture. Description of the Drawings

[0050] Figure 1A It is an example diagram of the method for establishing the radar coordinate system in the embodiment of the present application;

[0051] Figure 1B It is another example diagram of the method for establishing the radar coordinate system in the embodiment of the present application;

[0052] Figure 1C It is an example diagram of the grid terrain map of the radar monitoring area in the embodiment of the present application;

[0053] Figure 1D It is an example diagram of the method for establishing the pixel coordinate system in the embodiment of the present application;

[0054] Figure 2A It is an example diagram of a usage scenario of the parameter calibration method in the embodiment of the present application;

[0055] Figure 2B It is another example diagram of an applicable scenario of the parameter calibration method in the embodiment of the present application;

[0056] Figure 3 It is an example diagram of the GUI and data interaction of the parameter calibration method in the embodiment of the present application;

[0057] Figure 4AA flow chart of an embodiment of the parameter calibration method of the present application;

[0058] Figure 4B A flow chart of another embodiment of the parameter calibration method of the present application;

[0059] Figure 5 A flowchart of an embodiment of a target object tracking method of the present application;

[0060] Figure 6 This is a schematic diagram of the structure of an embodiment of a parameter calibration device of the present application;

[0061] Figure 7 This is a schematic diagram of the structure of an embodiment of a target object tracking device of the present application. DETAILED DESCRIPTION

[0062] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0063] First, the nouns involved in the embodiments of the present application are described in an illustrative but non-limiting manner.

[0064] In the existing implementation schemes, the parameter calibration methods include single-point calibration, multi-point calibration and GPS calibration. The single-point calibration method is mainly used when the horizontal distance between the installation positions of the radar and the camera is 0m. The parameter calibration between the radar and the camera is completed by collecting a set of calibration point data (including the coordinates of the calibration point in the radar coordinate system, and the coordinates of the camera in the camera coordinate system when the calibration point image is located at the center point of the camera image). The multi-point calibration method has no restrictions on the installation position relationship between the radar and the camera, and the parameter calibration between the radar and the camera is completed by collecting more than three sets of calibration point data. The GPS calibration method has no restrictions on the installation position relationship between the radar and the camera. The parameter calibration between the radar and the camera is completed by collecting the GPS data corresponding to the installation positions of the radar and the camera and collecting a set of calibration point data. However, after completing parameter calibration by the existing parameter calibration method (i.e., obtaining the conversion relationship between the coordinates of the radar coordinate system and the coordinates of the camera coordinate system, hereinafter referred to as the coordinate conversion relationship), when the radar is linked to the camera to track the target object, there is a situation where the camera's captured image only contains a local feature image of the target object or no target object image at all, and the main goal of the camera tracking shooting is to capture the complete target object image as much as possible, and the center point of the target object image is as close to or coincident with the center point of the captured image as possible, so the camera's tracking shooting effect on the target object is poor. For example, assuming that the target object is a person, there is a situation where the image captured by the camera only includes a partial image of the target person or no image of the target person at all.

[0065] After the inventor's statistics, it is found that when the target object is in the uneven terrain area in the radar monitoring area, it is easy to have the situation that the target object image in the image captured by the camera is incomplete or there is no target object image. Moreover, most of the incomplete target objects are incomplete in the vertical direction. For example, there is only the image of the upper half of the target object, only the image of the middle part of the target object, or only the image of the lower half of the target object, etc. The main reason for this situation is that the calibration methods such as single-point calibration, multi-point calibration, and GPS calibration in the prior art can only be applied to the case where the terrain of the radar monitoring area is flat. That is to say, only when the target object is in the flat terrain area, the camera shooting coordinates are calculated by using the coordinate conversion relationship obtained by the parameter calibration method of the prior art, and the shooting state of the camera is adjusted, can the target object image in the image captured by the camera be located at the appropriate position in the image, achieving a better shooting effect. If the target object is in the uneven terrain area, and the shooting coordinates are still calculated based on the above method to adjust the shooting state of the camera, since the vertical height of the position where the target object is located is higher or lower than the ground plane of the flat terrain area, the situation that the camera cannot capture the complete image of the target object or even cannot capture the target object image occurs.

[0066] Therefore, the embodiment of the present application proposes a parameter calibration method, a processing device and a system, which can improve the situation that only local features of the target object or no target object at all appear in the camera shooting picture when the radar-linked camera tracks the target object, and improve the tracking shooting effect of the camera on the target object when the radar-linked camera is used.

[0067] Furthermore, the embodiment of the present application also proposes a target object tracking method, a processing device and a system, which can set the camera shooting coordinates based on the calibration result obtained by the parameter calibration method of the embodiment of the present application, and improve the tracking shooting effect of the camera on the target object when the radar-linked camera is used.

[0068] First, an exemplary rather than limiting description is given to the nouns involved in the embodiment of the present application.

[0069] Radar: Radar, the transliteration of the English word Radar, originated from the abbreviation of radio detection and ranging, which means "radio detection and ranging", that is, using radio methods to detect targets and determine their spatial positions. Therefore, radar is also called "radio positioning". Radar is an electronic device that uses electromagnetic waves to detect targets. Radar emits electromagnetic waves to irradiate the target and receives its echo, thereby obtaining information such as the distance from the target to the electromagnetic wave emission point, the rate of change of distance (radial velocity), azimuth, altitude, etc. According to the radar frequency band, it can be divided into over-the-horizon radar, microwave radar, millimeter-wave radar, etc.

[0070] Camera: A camera converts optical image signals into electrical signals for storage or transmission. When shooting an object, the light reflected from the object is collected by the camera lens and focused on the light-receiving surface of the camera device. The camera device then converts the light into electrical energy, which is a "video signal". The photoelectric signal is very weak and needs to be amplified by a pre-amplifier circuit, and then processed and adjusted by various circuits. The final standard signal can be sent to a recording medium such as a video recorder for recording, or transmitted through a transmission system or sent to a monitor for display. Spherical camera: It is called spherical only because of its appearance. It can freely adjust the rotation angle P, pitch angle T, and zoom factor Z of the viewing angle. It is a PTZ camera suitable for target tracking and capture, as well as defense area surrounds.

[0071] The radar coordinate system is a coordinate system established based on the radar. In the embodiment of the present application, the radar coordinate system is taken as an example as a two-dimensional coordinate system. For details, see Figure 1A As shown, the projection point O of the physical center point O' of the radar 10 on the horizontal plane 20 can be used as the origin of the radar coordinate system. Figure 1B As shown, the monitoring area of the radar on the horizontal plane 20 is, for example Figure 1B The sector area shown in the figure can be taken as the positive direction of the y-axis, and the direction Ox perpendicular to Oy can be taken as the positive direction of the x-axis. Then, each position A in the radar monitoring area can be calculated to obtain its coordinates (x, y) in the radar coordinate system. In the embodiment of the present application, the coordinates in the radar coordinate system are referred to as radar coordinates. It should be noted that Figure 1B The radar coordinate system is only an example and is not intended to limit the method of establishing the radar coordinate system. For example, the direction of the bisector of the central angle of the sector area can be used as the positive direction of the x-axis, or the boundary line of the sector passing through the pole O can be used as the positive direction of the x-axis, etc. Figure 1B Different radar coordinate systems. It should be noted that, based on the conversion principle between polar coordinate system and rectangular coordinate system, the radar coordinate system can also be a polar coordinate system, and the specific establishment method will not be repeated in the embodiment of this application.

[0072] In the embodiment of the present application, a gridded topographic map of the radar monitoring area is established in the radar, and the radar monitoring area is divided into a number of sub-areas to facilitate the subsequent recording of preset parameters corresponding to the radar coordinates of different positions in the radar monitoring area. In the embodiment of the present application, the sub-areas obtained by division are referred to as grids. The areas of the grids obtained by division can be the same or different. For the convenience of management and calculation, it is preferred that the areas of the grids are the same. For example Figure 1C As shown, a method for establishing a gridded terrain map of a radar monitoring area is shown. Specifically: the radar monitoring area is a fan-shaped area, and a circumscribed rectangle ABCD of the radar monitoring area is established. The rectangle ABCD is evenly divided into m rows and n columns to obtain mn rectangular grids.Figure 1C The values of m and n being both 4 are only examples and are not used to limit the specific values of m and n. Figure 1C Dividing the radar monitoring area into rectangular grids is only an example and is not used to limit the way of dividing the grids in the parameter calibration method of the embodiments of the present application. For example, the rectangle ABCD can also be divided into oblique grids, honeycomb grids, etc. The radar can record each grid by recording the radar coordinates of the specified point, such as the center point, of each grid. It should be noted that the number of small areas obtained by dividing the radar monitoring area is not limited in the embodiments of the present application. However, theoretically speaking, the more small areas are divided, the more accurate the subsequent parameter calibration result will be, and when the radar is linked with the camera based on the parameter calibration result, the tracking and shooting effect of the camera on the target object will be better.

[0073] The camera coordinate system is a coordinate system established for the shooting state of the camera. In the embodiments of the present application, it is taken as an example that the camera coordinate system includes three dimensions: the rotation angle P, the pitch angle T, and the zoom ratio Z. Specifically, there are 3 motors in the camera. In the embodiments of the present application, the 3 motors are respectively called the horizontal motor, the vertical motor, and the zoom motor. Among them, the horizontal motor is used to control the rotation of the camera lens in the horizontal direction, so that the shooting direction of the camera rotates in the horizontal direction. The vertical motor is used to control the rotation of the camera lens in the vertical direction, so that the shooting direction of the camera rotates in the vertical direction. The zoom motor is used to drive the movement of the focusing lens group in the camera lens to change the zoom ratio of the camera lens. The moving distance of the focusing lens group relative to the initial position corresponds to the zoom ratio of the camera lens. Here, the rotation angle refers to the rotation angle of the horizontal motor relative to the horizontal initial direction; the pitch angle refers to the rotation angle of the vertical motor relative to the vertical initial direction; the zoom ratio refers to the distance that the zoom motor drives the focusing lens group to move relative to the initial position of the focusing lens group. The camera lens has an initial position, which can be set when the camera leaves the factory or set independently by the user during use. Correspondingly, in the initial position of the lens, the horizontal motor, the vertical motor, and the focusing lens group all have their own initial positions. The camera can obtain the rotation angle of the camera by obtaining the rotation angle of the horizontal motor relative to the horizontal initial position, obtain the pitch angle of the camera by obtaining the rotation angle of the vertical motor relative to the vertical initial position, and obtain the zoom ratio of the camera by obtaining the distance that the zoom motor drives the focusing lens group to move. In the embodiments of the present application, the coordinates in the camera coordinate system corresponding to the camera shooting are called PTZ values. The PTZ values in the camera coordinate system are used to record the shooting state information of the camera.

[0074] The pixel coordinate system is a coordinate system established for the pictures taken by the camera. See Figure 1D, in the embodiments of the present application, from the perspective of the user viewing the picture, the upper left vertex of the picture is used as the origin O, the right direction along the upper boundary of the picture is used as the positive direction of the x-axis, and the downward direction along the left boundary of the picture is used as the positive direction of the y-axis. Correspondingly, each pixel in the picture can obtain its coordinates in the pixel coordinate system. In the embodiments of the present application, the coordinates in the pixel coordinate system are referred to as pixel coordinates.

[0075] Through parameter calibration methods such as single-point calibration, multi-point calibration, or GPS calibration in the prior art, the coordinate conversion relationship between the radar coordinates in the radar coordinate system and the camera coordinates in the camera coordinate system can be obtained. For example, given a coordinate (x i , y i ) in the radar coordinate system, a camera coordinate (P i , T i , Z i ) in the camera coordinate system can be obtained based on this conversion relationship. In the prior art, after the parameter calibration is completed (that is, the coordinate conversion relationship between the polar coordinates in the radar coordinate system and the camera coordinates in the camera coordinate system is obtained), if the radar determines the target object, the radar coordinates of the target object in the radar coordinate system can be obtained, the radar coordinates can be converted into camera coordinates according to the above coordinate conversion relationship, and the camera coordinates are sent to the camera, and the camera is adjusted to the shooting state indicated by the camera coordinates for shooting. When the target object is located in a flat area, the target object image is located at a suitable position in the image captured by the camera, achieving a better shooting effect; while if the target object is located in an uneven area, there may be only a partial feature image of the target object or no target object image at all in the image captured by the camera, and the tracking shooting effect of the camera on the target object is poor.

[0076] Therefore, in addition to obtaining the coordinate conversion relationship between the radar coordinates in the radar coordinate system and the camera coordinates in the camera coordinate system by using the parameter calibration method of the prior art, the radar and the camera also execute the parameter calibration method of the embodiments of the present application to obtain another conversion relationship of the T value between the radar coordinates in the radar coordinate system and the camera coordinates in the camera coordinate system. When the target object is in an uneven area, the T value in the camera coordinates calculated by using the coordinate conversion relationship is corrected by using the conversion relationship of the embodiments of the present application, so that the target object image can be located at a suitable position in the image captured by the camera.

[0077] First, an example of the usage scenario of the parameter calibration method of the embodiments of the present application is given. Such as Figure 2AAs shown in the figure, it may include: a client device 21, a radar 22, a camera 23, and a processing device 24. Communication connections are respectively established between the processing device 24 and the client 21, the radar 22, and the camera 23 for data communication. The communication connection method may be a wired connection or a wireless connection, which is not limited in the embodiments of the present application. Among them, the radar 22 can be used to: if an object is detected, generate an object identifier and position information of the object; the camera 23 can be used to: take pictures of the object detected by the radar 22; the client device 21 can be used to: provide a graphical user interface (GUI, Graphical User Interface) for the user to realize the interaction between the user and the client device 21. For example, the client 21 can display a monitoring image generated based on the object identifier and position information detected by the radar 22 to the user, and / or display the image captured by the camera 23 to the user; the processing device 24 can be used to execute the parameter calibration method in the embodiments of the present application.

[0078] Optionally, the processing device 24 exists independently, or is integrated in the radar 22. Optionally, the above-mentioned client device 21, radar 22, camera 23, and processing device 24 exist independently, or the radar 22 and the processing device 24 are integrated into an all-in-one machine, or the radar 22, camera 23, and processing device 24 are integrated into an all-in-one machine, or the client device 21, radar 22, camera 23, and processing device 24 are integrated into an all-in-one machine.

[0079] Taking the example that the processing device 24 is integrated in the radar 22, as Figure 2B shown in the figure, it may include: a client device 21, a radar 22, and a camera 23; among them, communication connections are respectively established between the client device 21, the radar 22, and the camera 23 for data communication. The communication connection method may be a wired connection or a wireless connection, which is not limited in the embodiments of the present application. The client device 21 can be used to provide a graphical user interface (GUI, Graphical User Interface) for the user to realize the interaction between the user and the radar 22 and / or the user and the camera 23. For example, the client device 21 can display a monitoring image generated based on the target object position information sent by the radar 22 to the user, and / or display the image captured by the camera 23 to the user; and, the client device 21 can receive the operation instruction of the user for the radar 22, send the operation instruction to the radar 22 to realize the control of the radar 22 by the user, and / or the client device 21 can receive the operation instruction of the user for the camera 23, send the operation instruction to the camera 23 to realize the control of the camera 23 by the user. Among them, the client device 21 can be an electronic device such as a computer or a tablet (Pad, portable android device).

[0080] Figure 3 is the GUI of the parameter calibration method in the embodiments of the present application and an example diagram of data interaction. This example diagram is based on Figure 2B the shown system architecture to exemplarily illustrate the parameter calibration method provided in the embodiments of the present application.

[0081] Refer to Figure 3 as shown in part 31 in Figure 3 . The user enters the parameter calibration start interface in the client device 21 and performs a selection operation on the "Calibration Start" control in the parameter calibration start interface. Optionally, this selection operation can be implemented by the user performing a finger click operation as shown in Figure 3 or by the user operating the mouse to perform a click operation. Correspondingly, the client device 21 detects the user's selection operation on the "Calibration Start" control and sends a parameter calibration start notification to the radar 22 and the camera 23. Correspondingly, the radar 22 and the camera 23 respectively receive the parameter calibration start notification and, in response to this notification, perform data interaction between them to complete the parameter calibration. During this process, the first object can move in the monitoring area of the radar 22. Optionally, the first object can preferably move in the uneven area of the monitoring area of the radar 22 so that the camera parameters corresponding to the uneven area can be better learned during the parameter calibration. Subsequently, the radar and the camera can better capture the target object based on these parameters when tracking and photographing the target object. It should be noted that the first object can be one or more.

[0082] Refer to Figure 3 as shown in part 32 in Figure 3 . After the parameter calibration starts, the client device can set a "Calibration End" control in the interface presented to the user. When the user decides to end the parameter calibration process, the user can perform a selection operation on the "Calibration End" control in the interface presented by the client device 21. Optionally, this selection operation can be implemented by the user performing a finger click operation as shown in Figure 3 or by the user operating the mouse to perform a click operation. Correspondingly, the client device 21 detects the user's selection operation on the "Calibration End" control and sends a parameter calibration end notification to the radar 22 ( not shown in Figure 3 ) to notify the radar 22 to end the parameter calibration process of the embodiments of the present application.

[0083] In another possible implementation, the end of the parameter calibration process may not be triggered by the user, but by the processing device automatically ending the parameter calibration process after detecting that the parameter calibration meets a preset condition.

[0084] The following is a more detailed description of the parameter calibration method of the present application through Figure 4A the following. Figure 4A FIG. is a flowchart of an embodiment of the parameter calibration method according to an embodiment of the present application. This method can be applied to a radar including a processing device or integrated with a processing device, such as Figure 4A shown, and this method may include:

[0085] Step 401: Receive a parameter calibration start notification.

[0086] This step may correspond to Figure 3 part 31 in. At this time, this step may include: receiving a parameter calibration start notification sent by an electronic device, where the parameter calibration start notification is sent when the electronic device detects a user's selection operation on the first control. In Figure 3 an example where the first control is a "calibration start" control is taken.

[0087] Step 402: In response to the parameter calibration start notification, obtain a first picture captured by the camera; the first picture includes images of at least one object.

[0088] Among them, the first picture may be a video frame captured by the camera. The at least one object may be a calibration object set by the user for parameter calibration, or an object in the actual physical environment.

[0089] Step 403: Detect a first image from the first picture, where the first image is an image of a first object.

[0090] Optionally, a first model may be preset. The first model is a pre-trained model for detecting object images in pictures. Specifically, pictures with bounding boxes of object images marked on them can be used as samples to input into the initial model for model training to obtain the above-mentioned first model. The initial model may be a deep learning network. The input of the first model may be a picture, and the output is the bounding boxes of each object image in the picture.

[0091] If the bounding box is a rectangle, the pixel coordinates of the two diagonal vertices of the bounding box can be used to record a bounding box.

[0092] It should be noted that multiple object images may be detected from the picture. Each object image can be used as the image of the first object, or an object image can be selected as the image of the first object.

[0093] Step 404: Calculate the T value and P value corresponding to the target pixel in the first image; the T value is the T value when the camera can capture the second picture, and the P value is the P value when the camera can capture the second picture. The second picture is the picture where the target pixel is located at the specified position in the picture.

[0094] In a possible implementation, the target pixel can be the midpoint of the lower edge of the first image. If the first image is identified by an external bounding box, the target pixel can be the midpoint of the lower edge of the bounding box; the above-mentioned specified position in the picture can be the center point of the picture.

[0095] In this step, it is not necessary to actually adjust the PT values of the camera to make the camera actually capture the second picture. The T value and P value corresponding to the above target pixel can be directly calculated according to the PT values of the camera when shooting the first picture. Taking the target pixel as the midpoint of the lower edge of the first image and the specified position in the picture as the center point of the picture as an example, it illustrates how to calculate the P value and T value corresponding to the target pixel:

[0096] According to the Z value of the camera when shooting the first picture, determine the field of view angle range of the camera corresponding to the Z value. The field of view angle range includes: the range H of the T value and the range V of the P value. The method for determining the field of view angle range of the camera corresponding to the Z value is not limited in this embodiment of the present application. For example: In a possible implementation, the field of view angle range can be determined by looking up a table. The corresponding table of any zoom ratio of the camera and the field of view angle range can be obtained through pre-actual measurement. When using it, the field of view angle range corresponding to the Z value of the camera when shooting the first picture can be obtained by looking up the table; in another possible implementation, the field of view angle range can be calculated according to the camera internal parameters. The specific calculation formula is as follows:

[0097]

[0098]

[0099] Among them, f is the focal length corresponding to the Z value, h is the horizontal width of the camera target surface, and v is the vertical height of the camera target surface. The camera target surface refers to the photosensitive area of the image sensor in the camera.

[0100] Calculate the rotation angle range of the first shooting direction of the camera according to the P value, T value when shooting the first picture, and the obtained field of view angle range Pitch angle range P is the P value when the camera shoots the first picture, and T is the T value when the camera shoots the first picture.

[0101] For any pixel (u, v) in the first image, let the image width be imgW and the height be imgH, both of which are known quantities. At this time, the P value and T value corresponding to the target pixel can be calculated. In the following formulas, p represents the P value corresponding to the target pixel, and t represents the T value corresponding to the target pixel:

[0102] p = P + H * (u - imgW / 2) / imgW, and the first rotation angle t = T + V * (v - imgH / 2) / imgH. The above first shooting direction is the shooting direction when the camera captures the first image.

[0103] It should be noted that the above method for calculating the P value and T value of the target pixel is only an example, and other methods can also be used to calculate the P value and T value of the above target pixel.

[0104] Step 405: Obtain the first target object that matches the first object according to the P value; the first target object is the object detected by the radar.

[0105] This step may include:

[0106] Obtain the radar coordinates of the target object detected by the radar;

[0107] Calculate the P value of the target object according to the radar coordinates of the target object;

[0108] Search for the target object whose difference between the P value and the P value of the first object is less than the preset first threshold from the target objects, and determine the first target object according to the found target object.

[0109] Among them, there may be a situation where no target object is found. At this time, step 402 can be directly returned to obtain the next image captured by the camera, or step 403 can be returned to obtain the image of the next first object from the first image.

[0110] Among them, if the positions of multiple objects are relatively close, there may be a situation where multiple target objects are found. Therefore, the above determining the first target object according to the found target object may include:

[0111] If the number of found target objects is 1, take the found target object as the first target object;

[0112] If the number of found target objects is greater than 1, search for the object image whose distance from the first image is less than the preset second threshold from the object images included in the first image. If the number of found object images is the first value, the first value is the number of found target objects minus 1, sort the found target objects in descending order according to the radar coordinates, sort the found object images and the first image in ascending order, and obtain the target object with the same sorting position as the first image as the first target object.

[0113] Among them, if the number of objects found is not the first number, step 402 can be directly returned to obtain the next picture captured by the camera; or, step 403 can be returned to obtain the image of the next first object from the first picture.

[0114] Step 406: Search for the first grid where the radar coordinates of the first target object are located.

[0115] Among them, if the grid is recorded by the radar coordinates of the center point of the grid, the grid whose radar coordinates of the center point are closest to the radar coordinates of the first target object is the above-mentioned first grid.

[0116] Step 407: Update the reference value corresponding to the first grid according to the T value corresponding to the target pixel, and the reference value is the reference value of the T value of the camera when the radar and the camera are linked.

[0117] Optionally, updating the reference value corresponding to the first grid according to the T value may include:

[0118] Calculate the average value of the historical T value and the T value corresponding to the target pixel, and use the average value as the reference value. The historical T value is the T value used to update the reference value before updating the reference value according to the T value corresponding to the target pixel.

[0119] Specifically, the calculation of the above average value T2 can be implemented by the following formula:

[0120] T2 = ((T1 * N) + Tx) / (N + 1);

[0121] Among them, T1 is the reference value before update, Tx is the T value corresponding to the target pixel, and N is the historical update times of this reference value before update.

[0122] By repeatedly executing the above steps 402 to step 407, the reference values corresponding to several grids of the radar can be obtained. This parameter value is the reference value of the T value of the camera. As long as the first object moves in the uneven terrain area in the radar monitoring area as much as possible during the calibration process, the processing device can automatically learn the reference values of the grids corresponding to the uneven terrain area, so that when the camera tracks and shoots the target object at this grid, the T value of the camera can be corrected based on this reference value, so that the target object image in the image captured by the camera is located at a suitable position in the image captured by the camera.

[0123] See Figure 4B As shown, in order to make the reference value obtained during the parameter calibration process more accurate, between step 403 and step 404, the following may further be included:

[0124] Step 400: Determine whether the detected first image is a complete image of the first object. If it is, execute Step 404; if not, execute Step 408.

[0125] This step may include:

[0126] Judge whether the lower edge of the first image coincides with the lower edge of the first picture. If they coincide, determine that the detected first image is not a complete image of the first object; if not, determine that the first image is a complete image of the first object.

[0127] Step 408: Adjust the T value of the camera, return to Step 402 to re-obtain the first picture taken by the camera after the T value is adjusted, and detect the image of the first object from the first picture. Repeat this process until it is determined that the first image is a complete image of the first object.

[0128] Among them, when adjusting the T value of the camera, the image of the first object can be moved in the negative y-axis direction in the pixel coordinate system, so that after adjusting the T value of the camera several times, the image of the first object included in the first picture taken by the camera is a complete image of the first object. When adjusting the T value of the camera, it can be adjusted according to a preset step size, and the specific value of the step size is not limited in the embodiments of the present application.

[0129] For the re-obtained first picture, detecting the image of the first object from the first picture may include:

[0130] Detect the object images in the re-obtained first picture to obtain the first pixel coordinates of the specified pixel points of each object image. According to the second pixel coordinates of the specified pixel points of the first image detected from the first picture last time, find the first pixel coordinate with the shortest distance to the second pixel coordinate from the first pixel coordinates, and use the object image corresponding to the first pixel coordinate as the image of the first object in the re-obtained first picture. The specified pixel point can be any point of the object image, preferably points at special positions such as the center point of the image and the midpoint of the lower edge.

[0131] By adjusting the T value of the camera to make the complete image of the first object detected in the first picture, it is easier to make the second picture include as much as possible the complete image of the first object when calculating the T value and P value of the target pixel in Step 404, thereby optimizing the parameter values corresponding to the grid.

[0132] Optionally, refer to Figure 4B , an end condition for parameter calibration can be preset for the processing device. At this time, after Step 407, the method may further include:

[0133] Step 409: Determine that the proportion of calibrated grids in the specified area of the preset grid terrain map reaches a preset third threshold, and end the parameter calibration. The calibrated grids are the grids with corresponding parameter values.

[0134] If the proportion of calibrated grids in the specified area does not reach the preset third threshold, it is possible to return to Step 402 to re-obtain the first picture taken by the camera after the T value is adjusted, and detect the first image from the first picture, and so on in a loop until the proportion of calibrated grids in the specified area reaches the preset third threshold, and then end the parameter calibration.

[0135] Since there may be some areas in the monitoring area of the radar where it is not easy for target objects to appear. For example, a wall divides the monitoring area of the radar into two sub-areas. One sub-area is a forest with few people, and the other sub-area is the area where people are active. At this time, the area corresponding to the above-mentioned other sub-area in the grid terrain map can be set as the specified area, so that the end condition of the parameter calibration is more reasonable. Or, if only part of the area in the radar monitoring area is an uneven terrain area, the area corresponding to the uneven terrain area in the grid terrain map can also be set as the specified area, so that the end condition of the parameter calibration is more reasonable.

[0136] The third threshold is a value greater than 0 and less than or equal to 1, and the specific value is not limited in the embodiments of the present application. The larger the third threshold, the more calibrated grids there are in the specified area. Correspondingly, after the parameter calibration is completed, when the radar and the camera are linked, there can be more reference values to correct the T value of the camera, so that when the target object is in an uneven terrain area, the target object image can be in a suitable position in the picture taken by the camera.

[0137] Figure 5 It is a flowchart of an embodiment of the target object tracking method of the present application. This method can be applied to a radar, as Figure 5 shown, this method may include:

[0138] Step 501: When a target object is detected, obtain the radar coordinates of the target object.

[0139] The target object here is an intrusion object detected by the radar, or a monitoring object specified by the user for the radar.

[0140] Step 502: Calculate the PTZ value corresponding to the radar coordinates according to the preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system.

[0141] Among them, the preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system can be obtained by calibrating the radar and the camera using calibration methods such as single-point calibration, multi-point calibration, and GPS calibration in the prior art.

[0142] Step 503: If a reference value corresponding to the radar coordinates is found, correct the T value in the PTZ value according to the reference value; the reference value is the reference value of the T value of the camera when the radar and the camera are linked.

[0143] In this step, the corresponding relationship between the grid obtained by using the above Figure 4A and Figure 4B parameter calibration method and the reference value can be used to achieve this. Specifically, finding the reference value corresponding to the radar coordinates in this step may include:

[0144] Finding the second grid corresponding to the radar coordinates;

[0145] If the second grid has a corresponding reference value, use the reference value corresponding to the second grid as the reference value corresponding to the radar coordinates;

[0146] If the second grid does not have a corresponding reference value, find the third grid closest to the second grid among the grids with corresponding reference values. If the distance between the third grid and the second grid is not greater than a preset fourth threshold, use the reference value corresponding to the third grid as the reference value corresponding to the radar coordinates.

[0147] Among them, if the distance between the third grid and the second grid is greater than the preset fourth threshold, the PTZ value may not be corrected, and the PTZ value may be directly sent to the camera.

[0148] Correcting the T value in the PTZ value according to the reference value in this step may include:

[0149] Using the reference value to replace the T value in the PTZ value.

[0150] Step 504: Send the corrected PTZ value to the camera, and the corrected PTZ value is used to make the camera capture images according to the corrected PTZ value.

[0151] Figure 5 In the method shown, the conversion relationship between the radar coordinate system and the camera coordinate system obtained by using the parameter calibration method in the prior art, and the reference value corresponding to the grid polar coordinates obtained by the parameter calibration method of the embodiment of the present application are used together to determine the camera coordinates used by the camera to capture the target object, so as to ensure that no matter whether the target object is on a flat terrain or an uneven terrain, the target object image can be in a suitable position in the image captured by the camera.

[0152] It can be understood that some or all of the steps or operations in the above embodiments are only examples. The embodiments of the present application can also perform other operations or various deformations of the operations. In addition, each step can be executed in a different order presented in the above embodiments, and it is possible not to execute all the operations in the above embodiments.

[0153] Figure 6 The structural diagram of an embodiment of the parameter calibration device for this application. This device can be applied to a processing device or a radar integrated with a processing device, such as Figure 6 As shown, the device 60 may include:

[0154] A receiving unit 61, configured to receive a parameter calibration start notification;

[0155] A first acquisition unit 62, configured to, in response to the parameter calibration start notification, acquire a first picture captured by the camera; the first picture includes images of at least one object;

[0156] A detection unit 63, configured to detect a first image from the first picture, the first image being an image of a first object; the first object is one of the at least one object;

[0157] A calculation unit 64, configured to calculate the T value and the P value corresponding to a target pixel in the first image, the T value being the T value when the camera captures a second picture, the P value being the P value when the camera captures a second picture, and the second picture being a picture in which the target pixel is located at a specified position in the picture;

[0158] A second acquisition unit 65, configured to acquire a first target object that matches the first object according to the P value; the first target object is an object detected by the radar;

[0159] A search unit 66, configured to search for a first grid where the radar coordinates of the first target object are located; the radar coordinates are coordinates in the radar coordinate system;

[0160] An update unit 67, configured to update the reference value corresponding to the first grid according to the T value, the reference value being the reference value of the T value of the camera when the radar and the camera are linked.

[0161] In a possible implementation, the update unit 67 may specifically be configured to: calculate the average value of the historical T value and the T value corresponding to the target pixel, and use the average value as the reference value, where the historical T value is the T value used to update the reference value before updating the reference value according to the T value corresponding to the target pixel.

[0162] In a possible implementation, the second acquisition unit 65 may specifically be configured to: acquire the radar coordinates of the target object detected by the radar; calculate the P value of the target object according to the radar coordinates of the target object; search for a target object from the target objects where the difference between the P value and the P value of the first object is less than a preset first threshold, and determine the first target object according to the found target object.

[0163] In a possible implementation, the second acquisition unit 65 may specifically be configured to: if the number of target objects found is 1, use the found target object as the first target object; if the number of target objects found is greater than 1, find object images in the object images included in the first picture whose distance from the first image is less than a preset second threshold. If the number of found object images is a first value, where the first value is the number of found target objects minus 1, sort the found target objects in descending order according to the radar coordinates, sort the found object images and the first image in ascending order, and obtain the target object with the same sorting position as the first image as the first target object.

[0164] In a possible implementation, the first grid is a grid in a preset grid terrain map; the preset grid terrain map is obtained by dividing the monitoring area of the radar into grids;

[0165] The device 60 may further include: an end judgment unit, configured to determine that the proportion of calibrated grids in a specified area of the grid terrain map reaches a preset third threshold, and end the parameter calibration. The calibrated grids are grids with the parameter values.

[0166] Figure 7 This is a structural diagram of an embodiment of the target object tracking device of the present application. The device can be applied to a processing device or a radar integrated with a processing device, such as Figure 7 As shown, the device 70 may include:

[0167] An acquisition unit 71, configured to obtain the radar coordinates of a target object when the target object is detected; the radar coordinates are coordinates in a radar coordinate system;

[0168] A calculation unit 72, configured to calculate the PTZ value corresponding to the radar coordinates according to a preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system;

[0169] A correction unit 73, configured to, if a reference value corresponding to the radar coordinates is found, correct the T value in the PTZ value according to the reference value; the reference value is a reference value of the T value of the camera when the radar and the camera are linked;

[0170] A sending unit 74, configured to send the corrected PTZ value to the camera, and the corrected PTZ value is used to enable the camera to perform image capture according to the corrected PTZ value.

[0171] In a possible implementation, the correction unit 73 may specifically be configured to: replace the T value in the PTZ value with the reference value.

[0172] In a possible implementation, the correction unit 73 may specifically be configured to: find the second grid corresponding to the radar coordinate; if the second grid has a corresponding reference value, use the reference value corresponding to the second grid as the reference value corresponding to the radar coordinate; if the second grid does not have a corresponding reference value, find the third grid closest to the second grid among the grids with corresponding reference values, and if the distance between the third grid and the second grid is not greater than a preset fourth threshold, use the reference value corresponding to the third grid as the reference value corresponding to the radar coordinate.

[0173] Figure 6 The device 60 provided in the illustrated embodiment may be used to execute the Figure 4A and Figure 4B technical solutions of the method embodiment shown, and its implementation principle and technical effects may be further referred to the relevant descriptions in the method embodiment.

[0174] Figure 7 The device 70 provided in the illustrated embodiment may be used to execute the Figure 5 technical solutions of the method embodiment shown, and its implementation principle and technical effects may be further referred to the relevant descriptions in the method embodiment.

[0175] It should be understood that the division of each unit of the Figures 6 - 7 device shown above is only a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or physically separated. And these units may all be implemented in the form of software called by a processing element; they may also all be implemented in the form of hardware; or some units may be implemented in the form of software called by a processing element, and some units may be implemented in the form of hardware. For example, the acquisition unit may be a separately established processing element, or may be integrated in a certain chip of the electronic device. The implementation of other units is similar. In addition, all or part of these units may be integrated together or may be independently implemented. In the implementation process, each step of the above method or each of the above units may be completed by the integrated logic circuit in the processor element or the instruction in the form of software.

[0176] For example, the above units may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or, one or more Digital Singnal Processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, these units may be integrated together and implemented in the form of a System-On-a-Chip (SOC).

[0177] An embodiment of the present application further provides a parameter calibration system, including: a camera, a radar, and a processing device, where the processing device is respectively connected to the camera and the radar; wherein,

[0178] The camera is used to track and photograph an object;

[0179] The radar is used to generate the radar coordinates of the target object, and the radar coordinates are the coordinates in the radar coordinate system;

[0180] The processing device is used to receive a parameter calibration start notification; in response to the parameter calibration start notification, obtain a first picture taken by the camera; the first picture includes images of at least one object; detect a first image from the first picture, and the first image is an image of a first object; the first object is one of the at least one object; calculate the T value and the P value corresponding to the target pixel in the first image, where the T value is the T value when the camera takes a second picture, and the P value is the P value when the camera takes a second picture, and the second picture is a picture in which the target pixel is located at a specified position in the picture; obtain a first target object matching the first object according to the P value; the first target object is an object detected by the radar; find the first grid where the radar coordinates of the first target object are located; the radar coordinates are the coordinates in the radar coordinate system; update the reference value corresponding to the first grid according to the T value, and the reference value is the reference value of the T value of the camera when the radar and the camera are linked.

[0181] Optionally, the system may further include:

[0182] A client device, which is used to detect the parameter calibration start operation of the user and send the parameter calibration start notification to the processing device.

[0183] An embodiment of the present application further provides a target object tracking system, including: a camera, a radar, and a processing device, where the processing device is respectively connected to the camera and the radar; wherein,

[0184] A camera for tracking and photographing a target object;

[0185] A radar for generating radar coordinates of the target object, where the radar coordinates are coordinates in a radar coordinate system;

[0186] A processing device, when detecting a target object, obtains the radar coordinates of the target object; the radar coordinates are coordinates in a radar coordinate system; calculates the PTZ value corresponding to the radar coordinates according to a preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system; if a reference value corresponding to the radar coordinates is found, corrects the T value in the PTZ value according to the reference value; the reference value is a reference value of the T value of the camera when the radar and the camera are linked; sends the corrected PTZ value to the camera, and the corrected PTZ value is used to make the camera perform image shooting according to the corrected PTZ value.

[0187] An embodiment of the present application further provides a processing device, including: one or more processors; a memory; and one or more computer programs, where the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute Figures 4A - 5 the method provided by the embodiment shown.

[0188] The present application also provides a radar, the radar includes a storage medium and a central processing unit, the storage medium may be a non-volatile storage medium, a computer executable program is stored in the storage medium, the central processing unit is connected to the non-volatile storage medium, and executes the computer executable program to implement the present application Figures 4A - 5 the method provided by the embodiment shown.

[0189] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when it runs on a computer, it causes the computer to execute the present application Figures 4A - 5 the method provided by the embodiment shown.

[0190] An embodiment of the present application further provides a computer program product, the computer program product includes a computer program, and when it runs on a computer, it causes the computer to execute the present application Figures 4A - 5 the method provided by the embodiment shown.

[0191] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0192] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0193] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0194] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (hereinafter referred to as ROM), random access memories (hereinafter referred to as RAM), magnetic disks, or optical discs that can store program codes.

[0195] The above is only the specific implementation manner of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application and should be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A parameter calibration method, characterized in that, Including: Receiving a parameter calibration start notification; In response to the parameter calibration start notification, acquiring a first picture captured by a camera; The first picture includes images of at least one object; Detecting a first image from the first picture, where the first image is an image of a first object; The first object is one of the at least one object; Calculating a T value and a P value corresponding to a target pixel in the first image, where the T value is the T value when the camera captures a second picture, and the P value is the P value when the camera captures the second picture. The second picture is a picture in which the target pixel is located at a specified position in the picture. Herein, the T value refers to the T value in the PTZ value in the camera coordinate system when the camera captures an image, and the P value refers to the P value in the PTZ value in the camera coordinate system when the camera captures an image; Obtaining a first target object that matches the first object according to the P value; the first target object is an object detected by a radar; Searching for a first grid where the radar coordinate of the first target object is located; the radar coordinate is a coordinate in the radar coordinate system; Updating a reference value corresponding to the first grid according to the T value, where the reference value is a reference value of the T value of the camera when the radar and the camera are linked; 2. The method according to claim 1, wherein The updating the reference value corresponding to the first grid according to the T value includes: Calculating an average value of a historical T value and the T value corresponding to the target pixel, and using the average value as the reference value. The historical T value is the T value used to update the reference value before updating the reference value according to the T value corresponding to the target pixel; 3. The method according to claim 1 or 2, characterized in that, The obtaining a first target object that matches the first object according to the P value includes: Obtaining the radar coordinate of the target object detected by the radar; Calculating the P value of the target object according to the radar coordinate of the target object; Searching for a target object from the target objects where the difference between the P value of the target object and the P value of the first object is less than a preset first threshold, and determining the first target object according to the found target object; 4. The method according to claim 3, wherein The determining the first target object according to the found target object includes: If the number of found target objects is 1, using the found target object as the first target object; If the number of found target objects is greater than 1, searching for an object image whose distance from the first image is less than a preset second threshold from the object images included in the first picture; if the number of found object images is a first value, where the first value is the number of found target objects minus 1, sorting the found target objects in descending order of radar coordinates, sorting the found object images and the first image in ascending order, and obtaining a target object with the same sorting position as the first image as the first target object; 5. The method according to claim 1, wherein The first grid is a grid in a preset grid terrain map; The preset grid terrain map is obtained by dividing the monitoring area of the radar into grids; The method further includes: determining that the proportion of calibrated grids in a specified area of the grid terrain map reaches a preset third threshold, and ending the parameter calibration, where the calibrated grids are grids with corresponding parameter values.

6. A method for tracking a target object, characterized in that, Including: When a target object is detected, obtaining the radar coordinates of the target object; The radar coordinates are coordinates in the radar coordinate system; According to the preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system, calculating the PTZ value corresponding to the radar coordinates; If a reference value corresponding to the radar coordinates is found, correcting the T value in the PTZ value according to the reference value; The reference value is the reference value of the T value of the camera when the radar and the camera are linked; Sending the corrected PTZ value to the camera, where the corrected PTZ value is used to make the camera take images according to the corrected PTZ value.

7. The method according to claim 6, characterized in that The correcting the T value in the PTZ value according to the reference value includes: Replacing the T value in the PTZ value with the reference value.

8. The method according to claim 6 or 7, characterized in that, The finding the reference value corresponding to the radar coordinates includes: Finding the second grid corresponding to the radar coordinates; If the second grid has a corresponding reference value, using the reference value corresponding to the second grid as the reference value corresponding to the radar coordinates; If the second grid does not have a corresponding reference value, finding the third grid closest to the second grid among the grids with corresponding reference values, and if the distance between the third grid and the second grid is not greater than a preset fourth threshold, using the reference value corresponding to the third grid as the reference value corresponding to the radar coordinates.

9. A parameter calibration system, characterized in that, Including: A camera for tracking and photographing an object; A radar for generating the radar coordinates of a target object, where the radar coordinates are coordinates in the radar coordinate system; A processing device for receiving a parameter calibration start notification; In response to the parameter calibration start notification, obtaining a first picture taken by the camera; the first picture includes images of at least one object; detecting a first image from the first picture, where the first image is an image of a first object; The first object is one of the at least one object; Calculating the T value and the P value corresponding to the target pixel in the first image, where the T value is the T value when the camera takes a second picture, and the P value is the P value when the camera takes a second picture, and the second picture is a picture where the target pixel is located at a specified position in the picture; Obtaining a first target object matching the first object according to the P value; the first target object is an object detected by the radar; finding the first grid where the radar coordinates of the first target object are located; the radar coordinates are coordinates in the radar coordinate system; updating the reference value corresponding to the first grid according to the T value, where the reference value is the reference value of the T value of the camera when the radar and the camera are linked; where, the T value refers to the T value in the PTZ value in the camera coordinate system when the camera takes a picture; the P value refers to the P value in the PTZ value in the camera coordinate system when the camera takes a picture.

10. The parameter calibration system according to claim 9, wherein Further including: A client device for detecting a user's parameter calibration start operation and sending the parameter calibration start notification to the processing device.

11. A target object tracking system, characterized in that, It includes: A camera for tracking and photographing a target object; A radar for generating the radar coordinates of the target object, where the radar coordinates are coordinates in the radar coordinate system; A processing device for obtaining the radar coordinates of the target object when the target object is detected; the radar coordinates are coordinates in the radar coordinate system; calculating the corresponding PTZ value of the radar coordinates according to the preset coordinate conversion relationship between the radar coordinate system and the camera coordinate system; if a reference value corresponding to the radar coordinates is found, correcting the T value in the PTZ value according to the reference value; The reference value is the reference numerical value of the T value of the camera when the radar and the camera are linked; Sending the corrected PTZ value to the camera, and the corrected PTZ value is used to enable the camera to perform image shooting according to the corrected PTZ value.

12. A processing device, characterized in that, It includes: One or more processors; A memory; And one or more computer programs, where the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the method according to any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the method according to any one of claims 1 to 8.

Images