Pan-tilt tracking method, electronic equipment and storage medium

Abstract

The invention discloses a holder tracking method, electronic equipment and a storage medium, and the method comprises the steps: obtaining the moving speed of a target object under the condition that the target object triggers holder tracking; determining picture staying time corresponding to the moving speed, wherein the picture staying time represents the duration that the target object at the moving speed can continue to appear in a picture shot by an acquisition device arranged on the holder; and in response to the fact that a first relation between the picture staying time and preset intelligent tracking relay time consumption meets an intelligent tracking requirement, the cloud deck is controlled to track the target object by utilizing an intelligent tracking algorithm, and the intelligent tracking algorithm is that the cloud deck is controlled to move by utilizing an analysis result of an image shot by the acquisition device. Through the method, the target tracking effect can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of target tracking, and in particular to a pan-tilt tracking method, electronic equipment, and storage medium. Background Art

[0002] In the field of target tracking, target tracking is typically performed using radar or intelligent tracking algorithms. When using intelligent tracking algorithms, existing algorithms only consider locating the target near the center of the image and lack a predictive estimate of the success rate of tracking control. Simply put, existing intelligent tracking algorithms fail to consider tracking feasibility—that is, they fail to determine whether tracking is possible under current conditions. This can lead to ineffective tracking and poor target tracking performance. Summary of the Invention

[0003] The main technical problem solved by this application is to provide a pan-tilt tracking method, electronic equipment and storage medium, which can improve the target tracking effect.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is to provide a pan-tilt tracking method, which includes: when a target object triggers pan-tilt tracking, obtaining the moving speed of the target object; determining the image retention time corresponding to the moving speed, wherein the image retention time represents the length of time that the target object with the moving speed can continue to appear in the image captured by the acquisition device provided on the pan-tilt; in response to the first relationship between the image retention time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, using an intelligent tracking algorithm to control the pan-tilt movement, wherein the intelligent tracking algorithm uses the analysis results of the image captured by the acquisition device to control the movement of the pan-tilt.

[0005] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide an electronic device, which includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the above pan-tilt tracking method.

[0006] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed by a processor to implement the above-mentioned pan-tilt tracking method.

[0007] The above scheme obtains the moving speed of the target object when the target object triggers pan-tilt tracking. The screen dwell time corresponding to the moving speed is determined, and the screen dwell time represents the length of time that the target object with the moving speed can continue to appear in the picture taken by the acquisition device provided on the pan-tilt. In response to the first relationship between the screen dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, the intelligent tracking algorithm is used to control the pan-tilt to track the target object, wherein the intelligent tracking algorithm controls the movement of the pan-tilt according to the analysis results of the image taken by the acquisition device. The present application determines the feasibility of tracking by judging the relationship between the screen dwell time and the preset intelligent tracking relay time, thereby realizing the correct use of the intelligent tracking algorithm to control the pan-tilt to track the target object, reducing the occurrence of target tracking failures, and thus improving the effect of tracking the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a flow chart of an embodiment of a pan-tilt tracking method provided by the present application;

[0009] Figure 2 This is a schematic diagram of a target object meeting the tracking conditions of the pan-tilt tracking method provided by the present application;

[0010] Figure 3 This is a partial flow chart of a specific embodiment of the pan-tilt tracking method provided by this application;

[0011] Figure 4 This is a schematic diagram of the framework of an embodiment of the pan-tilt tracking device of the present application;

[0012] Figure 5 This is a schematic diagram of the framework of an embodiment of the pan-tilt tracking device of the present application;

[0013] Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and effects of this application clearer and more specific, this application is further described in detail below with reference to the accompanying drawings and examples.

[0015] It should be noted that the term "several" in this article means at least one, and the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. The term "and / or" is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0016] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the pan-tilt tracking method provided by the present application. It should be noted that if there are substantially the same results, this embodiment is not based on Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0017] Step S11: When the target object triggers pan / tilt tracking, the moving speed of the target object is obtained.

[0018] The execution subject of this embodiment can be a PTZ or any independent device that can communicate with the PTZ. There is no limitation on the specific type of the execution subject. For ease of explanation, the term "device end" will be used to refer to the execution subject in the following text.

[0019] In one embodiment, whether the target object triggers PTZ tracking can be determined by determining whether the target object triggers an alarm. Specifically, if the target object triggers an alarm, the target object is determined to have triggered PTZ tracking. In one specific embodiment, preset alarm conditions can be pre-set, such as area intrusion, speed limit violation, abnormal morphology, tripwire intrusion, etc. If the target object meets the preset alarm conditions, the target object is determined to have triggered an alarm, further triggering PTZ tracking. Of course, the above preset alarm conditions can also be flexibly combined, and this is not limited here.

[0020] In another embodiment, radar can be used to perform preliminary target tracking on the target object that triggers pan-tilt tracking to obtain the target object's movement speed. Specifically, radar feedback data can be used to obtain the target object's movement speed. In one specific embodiment, the radar can emit electromagnetic waves, then receive the waves (echoes) reflected back from the target object, and analyze the echo's time delay, frequency variation, and other information to detect the target object's position and movement speed. The specific method for obtaining the target object's movement speed can be flexibly set based on the needs of the actual application scenario and is not limited here.

[0021] In another embodiment, after obtaining the target object's moving speed, the radar can be continuously tracked for a period of time, for example, using the radar to continuously track the target object for 10 seconds, to ensure that the device has sufficient time to complete all its response actions, including a series of operations such as pan / tilt rotation and lens zoom adjustment, so that the target object can be clearly presented in the image, and the state of the target object can be clearly observed. In a specific embodiment, the radar control can be continuously monitored for instructions. If no instructions are received, the device is in an idle waiting mode. When the radar detects the target object and the device successfully receives instructions (such as absolute positioning guidance (PTZ coordinates)), the device uses the received absolute positioning guidance to track the target. The absolute positioning guidance can be obtained based on pre-calibrated data or in real time, which is not limited here. For example, when the absolute positioning guidance is obtained based on pre-calibrated data, the absolute positioning guidance matching the pre-calibrated data can be obtained based on the moving speed of the target object. The pre-calibrated data can be stored in a data table, a file system, or a data warehouse, and its specific storage method is not limited in this embodiment.

[0022] For example, after obtaining the target's speed, the radar can be continuously tracked for a period of time, obtaining absolute positioning guidance based on pre-calibrated data. The stored pre-calibrated data includes the target ID, speed, distance, type, first calibrated PT offset, second calibrated PT offset, and lens calibration magnification. The first calibrated PT offset includes a first calibrated P offset and a first calibrated T offset, the second calibrated PT offset includes a second calibrated P offset and a second calibrated T offset, and the lens calibration magnification includes a visible light lens calibration magnification and a thermal imaging lens calibration magnification. The first calibrated PT offset is based on the second calibrated PT offset, but with a manual intervention parameter to offset the target image to the edge of the image opposite to the target's movement direction. The second calibrated PT offset is a parameter used to control the pan / tilt rotation calibration, with the goal of keeping the target image relatively centered within the lens image. The lens calibration magnification is to manually control the visible light and thermal imaging lens magnification to a suitable magnification that is more suitable for monitoring and observation, and easier to detect and trigger an alarm (for example, under 60 pixels and 1280p conditions, a suitable magnification that is more suitable for monitoring and observation, and easier to detect and trigger an alarm). According to the acquired moving speed of the target object, the second calibration PT offset and the lens calibration magnification that match the corresponding moving speed are used as absolute positioning guidance. The received absolute positioning guidance is used to track the target, that is, the second calibration PT offset and the lens calibration magnification are used to adjust the pan / tilt and lens to the corresponding state to ensure that the target object is always in the center area of the lens image. It should be noted that when adjusting the lens calibration magnification, you can choose to adjust at least one of the visible light lens magnification and the thermal imaging lens magnification according to the actual environmental conditions, and there is no restriction here.

[0023] In a specific embodiment, in response to the target object triggering tracking, the radar is used to control the pan-tilt head to track the target object according to a normal scheme.

[0024] For example, radar feedback data can be used to select a corresponding second calibrated PT offset to control the pan / tilt movement. Another example is the use of radar feedback data to select a corresponding calibration magnification to adjust the image magnification of the acquisition device. Of course, radar feedback data can also be used to select a corresponding second calibrated PT offset to control the pan / tilt movement. The corresponding calibration magnification can also be used to adjust the image magnification of the acquisition device. This is not a limitation.

[0025] Of course, after obtaining the target object's moving speed, it is also possible to stop using the radar to track the target object. This can be set according to actual needs and is not limited here.

[0026] Step S12: Determine the image dwell time corresponding to the moving speed.

[0027] The image retention time represents the length of time that a target object with a moving speed can continue to appear in the image captured by the acquisition device located on the pan-tilt head.

[0028] In one embodiment, the image retention time is pre-calibrated according to the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

[0029] In a specific embodiment, the picture dwell time and pre-calibrated data such as the first calibration PT offset and lens calibration magnification mentioned in step S11 can be stored in the same storage structure, which will not be described in detail here.

[0030] Step S13: In response to the first relationship between the image dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirement, the intelligent tracking algorithm is used to control the pan / tilt head to track the target object.

[0031] Among them, the intelligent tracking algorithm controls the movement of the pan-tilt head using the analysis results of the images captured by the acquisition device.

[0032] In one embodiment, the intelligent tracking requirement is that the product of the image retention time and the first redundancy weight is greater than a preset intelligent tracking relay time.

[0033] In another embodiment, the preset intelligent tracking relay time includes at least one of the gimbal startup time and the process time of the intelligent tracking algorithm. The gimbal startup time is the maximum value of the startup time of the gimbal in different directions. The startup time in each direction is the time it takes for the gimbal to accelerate from stationary to a gimbal speed corresponding to the moving speed in the direction. When the gimbal moves at the said gimbal speed, the target object at the moving speed can be kept in the same area of the screen.

[0034] The startup time in each direction is the time it takes for the gimbal to accelerate from a stationary state to a gimbal speed corresponding to the moving speed in the direction, which can be stored in a data table or other data storage structure, and is not limited here.

[0035] For example, the intelligent tracking requirement is Sensitivity(i)%*Tleft(i)>max(pvptzCost,tvptzCost)+aiCost, that is, when the above inequality is satisfied, the first relationship between the confirmation screen dwell time and the preset intelligent tracking relay time meets the intelligent tracking requirement. Among them, Sensitivity(i)% is the first redundant weight. max(pvptzCost,tvptzCost) is the maximum value of the gimbal startup time in different directions, that is, the gimbal startup time, pvptzCost is the time taken to accelerate from a standstill in the horizontal direction to the gimbal speed corresponding to the moving speed, and tvptzCost is the time taken to accelerate from a standstill in the vertical direction to the gimbal speed corresponding to the moving speed. aiCost is the process time of the intelligent tracking algorithm. The process time of the intelligent tracking algorithm is the total time taken for the intelligent tracking algorithm detection, alarm, and tracking process. Specifically, Sensitivity(i)% can be configured by the user based on actual needs, ranging from (0,100%). When Sensitivity(i)% is 100%, the intelligent tracking algorithm is more frequently used to track the target object. When Sensitivity(i)% is 0%, the gimbal is more likely to always use the radar to control the gimbal to track the target object according to the normal method.

[0036] In another embodiment, before using an intelligent tracking algorithm to control the pan-tilt head to track the target object, a radar can be used to control the pan-tilt head to track the target object so that the target object is located in the target area in the picture taken by the acquisition device. The picture residence time represents the duration of the process of the target object moving from the target area to outside the picture at the moving speed.

[0037] In response to the first relationship between the image dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, the radar can be used to control the pan-tilt head to track the target object according to the intelligent tracking reservation plan within the first time period. The pan-tilt head can also be controlled to stop, and the image obtained after the pan-tilt head stops is used to detect whether the target object meets the tracking conditions. Of course, the radar can also be used to control the pan-tilt head to track the target object according to the intelligent tracking reservation plan within the first time period. The pan-tilt head is controlled to stop, and the image obtained after the pan-tilt head stops is used to detect whether the target object meets the tracking conditions. The specific settings can be made according to actual needs and are not limited here.

[0038] In one embodiment, in response to a first relationship between the image retention time and a preset intelligent tracking relay duration meeting an intelligent tracking requirement, the pan / tilt head (PTZ) is controlled to track a target object using a radar according to an intelligent tracking reservation scheme within a first time period. The intelligent tracking reservation scheme enables the target object to be located in a first target area of an image captured by a capture device, and the image retention time represents the duration of the target object moving from the first target area to outside the image at the moving speed. After the first time period, the PTZ is controlled to track the target object using an intelligent tracking algorithm.

[0039] In a specific embodiment, the first target area is located at an edge area of the picture, and the orientation of the first target area in the picture is opposite to the moving direction of the target object.

[0040] Specifically, the corresponding screen dwell time represents the length of time it takes for the target object to move from the edge area of the screen to outside the screen at the moving speed, wherein the edge area of the screen is opposite to the moving direction of the target object. For example, when the target object moves to the right, the screen dwell time represents the length of time it takes for the target object to move from the left edge area of the screen to outside the screen at the moving speed. That is, the first relationship between the length of time it takes for the target object to move from the left edge area of the screen to outside the screen at the moving speed and the preset intelligent tracking relay time meets the intelligent tracking requirements, and within the first time period, the radar is used to control the pan-tilt head to track the target object according to the intelligent tracking reservation plan.

[0041] In another specific embodiment, controlling the gimbal to track the target object using radar according to the intelligent tracking reservation scheme within the first time period can be controlling the gimbal movement by using radar feedback data and selecting a corresponding first calibrated PT offset within the first time period.

[0042] The first calibrated PT offset is based on the second calibrated PT offset and manually intervenes with parameters to shift the target image to the edge of the image opposite to the target object's movement direction. That is, the first calibrated PT offset is used to shift the target image to the edge of the image opposite to the target object's movement direction. The first calibrated PT offset can be pre-calibrated based on the type of target object, the distance between the target object and the acquisition device, and the target object's movement speed.

[0043] In another specific embodiment, the image magnification of the acquisition device can be adjusted by selecting a corresponding calibration magnification using data fed back by the radar during the first time period.

[0044] The specific description of the calibration magnification can refer to the relevant description of the calibration magnification in step S11, which will not be repeated here. The calibration magnification can be pre-calibrated according to the type of target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

[0045] In another specific embodiment, the radar feedback data may be used to select a corresponding first calibrated PT offset to control the movement of the pan / tilt platform within the first time period, and a corresponding calibrated magnification may be selected to adjust the image magnification of the acquisition device.

[0046] In another embodiment, in response to a first relationship between the image retention time and a preset intelligent tracking relay duration meeting an intelligent tracking requirement, the pan / tilt head is controlled to stop, and an image captured after the pan / tilt head stops is used to detect whether the target object meets tracking conditions. In response to the tracking conditions being met, the pan / tilt head is controlled to track the target object using an intelligent tracking algorithm.

[0047] Specifically, the corresponding screen dwell time is the time it takes for the target object to move from its current position to outside the screen at the moving speed. For example, if the target object's current position is at the center of the screen, the corresponding screen dwell time is the time it takes for the target object to move from the center of the screen to outside the screen at the moving speed. That is, if the first relationship between the time it takes for the target object to move from the center of the screen to outside the screen at the moving speed and the preset intelligent tracking relay time meets the intelligent tracking requirements, the PTZ is controlled to stop.

[0048] The PTZ can be stopped by shielding the command sent by the radar. Of course, the PTZ can also be stopped by using the stop command sent by the radar. This is not limited here.

[0049] In one specific embodiment, in response to not detecting an image meeting the tracking condition within the second time period, the radar is used to control the pan / tilt head to track the target object according to a normal scheme. The normal scheme enables the target object to be located in a second target area of the image captured by the acquisition device, where the second target area is closer to the center of the image than the first target area. The second target area may be the center of the image or an area near the center of the image.

[0050] In a specific embodiment, the radar feedback data can be used to select a corresponding second calibrated PT offset to control the movement of the gimbal.

[0051] The second calibrated PT offset is a parameter used to calibrate the pan / tilt (PTZ) rotation, aiming to keep the target object relatively centered within the camera's frame. Specifically, the PTZ is controlled using radar according to a standard method to track the target object, ensuring that the target object remains relatively centered within the camera's frame. The second calibrated PT offset can be pre-calibrated based on the target object's type, the distance between the target object and the acquisition device, and the target object's movement speed.

[0052] In another specific embodiment, a corresponding calibration magnification may also be selected to adjust the image magnification of the acquisition device.

[0053] The specific description of the calibration magnification can refer to the relevant description of the calibration magnification in step S11, which will not be repeated here. The calibration magnification can be pre-calibrated according to the type of target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

[0054] In another specific embodiment, the radar feedback data can be used to select a corresponding second calibration PT offset to control the movement of the pan / tilt platform, and a corresponding calibration magnification can be selected to adjust the image magnification of the acquisition device.

[0055] In another specific embodiment, to account for the potential for missed detections of target objects using an intelligent tracking algorithm, a foolproofing mechanism can be added. Specifically, after detecting whether the target object meets tracking conditions using an image captured after the pan / tilt system is stopped, the cumulative number of consecutive missed detections of the target object can be recorded in response to failure to detect that the image meets the tracking conditions within a second time period. In response to the cumulative number of consecutive missed detections exceeding a first threshold, the first relationship between the image dwell time and the preset intelligent tracking relay duration can be adjusted so that the adjusted first relationship no longer meets the intelligent tracking requirements.

[0056] The cumulative number of consecutive missed reports may be stored in a state table.

[0057] In a specific embodiment, the cumulative number of consecutive omissions and the cumulative number of consecutive omissions described in step S13 can be stored as the same variable in the state table. Of course, the cumulative number of consecutive omissions and the cumulative number of consecutive omissions described in step S13 can also be stored as different variables in the state table, which can be set according to actual needs and is not limited here.

[0058] For example, the cumulative number of consecutive omissions is initialized to 0, and the first count threshold is 1. In response to the fact that no image is detected to meet the tracking conditions within the first 200ms, the cumulative number of consecutive omissions of the target object is recorded, that is, the cumulative number of consecutive omissions is increased by 1, and the corresponding cumulative number of consecutive omissions is 1. In the case where the image is detected to meet the tracking conditions within the second 200ms, the cumulative number of consecutive omissions is initialized to 0. In the case where no image is detected to meet the tracking conditions within the second 200ms, the cumulative number of consecutive omissions is increased by 1, and the corresponding cumulative number of consecutive omissions is 2. In the case where the corresponding cumulative number of consecutive omissions is 2, that is, it exceeds the first count threshold, the first relationship between the image residence time and the preset intelligent tracking relay time is adjusted so that the adjusted first relationship does not meet the intelligent tracking requirements. For example, the first redundant weight can be adjusted so that the adjusted first relationship does not meet the intelligent tracking requirements. Specifically, the first redundant weight is set to 0%, and the adjusted first relationship does not meet the intelligent tracking requirements.

[0059] Tracking conditions can be determined by combining intelligent perimeter rules and target object information. Intelligent perimeter rules can include at least one basic perimeter rule, including tripwire intrusion and area intrusion. Target object information includes at least the target object's location information and corresponding timing information. Specifically, the target object information can be used to determine whether the target object satisfies the intelligent perimeter rules, thereby determining whether the target object meets the tracking conditions.

[0060] In a specific implementation, the target object information may be stored in the form of a data table.

[0061] In another specific embodiment, target object information can be further determined based on the image presented by the lens. The lens can include a visible light lens and a thermal imaging lens. The images captured by the visible light lens and the thermal imaging lens can be processed separately or simultaneously. The choice is not limited here and can be based on actual environmental requirements.

[0062] For example, taking the example of separate processing of visible light and thermal imaging cameras, the images captured by the visible light and thermal imaging cameras are input into the visible light and thermal imaging channels, respectively. Each channel runs a detection model, comparing it to the trained target model to determine whether the target object exists in the image. It also segments the target object's outline and outputs the coordinates of the point within its bounding box. The bounding box can be a rectangle, and the coordinates of the point within it can be the coordinates of two points at the top left and bottom right of the box, or four points at its four corners, without limitation. The visible light and thermal imaging channels each run a tracking model, processing multiple consecutive frames. The ID, corresponding temporal state, and position information of the same target object are recorded, and the corresponding target object information is updated based on this information. The target model and tracking model corresponding to the visible light and thermal imaging cameras can be the same or different. Furthermore, if the error in the output coordinates of the point within the bounding box of the target object remains within n% over a consecutive number of frames, the target ID is considered the same, and the coordinates are not updated. The error n% can be configured by the user.

[0063] In a specific embodiment, the target object information can be updated in real time, that is, the device can continuously obtain the ID of the target object, the timing status corresponding to the target object, and the location information corresponding to the target object, and update the updated information to the target object information.

[0064] See Figure 2 To illustrate, Figure 2 This is a schematic diagram of a target object meeting the tracking conditions of the pan-tilt tracking method provided by this application.

[0065] like Figure 2 As shown, the targets detected in detection area γ include target car a and target person b. The dotted rectangles around target car a and target person b are bounding boxes. Among them, t1, t2, t3...t(n+1) represent different time sequences. Target car a meets the intelligent perimeter rule, which means the target car triggers an alarm and the pan-tilt head performs tracking. Target person b does not meet the intelligent perimeter rule, which means the target car does not trigger an alarm and the pan-tilt head does not perform tracking. Therefore, there is an image of target car a at time sequence t(n+1), while there is no image of target person b at time sequence t(n+1). Target car a is the target object. The device can continuously obtain the ID of target car a, the corresponding time sequence status of target car a, and the corresponding location information of target car a, and update the updated information to the target object information.

[0066] In one specific embodiment, the target object information may only store information about the target object that triggered an alarm. Alternatively, information about all detected targets may be stored. Of course, information about all detected targets may also be stored, and when an alarmed target object is detected, information corresponding to the remaining targets that did not trigger an alarm may be deleted from the target object information. This is not a limitation.

[0067] In another embodiment, in response to a first relationship between the image dwell time and a preset intelligent tracking relay duration meeting an intelligent tracking requirement, the pan / tilt head (PTZ) is controlled to track the target object using radar according to an intelligent tracking reservation scheme within a first time period. The intelligent tracking reservation scheme enables the target object to be located in a first target area of the image captured by the acquisition device, and the image dwell time represents the duration of the target object moving from the first target area to outside the image at the moving speed. After the first time period, the PTZ is controlled to stop, and the image captured after the PTZ stops is used to detect whether the target object meets the tracking conditions. In response to the tracking conditions being met, the PTZ is controlled to track the target object using an intelligent tracking algorithm.

[0068] Specifically, the corresponding screen dwell time represents the time it takes for the target object to move from the edge area of the screen to the outside of the screen at the moving speed, wherein the edge area of the screen is in the opposite direction of the moving direction of the target object. For a detailed description, please refer to the relevant description in step S13 and will not be repeated here.

[0069] In another embodiment, a foolproof mechanism can also be implemented to reduce instances where the intelligent tracking algorithm poorly tracks the target object, taking into account the duration of tracking the target object using the intelligent tracking algorithm. Specifically, the actual duration of tracking the target object using the intelligent tracking algorithm is obtained. In response to a second relationship between the actual duration and the preset duration meeting normal tracking requirements, the intelligent tracking algorithm is terminated to control the pan / tilt head to track the target object.

[0070] In another embodiment, in response to the second relationship not meeting normal tracking requirements, a number of consecutive tracking failures of the target object is recorded. In response to the number of consecutive tracking failures exceeding a second threshold, the first relationship between the image dwell time and the preset intelligent tracking relay duration is adjusted so that the adjusted first relationship does not meet the intelligent tracking requirements.

[0071] The number of consecutive tracking failures may be stored in a state table.

[0072] In one specific embodiment, normal tracking requires that the actual tracking duration be greater than the product of the preset tracking duration and the second redundancy weight. For example, if the second threshold is 0, the number of consecutive tracking failures is initialized to 0, the normal tracking time is 3 seconds, the preset tracking duration is 30 seconds, and the second redundancy weight is 10%, and the actual tracking duration is no greater than the product of the preset tracking duration and the second redundancy weight, then the second relationship between the actual tracking duration and the preset tracking duration does not meet the normal tracking requirement, and the number of consecutive tracking failures is increased by 1. If the number of consecutive tracking failures exceeds the second threshold, the first relationship between the image dwell time and the preset intelligent tracking relay duration is adjusted so that the adjusted first relationship does not meet the intelligent tracking requirement.

[0073] Among them, the smart perimeter rules can also include normal tracking requirements.

[0074] It should be noted that when the intelligent tracking algorithm is used to control the gimbal to track the target object and it is determined that the target object tracking is lost, and the second relationship between the actual tracking time and the preset tracking time meets the normal tracking requirements, or the actual tracking time is equal to the preset tracking time, the target object can be updated or steps S11 to S13 can be repeated.

[0075] For example, when the intelligent tracking algorithm is used to control the pan-tilt head to track the target object and it is determined that the target object tracking is lost, the normal tracking time is 4 seconds, the preset tracking time is 30 seconds, and the second redundancy weight is 10%. That is, the actual tracking time is greater than the product of the preset tracking time and the second redundancy weight. The target object can be updated or steps S11 to S13 can be repeatedly executed.

[0076] For another example, when the intelligent tracking algorithm is used to control the gimbal to track the target object and it is determined that the target object tracking is lost, the normal tracking time is 4 seconds, and the preset tracking time is 4 seconds, that is, the actual tracking time is equal to the preset tracking time. The target object can be updated or steps S11 to S13 can be repeated.

[0077] In another embodiment, in response to the first relationship not meeting the intelligent tracking requirements, the radar is used to control the pan-tilt head to track the target object according to a normal scheme, wherein the normal scheme enables the target object to be located in a second target area of the image captured by the acquisition device. A detailed description of controlling the pan-tilt head to track the target object according to the normal scheme using the radar can be found in the relevant descriptions of steps S11 and S13 and is not repeated here.

[0078] This embodiment obtains the moving speed of the target object when the target object triggers pan-tilt tracking. The screen dwell time corresponding to the moving speed is determined, and the screen dwell time represents the length of time that the target object with the moving speed can continue to appear in the picture taken by the acquisition device provided on the pan-tilt. In response to the first relationship between the screen dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, the intelligent tracking algorithm is used to control the pan-tilt to track the target object, wherein the intelligent tracking algorithm controls the movement of the pan-tilt according to the analysis results of the image taken by the acquisition device. The present application determines the feasibility of tracking by judging the relationship between the screen dwell time and the preset intelligent tracking relay time, thereby correctly controlling the pan-tilt to track the target object using the intelligent tracking algorithm, reducing the occurrence of target tracking failures, and improving the effect of tracking the target.

[0079] See Figure 3 , Figure 3 This is a partial flow chart of a specific embodiment of the pan-tilt tracking method provided by this application. Figure 3 As shown, the pan / tilt tracking method provided in this embodiment may include the following steps, wherein step S34 and step S37 may be performed simultaneously or independently, and the selection may be made according to the actual environment, and is not limited here:

[0080] Step S31: When a target object triggers pan-tilt tracking, obtain the target object's movement speed. Step S32: Determine the image dwell time corresponding to the movement speed. Step S33: In response to a first relationship between the image dwell time and a preset intelligent tracking relay duration meeting intelligent tracking requirements, control the pan-tilt to remain stationary.

[0081] Step S34: Acquire a thermal image. Step S35: Use the thermal image to segment the outline of the target object in the image and output the coordinates of the points in the bounding box corresponding to the target object. Step S36: Use the outline of the target object and the coordinates of the points in the bounding box corresponding to the target object to record the target object information.

[0082] Step S37: Acquire a visible light image. Step S38: Use the visible light image to segment the outline of the target object in the image and output the coordinates of the points in the bounding box corresponding to the target object. Step S39: Use the outline of the target object and the coordinates of the points in the bounding box corresponding to the target object to record the target object information.

[0083] After executing step S33, execute step S310: using the target object information, determine whether the target object detection timeout has occurred. If the execution result of step S310 is negative, execute step S311: in response to the target object detection alarm, control the pan-tilt head to track the target object using the intelligent tracking algorithm. Step S312: determine whether the second relationship between the actual tracking duration and the preset tracking duration meets normal tracking requirements. If the execution result of step S312 is negative, execute step S313: update the status table. If the execution result of step S312 is positive, execute step S31 again.

[0084] If the result of executing step S310 is yes, step S313 is executed.

[0085] It should be noted that the status table in step S313 may store the cumulative number of consecutive missed reports and the number of consecutive tracking failures.

[0086] See Figure 4 , Figure 4 It is a framework diagram of an embodiment of the pan-tilt tracking device of the present application. The pan-tilt tracking device 400 includes a moving speed acquisition module 410, a picture dwell time acquisition module 420, and an intelligent tracking module 430. The moving speed acquisition module 410 is used to obtain the moving speed of the target object when the target object triggers pan-tilt tracking. The picture dwell time acquisition module 420 is used to determine the picture dwell time corresponding to the moving speed, and the picture dwell time represents the length of time that the target object with the moving speed can continue to appear in the picture taken by the acquisition device provided on the pan-tilt. The intelligent tracking module 430 is used to control the pan-tilt to track the target object using an intelligent tracking algorithm in response to the first relationship between the picture dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, wherein the intelligent tracking algorithm is to control the movement of the pan-tilt using the analysis results of the image taken by the acquisition device.

[0087] In some embodiments, the intelligent tracking requirement is that the product of the screen dwell time and the first redundancy weight is greater than a preset intelligent tracking relay time.

[0088] In some embodiments, the preset intelligent tracking relay time includes at least one of the gimbal startup time and the process time of the intelligent tracking algorithm. The gimbal startup time is the maximum value of the startup time of the gimbal in different directions. The startup time in each direction is the time it takes for the gimbal to accelerate from stationary to a gimbal speed corresponding to the moving speed in the direction. When the gimbal moves at the gimbal speed, the target object at the moving speed can be kept in the same area of the screen.

[0089] In some embodiments, before using an intelligent tracking algorithm to control the pan-tilt head to track the target object, a radar is first used to control the pan-tilt head to track the target object so that the target object is located in the target area in the picture taken by the acquisition device. The picture dwell time represents the duration of the process in which the target object moves from the target area to outside the picture at the moving speed.

[0090] In some embodiments, the image dwell time is pre-calibrated based on the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

[0091] In some embodiments, when the intelligent tracking module 430 controls the pan-tilt head to track the target object using an intelligent tracking algorithm in response to a first relationship between the image dwell time and a preset intelligent tracking relay time meeting the intelligent tracking requirement, the intelligent tracking module 430 controls the pan-tilt head to track the target object using a radar according to an intelligent tracking reservation scheme within a first time period in response to the first relationship between the image dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirement. The intelligent tracking reservation scheme enables the target object to be located in a first target area of the image captured by the acquisition device, and the image dwell time represents the duration of the process of the target object moving from the first target area to outside the image at the moving speed. After the first time period, the intelligent tracking algorithm is used to control the pan-tilt head to track the target object. Alternatively, in response to the first relationship between the image dwell time and the preset intelligent tracking relay time meeting the intelligent tracking requirement, the pan-tilt head is controlled to stop, and an image captured after the pan-tilt head stops is used to detect whether the target object meets the tracking condition. In response to the tracking condition being met, the intelligent tracking algorithm is used to control the pan-tilt head to track the target object. Alternatively, in response to a first relationship between the image dwell time and a preset intelligent tracking relay duration meeting an intelligent tracking requirement, the pan-tilt head (PTZ) is controlled to track the target object using radar according to an intelligent tracking reservation scheme within a first time period. The intelligent tracking reservation scheme enables the target object to be located in a first target area of the image captured by the acquisition device, and the image dwell time represents the duration of the target object moving from the first target area to outside the image at the moving speed. After the first time period, the PTZ is controlled to stop, and the image captured after the PTZ stops is used to detect whether the target object meets the tracking conditions. In response to the tracking conditions being met, the PTZ is controlled to track the target object using an intelligent tracking algorithm.

[0092] In some embodiments, the first target area is located at an edge area of the picture, and the orientation of the first target area in the picture is opposite to the moving direction of the target object.

[0093] In some embodiments, the intelligent tracking module 430 specifically controls the gimbal to track the target object using the radar according to the intelligent tracking reservation scheme within the first time period, including: within the first time period, using the radar feedback data to select the corresponding first calibrated PT offset to control the gimbal movement.

[0094] In some embodiments, when the intelligent tracking module 430 specifically uses the radar to control the pan-tilt head to track the target object according to the intelligent tracking reservation plan within the first time period, within the first time period, the radar feedback data is used to select the corresponding calibration magnification to adjust the image magnification of the acquisition device.

[0095] In some embodiments, when the intelligent tracking module 430 specifically controls the pan-tilt head (PT) to track a target object using a radar according to a reserved intelligent tracking scheme within a first time period, the first calibrated PT offset is selected to control the movement of the PT during the first time period using data fed back by the radar. The corresponding calibrated magnification is selected to adjust the image magnification of the acquisition device.

[0096] In some embodiments, the pan-tilt tracking device 400 also includes a radar tracking module. After the intelligent tracking module 430 specifically executes the image obtained by using the pan-tilt to stop and detects whether the target object meets the tracking conditions, the tracking duration determination module responds to the fact that no image is detected to meet the tracking conditions within the second time period, and uses the radar to control the pan-tilt to track the target object according to a normal scheme, wherein the normal scheme enables the target object to be located in the second target area of the picture taken by the acquisition device, and the second target area is closer to the center of the picture than the first target area.

[0097] In some embodiments, after detecting whether a target object meets tracking conditions using an image captured after the pan / tilt head is stopped, the intelligent tracking module 430 records a cumulative number of consecutive missed detections of the target object in response to not detecting that the image meets the tracking conditions within a second time period. In response to the cumulative number of consecutive missed detections exceeding a first threshold, the intelligent tracking module adjusts a first relationship between the image dwell time and a preset intelligent tracking relay duration so that the adjusted first relationship does not meet the intelligent tracking requirements.

[0098] In some embodiments, the pan-tilt tracking device 400 further includes a tracking duration determination module configured to obtain an actual tracking duration of the target object using the intelligent tracking algorithm. In response to a second relationship between the actual tracking duration and a preset tracking duration satisfying normal tracking requirements, the pan-tilt tracking device 400 terminates the tracking of the target object using the intelligent tracking algorithm.

[0099] In some embodiments, the pan-tilt tracking device 400 also includes a tracking duration determination module, which is used to record the number of consecutive tracking failures of the target object in response to the second relationship not meeting the normal tracking requirements; in response to the number of consecutive tracking failures exceeding the second number threshold, adjust the first relationship between the picture dwell time and the preset intelligent tracking relay time so that the adjusted first relationship does not meet the intelligent tracking requirements.

[0100] In some embodiments, the pan-tilt tracking device 400 further includes a tracking duration determination module configured to obtain an actual tracking duration of the target object using the intelligent tracking algorithm. In response to a second relationship between the actual tracking duration and a preset tracking duration satisfying normal tracking requirements, the pan-tilt tracking of the target object using the intelligent tracking algorithm is terminated. In response to a second relationship not satisfying normal tracking requirements, the number of consecutive tracking failures of the target object is recorded. In response to a second number threshold of consecutive tracking failures exceeding the second number threshold, the first relationship between the image dwell time and the preset intelligent tracking relay duration is adjusted so that the adjusted first relationship does not satisfy the intelligent tracking requirements.

[0101] In some embodiments, the normal tracking requirement is that the actual tracking duration is greater than the product of the preset tracking duration and the second redundancy weight.

[0102] In some embodiments, the gimbal tracking device 400 also includes a radar tracking module. After the intelligent tracking module 430 executes a judgment on whether the first relationship meets the intelligent tracking requirements, the radar tracking module is used to control the gimbal to track the target object according to a normal scheme using radar in response to the first relationship not meeting the intelligent tracking requirements, wherein the normal scheme enables the target object to be located in the second target area of the picture taken by the acquisition device.

[0103] In some embodiments, the gimbal tracking device 400 also includes a radar tracking module. Before the intelligent tracking module 430 executes the judgment of whether the first relationship meets the intelligent tracking requirements, the radar tracking module is used to trigger tracking in response to the target object and use the radar to control the gimbal to track the target object according to the normal scheme.

[0104] In some embodiments, the gimbal tracking device 400 also includes a radar tracking module, which is used to use radar feedback data to select a corresponding second calibrated PT offset to control the movement of the gimbal when specifically executing the normal scheme of controlling the gimbal to track the target object using radar.

[0105] In some embodiments, the gimbal tracking device 400 also includes a radar tracking module, which is used to use the radar feedback data to select the corresponding calibration magnification to adjust the image magnification of the acquisition device when specifically executing the use of radar to control the gimbal to track the target object according to the normal plan.

[0106] In some embodiments, the PTZ tracking device 400 further includes a radar tracking module configured to utilize radar feedback data to select a corresponding second calibrated PT offset to control PT movement when the PTZ is tracked using radar in a normal manner. The module also selects a corresponding calibrated magnification to adjust the image magnification of the acquisition device.

[0107] In some embodiments, the first calibrated PT offset, the second calibrated PT offset, and the calibration magnification are pre-calibrated according to the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

[0108] See also Figure 5 , Figure 5 Schematic diagram of a framework of an embodiment of a pan-tilt tracking device of the present application. Pan-tilt tracking device 50 includes a memory 51 and a processor 52 coupled to each other. Processor 52 is configured to execute program instructions stored in memory 51 to implement the steps of any of the aforementioned pan-tilt tracking method embodiments. In a specific implementation scenario, pan-tilt tracking device 50 may include, but is not limited to, a microcomputer and a server. Furthermore, pan-tilt tracking device 50 may also include mobile devices such as laptops and tablet computers, without limitation herein.

[0109] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in any of the above-mentioned gimbal tracking method embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0110] See also Figure 6 , Figure 6 The computer-readable storage medium 60 stores program instructions 61 that can be executed by a processor, and the program instructions 61 are used to implement the steps of any of the above-mentioned pan-tilt tracking method embodiments.

[0111] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0112] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0114] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A pan-tilt tracking method, characterized in that: The method comprises: When the target object triggers pan / tilt tracking, obtaining the moving speed of the target object; Determining a picture retention time corresponding to the moving speed, wherein the picture retention time represents a length of time that a target object moving at the moving speed can continue to appear in a picture captured by a collection device provided on the pan / tilt head; In response to the first relationship between the image retention time and the preset intelligent tracking relay time meeting the intelligent tracking requirements, the pan-tilt head is controlled to track the target object using an intelligent tracking algorithm, wherein the intelligent tracking algorithm controls the movement of the pan-tilt head using the analysis results of the image captured by the acquisition device.

2. The method according to claim 1, characterized in that The intelligent tracking requirement is that the product of the image dwell time and the first redundancy weight is greater than the preset intelligent tracking relay time; And / or, the preset intelligent tracking relay time includes at least one of a gimbal startup time and a process time of the intelligent tracking algorithm, the gimbal startup time is the maximum value of the startup time of the gimbal in different directions, the startup time in each direction is the time it takes for the gimbal to accelerate from a stationary state in the direction to a gimbal speed corresponding to the moving speed, and when the gimbal moves at the gimbal speed, the target object at the moving speed can be kept in the same area of the picture; and / or, before controlling the pan-tilt head to track the target object using an intelligent tracking algorithm, first controlling the pan-tilt head to track the target object using a radar so that the target object is located in a target area in the image captured by the acquisition device, and the image dwell time represents the duration of a process in which the target object moves from the target area to outside the image at the moving speed; And / or, the image retention time is pre-calibrated according to the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

3. The method according to claim 1, characterized in that In response to the first relationship between the image retention time and the preset intelligent tracking relay time meeting the intelligent tracking requirement, controlling the pan-tilt head to track the target object using an intelligent tracking algorithm includes: In response to a first relationship between the image retention time and a preset intelligent tracking relay time satisfying an intelligent tracking requirement, controlling the pan-tilt head to track the target object using the radar according to an intelligent tracking reservation scheme within a first time period, wherein the intelligent tracking reservation scheme enables the target object to be located in a first target area of the image captured by the acquisition device, and the image retention time represents the duration of a process in which the target object moves from the first target area to outside the image at the moving speed; after the first time period, controlling the pan-tilt head to track the target object using an intelligent tracking algorithm; Alternatively, in response to a first relationship between the image retention time and a preset intelligent tracking relay time satisfying an intelligent tracking requirement, the pan-tilt head is controlled to stop, and an image captured after the pan-tilt head stops is used to detect whether the target object meets a tracking condition; in response to the tracking condition being met, the pan-tilt head is controlled to track the target object using an intelligent tracking algorithm; Alternatively, in response to a first relationship between the image retention time and a preset intelligent tracking relay time satisfying an intelligent tracking requirement, the pan-tilt head is controlled to track the target object using a radar according to an intelligent tracking reservation scheme within a first time period, wherein the intelligent tracking reservation scheme enables the target object to be located in a first target area of the image captured by the acquisition device, and the image retention time represents the duration of a process in which the target object moves from the first target area to outside the image at the moving speed; after the first time period, the pan-tilt head is controlled to stop, and an image captured after the pan-tilt head stops is used to detect whether the target object meets the tracking conditions; in response to the tracking conditions being met, the pan-tilt head is controlled to track the target object using an intelligent tracking algorithm.

4. The method according to claim 3, characterized in that The first target area is located at an edge area of the picture, and the orientation of the first target area in the picture is opposite to the moving direction of the target object; And / or, controlling the pan-tilt head to track the target object according to the intelligent tracking reservation scheme using the radar within the first time period includes: During the first time period, the data fed back by the radar is used to select a corresponding first calibrated PT offset to control the movement of the pan / tilt platform, and / or a corresponding calibrated magnification is selected to adjust the image magnification of the acquisition device.

5. The method according to claim 4, characterized in that The first calibration PT offset and the calibration magnification are pre-calibrated according to the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

6. The method according to claim 3, characterized in that After detecting whether the target object meets the tracking condition using the image captured after the pan / tilt head stops, the method further includes: In response to not detecting that the image satisfies the tracking condition within a second time period, the radar is used to control the pan-tilt head to track the target object according to a normal scheme, wherein the normal scheme enables the target object to be located in a second target area of the picture taken by the acquisition device, and the second target area is closer to the center of the picture than the first target area.

7. The method according to claim 6, characterized in that After detecting whether the target object meets the tracking condition using the image captured after the pan / tilt head stops, the method further includes: In response to not detecting that the image satisfies the tracking condition within a second time period, recording a cumulative number of consecutive missed detections of the target object; In response to the cumulative number of consecutive missed reports exceeding a first threshold, a first relationship between the image dwell time and a preset intelligent tracking relay time is adjusted so that the adjusted first relationship does not meet intelligent tracking requirements.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining an actual tracking time of the target object using the intelligent tracking algorithm; In response to a second relationship between the actual tracking duration and the preset tracking duration satisfying a normal tracking requirement, ending the use of the intelligent tracking algorithm to control the pan / tilt head to track the target object; and / or, In response to the second relationship not meeting the normal tracking requirement, recording the number of consecutive tracking failures of the target object; in response to the number of consecutive tracking failures exceeding a second number threshold, adjusting the first relationship between the image dwell time and the preset intelligent tracking relay time so that the adjusted first relationship does not meet the intelligent tracking requirement.

9. The method according to claim 8, characterized in that The normal tracking requirement is that the actual tracking duration is greater than the product of the preset tracking duration and the second redundancy weight.

10. The method according to claim 1, characterized in that After determining whether the first relationship meets the intelligent tracking requirement, the method further includes: In response to the first relationship not meeting the intelligent tracking requirement, controlling the pan / tilt head to track the target object using a radar according to a normal scheme, wherein the normal scheme enables the target object to be located in a second target area of the image captured by the acquisition device; And / or, before determining whether the first relationship meets the intelligent tracking requirement, the method further includes: In response to the target object triggering tracking, the radar is used to control the pan-tilt head to track the target object according to the normal scheme.

11. The method according to claim 6 or 10, characterized in that Using the radar to control the pan-tilt head to track the target object according to the normal scheme includes: Using the data fed back by the radar, a corresponding second calibration PT offset is selected to control the movement of the pan / tilt platform, and / or a corresponding calibration magnification is selected to adjust the image magnification of the acquisition device.

12. The method according to claim 11, characterized in that The second calibration PT offset and the calibration magnification are pre-calibrated according to the type of the target object, the distance between the target object and the acquisition device, and the moving speed of the target object.

13. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the pan-tilt tracking method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the pan-tilt tracking method according to any one of claims 1 to 12.

Images