Pan-tilt deviation determination method and device, program product and pan-tilt camera equipment

Abstract

The invention belongs to the technical field of pan-tilt correction, and particularly relates to a pan-tilt deviation determination method and device, a computer program product and pan-tilt camera equipment. The method comprises the following steps: controlling the pan-tilt camera equipment to move back and forth at two different positions, and determining a rough pan-tilt deviation of the pan-tilt camera equipment based on an initial image and an end point image; based on the rough pan-tilt deviation, controlling the pan-tilt camera equipment to move from the final position to the initial position until the current image feature deviation is smaller than a feature deviation threshold value, and stopping the pan-tilt camera equipment from moving; wherein the current image feature deviation is the deviation between the features of the current image and the features of the initial image, and the current image is the image collected at the current position; and under the condition that the pan-tilt camera equipment stops moving, the accurate pan-tilt deviation is determined based on the final position and the current position.

Description

Technical Field

[0001] The present application belongs to the technical field of pan / tilt correction, and in particular, relates to a pan / tilt deviation determination method, device, computer program product and pan / tilt camera equipment. Background Art

[0002] A camera device with a pan / tilt motor (referred to as a pan / tilt camera device) can rotate at different angles, thereby acquiring images or videos at different angles.

[0003] When the gimbal camera device is affected by external factors or the overall structure of the device is loose, when it moves back and forth in a certain direction (such as horizontal dimension or vertical dimension) at the same step number, the starting position and the end position of the reciprocating motion do not coincide (that is, there is a gimbal deviation), resulting in inconsistent front and back shooting angles. When a gimbal deviation occurs, it is usually necessary to determine the gimbal deviation, and the gimbal camera device can be corrected based on the gimbal deviation.

[0004] However, the gimbal deviation determination method in the prior art often estimates the gimbal deviation by adding a motor angle rotation sensor, which results in high hardware cost. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, apparatus, computer program product and pan-tilt camera device for determining pan-tilt deviation to solve the problem of high hardware cost of the pan-tilt deviation determination method in the prior art.

[0006] A first aspect of an embodiment of the present application provides a method for determining a gimbal deviation, which may include:

[0007] Controlling the pan-tilt camera device to perform reciprocating motion at two different positions, and determining a rough pan-tilt deviation of the pan-tilt camera device based on a starting image and an ending image; wherein the starting image is an image captured at the starting position of the reciprocating motion, and the ending image is an image captured at the ending position of the reciprocating motion;

[0008] Based on the rough pan-tilt deviation, the pan-tilt camera device is controlled to move from the end position to the starting position until the current image feature deviation is less than the feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is the deviation between the feature of the current image and the feature of the starting image, and the current image is an image captured at the current position;

[0009] When the pan-tilt camera device stops moving, a precise pan-tilt deviation is determined based on the end position and the current position.

[0010] A second aspect of an embodiment of the present application provides a device for determining a gimbal deviation, which may include:

[0011] A rough estimation module, used to control the pan-tilt camera device to perform a reciprocating motion at two different positions, and determine a rough pan-tilt deviation of the pan-tilt camera device based on a start image and an end image; wherein the start image is an image captured at the start position of the reciprocating motion, and the end image is an image captured at the end position of the reciprocating motion;

[0012] A motion module, used for controlling the pan-tilt camera device to move from the end position to the starting position based on the rough pan-tilt deviation, until the current image feature deviation is less than a feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is a deviation between a feature of the current image and a feature of the starting image, and the current image is an image captured at the current position;

[0013] The deviation determination module is used to determine the precise pan-tilt deviation based on the end position and the current position when the pan-tilt camera device stops moving.

[0014] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned gimbal deviation determination methods are implemented.

[0015] The fourth aspect of an embodiment of the present application provides a gimbal camera device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the gimbal camera device implements the steps of any one of the above-mentioned gimbal deviation determination methods.

[0016] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program, which, when executed, enables any of the above-mentioned gimbal deviation determination methods to be executed.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application control the pan-tilt camera device to perform reciprocating motion at two different positions, and determine the rough pan-tilt deviation of the pan-tilt camera device based on the starting image and the end image; wherein the starting image is an image captured at the starting position of the reciprocating motion, and the end image is an image captured at the end position of the reciprocating motion; based on the rough pan-tilt deviation, the pan-tilt camera device is controlled to move from the end position to the starting position until the current image feature deviation is less than the feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is the deviation between the feature of the current image and the feature of the starting image, and the current image is an image captured at the current position; when the pan-tilt camera device stops moving, the precise pan-tilt deviation is determined based on the end position and the current position. In the embodiments of the present application, the precise pan-tilt deviation can be determined by image feature tracking, so that the pan-tilt deviation can be efficiently estimated without increasing hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flow chart of an embodiment of a method for determining a gimbal deviation in an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the reciprocating motion process of the pan-tilt camera device;

[0021] Figure 3 Schematic diagram of feature tracking of an image;

[0022] Figure 4 This is a structural diagram of an embodiment of a device for determining a pan / tilt deviation in an embodiment of the present application;

[0023] Figure 5 This is a schematic block diagram of a pan-tilt camera device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0026] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0027] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0029] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0030] A camera device with a pan / tilt motor (referred to as a pan / tilt camera device) can rotate at different angles, thereby acquiring images or videos at different angles.

[0031] When the gimbal camera device is affected by external factors or the overall structure of the device is loose, when it moves back and forth in a certain direction (such as horizontal dimension or vertical dimension) at the same step number, the starting position and the end position of the reciprocating motion do not coincide (that is, there is a gimbal deviation), resulting in inconsistent front and back shooting angles. When a gimbal deviation occurs, it is usually necessary to determine the gimbal deviation, and the gimbal camera device can be corrected based on the gimbal deviation.

[0032] However, the gimbal deviation determination method in the prior art often estimates the gimbal deviation by adding a motor angle rotation sensor, which results in high hardware cost.

[0033] In view of this, the embodiments of the present application provide a gimbal deviation determination method, apparatus, computer program product and gimbal camera device to solve the problem of high hardware cost of the gimbal deviation determination method in the prior art.

[0034] It should be noted that the executor of the method of the present application is a pan-tilt camera device, specifically, it can be a camera device equipped with a pan-tilt motor, such as a pan-tilt camera, pan-tilt monitoring, or any other common camera device in the prior art.

[0035] In an embodiment of the present application, a rough gimbal deviation estimate can be first made for the gimbal camera device to obtain a rough gimbal deviation, and the gimbal camera device can be controlled to move to a theoretical position when there is no gimbal deviation based on the rough gimbal deviation, so that the gimbal camera device is as close to the theoretical position as possible; during the movement, feature tracking can be performed through images collected in real time to determine the distance between the gimbal camera device and the theoretical position; when the distance between the position of the gimbal camera device and the theoretical position is small, the gimbal camera device can be controlled to stop moving, and the precise gimbal deviation can be determined based on the number of steps of the gimbal motor during the movement.

[0036] See also Figure 1 In an embodiment of the present application, a method for determining a gimbal deviation may include:

[0037] Step S101: Control the pan-tilt camera device to perform reciprocating motion at different positions, and determine a rough pan-tilt deviation of the pan-tilt camera device based on a start image and an end image.

[0038] In an embodiment of the present application, the pan-tilt camera device can be controlled to move back and forth between two different positions in a target dimension (e.g., a horizontal dimension or a vertical dimension). For example, the pan-tilt camera device can be controlled to move back and forth between position A and position B in the horizontal dimension.

[0039] Specifically, the pan-tilt camera device can be controlled to move back and forth between a starting position and a target position; here, the pan-tilt camera device can be controlled to move a first step from the starting position to a first direction, and the position when the pan-tilt camera device stops moving can be called the target position.

[0040] Among them, the first direction is the direction of movement from the starting position to the target position. Here, the previous movement direction of the gimbal camera device in the target dimension can be determined as the first direction; for example, the target dimension is the horizontal dimension, and the previous movement direction of the gimbal camera device in the horizontal dimension is horizontal to the left, then the first direction can be determined as horizontal to the left; the first step number can be set according to actual needs, and the embodiment of the present application does not limit this.

[0041] When the motor angle of the gimbal motor is at the boundary value of the motor angle, the gimbal motor will not be able to rotate. In order to smoothly perform reciprocating motion, the starting position can be set to a position far away from the boundary value of the motor angle.

[0042] Afterwards, the pan-tilt camera device can be controlled to move from the target position to the second direction for the first number of times and then stop moving, and the position when the pan-tilt camera device stops moving can be called the end position; wherein the second direction is the opposite direction to the first direction; for example, if the first direction is horizontally to the left, then the second direction is horizontally to the right.

[0043] As just an example, the above reciprocating motion process can be as follows Figure 2 As shown, here, the pan-tilt camera device can be controlled to move back and forth between position A and position B; specifically, position A can be used as the starting position, and the pan-tilt camera device can be controlled to move horizontally from position A to the left for the first step to position B; thereafter, the pan-tilt camera device can be controlled to move horizontally from position B to the right for the first step to position C, and position C is the end position; since position A and position C do not overlap, it can be considered that the pan-tilt camera device has a pan-tilt deviation.

[0044] In an embodiment of the present application, before controlling the pan-tilt camera device to move from a starting position to a target position, image acquisition can be performed at the starting position, and the image acquired at the starting position can be referred to as a starting image; when the pan-tilt camera device is located at an end position, image acquisition can be performed at the end position, and the image acquired at the end position can be referred to as an end image; based on the starting image and the end image, the pan-tilt deviation can be roughly estimated to obtain a rough pan-tilt deviation.

[0045] Specifically, feature extraction can be performed on the starting image and the end image to obtain features of the starting image and features of the end image respectively; then, feature matching can be performed on the features of the end image and the features of the starting image to obtain features in the end image that match the features of the starting image (called successful matching features); based on the deviation between the successful matching features of the end image and the corresponding features in the starting image, a rough gimbal deviation can be determined.

[0046] Here, the difference in pixel position between each successfully matched feature in the end image and the corresponding feature in the start image may be calculated, and then the average value of the difference may be calculated, and the average value of the difference may be determined as the first image feature deviation.

[0047] Based on the average value, the deviation angle corresponding to the average value can be determined; based on the deviation angle, the number of motor deviation steps between the end position and the starting position can be determined, and the number of motor deviation steps can be determined as the rough gimbal deviation.

[0048] The image features referred to in the embodiments of the present application may be any common image features, and may specifically include but are not limited to image feature points, line segments or object features. The following will take the image feature points as the image features as an example to illustrate the process of determining the rough gimbal deviation.

[0049] In this example, feature points of the starting image and feature points of the end image can be extracted respectively. The specific feature point extraction method can be any common image feature point extraction method in the prior art, for example, the feature point extraction method can be Oriented FAST and Rotated BRIEF (ORB) or Scale-Invariant Feature Transform (SIFT). Here, feature point extraction can be performed on the starting image I0 and the end image I1 based on the ORB method to obtain various features of the starting image (here, the ORB feature points of the starting image) and various features of the end image (here, the ORB feature points of the end image), and the set of various features of the starting image can be recorded as the feature set Pt0 of the starting image, and the set of various features of the end image can be recorded as the feature set Pt1 of the end image.

[0050] Afterwards, the feature set Pt1 of the end image can be matched with the feature set Pt0 of the starting image to obtain successfully matched feature pairs; based on each successfully matched feature pair, the average value Ps0 (i.e., the first image feature deviation) of the difference in pixel positions between the successfully matched feature pairs can be calculated, and the specific calculation formula is:

[0051]

[0052] in, is the i-th feature in Pt1, For Pt0 The features of successful matching, express At the pixel position in the destination image, express The pixel position in the starting image.

[0053] Based on the first image feature deviation Ps0, the deviation angle Ang0 corresponding to the end image and the start image can be calculated. If the end image and the start image are distorted images, the deviation angle Ang0 can be approximated as:

[0054] Ang0=Ps0 / f,

[0055] If the end image and the start image are images without distortion, the deviation angle Ang0 can be approximated as:

[0056] Ang0=tan -1 (Ps0 / f).

[0057] It should be noted that what is calculated here is only the gimbal deviation in the target dimension (single-direction dimension), and the obtained deviation angle Ang0 will ignore the angle deviations in other directional dimensions; for example, if the target dimension is the horizontal dimension, the calculated deviation angle Ang0 will ignore the angle deviation in the vertical dimension; for another example, if the target dimension is the vertical dimension, the calculated deviation angle Ang0 will ignore the angle deviation in the horizontal dimension.

[0058] Afterwards, the corresponding motor deviation step number Bs0 can be determined based on the deviation angle Ang0, and the motor deviation step number can be determined as the rough gimbal deviation. In order to improve the efficiency of determining the gimbal deviation, the embodiment of the present application can pre-establish a mapping relationship between the deviation angle and the motor deviation step number, and when determining the rough gimbal deviation, the motor deviation step number corresponding to the deviation angle can be efficiently determined directly based on the mapping relationship.

[0059] Step S102: Based on the rough pan / tilt deviation, the pan / tilt camera device is controlled to move from the end position to the start position until the current image feature deviation is less than the feature deviation threshold, and the pan / tilt camera device stops moving.

[0060] In an embodiment of the present application, the pan-tilt camera device can be controlled to move from the end position to the starting position so that the pan-tilt camera device is aligned with the starting position as much as possible; during this movement, image acquisition can be performed in real time, and the image acquired at the current position can be called the current image; wherein the current position is the position of the pan-tilt camera device at the current moment; at the same time, the current image feature deviation can also be calculated in real time, and the current image feature deviation is the deviation between the features of the current image and the features of the starting image.

[0061] If the current image feature deviation is greater than or equal to the preset feature deviation threshold, it indicates that the current image is significantly different from the starting image, that is, the distance between the current position and the starting position is large. At this time, the pan-tilt camera device can continue to be controlled to move from the current position to the starting position.

[0062] If the current image feature deviation is less than the feature deviation threshold, it indicates that the difference between the current image and the starting image is small, that is, the distance between the current position and the starting position is small. Therefore, it can be considered that the deviation correction of the gimbal camera device has been completed at this time, and the gimbal camera device can be controlled to stop moving.

[0063] The above characteristic deviation threshold can be specific and situationally set according to actual needs, and the embodiment of the present application does not limit this. In a specific implementation, the characteristic deviation threshold can be the rotation angle value corresponding to one step of the pan / tilt motor rotation.

[0064] In a specific implementation of the embodiment of the present application, during the movement of the pan-tilt camera device from the end position to the starting position, the number of motor deviation steps between the current position and the starting position can be determined, and the movement speed of the pan-tilt camera device can be determined based on the number of motor deviation steps, so that the movement speed of the pan-tilt camera device is positively correlated with the number of motor deviation steps. Specifically, if the distance between the current position of the pan-tilt camera device and the starting position is large, it can move to the starting position at a larger movement speed to reduce the time consumption of the overall process; if the distance between the current position of the pan-tilt camera device and the starting position is small, it can move to the starting position at a smaller movement speed, so that the pan-tilt camera device is more accurately aligned with the starting position.

[0065] In the above implementation, the motor deviation step number between the current position and the starting position can be determined based on the rough pan / tilt deviation and the current position; specifically, the motor steps rotated from the end position to the current position can be obtained, and the motor deviation step number Bs between the current position and the starting position can be calculated based on the rough pan / tilt deviation and the motor steps rotated; according to the motor deviation step number Bs, the movement speed y of the pan / tilt camera device can be determined, and the specific formula can be:

[0066] y=ke Bs ,

[0067] Here, k is a preset constant, and the specific value of k can be set according to actual needs, which is not limited in the embodiment of the present application.

[0068] When the pan-tilt camera device moves from the end position to the start position, the current image feature deviation can be calculated in real time so that the pan-tilt camera device can stop moving in time when it is aligned with the start position.

[0069] In a specific implementation method of an embodiment of the present application, the current image feature deviation can be determined based on feature matching between features of the current image and features of the starting image; specifically, the feature set of the current image can be feature matched with the feature set of the starting image, and based on each successfully matched feature pair, the average value of the difference in pixel positions between the successfully matched feature pairs can be calculated, and the average value can be used as the current image feature deviation; if the current image feature deviation is greater than or equal to a feature deviation threshold, the gimbal camera device can continue to be controlled to move from the current position to the starting position; if the current image feature deviation is less than the feature deviation threshold, the gimbal camera device can be controlled to stop moving and step S103 is executed.

[0070] In another specific implementation of the embodiment of the present application, the current image feature deviation can be determined by tracking the features of the previous image based on the features of the current image, where the previous image is an image captured by the pan-tilt camera device at the previous moment.

[0071] Specifically, the current image I k Perform feature extraction to obtain the features of the current image; then, the features of the current image can be compared with the features Pt successfully tracked in the previous image. k-1 (called the tracking success feature Pt of the previous image k-1 ) to match and obtain the successful tracking feature Pt of the current image k The feature tracking method here can be any common feature tracking method in the prior art, such as optical flow method, Kalman filtering, etc.

[0072] The above feature tracking will continue throughout the entire process of the PTZ camera moving from the end position to the starting position; as an example, see Figure 3At time 1, the pan-tilt camera device is located at position 1 (end position) and moves from position 1 to the starting position; at time 2, the pan-tilt camera device moves to position 2. At this time, the feature Pt1 of the previous image (the image captured at position 1, i.e., the end image I1) can be tracked based on the feature of the current image I2 (i.e., the image captured at position 2), and the tracking success feature Pt2 of the current image I2 can be obtained; at time 3, the pan-tilt camera device moves to position 3. At this time, the tracking success feature Pt2 in the previous image I2 (the image captured at position 2) can be tracked based on the feature of the current image I3 (i.e., the image captured at position 3), and the tracking success feature Pt3 of the current image I3 can be obtained; and so on, until the pan-tilt camera device stops moving.

[0073] Optionally, in order to avoid the influence of moving objects on the feature tracking process and resulting in inaccurate calculation of the gimbal deviation, foreground detection can be performed on the previous image to obtain the foreground features of the previous image, and the foreground features of the previous image can be eliminated from the tracking success features of the previous image to obtain the updated tracking success features of the previous image, and the features of the current image can be matched with the tracking success features of the updated previous image.

[0074] Optionally, in a long feature tracking process, the number of successfully tracked features may gradually decrease. In order to avoid too few features to be tracked, here, when the number of features to be tracked is small, each feature extracted from the current image can be directly tracked in the subsequent tracking. Specifically, the features of the current image can be matched with the successfully tracked features of the previous image to obtain each successfully matched feature of the current image; if the number of each successfully matched feature is greater than or equal to the successfully matched number threshold, then each successfully matched feature of the current image can be determined as the successfully tracked feature of the current image; if the number of each successfully matched feature is less than the successfully matched number threshold, it can be considered that the number of successfully tracked features is small. At this time, each feature of the current image can be directly determined as the successfully tracked feature of the current image, and at the next moment, each feature of the current image can be tracked to ensure that there are enough features to be tracked. The above-mentioned successfully matched number threshold can be specific and situational according to actual needs, and this application does not limit this.

[0075] Afterwards, the current image feature deviation can be determined based on the successful tracking feature of the current image; here, the deviation between the successful tracking feature of the current image and the corresponding feature in the endpoint image can be calculated to obtain the second image feature deviation; specifically, the average value of the difference in pixel positions of each pair of features that have been successfully tracked at adjacent moments can be calculated.

[0076] Taking the current moment (moment k) and the previous moment (moment k-1) as an example, the current image I k Tracking success characteristics Pt k With the previous image I k-1 The average value Ps of the difference in pixel position of the corresponding successful tracking feature k for:

[0077]

[0078] in, For the previous image Pt k-1 One of the features, is the current image Pt k Zhongyu The features that match successfully (i.e., and is the tracking successful feature pair), Indicated in The pixel position in the previous image, express In the current image Pt k The pixel position in .

[0079] In the period of movement from the end position to the current position, the average value of the difference in the pixel positions of each pair of features that are successfully tracked can be calculated according to the above method at any adjacent time to obtain the average value of the difference in each pixel position; by summing up each average value, the deviation between the tracked feature of the current image and the corresponding feature in the end image (i.e., the second image feature deviation PI k ):

[0080]

[0081] Among them, Ps i Represents the average value of the difference between the pixel positions of the successfully tracked features at time i and time i-1.

[0082] Based on the first image feature deviation Ps0 (the deviation between the matching successful feature of the end image and the corresponding feature in the starting image) and the second image feature deviation PI k , the current image feature deviation SPI can be determined k Specifically, the first image feature deviation Ps0 and the second image feature deviation PI k Add them together to get the current image feature deviation SPI k :

[0083] SPI k =PI k +Ps0,

[0084] If the current image feature deviation SPIk Greater than or equal to the feature deviation threshold T ps , the PTZ camera device can continue to be controlled to move from the current position to the starting position; if the current image feature deviation SPI k Less than the characteristic deviation threshold T ps , the pan-tilt camera device can be controlled to stop moving and step S103 is executed.

[0085] Step S103: When the pan-tilt camera device stops moving, determine the precise pan-tilt deviation based on the end position and the current position.

[0086] In the embodiment of the present application, when the pan-tilt camera device stops moving, the number of steps B of the pan-tilt motor during the movement of the pan-tilt camera device from the end position to the current position can be obtained. k , and the number of steps B can be k Determined as precise gimbal deviation.

[0087] Accordingly, the precise gimbal deviation can be determined based on the image feature tracking method, thereby eliminating the need to introduce additional hardware, thereby achieving efficient and accurate gimbal deviation estimation without increasing hardware costs.

[0088] In a specific implementation method of an embodiment of the present application, after determining the precise gimbal deviation of the target dimension, the precise gimbal deviation of other dimensions can be determined according to the same method. For example, if the target dimension is the horizontal dimension, after determining the precise gimbal deviation of the horizontal dimension, the precise gimbal deviation of the vertical dimension can be determined based on the same method.

[0089] In another specific implementation of the embodiment of the present application, after the precise pan-tilt deviation of the target dimension is determined, when the pan-tilt camera device performs a reciprocating motion in the target dimension, the pan-tilt deviation compensation can be performed based on the precise pan-tilt deviation. For example, the precise pan-tilt deviation of the pan-tilt camera device in the horizontal dimension is B k If the PTZ camera moves a specific number of steps from position 1 to position 2 in the horizontal dimension to position 2, and then the PTZ camera moves the same specific number of steps from position 2 to position 1, then the PTZ camera can be controlled to move to position 1 based on the specific number of steps. k , so that the gimbal camera device can smoothly return to position 1, which can greatly reduce the impact of the gimbal deviation on the normal use of the gimbal camera device and improve the user experience.

[0090] In summary, the embodiment of the present application controls the pan-tilt camera device to perform reciprocating motion at two different positions, and determines the rough pan-tilt deviation of the pan-tilt camera device based on the starting image and the end image; wherein the starting image is an image captured at the starting position of the reciprocating motion, and the end image is an image captured at the end position of the reciprocating motion; based on the rough pan-tilt deviation, the pan-tilt camera device is controlled to move from the end position to the starting position until the current image feature deviation is less than the feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is the deviation between the feature of the current image and the feature of the starting image, and the current image is an image captured at the current position; when the pan-tilt camera device stops moving, the precise pan-tilt deviation is determined based on the end position and the current position. In the embodiment of the present application, the precise pan-tilt deviation can be determined by image feature tracking, so that the pan-tilt deviation can be efficiently estimated without increasing hardware costs.

[0091] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0092] Corresponding to the method for determining the gimbal deviation described in the above embodiment, Figure 4 A structural diagram of an embodiment of a gimbal deviation determination device provided in an embodiment of the present application is shown.

[0093] In an embodiment of the present application, a device for determining a gimbal deviation may include:

[0094] A rough estimation module 401 is used to control the pan-tilt camera device to perform a reciprocating motion at two different positions, and determine a rough pan-tilt deviation of the pan-tilt camera device based on a start image and an end image; wherein the start image is an image captured at the start position of the reciprocating motion, and the end image is an image captured at the end position of the reciprocating motion;

[0095] A motion module 402 is used to control the pan-tilt camera device to move from the end position to the starting position based on the rough pan-tilt deviation until the current image feature deviation is less than a feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is a deviation between a feature of the current image and a feature of the starting image, and the current image is an image captured at the current position;

[0096] The deviation determination module 403 is used to determine the precise pan-tilt deviation based on the end position and the current position when the pan-tilt camera device stops moving.

[0097] In a specific implementation of the embodiment of the present application, the device for determining the gimbal deviation further includes:

[0098] A feature extraction module, used to extract features from the current image to obtain features of the current image;

[0099] A feature matching module, used for matching the feature of the current image with the tracking success feature of the previous image to obtain the tracking success feature of the current image; wherein the previous image is an image acquired at the position at the previous moment;

[0100] Furthermore, the deviation determination module is used to determine the feature deviation of the current image based on the tracking success feature of the current image.

[0101] In a specific implementation of the embodiment of the present application, the feature matching module includes:

[0102] A feature matching submodule, used for matching the features of the current image with the tracking success features of the previous image to obtain each matching success feature;

[0103] A feature determination submodule, configured to determine each of the successfully matched features as a successful tracking feature of the current image if the number of each of the successfully matched features is greater than or equal to a successfully matched number threshold;

[0104] The feature determination submodule is further configured to determine the feature of the current image as a successful tracking feature of the current image if the number of each of the successful matching features is less than the successful matching number threshold.

[0105] In a specific implementation of the embodiment of the present application, the feature matching module further includes:

[0106] A detection submodule, used to perform foreground detection on the previous image to obtain foreground features of the previous image;

[0107] A removal submodule, used for removing the foreground feature of the previous image from the tracking success feature of the previous image to obtain an updated tracking success feature of the previous image;

[0108] The feature matching module is also used to match the feature of the current image with the tracking success feature of the previous image to obtain the tracking success feature of the current image, which is: matching the feature of the current image with the updated tracking success feature of the previous image to obtain the tracking success feature of the current image.

[0109] In a specific implementation of the embodiment of the present application, the feature matching module includes:

[0110] A first deviation determination submodule is used to calculate the deviation between the successfully matched feature of the end image and the corresponding feature in the starting image to obtain a first image feature deviation;

[0111] A deviation determination second submodule is used to calculate the deviation between the tracking success feature of the current image and the corresponding feature in the endpoint image to obtain a second image feature deviation;

[0112] The deviation determination third submodule is used to determine the current image feature deviation based on the first image feature deviation and the second image feature deviation.

[0113] In a specific implementation of the embodiment of the present application, the motion module includes:

[0114] a step number determination submodule, for determining the motor deviation step number between the current position and the starting position based on the rough gimbal deviation and the current position;

[0115] A speed determination submodule, used to determine the movement speed of the pan-tilt camera device based on the motor deviation step number; wherein the movement speed is positively correlated with the motor deviation step number;

[0116] The first submodule of the motion submodule is used to control the pan-tilt camera device to move from the end position to the starting position based on the motion speed until the current image feature deviation is less than the feature deviation threshold.

[0117] In a specific implementation of the embodiment of the present application, the rough estimation module includes:

[0118] The second submodule of the motion submodule is used to control the pan / tilt camera device to move from the starting position to the first direction for the first number of steps to the target position;

[0119] The third submodule of the motion submodule is used to control the pan-tilt camera device to stop moving after the pan-tilt camera device moves from the target position to the second direction for the first number of times; wherein the second direction is a direction opposite to the first direction;

[0120] A position determination submodule, used to determine the position of the pan / tilt camera device when it stops moving as the end position;

[0121] The rough estimation submodule is used to determine the rough gimbal deviation based on the characteristics of the endpoint image and the characteristics of the starting image.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] Figure 5 A schematic block diagram of a pan-tilt camera device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0125] like Figure 5 As shown, the pan-tilt camera device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned pan-tilt deviation determination method embodiments are implemented, for example Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 4 The functions of modules 401 to 403 are shown.

[0126] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 52 in the pan-tilt camera device 5.

[0127] Those skilled in the art will understand that Figure 5 It is only an example of the pan-tilt camera device 5 and does not constitute a limitation of the pan-tilt camera device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the pan-tilt camera device 5 may also include input and output devices, network access devices, buses, etc.

[0128] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0129] The memory 51 may be an internal storage unit of the pan-tilt camera device 5, such as a hard disk or memory of the pan-tilt camera device 5. The memory 51 may also be an external storage device of the pan-tilt camera device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the pan-tilt camera device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the pan-tilt camera device 5. The memory 51 is used to store the computer program and other programs and data required by the pan-tilt camera device 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0130] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] In the embodiments provided in the present application, it should be understood that the disclosed device / pan-tilt camera equipment and method can be implemented in other ways. For example, the device / pan-tilt camera equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0136] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0137] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for determining a pan / tilt deviation, characterized in that: include: Controlling the pan-tilt camera device to perform reciprocating motion at two different positions, and determining a rough pan-tilt deviation of the pan-tilt camera device based on a starting image and an ending image; wherein the starting image is an image captured at the starting position of the reciprocating motion, and the ending image is an image captured at the ending position of the reciprocating motion; Based on the rough pan-tilt deviation, the pan-tilt camera device is controlled to move from the end position to the starting position until the current image feature deviation is less than the feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is the deviation between the feature of the current image and the feature of the starting image, and the current image is an image captured at the current position; When the pan-tilt camera device stops moving, a precise pan-tilt deviation is determined based on the end position and the current position.

2. The method for determining the pan / tilt deviation according to claim 1, characterized in that: The process of determining the current image feature deviation includes: Performing feature extraction on the current image to obtain features of the current image; Matching the features of the current image with the tracking success features of the previous image to obtain the tracking success features of the current image; wherein the previous image is an image acquired at the position at the previous moment; The current image feature deviation is determined based on the tracking success feature of the current image.

3. The method for determining the pan / tilt deviation according to claim 2, characterized in that: The matching of the feature of the current image with the tracking success feature of the previous image to obtain the tracking success feature of the current image includes: Matching the features of the current image with the tracking success features of the previous image to obtain respective matching success features; If the number of each of the successfully matched features is greater than or equal to a successfully matched number threshold, each of the successfully matched features is determined as a successfully tracked feature of the current image; If the number of each of the successfully matched features is less than the successfully matched number threshold, the feature of the current image is determined as a successfully tracked feature of the current image.

4. The method for determining the pan / tilt deviation according to claim 2, characterized in that: Before matching the feature of the current image with the tracking success feature of the previous image to obtain the tracking success feature of the current image, the method further includes: Performing foreground detection on the previous image to obtain foreground features of the previous image; Eliminating the foreground feature of the previous image from the tracking success feature of the previous image to obtain an updated tracking success feature of the previous image; The matching of the feature of the current image with the tracking success feature of the previous image to obtain the tracking success feature of the current image is: matching the feature of the current image with the updated tracking success feature of the previous image to obtain the tracking success feature of the current image.

5. The method for determining the pan / tilt deviation according to claim 2, characterized in that: The determining the current image feature deviation based on the tracking success feature of the current image includes: Calculating the deviation between the successfully matched feature of the end point image and the corresponding feature in the starting image to obtain a first image feature deviation; Calculating the deviation between the tracking success feature of the current image and the corresponding feature in the endpoint image to obtain a second image feature deviation; The current image feature deviation is determined based on the first image feature deviation and the second image feature deviation.

6. The method for determining the pan / tilt deviation according to claim 1, characterized in that: The step of controlling the pan-tilt camera device to move from the end position to the start position based on the rough pan-tilt deviation until the current image feature deviation is less than a feature deviation threshold and the pan-tilt camera device stops moving includes: Based on the rough gimbal deviation and the current position, determining the number of motor deviation steps between the current position and the starting position; Based on the motor deviation step number, determining the movement speed of the pan-tilt camera device; wherein the movement speed is positively correlated with the motor deviation step number; Based on the movement speed, the pan-tilt camera device is controlled to move from the end position to the starting position until the current image feature deviation is less than the feature deviation threshold.

7. The method for determining the pan / tilt deviation according to any one of claims 1 to 6, characterized in that: The step of controlling the pan-tilt camera device to perform reciprocating motion at two different positions and determining a rough pan-tilt deviation of the pan-tilt camera device based on a starting image and an end point image includes: Control the pan / tilt camera device to move from the starting position to the first direction for a first number of steps to the target position; After controlling the pan-tilt camera device to move from the target position to a second direction for the first number of times, the pan-tilt camera device stops moving; wherein the second direction is a direction opposite to the first direction; Determine the position of the pan / tilt camera device when it stops moving as the end position; The rough gimbal deviation is determined based on the features of the endpoint image and the features of the start image.

8. A device for determining a pan / tilt deviation, characterized in that: include: A rough estimation module, used to control the pan-tilt camera device to perform a reciprocating motion at two different positions, and determine a rough pan-tilt deviation of the pan-tilt camera device based on a start image and an end image; wherein the start image is an image captured at the start position of the reciprocating motion, and the end image is an image captured at the end position of the reciprocating motion; A motion module, used for controlling the pan-tilt camera device to move from the end position to the starting position based on the rough pan-tilt deviation, until the current image feature deviation is less than a feature deviation threshold, and the pan-tilt camera device stops moving; wherein the current image feature deviation is a deviation between a feature of the current image and a feature of the starting image, and the current image is an image captured at the current position; The deviation determination module is used to determine the precise pan-tilt deviation based on the end position and the current position when the pan-tilt camera device stops moving.

9. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables the pan / tilt deviation determination method according to any one of claims 1 to 7 to be executed.

10. A pan-tilt camera device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the pan-tilt camera device implements the steps of the pan-tilt deviation determination method according to any one of claims 1 to 7.

Images