Data deviation processing method and device

Abstract

The invention provides a data deviation processing method and device which are used for improving the accuracy of data deviation processing. The method comprises the following steps: acquiring N first images shot by first equipment at a first preset position and N second images shot by second equipment at a second preset position; determining a first target image and a second target image satisfying a similarity corresponding relation; acquiring a first position of the first target image and a second position of the second target image, and determining position deviation of the first equipment and the second equipment based on deviation of the first position and the second position; determining at least one group of image pairs satisfying a similarity corresponding relation in the N first images and the N second images; registering each group of image pairs to obtain a registration matrix; calculating the coordinate angle deviation of the first equipment and the second equipment according to the registration matrix, the width and height of the image and the field angle of the second equipment; and determining a target preset position coordinate of the second equipment based on the coordinate, the position deviation and the coordinate angle deviation of the first equipment.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a data deviation processing method and device. Background Art

[0002] When a camera device has an abnormality and needs to be disassembled for repair or needs to be updated and replaced, due to factors such as the installation location, differences in device hardware and software, if the replaced device is deployed according to the original preset position information, there will be a large data deviation. If the preset position information is reset manually, it will be extremely time-consuming and labor-intensive, and there will be errors caused by human subjective factors.

[0003] In view of this, how to improve the accuracy of data deviation processing is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a data deviation processing method and device for improving the accuracy of data deviation processing.

[0005] In a first aspect, an embodiment of the present application provides a data deviation processing method, wherein a first device and a second device are pan / tilt camera devices, and a shooting starting point and a shooting object of the first device and the second device are the same; the method comprises: According to the first step length, N first images taken by the first device at the first preset position according to the preset rule and N second images taken by the second device at the second preset position according to the preset rule are obtained; the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step length is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are arranged according to their respective shooting order; Determine a first target image from the N first images, and determine a second target image having a similarity correspondence relationship with the first target image from the N second images; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold; Acquire a first position of a first target image in the N first images and a second position of a second target image in the N second images, and determine a position deviation of the first device and the second device in a horizontal direction based on a deviation between the first position and the second position; Determine, according to the correspondence between the first position and the second position, a position correspondence between each first image in the N first images and each second image in the N second images; construct N groups of image pairs according to the position correspondence, each group of image pairs including a first image and a second image corresponding to the positions; Select at least one group of image pairs from the N groups of image pairs, the two images of which satisfy a similarity correspondence relationship; perform registration calculation on the two images in each image pair in the at least one group of image pairs to obtain a registration matrix; calculate the coordinate angle deviation of the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; Based on the coordinates, position deviation and coordinate angle deviation of the first device at the first preset position, the target preset position coordinates of the second device are determined so that the shooting coverage area of ​​the second device at the target preset position is the same as the shooting coverage area of ​​the first device at the first preset position.

[0006] In the present method, N first images taken by the first device at the first preset position and N second images taken by the second device at the second preset position are respectively obtained by the same step length, so as to facilitate the subsequent determination of the position correspondence between the first image and the second image according to the step length; and the first target image and the second target image are determined by similarity calculation, that is, two images with the same captured content and shooting angle are determined, and the position deviation of the first device and the second device in the horizontal direction can be determined according to the deviation between the first position of the first target image in the N first images and the second position of the second target image in the second image; further, based on the correspondence between the first position and the second position , the position correspondence between each first image and each second image can be determined, and then N groups of image pairs are constructed, and at least one group of image pairs that meet the similarity correspondence relationship is screened out from the N groups of image pairs for subsequent deviation calculation, which can improve the accuracy of the deviation calculation; then, each group of image pairs in the at least one group of image pairs is registered, and the coordinate angle deviation of the first device and the second device is calculated according to the preset algorithm based on the obtained registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; the relationship deviation and the coordinate angle deviation are processed by combining the obtained relationship deviation and the coordinate angle deviation, which can improve the accuracy of data deviation processing, so that the determined target preset position coordinates of the second device are more accurate.

[0007] Optionally, N first images taken by the first device at the first preset position according to a preset rule and N second images taken by the second device at the second preset position according to a preset rule are obtained according to a first step length, including: replacing the original magnification of the first device at the first preset position with the minimum magnification, according to the first vertical coordinate and the minimum magnification, starting with the horizontal coordinate 0°, the first step length is 360° divided by N, the gimbal of the first device is horizontally rotated, and N first images are captured; the N first images are arranged in the order of shooting; based on the minimum horizontal field of view corresponding to the minimum magnification of the first device, the first magnification of the second device at the minimum horizontal field of view is queried; according to the second vertical coordinate and the first magnification, starting with the horizontal coordinate 0°, the first step length is 360° divided by N, the gimbal of the second device is horizontally rotated, and N second images are captured; the N second images are arranged in the order of shooting.

[0008] Optionally, determining a first target image from N first images, and determining a second target image that has a similarity correspondence relationship with the first target image from N second images, includes: extracting M first images from the N first images according to a second step length; the second step length is 2n times the first step length, n is a positive integer; M is a positive integer less than N; extracting 2M second images from the N second images according to a third step length; the third step length is half of the second step length; obtaining any first image from the M first images, registering the any first image with each second image in the 2M second images, and calculating the registration. similarity between any first image and the second image after calibration; if there is at least one second image whose similarity with any first image is not less than a preset similarity threshold, the second image with the greatest similarity to any first image is used as the second target image, and any first image is used as the first target image; if there is no second image whose similarity with any first image is not less than the preset similarity threshold, the value of the second vertical coordinate is adjusted, and N second images of the second device at the second preset position are reacquired; the value range of the second vertical coordinate is within the vertical movement coordinate range of the gimbal of the second device.

[0009] Optionally, after taking the second image with the greatest similarity to any of the first images as the second target image and taking any of the first images as the first target image, the method further includes: based on the position correspondence between the first target image and the second target image, and the angle correspondence between the M first images and the 2M second images, acquiring M-1 second images other than the second target image from the second target image; the angle correspondence is determined according to the relationship between the second step length and the third step length; the M-1 second images correspond one-to-one with the M-1 first images other than the first target image in the M first images; forming a group of image pairs with each of the M-1 second images and the corresponding first image in the M-1 first images to obtain M-1 groups of image pairs; performing registration calculation on the two images in each of the M-1 groups of image pairs, and calculating the similarity between the two registered images; counting the number of image pairs whose similarity is not less than a preset similarity threshold; if the number is less than the first number threshold, adjusting the value of the second vertical coordinate, reacquiring the N second images of the second device at the second preset position, and re-determining the first target image and the second target image.

[0010] Optionally, the coordinate angle deviation of the first device and the second device is calculated according to the registration matrix, the width and draft of the image, and the field of view of the second device at the second preset position, including: for any image pair in at least one group of image pairs, performing the following operations: calculating the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation of the first device and the second device according to the linear transformation parameters, projection transformation parameters and translation transformation parameters included in the registration matrix, and the width and height of the image; converting the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into a horizontal coordinate angle deviation and a vertical coordinate angle deviation respectively according to the horizontal field of view and the vertical field of view of the second device at the second preset position, and the width and height of the image; comparing the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of any image pair and the horizontal coordinate angle deviation and the vertical coordinate angle deviation of an adjacent image pair, and if the difference is not less than a preset difference threshold, discarding any image pair; if the difference is less than the preset difference threshold, setting any image pair as a correct image pair; determining the coordinate angle deviation of the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs.

[0011] Optionally, the coordinate angle deviation of the first device and the second device is determined based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs, including: calculating the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the first device and the second device at each angle in the horizontal direction by interpolation based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs; and determining the coordinate angle deviation of the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle.

[0012] Optionally, based on the coordinates, position deviation and coordinate angle deviation of the first device at the first preset position, the target preset position coordinates of the second device are determined, including: obtaining a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from the original magnification to the minimum magnification of the first preset position; querying the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view corresponding to the original magnification; obtaining a second horizontal optical axis deviation value and a second vertical optical axis deviation value of the second device from the second magnification of the second preset position to the first magnification; obtaining an optical axis deviation between the first device and the second device based on the first horizontal optical axis deviation value and the first vertical optical axis deviation value superimposed on the second horizontal optical axis deviation value and the second vertical optical axis deviation value; determining the target preset position coordinates of the second device based on the coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device at the first preset position.

[0013] Optionally, determining the target preset position coordinates of the second device based on the coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device at the first preset position includes: P preset positions are randomly obtained from K preset positions, and J third images and J fourth images of the first device and the second device at each preset position in the P preset positions according to a preset rule are respectively obtained; K is a positive integer, P is a positive integer less than K, and J is a positive integer; J groups of image pairs are determined based on the J third images and the J fourth images in the order of shooting, and the shooting order of the two images in each group of image pairs is the same; the two images in each group of image pairs are registered, and the similarity between the two images in each group of image pairs after registration is calculated, and the number of image pairs whose similarity is not less than a preset similarity threshold is obtained; if the number of image pairs is less than a second number threshold, the target preset position coordinates of the second device are determined based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device; if the number of image pairs is not less than the second number threshold, the average of the coordinate angle deviations of all image pairs whose similarity is not less than the preset similarity threshold is calculated; based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, and the average of the coordinate angle deviation, the target preset position coordinates of the second device are determined.

[0014] In a second aspect, an embodiment of the present application provides a data deviation processing device, wherein a first device and a second device are pan / tilt camera devices, and a shooting starting point and a shooting object of the first device and the second device are the same; the device comprises: The acquisition module is used to: acquire, according to a first step length, N first images taken by a first device at a first preset position according to a preset rule and N second images taken by a second device at a second preset position according to a preset rule; the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step length is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are arranged in the order of their respective shooting sequence; A first calculation module is used to: determine a first target image from the N first images, and determine a second target image that has a similarity correspondence relationship with the first target image from the N second images; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold; obtain a first position of the first target image in the N first images, and a second position of the second target image in the N second images, and determine a position deviation of the first device and the second device in a horizontal direction based on a deviation between the first position and the second position; A second calculation module is used to: determine the position correspondence between each first image in the N first images and each second image in the N second images according to the correspondence between the first position and the second position; construct N groups of image pairs according to the position correspondence, each group of image pairs including a first image and a second image corresponding to the positions; select at least one group of image pairs in which the two images satisfy the similarity correspondence from the N groups of image pairs; perform registration calculation on the two images in each group of image pairs in the at least one group of image pairs to obtain a registration matrix; calculate the coordinate angle deviation of the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; The processing module is used to determine the target preset position coordinates of the second device based on the coordinates, position deviation and coordinate angle deviation of the first device at the first preset position, so that the shooting coverage area of ​​the second device at the target preset position is the same as the shooting coverage area of ​​the first device at the first preset position.

[0015] Optionally, an acquisition module is specifically used to: acquire, according to a first step length, N first images shot by the first device at a first preset position according to a preset rule and N second images shot by the second device at a second preset position according to a preset rule, including: replacing the original magnification of the first device at the first preset position with the minimum magnification, according to the first vertical coordinate and the minimum magnification, starting with the horizontal coordinate 0°, and the first step length is 360° divided by N, horizontally rotating the gimbal of the first device, and capturing N first images; arranging the N first images in order of shooting; querying the first magnification of the second device at the minimum horizontal field of view angle based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device; according to the second vertical coordinate and the first magnification, starting with the horizontal coordinate 0°, the first step length is 360° divided by N, horizontally rotating the gimbal of the second device, and capturing N second images; the N second images are arranged in order of shooting.

[0016] Optionally, when the first calculation module determines the first target image from N first images and determines the second target image that has a similarity correspondence relationship with the first target image from N second images, it is used to: extract M first images from the N first images according to the second step size; the second step size is 2n times the first step size, n is a positive integer; M is a positive integer less than N; extract 2M second images from the N second images according to the third step size; the third step size is half of the second step size; obtain any first image from the M first images, and align the any first image with each second image in the 2M second images. , calculate the similarity between any first image and the second image after registration; if there is at least one second image whose similarity with any first image is not less than a preset similarity threshold, take the second image with the greatest similarity with any first image as the second target image, and take any first image as the first target image; if there is no second image whose similarity with any first image is not less than the preset similarity threshold, adjust the value of the second vertical coordinate, and reacquire N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the gimbal of the second device.

[0017] Optionally, after taking the second image with the greatest similarity to any first image as the second target image and taking any first image as the first target image, the first calculation module is further used to: based on the position correspondence between the first target image and the second target image, and the angular correspondence between the M first images and the 2M second images, obtain M-1 second images other than the second target image from the second target image; the angular correspondence is determined according to the relationship between the second step length and the third step length; the M-1 second images are compared with the M first images other than the first target image in the M first images -1 first images correspond one to one; each second image in the M-1 second images is combined with the corresponding first image in the M-1 first images to form an image pair, so as to obtain M-1 groups of image pairs; the two images in each image pair in the M-1 groups of image pairs are registered, and the similarity between the two registered images is calculated; the number of image pairs whose similarity is not less than a preset similarity threshold is counted; if the number is less than the first number threshold, the value of the second vertical coordinate is adjusted, the N second images of the second device at the second preset position are re-acquired, and the first target image and the second target image are re-determined.

[0018] Optionally, when calculating the coordinate angle deviation of the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position, the second calculation module is used to: perform the following operations for any image pair in at least one group of image pairs: calculate the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation of the first device and the second device according to the linear transformation parameters, projection transformation parameters and translation transformation parameters included in the registration matrix, and the width and height of the image; convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into a horizontal coordinate angle deviation and a vertical coordinate angle deviation respectively according to the horizontal field of view and the vertical field of view of the second device at the second preset position, and the width and height of the image; compare the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of any image pair with the horizontal coordinate angle deviation and the vertical coordinate angle deviation of an adjacent image pair, and if the difference is not less than a preset difference threshold, discard the any image pair; if the difference is less than the preset difference threshold, set the any image pair as a correct image pair; and determine the coordinate angle deviation of the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs.

[0019] Optionally, when the second calculation module determines the coordinate angle deviation of the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs, it is also used to: calculate the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the first device and the second device at each angle in the horizontal direction by interpolation based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs; and determine the coordinate angle deviation of the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle.

[0020] Optionally, the device also includes a third calculation module; the third calculation module is used to: obtain a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from the original magnification of the first preset position to the minimum magnification; query the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view corresponding to the original magnification; obtain the second horizontal optical axis deviation value and the second vertical optical axis deviation value of the second device from the second magnification of the second preset position to the first magnification; obtain the optical axis deviation of the first device and the second device based on the first horizontal optical axis deviation value and the first vertical optical axis deviation value superimposed on the second horizontal optical axis deviation value and the first vertical optical axis deviation value; the processing module is used to: determine the target preset position coordinates of the second device based on the coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device at the first preset position.

[0021] Optionally, the device also includes a fourth calculation module; the fourth calculation module is used to: randomly obtain P preset positions from K preset positions, and respectively obtain J third images and J fourth images of the first device and the second device at each preset position in the P preset positions according to a preset rule; K is a positive integer, P is a positive integer less than K, and J is a positive integer; based on the J third images and the J fourth images, J groups of image pairs are determined according to the order of shooting, and the shooting order of the two images in each group of image pairs is the same; perform registration calculation on the two images in each group of image pairs, and calculate the similarity between the two images in each group of image pairs after registration , obtain the number of image pairs whose similarity is not less than a preset similarity threshold; if the number of image pairs is less than a second number threshold, the processing module is used to: determine the target preset position coordinates of the second device based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device; if the number of image pairs is not less than the second number threshold, calculate the average value of the coordinate angle deviation of all image pairs whose similarity is not less than the preset similarity threshold; the processing module is used to: determine the target preset position coordinates of the second device based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, and the average value of the coordinate angle deviation.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, comprising at least one processor, wherein the at least one processor is used to execute a computer program stored in a memory so that a method as in the first aspect or any optional implementation of the first aspect is implemented.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0024] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program code, which, when executed on a computer, enables the method in the first aspect or any optional implementation of the first aspect to be implemented.

[0025] The technical effects or advantages of one or more technical solutions provided in the second, third, fourth and fifth aspects of the embodiments of the present application can be correspondingly explained by the technical effects or advantages of one or more corresponding technical solutions provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flow chart of a data deviation processing method provided in an embodiment of the present application; Figure 2 A structural diagram of a data deviation processing device provided in an embodiment of the present application; Figure 3 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the technical solution of this application, the collection, dissemination, and use of data are in compliance with the requirements of relevant national laws and regulations.

[0028] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0029] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0030] It should be understood that "multiple" in the description of the embodiment of the present application refers to two or more. "First", "second" etc. in the embodiment of the present application are used to distinguish different objects, rather than to describe a specific order. The term "and / or" in the embodiment of the present application is merely a kind of association relationship describing associated objects, indicating that three relationships may exist, for example, A and / or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "includes" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. The module in the embodiment of the present application refers to a part with independent functions in a software system.

[0031] In scenarios such as substations, by pre-setting preset positions, cyclic monitoring of important devices and equipment can be achieved by relying on the inspection of PTZ camera equipment at the preset positions. The number of preset positions varies according to the scenario and focus points, and can reach up to several hundred. When the PTZ camera equipment has an abnormality and needs to be disassembled for repair and replacement, or when the equipment is updated and replaced, the original preset position information will become invalid due to factors such as the installation location and differences in equipment hardware and software. That is, if the new equipment is deployed at the original preset position, it will cause a large data deviation, so the preset position of the new equipment needs to be reset. If the preset position information is reset manually, it will be extremely time-consuming and labor-intensive, and there may be errors in human subjective judgment, resulting in reduced accuracy in data deviation processing.

[0032] In view of this, an embodiment of the present application provides a data deviation processing method, which obtains N first images taken by a first device at a first preset position and N second images taken by a second device at a second preset position respectively through the same step length, so as to facilitate the subsequent judgment of the position correspondence between the first image and the second image according to the step length; the first target image and the second target image are determined by similarity calculation, that is, two images with the same shooting content and shooting angle are determined, and the position deviation of the first device and the second device in the horizontal direction can be determined according to the deviation between the first position of the first target image in the N first images and the second position of the second target image in the second image; further, based on the first position and The correspondence between the first and second positions can determine the positional correspondence between each first image and each second image, and then construct N groups of image pairs, and select at least one group of image pairs that satisfies the similarity correspondence relationship from the N groups of image pairs for subsequent deviation calculation, which can improve the accuracy of the deviation calculation; then, each group of image pairs in the at least one group of image pairs is registered, and the coordinate angle deviation of the first device and the second device is calculated according to the preset algorithm based on the obtained registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; the position deviation and coordinate angle deviation obtained by comprehensive processing can improve the accuracy of data deviation processing, so that the determined target preset position coordinates of the second device are more accurate.

[0033] It is understandable that the data deviation processing method provided in the embodiment of the present application can be applied to any electronic device with processing capability, and the electronic device can be an independent electronic device or an electronic device deployed on the first device or the second device. The embodiment of the present application does not limit this.

[0034] The embodiments of the present application can be applied to any scenario where a pan-tilt camera device needs to be replaced, including when the first device (for example, the first device is an old device) fails and needs to be disassembled and repaired, the first device needs to be updated, etc., and a second device (for example, the second device is a new device) is used to replace the first device.

[0035] See also Figure 1 , is a flow chart of a data deviation processing method provided in an embodiment of the present application, wherein the first device and the second device are pan / tilt cameras, the shooting starting point and the shooting object of the first device and the second device are the same, and the second device is a new device and the first device is an old device that needs to be replaced as an example, the method includes the following steps S101 to S107: Step S101: according to the first step, N first images taken by a first device at a first preset position according to a preset rule and N second images taken by a second device at a second preset position according to a preset rule are obtained.

[0036] Among them, the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step size is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are arranged in their respective shooting order.

[0037] It can be understood that the setting of the preset position is determined by the user. In different scenarios, the preset position is manually set by the user according to the target object that the user needs to pay attention to. For example, in the substation scenario, the target object that the user needs to pay attention to may be the operation of certain electric meters or components. Therefore, the user can manually set the preset position of the PTZ camera. By adjusting the horizontal, vertical, magnification and other parameters, the PTZ camera can be aimed at the electric meter or component that the user needs to pay attention to. By setting multiple preset positions, rapid observation and monitoring of the target object can be achieved.

[0038] The preset positions of the pan-tilt camera device may include multiple ones, taking K as an example, where K is a positive integer; the coordinates of the preset positions can be recorded as [Pk, Tk, Zk], where Pk represents the horizontal coordinate of the kth preset position, Tk represents the vertical coordinate of the kth preset position, and Zk represents the original magnification of the kth preset position; the horizontal field of view corresponding to Zk can be recorded as fov_hk.

[0039] In a possible embodiment, the specific implementation of step S101 is as follows: The original magnification of the first device at the first preset position is replaced with the minimum magnification, and the gimbal of the first device is horizontally rotated according to the first vertical coordinate and the minimum magnification, starting from the horizontal coordinate 0°, with a first step length of 360° divided by N, and N first images are captured; the N first images are arranged in the order of shooting; Querying a first magnification of the second device at the minimum horizontal field of view angle based on a minimum horizontal field of view angle corresponding to the minimum magnification of the first device; According to the second vertical coordinate and the first magnification, starting from the horizontal coordinate 0°, the first step length is 360° divided by N, the gimbal of the second device is horizontally rotated, and N second images are captured; the N second images are arranged in the order of shooting.

[0040] For example, in order to ensure the accuracy of subsequent calculations, generally speaking, the value of N needs to be greater than 24, and the more points are set, the more accurate the calculation result will be, and the corresponding calculation cost and complexity will also be greater. Therefore, the value of N can be selected according to actual needs.

[0041] It can be understood that in order to ensure that during correction, the first device and the second device (ie, the old device and the new device) are aligned with the preset position and have enough overlapping area for calculation, the embodiment of the present application selects the minimum magnification Z of the first device. omin(That is, the minimum horizontal field of view is used.) Because, after environmental correction, there is a deviation between the preset point information of the second device and the preset point information of the first device. If it is directly adjusted to the maximum horizontal field of view, even a slight difference may cause the first device and the second device to align with completely different areas, that is, there is little or no overlap between the environmental scans of the first device and the second device, resulting in a large error in the subsequent registration calculation or even calculation failure, and the error of the preset point position cannot be calculated.

[0042] Moreover, usually, there is a correspondence between the field of view angle and the magnification inside the pan / tilt camera device, which is generally provided by the lens manufacturer or further expanded based on the information provided by the lens manufacturer. Therefore, it is possible to directly use the first correspondence between the field of view angle and the magnification of the first device according to the minimum magnification Z of the first device. omin The minimum field of view of the first device corresponding to the minimum magnification of the first device is queried from the first corresponding relationship. Similarly, based on the second corresponding relationship between the field of view and magnification of the second device, the first magnification Z corresponding to the minimum field of view of the first device is queried in the second corresponding relationship according to the minimum field of view of the first device. norres .

[0043] That is, the minimum magnification Z of the first device omin is the magnification corresponding to the minimum field of view angle of the first device in the first corresponding relationship; the first magnification Z norres is the magnification corresponding to the minimum field of view angle of the first device in the second corresponding relationship.

[0044] Optionally, when obtaining the first magnification of the second device, if there is a horizontal field of view angle that is the same as the minimum horizontal field of view angle of the first device among all the horizontal field of view angles of the second device, then the magnification corresponding to the horizontal field of view angle is directly obtained as the first magnification; if there is no horizontal field of view angle that is the same as the minimum horizontal field of view angle of the first device among all the horizontal field of view angles of the second device, but there is a horizontal field of view angle whose similarity with the minimum horizontal field of view angle of the first device is not less than a preset horizontal field of view angle similarity threshold, then the magnification corresponding to the horizontal field of view angle is used as the first magnification; if the similarity between all the horizontal field of view angles of the second device and the minimum horizontal field of view angle of the first device is less than the preset horizontal field of view angle similarity threshold, it means that the hardware difference between the first device and the second device is too large, and the first device cannot be replaced based on the second device, the calculation process is terminated, and an alarm prompt is output.

[0045] S102: Determine a first target image from the N first images, and determine a second target image having a similarity corresponding relationship with the first target image from the N second images.

[0046] The similarity correspondence is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold value. The preset similarity threshold value can be set according to actual needs.

[0047] In a possible embodiment, the specific implementation of step S102 is as follows: Extracting M first images from the N first images according to the second step length; the second step length is 2n times the first step length, n is a positive integer; M is a positive integer less than N; 2M second images are extracted from the N second images according to the third step length; the third step length is half of the second step length; Obtain any first image from the M first images, register the any first image with each second image in the 2M second images, and calculate the similarity between any first image and the second image after registration; If there is at least one second image whose similarity to any of the first images is not less than a preset similarity threshold, the second image with the greatest similarity to any of the first images is used as the second target image, and the any of the first images is used as the first target image; If there is no second image whose similarity with any first image is not less than a preset similarity threshold, the value of the second vertical coordinate is adjusted, and N second images of the second device at the second preset position are reacquired; the value range of the second vertical coordinate is within the vertical movement coordinate range of the gimbal of the second device.

[0048] Exemplarily, first, assuming that the N first images are old1~oldN, with the second step size as the angle interval, M first images are extracted from the N first images to obtain first images o1~oM; the N second images are new1~newN, with the third step size as the angle interval, 2M second images are extracted from the N second images to obtain second images n1~n2M.

[0049] It can be understood that the second step length needs to be a positive integer multiple of 2 of the first step length, that is, 2n times the first step length, n is a positive integer, that is, assuming that the first step length is s, the second step length is 2n·s, and the third step length is n·s. In addition, the multiple ratio (that is, the ratio of N to M) also needs to be an integer multiple of 2. It can be understood that the general PTZ camera device can achieve 360° rotation in order to observe the surrounding environment of the target object; when rotating to different angles, the observed area will also change. In order to establish the positional relationship between the first device and the second device (that is, the old device and the new device), the environmental scan map of the first device and the second device (that is, the N first images and the second images obtained) need to have an overlapping area. In theory, the larger the selected N, the higher the calculation accuracy, but the corresponding calculation speed will decrease. However, if the selected N is too small, the overlapping area between the environmental scan maps of the first device and the second device will be too small, resulting in a large calculation error or even calculation failure. Therefore, the multiple ratio is set here to balance the accuracy and speed.

[0050] Secondly, extract the first image om from o1~oM; om can be a randomly selected first image, or a first image selected in sequence from small to large or from large to small; perform registration calculations on om and n1~n2M respectively, and calculate the similarity of each group of images after registration, that is, calculate the similarity between the registered om and n1, calculate the similarity between the registered om and n2, and so on, to obtain the similarity between the registered om and each second image in n1~n2M.

[0051] It can be understood that the specific registration calculation method can be selected according to actual needs, and the embodiments of the present application do not limit this.

[0052] Then, obtain the second image nb with the highest similarity to the registered om, and determine whether the similarity between the registered om and nb is not less than the preset similarity threshold; if so, determine that om is the first target image and nb is the second target image; if not, re-extract om from o1~oM, and perform the above-mentioned registration calculation and similarity calculation. If all the first images in o1~oM are traversed and the first image and the second image that satisfy the similarity correspondence relationship are still not obtained, return to the above-mentioned step S101, modify the value of the second vertical coordinate, re-acquire the N second images of the second device at the second preset position, and repeat step S102 until the first target image and the second target image are determined.

[0053] It can be understood that, since the first device and the second device may have a large difference in form, there is a large distance in the vertical direction when they are installed in the same position. For the first device to be viewed horizontally, the second device may need to be viewed upward or downward. Therefore, by dynamically adjusting the value of the second vertical coordinate of the second device, the shooting angle and shooting range of the second device are the same as the shooting angle and shooting range of the first device at the first vertical coordinate. The value of the initial second vertical coordinate can be randomly set or manually preset, and the value of the second vertical coordinate needs to be within the vertical movement coordinate range of the gimbal of the second device.

[0054] Optionally, if all selectable values ​​of the second vertical coordinate are traversed and the first image and the second image that satisfy the similarity correspondence relationship are still not obtained, it means that the hardware and other factors of the first device and the second device are very different, and the operation of replacing the first device with the second device cannot be realized. The calculation process ends and an alarm prompt is output.

[0055] Furthermore, in a possible embodiment, after taking the second image with the greatest similarity to any first image as the second target image and taking any first image as the first target image, it is also necessary to verify the first target image and the second target image to ensure that the position correspondence between the N first images and the N second images can be determined based on the position correspondence between the first target image and the second target image. The specific verification method is as follows: Based on the position correspondence between the first target image and the second target image, and the angle correspondence between the M first images and the 2M second images, M-1 second images other than the second target image are acquired from the second target image; the angle correspondence is determined according to the relationship between the second step length and the third step length; the M-1 second images correspond one-to-one to the M-1 first images other than the first target image in the M first images; Each second image in the M-1 second images is combined with the corresponding first image in the M-1 first images to form a group of image pairs, thereby obtaining M-1 groups of image pairs; Perform registration calculation on the two images in each image pair in the M-1 groups of image pairs, and calculate the similarity between the two registered images; count the number of image pairs whose similarity is not less than a preset similarity threshold; if the number is less than the first number threshold, adjust the value of the second vertical coordinate, reacquire the N second images of the second device at the second preset position, and redetermine the first target image and the second target image.

[0056] Exemplarily, the first target image is om, that is, the position of the first target image in o1~oM is m, and the second target image is nb, that is, the position of the second target image in n1~n2M is b. According to the corresponding relationship between om and nb, it can be known that the position m in the first target image corresponds to the position b in the second target image.

[0057] Since om and nb correspond to each other, it can be understood that the shooting angles and shooting contents of om and nb are the same, that is, the angles corresponding to position m and position b are the same. Furthermore, according to the relationship between the second step length and the third step length, that is, the second step length is twice the third step length, it can be understood that the shooting angle interval corresponding to the second step length is twice the shooting angle interval of the third step length. It can be seen that the angles corresponding to position m+1 and position b+2 are also the same. Specifically, assuming that the angle corresponding to m and b is a, the angle corresponding to m+1 is a+second step length (that is, a+2n·s), and the angle corresponding to b+2 is a+2·third step length (that is, a+2n·s). The corresponding angles of the two are the same, that is, position m+1 and position b+2 are corresponding. Similarly, the position m+2 corresponds to the position b+4, and the position m-1 corresponds to the position b-2. It can be inferred that the position m+r in the first target image corresponds to b+2r (r is an integer, r is used to indicate the position interval) in the second target image. Therefore, based on this correspondence, M-1 groups of image pairs other than om and nb can be determined, that is, the second image corresponding to the position of each first image in the M-1 first images can be determined.

[0058] For each image pair in the M-1 groups of image pairs, perform registration calculation on the two images contained therein, calculate the similarity of the two registered images, and count the number of image pairs whose similarity is not less than a preset similarity threshold; If the number is less than the first number threshold, it means that the currently determined first target image and second target image are unreliable, and the position correspondence between each first image and each second image cannot be accurately determined based on the first target image and the second target image; therefore, it is necessary to adjust the value of the second vertical coordinate, reacquire the N second images of the second device at the second preset position, and re-determine the first target image and the second target image; If the number is not less than the first number threshold, it means that the currently determined first target image and second target image are reliable, and the current first target image and second target image can be used.

[0059] S103, obtaining a first position of the first target image in the N first images and a second position of the second target image in the N second target images, and determining a position deviation between the first device and the second device in the horizontal direction based on a deviation between the first position and the second position.

[0060] Exemplarily, continuing the example in the above step S102, the first target image is om, and its position in o1~oM is m. According to the relationship between o1~oM and old1~oldN, it can be known that the position of om in old1~oldN is (m-1)×ratio+1, where ratio represents N / M; similarly, the second target image is nb, and its position in n1~n2M is b. According to the corresponding relationship between n1~n2M and new1~newN, it can be known that the position of nb in new1~newN is (b-1)×ratio / 2+1; that is, the position (m-1)×ratio+1 in old1~oldN corresponds to the position (b-1)×ratio / 2+1 in new1~newN.

[0061] Therefore, the position deviation (also called relational deviation) rx of the first device and the second device in the horizontal direction can be determined according to the position deviation between (m-1)×ratio+1 and (b-1)×ratio / 2+1: rx=(b-1)×s / 2-(m-1)×s; The position deviation can reflect the deviation between the first device and the second device in horizontal installation, and also reflect the angular correspondence between the first device and the second device.

[0062] S104, determining a position correspondence between each first image in the N first images and each second image in the N second images according to the correspondence between the first position and the second position; and constructing N groups of image pairs according to the position correspondence.

[0063] Each group of image pairs includes a first image and a second image corresponding to each other in position.

[0064] Exemplarily, continuing the example in step S103 above, it can be seen that the position (m-1)×ratio+1 in old1~oldN corresponds to the position (b-1)×ratio / 2+1 in new1~newN; and since the step lengths of old1~oldN and new1~newN are the same, it can be seen that the position (m-1)×ratio+1+r in old1~oldN corresponds to the position (b-1)×ratio / 2+1+r (r is an integer) in new1~newN. Therefore, according to this position correspondence, N groups of image pairs can be constructed. The r value of the first image and the second image in each group of image pairs is the same.

[0065] S105, selecting at least one group of image pairs whose two images satisfy a similarity correspondence relationship from the N groups of image pairs; performing registration calculation on the two images in each image pair in the at least one group of image pairs to obtain a registration matrix.

[0066] Exemplarily, a registration calculation is performed on two images in each of the N groups of image pairs, and the similarity between the two registered images is calculated to obtain at least one group of image pairs whose similarity is greater than a preset similarity threshold.

[0067] Optionally, if the number of at least one group of image pairs is less than a third number threshold, it means that the hardware difference between the first device and the second device is too large, and the second device cannot replace the first device. The calculation process ends and an alarm prompt is output.

[0068] For at least one set of image pairs that satisfy the similarity correspondence relationship, a registration matrix is ​​obtained according to the registration calculation. The following is an example of a registration matrix t provided in an embodiment of the present application: ; Among them, m 0 , m 1 , m 2 , m 3 Represent four different linear transformation parameters respectively; p 0 , p 1 Represent two different projection transformation parameters; t 0 , t 1 They represent two different translation transformation parameters respectively, and 1 is used to represent the size transformation parameter of the image (the size transformation parameter of the image is 1, indicating that the image does not need to be resized).

[0069] S106: Calculate the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position.

[0070] In a possible embodiment, the specific implementation of step S106 is as follows: For any image pair in at least one set of image pairs, perform the following operations: Calculate the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation of the first device and the second device according to the linear transformation parameters, the projection transformation parameters and the translation transformation parameters included in the registration matrix, and the width and height of the image; Convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into the horizontal coordinate angle deviation and the vertical coordinate angle deviation respectively according to the horizontal field of view angle and the vertical field of view angle of the second device at the second preset position, and the width and height of the image; Compare the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of any image pair and the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the adjacent image pair, and if the difference is not less than a preset difference threshold, discard the any image pair; if the difference is less than the preset difference threshold, set the any image pair as the correct image pair; The coordinate angle deviation between the first device and the second device is determined based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs.

[0071] For example, taking the registration matrix t shown in step S105 as an example, the width of the image is w, the height is h, and the horizontal coordinate pixel deviation of the first device and the second device is calculated according to the following formula 1: and vertical pixel deviation : (Formula 1) Then, according to the horizontal field angle of the second device corresponding to the first magnification And vertical field of view , as well as the width w and height h of the image, the horizontal coordinate pixel deviation is converted to the horizontal coordinate angle deviation dx, and the vertical coordinate pixel deviation is converted to the vertical coordinate angle deviation dy according to the following formula 2: (Formula 2) According to the above formula 1 and formula 2, the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each group of image pairs at the corresponding angle can be obtained.

[0072] After obtaining the horizontal coordinate angle deviation and vertical coordinate angle deviation of each group of image pairs at the corresponding angle, it is also necessary to verify the obtained coordinate angle deviation to avoid mutation or misjudgment, that is, by comparing the horizontal coordinate angle deviation and vertical coordinate angle deviation of each group of image pairs with the horizontal coordinate angle deviation and vertical coordinate angle deviation of the adjacent image pairs of the group of image pairs. If the difference is less than the preset difference threshold, it means that there is no mutation in the image pair and it is set as the correct image pair; if the difference is not less than the preset difference threshold, it means that there is a mutation in the image pair and it is a misjudgment point and it is discarded.

[0073] Furthermore, since the coordinate angle deviation between the first device and the second device calculated according to the above embodiment is only the coordinate angle deviation of some image pairs at corresponding angles, it cannot cover the coordinate angle deviation of each angle in the 360° horizontal direction.

[0074] Therefore, the present application also provides a possible embodiment: based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs, the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle of the first device and the second device in the horizontal direction are calculated by interpolation; based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle, the coordinate angle deviation of the first device and the second device is determined. In this way, the coordinate angle deviation of the first device and the second device obtained is more comprehensive.

[0075] It can be understood that the specific interpolation method can be selected according to actual needs, and the embodiments of the present application do not limit this.

[0076] S107: Determine the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation, and the coordinate angle deviation.

[0077] In this way, the shooting coverage area of ​​the second device at the target preset position can be made the same as the shooting coverage area of ​​the first device at the first preset position.

[0078] Exemplarily, the coordinates of the first preset position of the first device and the position deviation can be used to roughly determine the rough coordinates corresponding to the first preset position of the first device in the second device. Based on the coordinate angle deviation, more precise coordinates of the first preset position of the first device in the second device (i.e., the target preset position coordinates) can be obtained, thereby realizing coordinate mapping from the environmental point position of the first device to the second device.

[0079] In a possible design, due to the influence of the optical axis deviation, the images of the pan / tilt camera device at different magnifications have deviations, the image centers are not consistent, and the aligned target areas are also inconsistent, so that the target preset position obtained only based on the position deviation and the coordinate angle deviation still has deviations. Therefore, the embodiment of the present application also provides a method for eliminating the influence of the optical axis deviation, and the specific implementation of the method is as follows: Acquire a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from an original magnification of a first preset position to a minimum magnification; Querying a second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; obtaining a second horizontal optical axis deviation value and a second vertical optical axis deviation value of the second device from the second magnification of the second preset position to the first magnification; Obtaining an optical axis deviation between the first device and the second device based on the first horizontal optical axis deviation value and the first vertical optical axis deviation value superimposed on the second horizontal optical axis deviation value and the second vertical optical axis deviation value; Based on the coordinates of the first device at the first preset position, the position deviation, the coordinate angle deviation, and the optical axis deviation, the target preset position coordinates of the second device are determined.

[0080] For example, assuming that the first preset position is preset position k, the horizontal coordinate of the first device at the original magnification Zk of the first preset position is Pk, the vertical coordinate is Tk, and the minimum magnification Zk of the first device at the first preset position is omin The horizontal and vertical coordinates of 0 and Tk 0 ; Get the first device from the original magnification Zk to the minimum magnification Z ominThe horizontal optical axis deviation value optx and the vertical optical axis deviation value opty, it can be seen that Pk 0 =Pk+optx,Tk 0 =Tk+opty.

[0081] According to the above embodiment, at the coordinates [Pk 0 , Tk 0 , Z omin ] After adding the position deviation and the coordinate angle deviation, the target preset position coordinates of the second device are obtained as [Pk 1 , Tk 1 , Z norres ], where Z norres is the first magnification of the second device. The target preset position coordinates are calculated based on the minimum magnification Z of the first device. omin The coordinates obtained by the mapping below, because the second device also has an optical axis deviation, therefore, it is also necessary to consider the impact of the optical axis deviation of the second device.

[0082] That is, get the second device from the second magnification Z korres To the first magnification Z norres The horizontal optical axis deviation and vertical optical axis deviation, in Pk 1 , Tk 1 After superimposing the horizontal optical axis deviation and the vertical optical axis deviation on the basis of the target preset position coordinates of the second device [Pk 2 , Tk 2 , Z korres ].

[0083] It can be understood that the optical axis deviation is a pre-set property of the pan / tilt camera and can be directly queried and obtained according to the magnification.

[0084] In this way, the optical axis deviation is also included in the data deviation processing, which can improve the accuracy of the data deviation processing and make the obtained target preset position coordinates more accurate.

[0085] In a possible design, considering the error of the optical axis deviation itself and the interpolation error when the coordinate angle deviation is expanded by the interpolation method, the embodiment of the present application also provides a method for further improving the accuracy of data deviation processing, and the specific implementation of the method is as follows: Randomly obtain P preset positions from K preset positions, and respectively obtain J third images and J fourth images at each of the P preset positions by the first device and the second device according to a preset rule; K is a positive integer, P is a positive integer less than K, and J is a positive integer; Based on the J third images and the J fourth images, J groups of image pairs are determined according to the shooting sequence, and the shooting sequence of the two images in each group of image pairs is the same; Performing registration calculation on the two images in each image pair, and calculating the similarity between the two images in each image pair after registration, and obtaining the number of image pairs whose similarity is not less than a preset similarity threshold; If the number of image pairs is less than a second number threshold, determining the target preset position coordinates of the second device based on the first preset position coordinates, the position deviation, the coordinate angle deviation, and the optical axis deviation of the first device; If the number of image pairs is not less than a second number threshold, the average coordinate angle deviation of all image pairs whose similarity is not less than a preset similarity threshold is calculated; based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, and the average coordinate angle deviation, the target preset position coordinates of the second device are determined.

[0086] Exemplarily, a specific proportion of preset positions are randomly and non-repetitively extracted from the K preset positions to obtain J preset positions; the specific proportion can be set according to actual needs, such as 20%. The first image and the second image of the first device and the second device at each preset position in the J preset positions are obtained; according to the registration calculation and similarity calculation method in the above embodiment method, the number of image pairs whose similarity is not less than the preset similarity threshold is determined.

[0087] If the number of image pairs is not less than the second number threshold, the mean value of the coordinate angle deviation of these image pairs is obtained, and the target preset position coordinates of the second device are determined based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, and the mean value of the coordinate angle deviation. 2 , Tk 2 On this basis, the mean horizontal coordinate angle deviation and the mean vertical coordinate angle deviation are superimposed respectively.

[0088] If the number of image pairs is less than the second number threshold, more preset positions can be obtained for calculation by increasing a specific ratio, or the number of first images and second images obtained can be increased, so that the number of image pairs that meet the similarity correspondence relationship is greater than the second number threshold. If the number of image pairs cannot be made not less than the second number threshold by increasing the specific ratio or increasing the number of first images and second images, the target preset position coordinates of the second device are determined directly based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, that is, the target preset position coordinates are [Pk 2 , Tk 2 , Z korres ].

[0089] In this way, by calculating the average value of the coordinate angle deviation of the first device and the second device at multiple preset positions, the influence of the error of the optical axis deviation itself and the interpolation error can be reduced, and the accuracy of data deviation processing can be further improved. And the accuracy requirement for the optical axis deviation can be reduced, which is more convenient for engineering implementation in practical applications.

[0090] In this embodiment, by calculating the position deviation and coordinate angle deviation of the first device and the second device, it is determined that the coordinates of the first device at the first preset position are mapped to the target preset position coordinates of the second device, which can solve the problem that the original preset position cannot be reused due to the replacement, disassembly and reinstallation of the pan-tilt camera device or the aging and deviation of the device, effectively improve the efficiency of introducing new pan-tilt camera equipment and reduce labor costs; and the influence of the optical axis deviation of the first device and the second device on the target preset position is also considered, and the optical axis deviation is superimposed on the position deviation and the coordinate angle deviation, so that the target preset position coordinates are more accurate, thereby improving the accuracy of data deviation processing; in addition, considering the error of the optical axis deviation itself and the interpolation error of the interpolation method used in calculating the coordinate angle deviation, by calculating the average of the coordinate angle deviations of the first device and the second device at multiple preset positions, the target preset position coordinates are optimized by the average of the coordinate angle deviations, thereby further improving the accuracy of data deviation processing.

[0091] The method provided by the embodiment of the present application is introduced above, and the device provided by the embodiment of the present application is introduced below.

[0092] Based on the same technical concept, the embodiment of the present application provides a data deviation processing device, wherein the first device and the second device are PTZ camera devices, and the shooting starting point and shooting object of the first device and the second device are the same; the device includes a module / unit / means for executing the method executed by the electronic device in the above method embodiment. The module / unit / means can be implemented by software, or by hardware, or by hardware executing the corresponding software implementation.

[0093] For example, Figure 2 As shown, the device 200 includes: The acquisition module 201 is used to: acquire, according to a first step length, N first images taken by a first device at a first preset position according to a preset rule and N second images taken by a second device at a second preset position according to a preset rule; the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step length is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are arranged in the order of their respective shooting sequence; The first calculation module 202 is used to: determine a first target image from the N first images, and determine a second target image that has a similarity correspondence relationship with the first target image from the N second images; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold; obtain a first position of the first target image in the N first images, and a second position of the second target image in the N second images, and determine a position deviation of the first device and the second device in the horizontal direction based on a deviation between the first position and the second position; The second calculation module 203 is used to: determine the position correspondence between each first image in the N first images and each second image in the N second images according to the correspondence between the first position and the second position; construct N groups of image pairs according to the position correspondence, each group of image pairs including a first image and a second image corresponding to the position; select at least one group of image pairs in which the two images satisfy the similarity correspondence from the N groups of image pairs; perform registration calculation on the two images in each group of image pairs in at least one group of image pairs to obtain a registration matrix; calculate the coordinate angle deviation of the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; The processing module 204 is used to determine the target preset position coordinates of the second device based on the coordinates, position deviation and coordinate angle deviation of the first device at the first preset position, so that the shooting coverage area of ​​the second device at the target preset position is the same as the shooting coverage area of ​​the first device at the first preset position.

[0094] It should be understood that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0095] Based on the same technical concept, see Figure 3 The embodiment of the present application further provides an electronic device 300, including: At least one processor 301; and a communication interface 303 that is communicatively connected to the at least one processor 301; the at least one processor 301 executes instructions stored in the memory 302, so that the electronic device 300 executes the method steps performed by the billboard in the above method embodiment through the communication interface 303.

[0096] Optionally, the memory 302 is located outside the electronic device 300 .

[0097] Optionally, the electronic device 300 includes the memory 302, the memory 302 is connected to the at least one processor 301, and the memory 302 has instructions that can be executed by the at least one processor 301. Figure 3The dashed lines indicate that the memory 302 is optional for the electronic device 300 .

[0098] The at least one processor 301 and the memory 302 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0099] The specific connection medium between the at least one processor 301, the memory 302 and the communication interface 303 is not limited in the embodiment of the present application. Figure 3 In the embodiment, at least one processor 301, a memory 302 and a communication interface 303 are connected via a bus 304. Figure 3 The connection between other components is only for schematic illustration and is not intended to be limiting. The bus part may be an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0100] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor implemented by reading software code stored in a memory.

[0101] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0102] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM).

[0103] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0104] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0105] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the computer executes the method steps executed by any device in the above method embodiments.

[0106] Based on the same technical concept, an embodiment of the present application also provides a computer program product, including computer program code. When the computer program code is executed on a computer, the method steps executed by any device in the above method embodiment are implemented.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for processing data deviation, characterized in that: The first device and the second device are pan / tilt camera devices, and the shooting starting point and the shooting object of the first device and the second device are the same; the method includes: According to the first step, N first images taken by the first device at the first preset position according to a preset rule and N second images taken by the second device at the second preset position according to the preset rule are obtained; N is a positive integer; the N first images and the N second images are arranged in the order of their respective shooting; Determine a first target image from the N first images, and determine a second target image from the N second images whose similarity to the first target image is not less than a preset similarity threshold; Acquire a first position of the first target image in the N first images, and a second position of the second target image in the N second images, and determine a position deviation of the first device and the second device in the horizontal direction based on a deviation of the first position and the second position; Determine, according to the correspondence between the first position and the second position, a position correspondence between each first image in the N first images and each second image in the N second images; construct N groups of image pairs according to the position correspondence, each group of image pairs including a first image and a second image corresponding in position; Selecting at least one group of image pairs from the N groups of image pairs, the similarity between the two images being not less than the preset similarity threshold; performing registration calculation on the two images in each image pair in the at least one group of image pairs to obtain a registration matrix; and calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; Based on the coordinates of the first device at the first preset position, the position deviation and the coordinate angle deviation, the target preset position coordinates of the second device are determined so that the shooting coverage area of ​​the second device at the target preset position is the same as the shooting coverage area of ​​the first device at the first preset position.

2. The method according to claim 1, characterized in that The step of acquiring, according to the first step, N first images taken by the first device at the first preset position according to a preset rule and N second images taken by the second device at the second preset position according to the preset rule comprises: Replace the original magnification of the first device at the first preset position with the minimum magnification, and according to the first vertical coordinate and the minimum magnification, start with the horizontal coordinate 0°, and the first step length is 360° divided by N, horizontally rotate the pan / tilt of the first device, and capture the N first images; Based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device, query the first magnification of the second device at the minimum horizontal field of view angle; According to the second vertical coordinate and the first magnification, starting from the horizontal coordinate 0°, the first step length is 360° divided by N, the gimbal of the second device is horizontally rotated, and the N second images are captured.

3. The method according to claim 2, characterized in that The step of determining a first target image from the N first images, and determining a second target image having a similarity corresponding relationship with the first target image from the N second images, comprises: Extracting M first images from the N first images according to a second step length; the second step length is 2n times the first step length, n is a positive integer; and M is a positive integer less than N; Extracting 2M second images from the N second images according to a third step length; the third step length is half of the second step length; Acquire any first image from the M first images, register the any first image with each second image in the 2M second images, and calculate the similarity between any first image and the second image after registration; If there is at least one second image whose similarity to any of the first images is not less than the preset similarity threshold, the second image with the greatest similarity to any of the first images is used as the second target image, and any of the first images is used as the first target image; If there is no second image whose similarity with any of the first images is not less than the preset similarity threshold, adjust the value of the second vertical coordinate and reacquire N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the gimbal of the second device.

4. The method according to claim 3, characterized in that After taking the second image having the greatest similarity to any of the first images as the second target image and taking any of the first images as the first target image, the method further includes: Based on the position correspondence between the first target image and the second target image, and the angular correspondence between the M first images and the 2M second images, M-1 second images other than the second target image are acquired from the second target image; the angular correspondence is determined according to the relationship between the second step length and the third step length; the M-1 second images correspond one-to-one to the M-1 first images other than the first target image in the M first images; Each second image in the M-1 second images is combined with a corresponding first image in the M-1 first images into a group of image pairs, to obtain M-1 groups of image pairs; Performing registration calculation on two images in each image pair in the M-1 groups of image pairs, and calculating the similarity between the two registered images; and counting the number of image pairs whose similarity is not less than the preset similarity threshold; If the number is less than a first number threshold, the value of the second vertical coordinate is adjusted, the N second images of the second device at the second preset position are reacquired, and the first target image and the second target image are re-determined.

5. The method according to claim 1, characterized in that The calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position includes: For any image pair in the at least one set of image pairs, perform the following operations: Calculate the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation of the first device and the second device according to the linear transformation parameters, the projection transformation parameters and the translation transformation parameters included in the registration matrix, and the width and height of the image; According to the horizontal field of view angle and the vertical field of view angle of the second device at the second preset position, and the width and height of the image, convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into a horizontal coordinate angle deviation and a vertical coordinate angle deviation respectively; Comparing the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of any image pair and the horizontal coordinate angle deviation and the vertical coordinate angle deviation of an adjacent image pair, if the difference is not less than a preset difference threshold, discarding any image pair; if the difference is less than the preset difference threshold, setting any image pair as a correct image pair; The coordinate angle deviation between the first device and the second device is determined based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs.

6. The method according to claim 5, characterized in that The determining of the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs includes: Based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs, the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle of the first device and the second device in the horizontal direction are determined by interpolation calculation; based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle, the coordinate angle deviation of the first device and the second device is determined.

7. The method according to any one of claims 1 to 6, characterized in that: The determining the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation, and the coordinate angle deviation includes: Acquire a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from an original magnification of the first preset position to a minimum magnification; querying a second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; Acquire a second horizontal optical axis deviation and a second vertical optical axis deviation of the second device from a second magnification of the second preset position to a first magnification; Obtaining optical axis deviations of the first device and the second device based on the superposition of the second horizontal optical axis deviation and the first vertical optical axis deviation value; The target preset position coordinates of the second device are determined based on the coordinates of the first device at the first preset position, the position deviation, the coordinate angle deviation, and the optical axis deviation.

8. A data deviation processing device, characterized in that: The first device and the second device are pan / tilt camera devices, and the shooting starting point and the shooting object of the first device and the second device are the same; the apparatus comprises: An acquisition module is used to: acquire, according to the first step, N first images taken by the first device at a first preset position according to a preset rule and N second images taken by the second device at a second preset position according to the preset rule; N is a positive integer; and the N first images and the N second images are arranged in a respective shooting order; A first calculation module is used to: determine a first target image from the N first images, and determine a second target image whose similarity with the first target image is not less than a preset similarity threshold from the N second images; obtain a first position of the first target image in the N first images, and a second position of the second target image in the N second images, and determine a position deviation of the first device and the second device in a horizontal direction based on a deviation of the first position and the second position; A second calculation module is used to: determine the position correspondence between each first image in the N first images and each second image in the N second images according to the correspondence between the first position and the second position; construct N groups of image pairs according to the position correspondence, each group of image pairs including a first image and a second image corresponding to the position; filter out at least one group of image pairs including two images whose similarity is not less than the similarity threshold from the N groups of image pairs; perform registration calculation on the two images in each group of image pairs in the at least one group of image pairs to obtain a registration matrix; calculate the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view of the second device at the second preset position; A processing module is used to determine the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation and the coordinate angle deviation, so that the shooting coverage area of ​​the second device at the target preset position is the same as the shooting coverage area of ​​the first device at the first preset position.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in the method according to any one of claims 1 to 7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to have a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images