A data deviation processing method and apparatus

Abstract

The present application provides a data deviation processing method and apparatus for improving the accuracy of data deviation processing. The method includes: acquiring N first images captured by a first device at a first preset position and N second images captured by a second device at a second preset position; determining a first target image and a second target image that satisfy a similarity correspondence relationship; acquiring a first position of the first target image and a second position of the second target image, and determining a position deviation between the first device and the second device based on the deviation between the first position and the second position; determining at least one set of image pairs that satisfy the similarity correspondence relationship among the N first images and the N second images; performing registration on each set of image pairs to obtain a registration matrix; calculating a coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device; and determining a target preset position coordinate of the second device based on the coordinate, the position deviation, and the coordinate angle deviation of the first device.

Description

Technical Field

[0001] This application relates to the technical field of video surveillance, and particularly to a method and device for processing data deviation. Background Art

[0002] When a camera device has an abnormality and needs to be disassembled and repaired or updated and replaced, due to factors such as installation location, differences in device software and hardware, etc., if the replaced device is deployed according to the original preset position information, there will be a large data deviation. If the preset position information is reset manually, it will be extremely time-consuming and laborious, and there will be errors caused by subjective human factors.

[0003] In view of this, how to improve the accuracy of data deviation processing is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method and device for processing data deviation, which is used to improve the accuracy of data deviation processing.

[0005] In a first aspect, an embodiment of this application provides a method for processing data deviation. The first device and the second device are pan-tilt camera devices, and the shooting starting points and shooting objects of the first device and the second device are the same. The method includes:

[0006] Obtain N first images captured by the first device at a first preset position according to a preset rule and N second images captured by the second device at a second preset position according to the preset rule with a first step size; the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step size is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are respectively arranged in their respective shooting sequences;

[0007] Determine a first target image from the N first images, and determine a second target image from the N second images that has a similarity correspondence relationship with the first target image; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold;

[0008] Obtain a first position of the first target image in the N first images and a second position of the second target image in the N second images, and determine the position deviation between the first device and the second device in the horizontal direction based on the deviation between the first position and the second position;

[0009] Determine the position correspondence relationship between each of the N first images and each of the N second images according to the correspondence relationship between the first position and the second position; construct N groups of image pairs according to the position correspondence relationship, and each group of image pairs includes a corresponding first image and a second image;

[0010] Select at least one pair of images from N pairs of images, where the two images included satisfy the similarity correspondence relationship; perform registration calculations on the two images in each pair of images in at least one pair of images to obtain a registration matrix; calculate the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position.

[0011] Based on the coordinates, position deviation, and coordinate angle deviation of the first device at the first preset position, determine the target preset position coordinates of the second device so that the shooting coverage area of the second device at the target preset position is the same as the shooting coverage area of the first device at the first preset position.

[0012] In this method, N first images captured by the first device at the first preset position and N second images captured by the second device at the second preset position are obtained respectively with the same step size, which is convenient for subsequently judging the position correspondence relationship between the first image and the second image according to the step size; and the first target image and the second target image are determined through similarity calculation, that is, two images with the same shooting content and shooting perspective are determined. According to the deviation between the first position of the first target image in the N first images and the second position of the second target image in the second images, the position deviation between the first device and the second device in the horizontal direction can be determined; further, based on the correspondence relationship between the first position and the second position, the position correspondence relationship between each first image and each second image can be determined, and then N pairs of images are constructed. Select at least one pair of images that satisfy the similarity correspondence relationship from the N pairs of images for subsequent deviation calculation, which can improve the accuracy of deviation calculation; then perform registration calculations on each pair of images in at least one pair of images, and calculate the coordinate angle deviation between the first device and the second device according to the obtained registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position according to a preset algorithm; comprehensively process the obtained relationship deviation and coordinate angle deviation, which can improve the accuracy of data deviation processing and make the determined target preset position coordinates of the second device more accurate.

[0013] Optionally, obtain N first images captured by the first device at the first preset position according to a preset rule and N second images captured by the second device at the second preset position according to the preset rule, including: replacing the original magnification of the first device at the first preset position with the minimum magnification, and according to the first vertical coordinate and the minimum magnification, starting from the horizontal coordinate of 0°, with the first step length being 360° divided by N, horizontally rotate the pan-tilt of the first device and intercept N first images; arrange the N first images in the order of shooting; query the first magnification of the second device at the minimum horizontal field of view angle based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device; according to the second vertical coordinate and the first magnification, starting from the horizontal coordinate of 0°, with the first step length being 360° divided by N, horizontally rotate the pan-tilt of the second device and intercept N second images; arrange the N second images in the order of shooting.

[0014] Optionally, determine a first target image from the N first images and determine a second target image that has a similarity correspondence relationship with the first target image from the N second images, including: extracting M first images from the N first images according to the second step length; the second step length is 2n times the first step length, where n is a positive integer; M is a positive integer less than N; extracting 2M second images from the N second images according to the third step length; the third step length is one-half of the second step length; obtain any one of the first images from the M first images, register the any one of the first images with each of the 2M second images, and calculate the similarity between the registered any one of the first images and the second images; if there is at least one second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then use the second image with the maximum similarity with the any one of the first images as the second target image and the any one of the first images as the first target image; if there is no second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then adjust the value of the second vertical coordinate and re-obtain N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the pan-tilt of the second device.

[0015] Optionally, after taking the second image with the highest similarity to any one of the first images as the second target image and the any one of the first images as the first target image, the method further includes: Based on the position correspondence between the first target image and the second target image, and the angle correspondence between the M first images and the 2M second images, obtaining M - 1 second images other than the second target image from the second target image; the angle correspondence is determined according to the relationship between the second step length and the third step length; the M - 1 second images correspond one by one to the M - 1 first images other than the first target image among the M first images; forming a pair of images by combining each of the M - 1 second images in the M - 1 second images with the corresponding first image in the M - 1 first images, to obtain M - 1 pairs of images; performing registration calculation on the two images in each pair of images in the M - 1 pairs of images, and calculating the similarity between the two registered images; counting the number of pairs of images with a similarity not less than a preset similarity threshold; if the number is less than the first quantity threshold, adjusting the value of the second vertical coordinate, re - obtaining N second images of the second device at the second preset position, and re - determining the first target image and the second target image.

[0016] Optionally, calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position, includes: For any pair of images in at least one group of pairs of images, perform the following operations: According to the linear transformation parameters, projection transformation parameters, and translation transformation parameters included in the registration matrix, and the width and height of the image, calculate the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation between the first device and the second device; According to the horizontal field of view angle and the vertical field of view angle of the second device at the second preset position, and the width and height of the image, respectively convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into a horizontal coordinate angle deviation and a vertical coordinate angle deviation; Compare the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the any pair of images and the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the adjacent pair of images. If the difference is not less than a preset difference threshold, discard the any pair of images; If the difference is less than the preset difference threshold, set the any pair of images as a correct pair of images; Based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct pairs of images, determine the coordinate angle deviation between the first device and the second device.

[0017] Optionally, determining the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct pairs of images, includes: Based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct pairs of images, interpolate to calculate the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle in the horizontal direction between the first device and the second device; Based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle, determine the coordinate angle deviation between the first device and the second device.

[0018] Optionally, determining the target preset position coordinates of the second device based on the coordinates, position deviation, and coordinate angle deviation of the first device at the first preset position includes: obtaining a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from the original magnification at the first preset position to the minimum magnification; querying the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; obtaining a second horizontal optical axis deviation value and a second vertical optical axis deviation value of the second device from the second magnification at the second preset position to the first magnification; obtaining the optical axis deviation between the first device and the second device by superimposing the first horizontal optical axis deviation value and the first vertical optical axis deviation value on the second horizontal optical axis deviation value and the second vertical optical axis deviation value; and determining the target preset position coordinates of the second device based on the coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device at the first preset position.

[0019] Optionally, determining the target preset position coordinates of the second device based on the coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device at the first preset position includes:

[0020] Randomly obtaining P preset positions from K preset positions, and respectively obtaining J third images and J fourth images of the first device and the second device at each of the P preset positions according to a preset rule; K is a positive integer, P is a positive integer less than K, and J is a positive integer; determining J groups of image pairs based on the J third images and the J fourth images in the order of shooting, where the shooting order of the two images in each group of image pairs is the same; performing registration calculation on the two images in each group of image pairs, and calculating the similarity between the two images in the registered group of image pairs, and obtaining the number of image pairs whose similarity is not less than a preset similarity threshold; if the number of image pairs is less than a second quantity threshold, determining the target preset position coordinates of the second device based on the first preset position coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device; if the number of image pairs is not less than the second quantity threshold, calculating the average value of the coordinate angle deviations of all image pairs whose similarity is not less than the preset similarity threshold; and determining the target preset position coordinates of the second device based on the first preset position coordinates, position deviation, coordinate angle deviation, optical axis deviation, and the average value of the coordinate angle deviations of the first device.

[0021] In a second aspect, an embodiment of the present application provides a data deviation processing device. The first device and the second device are pan-tilt camera devices, and the shooting starting points and shooting objects of the first device and the second device are the same. The device includes:

[0022] An acquisition module, configured to: acquire N first images captured by a first device at a first preset position according to a preset rule and N second images captured by a second device at a second preset position according to a preset rule, with a first step length; the preset position is used to indicate the coordinates and original magnification of a preset device; N is a positive integer; the step length is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are respectively arranged in their respective shooting sequences.

[0023] A first calculation module, configured to: determine a first target image from the N first images, and determine a second target image from the N second images that has a similarity correspondence relationship with the first target image; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold; obtain the first position of the first target image in the N first images and the second position of the second target image in the N second images, and determine the position deviation between the first device and the second device in the horizontal direction based on the deviation between the first position and the second position.

[0024] A second calculation module, configured to: determine the position correspondence relationship between each of the N first images and each of the N second images according to the correspondence relationship between the first position and the second position; construct N groups of image pairs based on the position correspondence relationship, where each group of image pairs includes a corresponding first image and a second image; screen out at least one group of image pairs whose included two images satisfy the similarity correspondence relationship from the N groups of image pairs; perform registration calculation on the two images in each group of the at least one group of image pairs to obtain a registration matrix; calculate the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position.

[0025] A processing module, configured to: determine the target preset position coordinates of the second device based on the coordinates, position deviation, and coordinate angle deviation of the first device at the first preset position, so that the shooting coverage area of the second device at the target preset position is the same as the shooting coverage area of the first device at the first preset position.

[0026] Optionally, the acquisition module is specifically configured to: acquire N first images captured by the first device at the first preset position according to a preset rule and N second images captured by the second device at the second preset position according to the preset rule, including: replacing the original magnification of the first device at the first preset position with the minimum magnification, and according to the first vertical coordinate and the minimum magnification, starting from the horizontal coordinate of 0°, with the first step length of 360° divided by N, horizontally rotating the pan-tilt of the first device, and intercepting N first images; arranging the N first images in the order of shooting; querying the first magnification of the second device at the minimum horizontal field of view angle based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device; according to the second vertical coordinate and the first magnification, starting from the horizontal coordinate of 0°, with the first step length of 360° divided by N, horizontally rotating the pan-tilt of the second device, and intercepting N second images; arranging the N second images in the order of shooting.

[0027] Optionally, when the first calculation module determines the first target image from the N first images and determines the second target image that has a similarity correspondence relationship with the first target image from the N second images, it is configured to: extract M first images from the N first images according to the second step length; the second step length is 2n times the first step length, where n is a positive integer; M is a positive integer less than N; extract 2M second images from the N second images according to the third step length; the third step length is one-half of the second step length; obtain any one of the first images from the M first images, register the any one of the first images with each of the 2M second images, and calculate the similarity between the registered any one of the first images and the second images; if there is at least one second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then use the second image with the maximum similarity with the any one of the first images as the second target image, and use the any one of the first images as the first target image; if there is no second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then adjust the value of the second vertical coordinate, and re-acquire N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the pan-tilt of the second device.

[0028] Optionally, after the first calculation module uses the second image with the maximum similarity to any one of the first images as the second target image and uses the any one of the first images as the first target image, it is further configured to: based on the position correspondence between the first target image and the second target image, and the angle correspondence between the M first images and the 2M second images, obtain M - 1 second images other than the second target image from the second target image; the angle correspondence is determined according to the relationship between the second step length and the third step length; the M - 1 second images correspond one by one to the M - 1 first images other than the first target image among the M first images; form a group of image pairs by combining each of the M - 1 second images with the corresponding first image among the M - 1 first images to obtain M - 1 groups of image pairs; perform registration calculation on the two images in each group of the M - 1 groups of image pairs, and calculate the similarity between the two registered images; count the number of image pairs with similarity not less than the preset similarity threshold; if the number is less than the first quantity threshold, adjust the value of the second vertical coordinate, re - obtain N second images of the second device at the second preset position, and re - determine the first target image and the second target image.

[0029] Optionally, when calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position, the second calculation module is configured to: for any one of at least one group of image pairs, perform the following operations: calculate the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation between the first device and the second device according to the linear transformation parameters, projection transformation parameters, and translation parameters included in the registration matrix, and the width and height of the image; according to the horizontal field of view angle and the vertical field of view angle of the second device at the second preset position, and the width and height of the image, convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into horizontal coordinate angle deviation and vertical coordinate angle deviation respectively; compare the difference between the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the any one of the image pairs and the horizontal coordinate angle deviation and the vertical coordinate angle deviation of the adjacent image pair, if the difference is not less than the preset difference threshold, discard the any one of the image pairs; if the difference is less than the preset difference threshold, set the any one of the image pairs as the correct image pair; determine the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all the correct image pairs.

[0030] Optionally, when determining the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs, the second calculation module is further configured to: calculate, through interpolation, the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle in the horizontal direction between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of all correct image pairs; and determine the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and the vertical coordinate angle deviation of each angle.

[0031] Optionally, the apparatus further includes a third calculation module; the third calculation module is configured to: obtain a first horizontal optical axis deviation value and a first vertical optical axis deviation value of the first device from the original magnification at the first preset position to the minimum magnification; query the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; obtain a second horizontal optical axis deviation value and a second vertical optical axis deviation value of the second device from the second magnification at the second preset position to the first magnification; and obtain the optical axis deviation between the first device and the second device by superimposing the first horizontal optical axis deviation value and the first vertical optical axis deviation value on the second horizontal optical axis deviation value and the second vertical optical axis deviation value; the processing module is configured to: determine the target preset position coordinate of the second device based on the coordinate, position deviation, coordinate angle deviation, and optical axis deviation of the first device at the first preset position.

[0032] Optionally, the apparatus further includes a fourth calculation module; the fourth calculation module is configured to: randomly obtain P preset positions from K preset positions, and respectively obtain J third images and J fourth images of the first device and the second device at each of the P preset positions according to a preset rule; K is a positive integer, P is a positive integer less than K, and J is a positive integer; determine J groups of image pairs based on the J third images and the J fourth images in the order of shooting, and the shooting order of the two images in each group of image pairs is the same; perform registration calculation on the two images in each group of image pairs, and calculate the similarity between the two images in each registered group of image pairs, and obtain the number of image pairs with a similarity not less than a preset similarity threshold; if the number of image pairs is less than a second quantity threshold, the processing module is configured to: determine the target preset position coordinate of the second device based on the first preset position coordinate, position deviation, coordinate angle deviation, and optical axis deviation of the first device; if the number of image pairs is not less than the second quantity threshold, calculate the mean value of the coordinate angle deviations of all image pairs with a similarity not less than the preset similarity threshold; the processing module is configured to: determine the target preset position coordinate of the second device based on the first preset position coordinate, position deviation, coordinate angle deviation, and optical axis deviation of the first device, and the mean value of the coordinate angle deviations.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, including at least one processor, and when the at least one processor executes a computer program stored in a memory, a method in the first aspect or any optional implementation manner of the first aspect is implemented.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions, and when the instructions are executed, a method in the first aspect or any optional implementation manner of the first aspect is implemented.

[0035] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer program code, and when the computer program code runs on a computer, a method in the first aspect or any optional implementation manner of the first aspect is implemented.

[0036] For the technical effects or advantages of one or more technical solutions provided in the second, third, fourth, and fifth aspects in the embodiments of the present application, they can all be correspondingly explained by the technical effects or advantages of the corresponding one or more technical solutions provided in the first aspect. Description of the Drawings

[0037] Figure 1 It is a flowchart of a data deviation processing method provided by an embodiment of the present application;

[0038] Figure 2 It is a structural diagram of a data deviation processing device provided by an embodiment of the present application;

[0039] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0040] In the technical solution of the present application, the collection, dissemination, use, etc. of data all comply with the requirements of relevant national laws and regulations.

[0041] It should be noted that in the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned, and they should be considered exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0042] The technical solution of the present application will be described in detail below through the drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0043] It should be understood that in the description of the embodiments of the present application, "a plurality of" means two or more. The "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. The term "and / or" in the embodiments of the present application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices. A module in the embodiments of the present application refers to a part with independent functions in a software system.

[0044] In scenarios such as substations, by presetting preset positions in advance, the cyclic monitoring of important devices and equipment can be achieved by relying on the inspection of a pan-tilt camera device at the preset positions. The number of preset positions set in advance varies according to different scenarios and focuses of attention, and can reach up to several hundred at most. When the pan-tilt camera device has an abnormality and needs to be disassembled, repaired, replaced, or the device is updated and replaced, due to factors such as installation location and differences in device software and hardware, all the original preset position information will become invalid. That is, if a new device is deployed at the original preset position, it will result in a large data deviation. Therefore, it is necessary to reset the preset positions of the new device. If the preset position information is reset manually, it will be extremely time-consuming and laborious, and there may be errors caused by subjective human judgment, resulting in a decrease in the accuracy of data deviation processing.

[0045] In view of this, the embodiments of the present application provide a method for processing data deviation. By obtaining N first images captured by a first device at a first preset position and N second images captured by a second device at a second preset position with the same step size, it is convenient to subsequently determine the position correspondence between the first image and the second image according to the step size. By calculating the similarity, a first target image and a second target image are determined, that is, two images with the same shooting content and shooting angle are determined. According to the deviation between the first position of the first target image in the N first images and the second position of the second target image in the second images, the position deviation between the first device and the second device in the horizontal direction can be determined. Further, based on the correspondence between the first position and the second position, the position correspondence between each first image and each second image can be determined, and then N groups of image pairs are constructed. At least one group of image pairs that meet the similarity correspondence relationship is selected from the N groups of image pairs for subsequent deviation calculation, which can improve the accuracy of deviation calculation. Then, registration calculation is performed on each group of image pairs in at least one group of image pairs. According to the obtained registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position, the coordinate angle deviation between the first device and the second device is calculated according to a preset algorithm. By comprehensively processing the obtained position deviation and coordinate angle deviation, the accuracy of data deviation processing can be improved, and the target preset position coordinates of the second device determined are more accurate.

[0046] It can be understood that the data deviation processing method provided by the embodiments of the present application can be applied to any electronic device with processing capabilities, and the electronic device can be an independent electronic device or an electronic device deployed on the first device or the second device. The embodiments of the present application do not limit this.

[0047] The embodiments of the present application can be applied to any scenario where a pan-tilt camera device needs to be replaced, including when the above-mentioned first device (for example, the first device is an old device) fails and needs to be disassembled and repaired, or the first device needs to be updated and replaced, etc., and the second device (for example, the second device is a new device) is used to replace the first device.

[0048] See Figure 1 , which is a flowchart of the data deviation processing method provided by the embodiments of the present application. The first device and the second device are pan-tilt cameras. The shooting starting points and shooting objects of the first device and the second device are the same, and taking the second device as the new device and the first device as the old device to be replaced as an example, the method includes the following steps S101 to S107:

[0049] Step S101: Obtain N first images captured by the first device at the first preset position according to a preset rule and N second images captured by the second device at the second preset position according to a preset rule with a first step size.

[0050] Among them, the preset position is used to indicate the coordinates and original magnification of the pre-set device; N is a positive integer; the step size is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are respectively arranged in their respective shooting sequences.

[0051] It can be understood that the setting of the preset position is determined by the user. In different scenarios, it is manually set according to the target object that the user needs to focus on. For example, in a substation scenario, the target objects that the user needs to focus on may be the operation conditions of certain meters or components. Therefore, the user can manually set the preset position of the pan-tilt camera. By adjusting parameters such as horizontal, vertical, and magnification, the pan-tilt camera can be aligned with the meters or components that the user needs to focus on. By setting multiple preset positions, rapid observation and monitoring of the target object can be achieved.

[0052] The pan-tilt camera device may include multiple preset positions. Taking K as an example, where K is a positive integer; then the coordinates of the preset position can be denoted as [Pk, Tk, Zk], where Pk represents the horizontal coordinate of the k-th preset position, Tk represents the vertical coordinate of the k-th preset position, and Zk represents the original magnification of the k-th preset position; the horizontal field of view angle corresponding to Zk can be denoted as fov_hk.

[0053] In a possible embodiment, the specific implementation manner of step S101 is as follows:

[0054] Replace the original magnification of the first device at the first preset position with the minimum magnification. According to the first vertical coordinate and the minimum magnification, starting from the horizontal coordinate of 0°, with the first step size being 360° divided by N, horizontally rotate the pan-tilt of the first device and intercept N first images; the N first images are arranged in the shooting sequence;

[0055] Query the first magnification of the second device at the minimum horizontal field of view angle based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device;

[0056] According to the second vertical coordinate and the first magnification, starting from the horizontal coordinate of 0°, with the first step size being 360° divided by N, horizontally rotate the pan-tilt of the second device and intercept N second images; the N second images are arranged in the shooting sequence.

[0057] Exemplarily, in order to ensure the accuracy of subsequent calculations, generally, the value of N needs to be greater than 24, and the more points are set, the more accurate the calculation result will be, and the corresponding calculation cost and complexity will also be greater. Therefore, the value of N can be selected according to actual needs.

[0058] It can be understood that, in order to ensure that during calibration, the first device and the second device (i.e., the old device and the new device) are aligned with the preset position and there is sufficient overlapping area between the two for calculation, in the embodiments of the present application, the minimum magnification Z of the first device is selected here omin (i.e., the minimum horizontal field of view angle is adopted). Because, after environmental calibration, there is a deviation between the preset point information of the second device and the preset point information of the first device. If directly adjusted to the maximum horizontal field of view angle, even a slight difference will cause the areas that the first device and the second device are aligned with to be completely different, that is, the overlapping part between the environmental scan maps of the first device and the second device is very small or non-existent, resulting in a large subsequent registration calculation error or even calculation failure, and thus the error of the preset point position cannot be calculated

[0059] Moreover, usually there is a corresponding relationship between the field of view angle and the magnification inside the pan-tilt camera device, which is generally provided by the lens manufacturer or further expanded based on the information provided by the lens manufacturer. Therefore, based on the first corresponding relationship between the field of view angle and the magnification of the first device, according to the minimum magnification Z of the first device omin query the minimum field of view angle of the first device corresponding to the minimum magnification of the first device from the first corresponding relationship. Similarly, based on the second corresponding relationship between the field of view angle and the magnification of the second device, query the first magnification Z corresponding to the minimum field of view angle of the first device in the second corresponding relationship according to the minimum field of view angle of the first device norres .

[0060] That is, the minimum magnification Z of the first device omin is the magnification corresponding to the minimum field of view angle of the first device in the first corresponding relationship; the first magnification Z norres is the magnification corresponding to the minimum field of view angle of the first device in the second corresponding relationship

[0061] Optionally, when obtaining the first magnification of the second device, if there is a horizontal field of view angle among all the horizontal field of view angles of the second device that is the same as the minimum horizontal field of view angle of the first device, directly obtain the magnification corresponding to this horizontal field of view angle as the first magnification; if there is no horizontal field of view angle among all the horizontal field of view angles of the second device that is the same as the minimum horizontal field of view angle of the first device, but there is a horizontal field of view angle whose similarity to the minimum horizontal field of view angle of the first device is not less than the preset horizontal field of view angle similarity threshold, then, use the magnification corresponding to this horizontal field of view angle as the first magnification; if the similarity between all the horizontal field of view angles of the second device and the minimum horizontal field of view angle of the first device is less than the preset horizontal field of view angle similarity threshold, it means that the hardware difference between the first device and the second device is too large, and it is impossible to replace the first device with the second device, end the calculation process, and output an alarm prompt

[0062] S102. Determine a first target image from N first images, and determine a second target image from N second images that has a similarity correspondence relationship with the first target image.

[0063] Among them, the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold. The preset similarity threshold can be set according to actual needs.

[0064] In a possible embodiment, the specific implementation manner of step S102 is as follows:

[0065] Extract M first images from N first images according to the second step length; the second step length is 2n times the first step length, where n is a positive integer; M is a positive integer less than N;

[0066] Extract 2M second images from N second images according to the third step length; the third step length is half of the second step length;

[0067] Obtain any one of the first images from the M first images, register this any one of the first images with each of the 2M second images, and calculate the similarity between the registered any one of the first images and the second images;

[0068] If there is at least one second image whose similarity with this any one of the first images is not less than the preset similarity threshold, then use the second image with the maximum similarity with this any one of the first images as the second target image, and use this any one of the first images as the first target image;

[0069] If there is no second image whose similarity with this any one of the first images is not less than the preset similarity threshold, then adjust the value of the second vertical coordinate, and re-obtain N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the pan-tilt of the second device.

[0070] Exemplarily, first, assume that the N first images are old1~oldN, and extract M first images from the N first images at an angle interval of the second step length to obtain the first images o1~oM; the N second images are new1~newN, and extract 2M second images from the N second images at an angle interval of the third step length to obtain the second images n1~n2M.

[0071] It can be understood that the second step length needs to be a positive integer multiple of 2 of the first step length, that is, 2n times the first step length, where n is a positive integer. That is, assuming the first step length is s, the second step length is 2n·s, and the third step length is n·s. In addition, the multiple ratio (i.e., the ratio of N and M) also needs to be an integer multiple of 2. It can be understood that generally, a pan-tilt camera device can achieve 360° rotation to observe the surrounding environment of the target object; when rotating to different angles, the observed area will also change. In order to establish the positional relationship between the first device and the second device (i.e., the old device and the new device), there needs to be an overlapping area between the environmental scan maps of the first device and the second device (i.e., the N first images and second images obtained). In theory, the larger the selected N, the higher the calculation accuracy, but the corresponding calculation speed will decrease. However, if the selected N is too small, it will lead to too small an overlapping area between the environmental scan maps of the first device and the second device, resulting in a large calculation error or even calculation failure. Therefore, the multiple ratio is set here, which can play a role in balancing accuracy and speed.

[0072] Secondly, extract the first image om from o1~oM; om can be a randomly extracted first image, or a first image extracted in ascending or descending order of the numbers; register and calculate om with n1~n2M respectively, and calculate the similarity of each group of registered images, that is, calculate the similarity between the registered om and n1, calculate the similarity between the registered om and n2, and so on, and the similarity between the registered om and each second image in n1~n2M can be obtained.

[0073] It can be understood that the specific registration calculation method can be selected according to actual needs, and the embodiments of the present application do not limit this.

[0074] Then, obtain the second image nb with the highest similarity to the registered om, and determine whether the similarity between the registered om and nb is not less than the preset similarity threshold; if so, determine om as the first target image and nb as the second target image; if not, re-extract om from o1~oM, and perform the above registration calculation and similarity calculation. If all the first images in o1~oM have been traversed and the first image and the second image that meet the similarity correspondence relationship have not been obtained, then return to step S101 above, modify the value of the second vertical coordinate, re-obtain the N second images of the second device at the second preset position, and repeat step S102 until the first target image and the second target image are determined.

[0075] It can be understood that due to the large morphological differences between the first device and the second device, when installed at the same position, there is a large distance in the vertical direction. For the case of the first device looking straight ahead, the second device may need to look up or down. Therefore, by dynamically adjusting the value of the second vertical coordinate of the second device, the shooting angle and shooting range of the second device are made the same as those of the first device at the first vertical coordinate. The initial value of the second vertical coordinate can be set randomly or preset manually, and the value of the second vertical coordinate needs to be within the vertical movement coordinate range of the pan-tilt of the second device.

[0076] Optionally, if all the selectable values of the second vertical coordinate are traversed and the first image and the second image that meet the similarity correspondence relationship are still not obtained, it means that the hardware and other factors of the first device and the second device are quite different, and the operation of replacing the first device with the second device cannot be realized. The calculation process is ended and an alarm prompt is output.

[0077] Further, in a possible embodiment, after taking the second image with the largest similarity to any one of the first images as the second target image and taking any one of the first images as the first target image, it is also necessary to verify the first target image and the second target image to ensure that the position correspondence relationship between the N first images and the N second images can be determined according to the position correspondence relationship between the first target image and the second target image. The specific verification method is as follows:

[0078] Based on the position correspondence relationship between the first target image and the second target image, and the angle correspondence relationship between the M first images and the 2M second images, obtain M - 1 second images other than the second target image from the second target image; the angle correspondence relationship is determined according to the relationship between the second step length and the third step length; the M - 1 second images correspond one by one to the M - 1 first images other than the first target image among the M first images;

[0079] Form a group of image pairs by combining each of the M - 1 second images in the M - 1 second images with the corresponding first image in the M - 1 first images, and obtain M - 1 groups of image pairs;

[0080] Perform registration calculation on the two images in each group of the M - 1 groups of image pairs, and calculate the similarity between the two registered images; count the number of image pairs with a similarity not less than the preset similarity threshold; if the number is less than the first quantity threshold, adjust the value of the second vertical coordinate, re - obtain the N second images of the second device at the second preset position, and re - determine the first target image and the second target image.

[0081] Exemplarily, the first target image is om, that is, the position of the first target image among o1~oM is m, and the second target image is nb, that is, the position of the second target image among n1~n2M is b. According to the correspondence between om and nb, it can be known that the position m in the first target image corresponds to the position b in the second target image.

[0082] Since om and nb are corresponding, it can be understood that the shooting angles and shooting contents of om and nb are the same, that is, the angles corresponding to the positions m and b are the same. Further, according to the relationship between the second step length and the third step length, that is, the second step length is twice the third step length, it can be understood that the shooting angle interval corresponding to the second step length is twice the shooting angle interval of the third step length. Then it can be known that the angles corresponding to the positions m+1 and b+2 are also the same. Specifically, assuming that the angle corresponding to m and b is a, then the angle corresponding to m+1 is a + the second step length (that is, a + 2n·s), and the angle corresponding to b+2 is a + 2·the third step length (that is, a + 2n·s), and the angles they correspond to are the same, that is, the positions m+1 and b+2 are corresponding. Similarly, the positions m+2 and b+4 are corresponding, and the positions m-1 and b-2 are corresponding. From this, it can be inferred that the position m+r in the first target image and b+2r in the second target image (r is an integer, and r is used to indicate the position interval) are also corresponding. Therefore, according to this correspondence, M-1 groups of image pairs other than om and nb can be determined, that is, the second images corresponding to the positions of each of the M-1 first images are determined.

[0083] For each group of image pairs in the M-1 groups of image pairs, perform registration calculation on the two images included therein, calculate the similarity of the two registered images, and count the number of image pairs whose similarity is not less than the preset similarity threshold;

[0084] If the number is less than the first quantity threshold, it means that the currently determined first target image and second target image are unreliable, and the position correspondence between each first image and each second image cannot be accurately determined based on the first target image and the second target image; therefore, it is necessary to adjust the value of the second vertical coordinate, re-obtain N second images of the second device at the second preset position, and re-determine the first target image and the second target image;

[0085] If the number is not less than the first quantity threshold, it means that the currently determined first target image and second target image are reliable, and the current first target image and second target image can be adopted.

[0086] S103. Obtain the first position of the first target image among the N first images and the second position of the second target image among the N second target images, and determine the position deviation in the horizontal direction between the first device and the second device based on the deviation between the first position and the second position.

[0087] Exemplarily, continuing with the example in the above step S102, the first target image is om, and its position among o1 to oM is m. According to the relationship between o1 to oM and old1 to oldN, it can be known that the position of om among old1 to oldN is (m - 1) × ratio + 1, where ratio represents N / M. Similarly, the second target image is nb, and its position among n1 to n2M is b. According to the corresponding relationship between n1 to n2M and new1 to newN, it can be known that the position of nb among new1 to newN is (b - 1) × ratio / 2 + 1. That is, the position (m - 1) × ratio + 1 among old1 to oldN corresponds to the position (b - 1) × ratio / 2 + 1 among new1 to newN.

[0088] Therefore, the position deviation (which can also be called the relationship deviation) rx in the horizontal direction between the first device and the second device can be determined according to the position deviation between (m - 1) × ratio + 1 and (b - 1) × ratio / 2 + 1 as follows:

[0089] rx = (b - 1) × s / 2 - (m - 1) × s;

[0090] This position deviation can reflect the deviation in the horizontal installation between the first device and the second device, and also reflects the angular correspondence relationship between the first device and the second device.

[0091] S104. Determine the position correspondence relationship between each of the N first images among the N first images and each of the N second images among the N second images according to the correspondence relationship between the first position and the second position; construct N pairs of images according to the position correspondence relationship.

[0092] Among them, each pair of images includes one first image and one second image with corresponding positions.

[0093] Exemplarily, continuing with the example in the above step S103, it can be known that the position (m - 1) × ratio + 1 in old1 to oldN corresponds to the position (b - 1) × ratio / 2 + 1 in new1 to newN; and based on the same step sizes of old1 to oldN and new1 to newN, it can be known that the position (m - 1) × ratio + 1 + r in old1 to oldN also corresponds to the position (b - 1) × ratio / 2 + 1 + r (r is an integer) in new1 to newN. Therefore, according to this position correspondence, N groups of image pairs can be constructed. The r values of the first image and the second image in each group of image pairs are the same.

[0094] S105. Select at least one group of image pairs from the N groups of image pairs whose two included images satisfy the similarity correspondence relationship; perform registration calculation on the two images in each group of the at least one group of image pairs to obtain a registration matrix.

[0095] Exemplarily, perform registration calculation on the two images in each group of the N groups of image pairs, calculate the similarity between the two registered images, and obtain at least one group of image pairs whose similarity is greater than a preset similarity threshold.

[0096] Optionally, if the number of the at least one group of image pairs is less than a third quantity threshold, it indicates that the hardware difference between the first device and the second device is too large, and the second device cannot replace the first device. End the calculation process and output an alarm prompt.

[0097] For at least one group of image pairs that satisfy the similarity correspondence relationship, obtain the registration matrix according to their registration calculation. The following is an example of a registration matrix t provided by an embodiment of the present application:

[0098] ;

[0099] Among them, m0, m1, m2, m3 respectively represent four different linear transformation parameters; p0, p1 respectively represent two different projection transformation parameters; t0, t1 respectively represent two different translation transformation parameters, and 1 is used to represent the image size transformation parameter (the image size transformation parameter being 1 indicates that the image does not need to be size-changed).

[0100] S106. Calculate the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position.

[0101] In a possible embodiment, the specific implementation manner of step S106 is as follows:

[0102] For any one of the at least one group of image pairs, perform the following operations:

[0103] Calculate the horizontal coordinate pixel deviation and vertical coordinate pixel deviation between the first device and the second device according to the linear transformation parameters, projection transformation parameters, and translation transformation parameters included in the registration matrix, as well as the width and height of the image.

[0104] According to the horizontal field of view angle and vertical field of view angle of the second device at the second preset position, as well as the width and height of the image, convert the horizontal coordinate pixel deviation and vertical coordinate pixel deviation into horizontal coordinate angle deviation and vertical coordinate angle deviation respectively.

[0105] Compare the difference between the horizontal coordinate angle deviation and vertical coordinate angle deviation of any one image pair and the horizontal coordinate angle deviation and vertical coordinate angle deviation of the adjacent image pair. If the difference is not less than the preset difference threshold, discard any one image pair; if the difference is less than the preset difference threshold, set any one image pair as the correct image pair.

[0106] Determine the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs.

[0107] Exemplarily, taking the registration matrix t shown in the above step S105 as the registration matrix, the width of the image is w, and the height is h. Calculate the horizontal coordinate pixel deviation between the first device and the second device according to the following formula 1 and the vertical coordinate pixel deviation :

[0108] (Formula 1)

[0109] Subsequently, according to the horizontal field of view angle and the vertical field of view angle of the second device corresponding to the first magnification, as well as the width w and height h of the image, convert the horizontal coordinate pixel deviation into the horizontal coordinate angle deviation dx and the vertical coordinate pixel deviation into the vertical coordinate angle deviation dy according to the following formula 2:

[0110] (Formula 2)

[0111] According to the above formula 1 and formula 2, the horizontal coordinate angle deviation and vertical coordinate angle deviation of each group of image pairs at the corresponding angles can be obtained.

[0112] After obtaining the horizontal coordinate angle deviation and vertical coordinate angle deviation of each group of image pairs at the corresponding angles, it is also necessary to verify the obtained coordinate angle deviation to avoid sudden changes or misjudgments. That is, by comparing the horizontal coordinate angle deviation and vertical coordinate angle deviation of each group of image pairs with the horizontal coordinate angle deviation and vertical coordinate angle deviation of the adjacent image pairs of the same group of image pairs, if the difference is less than the preset difference threshold, it means that there is no sudden change in this image pair, and it is set as the correct image pair; if the difference is not less than the preset difference threshold, it means that there is a sudden change in this image pair, which is a misjudgment point, and it is discarded.

[0113] Furthermore, since the coordinate angle deviations of the first device and the second device calculated according to the above embodiments are only the coordinate angle deviations of some image pairs at the corresponding angles, and cannot cover the coordinate angle deviations of each angle in the 360° horizontal direction.

[0114] Therefore, the present application also provides a possible embodiment: based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs, the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle in the horizontal direction of the first device and the second device are calculated by interpolation; based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle, the coordinate angle deviation of the first device and the second device is determined. In this way, the obtained coordinate angle deviations of the first device and the second device are more comprehensive.

[0115] It can be understood that the specific interpolation method can be selected according to actual needs, and the embodiments of the present application do not limit this.

[0116] S107. Based on the coordinates, position deviation, and coordinate angle deviation of the first device at the first preset position, determine the target preset position coordinates of the second device.

[0117] In this way, the shooting coverage area of the second device at the target preset position can be made the same as the shooting coverage area of the first device at the first preset position.

[0118] Exemplarily, through the coordinates and position deviation of the first device at the first preset position, the rough coordinates corresponding to the first preset position of the first device in the second device can be roughly determined. According to the coordinate angle deviation, the more accurate coordinates (i.e., the target preset position coordinates) of the first preset position of the first device in the second device can be obtained, thereby realizing the coordinate mapping from the environmental point position of the first device to the second device.

[0119] In a possible design, due to the influence of the optical axis deviation, there are deviations in the images of the pan-tilt camera device at different magnifications, the image centers are not consistent, and the aligned target areas are also not consistent. As a result, there are still deviations in the target preset positions obtained only based on the position deviation and the coordinate angle deviation. Therefore, the embodiments of the present application further provide a method for eliminating the influence of the optical axis deviation. The specific implementation manner of this method is as follows:

[0120] Obtain the first horizontal optical axis deviation value and the first vertical optical axis deviation value of the first device from the original magnification at the first preset position to the minimum magnification;

[0121] Query the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; obtain the second horizontal optical axis deviation value and the second vertical optical axis deviation value of the second device from the second magnification at the second preset position to the first magnification;

[0122] Based on the first horizontal optical axis deviation value and the first vertical optical axis deviation value, superimpose the second horizontal optical axis deviation value and the second vertical optical axis deviation value to obtain the optical axis deviation of the first device and the second device;

[0123] Based on the coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device at the first preset position, determine the target preset position coordinates of the second device.

[0124] Exemplarily, assume that the first preset position is preset position k, the horizontal coordinate of the first device at the original magnification Zk at the first preset position is Pk, the vertical coordinate is Tk, and the horizontal coordinate and vertical coordinate of the first device at the minimum magnification Z omin of the first device at the first preset position are Pk0 and Tk0 respectively; obtain the horizontal optical axis deviation value optx and the vertical optical axis deviation value opty of the first device from the original magnification Zk to the minimum magnification Z omin . It can be known that Pk0 = Pk + optx, Tk0 = Tk + opty.

[0125] According to the above embodiments, the target preset position coordinates of the second device obtained by superimposing the position deviation and the coordinate angle deviation on the coordinates [Pk0, Tk0, Z omin of the first device at the first preset position are [Pk1, Tk1, Z norres , where Z norres is the first magnification of the second device. The target preset position coordinates are the coordinates mapped according to the first device at the minimum magnification Z omin . Since the second device also has an optical axis deviation, the influence brought by the optical axis deviation of the second device needs to be considered.

[0126] That is, obtain the second device from the second magnification Z korres to the first magnification Z norresThe horizontal optical axis deviation and vertical optical axis deviation. Based on Pk1 and Tk1, after respectively superimposing the horizontal optical axis deviation and vertical optical axis deviation, the target preset position coordinates [Pk2, Tk2, Z korres .

[0127] It can be understood that the optical axis deviation is an attribute of the pre-set pan-tilt camera, which can be directly queried and obtained according to the magnification.

[0128] In this way, the optical axis deviation is also covered in the data deviation processing, which can improve the accuracy of the data deviation processing and make the obtained target preset position coordinates more accurate.

[0129] In a possible design, considering the error of the optical axis deviation itself and the interpolation error existing when expanding the coordinate angle deviation by the interpolation method, the embodiment of the present application also provides a method for further improving the accuracy of the data deviation processing. The specific implementation manner of this method is as follows:

[0130] Randomly obtain P preset positions from K preset positions, and respectively obtain J third images and J fourth images of the first device and the second device at each of the P preset positions according to a preset rule; K is a positive integer, P is a positive integer less than K, and J is a positive integer;

[0131] Based on the J third images and the J fourth images, determine J pairs of image pairs in the order of shooting. The shooting order of the two images in each pair of image pairs is the same;

[0132] Perform registration calculation on the two images in each pair of image pairs, calculate the similarity between the two images in each pair of registered image pairs, and obtain the number of image pairs whose similarity is not less than a preset similarity threshold;

[0133] If the number of image pairs is less than the second quantity threshold, determine the target preset position coordinates of the second device based on the first preset position coordinates, position deviation, coordinate angle deviation, and optical axis deviation of the first device;

[0134] If the number of image pairs is not less than the second quantity threshold, calculate the mean value of the coordinate angle deviations of all image pairs whose similarity is not less than the preset similarity threshold; based on the first preset position coordinates, position deviation, coordinate angle deviation, optical axis deviation, and the mean value of the coordinate angle deviations of the first device, determine the target preset position coordinates of the second device.

[0135] Exemplarily, a specific proportion of the preset positions are randomly and non-repeatedly selected from K preset positions to obtain J preset positions; the specific proportion can be set according to actual requirements, such as 20% or the like. And the first image and the second image of the first device and the second device at each of the J preset positions are obtained; according to the registration calculation and the similarity calculation method in the method of the above embodiment, the number of image pairs with a similarity not less than the preset similarity threshold is determined.

[0136] If the number of image pairs is not less than the second quantity threshold, then the average value of the coordinate angle deviations of these image pairs is obtained, and based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, and the average value of the coordinate angle deviations, the target preset position coordinates of the second device are determined. That is, on the basis of Pk2 and Tk2, the average value of the horizontal coordinate angle deviation and the average value of the vertical coordinate angle deviation are respectively superimposed.

[0137] If the number of image pairs is less than the second quantity threshold, then more preset positions can be obtained for calculation by increasing the specific proportion, or the number of the first images and the second images obtained can be increased, etc., so that the number of image pairs satisfying the similarity correspondence relationship is greater than the second quantity threshold. If the number of image pairs still cannot be made not less than the second quantity threshold by increasing the specific proportion or increasing the number of the first images and the second images, then the target preset position coordinates of the second device are directly determined based on the first preset position coordinates, position deviation, coordinate angle deviation and optical axis deviation of the first device, that is, the target preset position coordinates are [Pk2, Tk2, Z korres .

[0138] In this way, by calculating the average value of the coordinate angle deviations of the first device and the second device at multiple preset positions to reduce the influence caused by the error of the optical axis deviation itself and the interpolation error, the accuracy of data deviation processing can be further improved. And the accuracy requirement for the optical axis deviation can be reduced, which is more convenient for engineering implementation in practical applications.

[0139] In this embodiment, by calculating the position deviation and coordinate angle deviation between the first device and the second device, the coordinate mapping of the first device at the first preset position to the target preset position of the second device can be determined, which can solve the problem that the original preset position cannot be reused caused by problems such as replacement, disassembly and reinstallation of pan-tilt camera devices, or equipment aging and deviation, effectively improving the import efficiency of new pan-tilt camera devices and reducing labor costs. Moreover, the influence of the optical axis deviation between the first device and the second device on the target preset position is also considered, and the optical axis deviation is superimposed on the basis of the position deviation and the coordinate angle deviation, making the target preset position coordinates more accurate and improving the accuracy of data deviation processing. In addition, considering the error of the optical axis deviation itself and the interpolation error of the interpolation method used in calculating the coordinate angle deviation, by calculating the average value of the coordinate angle deviations of the first device and the second device at multiple preset positions, the target preset position coordinates are optimized by the average value of the coordinate angle deviations, further improving the accuracy of data deviation processing.

[0140] The method provided by the embodiments of the present application is introduced above. The device provided by the embodiments of the present application is introduced below.

[0141] Based on the same technical concept, the embodiments of the present application provide a data deviation processing device. The first device and the second device are pan-tilt camera devices, and the shooting starting points and shooting objects of the first device and the second device are the same. The device includes modules / units / means for executing the methods executed by the electronic device in the above method embodiments. The modules / units / means can be implemented by software, or by hardware, or by hardware executing corresponding software.

[0142] Exemplarily, as Figure 2 shown, the device 200 includes:

[0143] An acquisition module 201, configured to: acquire N first images captured by the first device at the first preset position according to a preset rule and N second images captured by the second device at the second preset position according to a preset rule in accordance with a first step size; the preset position is used to indicate the coordinates and original magnification of the preset device; N is a positive integer; the step size is used to indicate the shooting angle interval between two consecutive images; the N first images and the N second images are respectively arranged in the order of their respective shooting sequences;

[0144] The first calculation module 202 is configured to: determine a first target image from N first images, and determine a second target image from N second images, where there is a similarity correspondence relationship between the second target image and the first target image; the similarity correspondence relationship is used to indicate that the similarity between the first target image and the second target image is not less than a preset similarity threshold; obtain a first position of the first target image in the N first images, and a second position of the second target image in the N second images, and determine a position deviation between the first device and the second device in the horizontal direction based on the deviation between the first position and the second position;

[0145] The second calculation module 203 is configured to: determine a position correspondence relationship between each of the N first images in the N first images and each of the N second images in the N second images according to the correspondence relationship between the first position and the second position; construct N pairs of images according to the position correspondence relationship, where each pair of images includes a first image and a second image with corresponding positions; screen out at least one pair of images whose included two images satisfy the similarity correspondence relationship from the N pairs of images; perform registration calculation on the two images in each pair of images in at least one pair of images to obtain a registration matrix; calculate a coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position;

[0146] The processing module 204 is configured to: determine the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation, and the coordinate angle deviation, so that the shooting coverage area of the second device at the target preset position is the same as the shooting coverage area of the first device at the first preset position.

[0147] It should be understood that all relevant contents of the steps involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here.

[0148] Based on the same technical concept, see Figure 3 , this application embodiment also provides an electronic device 300, including:

[0149] At least one processor 301; and a communication interface 303 communicatively connected to the at least one processor 301; the at least one processor 301 causes the electronic device 300 to execute the method steps performed by the kanban in the above method embodiment through the communication interface 303 by executing instructions stored in the memory 302.

[0150] Optionally, the memory 302 is located outside the electronic device 300.

[0151] Optionally, the electronic device 300 includes the memory 302, which is connected to the at least one processor 301, and the memory 302 has instructions executable by the at least one processor 301. Attached Figure 3 The memory 302 is shown as optional for the electronic device 300 with a dashed line.

[0152] Wherein, the at least one processor 301 and the memory 302 can be coupled through an interface circuit or integrated together, without limitation here.

[0153] In the embodiments of the present application, the specific connection medium between the at least one processor 301, the memory 302, and the communication interface 303 is not limited. In the embodiments of the present application Figure 3 It is shown that the at least one processor 301, the memory 302, and the communication interface 303 are connected through a bus 304. The bus is Figure 3 shown as a thick line. The connection manners between other components are only for illustrative purposes and are not restrictive. This bus part can be an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 3 it is only shown as a thick line, but it does not mean that there is only one bus or one type of bus.

[0154] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor that realizes functions by reading software codes stored in the memory.

[0155] Exemplarily, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0156] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which serves as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0157] It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, the memory (storage module) may be integrated in the processor.

[0158] It should be noted that the memory described herein is intended to include but not be limited to these and any other suitable types of memory.

[0159] Based on the same technical concept, the embodiments of the present application also provide a computer-readable storage medium for storing instructions, which when executed cause a computer to execute the method steps performed by any of the devices in the above method embodiments.

[0160] Based on the same technical concept, the embodiments of the present application also provide a computer program product including computer program code, which when running on a computer causes the method steps performed by any of the devices in the above method embodiments to be implemented.

[0161] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0162] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0163] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realize the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0165] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A data deviation processing method, characterized in that, The first device and the second device are pan-tilt camera devices, and the shooting starting points and shooting objects of the first device and the second device are the same; the method includes: Obtaining N first images captured by the first device at a first preset position according to a preset rule and N second images captured by the second device at a second preset position according to the preset rule at a first step length; N is a positive integer; the N first images and the N second images are respectively arranged in their respective shooting sequences; Determining a first target image from the N first images and determining a second target image from the N second images, where the similarity between the second target image and the first target image is not less than a preset similarity threshold; Obtaining a first position of the first target image in the N first images and a second position of the second target image in the N second images, and determining the position deviation between the first device and the second device in the horizontal direction based on the deviation between the first position and the second position; Determining the position correspondence between each of the N first images in the N first images and each of the N second images in the N second images according to the correspondence between the first position and the second position; constructing N groups of image pairs according to the position correspondence, and each group of image pairs includes a first image and a second image with corresponding positions; Filtering out at least one group of image pairs whose similarity between the two included images is not less than the preset similarity threshold from the N groups of image pairs; performing registration calculation on the two images in each group of the at least one group of image pairs to obtain a registration matrix; calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position; Based on the coordinates of the first device at the first preset position, the position deviation, and the coordinate angle deviation, determining the target preset position coordinates of the second device so that the shooting coverage area of the second device at the target preset position is the same as the shooting coverage area of the first device at the first preset position.

2. The method according to claim 1, characterized in that The obtaining N first images captured by the first device at a first preset position according to a preset rule and N second images captured by the second device at a second preset position according to the preset rule at a first step length includes: Replacing the original magnification of the first device at the first preset position with the minimum magnification, and rotating the pan-tilt of the first device horizontally starting from a horizontal coordinate of 0° at a first vertical coordinate and the minimum magnification, where the first step length is 360° divided by N, and intercepting the N first images; Querying a first magnification of the second device under the minimum horizontal field of view angle based on the minimum horizontal field of view angle corresponding to the minimum magnification of the first device; Rotating the pan-tilt of the second device horizontally starting from a horizontal coordinate of 0° at a second vertical coordinate and the first magnification, where the first step length is 360° divided by N, and intercepting the N second images.

3. The method according to claim 2, characterized in that, Determining a first target image from the N first images and determining a second target image from the N second images that has a similarity correspondence relationship with the first target image includes: Extracting M first images from the N first images according to a second step length; the second step length is 2n times the first step length, where n is a positive integer; M is a positive integer less than N; Extracting 2M second images from the N second images according to a third step length; the third step length is one-half of the second step length; Obtaining any one of the M first images, registering the any one of the first images with each of the 2M second images, and calculating the similarity between the registered any one of the first images and the second images; If there is at least one second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then taking the second image with the maximum similarity with the any one of the first images as the second target image and taking the any one of the first images as the first target image; If there is no second image whose similarity with the any one of the first images is not less than the preset similarity threshold, then adjust the value of the second vertical coordinate and re-obtain the N second images of the second device at the second preset position; the value range of the second vertical coordinate is within the vertical movement coordinate range of the pan-tilt of the second device.

4. The method according to claim 3, wherein After taking the second image with the maximum similarity with the any one of the first images as the second target image and taking the any one of the first images as the first target image, the method further includes: Based on the position correspondence relationship between the first target image and the second target image, and the angle correspondence relationship between the M first images and the 2M second images, obtaining M-1 second images other than the second target image from the second target image; the angle correspondence relationship is determined according to the relationship between the second step length and the third step length; the M-1 second images correspond one-to-one with the M-1 first images other than the first target image among the M first images; Forming a pair of images by combining each of the M-1 second images with the corresponding first image among the M-1 first images to obtain M-1 pairs of images; Performing registration calculation on the two images in each pair of images among the M-1 pairs of images and calculating the similarity between the registered two images; counting the number of pairs of images whose similarity is not less than the preset similarity threshold; If the number is less than the first quantity threshold, then adjust the value of the second vertical coordinate, re-obtain the N second images of the second device at the second preset position, and re-determine the first target image and the second target image.

5. The method according to claim 1, characterized in that, Calculating the coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the image, and the field of view angle of the second device at the second preset position includes: For any pair of images in the at least one pair of images, perform the following operations: Calculate the horizontal coordinate pixel deviation and vertical coordinate pixel deviation between the first device and the second device according to the linear transformation parameters, projection transformation parameters, and translation transformation parameters included in the registration matrix, and the width and height of the image; According to the horizontal field of view angle and vertical field of view angle of the second device at the second preset position, and the width and height of the image, convert the horizontal coordinate pixel deviation and the vertical coordinate pixel deviation into horizontal coordinate angle deviation and vertical coordinate angle deviation respectively; Compare the difference between the horizontal coordinate angle deviation and vertical coordinate angle deviation of any image pair and the horizontal coordinate angle deviation and vertical coordinate angle deviation of the adjacent image pair. If the difference is not less than the preset difference threshold, discard the any image pair; if the difference is less than the preset difference threshold, set the any image pair as the correct image pair; Determine the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs.

6. The method according to claim 5, characterized in that, The determining the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs includes: Based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of all correct image pairs, determine the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle in the horizontal direction of the first device and the second device through interpolation calculation; determine the coordinate angle deviation between the first device and the second device based on the horizontal coordinate angle deviation and vertical coordinate angle deviation of each angle.

7. The method according to any one of claims 1 to 6, characterized in that The determining the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation, and the coordinate angle deviation includes: Obtain the first horizontal optical axis deviation value and the first vertical optical axis deviation value of the first device from the original magnification at the first preset position to the minimum magnification; Query the second magnification of the second device at the original horizontal field of view angle based on the original horizontal field of view angle corresponding to the original magnification; Obtain the second horizontal optical axis deviation and the second vertical optical axis deviation of the second device from the second magnification at the second preset position to the first magnification; Superimpose the second horizontal optical axis deviation and the second vertical optical axis deviation based on the first horizontal optical axis deviation value and the first vertical optical axis deviation value to obtain the optical axis deviation between the first device and the second device; Determine the target preset position coordinates of the second device based on the coordinates of the first device at the first preset position, the position deviation, the coordinate angle deviation, and the optical axis deviation.

8. A data deviation processing device, characterized in that, The first device and the second device are pan-tilt camera devices, and the shooting starting points and shooting objects of the first device and the second device are the same; the device includes: An acquisition module, configured to: acquire N first images captured by the first device at a first preset position according to a first step length and N second images captured by the second device at a second preset position according to the preset rule; N is a positive integer; the N first images and the N second images are respectively arranged in the order of their respective shooting sequences. A first calculation module, configured to: determine a first target image from the N first images, and determine a second target image from the N second images, the similarity between the second target image and the first target image being not less than a preset similarity threshold; obtain a first position of the first target image in the N first images and a second position of the second target image in the N second images, and determine a position deviation between the first device and the second device in the horizontal direction based on the deviation between the first position and the second position. A second calculation module, configured to: determine a position correspondence between each of the N first images and each of the N second images according to the correspondence between the first position and the second position; construct N pairs of images according to the position correspondence, each pair of images including a first image and a second image with corresponding positions; screen out at least one pair of images from the N pairs of images, the similarity between the two images included in each pair of images being not less than the similarity threshold; perform registration calculation on the two images in each pair of images in the at least one pair of images to obtain a registration matrix; calculate a coordinate angle deviation between the first device and the second device according to the registration matrix, the width and height of the images, and the field of view angle of the second device at the second preset position. A processing module, configured to: determine a target preset position coordinate of the second device based on the coordinate of the first device at the first preset position, the position deviation, and the coordinate angle deviation, so that the shooting coverage area of the second device at the target preset position is the same as the shooting coverage area of the first device at the first preset position.

9. An electronic device, characterized in that, Comprising: A memory, configured to store program instructions. A processor, configured to call the program instructions stored in the memory and execute the steps included in the method according to any one of claims 1-7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a computer, the method according to any one of claims 1-7 is implemented.

Images