Method for ranging with coincident center view points of binocular PTZ cameras

Abstract

The application discloses a ranging method for coinciding central viewpoints of binocular PTZ cameras, solves the problem that a calibration plate cannot be used for recalibration in an online environment during three-dimensional detection of active vision, and belongs to the technical field of active vision.The application comprises the following steps: left and right cameras of binocular PTZ cameras are used to shoot target images at respective positions in a spherical cavity with various pitch angles and azimuth angles, and an angle-image encoder is obtained; during ranging, the target is shot by the left and right cameras; according to the shot images, the azimuth angle and the pitch angle corresponding to the image pair shot by the left and right cameras are found in the angle-image encoder; then, the included angles alpha0 and beta0 between the left and right cameras and the target point are obtained in combination with the geometric relationship between the positions of the left and right cameras and the target; and finally, the three-dimensional distance of the target point is solved according to alpha0, beta0 and the distance D between the position points of the rotation centers of the left and right cameras.The application can still be used for ranging when the central viewpoints coincide without self-calibration under the condition that the internal and external parameters change after the focusing and the shooting angle and range are adjusted.

Description

TECHNICAL FIELD

[0001] The application relates to a ranging method for center point coincidence of a binocular PTZ camera, and belongs to the technical field of active vision. BACKGROUND

[0002] A plurality of diagnostic instruments are installed in a large spherical vacuum target chamber, and during work and debugging, the diagnostic instruments are in and out of the vicinity of the target chamber through flange openings, collision risks exist between the instruments and equipment, and collision detection problems between the moving instruments and equipment need to be solved.

[0003] Key point distance measurement through a plurality of cameras distributed outside the target chamber is a main approach to solving the problem, a traditional binocular vision system is limited by the installation position of the camera, the shooting angle and range are fixed, the target cannot be dynamically tracked, the observation angle cannot be changed, and the target is easily lost due to target movement or camera blocking.

[0004] An active vision system has horizontal swing (pan) (i.e. azimuth angle), tilt and zoom (Zoom) capabilities, the shooting angle and range can be adjusted, and is more suitable for tracking and detecting a moving target. However, after focusing and adjusting the shooting angle and range, the internal and external parameters change, and in an online environment, a calibration plate cannot be used for re-calibration, and the problem of self-calibration of a PTZ binocular camera needs to be solved. SUMMARY

[0005] In view of the problem that the internal and external parameters change after focusing and pointing adjustment during three-dimensional detection of active vision, and a calibration plate cannot be used for re-calibration in an online environment, the application provides an active stereo vision three-dimensional detection method which does not need to calibrate the internal and external parameters and can still be performed

[0006] The ranging method for center point coincidence of the binocular PTZ camera comprises the following steps:

[0007] S1, the left and right cameras of the binocular PTZ camera respectively take target images at respective positions in the spherical cavity by using various tilt angles and azimuth angles, and an angle-image encoder is obtained, wherein the angle comprises the tilt angle and the azimuth angle;

[0008] S2, the target is shot by using the left and right cameras, the corresponding azimuth angle and tilt angle of the image pair shot by the left and right cameras are found in the angle-image encoder according to the shot image, and the included angle alpha0 and beta0 of the left and right cameras are obtained according to the found azimuth angle and tilt angle and in combination with the positional geometric relationship of the left and right cameras and the target, L O represents the rotation center position point of the left camera, R P0 represents the target point, and alpha0 represents the straight line P0O L and the straight line O L O Rangle, β0 represents the included angle of straight line P0O R and straight line O L O R ;

[0009] S3, according to the included angles α0 and β0 of the left and right cameras, the distance D between the rotation center positions of the left and right cameras, and the three-dimensional distance d0 of the target point is solved:

[0010]

[0011] As a preferred, S1 comprises:

[0012] S11, an initial pitch angle θ0 and an initial azimuth angle φ0 are selected;

[0013] S12, the pitch angle is fixed, the camera is controlled to change the azimuth angle in turn, the target is photographed by the camera, the shooting area between adjacent images has overlap, and after one rotation, an image sequence is obtained:

[0014] (p0, p1, p2…p n , p n+1 , …, p n+k )

[0015] S13, the image p n with the highest similarity to the image photographed at the initial angle in the image sequence is found, the end position image is determined, and the images of (p n+1 , …, p n+k ) are eliminated;

[0016] S14, the overlapping parts of each adjacent image in the image sequence (p0, p1, p2…p n ) are calculated, the overlapping part of the previous image in the adjacent images is covered by the next image, after covering, the previous image only includes the remaining image after covering; the total width of the n+1 images after covering is w pixels, and the azimuth angle of the image p0 and the image p n is 2π, the azimuth angle of a unit pixel is 2π / w, the azimuth angle corresponding to the center of each image in the image sequence after covering is calculated, and n+1 azimuth angles are obtained;

[0017] S15, the pitch angle is changed, and S12 to S14 are repeated to obtain an angle-image encoder composed of images corresponding to different pitch angles and different azimuth angles.

[0018] As a preferred, in S12, the proportion of the pixels of the overlapping part to the total width of the image is not less than 1 / 4.

[0019] As a preferred, in S2, according to the photographed images, the method for finding the azimuth angle and the pitch angle corresponding to the images photographed by the left and right cameras in the angle-image encoder comprises:

[0020] Based on the images captured by the left and right cameras, find the image with the highest similarity to the image captured by the left and right cameras in the angle-image encoder. Using this image as the center, select M images and normalize the selected M images to obtain the pitch angle and azimuth angle corresponding to the image captured by the left and right cameras.

[0021] The pitch and azimuth angles corresponding to the images captured by the left camera are: The pitch and azimuth angles corresponding to the images captured by the right camera are:

[0022] Preferably, based on the background information in the captured image, the currently captured image is matched with the image in the angle-image encoder. The number of matching feature points is used as the similarity score to find the image with the highest similarity to the image captured by the left / right camera.

[0023] As a preferred method, the normalization method is:

[0024]

[0025] w i This represents the weight corresponding to the selected image i.

[0026] Preferably, in S2, the method for obtaining the camera angles α0 and β0 based on the found azimuth and elevation angles, combined with the positional geometric relationship between the left and right cameras and the target, includes:

[0027] Based on camera pose and Using geometric relationships, solve for the rotation center O of the left camera. L Unit vector pointing to target point P0 And the right camera rotation center O R Unit vector pointing to target point P0

[0028] Using vectors and Translation vectors relative to the rotation centers of the left and right cameras The vector relationships are used to calculate the included angles α0 and β0.

[0029] The beneficial effect of this invention is that it can still perform distance measurement when the center viewpoint coincides, even when the internal and external parameters change after focusing and shooting angle and range adjustment, without self-calibration. Attached Figure Description

[0030] Figure 1 Schematic diagram of PTZ camera motion model

[0031] Figure 2Rotate the camera to take a picture;

[0032] Figure 3 For the initial image sequence;

[0033] Figure 4 For the overlapping part of the adjacent image;

[0034] Figure 5 For the image covering the overlapping part;

[0035] Figure 6 For the correspondence between the azimuth angle and the image;

[0036] Figure 7 For the principle of changing the pitch angle θ;

[0037] Figure 8 For the image-angle encoding matrix diagram;

[0038] Figure 9 For the working environment diagram;

[0039] Figure 10 For the image feature matching diagram;

[0040] Figure 11 For the image selection diagram;

[0041] Figure 12 For the camera pose relationship diagram;

[0042] Figure 13 For the distance solution relationship diagram. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0044] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0045] The application will be further described below with reference to the drawings and specific embodiments, but not as a limitation of the application.

[0046] The ranging method of the binocular PTZ camera center viewpoint coincidence of the embodiment includes:

[0047] Step 1, establish a PTZ camera stereoscopic spherical space angle-image encoder:

[0048] The azimuth rotation axis and the pitch rotation axis of the PTZ camera intersect at a point, which is called the rotation center. The motion model of the PTZ camera is shown in Figure 1 .

[0049] A rectangular coordinate system o-xyz with the camera rotation center as the coordinate origin is established. By changing the azimuth angle and the pitch angle, each azimuth image can be collected. The left and right cameras of the binocular PTZ camera in the embodiment capture target images at various pitch angles and azimuth angles at their respective positions in the spherical cavity, to obtain an angle-image encoder. The angle includes the pitch angle and the azimuth angle.

[0050] Step 2, the target is captured by the left and right cameras. According to the captured images, the corresponding azimuth angle and pitch angle of the images captured by the left and right cameras are found in the angle-image encoder. According to the found azimuth angle and pitch angle, the camera included angle α0, β0 is obtained in combination with the geometric relationship between the positions of the left and right cameras and the target, as shown in Figure 12 and Figure 13 . L O R represents the rotation center position point of the right camera, P0 represents the target point, α0 represents the included angle of the straight line P0O L and the straight line O L O R , and β0 represents the included angle of the straight line P0O R and the straight line O L O R .

[0051] Step 3, according to the camera included angle α0, β0, and the distance D between the rotation center position points of the left and right cameras, the three-dimensional distance d0 of the target point is solved.

[0052]

[0053] By establishing the angle-image encoder, the azimuth angle and the pitch angle when the target image is captured can be determined by using the angle-image encoder under the condition that the internal and external parameters change after the focusing and the shooting angle and range are adjusted. Without self-calibration, the distance can still be measured when the center view point coincides.

[0054] In the preferred embodiment, step 1 in the embodiment includes:

[0055] Step 11, an initial pitch angle θ0 and an initial azimuth angle φ0 are selected.

[0056] Step 12, the pitch angle is fixed, and the camera is controlled to change the azimuth angle in turn, that is, the camera is horizontally rotated, to capture a target image of one latitude, as shown in Figure 2As shown, each rotation of the horizontal angle corresponds to one image, and the adjacent images have an overlap, and the proportion of the overlap to the pixel width of the image is not less than 1 / 4. After one rotation of the camera, a sequence of images (p0, p1, p2…p n , p n+1 , …, p n+k ) is obtained, as shown in FIG. 2. Figure 3

[0057] Step 13, find the image p n with the highest similarity to the initial image in the image sequence, determine the end position image, and eliminate the images (p n+1 , …, p n+k );

[0058] Step 14, after the camera rotates about 2π, the image is taken again, and the starting scene is taken again. In order to obtain the images taken at the beginning and the end that are close to each other, the end position image is determined by finding the image with the highest similarity to the initial image. Assuming that the image p n has the highest similarity to the initial image p0, then the image p n is the corresponding image after the camera rotates about 2π, and the images (p n+1 , …, p n+k ) are eliminated. Each image in the remaining image sequence (p0, p1, p2…p n ) corresponds to a unique longitude, and the overlapping part of each adjacent image in the image sequence (p0, p1, p2…p n ) is calculated. As shown in the figure, only the overlapping part of the image p0 and the image p1 is shown. The overlapping part of the previous image in the adjacent images is covered by the next image. After covering, the previous image only includes the remaining image after covering. The total width of the n+1 images after covering is w pixels, and the azimuth angle of the image p0 and the image p n is different by 2π, as shown in FIG. 21. Figure 5 The azimuth angle of a unit pixel is 2π / w. The azimuth angle corresponding to the center of each image in the covered image sequence is calculated, and the azimuth angle sequence is obtained, as shown in FIG. 22. Figure 6

[0059] Step 15, change the pitch angle, i.e., change the latitude of the taken image, repeat steps 12 to 14, and the image sequence (p0, p1, p2…p n ) and the azimuth angle sequence corresponding to different pitch angles θ and azimuth angles are obtained.

[0060] ​​Both left and right cameras use the above method to establish the angle-image encoder;

[0061] Step 1 of the embodiment removes redundant points after the image taken in one rotation is obtained, obtains the actual pixel number between the starting point of shooting and the starting point of shooting again, re-determines the azimuth angle after the redundancy is removed, completes self-checking, and ensures the accuracy of the angle.

[0062] In the preferred embodiment, step 2 of the embodiment includes:

[0063] The working environment of the binocular PTZ camera is inside a sphere, and the translation vector of the rotation center of the left and right cameras can be obtained through initial calibration and remains unchanged during camera movement, as shown in Figure 9 .

[0064] When the left and right cameras observe the target point at the same time, the camera pose is constantly controlled to adjust the position so that the target point is at the center of the binocular camera, accurate positioning is achieved, and the photos taken by the left and right cameras are returned.

[0065] Step 21, according to the images taken by the left camera / right camera, find the image with the highest similarity in the angle-image encoder respectively, frame M images around the image as the center, normalize the M framed images, and obtain the pitch angle and azimuth angle corresponding to the images taken by the left camera / right camera;

[0066] The pitch angle and azimuth angle corresponding to the image taken by the left camera are The pitch angle and azimuth angle corresponding to the image taken by the right camera are

[0067] In the embodiment, the image taken at present can be matched with the images in the image-angle encoding matrix according to the background information of the taken image, and the number of feature point matches is used as the similarity, as shown in Figure 10 ;

[0068] Frame 9 images around the image with the highest similarity as the center, as shown in Figure 11 ;

[0069] Normalize the 9 images according to the similarity, and perform weighted calculation on the pose of the input image according to the weight w i and the known pose of the framed image . Weighted calculation is performed on the pitch angle and azimuth angle corresponding to the images taken by the left and right cameras. Taking the left camera as an example:

[0070]

[0071] w i This represents the weight corresponding to the selected image i.

[0072] The current pose of the camera can be determined by using the same method for both the left and right cameras. and

[0073] Based on camera pose and Using geometric relationships, solve for the rotation center O of the left camera. L Unit vector pointing to target point P0 And the right camera rotation center O R Unit vector pointing to target point P0 like Figure 12 As shown;

[0074] Taking the left camera as an example, based on the pose of the left camera... It can be found that the relationship with o L -x L y L z L The angle α between the three coordinate axes of the coordinate system l ,β l γ l

[0075]

[0076]

[0077] cosγ l =sinθ l

[0078] In o L -x L y L z L In coordinate system, let the unit vector of the target point direction be... According to o L -x L y L z L The transformation relationship between the coordinate system and the spherical cavity coordinate system O-XYZ is derived. unit vector in spherical coordinate system

[0079] Similarly,

[0080] Using vectors and Translation vectors relative to the rotation centers of the left and right cameras The vector relationships are used to calculate the included angles α0 and β0.

[0081] Step 3 of the present embodiment:

[0082] During the continuous movement of the binocular PTZ camera, the operations such as zooming and focusing are involved to make the camera image center coincide with the target point more accurately, and the camera intrinsic parameters change, but the three-dimensional information of the target point can be solved according to the camera angles a0 and b0, the calibration of the camera intrinsic parameters is bypassed, the plane triangle relationship formed by the rotation centers of the left and right cameras and the target point is used to directly solve the three-dimensional distance. The three-dimensional solving model is shown in Figure 13 .

[0083] The distance d0 of the target point P0 to the straight line O L O R can be calculated according to the camera angles a0, b0 and the distance D, and the equation

[0084]

[0085] where D1 and D2 are the distances from the rotation centers of the left and right cameras to the foot point, and the translation vector of the rotation centers of the left and right cameras is The distance The three-dimensional distance d0 of the target point can be solved

[0086] Although the present application is described herein with reference to particular embodiments, it is to be understood that these examples are merely illustrative of the principles and applications of the present application. It is therefore to be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the present application as defined by the appended claims. It is to be understood that different combinations of the features described herein can be provided by different embodiments other than the original claims. It is also to be understood that features described in relation to one embodiment can be used in other embodiments.

Claims

1. A ranging method based on the coincidence of the central viewpoints of a binocular PTZ camera, characterized in that, The method includes: S1. The left and right cameras of the binocular PTZ camera capture target images at their respective positions within the spherical cavity using various pitch and azimuth angles to obtain an angle-image encoder, wherein the angles include pitch and azimuth angles. S2. Use the left and right cameras to photograph the target. Based on the photographed images, find the azimuth and pitch angles corresponding to the images taken by the left and right cameras in the angle-image encoder. Based on the found azimuth and pitch angles, and combined with the geometric relationship between the positions of the left and right cameras and the target, obtain the included angle between the left and right cameras. , , Indicates the position of the rotation center of the left camera. Indicates the position of the rotation center of the right camera. Indicates the target point. Represents a straight line With a straight line The included angle, Represents a straight line With a straight line The included angle; S3, based on the angle between the left and right cameras , Spacing between the center points of left and right camera rotation Solve for the three-dimensional distance of the target point. : ； S1 includes: S11. Select an initial pitch angle. and initial azimuth angle ; S12. With the pitch angle fixed, control the camera to change the azimuth angle sequentially, and use the camera to capture images of the target. The captured areas overlap between adjacent images. After rotating one revolution, an image sequence is obtained: （ ） S13. Find the image in the image sequence that has the highest similarity to the image taken from the initial angle. Determine the endpoint location image and remove [images]. (image) S14, Calculate the image sequence ( In the n+1 images, the overlapping portion of each adjacent image is covered by the next image. After covering, the previous image only includes the image remaining after being covered. The total width of the image after covering with n+1 images is... Pixels, and image With images azimuth phase difference The azimuth angle of a unit pixel is Calculate the azimuth angle corresponding to the center of each image in the covered image sequence to obtain n+1 azimuth angles; S15. Change the pitch angle and repeat S12 to S14 to obtain the image composition angle-image encoder corresponding to different pitch angles and different azimuth angles. In step S2, the method for finding the azimuth and pitch angles corresponding to the images captured by the left and right cameras in the angle-image encoder based on the captured images includes: Based on the images captured by the left and right cameras, find the image with the highest similarity to the image captured by the left and right cameras in the angle-image encoder. Using this image as the center, select M images and normalize the selected M images to obtain the pitch angle and azimuth angle corresponding to the image captured by the left and right cameras. The pitch and azimuth angles corresponding to the image captured by the left camera are ( ). , The pitch and azimuth angles corresponding to the image captured by the right camera are ( ). , ); In step S2, based on the found azimuth and elevation angles, and combined with the geometric relationship between the positions of the left and right cameras and the target, the camera included angle is obtained. , The methods include: Based on camera pose ( , )and( , Using geometric relationships, the rotation center of the left camera is calculated. Point to target point unit vector and the right camera rotation center Point to target point unit vector ; Using vectors and Translation vectors relative to the rotation centers of the left and right cameras The vector relationships, and the angles calculated respectively. and .

2. The ranging method based on the coincidence of the center viewpoints of a binocular PTZ camera according to claim 1, characterized in that, In S12, the pixels of the overlapping part account for no less than 1 / 4 of the total width of the image.

3. The ranging method based on the coincidence of the center viewpoints of a binocular PTZ camera according to claim 1, characterized in that, Based on the background information in the captured image, the currently captured image is matched with the image in the angle-image encoder. The similarity is calculated based on the number of matching feature points, and the image with the highest similarity to the image captured by the left / right camera is found.

4. The ranging method based on the coincidence of the center viewpoints of a binocular PTZ camera according to claim 1, characterized in that, The normalization method is as follows: Indicates the selection box. The weights corresponding to the images.

5. The ranging method for coinciding center viewpoints of a binocular PTZ camera according to claim 4, characterized in that, In S2, M equals 9.

6. A computer-readable storage device storing a computer program, characterized in that, When the computer program is executed, it implements the ranging method for coinciding the central viewpoints of the binocular PTZ camera as described in any one of claims 1 to 5.

7. A ranging device for coinciding center viewpoints of a binocular PTZ camera, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that, The processor executes the computer program to implement the ranging method for coinciding the central viewpoints of the binocular PTZ camera as described in any one of claims 1 to 5.

Images