Joint calibration method, apparatus, and computer equipment based on cameras with different fields of view

Abstract

This application relates to a joint calibration method, apparatus, computer device, storage medium, and computer program product based on cameras with different fields of view. The method includes: determining the coordinate system type corresponding to the platform, and establishing an associated coordinate system corresponding to the platform based on the coordinate system type; calibrating the platform camera that captures images of the platform based on the associated coordinate system to obtain a first matrix corresponding to the platform camera; determining the pixel coordinates of calibration points based on calibration images acquired by a non-platform camera; calculating a second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration points in the associated coordinate system; the first matrix and the second matrix are used to move the platform to a target position that aligns with the non-platform camera. This method can improve the accuracy of platform-non-platform alignment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a joint calibration method, apparatus, computer device, storage medium, and computer program product based on cameras with different fields of view. Background Technology

[0002] With the development of computer technology, machine vision can be used to control the movement of a platform, allowing objects transported on the platform to conform to objects transported outside the platform. Specifically, corresponding cameras are deployed for both the platform and the non-platform, and then the cameras are visually calibrated. After camera calibration, the platform's position coordinates in three-dimensional space can be determined based on the images captured by the cameras, and the platform's movement can be controlled using these coordinates. In traditional techniques, if the fields of view of the cameras capturing images of the platform and the non-platform differ significantly, it can lead to large errors in camera calibration, making it impossible to accurately control the platform's movement based on the captured images, resulting in low accuracy in object conformation. Summary of the Invention

[0003] Therefore, it is necessary to provide a joint calibration method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on cameras with different fields of view to address the above-mentioned technical problems, which can improve the fitting accuracy between objects on the platform and objects on the non-platform.

[0004] Firstly, this application provides a joint calibration method based on cameras with different fields of view. The method includes:

[0005] Determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0006] Based on the associated coordinate system, the platform camera that captures the platform is calibrated to obtain the first matrix corresponding to the platform camera;

[0007] The pixel coordinates of the calibration points are determined based on the calibration images acquired by a non-platform camera.

[0008] Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, a second matrix corresponding to the non-platform camera is calculated; the first matrix and the second matrix are used to move the platform to a target position that matches the non-platform corresponding to the non-platform camera.

[0009] Secondly, this application also provides a joint calibration device based on cameras with different fields of view. The device includes:

[0010] The determination module is used to determine the coordinate system type corresponding to the platform, and to establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0011] The calibration module is used to calibrate the platform camera that captures the platform based on the associated coordinate system, and obtain the first matrix corresponding to the platform camera;

[0012] The determining module is also used to determine the pixel coordinates of the calibration point based on the calibration image acquired by the non-platform camera;

[0013] The calculation module is used to calculate the second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system; the first matrix and the second matrix are used to move the platform to the target position that matches the non-platform corresponding to the non-platform camera.

[0014] In one embodiment, the apparatus further includes:

[0015] The determining module is further configured to determine the first pixel coordinates of the platform based on a first image captured by the platform camera on the platform, and to determine the second pixel coordinates of the non-platform based on a second image captured by the non-platform camera on the non-platform.

[0016] The calculation module is further configured to calculate the first pixel coordinates based on the first matrix to obtain the first position coordinates of the platform in the associated coordinate system, and to calculate the second pixel coordinates based on the second matrix to obtain the second position coordinates of the non-platform in the associated coordinate system;

[0017] The moving module is used to move the platform to a target position that is aligned with the non-platform according to the first position coordinates and the second position coordinates.

[0018] In one embodiment, the determining module is further configured to:

[0019] Control the platform to move in the target direction, and take pictures of the platform during the movement;

[0020] Determine the direction of motion of the platform in the captured image;

[0021] Based on the relationship between the direction of motion and the target direction, the coordinate system type corresponding to the platform is determined.

[0022] In one embodiment, the determining module is further configured to:

[0023] The calibration board is photographed using a non-platform camera to obtain calibration images;

[0024] Identify the rectangular edges of the rectangular blocks in the calibration image;

[0025] The intersection of the rectangle's sides is used as a calibration point, and the pixel coordinates of the intersection point in the calibration image are determined.

[0026] In one embodiment, the calibration module is further configured to:

[0027] Based on the associated coordinate system, determine the feature point position coordinates of the feature points on the platform;

[0028] The feature points on the platform are captured by the platform camera to obtain feature point images;

[0029] In the feature point image, determine the pixel coordinates of the feature points;

[0030] The first matrix corresponding to the platform camera is obtained by calculating based on the pixel coordinates of the feature points and the position coordinates of the feature points.

[0031] In one embodiment, the field of view difference between the platform camera and the non-platform camera is greater than a preset value.

[0032] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0033] Determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0034] Based on the associated coordinate system, the platform camera that captures the platform is calibrated to obtain the first matrix corresponding to the platform camera;

[0035] The pixel coordinates of the calibration points are determined based on the calibration images acquired by a non-platform camera.

[0036] Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, a second matrix corresponding to the non-platform camera is calculated; the first matrix and the second matrix are used to move the platform to a target position that matches the non-platform corresponding to the non-platform camera.

[0037] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0038] Determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0039] Based on the associated coordinate system, the platform camera that captures the platform is calibrated to obtain the first matrix corresponding to the platform camera;

[0040] The pixel coordinates of the calibration points are determined based on the calibration images acquired by a non-platform camera.

[0041] Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, a second matrix corresponding to the non-platform camera is calculated; the first matrix and the second matrix are used to move the platform to a target position that matches the non-platform corresponding to the non-platform camera.

[0042] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0043] Determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0044] Based on the associated coordinate system, the platform camera that captures the platform is calibrated to obtain the first matrix corresponding to the platform camera;

[0045] The pixel coordinates of the calibration points are determined based on the calibration images acquired by a non-platform camera.

[0046] Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, a second matrix corresponding to the non-platform camera is calculated; the first matrix and the second matrix are used to move the platform to a target position that matches the non-platform corresponding to the non-platform camera.

[0047] The aforementioned joint calibration method, apparatus, computer equipment, storage medium, and computer program products based on cameras with different fields of view determine the coordinate system type corresponding to the platform and establish an associated coordinate system corresponding to the platform based on the coordinate system type. Based on the associated coordinate system, the platform camera of the shooting platform is calibrated to obtain the first matrix corresponding to the platform camera. Then, the coordinate system of the non-platform is associated with the associated coordinate system, and the non-platform camera is calibrated based on the associated coordinate system. Specifically, based on the calibration image acquired by the non-platform camera, the pixel coordinates of the calibration point are determined. Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, the second matrix corresponding to the non-platform camera is calculated. The first and second matrices are used to move the platform to the target position corresponding to the non-platform camera. Since the first and second matrices are determined based on the associated coordinate system, the pixel coordinates of both the platform and non-platform can be converted into position coordinates in the associated coordinate system. Even when the fields of view of the platform camera and the non-platform camera differ significantly, the positions of the platform and non-platform can be accurately determined in the associated coordinate system, thereby enabling precise alignment of objects on the platform and non-platform, improving the alignment accuracy. Attached Figure Description

[0048] Figure 1 This is an application environment diagram of a joint calibration method based on cameras with different fields of view in one embodiment;

[0049] Figure 2 This is a flowchart illustrating a joint calibration method based on cameras with different fields of view in one embodiment;

[0050] Figure 3 This is a schematic diagram of various types of coordinate systems in one embodiment;

[0051] Figure 4 This is a schematic diagram of a calibration plate in one embodiment;

[0052] Figure 5 This is a schematic diagram of pixel coordinates in one embodiment;

[0053] Figure 6 This is a flowchart illustrating a mobile platform method in one embodiment;

[0054] Figure 7 This is a flowchart illustrating a method for determining the pixel coordinates of a calibration point in one embodiment;

[0055] Figure 8 This is a schematic diagram of the layout of a non-platform camera in one embodiment;

[0056] Figure 9 This is a schematic diagram of an image of a calibration board captured by a non-platform camera in one embodiment;

[0057] Figure 10 This is a schematic diagram showing the coordinates of the calibration points on the calibration plate in the associated coordinate system in one embodiment;

[0058] Figure 11 This is a flowchart illustrating a joint calibration method based on cameras with different fields of view in another embodiment;

[0059] Figure 12 This is a structural block diagram of a joint calibration device based on cameras with different fields of view in one embodiment;

[0060] Figure 13 This is a structural block diagram of a joint calibration device based on cameras with different fields of view in another embodiment;

[0061] Figure 14 This is an internal structural diagram of a computer device in one embodiment;

[0062] Figure 15 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0065] The joint calibration method based on cameras with different fields of view provided in this application can be applied to, for example... Figure 1In the application environment shown, computer device 102 communicates with platform 104 and non-platform 106 via a network, and controls the movement of platform 104 and non-platform 106 through communication. A data storage system can store the data that computer device 102 needs to process. The data storage system can be integrated into computer device 102 or placed in the cloud or on other network servers. Computer device 102 determines the coordinate system type corresponding to the platform and establishes an associated coordinate system corresponding to the platform based on the coordinate system type; based on the associated coordinate system, it calibrates the platform camera of the shooting platform to obtain the first matrix corresponding to the platform camera; based on the calibration image acquired by the non-platform camera, it determines the pixel coordinates of the calibration point; based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, it calculates the second matrix corresponding to the non-platform camera; the first matrix and the second matrix are used to move the platform to a target position that aligns with the non-platform corresponding to the non-platform camera. Computer device 102 can be a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0066] In one embodiment, such as Figure 2 As shown, a joint calibration method based on cameras with different fields of view is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0067] S202, determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type.

[0068] The platform is a mechanical component in automated equipment. For example, it can be a three-axis component with X, Y, and θ axes, capable of parallel or vertical movement and rotation. Automated equipment can be, for example, bonding equipment, assembly equipment, or cleaning equipment. For instance, a platform can be a three-axis component in polarizer bonding equipment used to support LCDs (Liquid Crystal Displays). Another example is a three-axis component in assembly equipment used to support components to be assembled. Yet another example is a three-axis component in cleaning equipment used to support objects to be cleaned.

[0069] The coordinate system type is a classification of the coordinate system, which can be a rectangular coordinate system, a planar polar coordinate system, or a cylindrical coordinate system, etc. For example, Figure 3As shown, the coordinate system type can include Type 1, Type 2, Type 3, or Type 4. For Type 1 coordinate system, the positive X-axis points to the right, the positive Y-axis points downwards, and the positive θ-axis is clockwise. For Type 2 coordinate system, the positive X-axis points to the left, the positive Y-axis points downwards, and the positive θ-axis is counterclockwise. For Type 3 coordinate system, the positive X-axis points to the right, the positive Y-axis points downwards, and the positive θ-axis is counterclockwise. For Type 4 coordinate system, the positive X-axis points to the left, the positive Y-axis points downwards, and the positive θ-axis is clockwise.

[0070] The associated coordinate system is a coordinate system used to represent three-dimensional spatial position coordinates, which can associate the spatial position coordinates of the platform with those of non-platforms. In one embodiment, the associated coordinate system can be any absolute coordinate system. For example, the associated coordinate system can be the world coordinate system, or it can be the mechanical coordinate system corresponding to the platform. The computer device establishes the associated coordinate system corresponding to the platform according to the coordinate system type, and the rotation directions of the X-axis, Y-axis, and θ of the established associated coordinate system are consistent with the rotation directions of the X-axis, Y-axis, and θ corresponding to the coordinate system type.

[0071] S204. Based on the associated coordinate system, the platform camera of the shooting platform is calibrated to obtain the first matrix corresponding to the platform camera.

[0072] The platform camera is used to acquire images of the platform. It can be deployed perpendicular to the platform and may include monochrome or color cameras. To capture clear images of the platform, the field of view of the platform camera is matched to the size of the platform. Specifically, by adjusting the field of view of the platform camera, images of the platform or feature points on the platform can be clearly captured. The first matrix is ​​used to convert the pixel coordinates of the images captured by the platform camera into spatial position coordinates.

[0073] In one embodiment, the computer device can calibrate the platform camera using a nine-point calibration method to obtain the first matrix corresponding to the platform camera. The nine-point calibration method is a method for determining the transformation relationship between pixel coordinates and spatial position coordinates by using nine sets of corresponding pixel coordinates and spatial position coordinates.

[0074] S206 determines the pixel coordinates of the calibration points based on calibration images acquired by a non-platform camera.

[0075] The non-platform camera is used to acquire images from a non-platform environment. It can be mounted perpendicular to the non-platform and includes monochrome or color cameras. The field of view of the non-platform camera is matched to the size of the non-platform. Specifically, by adjusting the field of view of the non-platform camera, images of the non-platform can be clearly captured. The field of view of the platform camera and the non-platform camera can be the same or different. In one embodiment, the difference in field of view between the platform camera and the non-platform camera is greater than a preset value. Specifically, the difference in field of view between the horizontal field of view of the platform camera and the non-platform camera is greater than a preset value; or the difference in field of view between the vertical field of view of the platform camera and the non-platform camera is greater than a preset value; or the differences in both the horizontal and vertical field of view of the platform camera and the non-platform camera are greater than preset values. For example, the difference in both the horizontal and vertical field of view between the platform camera and the non-platform camera is more than 5 times.

[0076] The calibration image is an image containing calibration points. In one embodiment, the calibration image can be an image acquired from a calibration board. The calibration board is a flat plate containing a pattern array, wherein the pattern array can be a rectangular pattern array, a circular pattern array, or other pattern array. For example, ... Figure 4 As shown, the calibration board is a flat plate containing an array of black and white rectangular blocks. Calibration points are points used for calibrating non-platform cameras and can be points on the calibration board. For example, such as... Figure 4 As shown, the calibration point is the intersection of the rectangular sides of the rectangular block on the calibration plate.

[0077] Here, pixel coordinates are the coordinates in the image, expressed in pixels. For example, as... Figure 5 As shown, pixel coordinates are the coordinates of a pixel in the image, with the top-left corner as the origin, and expressed in pixels. For example, the pixel coordinates of a calibration point could be (300, 430), indicating that the calibration point is located at the 300th pixel horizontally and the 430th pixel vertically in the calibration image.

[0078] S208, calculate the second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system; the first matrix and the second matrix are used to move the platform to the target position that matches the non-platform camera.

[0079] The second matrix is ​​used to convert the pixel coordinates of the image captured by the non-platform camera into spatial position coordinates. The computer equipment calculates the second matrix corresponding to the non-platform camera based on the pixel coordinates of the calibration point in the calibration image and the position coordinates of the calibration point in the associated coordinate system. The non-platform is a mechanical component in automated equipment; it can be a component with X and Y axes that can move parallel or horizontally, but lacks a θ axis and cannot rotate. For example, the non-platform can be a component in a polarizer attachment device used to support the POL (polarizer).

[0080] In the above embodiments, the coordinate system type corresponding to the platform is determined, and an associated coordinate system corresponding to the platform is established based on the coordinate system type. Based on the associated coordinate system, the platform camera of the shooting platform is calibrated to obtain the first matrix corresponding to the platform camera. Then, the coordinate system of the non-platform is associated with the associated coordinate system, and the non-platform camera is calibrated based on the associated coordinate system. Specifically, based on the calibration image acquired by the non-platform camera, the pixel coordinates of the calibration point are determined. Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, the second matrix corresponding to the non-platform camera is calculated. The first and second matrices are used to move the platform to the target position that matches the non-platform corresponding to the non-platform camera. Since the first and second matrices are determined based on the associated coordinate system, the pixel coordinates of both the platform and non-platform can be converted into position coordinates in the associated coordinate system. Even when the field of view of the platform camera and the non-platform camera differs significantly, the positions of the platform and non-platform can be accurately determined in the associated coordinate system, thereby enabling precise alignment of objects on the platform and non-platform, improving the alignment accuracy.

[0081] In one embodiment, such as Figure 6 As shown, the following steps are included after S208:

[0082] S602, based on a first image of the platform captured by a platform camera, determines the first pixel coordinates of the platform, and based on a second image of the non-platform captured by a non-platform camera, determines the second pixel coordinates of the non-platform.

[0083] Wherein, the first pixel coordinates are the coordinates of the platform in the first image, which can be the coordinates of the target point marked on the platform in the first image. The second pixel coordinates are the coordinates of the non-platform in the second image, which can be the coordinates of the target point marked on the non-platform in the second image.

[0084] In one embodiment, S602 specifically includes: the computer device performing image recognition on the first image to obtain the first pixel coordinates, and performing image recognition on the second image to obtain the second pixel coordinates.

[0085] S604, calculate the first pixel coordinates based on the first matrix to obtain the first position coordinates of the platform in the associated coordinate system, and calculate the second pixel coordinates based on the second matrix to obtain the second position coordinates of the non-platform in the associated coordinate system.

[0086] Wherein, the first position coordinates are the three-dimensional spatial coordinates corresponding to the first pixel coordinates. The second position coordinates are the three-dimensional spatial coordinates corresponding to the second pixel coordinates. In one embodiment, S604 specifically includes: the computer device multiplying the first matrix by the first pixel coordinates to obtain the first position coordinates; and multiplying the second matrix by the second pixel coordinates to obtain the second position coordinates.

[0087] S606, based on the first position coordinates and the second position coordinates, moves the platform to the target position that is in contact with the non-platform.

[0088] The target position is the location where the platform and the non-platform are bonded. For example, the platform is the component that supports the LCD in a polarizer bonding machine, the non-platform is the device that supports the POL in the polarizer bonding machine, and the target position is the location where the POL is bonded to the LCD.

[0089] Since both the first and second position coordinates are coordinates in the associated coordinate system, the distance and direction between the platform and the non-platform can be determined based on the first and second position coordinates. By controlling the platform to move according to the determined distance and direction, the platform can be moved to the target position.

[0090] When there is a significant difference in the field of view between the platform camera and the non-platform camera, since both cameras acquire images at the same resolution, the image captured by the camera with the larger field of view is unclear. Therefore, it is impossible to accurately determine the positions of the platform and non-platform based on the pixel coordinates of the two images, resulting in low alignment accuracy between the platform and non-platform. In the above embodiment, the first pixel coordinates corresponding to the platform and the second pixel coordinates corresponding to the non-platform are both converted to coordinates in an associated coordinate system. Then, the platform movement is controlled by using these coordinates in the associated coordinate system. This allows for accurate determination of the platform and non-platform position coordinates in the associated coordinate system, improving the alignment accuracy between the platform and non-platform.

[0091] In one embodiment, S202 specifically includes: controlling the platform to move in the target direction and taking pictures of the platform during the movement; determining the direction of movement of the platform in the captured image; and determining the coordinate system type corresponding to the platform based on the relationship between the direction of movement and the target direction.

[0092] The target direction can be the direction of a coordinate axis in the platform's mechanical coordinate system, and can be either the positive or negative direction of the coordinate axis. The mechanical coordinate system is set at the factory, with a specific point on the platform as the origin. For example, the target direction could be the positive direction of the X-axis in the platform's mechanical coordinate system. Another example is the positive direction of the Y-axis in the platform's mechanical coordinate system.

[0093] Specifically, the computer equipment can first use the platform's JOG (slow motion) function to move the platform in the positive X-axis direction, and then take pictures of the platform during this movement to determine the platform's direction of motion in the captured images. For example, as Figure 3 As shown, if the platform's motion direction in the captured image is consistent with the positive X-axis direction, the platform's coordinate system type can be Type 1 or Type 3; if the platform's motion direction in the captured image is opposite to the positive X-axis direction, the platform's coordinate system type can be Type 2 or Type 4. Then, the computer device can move the platform clockwise along the positive θ-axis. If the platform's motion direction in the captured image is clockwise, the platform's coordinate system type can be Type 1 or Type 4; if the platform's motion direction in the captured image is counterclockwise, the platform's coordinate system type can be Type 2 or Type 3. Similarly, when the platform moves in the positive Y-axis direction, the platform's coordinate system type can also be determined based on the platform's motion direction in the captured image.

[0094] In the above embodiments, the coordinate system type corresponding to the platform is determined, and an associated coordinate system is established based on this coordinate system type. Then, the spatial coordinates of both the platform and non-platform can be associated with the associated coordinate system. Even if the field of view of the platform camera and the non-platform camera differs greatly, the spatial position of the platform and non-platform can be accurately determined in the associated coordinate system. This allows for precise control of the platform's movement and improves the accuracy of the platform and non-platform alignment.

[0095] In one embodiment, such as Figure 7 As shown, S206 specifically includes the following steps:

[0096] The S702 uses a non-platform camera to photograph the calibration board and obtain calibration images.

[0097] Specifically, the non-platform camera takes a picture of the calibration plate placed in front of it, for example, such as Figure 8 As shown, the calibration board is placed in front of four non-platform cameras. Non-platform camera 1 takes a picture of the rectangular block in the upper left corner of the calibration board, non-platform camera 2 takes a picture of the rectangular block in the upper right corner of the calibration board, non-platform camera 3 takes a picture of the rectangular block in the lower right corner of the calibration board, and non-platform camera 4 takes a picture of the rectangular block in the lower right corner of the calibration board to obtain the calibration image.

[0098] S704 identifies the rectangular edges of rectangular blocks in a calibration image.

[0099] The calibration image obtained by a non-platform camera from the calibration board consists of black and white rectangular blocks. Computer equipment can identify the rectangular edges of the rectangular blocks using edge detection algorithms. For example, the computer equipment can identify the rectangular edges of the rectangular blocks using the Sobel algorithm.

[0100] S706, using the intersection of the rectangle's sides as the calibration point, and determining the pixel coordinates of the intersection point in the calibration image.

[0101] The computer device identifies the rectangular edges within the rectangular block, using the intersections of these edges as calibration points. For example, ... Figure 9 As shown, the intersection of rectangle 1 and rectangle 2 is point 1, the intersection of rectangle 2 and rectangle 3 is point 2, the intersection of rectangle 3 and rectangle 4 is point 3, and the intersection of rectangle 4 and rectangle 1 is point 4. Points 1 to 4 are used as calibration points, and the pixel coordinates of points 1 to 4 in the calibration image are determined.

[0102] In one embodiment, the computer device determines the coordinates of the intersection points of the rectangular sides of the rectangular block on the calibration plate in an associated coordinate system, that is, determines the coordinates of the calibration points in the associated coordinate system. For example, as... Figure 10 As shown, the rectangular blocks on the calibration plate are spaced 10 mm apart. The computer equipment can use the vertex of the upper left rectangular block on the calibration plate as the origin of the associated coordinate system. Table 1 shows the position coordinates of each calibration point on the calibration plate.

[0103] Table 1

[0104]

[0105]

[0106] In the above embodiments, a calibration image is obtained by capturing an image of the calibration board using a non-platform camera. The rectangular edges of the rectangular blocks in the calibration image are identified. The intersection points of the rectangular edges are used as calibration points, and the pixel coordinates of these intersection points in the calibration image are determined. A second matrix corresponding to the non-platform camera is calculated using the pixel coordinates and the position coordinates of the calibration points in the associated coordinate system. This second matrix allows the pixel coordinates in the non-platform camera's image of the non-platform to be converted into position coordinates in the associated coordinate system, thus accurately determining the non-platform's position within the associated coordinate system.

[0107] In one embodiment, S204 specifically includes: determining the feature point position coordinates of the feature points on the platform based on the associated coordinate system; capturing images of the feature points on the platform using the platform camera to obtain feature point images; determining the pixel coordinates of the feature points in the feature point images; and calculating the first matrix corresponding to the platform camera based on the pixel coordinates and the feature point position coordinates.

[0108] Feature points are points marked on the platform for platform calibration. After establishing the associated coordinate system, the computer device can acquire the position coordinates of the feature points on the platform within that system. Then, the platform camera captures images of the feature points, and image recognition methods can be used to obtain the pixel coordinates of these feature points from the resulting images.

[0109] In the above embodiments, the computer device calibrates the platform camera by selecting feature points on the platform to obtain a first matrix. This first matrix then converts the pixel coordinates of the image captured by the platform camera onto the platform into position coordinates in the associated coordinate system. Even when the field of view of the platform camera and the non-platform camera differs significantly, the platform's position can be accurately determined in the associated coordinate system, improving the accuracy of the platform-non-platform alignment.

[0110] In one embodiment, prior to S202, the method further includes adjusting the platform camera and the non-platform camera so that both cameras can clearly capture images of the subject. The adjustments to the platform camera and the non-platform camera include adjusting their focal length, position, and distance from the subject.

[0111] In one embodiment, such as Figure 11 As shown, the joint calibration method based on cameras with different fields of view includes the following steps:

[0112] S1102, control the platform to move in the direction of the target and take pictures of the platform during the movement.

[0113] S1104, determine the direction of motion of the platform in the captured image, and determine the coordinate system type corresponding to the platform based on the relationship between the direction of motion and the target direction.

[0114] S1106, Establish the associated coordinate system corresponding to the platform according to the coordinate system type.

[0115] S1108, based on the associated coordinate system, calibrate the platform camera of the shooting platform to obtain the first matrix corresponding to the platform camera.

[0116] S1110 uses a non-platform camera to capture images of the calibration board, obtains calibration images, and identifies the rectangular edges of the rectangular blocks in the calibration images.

[0117] S1112, using the intersection of the rectangle's sides as the calibration point, and determining the pixel coordinates of the intersection point in the calibration image.

[0118] S1114: Calculate the second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system.

[0119] S1116, based on a first image of the platform captured by a platform camera, determine the first pixel coordinates of the platform, and based on a second image of the non-platform captured by a non-platform camera, determine the second pixel coordinates of the non-platform.

[0120] S1118, calculate the first pixel coordinates based on the first matrix to obtain the first position coordinates of the platform in the associated coordinate system, and calculate the second pixel coordinates based on the second matrix to obtain the second position coordinates of the non-platform in the associated coordinate system.

[0121] S1120, based on the first position coordinates and the second position coordinates, move the platform to the target position that is in contact with the non-platform.

[0122] The specific content of S1102 to S1120 can be found in the implementation process described above.

[0123] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0124] Based on the same inventive concept, this application also provides a joint calibration device for cameras with different fields of view, used to implement the joint calibration method based on cameras with different fields of view described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the joint calibration device based on cameras with different fields of view provided below can be found in the limitations of the joint calibration method based on cameras with different fields of view described above, and will not be repeated here.

[0125] In one embodiment, such as Figure 12 As shown, a joint calibration device based on cameras with different fields of view is provided, including: a determination module 1202, a calibration module 1204, and a calculation module 1206, wherein:

[0126] The determination module 1202 is used to determine the coordinate system type corresponding to the platform and establish the associated coordinate system corresponding to the platform based on the coordinate system type;

[0127] The calibration module 1204 is used to calibrate the platform camera of the shooting platform based on the associated coordinate system to obtain the first matrix corresponding to the platform camera;

[0128] The determination module 1202 is also used to determine the pixel coordinates of the calibration points based on the calibration images acquired by the non-platform camera;

[0129] The calculation module 1206 is used to calculate the second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system; the first matrix and the second matrix are used to move the platform to the target position that matches the non-platform camera.

[0130] In the above embodiments, the coordinate system type corresponding to the platform is determined, and an associated coordinate system corresponding to the platform is established based on the coordinate system type. Based on the associated coordinate system, the platform camera of the shooting platform is calibrated to obtain the first matrix corresponding to the platform camera. Then, the coordinate system of the non-platform is associated with the associated coordinate system, and the non-platform camera is calibrated based on the associated coordinate system. Specifically, based on the calibration image acquired by the non-platform camera, the pixel coordinates of the calibration point are determined. Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, the second matrix corresponding to the non-platform camera is calculated. The first and second matrices are used to move the platform to the target position that matches the non-platform corresponding to the non-platform camera. Since the first and second matrices are determined based on the associated coordinate system, the pixel coordinates of both the platform and non-platform can be converted into position coordinates in the associated coordinate system. Even when the field of view of the platform camera and the non-platform camera differs significantly, the positions of the platform and non-platform can be accurately determined in the associated coordinate system, thereby enabling precise alignment of objects on the platform and non-platform, improving the alignment accuracy.

[0131] In one embodiment, such as Figure 13 As shown, the device also includes:

[0132] The determining module 1202 is further configured to determine the first pixel coordinates of the platform based on the first image obtained by capturing the platform through the platform camera, and to determine the second pixel coordinates of the non-platform based on the second image obtained by capturing the non-platform through the non-platform camera.

[0133] The calculation module 1206 is also used to calculate the first pixel coordinates based on the first matrix to obtain the first position coordinates of the platform in the associated coordinate system, and to calculate the second pixel coordinates based on the second matrix to obtain the second position coordinates of the non-platform in the associated coordinate system.

[0134] The moving module 1208 is used to move the platform to a target position that is in contact with the non-platform according to the first position coordinates and the second position coordinates.

[0135] In one embodiment, the determining module 1202 is further configured to:

[0136] The control platform moves towards the target direction, and the platform is photographed during the movement;

[0137] Determine the direction of motion of the platform in the captured image;

[0138] Based on the relationship between the direction of motion and the target direction, the coordinate system type corresponding to the platform is determined.

[0139] In one embodiment, the determining module 1202 is further configured to:

[0140] The calibration board is photographed using a non-platform camera to obtain calibration images;

[0141] Identify the rectangular edges of rectangular blocks in the calibration image;

[0142] The intersection of the rectangle's sides is used as the calibration point, and the pixel coordinates of the intersection point in the calibration image are determined.

[0143] In one embodiment, the calibration module 1204 is further configured to:

[0144] Based on the associated coordinate system, determine the feature point coordinates of the feature points on the platform;

[0145] Feature point images are obtained by capturing images of feature points on the platform using the platform's camera;

[0146] In the feature point image, determine the pixel coordinates of the feature points;

[0147] The first matrix corresponding to the platform camera is obtained by calculating the pixel coordinates and position coordinates of the feature points.

[0148] In one embodiment, the field of view difference between the platform camera and the non-platform camera is greater than a preset value.

[0149] The modules in the aforementioned joint calibration device based on cameras with different fields of view can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0150] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 14As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores joint calibration data based on cameras with different fields of view. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a joint calibration method based on cameras with different fields of view.

[0151] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 15 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a joint calibration method based on cameras with different fields of view. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0152] Those skilled in the art will understand that Figure 14 , 15The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A joint calibration method based on cameras with different fields of view, characterized in that, The method includes: Determine the coordinate system type corresponding to the platform, and establish the associated coordinate system corresponding to the platform based on the coordinate system type; Based on the associated coordinate system, the platform camera that captures the platform is calibrated to obtain the first matrix corresponding to the platform camera; The pixel coordinates of the calibration points are determined based on the calibration images acquired by a non-platform camera. Based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system, a second matrix corresponding to the non-platform camera is calculated; the first matrix and the second matrix are used to move the platform to a target position that matches the non-platform corresponding to the non-platform camera.

2. The method according to claim 1, characterized in that, The method further includes: The first pixel coordinates of the platform are determined based on a first image of the platform captured by the platform camera, and the second pixel coordinates of the non-platform are determined based on a second image of the non-platform captured by the non-platform camera. The first position coordinates of the platform in the associated coordinate system are obtained by calculating the first pixel coordinates based on the first matrix, and the second position coordinates of the non-platform in the associated coordinate system are obtained by calculating the second pixel coordinates based on the second matrix. The platform is moved to a target position that aligns with the non-platform based on the first and second position coordinates.

3. The method according to claim 1, characterized in that, The coordinate system type corresponding to the platform includes: Control the platform to move in the target direction, and take pictures of the platform during the movement; Determine the direction of motion of the platform in the captured image; Based on the relationship between the direction of motion and the target direction, the coordinate system type corresponding to the platform is determined.

4. The method according to claim 1, characterized in that, The process of determining the pixel coordinates of the calibration points based on calibration images acquired by a non-platform camera includes: The calibration board is photographed using a non-platform camera to obtain calibration images; Identify the rectangular edges of the rectangular blocks in the calibration image; The intersection of the rectangle's sides is used as a calibration point, and the pixel coordinates of the intersection point in the calibration image are determined.

5. The method according to claim 1, characterized in that, The calibration of the platform camera that captures images of the platform based on the associated coordinate system, to obtain the first matrix corresponding to the platform camera, includes: Based on the associated coordinate system, determine the feature point position coordinates of the feature points on the platform; The feature points on the platform are captured by the platform camera to obtain feature point images; In the feature point image, determine the pixel coordinates of the feature points; The first matrix corresponding to the platform camera is obtained by calculating based on the pixel coordinates of the feature points and the position coordinates of the feature points.

6. The method according to any one of claims 1 to 5, characterized in that, The difference in field of view between the platform camera and the non-platform camera is greater than a preset value.

7. A joint calibration device based on cameras with different fields of view, characterized in that, The device includes: The determination module is used to determine the coordinate system type corresponding to the platform, and to establish the associated coordinate system corresponding to the platform based on the coordinate system type; The calibration module is used to calibrate the platform camera that captures the platform based on the associated coordinate system, and obtain the first matrix corresponding to the platform camera; The determining module is also used to determine the pixel coordinates of the calibration point based on the calibration image acquired by the non-platform camera; The calculation module is used to calculate the second matrix corresponding to the non-platform camera based on the pixel coordinates and the position coordinates of the calibration point in the associated coordinate system; the first matrix and the second matrix are used to move the platform to the target position that matches the non-platform corresponding to the non-platform camera.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images