Visual measurement method, device and equipment for large-size equipment docking, and medium

Abstract

The application relates to the technical field of visual measurement, in particular to a visual measurement method, device and equipment for butt joint of large-size equipment and a medium. The method comprises the following steps: determining two target butt joint objects from a target large-size equipment; calibrating external parameters of a plurality of camera devices with pre-calibrated internal parameters; capturing butt joint surfaces of the two target butt joint objects by using the plurality of camera devices with calibrated external parameters to obtain a plurality of local images, wherein the plurality of local images correspond to a plurality of different areas of the butt joint surfaces; and measuring pose parameters of the two target butt joint objects based on imaging features of butt joint holes in the plurality of local images. The application realizes butt joint of the two target butt joint objects in the large-size equipment without a target, and the imaging features of the butt joint holes in the butt joint surface images are used to obtain the pose parameters of the target butt joint objects, so that the measurement error is small and the precision is high.

Description

Technical Field

[0001] This application relates to the field of visual measurement technology, and in particular to visual measurement methods, devices, equipment and media for the docking of large-size equipment. Background Technology

[0002] Visual measurement technology plays a crucial role in equipment docking, helping to ensure the quality, safety, and performance of equipment by providing high-precision measurement and real-time monitoring. For docking large targets, existing vision-based measurement methods offer advantages such as non-contact operation and high measurement accuracy.

[0003] Existing vision-based solutions require mounting targets on the object to be measured, and indirectly obtaining the position and orientation of the docking target by measuring the targets. However, there are several problems: first, high precision is required for target mounting; second, the use of targets is limited, and targets cannot be used in some scenarios; and third, errors are introduced during the conversion of measurement results, resulting in low measurement accuracy. Summary of the Invention

[0004] Based on the aforementioned technical problems, this application aims to provide a visual measurement method, apparatus, equipment, and medium for large-size equipment docking, so as to solve at least one of the aforementioned technical problems.

[0005] The first aspect of this application provides a visual measurement method for large-size equipment docking, the method comprising:

[0006] Identify two targets to be docked from the large-sized equipment;

[0007] Calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated;

[0008] Multiple cameras with calibrated external parameters are used to simultaneously capture images of the docking surfaces of the two targets to be docked, resulting in multiple local images, wherein the multiple local images correspond to multiple different regions of the docking surfaces;

[0009] Based on the imaging features of the docking holes in the multiple local images, the pose parameters of the two targets to be docked are measured.

[0010] In some embodiments of this application, measuring the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the plurality of local images includes:

[0011] Extract feature information from the multiple local images, and identify the docking holes of the two targets to be docked based on the feature information;

[0012] Using any one of the multiple camera devices calibrated with the external parameters as the measurement reference coordinate system, solve for the position parameters of the center of at least one docking hole within the field of view of the other camera devices;

[0013] By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole, the pose parameters of the two targets to be docked are measured to obtain the pose measurement values ​​of the two targets to be docked.

[0014] In some embodiments of this application, the step of extracting feature information from the plurality of local images and identifying the docking holes of the two targets to be docked based on the feature information includes:

[0015] For each local image, feature information of each local image is extracted based on region segmentation;

[0016] The shape, size, and location of the mating hole are determined from the feature information;

[0017] The docking holes of the two targets to be docked are identified by their shape, size, and position.

[0018] In some embodiments of this application, solving for the position parameters of at least one docking hole center within the field of view of other camera devices includes:

[0019] It was determined that at least one docking hole existed within the field of view of other camera devices, and the position parameters of the center of at least one docking hole were solved based on a monocular circular pose measurement algorithm.

[0020] In some embodiments of this application, the step of combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole to measure the pose parameters of the two targets to be docked, and obtaining the pose measurement values ​​of the two targets to be docked, includes:

[0021] Based on the external parameter calibration results, the position parameters of at least one docking hole center within the field of view of the other camera equipment are converted to the measurement reference coordinate system, and the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system are calculated.

[0022] By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, the coordinates of the original center imaging point of the docking hole within the reference field of view are obtained.

[0023] Based on the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system and the original center imaging point coordinates of the docking hole in the reference field of view, all docking hole combinations in the same preset coordinate system are obtained.

[0024] Solve the pose parameters for all the docking hole combinations to obtain the initial pose values ​​of the two targets to be docked;

[0025] The initial pose value is optimized to obtain the pose measurement values ​​of the two targets to be docked.

[0026] In some embodiments of this application, optimizing the initial pose value to obtain the pose measurement values ​​of the two targets to be docked includes:

[0027] Based on the external parameter calibration results, the initial pose value is converted to the camera coordinate system corresponding to the plurality of camera devices;

[0028] The projection curve of the spatial circle where the target docking hole is located and the projection point of the center of the docking hole are calculated in the camera coordinate system, and the projection curve and the projection point are compared with the actual detected feature points to obtain the difference.

[0029] The difference is used as the value of the objective function for pose iteration optimization. The initial pose value is adjusted until the value of the objective function reaches its minimum, and the pose measurement values ​​of the two targets to be docked are obtained.

[0030] In some embodiments of this application, calibrating the external parameters of multiple camera devices whose internal parameters have been pre-calibrated includes:

[0031] Two cameras from a set of pre-calibrated internal parameters are used as the cameras to be calibrated together.

[0032] A checkerboard calibration board for arranging dual-camera equipment;

[0033] Simultaneously, two devices from the calibration equipment are used to capture images of the calibration board, so as to obtain high-resolution corner points on the calibration board under the field of view of the two devices.

[0034] Based on the corner points on the calibration board, the PnP algorithm is used to calculate the external parameters between the two devices.

[0035] A second aspect of this application provides a visual measurement device for large-size equipment docking, the device comprising:

[0036] The target identification module is used to identify two targets to be docked from a large-sized target equipment.

[0037] The parameter calibration module is used to calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated;

[0038] The docking and imaging module is used to capture images of the docking surfaces of the two targets to be docked using multiple cameras with calibrated external parameters, thereby obtaining multiple local images, wherein the multiple local images correspond to multiple different regions of the docking surfaces;

[0039] The pose measurement module is used to determine the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the multiple local images.

[0040] A third aspect of this application provides an electronic device, including a memory and a processor. The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the visual measurement method for large-size equipment docking as described in various embodiments of this application.

[0041] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the visual measurement method for large-size equipment docking as described in various embodiments of this application.

[0042] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0043] The visual measurement method for large-scale equipment docking described in various embodiments of this application identifies two targets to be docked from the target large-scale equipment. Multiple cameras with pre-calibrated internal parameters are used to calibrate external parameters. The multiple cameras with calibrated external parameters simultaneously capture images of the docking surfaces of the two targets, obtaining multiple local images. These local images correspond to different regions of the docking surface. Based on the imaging features of the docking holes in the multiple local images, the pose parameters of the two targets to be docked are measured. Thus, because both targets are simultaneously observed, the impact of suboptimal imaging on measurement accuracy is reduced to some extent. Furthermore, because there is no need to install or remove targets, the measurement speed is faster and the applicability is wider, achieving docking of two large-scale equipment targets without the need for targets. Moreover, this application utilizes the imaging features of the docking holes in the docking surface images to obtain the pose parameters of the targets to be docked, resulting in smaller measurement errors and higher accuracy.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0045] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0046] Figure 1 This is a schematic diagram illustrating the steps of a visual measurement method for docking large-size equipment in an exemplary embodiment of this application;

[0047] Figure 2This is a calibration diagram of a camera device according to an exemplary embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the docking surfaces of two targets to be docked, captured by multiple camera devices in an exemplary embodiment of this application;

[0049] Figure 4 This is a schematic diagram illustrating the working process of visual measurement for docking of large-size equipment in an exemplary embodiment of this application;

[0050] Figure 5 This is a schematic diagram of docking of docking target 1 and docking target 2 in an exemplary embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the structure of a vision measurement and control system for large-size equipment docking in an exemplary embodiment of this application;

[0052] Figure 7 This is a schematic diagram of the structure of a visual measurement device for docking large-size equipment in an exemplary embodiment of this application;

[0053] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0055] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments depicted herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0056] Existing vision-based solutions require mounting targets on the object to be measured, and indirectly obtaining the position and orientation of the docking target by measuring the targets. However, this approach has several drawbacks: first, it requires high precision in target mounting; second, the use of targets is limited, and targets cannot be used in some scenarios; and third, errors are introduced during the conversion of measurement results, leading to low measurement accuracy.

[0057] Therefore, in some embodiments of this application, a visual measurement method for large-size equipment docking is provided, such as... Figure 1 As shown, the method includes steps S1 to S4.

[0058] S1. Identify two targets to be docked from the large-sized equipment.

[0059] Large-scale equipment refers to large objects that are inconvenient to measure directly, such as launch vehicles and aircraft. Rapid and automatic docking of launch vehicle or aircraft modules requires precise measurement of the module's position and attitude parameters. If the module's geometric parameters are known, a cylinder can be used as an approximation, and the docked modules are connected by radial bolts. During automatic docking, the position and attitude parameters of the two modules to be docked are first measured. Then, the position and attitude of one module are adjusted so that the attitude parameters of the two modules are identical, with the position parameters offset only in the radial direction, ensuring that the docking holes of the radial bolt connections overlap. Here, this application defines the two modules of an aircraft to be docked as two docking targets.

[0060] S2. Calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated.

[0061] The internal parameters of imaging equipment, such as cameras, are generally calibrated by the manufacturer before leaving the factory. These internal parameters include focal length, principal point coordinates, and distortion parameters, including radial and tangential distortion. When calibrating these internal parameters, the target is first photographed from different positions or angles using the imaging equipment being calibrated, acquiring 10 to 20 images. These images are then used in conjunction with a calibration algorithm, and the target must be within the clear imaging range of the imaging equipment.

[0062] In one possible implementation, the specific intrinsic parameter calibration process includes: preparing a checkerboard calibration board of known size, ensuring that each square and corner of the calibration board can be clearly captured by a camera; taking 15-20 images containing the calibration board, ensuring that the calibration board has different poses and positions within the camera's field of view during the shooting process; for each calibration image, detecting the pixel coordinates of the corner points on the calibration board, and associating the detected corner point pixel coordinates with corresponding points in the world coordinate system; using the known world coordinate system points and corresponding pixel coordinates, calculating the camera's intrinsic parameter matrix according to the Zhang Zhengyou calibration method. The Zhang Zhengyou calibration method is a commonly used camera calibration method based on a checkerboard pattern captured by the camera. This method calculates the camera's intrinsic parameter matrix by capturing the checkerboard corner points in the image and establishing a mapping relationship between the object coordinate system and the image coordinate system through these corner points. This intrinsic parameter matrix is ​​typically represented as:

[0063] K = [f x 0c x 0f y c y 0 0 1]

[0064] Among them, f x and f y It refers to the camera's horizontal and vertical focal lengths, c x and c yThese are the principal point coordinates. This internal parameter matrix allows for more accurate image processing. Because this application requires the use of multiple camera devices to photograph the docking surface of the target, the camera devices still need external parameter calibration after the internal parameters are calibrated.

[0065] The camera's extrinsic parameters describe the camera's position and orientation in the world coordinate system, typically including the camera's position (translation vector) and orientation (rotation matrix). These parameters are used to map points in the camera coordinate system to the world coordinate system, or vice versa. The extrinsic parameters are usually composed of a rotation matrix and a translation vector. Together, they form the camera's extrinsic parameter matrix.

[0066] In a specific implementation, such as Figure 2 As shown, each pair of cameras with calibrated internal parameters is used as a calibration device to be calibrated together, and a checkerboard calibration board with dual cameras is arranged. At the same time, two of the cameras to be calibrated together are used to capture images of the calibration board to obtain high-resolution corner points on the calibration board from the field of view of the two cameras. Based on the corner points on the calibration board, the external parameters between the two cameras are calculated using the PnP (Perspective-n-Point) algorithm. More specifically, when calculating the extrinsic parameters between the two devices using the PnP algorithm based on the corner points on the calibration board, firstly, for the image of each of the two devices, the pixel coordinates of the corner points on the calibration board are detected using a corner detection algorithm. Then, a shared world coordinate system is defined, with a corner point on the calibration board as the origin, and the grid points on the calibration board as reference points for the coordinate axes. The unit length of the world coordinate system is determined, and the detected corner pixel coordinates are associated with the corresponding points in the world coordinate system. Finally, based on the association results, the PnP algorithm is used to calculate the extrinsic parameters between the two devices, where the extrinsic parameters include the rotation matrix R and the translation vector T.

[0067] The rotation matrix R describes the camera's pose or rotation; it represents how the camera coordinate system rotates relative to the world coordinate system and is a 3x3 matrix. The translation vector T describes the camera's position translation relative to the world coordinate system; it represents the displacement of the camera from one position to another and is a 3x1 vector. Assume there are n feature points, each with coordinates P (3D coordinates) in the camera coordinate system and p (2D pixel coordinates) in the image. The camera intrinsic matrix is ​​K. The goal of the PnP algorithm is to find the optimal rotation matrix R and translation vector T such that the 3D point P in the world coordinate system is mapped to the pixel coordinate p in the image through the camera intrinsic matrix K, i.e., p = K·[R|T]·P, which can be further expressed as:

[0068]

[0069] For camera 2, R' and T' are obtained in the same way, and the relative pose between the two cameras is finally obtained according to the following formula.

[0070] R rel =R T R′

[0071] T rel =-R T T′+T

[0072] Among them, R rel Let T be the rotation matrix between the two cameras. rel It is a translation vector.

[0073] Since multiple cameras are required for measurement, external parameter calibration is performed by calibrating two cameras together, with each camera calibrated once. For example, when there are three cameras in the system, each camera is calibrated twice: camera 1 with camera 2, and camera 1 with camera 3. The calibration results are then transformed to obtain the rotation matrix and translation vector between the three cameras. Let... and These are the rotation matrices between camera 2 and camera 1, and between camera 3 and camera 1, respectively. and P represents the translation vectors between camera 2 and camera 1, and between camera 3 and camera 1, respectively. 2 Let P be the coordinates of a point on the target to be docked in the camera 2 coordinate system. Then, the coordinates of that point in the camera 1 coordinate system are P. 1 for:

[0074]

[0075] Similarly, let P 3 Let P be the coordinates of a point on the target to be docked in the camera 3 coordinate system. 1 for:

[0076]

[0077] Only after the external parameters of multiple camera devices are calibrated can they be used together to photograph the docking surface of the target to be docked.

[0078] S3. Using multiple cameras with calibrated external parameters, simultaneously capture images of the docking surfaces of the two targets to be docked, obtaining multiple local images.

[0079] Figure 3 This illustration shows a scenario where multiple cameras capture images of the docking surfaces of two targets to be docked. Figure 3Three cameras are used to simultaneously capture images of the docking surface from different angles. By using multiple cameras with different perspectives, the attitude or position of the target to be docked can be obtained, which is very useful for understanding the target's direction and orientation during the docking process. Large equipment, such as two sections of an aircraft to be docked, represent two targets. Due to the large size of the docking surfaces and the potential for obstruction, close-range measurements suffer from poor image completeness, requiring high-resolution camera equipment and making it difficult to perform high-precision pose parameter measurements based on external contour features. Considering the large size of the two targets, a single camera can only capture a local area, and measurement accuracy would be low if only a local area was used. Therefore, multiple cameras are used to simultaneously capture images of the docking surfaces of the two targets. Subsequently, the docking holes can be identified from the feature information of the local images, and measurement data can be obtained based on the docking holes.

[0080] S4. Based on the imaging features of the docking holes in the multiple local images, measure the pose parameters of the two targets to be docked.

[0081] In one specific implementation, the pose parameters of the two targets to be docked are measured based on the imaging features of the docking holes in multiple local images, including steps S41-S43.

[0082] S41. Extract feature information from multiple local images and identify the docking holes of the two targets to be docked based on the feature information.

[0083] Specifically, refer to Figure 4 For each local image, feature information is extracted based on region segmentation. The shape, size, and position of the docking holes are determined from these feature information. The docking holes of the two targets to be docked are then identified based on their shape, size, and position. For example... Figure 4 As shown, three camera devices, namely camera device 1, camera device 2, and camera device 3, simultaneously acquired images, obtaining local images. After region segmentation and feature detection, the docking hole can be detected. Region segmentation is an image segmentation technique that aims to divide an image into several regions, so that pixels within the same region have similar characteristics, while different regions have obvious differences.

[0084] In one implementation, the shape, size, and position of the docking hole are determined from the feature information. When identifying the docking holes of two targets to be docked based on their shape, size, and position, a closed contour detection algorithm can be used. Specifically, the algorithm detects the contours of objects in the image, searching for contours that form closed regions. Based on the shape of the docking hole, several filtering conditions are used to select closed contours, which may include the area and perimeter of the contour. Then, shape fitting is performed on the selected contours, providing information about the position and size of the docking hole. The dimensions of the docking hole are then measured using the fitted shape information, and the center position of the docking hole is obtained using the center coordinates obtained after fitting the shape.

[0085] S42. Using any one of the multiple camera devices with calibrated external parameters as the measurement reference coordinate system, solve for the position parameters of the center of at least one docking hole within the field of view of the other camera devices.

[0086] Specifically, it is determined that at least one docking hole exists within the field of view of other camera devices, and the position parameters of the center of at least one docking hole are solved based on a monocular circular pose measurement algorithm. Assume there are three camera devices A, B, and C, where A is selected as the measurement reference coordinate system. Devices B and C each have one docking hole within their field of view, and the center position parameters of these two docking holes need to be solved. The docking hole is located within the field of view of device B, and the position parameters of its center are determined using a monocular circular pose measurement algorithm. Similarly, the docking hole is located within the field of view of device C, and its center position parameters are solved using the same algorithm.

[0087] S43. Combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of at least one docking hole center, measure the pose parameters of the two targets to be docked, and obtain the pose measurement values ​​of the two targets to be docked.

[0088] Specifically, based on the external parameter calibration results, the position parameters of at least one docking hole center within the field of view of other camera devices are transformed to the measurement reference coordinate system, and the imaging point coordinates of at least one docking hole in the measurement reference coordinate system are calculated; combining all feature information within the reference field of view of the camera device serving as the measurement reference coordinate system, the original docking hole center imaging point coordinates within the reference field of view are obtained; based on the imaging point coordinates of at least one docking hole in the measurement reference coordinate system and the original docking hole center imaging point coordinates within the reference field of view, all docking hole combinations (corresponding to) in the same preset coordinate system are obtained. Figure 4 The process involves: (1) combining features in the model; solving for the pose parameters of all docking hole combinations to obtain the initial pose values ​​of the two targets to be docked; and optimizing the initial pose values ​​to obtain the measured pose values ​​of the two targets to be docked. The imaging point coordinates involve the camera projection model, which maps points in three-dimensional space onto a two-dimensional image and calculates the corresponding pixel coordinates.

[0089] In one possible implementation, the initial pose value is optimized to obtain the pose measurement values ​​of the two targets to be docked. This includes: transforming the initial pose value to a camera coordinate system corresponding to multiple camera devices based on the external parameter calibration results; calculating the projection curve of the spatial circle where the target docking hole is located and the projection point of the center of the docking hole in the camera coordinate system, and comparing the projection curve and projection point with the actual detected feature points to obtain the difference; using the difference as the value of the objective function for iterative pose optimization, adjusting the initial pose value until the value of the objective function reaches its minimum, thus obtaining the pose measurement values ​​of the two targets to be docked.

[0090] Specifically, calculating the projection curve of the spatial circle containing the target docking hole and the projection point of the hole's center in the camera coordinate system typically involves the camera projection model and the target geometry. The camera's projection can be described using a perspective projection model. This model maps points in three-dimensional space onto a two-dimensional image plane. The camera projection model can be expressed by the following formula:

[0091] s[uv 1]=K[Rt][XYZ 1]

[0092] Where s is the scaling factor, (u,v) are the coordinates on the image plane, K is the camera intrinsic matrix, R is the rotation matrix, t is the translation vector, and (X,Y,Z) are the coordinates of the target point in the camera coordinate system. Assuming the target docking hole is circular, its representation in three-dimensional space can be expressed by parametric equations:

[0093] [X(θ)Y(θ)Z(θ)]=[Xc+rcos(θ)Yc+rsin(θ)Zc]

[0094] Where (Xc,Yc,Zc) are the coordinates of the center of the docking hole in the camera coordinate system, r is the radius of the docking hole, and θ is the angle parameter, that is, the angle of a point on the circle relative to the center of the circle.

[0095] When calculating the projection curve and projection points, the target's three-dimensional coordinates are substituted into the camera projection model to obtain the corresponding two-dimensional image coordinates. For a target shape represented by parametric equations, θ can be considered a parameter, and then the projection points (u,v) corresponding to different θ values ​​are calculated. This yields the projection curve on the image. Simultaneously, the three-dimensional coordinates corresponding to θ, i.e., the coordinates of the docking hole center in the camera coordinate system, can be calculated as the projection point of the docking hole center. In other words, in the camera projection model, the three-dimensional coordinates of the target point are projected onto the image plane as coordinates (u,v) through a combination of the camera intrinsic parameter matrix K, the rotation matrix R, and the translation vector t. In this process, θ is considered a parameter; by changing the value of θ, the projection points (u,v) corresponding to different θ values ​​can be calculated.

[0096] refer to Figure 5 The pose measurements of the two targets to be docked are obtained, such as... Figure 5 As shown, the first target to be docked ( Figure 5 The pose parameters of the docking target 1) include the center coordinates O1 of the docking surface of the first docking target, the axis normal vector V1 of the first docking target, and the rotation angle θ1 of the first docking target relative to its own axis; the second docking target ( Figure 5 The pose parameters of the docking target 2) include the center coordinates O2 of the docking surface of the second docking target, the axis normal vector V2 of the second docking target, and the rotation angle θ2 of the second docking target relative to its own axis.

[0097] By using multiple cameras with calibrated external parameters to simultaneously capture images of the docking surfaces of two targets to be docked, multiple local images are obtained. Based on the imaging characteristics of the docking holes in the multiple local images, the pose parameters of the two targets to be docked are measured. This application is equivalent to simultaneously "seeing" the two targets to be docked. During measurement, each camera simultaneously captures images of the docking holes on docking target 1 and docking target 2. The position and attitude of docking target 1 and docking target 2 in the camera coordinate system are determined according to the position of the docking holes, which makes the measurement error small and the accuracy high. To a certain extent, it can reduce the impact of unsatisfactory imaging effect on measurement accuracy. Furthermore, since there is no need to install and disassemble the target, the measurement speed is faster and the application range is wider. That is, docking of two large-sized equipment targets to be docked is achieved without the need for a target.

[0098] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.

[0099] In some embodiments of this application, a vision measurement and control system for large-size equipment docking is provided, such as... Figure 6 As shown, it mainly includes camera equipment, an image vision processing unit, and a control system. Multiple camera devices synchronously acquire images under the control of the image vision processing unit; the image vision processing unit performs tasks such as image acquisition control, image preprocessing, feature extraction, pose calculation, and coordinate transformation, and finally sends the pose measurement results to the control system; the control system then controls the target to be docked based on the pose measurement results.

[0100] In other embodiments of this application, a visual measurement device for large-size equipment docking is also provided, which performs the visual measurement method for large-size equipment docking described in various embodiments, such as... Figure 7 As shown, the device includes:

[0101] The target determination module 701 is used to identify two targets to be docked from a large-sized target equipment.

[0102] The parameter calibration module 702 is used to calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated;

[0103] The docking and imaging module 703 is used to capture images of the docking surfaces of the two targets to be docked using multiple cameras with externally calibrated parameters, thereby obtaining multiple local images, wherein the multiple local images correspond to multiple different regions of the docking surfaces;

[0104] The pose measurement module 704 is used to determine the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the plurality of local images.

[0105] The aforementioned visual measurement device for large-size equipment docking utilizes the imaging features of the docking holes in the docking surface image to obtain the pose parameters of the target to be docked, resulting in small measurement errors and high accuracy. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application.

[0106] Please refer to the following. Figure 8 This is a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device 2 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program that can run on the processor 200. When the processor 200 runs the computer program, it executes a visual measurement method for docking large-size equipment as described in any of the embodiments of this application.

[0107] The memory 201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 203 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0108] Bus 202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. Memory 201 is used to store programs. After receiving an execution instruction, processor 200 executes the program. The visual measurement method for large-size equipment docking disclosed in any of the foregoing embodiments of this application can be applied to processor 200, or implemented by processor 200.

[0109] The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 200 or by instructions in software form. The processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 201. Processor 200 reads the information in memory 201 and, in conjunction with its hardware, completes the steps of the visual measurement method for docking large-size equipment.

[0110] This application also provides a computer-readable storage medium corresponding to the visual measurement method for large-size equipment docking provided in the foregoing embodiments. The medium stores a computer program, which, when executed by a processor, performs the visual measurement method for large-size equipment docking provided in any of the foregoing embodiments. Furthermore, examples of the computer-readable storage medium may include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical or magnetic storage media, which will not be elaborated upon here.

[0111] In addition, this application also provides a computer program product, including a computer program that, when executed by a processor, implements a visual measurement method for docking large-size equipment in any of the foregoing embodiments.

[0112] Those skilled in the art will understand that the various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art should understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation apparatus according to embodiments of this application.

[0113] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A visual measurement method for docking large-size equipment, characterized in that, The method includes: Identify two targets to be docked from the large-sized equipment; Calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated; Multiple cameras with calibrated external parameters are used to simultaneously capture images of the docking surfaces of the two targets to be docked, resulting in multiple local images, wherein the multiple local images correspond to multiple different regions of the docking surfaces; Based on the imaging features of the docking holes in the multiple local images, the pose parameters of the two targets to be docked are measured. The step of measuring the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the multiple local images includes: Extract feature information from the multiple local images, and identify the docking holes of the two targets to be docked based on the feature information; Using any one of the multiple camera devices calibrated with the external parameters as the measurement reference coordinate system, solve for the position parameters of the center of at least one docking hole within the field of view of the other camera devices; By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole, the pose parameters of the two targets to be docked are measured to obtain the pose measurement values ​​of the two targets to be docked. The method of combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole, to measure the pose parameters of the two targets to be docked, and obtaining the pose measurement values ​​of the two targets to be docked, includes: Based on the external parameter calibration results, the position parameters of at least one docking hole center within the field of view of the other camera equipment are converted to the measurement reference coordinate system, and the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system are calculated. By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, the coordinates of the original center imaging point of the docking hole within the reference field of view are obtained. Based on the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system and the original center imaging point coordinates of the docking hole in the reference field of view, all docking hole combinations in the same preset coordinate system are obtained. Solve the pose parameters for all the docking hole combinations to obtain the initial pose values ​​of the two targets to be docked; The initial pose value is optimized to obtain the pose measurement values ​​of the two targets to be docked.

2. The visual measurement method for large-size equipment docking according to claim 1, characterized in that, The step of extracting feature information from the multiple local images and identifying the docking holes of the two targets to be docked based on the feature information includes: For each local image, feature information of each local image is extracted based on region segmentation; The shape, size, and location of the mating hole are determined from the feature information; The docking holes of the two targets to be docked are identified by their shape, size, and position.

3. The visual measurement method for large-size equipment docking according to claim 1, characterized in that, The method of solving for the position parameters of the center of at least one docking hole within the field of view of other camera devices includes: It was determined that at least one docking hole existed within the field of view of other camera devices, and the position parameters of the center of at least one docking hole were solved based on a monocular circular pose measurement algorithm.

4. The visual measurement method for large-size equipment docking according to claim 1, characterized in that, The optimization of the initial pose value to obtain the pose measurement values ​​of the two targets to be docked includes: Based on the external parameter calibration results, the initial pose value is converted to the camera coordinate system corresponding to the plurality of camera devices; The projection curve of the spatial circle where the target docking hole is located and the projection point of the center of the docking hole are calculated in the camera coordinate system, and the projection curve and the projection point are compared with the actual detected feature points to obtain the difference. The difference is used as the value of the objective function for pose iteration optimization. The initial pose value is adjusted until the value of the objective function reaches its minimum, and the pose measurement values ​​of the two targets to be docked are obtained.

5. The visual measurement method for large-size equipment docking according to claim 1, characterized in that, The calibration of external parameters for multiple pre-calibrated internal camera devices includes: Two cameras from a set of pre-calibrated internal parameters are used as the cameras to be calibrated together. A checkerboard calibration board for arranging dual-camera equipment; Simultaneously, two devices from the calibration equipment are used to capture images of the calibration board, so as to obtain high-resolution corner points on the calibration board under the field of view of the two devices. Based on the corner points on the calibration board, the PnP algorithm is used to calculate the external parameters between the two devices.

6. A visual measurement device for large-size equipment docking, characterized in that, The device includes: The target identification module is used to identify two targets to be docked from a large-sized target equipment. The parameter calibration module is used to calibrate the external parameters of multiple camera devices whose internal parameters have been pre-calibrated; The docking and imaging module is used to capture images of the docking surfaces of the two targets to be docked using multiple cameras with calibrated external parameters, thereby obtaining multiple local images, wherein the multiple local images correspond to multiple different regions of the docking surfaces; The pose measurement module is used to measure the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the multiple local images. The step of measuring the pose parameters of the two targets to be docked based on the imaging features of the docking holes in the multiple local images includes: Extract feature information from the multiple local images, and identify the docking holes of the two targets to be docked based on the feature information; Using any one of the multiple camera devices calibrated with the external parameters as the measurement reference coordinate system, solve for the position parameters of the center of at least one docking hole within the field of view of the other camera devices; By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole, the pose parameters of the two targets to be docked are measured to obtain the pose measurement values ​​of the two targets to be docked. The method of combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, and the position parameters of the center of at least one docking hole, to measure the pose parameters of the two targets to be docked, and obtaining the pose measurement values ​​of the two targets to be docked, includes: Based on the external parameter calibration results, the position parameters of at least one docking hole center within the field of view of the other camera equipment are converted to the measurement reference coordinate system, and the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system are calculated. By combining all feature information within the reference field of view of the camera device, which serves as the measurement reference coordinate system, the coordinates of the original center imaging point of the docking hole within the reference field of view are obtained. Based on the imaging point coordinates of the at least one docking hole in the measurement reference coordinate system and the original center imaging point coordinates of the docking hole in the reference field of view, all docking hole combinations in the same preset coordinate system are obtained. Solve the pose parameters for all the docking hole combinations to obtain the initial pose values ​​of the two targets to be docked; The initial pose value is optimized to obtain the pose measurement values ​​of the two targets to be docked.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the visual measurement method for docking of large-size equipment as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the visual measurement method for docking of large-size equipment as described in any one of claims 1-5.

Images