Calibration cloth manufacturing method, calibration method and image splicing method for container spreader four-eye camera

Abstract

The invention belongs to the technical field of port equipment, and particularly provides a manufacturing method, a calibration method and an image splicing method of calibration cloth for a four-eye camera of a container spreader. The calibration cloth manufacturing method comprises the steps that base cloth is arranged below a lifting appliance; the lifting appliance is controlled to rise to a preset height, and the lifting appliance is contracted to the minimum size; acquiring a pixel point set of the edge of the lifting appliance under the current visual angle of one camera; acquiring a pixel region comprising a plurality of pixel points based on the pixel point set and a view center point of the camera at the current view angle; performing ground coordinate conversion to obtain ground coordinates of the pixel points; obtaining the size of the calibration grid based on the ground coordinates of the pixel points; determining the size of the calibration cloth based on the size of the calibration grid and a preset proportion, cutting based on the size of the calibration cloth to obtain the calibration cloth, and printing each feature point based on the ground coordinates of the pixel points to form the calibration grid; and printing other calibration grids by taking the center of the calibration cloth as a symmetric point. The method is convenient to deploy and high in precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of port equipment, in particular to a calibration cloth manufacturing method, a calibration method and an image splicing method for a four-camera of a container spreader. BACKGROUND

[0002] In the intelligent port application scenario, the automation and intelligentization of the spreader container grabbing and placing function is one of the important measures to improve market competitiveness. At present, the problem of the laser-based spreader container grabbing and placing modification scheme is that the modification cost is relatively high, the system is complex, and the false detection rate of the bulging container (side deformation) is high. The spreader four-corner monitoring camera-based spreader deviation detection scheme has a relatively low cost and is suitable for large-area application. The spreader four-corner monitoring camera needs to be calibrated before deviation detection to ensure that the images collected by multiple cameras can be spliced into a complete spreader overhead image under the same ground coordinate system. However, the calibration of the spreader four-corner monitoring camera currently often uses a special indoor calibration device for structured light calibration or self-calibration in an indoor controllable light environment, that is, the spreader needs to be calibrated away from the normal operation environment, which greatly affects the operation efficiency of the equipment. In addition, the indoor environment has high requirements for the site, equipment layout, and lighting conditions, which is difficult to meet the actual needs of complex operation scenarios such as ports and terminals. In addition, the traditional calibration method has low splicing precision when facing multiple camera layouts and other situations due to the lack of calibration assistance. Therefore, there is an urgent need for a calibration device and method that is suitable for the spreader operation site of the port, easy to deploy, and has high adaptability and precision. SUMMARY

[0003] Therefore, the present application provides a calibration cloth manufacturing method, a calibration method and an image splicing method for a four-camera of a container spreader, which are easy to deploy and have high precision.

[0004] To solve at least one of the above technical problems, the present application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a calibration cloth manufacturing method for a four-camera of a container spreader. Four corners of the calibration cloth are respectively formed with four calibration squares. Each calibration square is provided with a rectangular array including a plurality of feature points. The manufacturing method comprises the following steps:

[0006] Step S1, disposing a base cloth below the spreader;

[0007] Step S2, controlling the spreader to rise to a preset height and shrink to a minimum size;

[0008] Step S3, acquiring a set of pixel points of the edge of the spreader under the current angle of view of one camera of the spreader;

[0009] Step S4, based on the pixel point set and the field of view center point under the current perspective of the camera, a pixel region for setting the calibration grid corresponding to the camera is obtained, the pixel region includes a plurality of pixel points corresponding to the feature points respectively;

[0010] Step S5, ground coordinate conversion is performed on the pixel coordinates of the pixel points in the pixel region to obtain the ground coordinates of the pixel points;

[0011] Step S6, the size of the calibration grid is obtained based on the ground coordinates of the pixel points respectively;

[0012] Step S7, the size of the calibration cloth is determined based on the size of the calibration grid and the preset proportion, the cutting device is used to cut the base cloth to obtain the calibration cloth based on the size of the calibration cloth, and the printing device is used to print the feature points on the calibration cloth to form the calibration grid corresponding to the camera based on the ground coordinates of the pixel points;

[0013] Step S8, the center of the calibration cloth is taken as the symmetry point, and the calibration grids of the other three corners are printed.

[0014] In an embodiment of the present application, step S3 comprises:

[0015] An image under the current perspective of the camera is obtained and preprocessed to obtain a preprocessed image, the preprocessing including one or more of denoising, brightness enhancement or contrast enhancement;

[0016] Based on the preprocessed image, the edges of the sling in the image are identified by performing semantic segmentation on the preprocessed image through a pre-trained segmentation network model, and the pixel points of the edges of the sling are collected into the pixel point set.

[0017] In an embodiment of the present application, step S4 comprises:

[0018] Step S41, a combined bounding box corresponding to the sling edge is constructed based on the pixel point set, and the shortest distance from the field of view center point to the sling edge is obtained based on the positional relationship between the combined bounding box and the field of view center point;

[0019] Step S42, offset calculation is performed based on the shortest distance and the arrangement relationship of the plurality of feature points in the rectangular array to obtain the pixel region.

[0020] In an embodiment of the present application, step S42 comprises:

[0021] Step S421, based on the shortest distance and the arrangement relationship of the four corner feature points and the center feature point in the rectangular array, offset calculation is performed with the field of view center point as the reference point to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points respectively;

[0022] Step S422, obtaining a pixel region corresponding to the calibration grid under the current view angle of the camera based on the pixel coordinates of the four corner pixels;

[0023] The field of view center point corresponds to the center feature point.

[0024] In an embodiment of the present application, step S5 comprises:

[0025] Step S51, obtaining the intrinsic matrix, distortion parameters and pose parameters of the camera;

[0026] Step S52, based on the intrinsic matrix, distortion parameters, pose parameters and the preset height, performing ground coordinate conversion on the pixel coordinates of each pixel in the pixel region by using a projection equation to obtain the ground coordinates of each pixel.

[0027] In an embodiment of the present application, step S6 comprises:

[0028] Step S61, obtaining the distance between the ground coordinates of the two adjacent pixels based on the ground coordinates of the two adjacent pixels;

[0029] Step S62, determining the size of the calibration grid based on the distance between the ground coordinates of the two adjacent pixels and the arrangement relationship of the plurality of feature points in the rectangular array.

[0030] In an embodiment of the present application, the area of the calibration cloth is greater than the projection area of the spreader, and step S7 comprises:

[0031] Step S71, obtaining the proportional relationship between the size of the calibration grid and the size of the calibration cloth based on the preset proportion;

[0032] Step S72, determining the size of the calibration cloth based on the proportional relationship between the size of the calibration grid and the size of the calibration cloth, and making the cutting device cut the base cloth to obtain the calibration cloth based on the size of the calibration cloth;

[0033] Step S73, making the printing device print each feature point on the calibration cloth to form the calibration grid corresponding to the camera based on the ground coordinates of the pixels.

[0034] In an embodiment of the present application, the calibration cloth is provided with four right-angle marks corresponding to the calibration grids respectively, each right-angle mark is aligned with the projection edge of the corner lock cylinder of the spreader, and step S7 further comprises:

[0035] Based on the projection of the corner lock cylinder of the spreader, the printing device prints a plurality of right-angle marks on the calibration cloth.

[0036] The second aspect of the present application further provides a calibration method of a calibration cloth for a four-eye camera of a container spreader, comprising:

[0037] The calibration cloth is laid under the lifting appliance, the lifting appliance is controlled to be lifted to a preset height, and the straight angle lines on the calibration cloth are aligned with the projected edges of the corresponding corner lock cores on the lifting appliance respectively;

[0038] The four groups of cameras on the lifting appliance are controlled to capture images of the calibration cloth under the current visual angle respectively, wherein four calibration squares are formed at four corners of the calibration cloth respectively, and a rectangular array including a plurality of feature points is arranged in each calibration square;

[0039] The intrinsic matrix and the distortion parameters of each camera are acquired respectively, and the images captured by each camera are corrected based on the intrinsic matrix and the distortion parameters to obtain corrected images;

[0040] The ground coordinates of each feature point in each calibration square and the pixel coordinates of the pixel points corresponding to each feature point in each corrected image are acquired, and plane calibration calculation is performed based on the ground coordinates of each feature point and the pixel coordinates of the pixel points corresponding to each feature point to obtain a homography matrix corresponding to each camera.

[0041] In an embodiment of the present application, acquiring the ground coordinates of each feature point in each calibration square and the pixel coordinates of the pixel points corresponding to each feature point in each corrected image, and performing plane calibration calculation based on the ground coordinates of each feature point and the pixel coordinates of the pixel points corresponding to each feature point to obtain a homography matrix corresponding to each camera comprises:

[0042] Based on the arrangement relationship of the plurality of feature points in the calibration square, the ground coordinates of each feature point in each calibration square are acquired respectively;

[0043] Point position detection is performed on each corrected image to obtain the pixel coordinates of the pixel points corresponding to each feature point in each corrected image;

[0044] Plane calibration is performed based on the ground coordinates of each feature point and the pixel coordinates of the pixel points corresponding to each feature point, and a least square method is used to obtain a homography matrix corresponding to each camera.

[0045] The third aspect of the present application further provides a container lifting appliance four-camera image splicing method based on calibration cloth calibration parameters, comprising:

[0046] After the lifting appliance grasps the container, the lifting appliance is controlled to be lifted to a preset height;

[0047] The four groups of cameras on the lifting appliance are controlled to capture images of the four corners of the container under the current visual angle respectively;

[0048] Based on the intrinsic matrix and the distortion parameters of each camera, the images captured by each camera are corrected respectively to obtain corrected corner images of the container;

[0049] The homography matrix corresponding to each camera is acquired by using the method in any one of the embodiments of the second aspect, perspective transformation is performed on each corrected corner image, and corner orthographic projection images of the four corners of the container in the ground coordinate system are respectively acquired;

[0050] The orthographic projection images of the four corners of the container in the ground coordinate system are spliced to acquire a complete orthographic projection image of the container in the ground coordinate system.

[0051] The above technical solutions of the present application have at least one of the following beneficial effects:

[0052] According to the calibration cloth manufacturing method, on the basis of acquiring the pixel point set of the edge of the spreader, the pixel region of the calibration grid corresponding to the camera is set by using the field of view center point under the current view angle of the camera, the ground coordinate conversion is performed on the pixel coordinates of each pixel point in the pixel region, the ground coordinates of the pixel points are acquired, and then the size of the calibration grid and the calibration cloth is acquired, so that the calibration cloth required for calibration can be acquired conveniently and quickly.

[0053] According to the calibration cloth calibration method, the calibration cloth is laid under the spreader, the images of the calibration cloth under the current view angle are respectively shot by the four groups of cameras on the spreader after distortion correction, the plane calibration calculation is performed on the ground coordinates of each feature point and the pixel coordinates of the pixel points corresponding to each feature point, so as to acquire the homography matrix of each camera for subsequent image splicing, thereby without indoor calibration, the method is suitable for the spreader operation site of the port, is convenient to deploy, and has high adaptability and precision.

[0054] According to the image splicing method, the images of the four corners of the container under the current view angle are respectively shot by the four groups of cameras, the homography matrix corresponding to each camera is acquired by using the calibration cloth calibration, perspective transformation is performed on each corrected corner image, the corner orthographic projection images of the four corners of the container in the ground coordinate system are respectively acquired, and the orthographic projection images of the four corners of the container in the ground coordinate system are spliced into a complete orthographic projection image, so that the image splicing precision can be effectively improved by using the homography matrix after calibration for perspective transformation. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 An implementation environment schematic diagram provided for an embodiment of the present application;

[0056] Figure 2 A structure schematic diagram of the calibration cloth provided for an embodiment of the present application;

[0057] Figure 3 A flowchart of the calibration cloth manufacturing method for the four-camera spreader of the container provided for an embodiment of the present application;

[0058] Figure 4 A flowchart for obtaining a pixel region according to an embodiment of the present application is provided;

[0059] Figure 5 A flowchart for obtaining a pixel region based on a shortest distance and an arrangement relationship according to an embodiment of the present application is provided;

[0060] Figure 6 A flowchart for obtaining a ground coordinate of each pixel point according to an embodiment of the present application is provided;

[0061] Figure 7 A flowchart for obtaining a calibration grid size according to an embodiment of the present application is provided;

[0062] Figure 8 A flowchart for a calibration method of a calibration cloth for a container spreader four-camera according to an embodiment of the present application is provided;

[0063] Figure 9 A flowchart for obtaining a homography matrix corresponding to each camera according to an embodiment of the present application is provided;

[0064] Figure 10 A flowchart for a container spreader four-camera image splicing method based on calibration parameters of a calibration cloth according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the described embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0066] Reference is made to the accompanying drawings described in the specification Figure 1 which shows an implementation environment schematic diagram provided by an embodiment of the present application for manufacturing a calibration cloth for a container spreader four-camera. As shown in the figure, Figure 1 the implementation environment can include a container spreader 100 and a computer device 120, and one camera 110 is arranged at each corner of the container spreader 100. The container spreader 100 and the computer device 120 can be directly or indirectly connected through wired or wireless communication, and the embodiments of the present application do not limit this.

[0067] The computer device 120 can be, but is not limited to, various servers, personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, and the like. The server can be a stand-alone server or a server cluster composed of multiple servers or a distributed system. The server can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0068] As shown in Figure 1 and Figure 2 , in the embodiment of the present application, the base cloth can be first arranged below the container spreader 100, and then the computer device 120 controls the container spreader 100 to rise to a preset height and shrinks the container spreader 100 to the minimum size. The camera 110 at any corner of the container spreader 100 is used to shoot the base cloth below the container spreader 100 to obtain an image under the current view angle of the camera 110. Then the computer device 120 can identify and obtain a set of pixel points of the edge of the container spreader 100 under the current view angle, and based on the set of pixel points and the center point of the field of view under the current view angle of the camera 110, obtain a pixel area for setting the calibration square 210 corresponding to the camera 110, and then perform ground coordinate conversion on the pixel coordinates of each pixel point in the pixel area to obtain the ground coordinates of the pixel points. After obtaining the ground coordinates of the pixel points, the size of the calibration square 210 is obtained based on the ground coordinates of the pixel points. After obtaining the size of the calibration square 210, since the size of the calibration square 210 is proportional to the size of the calibration cloth 200, the cutting device can be used to cut the base cloth to obtain the calibration cloth 200 based on the size of the calibration cloth 200, and the printing device can be used to print the feature points 211 of each corner on the calibration cloth 200 with the center of the calibration cloth 200 as the symmetry point to form the corresponding calibration square 210. In this way, the calibration cloth 200 required for calibration can be conveniently obtained, and by using the calibration cloth 200, the container spreader 100 does not need to be calibrated indoors, is suitable for port operation sites, is easy to deploy, and has high adaptability and precision.

[0069] As shown in Figure 2 , an embodiment of the present application provides that four corners of the calibration cloth 200 are respectively formed with four calibration squares 210, and each calibration square 210 is provided with a rectangular array including a plurality of feature points 211. Figure 3 , which shows a flowchart of a method for manufacturing a calibration cloth for a four-camera container spreader according to an embodiment of the present application. The method can be applied to the computer device in Figure 1 , specifically, the method can include the following steps:

[0070] Step S1, setting the base cloth below the lifting appliance.

[0071] Step S2, controlling the lifting appliance to rise to a preset height, and shrinking the lifting appliance to a minimum size.

[0072] In this embodiment, in order to enable the camera to completely capture the picture corresponding to the corner of the calibration cloth on the base cloth, the lifting appliance can be controlled to rise to a preset height, for example, the preset height can be 3-5 meters. In addition, by shrinking the lifting appliance to a minimum size, the lifting appliance can be prevented from causing obstruction to the field of view of the camera.

[0073] Step S3, obtaining a pixel point set of the edge of the lifting appliance in the current visual angle of one of the cameras of the lifting appliance.

[0074] In this embodiment, the original image in the current preset height visual angle of the camera can be obtained first, and then the original image is preprocessed to obtain a preprocessed image. Specifically, the preprocessing can include one or more of denoising, brightness enhancement or contrast enhancement, wherein the original image is denoised to eliminate environmental interference noise points, the original image is brightness enhanced to improve the imaging quality under low light conditions, and the original image is contrast enhanced to highlight the gray difference of different regions in the image, so that the lifting appliance profile is clearer in the image.

[0075] Then, based on the preprocessed image, the preprocessed image is subjected to semantic segmentation by a pre-trained segmentation network model to identify the edge of the lifting appliance in the image, and the pixel points of the edge of the lifting appliance are collected into a pixel point set. Specifically, the segmentation network model is constructed based on a deep convolutional neural network, and can output a semantic class label of each pixel at a pixel level, and the semantic class label can include at least three categories of “lifting appliance edge”, “lifting appliance main body” and “background”, wherein the “lifting appliance edge” category is used to identify the pixel points at the outer contour line of the lifting appliance. In one specific embodiment of the present application, for example, the preprocessed image I pre (u, v) is input into the segmentation network model f θ , the segmentation network model can segment and identify the lifting appliance through feature extraction, feature fusion and multi-scale prediction to obtain a semantic segmentation result M(u, v), and the process of semantic segmentation of the segmentation network model f θ can be represented as:

[0076] M(u, v) = f θ (I pre (u, v))

[0077] wherein the semantic segmentation result M(u, v) is a two-dimensional label matrix consistent with the resolution of the original image, and M(u, v) e {0, 1, 2} can be used to represent the class label of the pixel (u, v), for example, 0 is used to represent the background, 1 represents the main body of the spreader, and 2 represents the edge of the spreader. Further, in order to extract the edge region of the spreader, the coordinates of all pixel points satisfying M(u, v) = 2 can be extracted to form an unclustered pixel point set ε:

[0078] ε = {(u, v) | M(u, v) = 2}

[0079] Then, the pixel points in the unclustered pixel point set ε can be analyzed for connectivity to cluster them in space, and the isolated or misrecognized edge points can be removed to obtain the pixel point set ε' of the edge of the spreader under the current view angle of the camera. Thus, a reference is provided for obtaining the accurate boundary of the calibration grid in the subsequent steps.

[0080] Step S4, based on the pixel point set and the center point of the field of view under the current view angle of the camera, a pixel region for setting the calibration grid corresponding to the camera is obtained.

[0081] In the present embodiment, since a plurality of feature points are arranged in each calibration grid, and these feature points form a regular rectangular array according to a fixed pitch, i.e., the arrangement relationship and relative position are fixed, for example, a 9*5 rectangular array can be arranged in the calibration grid, which contains 45 feature points, and the horizontal and vertical pitches between the feature points remain consistent. Based on this, the shortest distance from the center point of the field of view to the edge of the spreader can be obtained based on the pixel point set of the edge of the spreader obtained under the current view angle of the camera and the center point of the field of view, and the offset is calculated based on the shortest distance and the arrangement relationship of the plurality of feature points in the rectangular array, so as to obtain a pixel region, which includes a plurality of pixel points, and each pixel point corresponds to one feature point in the calibration grid, thereby providing accurate pixel positioning for subsequent ground coordinate conversion and calibration printing. Specifically, as shown in FIG. 4, the step S4 can include: Figure 4

[0082] Step S41, based on the pixel point set, a combined bounding box corresponding to the edge of the spreader is constructed, and the shortest distance from the center point of the field of view to the edge of the spreader is obtained based on the position relationship between the combined bounding box and the center point of the field of view.

[0083] In the present embodiment, after obtaining the pixel point set of the edge of the spreader, a combined bounding box corresponding to the edge of the spreader can be further constructed based on the pixel point set. Then, based on the resolution information of the image under the current view angle of the camera, the horizontal pixel width width and the vertical pixel height height of the image are obtained, so as to determine the pixel coordinates P center :

[0084] ​P center = (width / 2, height / 2)

[0085] Then, the Euclidean distance between the camera field of view center point and all pixel points of the spreader edge is calculated based on the pixel coordinates P center and the pixel point set ε' of the spreader edge, and the minimum value is obtained therefrom as the shortest distance r from the field of view center point to the spreader edge:

[0086] r = min(distance(P center , ε'))

[0087] where distance(P center , ε') can represent the distance between the field of view center point and each pixel point of the spreader edge. The shortest distance r finally obtained can be used to reflect the spatial relationship between the field of view center point and the spreader under the current camera view angle, and to provide a basis for subsequent offset calculation to obtain the pixel region corresponding to the calibration grid under the current view angle.

[0088] Step S42, offset calculation is performed based on the shortest distance and the arrangement relationship of the plurality of feature points in the rectangular array to obtain the pixel region.

[0089] In this embodiment, in order to determine the pixel region corresponding to the calibration grid in the image, offset calculation needs to be performed based on the shortest distance and the arrangement relationship of the plurality of feature points in the rectangular array under the current camera view angle. Specifically, since the calibration grid is composed of a plurality of feature points arranged at a fixed row and column pitch to form a rectangular array, and the positional relationship of the four corner feature points in the array relative to the position of the center feature point is determined, the arrangement relationship can be used to perform spatial offset along a certain direction with the field of view center point as the initial reference position, thereby obtaining the pixel region corresponding to the entire rectangular grid. As shown in FIG. 4, the step S42 can include: Figure 5

[0090] Step S421, offset calculation is performed based on the shortest distance and the arrangement relationship of the four corner feature points and the center feature point in the rectangular array, with the field of view center point as the reference point, to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points, respectively.

[0091] In this embodiment, the field of view center point corresponds to the center feature point, and then offset calculation can be performed based on the shortest distance between the field of view center point and the spreader edge, and the arrangement relationship of the four corner feature points and the center feature point in the rectangular array to determine the position of the calibration region. Specifically, taking the 9*5 rectangular array containing 45 feature points as an example, the pixel coordinates P center ​As an initial reference point, the offset relationship of the two corner feature points (for example, the upper left corner and the lower right corner) on the diagonal relative to the center point P is calculated in combination with the pixel spacing box_len between each feature point in the rectangular array, so as to obtain a coordinate set K composed of the coordinates of the two corner pixel points on the diagonal: center

[0092]

[0093] Then, according to the arrangement rule of the rectangular array, the pixel coordinates of the remaining two corner feature points, for example, the pixel coordinates of the upper right corner and the lower left corner, are derived, so as to finally determine the complete coordinate set of the four corner pixel points. The four corner pixel points jointly enclose the pixel region of the calibration grid under the current image view angle, providing a basis for subsequent image region extraction and ground coordinate mapping.

[0094] Step S422, based on the pixel coordinates of the four corner pixel points, the pixel region corresponding to the calibration grid under the current view angle of the camera is obtained.

[0095] In this embodiment, after obtaining the coordinates of the four corner pixel points, the complete pixel region corresponding to the calibration grid under the current view angle of the camera can be further determined based on the region enclosed by the four corner pixel points. The pixel region contains the pixel positions corresponding to all feature points in the image in the rectangular array, forming a pixel grid region with a determined length and width boundary.

[0096] Step S5, performing ground coordinate conversion on the pixel coordinates of each pixel point in the pixel region to obtain the ground coordinates of the pixel points.

[0097] In this embodiment, based on the intrinsic parameters, distortion parameters, pose information, and ground height of the camera, etc., the pixel coordinates of each pixel point in the calibration grid pixel region in the image are converted into the spatial position under the actual ground coordinate system through analysis and calculation, so as to establish the geometric correspondence between the image coordinates and the actual calibration layout. Specifically, as shown in Figure 6

[0098] Step S51, obtaining the intrinsic matrix, distortion parameters, and pose parameters of the camera.

[0099] ​​In this embodiment, in order to realize accurate mapping of pixel coordinates in the image to ground coordinates, the camera needs to be calibrated offline first to obtain the intrinsic matrix A, distortion parameters D, and pose parameters [R|t] of the camera coordinate system relative to the world coordinate system for subsequent geometric conversion calculation. The camera can be calibrated offline in advance by Zhang Zhengyou calibration method. Specifically, a black and white checkerboard calibration plate can be arranged below the camera in advance, and the black and white checkerboard calibration plate is photographed from multiple different angles and different postures, and the checkerboard pattern is completely captured in each image. Then, corner detection is performed on each image to extract the pixel coordinates of each inner corner point of the checkerboard. Since the size and the number of grid points of the checkerboard are known, the world coordinates of each corner point can be established. Based on the world coordinates of each corner point and the corresponding image pixel coordinates, the intrinsic matrix A, the distortion parameters D, and the pose parameter matrix [R|t] are solved by using a minimization re-projection error algorithm. Specifically, the intrinsic matrix A is:

[0100]

[0101] wherein the intrinsic matrix A can be used to describe the imaging geometry of the camera, f x and f y are the focal lengths in the horizontal and vertical directions respectively, and (u0, v0) is the coordinate of the image center. Then the distortion parameters D can also be solved:

[0102] D = [k1, k2, k3, p1, p2]

[0103] wherein k1, k2, and k3 are radial distortion parameters that can correct the image edge enlargement or reduction phenomenon caused by the spherical structure of the lens, and p1 and p2 are tangential distortion parameters that can correct the image offset caused by factors such as lens assembly asymmetry. Further, a perspective projection model can be established based on the projection relationship between the image coordinates of the detected checkerboard corner points in each image and their physical coordinates in the world coordinate system, and a nonlinear least squares optimization calculation is performed to obtain the rotation matrix R and the translation vector t of the camera coordinate system relative to the world coordinate system. Thus, the camera parameters can be provided for subsequent construction of the inverse projection model from the image plane to the ground plane. The specific implementation details of the above method belong to the public knowledge and will not be described here.

[0104] In step S52, based on the intrinsic matrix, the distortion parameters, the pose parameters, and the preset height, the pixel coordinates of each pixel point in the pixel region are converted to ground coordinates by using a projection equation to obtain the ground coordinates of each pixel point.

[0105] In this embodiment, after camera calibration is completed, the pixel region corresponding to the calibration square in the image can be converted into two-dimensional coordinates in the ground coordinate system based on the geometric mapping relationship between pixels and ground points. That is, a mapping relationship is established between the image coordinate system, camera coordinate system, and ground coordinate system. Then, the projection equation is used to perform back-projection calculations on each pixel to obtain its ground coordinates. Specifically, distortion correction can first be performed on each pixel (u, v) within the pixel region using the distortion parameter D to obtain its corrected coordinates under the ideal pinhole imaging model. Subsequently, based on the corrected pixel coordinates (u, v), a mapping relationship from the image coordinate system to the ground coordinate system is constructed. This mapping can be represented by the following projection formula:

[0106]

[0107] Among them, the one on the left Z is the homogeneous form of pixel coordinates. c With a preset camera height, the first and second matrices on the right are the pixel normalization matrix and the pinhole projection matrix, respectively. The pixel normalization matrix consists of the pixel spacing dx, dy and the coordinates (u0, v0) of the image center. In the pinhole projection matrix, f is the camera focal length, and the product of the first and second matrices is the intrinsic parameter matrix A. The third matrix is ​​the camera pose parameter matrix [R|t], and the fourth matrix is ​​the ground coordinates of the pixel. The ground coordinates of each pixel can be obtained by solving this projection equation. Furthermore, since the calibration cloth is located on the ground, the Z-axis coordinate of the ground coordinates of each pixel is 0. At this time, the projection relationship can be simplified to a planar homography transformation relationship. In order to uniformly represent the mapping relationship using homogeneous coordinates, the Z-axis coordinate can be... c Replace with a scale factor s in a homogeneous coordinate system, and further simplify the above projection relationship to:

[0108]

[0109] Specifically, the distortion-free pixel coordinates (u, v) can be further normalized, and the ground coordinates (X, Y) of each pixel can be obtained by back-projecting the part of the pose matrix [R|t] that is parallel to the ground (i.e., the first two columns r1, r2 of the rotation matrix R and the translation vector t) into the camera intrinsic parameter matrix A and the pose matrix [R|t].

[0110] Step S6: Obtain the size of the calibration square based on the ground coordinates of each pixel.

[0111] In this embodiment, after obtaining the ground coordinates of each pixel point, the size of the calibration grid can be obtained based on the ground coordinates of the pixel points. Since the calibration grid is composed of regularly arranged feature points, for example, it can be a 9*5 rectangular array, and its arrangement direction (horizontal, vertical) is fixed, so the distance between adjacent feature points can be selected as the basic unit to gradually accumulate the total size of the entire calibration grid. Specifically, as shown in Figure 7 The step S6 can include:

[0112] Step S61, obtaining the distance between the ground coordinates of the two adjacent pixel points based on the ground coordinates of the two adjacent pixel points.

[0113] In this embodiment, any pair of adjacent pixel points in the horizontal or vertical direction can be selected, respectively denoted as P i =(X i ,Y i ) and P j =(X j ,Y j ), then the distance d ij of the adjacent pixel points in the ground coordinate system is:

[0114]

[0115] Step S62, determining the size of the calibration grid based on the distance between the ground coordinates of the two adjacent pixel points and the arrangement relationship of the plurality of feature points in the rectangular array.

[0116] In this embodiment, after obtaining the distance d ij of the adjacent pixel points in the ground coordinate system, the physical size of the entire calibration grid can be determined in combination with the arrangement structure of the feature points in the calibration grid. For example, if the corresponding feature points under the current view angle are a 9*5 rectangular array, then the length of the calibration grid is 8*d ij , and the width is 4*d ij .

[0117] Step S7, determining the size of the calibration cloth based on the size of the calibration grid and the preset proportion, and making the cutting device cut the base cloth to obtain the calibration cloth based on the size of the calibration cloth, and making the printing device print each feature point on the calibration cloth to form a calibration grid corresponding to the camera based on the ground coordinates of the pixel points.

[0118] In the embodiment, the area of the calibration cloth is greater than the projection area of the spreader, and the size of the calibration grid is set in proportion to the size of the calibration cloth, for example, the length of the calibration cloth is three times the length of the calibration grid, and the width of the calibration cloth is three times the width of the calibration grid. Thus, the cutting device can cut the base cloth to obtain the calibration cloth, and the printing device can print each feature point on the calibration cloth based on the ground coordinates of the pixel points to form the calibration grid corresponding to the camera. Further, the feature points in each column of the rectangular array can be printed using different colors to improve the identification and calibration efficiency of the subsequent calibration board calibration. In addition, four auxiliary alignment straight lines are arranged on the calibration cloth. Each straight line can form an L-shaped pattern, which is used to align with the projection edges of the lock core structure at the four corners of the spreader, so that the calibration cloth and the spreader can be quickly positioned and accurately aligned.

[0119] Step S8, taking the center of the calibration cloth as the symmetry point, printing the calibration grids of the remaining three corners.

[0120] In summary, the calibration cloth manufacturing method of the embodiment of the present application, on the basis of obtaining the pixel point set of the edge of the spreader, uses the center point of the field of view under the current view angle of the camera to obtain the pixel region for setting the calibration grid corresponding to the camera, then converts the pixel coordinates of each pixel point in the pixel region to ground coordinates to obtain the ground coordinates of the pixel points, and further obtains the size of the calibration grid and the calibration cloth. Thus, the calibration cloth required for calibration can be obtained conveniently and quickly.

[0121] Reference is made to the accompanying drawings Figure 8 which shows a calibration cloth calibration method for a four-eye camera of a container spreader provided by an embodiment of the present application. The method can be applied to a computer device in Figure 1 , and specifically as shown in Figure 8 , the method can include the following steps:

[0122] Step S91, laying the calibration cloth under the spreader, and controlling the spreader to rise to a preset height so that each straight line on the calibration cloth is aligned with the projection edge of the corresponding corner lock core on the spreader.

[0123] In the embodiment, in order for the camera to completely capture the picture of the calibration cloth, the spreader can be controlled to rise to a preset height, for example, the preset height can be 3-5 meters. After reaching the preset height, each straight line on the calibration cloth is aligned with the projection edge of the corresponding corner lock core on the spreader.

[0124] Step S92, controlling the four groups of cameras on the spreader to capture images of the calibration cloth under the current view angle.

[0125] In the embodiment, four corners of the calibration cloth are respectively formed with four calibration grids, each of which is provided with a rectangular array including a plurality of feature points, and the feature points in each column of the rectangular array are in different color systems. Four sets of original images of the calibration cloth at the preset height of the cameras can be obtained respectively, and then the original images of the calibration cloth are preprocessed to obtain preprocessed images. Specifically, the preprocessing can include one or more of denoising, brightness enhancement or contrast enhancement. The original image is denoised to eliminate environmental interference noise points, the original image is brightness enhanced to improve the imaging quality under low light conditions, and the contrast of the original image is enhanced to highlight the gray difference of different regions in the image, so that the calibration cloth is clearer in the image.

[0126] In step S93, the intrinsic matrix and the distortion parameters of each camera are obtained respectively, and the images captured by each camera are corrected based on the intrinsic matrix and the distortion parameters to obtain corrected images.

[0127] In the embodiment, the image can be corrected based on the intrinsic matrix A and the distortion parameters D of the camera obtained in step S51, so as to correct the pixel coordinates in the original image to the geometric coordinates conforming to the ideal pinhole imaging model. Specifically, the coordinate of each pixel point (u, v) in the pixel region of the image can be first normalized using the distortion parameters D to obtain its coordinates (x, y) on the normalized imaging plane, that is:

[0128]

[0129] wherein (u0, v0) is the coordinate of the center of the image captured by the camera in the intrinsic matrix A, f x and f y are the focal lengths in the horizontal and vertical directions of the intrinsic matrix A. Then, the normalized coordinates can be corrected based on the distortion parameters D to calculate the corrected normalized coordinates (x cor , y cor ):

[0130] r 2 =x 2 +y 2

[0131] x cor =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0132] y cor =y(1+k1r 2 +k2r4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0133] Then, the corrected coordinates are remapped back to the pixel plane to obtain the corrected pixel coordinates (u cor , v cor ):

[0134] u cor = f x x cor + u0

[0135] v cor = f y y cor + v0

[0136] Thus, by using the intrinsic matrix and the distortion parameters to correct the distortion of the images captured by each camera, all the pixel points in the images can be mapped from the image plane affected by the distortion back to the positions under the ideal imaging model, thereby completing the distortion correction of the images. The corrected images are close to the real scene in geometric structure, which is conducive to improving the accuracy of subsequent plane calibration.

[0137] In step S94, the ground coordinates of each feature point in each calibration square and the pixel coordinates of the pixel points corresponding to each feature point in each corrected image are obtained, and plane calibration calculation is performed based on the ground coordinates of each feature point and the pixel coordinates of the pixel points corresponding to each feature point, to obtain the homography matrix corresponding to each camera.

[0138] In this embodiment, after the distortion correction of the images is completed, it is necessary to further establish the mapping relationship between each feature point on the calibration cloth and the pixel plane and the ground plane. Therefore, the coordinates of each feature point in each calibration square in the ground coordinate system and the pixel coordinates corresponding thereto in the image coordinate system need to be obtained, and plane calibration is performed to obtain the homography matrix between the image coordinate system and the ground coordinate system. Specifically, as shown in FIG. 9, step S94 can include: Figure 9

[0139] In step S941, the ground coordinates of each feature point in each calibration square are obtained based on the arrangement relationship of the plurality of feature points in the calibration square.

[0140] ​In this embodiment, the coordinates of all feature points on the ground plane can be derived based on the arrangement relationship of the plurality of feature points in the calibration grid, in combination with the size information of the calibration cloth and the calibration grid that has been acquired. Specifically, the center of the calibration cloth can be taken as the origin, and the ground coordinates of each feature point can be calculated based on the spacing between each feature points. In this way, a complete set of ground coordinates can be constructed in the ground plane, providing a reference frame for subsequent matching and calculation.

[0141] In step S942, point position detection is performed on each corrected image to obtain the pixel coordinates of the pixel points corresponding to the feature points in each corrected image.

[0142] In this embodiment, point position detection can be performed on each corrected image to identify the pixel coordinates of the feature points on the calibration grid in the image. Since the image has been corrected, the feature points are regularly distributed and the edges are clear, making them easy to identify and having high detection accuracy. Therefore, the feature points in the image can be traversed, the pixel coordinates of each feature point can be determined according to the arrangement rule of the feature points in the rectangular array, and the image coordinate values can be recorded in order to construct a pixel coordinate set corresponding to the ground coordinates of the feature points. The specific implementation details of the above method belong to common knowledge and will not be described here.

[0143] In step S943, plane calibration is performed based on the ground coordinates of the feature points and the pixel coordinates of the pixel points corresponding to the feature points, and a homography matrix corresponding to each camera is obtained using the least squares method.

[0144] In this embodiment, after obtaining the ground coordinate set and the pixel coordinate set, the homography matrix can be solved. The homography matrix H can be used to describe the projection transformation relationship between the two sets of two-dimensional plane coordinates, and can be used for image mapping and coordinate transformation when image stitching is performed subsequently. Specifically, the projection transformation formula is:

[0145]

[0146] where s is a normalization scale factor used to maintain the homogeneity of the vectors on both sides, (X i , Y i ) are the ground coordinates of the i-th feature point in the ground coordinate system, and (u i , v i ) are the pixel coordinates of the i-th feature point in the image coordinate system. Further, at least four pairs of non-collinear feature point coordinates can be used to construct a linear equation, and the least squares method can be used to globally optimize and solve the equation set to obtain a homography matrix H with the smallest projection error.

[0147] In summary, the calibration cloth calibration method of this invention involves laying a calibration cloth under the spreader, using four sets of cameras on the spreader to capture images of the calibration cloth from the current viewpoint, and then performing base surface correction. Planar calibration calculations are then performed on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixels to obtain the homography matrix for each camera used in subsequent image stitching. This eliminates the need for indoor calibration, making it suitable for port spreader operations, easy to deploy, and possessing high adaptability and accuracy.

[0148] Reference manual attached Figure 10 This invention illustrates a method for stitching images from a four-eye camera on a container spreader based on calibration parameters of a calibration cloth, according to an embodiment of the present invention. This method can be applied to... Figure 1 Among the computer devices in the system, specific examples include... Figure 10 As shown, the method may include the following steps:

[0149] Step S1010: After the spreader grabs the container, control the spreader to lift it to the preset height.

[0150] In this embodiment, to ensure the camera can capture a complete image of the container, the spreader can be raised to a preset height after completing the grabbing action. This height can be set according to the camera's field of view and the actual working scenario, for example, it can be 3-5 meters, so that the camera can capture a complete image of the container at a suitable angle and distance.

[0151] Step S1020: Control the four sets of cameras on the spreader to take images of the four corners of the container from the current perspective.

[0152] Step S1030: Based on the intrinsic parameter matrix and distortion parameters of each camera, perform distortion correction on the images captured by each camera to obtain the corrected corner image of the container.

[0153] In this embodiment, distortion correction can be performed based on the camera's intrinsic parameter matrix A and distortion parameter D image obtained in step S51, thereby correcting the pixel coordinates in the original images of the container captured by each camera to geometric coordinates that conform to the ideal pinhole imaging model. Specifically, the distortion parameter D can first be used to normalize the coordinates of each pixel point (u, v) within the pixel region to obtain its coordinates (x, y) on the unnormalized imaging plane, that is:

[0154]

[0155] Where (u0, v0) are the coordinates of the center of the image captured by the camera in the intrinsic parameter matrix A, f x and f yrespectively, are the focal lengths in horizontal and vertical directions of the intrinsic matrix A. Then, the normalized coordinates can be rectified based on the distortion parameters D, to calculate the rectified normalized coordinates (x cor , y cor ):

[0156] r 2 =x 2 +y 2

[0157] x cor =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0158] y cor =y(1+k1r 2 +k2r 4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0159] Then, the rectified coordinates are remapped back to the pixel plane to obtain the rectified pixel coordinates (u cor , v cor ):

[0160] u cor =f x x cor +u0

[0161] v cor =f y y cor +v0

[0162] Thus, by rectifying the images captured by each camera using the intrinsic matrix and the distortion parameters, all the pixel points in the images can be mapped from the image plane affected by distortion back to the positions under the ideal imaging model, thereby completing the rectification of the images. The rectified images are close to the real scene of the container in geometric structure.

[0163] Step S1040, perspective transformation is performed on each rectified corner image using the homography matrix corresponding to each camera, to obtain corner orthographic images of the four corners of the container in the ground coordinate system.

[0164] In this embodiment, based on the homography matrix H corresponding to each camera obtained in step S943 and the corrected corner images of the container, perspective transformation is performed on the corrected corner images corresponding to each camera to map the corner images in the image coordinate system to the orthographic perspective under the ground coordinate system. Specifically, the pixel coordinates of the pixel points in the corrected corner images are first obtained, and then the pixel coordinates of the pixel points are perspective transformed to the ground coordinates on the ground plane by the homography matrix H, and the formula of the perspective transformation is:

[0165]

[0166] wherein s is a normalized scale factor, (u, v) is the pixel coordinates of the pixel points in the corner image, and (X, Y) is the ground coordinates of the pixel points on the ground plane. Thus, each pixel point in the corrected corner image can be mapped to the ground plane, and the corner orthographic image under the unified coordinate system is further generated.

[0167] In step S1050, the orthographic images of the four corners of the container under the ground coordinate system are spliced to obtain the complete orthographic image of the container under the ground coordinate system.

[0168] In this embodiment, after obtaining the orthographic images of the four corners of the container under the ground coordinate system, the four images with consistent perspective and unified scale can be further processed by image splicing to reconstruct the orthographic image of the whole container on the ground plane. Since all the corner images have been transformed to the unified ground coordinate system by the corresponding homography matrix, no additional scale correction or angle alignment processing is required during the splicing process. Specifically, the arrangement relationship during image splicing can be determined according to the spatial position of each corner image in the ground coordinate system. For example, the top-left corner image should be spliced in the top-left area of the whole image, and the bottom-right corner image should be spliced in the bottom-right area. According to the two-dimensional position relationship of the ground coordinate, the orthographic images are aligned and superimposed according to their actual coordinate boundaries, so as to ensure that the image accurately reflects the structural layout of the container in the real space after splicing.

[0169] In summary, the image splicing method of the embodiment of the present application can obtain the orthographic images of the four corners of the container under the ground coordinate system by using four groups of cameras to respectively capture the images of the four corners of the container under the current perspective, performing perspective transformation on each corrected corner image using the homography matrix corresponding to each camera obtained by the calibration and marking, and splicing the orthographic images of the four corners of the container under the ground coordinate system into a complete orthographic image. Perspective transformation using the homography matrix after calibration can effectively improve the image splicing accuracy.

[0170] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the ordinary meaning of such terms for a person skilled in the art to which the present application pertains. The terms "first", "second" and similar terms are used herein merely to distinguish one element from another, and are not intended to imply any order or sequence, or any importance. Similarly, the terms "one" or "a" or "an" are not limited to one, but rather mean at least one. The terms "connected" or "coupled" or similar terms are not limited to a direct connection or coupling, but also include an indirect connection or coupling, such as through an intermediate party, an electrical connection or a mechanical connection.

[0171] The above is the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for manufacturing a calibration cloth for a four-lens camera on a container spreader, characterized in that, The calibration cloth has four calibration squares formed at its four corners, and each calibration square contains a rectangular array of multiple feature points. The manufacturing method includes: Step S1: Place the base fabric below the lifting device; Step S2: Control the lifting device to rise to a preset height and then retract the lifting device to its minimum size; Step S3: Obtain the set of pixels on the edge of the lifting device from the current view of one of the cameras; Step S4: Based on the set of pixels and the center point of the field of view under the current view of the camera, obtain a pixel region for setting the calibration grid corresponding to the camera. The pixel region includes multiple pixels, each corresponding to a feature point. Step S5: Perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of the pixel; Step S6: Obtain the size of the calibration grid based on the ground coordinates of the pixel; Step S7: Determine the size of the calibration cloth based on the size of the calibration square and the preset ratio, and cut the base cloth with a cutting device based on the size of the calibration cloth to obtain the calibration cloth, and print each feature point on the calibration cloth based on the ground coordinates of the pixel to form the calibration square corresponding to the camera. Step S8: Using the center of the calibration cloth as the symmetrical point, print the calibration squares at the other three corners.

2. The method for manufacturing calibration cloth according to claim 1, characterized in that, Step S3 includes: The image is acquired from the current viewpoint of the camera and preprocessed to obtain a preprocessed image. The preprocessing includes one or more of noise reduction, brightness enhancement, or contrast enhancement. Based on the preprocessed image, a pre-trained segmentation network model is used to perform semantic segmentation on the preprocessed image to identify the edges of the lifting device in the image, and the pixels of the edges of the lifting device are grouped into the pixel set.

3. The method for manufacturing calibration cloth according to claim 2, characterized in that, Step S4 includes: Step S41: Construct a bounding box corresponding to the edge of the lifting device based on the set of pixels, and obtain the shortest distance between the center point of the field of view and the edge of the lifting device based on the positional relationship between the bounding box and the center point of the field of view; Step S42: Based on the shortest distance and the arrangement relationship of multiple feature points in the rectangular array, offset calculation is performed to obtain the pixel region.

4. The method for manufacturing calibration cloth according to claim 3, characterized in that, Step S42 includes: Step S421: Based on the shortest distance and the arrangement relationship between the four corner feature points and the center feature point in the rectangular array, offset calculation is performed with the center point of the field of view as the reference point to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points respectively; Step S422: Obtain the pixel region corresponding to the calibration square in the current view of the camera based on the pixel coordinates of the four corner pixels; The center point of the field of view corresponds to the center feature point.

5. The method for manufacturing calibration cloth according to claim 3, characterized in that, Step S5 includes: Step S51: Obtain the intrinsic parameter matrix, distortion parameters, and pose parameters of the camera; Step S52: Based on the intrinsic parameter matrix, the distortion parameter, the pose parameter, and the preset height, the pixel coordinates of each pixel in the pixel region are transformed into ground coordinates using the projection equation to obtain the ground coordinates of each pixel.

6. The method for manufacturing calibration cloth according to claim 5, characterized in that, Step S6 includes: Step S61: Obtain the distance between the ground coordinates of two adjacent pixels based on their ground coordinates; Step S62: Determine the size of the calibration grid based on the distance between the ground coordinates of two adjacent pixels and the arrangement relationship of multiple feature points in the rectangular array.

7. The method for manufacturing calibration cloth according to claim 6, characterized in that, The area of ​​the calibration cloth is larger than the projected area of ​​the lifting device, and step S7 includes: Step S71: Obtain the proportional relationship between the size of the calibration grid and the size of the calibration cloth based on the preset ratio; Step S72: Determine the size of the calibration fabric based on the proportional relationship between the size of the calibration square and the size of the calibration fabric, and cut the base fabric with a cutting device based on the size of the calibration fabric to obtain the calibration fabric; Step S73: Based on the ground coordinates of the pixel, the printing device prints each feature point on the calibration cloth to form the calibration grid corresponding to the camera.

8. The method for manufacturing calibration cloth according to claim 7, characterized in that, The calibration cloth has four right-angled lines corresponding to each of the calibration squares, and each right-angled line is aligned with the projected edge of the corner lock cylinder of the lifting device. Step S7 further includes: Based on the projection of the corner lock core of the lifting device, the printing equipment prints a plurality of right-angle markings on the calibration cloth.

9. A calibration method for a four-lens camera on a container spreader, characterized in that, include: Lay the calibration cloth under the lifting device and control the lifting device to raise it to a preset height so that each right-angle mark on the calibration cloth is aligned with the projected edge of the corresponding corner lock cylinder on the lifting device. The four cameras on the lifting device are controlled to capture images of the calibration cloth from the current perspective. The calibration cloth has four calibration squares formed at its four corners, and each calibration square contains a rectangular array of multiple feature points. The intrinsic parameter matrix and distortion parameters of each camera are obtained respectively. Based on the intrinsic parameter matrix and distortion parameters, distortion correction is performed on the images captured by each camera to obtain the corrected images. The ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image are obtained. Planar calibration calculation is performed based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel to obtain the homography matrix corresponding to each camera.

10. The calibration method for a four-eye camera on a container spreader according to claim 9, characterized in that, The step of obtaining the ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image, and performing planar calibration calculation based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel to obtain the homography matrix corresponding to each camera includes: Based on the arrangement of multiple feature points in the calibration grid, the ground coordinates of each feature point in each calibration grid are obtained respectively; Point detection is performed on each of the corrected images to obtain the pixel coordinates of the pixel points corresponding to each feature point in each of the corrected images; Planar calibration is performed based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel points, and the homography matrix corresponding to each camera is obtained using the least squares method.

11. A method for stitching images from four cameras on a container spreader based on calibration parameters of a calibration cloth, characterized in that, include: After the spreader grabs the container, control the spreader to lift it to a preset height; The four cameras on the spreader are controlled to capture images of the four corners of the container from the current viewpoint. Based on the intrinsic parameter matrix and distortion parameters of each camera, distortion correction is performed on the images captured by each camera to obtain the corrected corner image of the container; Using the homography matrix corresponding to each camera obtained by the method as described in any one of claims 9 to 10, perspective transformation is performed on each corrected corner image to obtain the corner orthographic projection images of the four corners of the container in the ground coordinate system. The orthographic projection images of the four corners of the container in the ground coordinate system are stitched together to obtain a complete orthographic projection image of the container in the ground coordinate system.

Images