Calibration cloth manufacturing method for container spreader four-eye camera, calibration method and image splicing method

Abstract

The application belongs to the technical field of port equipment, and specifically provides a calibration cloth manufacturing method for a four-camera container spreader, a calibration method and an image splicing method. The calibration cloth manufacturing method comprises the following steps: arranging a base cloth under the spreader; controlling the spreader to rise to a preset height and contract to the minimum size; obtaining a pixel point set of the edge of the spreader under the current view angle of one camera; obtaining a pixel region comprising a plurality of pixel points based on the pixel point set and the center point of the field of view under the current view angle of the camera; performing ground coordinate conversion to obtain the ground coordinates of the pixel points; obtaining the size of the calibration grid based on the ground coordinates of the pixel points; determining the size of the calibration cloth based on the size of the calibration grid and a preset ratio, cutting the calibration cloth based on the size of the calibration cloth, and printing each feature point to form a calibration grid based on the ground coordinates of the pixel points; and printing the remaining calibration grids with the center of the calibration cloth as the symmetry point. The method is convenient to deploy and has high precision.

Description

Technical Field

[0001] This invention relates to the field of port equipment technology, specifically to a method for manufacturing calibration cloth, calibration method, and image stitching method for a four-eye camera used on a container spreader. Background Technology

[0002] In smart port applications, the automation and intelligence of spreader container handling functions are crucial for enhancing market competitiveness. Currently, laser-based spreader container handling retrofit solutions suffer from high retrofit costs, system complexity, and a high false detection rate for bulging containers (side deformation). In contrast, spreader deviation detection solutions based on four-corner monitoring cameras are less expensive and suitable for large-scale applications. Before deviation detection, these cameras need calibration to ensure that images from multiple cameras can be stitched together into a complete top-view image of the spreader in the same ground coordinate system. However, current calibration methods often employ dedicated indoor calibration devices for structural calibration or self-calibration under controlled indoor lighting conditions. This means the spreader needs to be calibrated outside of normal operating environments, significantly impacting equipment efficiency. Furthermore, indoor environments have high requirements for site conditions, equipment layout, and lighting, making them unsuitable for complex operating scenarios like ports and docks. Additionally, traditional calibration methods suffer from low stitching accuracy when dealing with multiple camera layouts due to a lack of calibration assistance. Therefore, there is an urgent need for a calibration device and calibration method that is suitable for port spreader operation sites, easy to deploy, and has high adaptability and accuracy. Summary of the Invention

[0003] In view of this, the present invention provides a method for manufacturing calibration cloth, calibration method and image stitching method for a four-eye camera for a container spreader, which is easy to deploy and has high accuracy.

[0004] To solve at least one of the above-mentioned technical problems, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a method for manufacturing a calibration cloth for a four-eye camera on a container spreader. The calibration cloth has four calibration squares formed at its four corners, and each calibration square contains a rectangular array of multiple feature points. The manufacturing method includes:

[0006] Step S1: Place the base fabric below the lifting device;

[0007] Step S2: Control the lifting device to rise to the preset height and retract the lifting device to its minimum size;

[0008] Step S3: Obtain the set of pixels on the edge of the rigging from the current view of one of the cameras;

[0009] Step S4: Based on the set of pixels and the center point of the field of view under the current view of the camera, obtain the pixel region used to set the calibration grid corresponding to the camera. The pixel region includes multiple pixels, which correspond to each feature point respectively.

[0010] Step S5: Perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of the pixel;

[0011] Step S6: Obtain the size of the calibration square based on the ground coordinates of each pixel;

[0012] Step S7: Determine the size of the calibration cloth based on the size of the calibration square and the preset ratio, and cut the base cloth with the cutting device based on the size of the calibration cloth to obtain the calibration cloth, and print each feature point on the calibration cloth based on the ground coordinates of the pixel to form the calibration square corresponding to the camera.

[0013] Step S8: Using the center of the calibration cloth as the symmetrical point, print the calibration squares at the other three corners.

[0014] In one embodiment of the present invention, step S3 includes:

[0015] The image is acquired from the current viewpoint of the camera and preprocessed to obtain a preprocessed image. The preprocessing includes one or more of the following: noise reduction, brightness enhancement, or contrast enhancement.

[0016] Based on the preprocessed image, a pre-trained segmentation network model is used to perform semantic segmentation on the preprocessed image to identify the edges of the lifting device in the image, and the pixels of the lifting device edges are grouped into a pixel set.

[0017] In one embodiment of the present invention, step S4 includes:

[0018] Step S41: Construct a bounding box corresponding to the edge of the scaffold based on the set of pixels, and obtain the shortest distance from the center of the field of view to the edge of the scaffold based on the positional relationship between the bounding box and the center of the field of view;

[0019] Step S42: Calculate the offset based on the shortest distance and the arrangement of multiple feature points in the rectangular array to obtain the pixel region.

[0020] In one embodiment of the present invention, step S42 includes:

[0021] Step S421: Based on the shortest distance and the arrangement relationship between the four corner feature points and the center feature point in the rectangular array, offset calculation is performed with the center point of the field of view as the reference point to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points respectively.

[0022] Step S422: Obtain the pixel region corresponding to the calibration square in the current view of the camera based on the pixel coordinates of the four corner pixels;

[0023] Among them, the center point of the field of view corresponds to the center feature point.

[0024] In one embodiment of the present invention, step S5 includes:

[0025] Step S51: Obtain the intrinsic parameter matrix, distortion parameters, and pose parameters of the camera;

[0026] Step S52: Based on the intrinsic parameter matrix, distortion parameters, pose parameters, and preset height, use the projection equation to perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of each pixel.

[0027] In one embodiment of the present invention, step S6 includes:

[0028] Step S61: Obtain the distance between the ground coordinates of two adjacent pixels based on their ground coordinates;

[0029] Step S62: Determine the size of the calibration grid based on the distance between the ground coordinates of two adjacent pixels and the arrangement relationship of multiple feature points in the rectangular array.

[0030] In one embodiment of the present invention, the area of ​​the calibration cloth is larger than the projected area of ​​the hanging device, and step S7 includes:

[0031] Step S71: Obtain the proportional relationship between the size of the calibration grid and the size of the calibration cloth based on a preset ratio;

[0032] Step S72: Determine the size of the calibration fabric based on the proportional relationship between the size of the calibration square and the size of the calibration fabric, and use the cutting equipment to cut the base fabric to obtain the calibration fabric based on the size of the calibration fabric;

[0033] Step S73: Based on the ground coordinates of the pixels, the printing equipment prints each feature point on the calibration cloth to form a calibration grid corresponding to the camera.

[0034] In one embodiment of the present invention, four right-angled lines are provided on the calibration cloth, each corresponding to a calibration square, and each right-angled line is aligned with the projected edge of the corner lock core of the lifting device. Step S7 further includes:

[0035] Based on the projection of the corner lock core of the lifting device, the printing equipment prints multiple right-angle markings on the calibration cloth.

[0036] A second aspect of the present invention also provides a calibration method for a calibration cloth used for a four-eye camera on a container spreader, comprising:

[0037] Lay the calibration cloth under the lifting device, control the lifting device to raise to the preset height, so that each right-angle mark on the calibration cloth is aligned with the projected edge of the corresponding corner lock cylinder on the lifting device;

[0038] The control device has four sets of cameras that capture images of the calibration cloth from the current viewpoint. The four corners of the calibration cloth form four calibration squares, and each calibration square contains a rectangular array of multiple feature points.

[0039] The intrinsic parameter matrix and distortion parameters of each camera are obtained respectively. Based on the intrinsic parameter matrix and distortion parameters, distortion correction is performed on the images captured by each camera to obtain the corrected images.

[0040] The ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image are obtained. Based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel, a plane calibration calculation is performed to obtain the homography matrix corresponding to each camera.

[0041] In one embodiment of the present invention, the ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image are obtained. Planar calibration calculation is performed based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel to obtain the homography matrix corresponding to each camera, including:

[0042] Based on the arrangement of multiple feature points in the calibration grid, the ground coordinates of each feature point in each calibration grid are obtained respectively;

[0043] For each corrected image, perform point detection to obtain the pixel coordinates of the pixel points corresponding to each feature point in each corrected image;

[0044] Planar calibration is performed based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel points. The homography matrix corresponding to each camera is obtained by using the least squares method.

[0045] A third aspect of the present invention also provides a method for stitching images from four cameras on a container spreader based on calibration parameters of a calibration cloth, comprising:

[0046] After the spreader grabs the container, control the spreader to lift it to the preset height;

[0047] The four cameras on the control spreader capture images of the four corners of the container from the current viewpoint.

[0048] Based on the intrinsic parameter matrix and distortion parameters of each camera, distortion correction is performed on the images captured by each camera to obtain the corrected corner image of the container;

[0049] Using the homography matrix corresponding to each camera obtained by the method of any one of the second aspect embodiments, perspective transformation is performed on each corrected corner image to obtain the corner orthographic projection images of the four corners of the container in the ground coordinate system.

[0050] The orthographic images of the four corners of the container in the ground coordinate system are stitched together to obtain a complete orthographic image of the container in the ground coordinate system.

[0051] The above-described technical solution of the present invention has at least one of the following beneficial effects:

[0052] According to the calibration cloth manufacturing method of the present invention, based on obtaining the set of pixels on the edge of the sling, the pixel area for setting the calibration grid corresponding to the camera is obtained by using the center point of the field of view under the current view of the camera. Then, the pixel coordinates of each pixel in the pixel area are transformed into ground coordinates to obtain the ground coordinates of the pixel, and then the size of the calibration grid and the calibration cloth are obtained. Thus, the calibration cloth required for calibration can be obtained conveniently and quickly.

[0053] According to the calibration cloth calibration method of the present invention, a calibration cloth is laid under the spreader, and images of the calibration cloth are taken from the current viewpoint by four sets of cameras on the spreader. After distortion correction, planar calibration calculation is performed on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel points to obtain the homography matrix of each camera for subsequent image stitching. Thus, calibration is not required indoors, which is suitable for port spreader operation sites, easy to deploy and has high adaptability and accuracy.

[0054] According to the image stitching method of this invention, four sets of cameras are used to capture images of the four corners of a container from the current viewpoint. After image correction, the homography matrix corresponding to each camera is obtained by calibration using a calibration cloth. Perspective transformation is performed on each corrected corner image to obtain the orthographic projection images of the four corners of the container in the ground coordinate system. Thus, the orthographic projection images of the four corners of the container in the ground coordinate system are stitched into a complete orthographic projection image. By using the calibrated homography matrix for perspective transformation, the image stitching accuracy can be effectively improved. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of an implementation environment provided for one embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the structure of a calibration cloth provided in one embodiment of the present invention;

[0057] Figure 3 A flowchart illustrating a method for manufacturing calibration cloth for a four-lens camera on a container spreader according to an embodiment of the present invention;

[0058] Figure 4 A flowchart for obtaining a pixel region is provided in one embodiment of the present invention;

[0059] Figure 5 A flowchart for obtaining pixel regions based on shortest distance and arrangement relationship is provided as an embodiment of the present invention;

[0060] Figure 6 This is a flowchart of obtaining the ground coordinates of each pixel according to an embodiment of the present invention;

[0061] Figure 7 A flowchart for obtaining the size of the calibration grid is provided in one embodiment of the present invention;

[0062] Figure 8 A flowchart illustrating a calibration method for a four-lens camera on a container spreader provided in one embodiment of the present invention;

[0063] Figure 9 A flowchart for obtaining the homography matrix corresponding to each camera is provided in one embodiment of the present invention;

[0064] Figure 10 This is a flowchart of a method for stitching images from a four-eye camera on a container spreader based on calibration parameters of a calibration cloth, provided as an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0066] Reference manual attached Figure 1 This illustrates a schematic diagram of an implementation environment provided by an embodiment of the present invention for manufacturing a calibration cloth for a four-lens camera on a container spreader. For example... Figure 1 As shown, the implementation environment may include a container spreader 100 and a computer device 120. A camera 110 is respectively installed at each of the four corners of the container spreader 100. The container spreader 100 and the computer device 120 can be directly or indirectly connected through wired or wireless communication. This embodiment of the invention does not limit this.

[0067] The computer device 120 may be, but is not limited to, various servers, personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be an independent server or a server cluster or distributed system composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0068] like Figure 1 and Figure 2 As shown, in this embodiment of the invention, the base fabric can first be placed below the container spreader 100. Then, the computer device 120 controls the container spreader 100 to rise to a preset height and retracts the container spreader 100 to its minimum size. A camera 110 at any corner of the container spreader 100 captures an image of the base fabric below the container spreader 100 from the current viewpoint of the camera 110. The computer device 120 can then identify and acquire the set of pixels at the edge of the container spreader 100 from the current viewpoint. Based on the set of pixels and the center point of the field of view of the camera 110, it acquires a pixel region for setting the calibration grid 210 corresponding to the camera 110. Then, it performs ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of the pixels. After obtaining the ground coordinates of the pixels, the size of the calibration grid 210 is obtained based on the ground coordinates of the pixels. After obtaining the dimensions of the calibration grid 210, since the dimensions of the calibration grid 210 are proportional to the dimensions of the calibration cloth 200, the base fabric can be cut using a cutting device based on the dimensions of the calibration cloth 200 to obtain the calibration cloth 200. Then, based on the ground coordinates of the pixels, a printing device prints the feature points 211 at each corner of the calibration cloth 200 with the center of the calibration cloth 200 as the symmetrical point, thereby forming the corresponding calibration grid 210. Thus, the calibration cloth 200 required for calibration can be obtained conveniently. By using the calibration cloth 200, the container spreader 100 does not need to be calibrated indoors, making it suitable for port operations, easy to deploy, and possessing high adaptability and accuracy.

[0069] like Figure 2 As shown, in one embodiment of the present invention, the calibration cloth 200 has four calibration squares 210 formed at its four corners, and each calibration square 210 contains a rectangular array including multiple feature points 211. (See attached specification.) Figure 3 This document illustrates the process of manufacturing a calibration cloth for a four-lens camera on a container spreader according to an embodiment of the present invention. This method can be applied to... Figure 1 In the computer device, specifically, the method may include the following steps:

[0070] Step S1: Place the base fabric below the lifting device.

[0071] Step S2: Control the lifting device to rise to the preset height and retract the lifting device to its minimum size.

[0072] In this embodiment, in order for the camera to capture the entire image of the base fabric corresponding to one corner of the calibration fabric, the hoist can be raised to a preset height, such as 3-5 meters. Furthermore, by minimizing the hoist's size, obstruction of the camera's field of view can be avoided.

[0073] Step S3: Obtain the set of pixels on the edge of the rigging from the current view of one of the cameras.

[0074] In this embodiment, the original image at the current preset height of the camera can be acquired first, and then the image can be preprocessed to obtain a preprocessed image. Specifically, the preprocessing can include one or more of denoising, brightness enhancement, or contrast enhancement. Denoising the original image can eliminate environmental noise, brightness enhancement can improve the imaging quality under low light conditions, and contrast enhancement can highlight the grayscale differences in different areas of the image, thereby making the outline of the rigging clearer in the image.

[0075] Then, based on the preprocessed image, a pre-trained segmentation network model is used to perform semantic segmentation on the preprocessed image to identify the edges of the lifting device in the image, and the pixels of the lifting device edges are grouped into a pixel set. Specifically, this segmentation network model is built based on a deep convolutional neural network, which can output the semantic category label of each pixel at the pixel level. The semantic category label can include at least three categories: "lifting device edge", "lifting device body", and "background", where the "lifting device edge" category is used to identify pixels at the outer contour line of the lifting device. In a specific embodiment of the present invention, for example, the preprocessed image I can be... pre (u, v) are input into the segmentation network model f θ In this context, the segmentation network model can segment and identify the lifting device through feature extraction, feature fusion, and multi-scale prediction to obtain the semantic segmentation result M(u, v). The segmentation network model f θ The process of semantic segmentation can be represented as:

[0076] M(u, v) = f θ (I pre (u, v))

[0077] The semantic segmentation result M(u, v) is a two-dimensional label matrix with the same resolution as the original image, and M(u, v) ∈ {0, 1, 2}. It can be used to represent the category label of pixels (u, v), for example, 0 represents the background, 1 represents the main body of the lifting device, and 2 represents the edge of the lifting device. Furthermore, to extract the edge region of the lifting device, the coordinates of all pixels satisfying M(u, v) = 2 can be extracted to form a set of unclustered pixels ε.

[0078] ε={(u,v)|M(u,v)=2}

[0079] Then, connectivity analysis can be performed on the pixels of the unclustered pixel set ε to spatially cluster them, removing isolated or misidentified edge points, thus obtaining the pixel set ε′ of the rigging edge from the camera's current viewpoint. This provides a reference for obtaining the precise boundary of the calibration grid in subsequent steps.

[0080] Step S4: Based on the set of pixels and the center point of the field of view under the current view of the camera, obtain the pixel area used to set the calibration square corresponding to the camera.

[0081] In this embodiment, each calibration square contains multiple feature points, and these feature points form a regular rectangular array with fixed spacing, meaning their arrangement and relative positions are fixed. For example, a 9*5 rectangular array can be set within the calibration square, containing 45 feature points, with consistent horizontal and vertical spacing between each feature point. Based on this, the shortest distance from the center of the field of view to the edge of the rigging can be obtained based on the set of pixel points at the edge of the rigging acquired from the camera's current viewpoint and the center of the camera's field of view. Offset calculations are then performed based on this shortest distance and the arrangement of multiple feature points in the rectangular array to obtain a pixel region. This pixel region includes multiple pixels, and each pixel corresponds one-to-one with a feature point in the calibration square, thus providing accurate pixel positioning for subsequent ground coordinate transformation and calibration cloth printing. Specifically, as... Figure 4 As shown, step S4 may include:

[0082] Step S41: Construct a bounding box corresponding to the edge of the rigging based on the set of pixels, and obtain the shortest distance from the center of the field of view to the edge of the rigging based on the positional relationship between the bounding box and the center of the field of view.

[0083] In this embodiment, after obtaining the pixel set of the lifting device's edge, a bounding box corresponding to the lifting device's edge can be constructed based on this pixel set. Then, based on the resolution information of the image under the camera's current viewpoint, the horizontal pixel width and vertical pixel height of the image are obtained, thereby determining the pixel coordinates P of the center point of the field of view under the camera's current viewpoint. center :

[0084] P center = (width / 2, height / 2)

[0085] Then, based on the pixel coordinates P of the center point of the field of view center Calculate the Euclidean distance between the camera's field of view center point and all pixels on the edge of the rigging, using the set of pixels ε′ along with the edge of the rigging. Then, find the minimum value among these distances and take it as the shortest distance r from the field of view center point to the edge of the rigging.

[0086] r = min(distance(P) center ,ε′))

[0087] Where, distance(P) center ε′) can represent the distance between each pixel point at the center of the field of view and the edge of the rigging. The shortest distance r obtained can be used to reflect the spatial relationship between the center of the field of view and the rigging under the current view of the camera, providing a basis for subsequent offset calculations to obtain the pixel area corresponding to the calibration grid under the current view.

[0088] Step S42: Calculate the offset based on the shortest distance and the arrangement of multiple feature points in the rectangular array to obtain the pixel region.

[0089] In this embodiment, to determine the pixel region corresponding to the calibration square in the image, offset calculation is required based on the shortest distance and the arrangement relationship of multiple feature points in the rectangular array under the current camera viewpoint. Specifically, since the calibration square is a rectangular array composed of multiple feature points arranged at fixed row and column intervals, and the positional relationship of the four corner feature points relative to the center feature point is determined, this arrangement relationship can be used to perform spatial offset along a certain direction, with the center point of the field of view as the initial reference position, thereby obtaining the pixel region corresponding to the entire rectangular square. Figure 5 As shown, step S42 may include:

[0090] Step S421: Based on the shortest distance and the arrangement relationship between the four corner feature points and the center feature point in the rectangular array, offset calculation is performed with the center point of the field of view as the reference point to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points respectively.

[0091] In this embodiment, the center point of the field of view corresponds to the center feature point. Therefore, the location of the calibration area can be determined by offset calculation based on the shortest distance between the center point of the field of view and the edge of the rigging, and the arrangement relationship between the four corner feature points and the center feature point in the rectangular array. Specifically, taking a 9*5 rectangular array containing 45 feature points as an example, the pixel coordinates P of the center point of the image under the current camera view can be used as the starting point. centerUsing the initial reference point and the pixel spacing box_len between each feature point, the coordinates of the two diagonal corner feature points (e.g., the top left and bottom right corners) relative to the center point P in the rectangular array are calculated. center The offset relationship is used to obtain the coordinate set K consisting of the coordinates of the two corner pixels of the diagonal:

[0092]

[0093] Then, based on the arrangement rules of the rectangular array, the pixel coordinates of the remaining two corner feature points are derived, such as the pixel coordinates of the upper right and lower left corner feature points, thus finally determining the complete coordinate set of the four corner pixels. The four corner pixels together enclose the pixel region of the calibration grid in the current image viewpoint, providing a basis for subsequent image region extraction and ground coordinate mapping.

[0094] Step S422: Obtain the pixel region corresponding to the calibration grid in the current view of the camera based on the pixel coordinates of the four corner pixels.

[0095] In this embodiment, after obtaining the coordinates of the four corner pixels, the complete pixel region corresponding to the calibration grid under the current viewpoint of the camera can be further determined based on the area enclosed by the four corner pixels. This pixel region includes the pixel positions of all feature points in the rectangular array in the image, forming a pixel grid region with defined length and width boundaries.

[0096] Step S5: Perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of the pixel.

[0097] In this embodiment, based on camera intrinsic parameters, distortion parameters, pose information, and ground height parameters, analytical calculations can be performed to convert the coordinates of each pixel within the calibration grid pixel area in the image into its spatial position in the actual ground coordinate system, thereby establishing a geometric correspondence between the image coordinates and the actual calibration grid. Specifically, as... Figure 6 As shown, step S5 may include:

[0098] Step S51: Obtain the intrinsic parameter matrix, distortion parameters, and pose parameters of the camera.

[0099] In this embodiment, to achieve accurate mapping of pixel coordinates in the image to ground coordinates, the camera needs to be calibrated offline first to obtain the intrinsic parameter matrix A, distortion parameter D, and pose parameter [R|t] of the camera coordinate system relative to the world coordinate system for subsequent geometric transformation calculations. This can be achieved by pre-calibrating the camera offline using the Zhang Zhengyou calibration method. Specifically, a black and white checkerboard calibration board can be placed below the camera, and the board can be photographed from multiple different angles and postures, ensuring that the checkerboard pattern is fully captured in each image. Then, corner detection is performed on each image to extract the pixel coordinates of each inner corner point on the checkerboard. Since the size and number of checkerboard points are known, the world coordinates of each corner point can be established. Based on the world coordinates of each corner point and its corresponding image pixel coordinates, the intrinsic parameter matrix A, distortion parameter D, and pose parameter matrix [R|t] are solved using the algorithm for minimizing reprojection error. Specifically, the intrinsic parameter matrix A is:

[0100]

[0101] Among them, the intrinsic parameter matrix A can be used to describe the imaging geometry of the camera, f x and f y These are the focal lengths in the horizontal and vertical directions, respectively, and (u0, v0) are the coordinates of the image center. The distortion parameter D can then be calculated.

[0102] D = [k1, k2, k3, p1, p2]

[0103] Where k1, k2, and k3 are radial distortion parameters, which can correct the magnification or reduction of image edges caused by the spherical structure of the lens, and p1 and p2 are tangential distortion parameters, which can correct image shifts caused by factors such as lens assembly asymmetry. Furthermore, a perspective projection model can be established based on the projection relationship between the image coordinates of the detected checkerboard corner points in each image and their physical coordinates in the world coordinate system. Nonlinear least squares optimization calculations are then performed to obtain the rotation matrix R and translation vector t of the camera coordinate system relative to the world coordinate system. This provides camera parameters for subsequently constructing a back-projection model from the image plane to the ground plane. The specific implementation details of the above method are common knowledge and will not be elaborated here.

[0104] Step S52: Based on the intrinsic parameter matrix, distortion parameters, pose parameters, and preset height, use the projection equation to perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of each pixel.

[0105] In this embodiment, after camera calibration is completed, the pixel region corresponding to the calibration square in the image can be converted into two-dimensional coordinates in the ground coordinate system based on the geometric mapping relationship between pixels and ground points. That is, a mapping relationship is established between the image coordinate system, camera coordinate system, and ground coordinate system. Then, the projection equation is used to perform back-projection calculations on each pixel to obtain its ground coordinates. Specifically, distortion correction can first be performed on each pixel (u, v) within the pixel region using the distortion parameter D to obtain its corrected coordinates under the ideal pinhole imaging model. Subsequently, based on the corrected pixel coordinates (u, v), a mapping relationship from the image coordinate system to the ground coordinate system is constructed. This mapping can be represented by the following projection formula:

[0106]

[0107] Among them, the one on the left Z is the homogeneous form of pixel coordinates. c With a preset camera height, the first and second matrices on the right are the pixel normalization matrix and the pinhole projection matrix, respectively. The pixel normalization matrix consists of the pixel spacing dx, dy and the coordinates (u0, v0) of the image center. In the pinhole projection matrix, f is the camera focal length, and the product of the first and second matrices is the intrinsic parameter matrix A. The third matrix is ​​the camera pose parameter matrix [R|t], and the fourth matrix is ​​the ground coordinates of the pixel. The ground coordinates of each pixel can be obtained by solving this projection equation. Furthermore, since the calibration cloth is located on the ground, the Z-axis coordinate of the ground coordinates of each pixel is 0. At this time, the projection relationship can be simplified to a planar homography transformation relationship. In order to uniformly represent the mapping relationship using homogeneous coordinates, the Z-axis coordinate can be... c Replace with a scale factor s in a homogeneous coordinate system, and further simplify the above projection relationship to:

[0108]

[0109] Specifically, the distortion-free pixel coordinates (u, v) can be further normalized, and the ground coordinates (X, Y) of each pixel can be obtained by back-projecting the part of the pose matrix [R|t] that is parallel to the ground (i.e., the first two columns r1, r2 of the rotation matrix R and the translation vector t) into the camera intrinsic parameter matrix A and the pose matrix [R|t].

[0110] Step S6: Obtain the size of the calibration square based on the ground coordinates of each pixel.

[0111] In this embodiment, after obtaining the ground coordinates of each pixel, the size of the calibration grid can be obtained based on the ground coordinates of the pixels. Since the calibration grid is composed of regularly arranged feature points, such as a 9*5 rectangular array with a fixed arrangement direction (horizontal or vertical), the distance between adjacent feature points can be selected as the basic unit, and the total size of the entire calibration grid can be obtained by gradually accumulating these distances. Specifically, as shown... Figure 7 As shown, step S6 may include:

[0112] Step S61: Obtain the distance between the ground coordinates of two adjacent pixels based on their ground coordinates.

[0113] In this embodiment, any pair of adjacent pixels in the horizontal or vertical direction can be selected and denoted as P. i =(X i ,Y i ) and P j =(X j ,Y j If ), then the distance d between adjacent pixels in the ground coordinate system is... ij for:

[0114]

[0115] Step S62: Determine the size of the calibration grid based on the distance between the ground coordinates of two adjacent pixels and the arrangement relationship of multiple feature points in the rectangular array.

[0116] In this embodiment, the distance d between adjacent pixels in the ground coordinate system is obtained. ij Then, the overall physical dimensions of the calibration grid can be determined by combining the arrangement of feature points within the grid. For example, if the feature points corresponding to the current viewpoint are a 9*5 rectangular array, then the length of the calibration grid is 8*d. ij Width is 4*d ij .

[0117] Step S7: Determine the size of the calibration cloth based on the size of the calibration square and the preset ratio, and cut the base cloth with the cutting device based on the size of the calibration cloth to obtain the calibration cloth, and print each feature point on the calibration cloth based on the ground coordinates of the pixel to form the calibration square corresponding to the camera.

[0118] In this embodiment, the area of ​​the calibration cloth is larger than the projected area of ​​the lifting device, and the size of the calibration square is proportional to the size of the calibration cloth. For example, the length of the calibration cloth is three times the length of the calibration square, and the width of the calibration cloth is three times the width of the calibration square. This allows the cutting device to cut the base cloth to obtain the calibration cloth, and the printing device to print each feature point on the calibration cloth based on the ground coordinates of the pixels, thereby forming a calibration square corresponding to the camera. Furthermore, the feature points in each column of the rectangular array can be printed with different colors to improve the recognition and calibration efficiency of the subsequent calibration plate. In addition, four right-angled markings for auxiliary alignment are also provided on the calibration cloth. Each right-angled marking can form an L-shaped pattern for alignment with the projected edges of the lock cylinder structure at the four corners of the lifting device, thereby enabling rapid positioning and precise alignment of the calibration cloth and the lifting device.

[0119] Step S8: Using the center of the calibration cloth as the symmetrical point, print the calibration squares at the other three corners.

[0120] In summary, the calibration cloth manufacturing method of this invention, based on obtaining the set of pixels on the edge of the rigging, uses the center point of the field of view under the current view of the camera to obtain the pixel area for setting the calibration grid corresponding to the camera, and then performs ground coordinate transformation on the pixel coordinates of each pixel in the pixel area to obtain the ground coordinates of the pixel, thereby obtaining the calibration grid and the size of the calibration cloth. This allows for convenient and quick acquisition of the calibration cloth required for calibration.

[0121] Reference manual attached Figure 8 This invention illustrates a calibration method for a four-eye camera on a container spreader provided by an embodiment of the present invention. This method can be applied to... Figure 1 Among the computer devices in the system, specific examples include... Figure 8 As shown, the method may include the following steps:

[0122] Step S91: Lay the calibration cloth under the lifting device and control the lifting device to raise it to the preset height so that each right-angle mark on the calibration cloth is aligned with the projected edge of the corresponding corner lock core on the lifting device.

[0123] In this embodiment, in order for the camera to capture a complete image of the calibration cloth, the lifting device can be controlled to rise to a preset height, such as 3-5 meters. After reaching the preset height, each right-angle mark on the calibration cloth is aligned with the projected edge of the corresponding corner lock cylinder on the lifting device.

[0124] Step S92: Control the four sets of cameras on the lifting device to take images of the fixed cloth from the current viewpoint.

[0125] In this embodiment, four calibration squares are formed at the four corners of the calibration cloth. Each calibration square contains a rectangular array of multiple feature points, and the feature points in each column of the rectangular array are of different colors. Original images of the calibration cloth at the current preset camera height can be acquired separately for four sets of cameras. These original images are then preprocessed to obtain preprocessed images. Specifically, preprocessing can include one or more of denoising, brightness enhancement, or contrast enhancement. Denoising the original image eliminates environmental noise; brightness enhancement improves imaging quality under low light conditions; and contrast enhancement highlights grayscale differences in different areas of the image, making the calibration cloth clearer in the image.

[0126] Step S93: Obtain the intrinsic parameter matrix and distortion parameters of each camera respectively, and perform distortion correction on the images captured by each camera based on the intrinsic parameter matrix and distortion parameters to obtain the corrected images.

[0127] In this embodiment, distortion correction can be performed on the image based on the camera's intrinsic parameter matrix A and distortion parameter D obtained in step S51, thereby correcting the pixel coordinates in the original image to geometric coordinates that conform to the ideal pinhole imaging model. Specifically, the distortion parameter D can first be used to normalize the coordinates of each pixel (u, v) within the pixel region of the image to obtain its coordinates (x, y) on the unnormalized imaging plane, that is:

[0128]

[0129] Where (u0, v0) are the coordinates of the center of the image captured by the camera in the intrinsic parameter matrix A, f x and f y These represent the focal lengths in the horizontal and vertical directions of the intrinsic parameter matrix A, respectively. Next, distortion correction can be applied to the normalized coordinates based on the distortion parameter D, and the corrected normalized coordinates (x, y, d) can be calculated. cor y cor ):

[0130] r 2 =x 2 +y 2

[0131] x cor =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0132] y cor =y(1+k1r 2 +k2r4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0133] Then, the corrected coordinates are remapped back to the pixel plane to obtain the corrected pixel coordinates (u). cor v cor ):

[0134] u cor =f x x cor +u0

[0135] v cor =f y y cor +v0

[0136] Therefore, by using intrinsic parameter matrices and distortion parameters to correct the distortion of images captured by each camera, all pixels in the image can be mapped back to their positions under the ideal imaging model from the image plane affected by distortion, thus completing the image distortion correction. The corrected image is geometrically closer to the real scene, which helps improve the accuracy of subsequent planar calibration.

[0137] Step S94: Obtain the ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image. Perform planar calibration calculation based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel to obtain the homography matrix for each camera.

[0138] In this embodiment, after completing image distortion correction, it is necessary to further establish the mapping relationship between each feature point on the calibration cloth and the pixel plane and the ground plane. Therefore, it is necessary to obtain the coordinates of each feature point in each calibration square in the ground coordinate system and the corresponding pixel coordinates in the image coordinate system, and perform planar calibration to obtain the homography matrix between the image coordinate system and the ground coordinate system. Specifically, as shown... Figure 9 As shown, step S94 may include:

[0139] Step S941: Based on the arrangement of multiple feature points in the calibration grid, obtain the ground coordinates of each feature point in each calibration grid.

[0140] In this embodiment, the coordinates of all feature points on the ground plane can be derived based on the arrangement of multiple feature points in the calibration grid, combined with the obtained size information of the calibration cloth and calibration grid. Specifically, the center of the calibration cloth can be used as the origin, and the ground coordinates of each feature point can be calculated based on the spacing between each feature point. Thus, a complete set of ground coordinates can be constructed on the ground plane, providing a reference system for subsequent matching and calculation.

[0141] Step S942: Perform point detection on each corrected image to obtain the pixel coordinates of the pixel points corresponding to each feature point in each corrected image.

[0142] In this embodiment, point detection can be performed on each corrected image to identify the pixel coordinates of each feature point on the marked square in the image. Since the image has been corrected, the feature points are regularly distributed with clear edges, making them easy to identify and resulting in high detection accuracy. Therefore, the feature points in the image can be traversed, and the pixel coordinates of each feature point can be determined according to the arrangement rules of the feature points in the rectangular array. Their image coordinate values ​​are recorded sequentially, constructing a set of pixel coordinates corresponding to the ground coordinates of the feature points. The specific implementation details of the above method are common knowledge and will not be elaborated here.

[0143] Step S943: Perform plane calibration based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel points, and use the least squares method to obtain the homography matrix corresponding to each camera.

[0144] In this embodiment, after obtaining the ground coordinate set and pixel coordinate set, the homography matrix can be solved. The homography matrix H can be used to describe the projection transformation relationship between the two sets of two-dimensional planar coordinates, and can be used for image mapping and coordinate transformation during subsequent image stitching. Specifically, the projection transformation formula is:

[0145]

[0146] Where s is the normalization scaling factor, used to maintain the homogeneity of the two-sided vectors, (X i Y i Let (u) be the ground coordinates of the i-th feature point in the ground coordinate system. i v i Let be the pixel coordinates of the i feature points in the image coordinate system. Further, a linear equation can be constructed using at least four pairs of non-collinear feature point coordinates. The above equations can then be solved globally using the least squares method to obtain the homography matrix H with the minimum projection error.

[0147] In summary, the calibration cloth calibration method of this invention involves laying a calibration cloth under the spreader, using four sets of cameras on the spreader to capture images of the calibration cloth from the current viewpoint, and then performing base surface correction. Planar calibration calculations are then performed on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixels to obtain the homography matrix for each camera used in subsequent image stitching. This eliminates the need for indoor calibration, making it suitable for port spreader operations, easy to deploy, and possessing high adaptability and accuracy.

[0148] Reference manual attached Figure 10 This invention illustrates a method for stitching images from a four-eye camera on a container spreader based on calibration parameters of a calibration cloth, according to an embodiment of the present invention. This method can be applied to... Figure 1 Among the computer devices in the system, specific examples include... Figure 10 As shown, the method may include the following steps:

[0149] Step S1010: After the spreader grabs the container, control the spreader to lift it to the preset height.

[0150] In this embodiment, to ensure the camera can capture a complete image of the container, the spreader can be raised to a preset height after completing the grabbing action. This height can be set according to the camera's field of view and the actual working scenario, for example, it can be 3-5 meters, so that the camera can capture a complete image of the container at a suitable angle and distance.

[0151] Step S1020: Control the four sets of cameras on the spreader to take images of the four corners of the container from the current perspective.

[0152] Step S1030: Based on the intrinsic parameter matrix and distortion parameters of each camera, perform distortion correction on the images captured by each camera to obtain the corrected corner image of the container.

[0153] In this embodiment, distortion correction can be performed based on the camera's intrinsic parameter matrix A and distortion parameter D image obtained in step S51, thereby correcting the pixel coordinates in the original images of the container captured by each camera to geometric coordinates that conform to the ideal pinhole imaging model. Specifically, the distortion parameter D can first be used to normalize the coordinates of each pixel point (u, v) within the pixel region to obtain its coordinates (x, y) on the unnormalized imaging plane, that is:

[0154]

[0155] Where (u0, v0) are the coordinates of the center of the image captured by the camera in the intrinsic parameter matrix A, f x and f yThese represent the focal lengths in the horizontal and vertical directions of the intrinsic parameter matrix A, respectively. Next, distortion correction can be applied to the normalized coordinates based on the distortion parameter D, and the corrected normalized coordinates (x, y, d) can be calculated. cor y cor ):

[0156] r 2 =x 2 +y 2

[0157] x cor =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0158] y cor =y(1+k1r 2 +k2r 4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0159] Then, the corrected coordinates are remapped back to the pixel plane to obtain the corrected pixel coordinates (u). cor v cor ):

[0160] u cor =f x x cor +u0

[0161] v cor =f y y cor +v0

[0162] Therefore, by using intrinsic parameter matrices and distortion parameters to correct the distortion of images captured by each camera, all pixels in the image can be mapped back to their positions under the ideal imaging model from the image plane affected by distortion, thus completing the image distortion correction. The corrected image is geometrically close to the real scene of the container.

[0163] Step S1040: Using the homography matrix corresponding to each camera, perform perspective transformation on each corrected corner image to obtain the corner orthographic projection images of the four corners of the container in the ground coordinate system.

[0164] In this embodiment, perspective transformation can be performed on the homography matrix H corresponding to each camera obtained in step S943, the corrected corner image of the container, and the corrected corner image corresponding to each camera, to uniformly map the corner image in the image coordinate system to the orthographic projection viewpoint in the ground coordinate system. Specifically, the pixel coordinates of the pixels in the corrected corner image can be obtained first, and then the pixel coordinates of the pixels can be perspective transformed to their ground coordinates on the ground plane through the homography matrix H. The formula for perspective transformation is:

[0165]

[0166] Where s is the normalized scaling factor, (u, v) are the pixel coordinates of the pixels in the corner image, and (X, Y) are the ground coordinates of the corresponding pixels on the ground plane. Thus, each pixel in the corrected corner image can be mapped to the ground plane, thereby generating a corner orthographic projection image in a unified coordinate system.

[0167] Step S1050: Stitch together the orthographic projection images of the four corners of the container in the ground coordinate system to obtain a complete orthographic projection image of the container in the ground coordinate system.

[0168] In this embodiment, after acquiring the orthographic projection images of the four corners of the container in the ground coordinate system, these four images with consistent perspectives and uniform scale can be further stitched together to reconstruct the orthographic projection image of the entire container on the ground plane. Since all corner images have been transformed to a unified ground coordinate system through the corresponding homography matrix, no additional scale correction or angle alignment processing is required during the stitching process. Specifically, the arrangement relationship during image stitching can be determined first based on the spatial position of each corner image in the ground coordinate system. For example, the upper left corner image should be stitched into the upper left region of the overall image, and the lower right corner image should be stitched into the lower right region. Based on the two-dimensional positional relationship corresponding to the ground coordinates, each orthographic projection image can be aligned and superimposed according to its actual coordinate boundaries, thereby ensuring that the stitched image accurately reflects the structural layout of the container in real space.

[0169] In summary, the image stitching method of this invention uses four sets of cameras to capture images of the four corners of a container from the current viewpoint. After image correction, the homography matrix corresponding to each camera is obtained by calibration using a calibration cloth. Perspective transformation is then performed on each corrected corner image to obtain the orthographic projection images of the four corners of the container in the ground coordinate system. This allows the orthographic projection images of the four corners of the container in the ground coordinate system to be stitched together into a complete orthographic projection image. By using the calibrated homography matrix for perspective transformation, the image stitching accuracy can be effectively improved.

[0170] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0171] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for manufacturing a calibration cloth for a four-lens camera on a container spreader, characterized in that, The calibration cloth has four calibration squares formed at its four corners, and each calibration square contains a rectangular array of multiple feature points. The manufacturing method includes: Step S1: Place the base fabric below the lifting device; Step S2: Control the lifting device to rise to a preset height and then retract the lifting device to its minimum size; Step S3: Obtain the set of pixels on the edge of the lifting device from the current view of one of the cameras; Step S4: Based on the set of pixels and the center point of the field of view under the current view of the camera, obtain a pixel region for setting the calibration grid corresponding to the camera. The pixel region includes multiple pixels, each corresponding to a feature point. Step S5: Perform ground coordinate transformation on the pixel coordinates of each pixel in the pixel region to obtain the ground coordinates of the pixel; Step S6: Obtain the size of the calibration grid based on the ground coordinates of the pixel; Step S7: Determine the size of the calibration cloth based on the size of the calibration square and the preset ratio, and cut the base cloth with a cutting device based on the size of the calibration cloth to obtain the calibration cloth, and print each feature point on the calibration cloth based on the ground coordinates of the pixel to form the calibration square corresponding to the camera. Step S8: Using the center of the calibration cloth as the symmetrical point, print the calibration squares at the remaining three corners; Step S4 includes: Step S41: Construct a bounding box corresponding to the edge of the lifting device based on the set of pixels, and obtain the shortest distance between the center point of the field of view and the edge of the lifting device based on the positional relationship between the bounding box and the center point of the field of view; Step S42: Based on the shortest distance and the arrangement relationship of multiple feature points in the rectangular array, perform offset calculation to obtain the pixel region; The calibration cloth has four right-angled lines corresponding to each of the calibration squares, and each right-angled line is aligned with the projected edge of the corner lock cylinder of the lifting device. Step S7 further includes: Based on the projection of the corner lock core of the lifting device, the printing equipment prints a plurality of right-angle markings on the calibration cloth.

2. The method for manufacturing calibration cloth according to claim 1, characterized in that, Step S3 includes: The image is acquired from the current viewpoint of the camera and preprocessed to obtain a preprocessed image. The preprocessing includes one or more of noise reduction, brightness enhancement, or contrast enhancement. Based on the preprocessed image, a pre-trained segmentation network model is used to perform semantic segmentation on the preprocessed image to identify the edges of the lifting device in the image, and the pixels of the edges of the lifting device are grouped into the pixel set.

3. The method for manufacturing calibration cloth according to claim 1, characterized in that, Step S42 includes: Step S421: Based on the shortest distance and the arrangement relationship between the four corner feature points and the center feature point in the rectangular array, offset calculation is performed with the center point of the field of view as the reference point to obtain the pixel coordinates of the four corner pixel points corresponding to the four corner feature points respectively. Step S422: Obtain the pixel region corresponding to the calibration square in the current view of the camera based on the pixel coordinates of the four corner pixels; The center point of the field of view corresponds to the center feature point.

4. The method for manufacturing calibration cloth according to claim 1, characterized in that, Step S5 includes: Step S51: Obtain the intrinsic parameter matrix, distortion parameters, and pose parameters of the camera; Step S52: Based on the intrinsic parameter matrix, the distortion parameter, the pose parameter, and the preset height, the pixel coordinates of each pixel in the pixel region are transformed into ground coordinates using the projection equation to obtain the ground coordinates of each pixel.

5. The method for manufacturing calibration cloth according to claim 4, characterized in that, Step S6 includes: Step S61: Obtain the distance between the ground coordinates of two adjacent pixels based on their ground coordinates; Step S62: Determine the size of the calibration grid based on the distance between the ground coordinates of two adjacent pixels and the arrangement relationship of multiple feature points in the rectangular array.

6. The method for manufacturing calibration cloth according to claim 5, characterized in that, The area of ​​the calibration cloth is larger than the projected area of ​​the lifting device, and step S7 includes: Step S71: Obtain the proportional relationship between the size of the calibration grid and the size of the calibration cloth based on the preset ratio; Step S72: Determine the size of the calibration fabric based on the proportional relationship between the size of the calibration square and the size of the calibration fabric, and cut the base fabric with a cutting device based on the size of the calibration fabric to obtain the calibration fabric; Step S73: Based on the ground coordinates of the pixel, the printing device prints each feature point on the calibration cloth to form the calibration grid corresponding to the camera.

7. A calibration method for a four-lens camera on a container spreader, characterized in that, The calibration cloth is applied to a calibration cloth manufactured by the calibration cloth manufacturing method for a container spreader four-eye camera according to any one of claims 1-6, wherein the calibration method for the calibration cloth for a container spreader four-eye camera includes: The calibration cloth is laid under the lifting device, and the lifting device is raised to a preset height so that each right-angle mark on the calibration cloth is aligned with the projected edge of the corresponding corner lock cylinder on the lifting device. The four sets of cameras on the lifting device are controlled to capture images of the calibration cloth from the current perspective. The calibration cloth has four calibration squares formed at its four corners, and each calibration square contains a rectangular array of multiple feature points. The intrinsic parameter matrix and distortion parameters of each camera are obtained respectively. Based on the intrinsic parameter matrix and distortion parameters, distortion correction is performed on the images captured by each camera to obtain the corrected images. The ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image are obtained. Based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel, a plane calibration calculation is performed to obtain the homography matrix corresponding to each camera.

8. The calibration method for a four-eye camera on a container spreader according to claim 7, characterized in that, The step of obtaining the ground coordinates of each feature point in each calibration grid and the pixel coordinates of the corresponding pixel in each corrected image, and performing planar calibration calculation based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel to obtain the homography matrix corresponding to each camera includes: Based on the arrangement of multiple feature points in the calibration grid, the ground coordinates of each feature point in each calibration grid are obtained respectively; Point detection is performed on each of the corrected images to obtain the pixel coordinates of the pixel points corresponding to each feature point in each of the corrected images; Planar calibration is performed based on the ground coordinates of each feature point and the pixel coordinates of the corresponding pixel points, and the homography matrix corresponding to each camera is obtained using the least squares method.

9. A method for stitching images from four cameras on a container spreader based on calibration parameters of a calibration cloth, characterized in that, include: After the spreader grabs the container, control the spreader to lift it to a preset height; The four cameras on the spreader are controlled to capture images of the four corners of the container from the current viewpoint. Based on the intrinsic parameter matrix and distortion parameters of each camera, distortion correction is performed on the images captured by each camera to obtain the corrected corner image of the container; Using the homography matrix corresponding to each camera obtained by the method as described in any one of claims 7 to 8, perspective transformation is performed on each corrected corner image to obtain the corner orthographic projection images of the four corners of the container in the ground coordinate system; The orthographic projection images of the four corners of the container in the ground coordinate system are stitched together to obtain a complete orthographic projection image of the container in the ground coordinate system.

Images