Goods stack three-dimensional pose detection method and device based on two-dimensional image and geometric constraint

Abstract

The invention relates to a two-dimensional image and geometric constraint-based cargo stack three-dimensional pose detection method and device, which are applied to the technical field of computer vision and logistics automation, and comprises the following steps of: through an edge straight line screening strategy, combining robust fitting of a random sampling consistency algorithm, calculating a three-dimensional pose of a cargo stack; a real physical boundary can be effectively and accurately recognized from complex visual interference such as adhesive tape, labels and printed patterns on the surface of a container, the accuracy and robustness of edge detection are remarkably improved, and therefore the high precision of final three-dimensional pose calculation is guaranteed; and secondly, only a common industrial camera is needed, and an expensive three-dimensional sensor such as a laser radar or a depth camera is not needed, so that the hardware cost and the maintenance complexity of the system are greatly reduced, and the cost benefit is extremely high. And thirdly, an efficient image processing and geometric calculation algorithm is mainly adopted, the calculation complexity is low, and the requirement for real-time detection of the goods stack on an industrial automatic assembly line can be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and logistics automation, and particularly relates to a method and device for detecting three-dimensional pose of a cargo stack based on two-dimensional images and geometric constraints. BACKGROUND

[0002] In modern automated warehouses, accurate three-dimensional pose detection of cargo stacks on conveying lines is a key link to ensure safe, efficient transfer, stacking and storage of goods. Traditional detection methods, such as manual visual inspection, are inefficient and unreliable, and cannot meet the high-speed automated logistics requirements.

[0003] To achieve automated detection, there are various solutions in the prior art. For example, photoelectric sensors are deployed on both sides of the conveying line to determine whether goods exist or are out of limits. This solution is cost-effective, but can only provide rough, discrete position information and cannot obtain accurate three-dimensional pose of the cargo stack. Another type of solution uses three-dimensional sensors such as laser radars or depth cameras to directly obtain three-dimensional point cloud data of the cargo stack by actively emitting laser or structured light, which can achieve high-precision pose measurement. However, such three-dimensional sensors have high hardware costs, complex system integration and maintenance, and their performance may decrease in certain specific materials (such as transparent or highly reflective surfaces) or complex lighting environments, limiting their large-scale industrial applications.

[0004] To strike a balance between cost and accuracy, some technical solutions attempt to use only ordinary two-dimensional image cameras for detection. Such solutions typically perform edge detection on the cargo box image, determine the intersection of the edge lines to obtain the profile vertices of the cargo box, and finally use perspective projection algorithms and re-projection error optimization to calculate the pose. However, in actual industrial environments, the cargo box surface often has packing tape, printed patterns, labels, and complex background lighting, which can generate a large number of false edges and lines in the image. The prior art does not provide a sufficient solution to effectively handle these visual disturbances, resulting in a serious lack of robustness in filtering edges and vertices, making it difficult to accurately distinguish between real cargo box physical boundaries and false internal lines, ultimately leading to failed or low-precision pose detection. Therefore, how to use only low-cost two-dimensional images to effectively overcome the above visual disturbances and achieve high-precision, high-robustness three-dimensional pose detection of cargo stacks is a technical problem that needs to be solved in the field. SUMMARY

[0005] Therefore, the present application aims to provide a method and device for detecting the three-dimensional pose of a cargo stack based on a two-dimensional image and geometric constraints, so as to solve the technical problem that, when a two-dimensional image is used to detect the pose of a cargo stack, visual interference caused by factors such as the surface tape, labels, patterns, and complex background lighting of the cargo box makes it difficult to accurately and robustly identify the real boundary of the cargo box, and finally makes it difficult to balance the cost, accuracy, robustness, and real-time performance of the detection.

[0006] According to a first aspect of an embodiment of the present application, a method for detecting the three-dimensional pose of a cargo stack based on a two-dimensional image and geometric constraints is provided, and the method comprises the following steps: A plurality of cargo stack images are acquired by cameras with different viewing angles, an arbitrary single-view cargo stack image is selected, target detection is performed on the surface of the cargo stack in the single-view cargo stack image, and a ROI candidate region is obtained; The ROI candidate region is preprocessed to obtain a ROI target region; Edge detection is performed on the ROI target region by using a Canny operator to obtain edge points; Hough transform line detection is performed on the ROI target region to obtain candidate straight line segments, and an edge straight line screening strategy is executed to screen out optimal edge line segments representing the boundary of the cargo box from the candidate straight line segments; The edge points are assigned to the nearest optimal edge line segment according to the distance between the edge points and each optimal edge line segment to form an edge point set, and a RANSAC algorithm is used to perform line fitting on each set, and outliers are removed through iterative optimization to obtain the boundary straight line corresponding to each edge point set; The intersection points of the boundary straight lines are calculated, each intersection point is screened according to a preset spatial geometric constraint, and the four corner points of the cargo box are determined according to the screening result of the intersection points; An internal parameter matrix and distortion coefficients are obtained by calibrating the camera, and a PnP algorithm is used to obtain the 3D pose of the box body through the four corner points of the cargo box based on the internal parameter matrix and the distortion coefficients; The 3D pose of the box body under different viewing angles is obtained; A unified global coordinate system is established through joint calibration of multiple cameras, the 3D pose of the box body under different viewing angles is projected into the unified global coordinate system, and the optimal three-dimensional pose of the cargo box is obtained according to a preset constraint condition.

[0007] Preferably, The preprocessing of the ROI candidate region to obtain the ROI target region comprises the following steps: The ROI candidate region is subjected to grayscale processing; The image after grayscale processing is subjected to adaptive contrast enhancement by using a CLAHE algorithm; The grayscale image after adaptive contrast enhancement is subjected to Gaussian filtering processing; performing morphological opening and closing operations on the Gaussian filtered gray image; performing median filtering on the gray image after the morphological opening and closing operations to obtain the ROI target region.

[0008] Preferably, The edge detection on the ROI target region using the Canny operator to obtain edge points includes: using a Sobel operator on the ROI target region to calculate the gradient of the image in the x and y directions respectively to obtain gradient amplitude and gradient direction; performing non-maximum suppression on the gradient amplitude according to the gradient direction to obtain single-pixel edges; setting high and low thresholds, marking edge points greater than the high threshold in the single-pixel edges as strong edge points, marking edge points between the high and low thresholds as weak edge points, and suppressing edge points less than the low threshold to 0, and connecting the weak edge points, if the weak edge points are connected to the strong edge points, then they are retained, otherwise they are suppressed to 0.

[0009] Preferably, The edge straight line screening strategy includes: classifying the candidate straight line segments and assigning them to a plurality of edge regions, calculating a comprehensive score for each candidate straight line segment according to a preset scoring rule, and selecting the straight line segment with the highest comprehensive score in each edge region as the optimal edge line segment.

[0010] Preferably, The comprehensive score is calculated by weighting based on the normalized line segment length, position validity, and edge region attribution of the candidate straight line segment; The position validity is determined according to whether the distance between the two endpoints of the candidate straight line segment and the image boundary is greater than a preset boundary safety distance, if both are greater than the preset boundary safety distance, the position validity is assigned a value of 1, otherwise it is 0.

[0011] Preferably, The edge region is a preset width region of the four sides of the image.

[0012] Preferably, The screening of each intersection point according to the preset spatial geometric constraint includes: The intersection point is located in the ROI region, and the angle between the two intersecting straight lines constituting the intersection point is within a preset right angle tolerance range, and the distance from the intersection point to the straight line is within a preset distance threshold range; The intersection point that meets the conditions is taken as an effective corner point.

[0013] Preferably, The 3D pose of the box is obtained by the PnP algorithm through four corner points of the box, and the PnP algorithm includes the following steps: The PnP algorithm based on the Levenberg-Marquardt optimization method is used to solve the 3D pose of the box surface by iteratively minimizing the re-projection error between the coordinates of the three-dimensional model corner points corresponding to the three-dimensional size of the box projected onto the image plane under the camera model and the coordinates of the corner points of the box on the two-dimensional image.

[0014] Preferably, The optimal three-dimensional pose of the box is obtained according to the preset constraint condition, and the method includes the following steps: The epipolar constraint is performed on the corresponding edge points detected across views, so that the corresponding points in different views are located on the conjugate epipolar line, and the epipolar constraint error is obtained; The epipolar constraint error and the re-projection error obtained by the PnP algorithm are fused to obtain a combined cost function, and the final pose parameter is obtained by minimizing the combined cost function, and the optimal three-dimensional pose of the box is obtained through the final pose parameter.

[0015] According to a second aspect of the embodiment of the present application, a device for detecting the three-dimensional pose of a cargo stack based on two-dimensional images and geometric constraints is provided, and the device includes: An ROI candidate region acquisition module is configured to acquire a plurality of cargo stack images through cameras with different viewing angles, select an arbitrary single-view cargo stack image, perform target detection on the surface of the single-view cargo stack image, and obtain an ROI candidate region; A preprocessing module is configured to pre-process the ROI candidate region to obtain an ROI target region; An edge detection module is configured to perform edge detection on the ROI target region using a Canny operator to obtain edge points; An optimal edge line segment acquisition module is configured to perform Hough transform line detection on the ROI target region to obtain candidate straight line segments, and perform edge straight line screening strategy to screen out optimal edge line segments representing the boundary of the box from the candidate straight line segments; A boundary straight line acquisition module is configured to assign the edge points to the nearest optimal edge line segment according to the distance between the edge points and each optimal edge line segment to form an edge point set, and perform line fitting on each set using a RANSAC algorithm, and remove outliers through iterative optimization to obtain a boundary straight line corresponding to each edge point set; An angle point acquisition module is configured to calculate the intersection points of the boundary straight lines, and screen the intersection points according to a preset spatial geometric constraint, and determine four corner points of the box through the screening result of the intersection points; 3D pose acquisition module: for calibrating the camera to acquire the intrinsic matrix and distortion coefficient; based on the intrinsic matrix and distortion coefficient, the PnP algorithm is adopted to acquire the box 3D pose through the four corner points of the box; Multi-view pose acquisition module: for acquiring the box 3D pose under different views; Multi-view fusion module: for establishing a unified global coordinate system through multi-camera joint calibration, projecting the box 3D pose under different views into the unified global coordinate system, and acquiring the optimal three-dimensional pose of the box according to the preset constraint condition.

[0016] The technical scheme provided by the embodiment of the application can include the following beneficial effects: The application can accurately identify the real physical boundary from the complex visual interference such as the adhesive tape, label and printed pattern on the surface of the box through the designed "partition competition" type edge straight line screening strategy combined with the robust fitting of the random sample consensus algorithm, significantly improves the accuracy and robustness of edge detection, and thus guarantees the high precision of the final three-dimensional pose solution. Secondly, the application only needs to use an ordinary industrial camera, without the need of expensive three-dimensional sensors such as laser radar or depth camera, greatly reduces the hardware cost and maintenance complexity of the system, and has extremely high cost effectiveness. Thirdly, the application mainly adopts efficient image processing and geometric calculation algorithm, and the calculation complexity is low, which can meet the real-time detection demand of the cargo pile on the industrial automation assembly line. Finally, through the multi-level image preprocessing process, the problem of uneven light in the actual scene is effectively solved, and the optional multi-view integrated framework can solve the occlusion problem under single view, further enhances the adaptability and detection reliability of the system under complex working conditions.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings incorporated into the specification and forming a part thereof, show embodiments consistent with the application, and together with the specification serve to explain the principles of the application.

[0019] Figure 1 is a flowchart of a cargo pile three-dimensional pose detection method based on two-dimensional images and geometric constraints according to an exemplary embodiment; Figure 2 is a detailed flowchart of a box edge detection method according to another exemplary embodiment; Figure 3 is an image preprocessing flowchart according to another exemplary embodiment; Figure 4 is an edge straight line screening and scoring flowchart according to another exemplary embodiment; Figure 5 is a multi-view integration method flow chart shown according to another exemplary embodiment; Figure 6 is a system hardware implementation environment schematic diagram shown according to another exemplary embodiment; In the drawings: 1 - conveyor belt, 2 - pallet, 3 - gantry, 4 - camera. DETAILED DESCRIPTION

[0020] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, unless otherwise indicated, like numbers in the attached drawings refer to the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0021] Embodiment One Figure 1 is a flow chart of a pallet 3D pose detection method based on two-dimensional images and geometric constraints according to an exemplary embodiment, as shown in Figure 1 , the method comprises: S1, obtaining multiple pallet images through cameras of different views, selecting an arbitrary single-view pallet image, performing target detection on the pallet surface in the single-view pallet image to obtain an ROI candidate region; S2, pre-processing the ROI candidate region to obtain an ROI target region; S3, performing edge detection on the ROI target region using a Canny operator to obtain edge points; S4, performing Hough transform line detection on the ROI target region to obtain candidate straight line segments; performing edge straight line screening strategy to screen out optimal edge line segments representing the boundaries of the container from the candidate straight line segments; S5, assigning the edge points to the nearest optimal edge line segment according to their distance from each optimal edge line segment to form an edge point set, and using a RANSAC algorithm to perform line fitting on each set, and removing outliers through iterative optimization to obtain the boundary straight line corresponding to each edge point set; S6, calculating the intersection points of each boundary straight line, and screening each intersection point according to a predetermined spatial geometric constraint, and determining the four corner points of the container through the intersection point screening result; S7, calibrating the camera to obtain an intrinsic matrix and distortion coefficients; based on the intrinsic matrix and distortion coefficients, using a PnP algorithm to obtain the 3D pose of the container through the four corner points of the container; S8, obtaining the 3D pose of the container under different views; S9, by establishing a unified global coordinate system through multi-camera joint calibration, projecting the 3D pose of the container under different perspectives into the unified global coordinate system, and obtaining the optimal 3D pose of the container according to a preset constraint condition; It can be understood that the embodiment provides a basic scheme for detecting the 3D pose of a container based on a two-dimensional image collected by a single camera. It should be noted that the scheme is intended to fully embody the complete technical process of image collection, preprocessing, core edge detection and screening, and final pose solving provided by the present application.

[0022] Please refer to Figure 6 which shows a system hardware implementation environment to which the embodiments of the present application can be applied. In a typical automated warehouse scenario, the system can include a conveyor belt 1 for transporting a pallet 2. A gantry 3 can be erected above the conveyor belt 1, and one or more industrial cameras 4 can be fixedly installed on the gantry 3. The lens of the camera 4 is perpendicular or at a certain angle to the downward direction to ensure that when the pallet 2 passes through, a clear and complete image of the top or side of the pallet can be captured. In the present embodiment, it is assumed that one camera 4 is used, which is installed at a position and angle that have been adjusted to be able to capture the main visible surface of the target container.

[0023] When a pallet carrying a pallet 2 enters the field of view of the camera 4 and triggers a shooting signal, the system starts the detection process.

[0024] First, the system performs image acquisition. The camera 4 captures a two-dimensional color image containing the pallet 2, for example, an image with a resolution of 1920x1080 pixels.

[0025] As an optional implementation manner, in some application scenarios, for example, when multiple containers are included in the image or the background is relatively complex, region of interest target detection can be performed first. This step can use a lightweight deep learning target detection model (such as the YOLO series) or a traditional image segmentation method to quickly locate the approximate bounding box of each container to be detected in the entire image. This step helps to focus the subsequent processing process on the effective area containing a single container, thereby reducing the interference of irrelevant backgrounds and improving processing efficiency. For the description of the subsequent steps, it is assumed that an image region containing only a single target container has been obtained.

[0026] Correspondingly, the process enters the core container edge detection. The goal of this step is to accurately extract the geometric information representing the physical boundary of the container from the container image that can be full of visual interference. Please refer to Figure 2 This step is further divided into a series of sub-steps.

[0027] The first step is image preprocessing S31: due to the variable lighting conditions in the industrial field, and the presence of stains, reflections and other noise on the surface of the box, it is essential to preprocess the original image. Please refer to the following description Figure 3 In an embodiment of the present application, the preprocessing S31 process includes the following consecutive operations: first, perform grayscale processing (step S311) to convert the three-channel color image captured by the camera into a single-channel grayscale image. This is intended to reduce the data dimension and reduce the computational complexity of subsequent image processing, while retaining the brightness information required for edge detection, which specifically includes: Color images contain RGB three-channel color information. In the box corner detection task, the focus is on the structural features of the image rather than the color information, and it is necessary to convert the RGB image to a grayscale image. The conversion from RGB to grayscale uses the weighted average method:

[0028] where R, G, and B represent the red, green, and blue channel values of the pixel point, respectively, and the weight coefficient is determined by the sensitivity of the human eye to different colors. Grayscale conversion converts three-channel images to a single channel, reducing the data volume while preserving the main structural information of the image.

[0029] Subsequently, to enhance the image contrast (step S312), an adaptive histogram equalization algorithm with limited contrast can be used to process the grayscale image. It can be understood that, unlike global histogram equalization, this algorithm divides the image into several sub-regions and performs histogram equalization on each sub-region independently, which can effectively enhance the local contrast of the image under different lighting conditions, so that the edges of the box can be clearly highlighted even in uneven light conditions. To avoid excessive amplification of noise, the algorithm can also set an upper limit for the contrast gain of each sub-region, which specifically includes: To enhance the local contrast of the image, the CLAHE algorithm is used to process the grayscale image. The histogram equalization of each local region in the image can be expressed as:

[0030] where, is the number of pixels at the gray level is the total number of pixels in the region, is the number of gray levels, and are the target gray range.

[0031] ​Next, Gaussian filtering is performed (step S313), for example, by convolving the enhanced image with a 5x5 Gaussian kernel. Gaussian filtering, as a linear smoothing filter, effectively suppresses normally distributed Gaussian noise in the image, making the image smoother and laying the foundation for subsequent edge detection. Specifically, it includes: To suppress noise in an image, a Gaussian filter is applied to a grayscale image. The Gaussian filter is a linear smoothing filter whose weight coefficients follow a two-dimensional Gaussian distribution.

[0032] Where (x,y) represents the position of the pixel relative to the center of the kernel. Let be the standard deviation of the Gaussian distribution. According to the Gaussian distribution, the filter assigns higher weights to the center pixels and gradually decreases the weights of surrounding pixels, creating a balanced and smooth effect. A discrete Gaussian kernel is used to perform convolution operations on the image:

[0033] in, For the filtered image at position Pixel value at that location, This represents the pixel value at the corresponding position in the original image. The coefficients are Gaussian kernels. During image processing, convolution operations can preserve important structural features while suppressing high-frequency noise.

[0034] Furthermore, to eliminate any tiny bright or dark spots in the image (e.g., noise caused by dust or fine reflections) and fill any tiny breaks in the edges, this embodiment can also sequentially perform morphological opening and closing operations (step S314). Specifically, the opening operation can break fine connections and eliminate isolated bright spots, while the closing operation can fill tiny holes and connect adjacent areas. For example, this operation can be performed using a 3x3 structuring element, specifically including: To further optimize image quality, morphological opening and closing operations are employed, with the structuring element set to... The original image is The formulas for opening and closing operations are as follows:

[0035] in, and These represent erosion and dilation operations, respectively. Opening removes small bright details while preserving the overall shape and size of larger objects; closing fills small holes and connects adjacent objects.

[0036] As the last step of the preprocessing, the image can be further processed by median filtering (step S315), for example, using a 3x3 size median filter. Median filtering is a nonlinear filtering technique that can effectively remove salt and pepper noise by sorting the pixel values in the neighborhood of a pixel and replacing the center pixel value with the median value. Compared with Gaussian filtering, median filtering can better preserve the edge definition. Specifically, the median filtering includes: Finally, the median filtering is used to further suppress noise while preserving edge features. For the median filtering with a window size of 3x3:

[0037] As a nonlinear technique, the median filtering replaces each pixel with the median value of its neighborhood, which can effectively remove salt and pepper noise and better preserve edge integrity compared with linear filters.

[0038] After the above series of preprocessing S31, a gray-scale image with suppressed noise and enhanced edges can be obtained, which lays a solid foundation for subsequent edge detection S32.

[0039] In step S32, the Canny operator is first used to detect the edges of the preprocessed image. As a multi-stage edge detection algorithm, the Canny operator can detect relatively accurate and continuous edge points by calculating the image gradient, performing non-maximum suppression to refine the edges, and using double thresholds and hysteresis tracking to connect the edges, thereby forming a binary edge map. Specifically, the Canny operator includes: First, the Sobel operator is used to calculate the gradients of the image in the x and y directions, respectively. The mathematical expression of the Sobel operator is:

[0040] wherein, is the input image, denotes convolution operation; Then, the gradient amplitude and direction are calculated:

[0041] Non-maximum suppression: In order to obtain single-pixel edges , the non-maximum suppression is performed on the gradient amplitude . First, the gradient direction is quantized into 0°, 45°, 90°, and 135°. Then, interpolation is performed on each pixel point along the gradient direction. If the gradient amplitude of the point is not the local maximum value, it is suppressed to 0. The non-maximum suppression is represented as: ​

[0042] Double threshold detection and edge connection: The double threshold algorithm screens and connects edges, sets high and low thresholds and , marks the edge points greater than as strong edge points; marks the edge points between and as weak edge points; suppresses the edge points less than to 0, and judges the connection of weak edge points, if the weak edge points are connected with strong edge points, they are retained, otherwise they are suppressed to 0.

[0043] Subsequently, in order to extract structured straight line information from discrete edge points, the embodiment adopts a probabilistic Hough transform to detect all candidate straight line segments on the Canny edge map. Compared with the standard Hough transform, the probabilistic Hough transform does not accumulate all points, but only analyzes a random subset of the edge point set and estimates the probability that these points form a straight line, so the calculation efficiency is higher. The output of the transform is a set of straight line segments, each of which is defined by its start and end point coordinates. Parameters such as minimum line segment length (e.g. 50 pixels) and maximum allowed gap between discontinuous line segments can be set to filter out too short or discontinuous line segments, which specifically include: In order to improve the calculation efficiency, the probabilistic Hough transform (PHT) is used to map the straight line from the image space to the parameter space. In polar coordinate representation, any straight line can be represented as:

[0044] wherein represents the vertical distance of the straight line to the coordinate origin, represents the angle between the vertical line and the axis, represents any point on the straight line, the main parameters of PHT include resolution (pixels), resolution (radians), minimum voting threshold , minimum line segment length and maximum line segment gap , the line segment detection function of PHT is:

[0045] wherein is the voting accumulation value in the parameter space, is the length of the detected line segment, is the gap distance between line segments. Due to the influence of background and box surface texture, PHT realizes robust straight line detection and recognition of a large number of candidate line segments through multiple parameter constraints. ​

[0046] To accurately identify the straight lines representing the four physical boundaries of the container from the candidate straight line segments, the present application proposes a set of core edge filtering and classification strategies, i.e., step S33, please refer to Figure 4 The detailed process of the strategy is as follows: Step S331, straight line classification and partitioning, this step aims to structurally organize all candidate straight line segments. First, classify each straight line segment according to its angle, for example, classify the straight lines into two categories of horizontal lines and vertical lines by an angle threshold, wherein the angle threshold is set to 45°. Then, according to the position of the straight line segment in the image, it is classified into a predefined edge region. In this embodiment, the image can be divided into four edge regions of upper, lower, left and right along the horizontal and vertical directions. For example, the region of the top 15% height of the image is defined as the "upper edge region", the region of the bottom 15% height of the image is defined as the "lower edge region", the region of the left side 15% width of the image is defined as the "left edge region", and the region of the right side 15% width of the image is defined as the "right edge region". In this way, a candidate straight line segment classified as a "horizontal line" will be classified into the "upper edge region" if most of it is located in the upper part of the image, which specifically includes: First, determine the pixel range of the edge region by the edge threshold calculation formula:

[0047] wherein the default setting is 0.15, i.e., 15% of the minimum side length of the image as the width of the edge region. For each detected straight line, the system classifies it into the corresponding edge region according to its geometric characteristics and position information. The vertical coordinate range of the horizontal line segment is used to determine whether it belongs to the upper edge region or the lower edge region, and the horizontal coordinate range of the vertical line segment is used to determine whether it belongs to the left edge region or the right edge region. The determination criteria is whether the line segment endpoint coordinates satisfy the corresponding edge threshold condition.

[0048] Step S332, calculate the comprehensive score. In this step, a comprehensive score is calculated for each candidate straight line segment to quantify its possibility as a true container boundary. The score can be calculated by weighting multiple dimensional features, which specifically includes: The comprehensive score of each straight line is calculated to select the optimal edge, and the straight line scoring formula is as follows:

[0049] wherein, denotes the normalized line segment length, which is calculated as wherein is the line segment length, and are the image height and width, respectively; represents the position validity score, which is 1.0 when both ends of the line segment are far away from the image boundary, otherwise 0.0; represents the edge region score, which is 1.0 when the line segment is located in the edge region, otherwise 0.0; , and are the length weight, position weight and convenience weight respectively, which are set as , , ; The position validity judgment adopts the boundary distance criterion, which requires that the coordinates of the end points of the line segment satisfy:

[0050] where margin is the boundary safety distance, which is set as 5 pixels; Finally, the system selects the line with the highest comprehensive score from the line set in each of the four edge regions as the optimal edge line segment of the box, effectively excluding the interfering lines inside the image (such as the horizontal packing tape), ensuring the accuracy and stability of the detection results.

[0051] Step S333, selecting the optimal line in each region. After completing the comprehensive score calculation of all candidate line segments, the system executes a "partition competition" selection mechanism. Specifically, among all the candidate lines in the "upper edge region", the line with the highest comprehensive score is selected as the optimal edge line segment of the region. Similarly, the line with the highest score is also selected in the "lower edge region", "left edge region" and "right edge region" respectively. In this way, from hundreds of candidate lines, the four optimal edge line segments that best represent the upper, lower, left and right physical boundaries of the box can be finally selected.

[0052] Returning to the flow of Figure 2 , after obtaining the four optimal edge line segments, the flow enters step S34 to perform more accurate line fitting. Although the optimal edge line segments are very close to the true boundary, they are still approximate results based on the Hough transform. To obtain a more accurate line equation, the embodiment uses all the edge points previously detected by the Canny operator for robust fitting. First, all the edge points are assigned to the line segment closest to them according to their perpendicular distance to the four optimal edge line segments, thereby forming four edge point sets. In theory, each point set corresponds to a boundary of the box, but it may still contain a small number of noise points (i.e. outliers) that do not belong to the boundary, which specifically include: Given the edge image and the image size , the distance of each edge point to the optimal edge line is calculated and filtered by a distance threshold:

[0053] wherein, denotes the line parameter, denotes the distance of a point to a line, when the distance is below a threshold, each edge point is assigned to the nearest line, the edge point is successfully classified into four different box boundaries (upper, lower, left, right edge), effectively separating the real box edge from the noise generated by the label or background texture.

[0054] To eliminate the influence of these outliers, the embodiment adopts a random sample consensus algorithm to perform line fitting on each edge point set, the random sample consensus algorithm is an iterative process: in each iteration, two points are randomly selected from the point set to fit a temporary line; then, the distances of all other points in the point set to the temporary line are calculated, and the points with a distance less than a preset threshold are defined as "inliers"; this process is repeated multiple times, and finally the line corresponding to the model with the most inliers is selected as the final fitting result of the point set, by executing the algorithm on the four edge point sets respectively, four line equations that can accurately describe the box boundary are finally obtained, specifically including: On the basis of obtaining the edge point clustering result, the RANSAC (Random Sample Consensus) algorithm is used to perform line fitting on each set, and the outliers are removed through iterative optimization to obtain accurate boundary lines, the edge point set , the line can be represented as:

[0055] wherein, k is the slope of the line, b is the intercept of the line, is the coordinate of the point in the point set. The RANSAC algorithm finds the optimal fitting result through iterative optimization, first estimates the model for the two points randomly selected, calculates the line parameter; then counts the number of inliers and evaluates the model; finally, the inliers are screened by judging whether the perpendicular distance of the point to the line is less than the set threshold, the algorithm iterates multiple times to select the optimal model, and the inlier set is retained.

[0056] Subsequently, in step S35, corner point detection and sorting are performed. By simultaneously solving the equations of the four boundary lines obtained in the previous step, the intersection points between them can be calculated, thereby obtaining a set of candidate corner points. In theory, there are six intersection points of the four lines, but due to the perspective effect, the embodiment is concerned with the four corner points that constitute the visible top surface of the container. To filter out the correct corner points from the candidate corner points, the embodiment imposes spatial geometric constraints. A key constraint is the angle constraint: since the container is a cuboid, its adjacent boundary lines are perpendicular to each other in three-dimensional space, and after projection onto the two-dimensional image, their included angle should also be close to 90 degrees. Therefore, those corner points formed by the intersection of two lines with an included angle within a predetermined right angle tolerance range (e.g., 80° to 100°) can be filtered out. In addition, the region of interest information obtained in the above can be combined to eliminate corner points located outside the region. Through these filters, the four corner point coordinates of the container on the two-dimensional image are finally determined, and they can be sorted according to their positional relationship (e.g., clockwise or counterclockwise). The specific steps include: To obtain the accurate corner point coordinates of the upper surface of the container, based on the results of line fitting, an angle point detection method based on line intersection calculation and clustering is proposed, and spatial geometric constraints are used to filter and sort the corner points. For two lines and , the intersection point coordinates can be obtained by solving the linear equation system:

[0057] The intersection point coordinates are obtained:

[0058] An effective corner point is defined as a point that satisfies all the following conditions : (1) The point is located within the ROI region: ; (2) The angle between the two intersecting lines and forming the corner point is close to a right angle: ; (3) The distance from the corner point to the line is within a threshold range: ; where the angle tolerance is set to 10° to accommodate slight deviations from perfect perpendicularity in actual images, and the distance threshold and are used to filter out false intersection points while retaining effective corner points. The specific values are determined through experimental calibration of the target container type.

[0059] 至此,货箱边缘检测步骤S30完成,然后执行位姿解算,该步骤旨在根据已知的二维图像点和对应的三维空间点,求解相机与物体之间的相对位姿,该问题在计算机视觉领域中被称为PnP问题, 在本实施例中,位姿解算的输入包括: 预先标定好的相机内参矩阵K和畸变系数d; 已知的货箱三维模型,具体地,是货箱在自身模型坐标系下的四个顶点的三维坐标,例如,可定义一个以货箱某个角点为原点的坐标系,则四个顶点坐标分别为(0, 0, 0),(L, 0, 0), (L, W, 0), (0, W, 0),其中L和W是货箱的长度和宽度; 本实施例采用一种基于Levenberg-Marquardt优化方法的迭代PnP算法来求解位姿,该算法的目标是寻找一个旋转矩阵R和平移向量T(它们共同构成了从模型坐标系到相机坐标系的变换),使得三维模型角点经过该(R, T)变换、再通过相机内参K投影到图像平面上的二维坐标,与实际观测到的二维角点坐标之间的距离(即重投影误差)之和最小,Levenberg-Marquardt算法是一种高效的非线性优化算法,它结合了梯度下降法和高斯-牛顿法的优点,能够快速稳定地收敛到最优解,算法的输出即为货箱相对于相机坐标系的旋转矩阵R和平移向量T,并基于相机坐标系的旋转矩阵R和平移向量T求解四个角点的三维坐标,也就是货箱的3D位姿。

[0060] 通过本实施例的完整流程,即使在货箱表面存在反光胶带和印刷文字等复杂视觉干扰的情况下,系统也能够准确地计算出其在相机坐标系下的三维位姿,实验表明,位置误差可控制在5mm以内,姿态误差可小于1°,充分展示了本申请方法的鲁棒性和高精度。

[0061] It should be emphasized that before executing the PnP algorithm, the camera 4 needs to be calibrated. This calibration process can be completed using a standard checkerboard calibration board. Specifically, by taking images of the calibration board at different positions and poses and using mature calibration algorithms such as Zhang's calibration method, the internal parameter matrix K and distortion coefficients d of the camera 4 can be accurately calculated. It can be understood that the internal parameter matrix K describes the projection relationship from three-dimensional points in the camera coordinate system to pixel coordinates in the two-dimensional image plane, and the distortion coefficients d are used to correct the radial and tangential distortions of the image caused by the physical characteristics of the lens. These parameters are the basis for subsequent accurate three-dimensional pose calculation and will be stored in the system for later use after calibration. At the same time, the three-dimensional physical dimensions of the cargo box to be detected (for example, length 400mm, width 300mm, height 200mm) are input into the system as prior knowledge; the specific PnP algorithm includes: The specific solution process is as follows: The first stage: Camera calibration; Input: n checkerboard images (n = 9); Output: Intrinsic parameter matrix K, distortion coefficients d; The second stage: PnP for the corner points of the cargo box; Input: Image coordinates of 4 corner points, dimensions L×W of the cargo box, camera parameters K, d; Output: Extrinsic parameters R, t of the camera; The third stage: Three-dimensional coordinate calculation; Input: World coordinates of the corner points, extrinsic parameters R, t; Output: Three-dimensional coordinates of the corner points (in the camera coordinate system); The first stage: Known: Images of the cargo box taken by camera A; Two-dimensional image coordinates of the four corner points on the surface of the cargo box (rectangle);

[0062] Actual dimensions of the surface of the cargo box: length L, width W Solution target: Three-dimensional world coordinates of the four corner points The first step: Solve K and d using the checkerboard calibration method Data acquisition for checkerboard calibration Use a standard checkerboard (such as 9×6 grids) to take n images (n≥3) from multiple angles, and m corner points are detected in each image.

[0063] Setting of the world coordinates of the checkerboard corner points: (i, j are grid indices) is the size of each single grid in the checkerboard Homography matrix estimation (solution )

[0064] For each chessboard image k, establish a planar homography transformation:

[0065] Each problem is solved using the DLT method. : Establish 2m linear equations for m corner points and solve for 9 parameters.

[0066] Solving the intrinsic parameter matrix K Using the orthogonality constraint of rotation matrices: (Orthogonal constraint) (Modulus length constraint) set up Each image provides 2 constraint equations, and n images provide 2n equations.

[0067] b is solved by SVD decomposition, and then the intrinsic parameters are calculated:

[0068]

[0069]

[0070]

[0071]

[0072] Intrinsic parameter matrix:

[0073] Initial extrinsic parameter estimation For each image ,from Calculate the initial extrinsic parameters:

[0074]

[0075]

[0076]

[0077]

[0078] in

[0079] Solving for distortion coefficient d Distortion model (horizontal + vertical):

[0080]

[0081] Total weight projection error:

[0082] Optimize all parameters using the Levenberg-Marquardt algorithm: Intrinsic parameters:

[0083] Distortion coefficient:

[0084] External parameters:

[0085] .

[0086] Phase 2: Solving for camera extrinsic parameters R and t Establish a world coordinate system for cargo containers Set the world coordinates of the four corner points on the surface of the cargo box:

[0087] PnP issue settings Given: Camera parameters obtained from calibration: K, d World coordinates of the four corner points:

[0088] Corresponding distortion-corrected image coordinates:

[0089] Solving for camera extrinsic parameters.

[0090] The projection equation is:

[0091] PnP algorithm solution Control point selection

[0092] Calculation of centroid coordinates express For linear combinations of other control points:

[0093] in It can be solved by a system of linear equations.

[0094] Linear constraint construction Establish constraint equations for each observation point:

[0095] Construct a linear system of equations Mx = 0, where x contains 12 control point coordinate components.

[0096] Distance constraint solution Utilizing the invariance of distance between control points (World coordinate system = Camera coordinate system) Establish a system of linear equations concerning the scale parameters to solve for the coordinates of the control points in the camera coordinate system.

[0097] extrinsic parameter calculation R and t are calculated using the absolute orientation method: Centroid calculation:

[0098]

[0099] Covariance matrix:

[0100] SVD decomposition Rotation matrix:

[0101] Translation vector:

[0102] Phase 3: Calculation of 3D coordinates of corner points Camera coordinate system coordinates The coordinates of the four corner points in the camera coordinate system:

[0103]

[0104]

[0105] World coordinate system coordinates Coordinates relative to a certain world coordinate system, through inverse transformation:

[0106] Reprojection verification Reproject the calculated 3D coordinates onto the image: Camera coordinate projection:

[0107]

[0108]

[0109] Distortion correction:

[0110]

[0111]

[0112]

[0113]

[0114] Reprojection error (calculated only):

[0115] Multi-view integration: Please see Figure 5 It demonstrates the process of a multi-view integration method. First, during the system deployment phase, joint calibration of camera A and camera B is required (step S73), which specifically includes: To extend the single-view method to multi-camera scenes, a multi-view ensemble framework is proposed, based on N cameras. The observed cargo stacks are processed independently using a single-view workflow, while cross-view information is integrated through a unified optimization framework.

[0116] First, a global coordinate system is established through a unified camera calibration process:

[0117] in Represents a point in the world coordinate system; Represents a view The camera coordinates; and These are the rotation and translation parameters for each camera. Rigid body transformation is applied to establish a common reference coordinate system for all cameras, integrating information from multiple perspectives.

[0118] Once the stack 2 enters the overlapping field of view, in step S71, the two cameras simultaneously or sequentially acquire images. In step S72, the system performs independent processing for each single viewpoint. Based on this, the system performs cross-view fusion and optimization in step S74. This step utilizes the geometric consistency between multiple views to solve for or optimize the pose. Specifically, it can be done in the following way: Using the preliminary pose calculated by camera B, we obtain the estimated coordinates of the four three-dimensional corner points of the cargo box in the global coordinate system.

[0119] For a corner point that can be observed by both camera A and camera B, find its two-dimensional coordinates in the image of camera A and its corresponding two-dimensional coordinates in the image of camera B.

[0120] Construct a combinatorial optimization objective function that aims to find the globally optimal cargo container pose by simultaneously minimizing multiple error terms, specifically including: Perform cross-view detection fusion, and for each detected cargo surface, project the detection results from multiple views onto a unified coordinate system:

[0121] in It's a camera. The projection matrix, containing the view 2D point coordinates detected The Direct Linear Transformation (DLT) algorithm is used to efficiently solve for uniform coordinates. Treating this as a least-squares problem, the algorithm employs triangulation to find the optimal 3D point that minimizes reprojection errors across all views. Improve PnP-based pose estimation through multi-view constraints. For corresponding edge points detected across views. and Perform polar constraints:

[0122] in It is a view and The fundamental matrix between them ensures that corresponding points in different views lie on the conjugate epipolar line, providing a geometric verification mechanism for feature matching. The final pose parameters are obtained by minimizing the combined cost function.

[0123] in This represents the reprojection error of the PnP estimate. This indicates the polar constraint error. It is a weighting factor that, by combining reprojection error with epipolar constraints, balances single-view pose accuracy with multi-view geometric consistency in the cost function.

[0124] Finally, in step S75, a more accurate and robust final cargo container pose (in the global world coordinate system) is output after joint optimization of multi-view geometric constraints.

[0125] It is understandable that the solution provided in this embodiment, by fusing information from multiple perspectives, not only effectively solves the problem of incomplete information due to occlusion in a single perspective, but also, due to the use of more observation data and geometric constraints, its final output pose accuracy is usually better than the result of independent calculation from any single perspective.

[0126] Example 2 This embodiment provides a three-dimensional pose detection device for a stack of goods based on two-dimensional images and geometric constraints. The device includes: ROI candidate region acquisition module: used to acquire multiple stack images through cameras from different perspectives, select any single-view stack image, perform target detection on the stack surface in the single-view stack image, and obtain ROI candidate regions; Preprocessing module: used to preprocess the candidate ROI regions to obtain the target ROI regions; Edge detection module: used to perform edge detection on the ROI target region using the Canny operator to obtain edge points; Optimal edge segment acquisition module: used to obtain candidate line segments by performing Hough transform line detection on the ROI target area; execute the edge line filtering strategy to select the optimal edge segment representing the cargo box boundary from the candidate line segments; Boundary line acquisition module: It is used to assign the edge points to the nearest optimal edge line segment according to their distance from each optimal edge line segment to form an edge point set, and use the RANSAC algorithm to perform line fitting on each set, and remove outliers through iterative optimization to obtain the boundary line corresponding to each edge point set; Corner point acquisition module: It is used to calculate the intersection points of each boundary line, filter each intersection point according to the preset spatial geometric constraints, and determine the four corner points of the cargo box based on the intersection point filtering results; 3D pose acquisition module: used to calibrate the camera to obtain the intrinsic parameter matrix and distortion coefficients; based on the intrinsic parameter matrix and distortion coefficients, the PnP algorithm is used to obtain the 3D pose of the container through the four corner points of the container; Multi-view pose acquisition module: used to acquire the 3D pose of the box from different viewpoints; Multi-view fusion module: used to establish a unified global coordinate system through joint calibration of multiple cameras, project the 3D pose of the container from different viewpoints onto the unified global coordinate system, and obtain the optimal three-dimensional pose of the container according to preset constraints.

[0127] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0128] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0129] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0130] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0131] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0132] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0133] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0134] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0135] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting the three-dimensional pose of a stack of goods based on two-dimensional images and geometric constraints, characterized in that, The method includes: Multiple images of the stack of goods are acquired by cameras from different perspectives. Any single-view stack of goods image is selected, and target detection is performed on the surface of the stack of goods in the single-view stack of goods image to obtain ROI candidate regions. The candidate ROI regions are preprocessed to obtain the target ROI regions; Edge points are obtained by performing edge detection on the target ROI region using the Canny operator; Hough transform is used to detect candidate line segments in the ROI target area; an edge line filtering strategy is executed to select the optimal edge line segment representing the cargo box boundary from the candidate line segments; The edge points are assigned to the nearest optimal edge segment according to their distance from each optimal edge segment to form an edge point set. The RANSAC algorithm is used to fit a straight line to each set. Outliers are eliminated through iterative optimization to obtain the boundary line corresponding to each edge point set. By calculating the intersection points of each boundary line and filtering each intersection point according to the preset spatial geometric constraints, the four corner points of the cargo box are determined based on the intersection point filtering results. The camera is calibrated to obtain the intrinsic parameter matrix and distortion coefficients; based on the intrinsic parameter matrix and distortion coefficients, the PnP algorithm is used to obtain the 3D pose of the cargo box through the four corner points; Obtain the 3D pose of the box from different perspectives; A unified global coordinate system is established through joint calibration of multiple cameras. The 3D poses of the container from different perspectives are projected into the unified global coordinate system, and the optimal 3D pose of the container is obtained according to preset constraints.

2. The method according to claim 1, characterized in that, The preprocessing of the candidate ROI regions to obtain the target ROI regions includes: The ROI candidate regions are then grayscaled. The grayscale image is then subjected to adaptive contrast enhancement using the CLAHE algorithm. The grayscale image after adaptive contrast enhancement is processed by Gaussian filtering. Morphological opening and closing operations are applied to the grayscale image after Gaussian filtering; The grayscale image after morphological opening and closing operations is processed by median filtering to obtain the ROI target region.

3. The method according to claim 2, characterized in that, The step of using the Canny operator to perform edge detection and obtain edge points in the ROI target region includes: The Sobel operator is used to calculate the gradient of the image in the x and y directions for the ROI target region, respectively, to obtain the gradient magnitude and gradient direction; The gradient magnitude is subjected to non-maximum suppression based on the gradient direction to obtain the single-pixel edge; Two thresholds are set: a high threshold and a low threshold. Edge points with a single pixel edge greater than the high threshold are marked as strong edge points, and edge points with a threshold between the high and low thresholds are marked as weak edge points. Edge points with a threshold less than the low threshold are suppressed to 0. For weak edge points, a connection judgment is made. If a weak edge point is connected to a strong edge point, it is retained; otherwise, it is suppressed to 0.

4. The method according to claim 3, characterized in that, The edge line filtering strategy includes: The candidate line segments are classified and assigned to multiple preset edge regions. A comprehensive score is calculated for each candidate line segment according to preset scoring rules. The line segment with the highest comprehensive score in each edge region is selected as the optimal edge line segment.

5. The method according to claim 4, characterized in that, The comprehensive score is calculated by weighting the normalized line segment length, positional validity, and edge region affiliation of the candidate line segments. The position validity is determined by whether the distance between the two endpoints of the candidate line segment and the image boundary is greater than a preset boundary safety distance. If both are greater, the position validity is assigned a value of 1; otherwise, it is assigned a value of 0.

6. The method according to claim 5, characterized in that, The edge region is a preset width region on the four sides of the image.

7. The method according to claim 6, characterized in that, The step of filtering each intersection point according to preset spatial geometric constraints includes: The intersection point is located within the ROI region, and the included angle between the two intersecting lines constituting the intersection point is within a preset right angle tolerance range, while the distance from the intersection point to the line is within a preset distance threshold range. Intersections that meet the conditions are considered valid corner points.

8. The method according to claim 7, characterized in that, The step of obtaining the 3D pose of the cargo box using the PnP algorithm through the four corner points includes: The PnP algorithm based on the Levenberg-Marquardt optimization method is used to solve the 3D pose of the container surface by iteratively minimizing the reprojection error between the coordinates of the corner points of the 3D model corresponding to the 3D dimensions of the container projected onto the image plane under the camera model and the coordinates of the corner points of the container on the 2D image.

9. The method according to claim 8, characterized in that, The process of obtaining the optimal three-dimensional pose of the cargo box according to preset constraints includes: Apply epipolar constraints to the corresponding edge points detected across views, so that the corresponding points in different views lie on the conjugate epipolar line, and obtain the epipolar constraint error; The epipolar constraint error and the reprojection error obtained by the PnP algorithm are fused to obtain the combined cost function. The final pose parameters are obtained by minimizing the combined cost function, and the optimal three-dimensional pose of the cargo box is obtained by the final pose parameters.

10. A three-dimensional pose detection device for a stack of goods based on two-dimensional images and geometric constraints, characterized in that, The device includes: ROI candidate region acquisition module: used to acquire multiple stack images through cameras from different perspectives, select any single-view stack image, perform target detection on the stack surface in the single-view stack image, and obtain ROI candidate regions; Preprocessing module: used to preprocess the candidate ROI regions to obtain the target ROI regions; Edge detection module: used to perform edge detection on the ROI target region using the Canny operator to obtain edge points; Optimal edge segment acquisition module: used to obtain candidate line segments by performing Hough transform line detection on the ROI target area; execute the edge line filtering strategy to select the optimal edge segment representing the cargo box boundary from the candidate line segments; Boundary line acquisition module: It is used to assign the edge points to the nearest optimal edge line segment according to their distance from each optimal edge line segment to form an edge point set, and use the RANSAC algorithm to perform line fitting on each set, and remove outliers through iterative optimization to obtain the boundary line corresponding to each edge point set; Corner point acquisition module: It is used to calculate the intersection points of each boundary line, filter each intersection point according to the preset spatial geometric constraints, and determine the four corner points of the cargo box based on the intersection point filtering results; 3D pose acquisition module: used to calibrate the camera to obtain the intrinsic parameter matrix and distortion coefficients; based on the intrinsic parameter matrix and distortion coefficients, the PnP algorithm is used to obtain the 3D pose of the container through the four corner points of the container; Multi-view pose acquisition module: used to acquire the 3D pose of the box from different viewpoints; Multi-view fusion module: used to establish a unified global coordinate system through joint calibration of multiple cameras, project the 3D pose of the container from different viewpoints onto the unified global coordinate system, and obtain the optimal three-dimensional pose of the container according to preset constraints.

Images