Container Remaining Capacity Detection Method
Automatically detecting the remaining capacity of the container through image vision technology, solving the problem of time-consuming, labor-intensive and inaccurate manual inspection, achieving efficient and accurate capacity calculation in the container, and improving logistics and transportation efficiency.
Patent Information
- Application Number
- CN202411207974.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2044-08-30
AI Technical Summary
When detecting the remaining capacity of containers in the prior art, manual reporting is time-consuming and laborious and difficult to accurately estimate, resulting in untimely information acquisition and large errors, which affects warehouse management and logistics transportation efficiency.
Using an automatic intelligent detection method based on image vision, by obtaining images in the empty box and loading cargo state, marking the corner points inside the container, calculating the corner point projection relationship, extracting the cargo profile endpoints, estimating the ratio of the physical dimensions of the cargo to the container size, and calculating the container Remaining capacity.
Convenient and automatic container residual capacity detection is realized, loading efficiency and logistics chain economy are improved, and information processing is reduced.
Smart Images
Figure CN118735988B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and more specifically, to a method for detecting the remaining capacity of a container. Background Art
[0002] The efficiency of container loading has become a core factor affecting the cost and efficiency of truck transportation. Reasonable cargo loading can not only improve the vehicle's loading capacity, but also optimize the freight process and reduce the overall logistics cost. Therefore, testing the remaining capacity of the container can ensure that each trip achieves the best space utilization, thereby improving the efficiency and economy of the entire logistics chain. When the logistics volume is huge, manual reporting of the container's loading and remaining capacity is time-consuming and laborious, and it is difficult to accurately estimate the remaining space in the container. Information processing is also very cumbersome, which can easily lead to problems such as untimely information acquisition and large errors, causing problems for warehouse management and logistics transportation. Summary of the invention
[0003] With respect to the prior art, the technical problem to be solved by the present application is to provide a method for detecting the remaining capacity of a container based on image vision to conveniently and automatically and intelligently count the remaining capacity in the container.
[0004] In order to solve the above technical problems, in a first aspect, the present application provides a method for detecting the remaining capacity of a container, which comprises:
[0005] Acquire a first image of an interior scene of a container in an empty state, and mark a plurality of corner points corresponding to a plurality of corners inside the container in the first image;
[0006] Selecting a plurality of reference points in the first image, and calculating a mapping relationship between projections of a plurality of corner points and a plurality of reference points;
[0007] Acquire a second image of the interior scene of the container in a cargo-loaded state, extract the contour of the cargo in the second image, and extract a left contour endpoint and a right contour endpoint in the contour;
[0008] The first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint are respectively converted and obtained by the mapping relationship;
[0009] According to the first projection point and the selected reference point, a ratio of the physical height of the cargo on the left side to the physical height of the container and a ratio of the physical length of the cargo on the left side to the physical length of the container in the second image are obtained; according to the second projection point and the selected reference point, a ratio of the physical height of the cargo on the right side to the physical height of the container and a ratio of the physical length of the cargo on the right side to the physical length of the container in the second image are obtained;
[0010] Calculate the volume V of the cargo based on the ratio and container dimensions g , and according to the internal volume V of the containerB and the volume of the cargo V g Get the remaining capacity V of the container S ;
[0011] The horizontal axis bisector of the image is used as the dividing line, the positive direction of the horizontal axis corresponds to the right side and the negative direction of the horizontal axis corresponds to the left side.
[0012] In a possible implementation, the upper left corner of the image is taken as the origin O of the image coordinate system, and the four selected reference points include the origin O (0, 0), point A (a, 0) on the horizontal axis, point B (0, b) on the vertical axis, and point C (a, b);
[0013] Wherein, a is the horizontal coordinate of the set point A on the horizontal axis, and b is the vertical coordinate of the set point B on the vertical axis.
[0014] In a possible implementation, a panoramic camera disposed at the door of the container is used to photograph the interior of the container to obtain an image, and the marked multiple corner points are four corner points corresponding to the four corners of the bottom of the container and four corner points corresponding to the four corners of the top;
[0015] The calculation of the mapping relationship between the projection of multiple corner points to multiple reference points includes:
[0016] The plane perpendicular to the bottom and top of the container and bisecting the container door is used as the boundary dividing the left and right sides. When a person faces the door, the left hand side is the left side and the right hand side is the right side.
[0017] Mark the four corner points on the left side: mark the two left corners of the bottom of the container as corner points P1 and P2, and mark the two left corners of the top of the container as corner points P3 and P4, wherein corner point P1 is farther from the container door than corner point P2, and corner point P3 is farther from the container door than corner point P4;
[0018] Mark the four corner points on the right side: mark the two right corners of the bottom of the container as corner points P5 and P6, and mark the two right corners of the top of the container as corner points P7 and P8, wherein corner point P5 is farther from the container door than corner point P6, and corner point P7 is farther from the container door than corner point P8;
[0019] Map the four left corner points P1, P2, P3 and P4 to points O(0,0), A(a,0), C(0,b), O(a,b) respectively to obtain the first transformation matrix M1;
[0020] The four right corner points P5, P6, P7 and P8 are respectively mapped to points O(0,0), A(a,0), C(0,b), O(a,b) to obtain the first transformation matrix M2.
[0021] In a possible implementation, the first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint are respectively converted and obtained by the mapping relationship as follows:
[0022] Assume that the first transformation matrix M1 = ;
[0023] Assume that the first transformation matrix M2 = ;
[0024] Transform to obtain the first projection point P corresponding to the left contour endpoint L2 (x L2 ,y L2 )for:
[0025] x L2 =(f 11 *x L1 +f 12 *y L1 +f 13 ) / (f 31 *x L1 +f 32 *y L1 +f 33 );
[0026] y L2 =(f 21 *x L1 +f 22 *y L1 +f 23 ) / (f 31 *x L1 +f 32 *y L1 +f 33 );
[0027] Transform to obtain the first projection point P corresponding to the endpoint of the right contour R2 (x R2 ,y R2 )for:
[0028] x R2 =(g 11 *x R1 +g 12 *y R1 +g 13 ) / (g 31 *x R1 +g 32 *y R1 +g 33 );
[0029] y R2 =(g 21 *x R1 +g22 *y R1 +g 23 ) / (g 31 *x R1 +g 32 *y R1 +g 33 );
[0030] Among them, x L1 and L1 are the horizontal and vertical coordinates of the left contour endpoint, respectively. L2 and L2 are the horizontal and vertical coordinates of the first projection point respectively; R1 and R1 are the horizontal and vertical coordinates of the right contour endpoint, respectively. R2 and R2 are the horizontal and vertical coordinates of the second projection point respectively.
[0031] In a possible implementation, a and b are set to satisfy: a / b=W / H; wherein W and H are respectively the width and height of the inner space of the empty container.
[0032] In a possible implementation, extracting the contour of the goods in the second image, and extracting the left contour endpoint and the right contour endpoint in the contour, includes:
[0033] Convert the second image in RGB format to HSV color space;
[0034] Extract the S channel image in the HSV color space;
[0035] The S channel image is binarized to obtain a binarized image with the cargo as the foreground object, and the cargo contour is extracted from the binarized image to obtain a contour image;
[0036] Get the left contour endpoint: use the bisector on the horizontal axis of the contour image as the boundary; start from the upper left corner of the contour image and traverse row by row from left to right: in each row traversal process, when the first foreground point is traversed, record the first foreground point and terminate the row traversal; compare the ordinates of all recorded foreground points, and select the foreground point with the smallest ordinate as the left contour endpoint;
[0037] Get the endpoint of the right contour: use the bisector on the horizontal axis of the contour image as the boundary, and traverse the contour image row by row from right to left starting from the upper right corner point of the contour image: in each row traversal process, when the first foreground point is traversed, record the first foreground point and terminate the row traversal; compare the vertical coordinates of all recorded foreground points, and select the foreground point with the smallest vertical coordinate as the endpoint of the right contour.
[0038] In a possible implementation, the ratio of the physical height of the cargo on the left side of the second image to the physical height of the container is obtained according to the first projection point and the selected reference point. and the ratio of the physical length of the cargo on the left to the physical length of the container : ;
[0039] According to the second projection point and the selected reference point, the ratio of the physical height of the cargo on the right side of the second image to the physical height of the container is obtained. and the ratio of the physical length of the cargo on the right to the physical length of the container : .
[0040] In a possible implementation, the volume V of the cargo is obtained based on the ratio g for:
[0041] ;
[0042] in, , and They are respectively the length, width and height of the interior space of the container when it is empty.
[0043] In a possible implementation, according to the internal volume V of the container B and the volume of the cargo V g Get the remaining capacity V of the container S :V S =V B -V g .
[0044] The beneficial effects of the container remaining capacity detection method provided by the present application are: marking multiple corner points inside the container when it is empty, and obtaining the mapping relationship between the multiple corner points and the selected multiple reference points; extracting the cargo contour in the second image of the container loaded with cargo at the same time, extracting the left contour endpoint and the right contour endpoint of the cargo contour; converting the first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint respectively by the mapping relationship; then estimating the proportional relationship between the physical size of the cargo and the physical size of the container based on the projection point and the selected reference point, and finally obtaining the cargo volume based on the aforementioned proportional relationship and the known container size in the empty state, so as to obtain the remaining volume capacity inside the container. The container remaining capacity detection method is based on image vision to conveniently and automatically calculate the remaining capacity of the container when it is loaded with cargo. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0046] Figure 1 This is a flow chart of the steps of the method for detecting the remaining capacity of a container according to an embodiment of the present application.
[0047] Figure 2 A schematic diagram of a container structure and eight corner points on the container according to an embodiment of the present application.
[0048] Figure 3 This is a schematic diagram of an embodiment of the present application using a panoramic camera disposed at the door of the container to photograph the interior of a container loaded with goods.
[0049] Figure 4 This is a schematic diagram of dividing the left and right sides of a container according to an embodiment of the present application.
[0050] Figure 5 The left contour endpoint P is obtained in the embodiment of the present application. L Schematic diagram of row traversal from left to right when .
[0051] Figure 6 The right contour endpoint P is obtained in the embodiment of the present application. R Diagram of row traversal from right to left when . DETAILED DESCRIPTION
[0052] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0053] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.
[0054] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0055] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0056] The container remaining capacity detection method of the present application is now described in detail with reference to the accompanying drawings.
[0057] Please refer to Figure 1 The container remaining capacity detection method provided in the embodiment of the present application includes the following steps S100 to S600:
[0058] Step S100: Acquire a first image of the interior scene of a container in an empty state, and mark a plurality of corner points corresponding to a plurality of corners inside the container in the first image;
[0059] Step S200: selecting a plurality of reference points in the first image, and calculating a mapping relationship between projections of a plurality of corner points and a plurality of reference points;
[0060] Step S300: acquiring a second image of the interior scene of the container in a cargo-loaded state, extracting the contour of the cargo in the second image, and extracting the left contour endpoint and the right contour endpoint in the contour;
[0061] Step S400: converting and obtaining the first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint respectively according to the mapping relationship;
[0062] Step S500: obtaining, according to the first projection point and the selected reference point, a ratio of the physical height of the cargo on the left side to the physical height of the container and a ratio of the physical length of the cargo on the left side to the physical length of the container in the second image; obtaining, according to the second projection point and the selected reference point, a ratio of the physical height of the cargo on the right side to the physical height of the container and a ratio of the physical length of the cargo on the right side to the physical length of the container in the second image;
[0063] Step S600: Calculate the volume V of the cargo based on the ratio and the container size g, and according to the internal volume V of the container B and the volume of the cargo V g Get the remaining capacity V of the container S ;
[0064] It is worth mentioning that in the above steps, the bisector of the horizontal axis of the image is used as the dividing line, the positive direction of the horizontal axis corresponds to the right side and the negative direction of the horizontal axis corresponds to the left side, that is, whether in the first image or the second image, the bisector of the horizontal axis of the image is used as the dividing line, the positive direction of the horizontal axis corresponds to the right side and the negative direction of the horizontal axis corresponds to the left side.
[0065] It is worth noting that if Figure 2 As shown, the camera for shooting the interior of the container is installed at the door position of the container, and the camera is installed on the top of the container with the camera collection surface facing the inside of the container. The camera is a panoramic camera that can completely shoot the interior view of the container. The container is a rectangular structure. The side of the container parallel to the ground and closest to the ground is the bottom and the side opposite to the bottom is the top. The door of the container is set between the bottom and the top. It can be understood that when loading goods into the container, the goods are stacked in a relatively regular manner in the box, and the goods are stacked from the side opposite to the door. Furthermore, Figure 2 The figure shows the physical width W, physical length L and physical height H of the internal space of the container. Figure 3 As shown, Figure 3 This is a schematic diagram of the interior of a container loaded with goods, captured using a panoramic camera installed at the container door. Figure 3 It also shows the positions of the eight corner points P1, P2, P3, P4, P5, P6, P7 and P8 corresponding to the eight corners inside the container. Figure 2 The eight corner points P1, P2, P3, P4, P5, P6, P7 and P8 in Figure 3 The positions of the eight corner points P1, P2, P3, P4, P5, P6, P7 and P8 are in a one-to-one correspondence, wherein the multiple corner points include four corner points corresponding to the four corners of the bottom of the container and four corner points corresponding to the four corners of the top.
[0066] In the embodiment of the present application, the upper left corner of the image is taken as the origin O of the image coordinate system, and the four selected reference points include the origin O (0, 0), point A (a, 0) on the horizontal axis, point B (0, b) on the vertical axis, and point C (a, b); where a is the horizontal coordinate of point A on the horizontal axis, and b is the vertical coordinate of point B on the vertical axis. Figures 2 to 4 , for step S200: calculating the mapping relationship between the projection of multiple corner points to multiple reference points, including steps S210 to S250:
[0067] Step S210: The plane perpendicular to the bottom and top of the container and bisecting the container door is used as the boundary dividing the left and right sides. Figure 4 Plane in the image; and when a person faces the door, the left hand side is the left side, and the right hand side is the right side; understandably, the left and right sides of the container also correspond to the left and right sides of the image respectively;
[0068] Step S220: Mark four corner points on the left side: mark the two left corners of the bottom of the container as corner points P1 and P2, and mark the two left corners of the top of the container as corner points P3 and P4, wherein corner point P1 is farther from the container door than corner point P2, and corner point P3 is farther from the container door than corner point P4;
[0069] Step S230: Mark four corner points on the right side: mark the two right corners of the bottom of the container as corner points P5 and P6, and mark the two right corners of the top of the container as corner points P7 and P8, wherein corner point P5 is farther from the container door than corner point P6 and corner point P7 is farther from the container door than corner point P8;
[0070] Step S240: Map the four left corner points P1, P2, P3 and P4 to points O(0,0), A(a,0), C(0,b), O(a,b) respectively to obtain a first transformation matrix M1;
[0071] Step S250: Map the four right corner points P5, P6, P7 and P8 to points O(0,0), A(a,0), C(0,b), O(a,b) respectively to obtain a first transformation matrix M2.
[0072] Among them, four points are mapped to another four points to obtain a transformation matrix that reflects the mapping relationship. This is the perspective transformation process. The transformation matrix is a 3×3 matrix. For example, the getPerspectiveTransform() function method in the opencv function library can be used to obtain the transformation matrix in the perspective transformation process. The first transformation matrix M1 and the first transformation matrix M2 are both 3×3 matrices. In an application embodiment, for step S400, the first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint are respectively converted from the mapping relationship to obtain:
[0073] Assume that the first transformation matrix M1 = ;
[0074] Assume that the first transformation matrix M2 = ;
[0075] Transform to obtain the first projection point P corresponding to the left contour endpoint L2 (x L2 ,yL2 )for:
[0076] x L2 =(f 11 *x L1 +f 12 *y L1 +f 13 ) / (f 31 *x L1 +f 32 *y L1 +f 33 );
[0077] y L2 =(f 21 *x L1 +f 22 *y L1 +f 23 ) / (f 31 *x L1 +f 32 *y L1 +f 33 );
[0078] Transform to obtain the first projection point P corresponding to the endpoint of the right contour R2 (x R2 ,y R2 )for:
[0079] x R2 =(g 11 *x R1 +g 12 *y R1 +g 13 ) / (g 31 *x R1 +g 32 *y R1 +g 33 );
[0080] y R2 =(g 21 *x R1 +g 22 *y R1 +g 23 ) / (g 31 *x R1 +g 32 *y R1 +g 33 );
[0081] Among them, x L1 and L1 are the horizontal and vertical coordinates of the left contour endpoint, respectively. L2 and L2are the horizontal and vertical coordinates of the first projection point, respectively, R2 and R2 are the horizontal and vertical coordinates of the right contour endpoint, respectively. R2 and R2 are the horizontal axis coordinate and the vertical axis coordinate of the second projection point respectively. It can be understood that the first transformation matrix and the second transformation matrix of the determined value can be obtained through step S230 and step S240, then f in the first transformation matrix 11 、f 12 、f 13 The values of the nine elements can be determined. Similarly, the g in the second transformation matrix 11 , g 12 , g 13 The values of the other nine elements can also be determined.
[0082] In one embodiment of the application, when setting a and b, the condition is satisfied: a / b=W / H, where, and They are respectively the physical width and height of the internal space of the container when it is empty.
[0083] In one embodiment of the application, for step S500, the ratio of the physical height of the cargo on the left side of the second image to the physical height of the container is obtained according to the first projection point and the selected reference point. and the ratio of the physical length of the cargo on the left to the physical length of the container ; ; According to the second projection point and the selected reference point, obtain the ratio of the physical height of the cargo on the right side of the second image to the physical height of the container and the ratio of the physical length of the cargo on the right to the physical length of the container : .
[0084] In step S600, the volume V of the cargo is obtained according to the ratio g for:
[0085] ,in, , and They are respectively the physical length, physical width and physical height of the internal space of the container when it is empty.
[0086] in, is the volume of the cargo on the left, is the volume of the cargo on the right.
[0087] Furthermore, according to the internal volume V of the container B and the volume of the cargo V g Get the remaining capacity V of the containerS :V S =V B -V g , V B =W×L×H.
[0088] For step S300, refer to Figure 5 and Figure 6 , obtain a second image of the interior scene of the container in the state of loading cargo, extract the contour of the cargo in the second image, and extract the left contour endpoint P in the contour L and the right contour endpoint P R ,include:
[0089] Step S310: converting the second image in RGB format into HSV color space;
[0090] Step S320: extracting the S channel image in the HSV color space;
[0091] Step S330: binarizing the S channel image to obtain a binarized image with the cargo as the foreground object, and extracting the cargo contour from the binarized image to obtain a contour image;
[0092] Step S340: Obtain the left contour endpoint P L : Take the bisector on the horizontal axis of the contour image as the boundary; start from the upper left corner of the contour image and traverse row by row from left to right: in each row traversal process, when the first foreground point is traversed, record the first foreground point and end the row traversal; compare the ordinates of all recorded foreground points, and select the foreground point with the smallest ordinate as the left contour endpoint P L ;
[0093] Step S350: Obtain the right contour endpoint P R : Take the bisector on the horizontal axis of the contour image as the boundary, and take the upper right corner point of the contour image as the starting point to traverse row by row from right to left: In each row traversal process, when the first foreground point is traversed, the first foreground point is recorded and the row traversal is terminated; compare the ordinates of all recorded foreground points, and select the foreground point with the smallest ordinate as the right contour endpoint P R .
[0094] It is worth noting that in a binary image, the pixel value of the foreground point is 255, and the pixel value of the background point is 0. In the embodiment of the present application, the horizontal axis is the X-axis, and the vertical axis is the Y-axis; the horizontal axis coordinate and the horizontal coordinate are both X-axis coordinates, and the vertical axis coordinate and the vertical coordinate are both Y-axis coordinates. The width involved in the image is the length in the X-axis direction, and the height involved in the image is the length in the Y-axis direction. Figure 4 and Figure 5In FIG. 1 , multiple horizontal dashed lines illustrate the process of traversing the lines from top to bottom; specifically, for obtaining the left contour endpoint P in step S340, L In the process of traversing each row, the row is traversed from the left side of the image to the right, and the horizontal axis bisector is used as the boundary during each row traversal, that is, the horizontal coordinate of the rightmost pixel point of each row traversal is half the width of the second image. In addition, when the traversal encounters the first foreground point, the foreground point is recorded and the row traversal is terminated, and the next row traversal is performed. Specifically, for the step S350 of obtaining the right contour endpoint P R In the process, each time a row is traversed, the row is traversed from the right side of the image, and the horizontal axis bisector is used as the boundary during each row traversal, that is, the horizontal coordinate of the leftmost pixel point of each row traversal is half the width of the second image. Furthermore, when the traversal encounters the first foreground point, the foreground point is recorded and the row traversal is terminated, and then the next row traversal is performed.
[0095] It is worth noting that the volume V of the cargo is obtained from the ratio in step S600. g When calculating the volume of the cargo, the volume is divided into the left and right sides. In order to obtain the volume of the cargo on the left and the right, the left contour endpoint P must be obtained first. L and the right contour endpoint P R , that is, to obtain the left and right endpoints respectively, then obtain the left contour endpoint P through image traversal L Or obtain the right contour endpoint P by image traversal R During the process, there is no need to traverse the entire image. The image's horizontal axis bisector is used as the restriction condition for traversal. This can effectively avoid unnecessary processes and save time.
[0096] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for detecting remaining capacity of a container, characterized in that: include: Acquire a first image of an interior scene of a container in an empty state, and mark a plurality of corner points corresponding to a plurality of corners inside the container in the first image; Selecting a plurality of reference points in the first image, and calculating a mapping relationship between projections of a plurality of corner points and a plurality of reference points; Acquire a second image of the interior scene of the container in a cargo-loaded state, extract the contour of the cargo in the second image, and extract a left contour endpoint and a right contour endpoint in the contour; The first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint are respectively converted and obtained by the mapping relationship; According to the first projection point and the selected reference point, a ratio of the physical height of the cargo on the left side to the physical height of the container and a ratio of the physical length of the cargo on the left side to the physical length of the container in the second image are obtained; according to the second projection point and the selected reference point, a ratio of the physical height of the cargo on the right side to the physical height of the container and a ratio of the physical length of the cargo on the right side to the physical length of the container in the second image are obtained; Calculate the volume V of the cargo based on the ratio and container dimensions g , and according to the internal volume V of the container B and the volume of the cargo V g Get the remaining capacity V of the container S ; Among them, the horizontal axis bisector of the image is used as the dividing line, the positive direction of the horizontal axis corresponds to the right side and the negative direction of the horizontal axis corresponds to the left side; The upper left corner of the image is taken as the origin O of the image coordinate system. The four selected reference points include the origin O(0,0), point A(a,0) on the horizontal axis, point B(0,b) on the vertical axis, and point C(a,b); Wherein, a is the horizontal coordinate of the set point A on the horizontal axis, and b is the vertical coordinate of the set point B on the vertical axis; The interior of the container is photographed with a panoramic camera arranged at the door of the container to obtain an image, and the marked multiple corner points are four corner points corresponding to the four corners of the bottom of the container and four corner points corresponding to the four corners of the top; The calculation of the mapping relationship between the projection of multiple corner points to multiple reference points includes: The plane perpendicular to the bottom and top of the container and bisecting the container door is used as the boundary dividing the left and right sides. When a person faces the door, the left hand side is the left side and the right hand side is the right side. Mark the four corner points on the left side: mark the two left corners of the bottom of the container as corner points P1 and P2, and mark the two left corners of the top of the container as corner points P3 and P4, wherein corner point P1 is farther from the container door than corner point P2, and corner point P3 is farther from the container door than corner point P4; Mark the four corner points on the right side: mark the two right corners of the bottom of the container as corner points P5 and P6, and mark the two right corners of the top of the container as corner points P7 and P8, wherein corner point P5 is farther from the container door than corner point P6, and corner point P7 is farther from the container door than corner point P8; Map the four left corner points P1, P2, P3 and P4 to points O(0,0), A(a,0), C(0,b), O(a,b) respectively to obtain the first transformation matrix M1; Map the four right corner points P5, P6, P7 and P8 to points O(0,0), A(a,0), C(0,b), O(a,b) respectively to obtain a second transformation matrix M2; The first projection point corresponding to the left contour endpoint and the second projection point corresponding to the right contour endpoint are obtained by conversion from the mapping relationship: Assume that the first transformation matrix Assume that the second transformation matrix Transform to obtain the first projection point P corresponding to the left contour endpoint L2 (x L2 ,y L2 )for: x L2 =(f 11 *x L1 +f 12 *y L1 +f 13 ) / (f 31 *x L1 +f 32 *y L1 +f 33 ); y L2 =(f 21 *x L1 +f 22 *y L1 +f 23 ) / (f 31 *x L1 +f 32 *y L1 +f 33 ); Transform to obtain the first projection point P corresponding to the endpoint of the right contour R2 (x R2 ,y R2 )for: x R2 =(g 11 *x R1 +g 12 *y R1 +g 13 ) / (g 31 *x R1 +g 32 *y R1 +g 33 ); y R2 =(g 21 *x R1 +g 22 *y R1 +g 23 ) / (g 31 *x R1 +g 32 *y R1 +g 33 ); Among them, x L1 and L1 are the horizontal and vertical coordinates of the left contour endpoint, respectively. L2 and L2 are the horizontal and vertical coordinates of the first projection point respectively; R1 and R1 are the horizontal and vertical coordinates of the right contour endpoint, respectively. R2 and R2 are the horizontal and vertical coordinates of the second projection point respectively.
2. The method for detecting remaining capacity of a container according to claim 1, characterized in that: The settings of a and b satisfy: a / b=W / H; where W and H are respectively the width and height of the inner space of the empty container.
3. The method for detecting remaining capacity of a container according to claim 1, characterized in that: Extracting the contour of the goods in the second image, and extracting the left contour endpoint and the right contour endpoint in the contour, including: Convert the second image in RGB format to HSV color space; Extract the S channel image in the HSV color space; The S channel image is binarized to obtain a binarized image with the cargo as the foreground object, and the cargo contour is extracted from the binarized image to obtain a contour image; Get the left contour endpoint: use the bisector on the horizontal axis of the contour image as the boundary; start from the upper left corner of the contour image and traverse row by row from left to right: in each row traversal process, when the first foreground point is traversed, record the first foreground point and terminate the row traversal; compare the ordinates of all recorded foreground points, and select the foreground point with the smallest ordinate as the left contour endpoint; Get the endpoint of the right contour: use the bisector on the horizontal axis of the contour image as the boundary, and traverse the contour image row by row from right to left starting from the upper right corner point of the contour image: in each row traversal process, when the first foreground point is traversed, record the first foreground point and terminate the row traversal; compare the vertical coordinates of all recorded foreground points, and select the foreground point with the smallest vertical coordinate as the endpoint of the right contour.
4. The method for detecting remaining capacity of a container according to claim 1, characterized in that: According to the first projection point and the selected reference point, the ratio B of the physical height of the left cargo to the physical height of the container in the second image is obtained. L_h And the ratio of the physical length of the cargo on the left to the physical length of the container B L_L : According to the second projection point and the selected reference point, the ratio B of the physical height of the cargo on the right side of the second image to the physical height of the container is obtained. R_h And the ratio B of the physical length of the cargo on the right to the physical length of the container R_L :
5. The method for detecting remaining capacity of a container according to claim 4, characterized in that: Calculate the volume V of the cargo based on the ratio g for: Among them, L, W and H are the length, width and height of the internal space of the container when it is empty.
6. The method for detecting remaining capacity of a container according to claim 1, characterized in that: And according to the internal volume V of the container B and the volume of the cargo V g Get the remaining capacity V of the container S :V S =V B -V g .