Container distance measurement method, system and storage medium

Abstract

Embodiments of the present invention provide a container distance measurement method, system, and storage medium. The method includes: using a calibrated camera to obtain outline information of a first container, detecting the four corner fittings of the first container, and obtaining the center point coordinates of the four corner fittings; calculating a first distance based on the center point coordinates of the four corner fittings; obtaining vehicle information and container information of a vehicle, and determining whether the vehicle's trailer can load a second container based on the vehicle information, container information, and the first distance; and if the vehicle's trailer can load the second container, calculating the distance the vehicle needs to travel based on the vehicle information, container information, and the first distance so that the spreader can completely place the second container on the vehicle's trailer without interfering with the first container. By utilizing the characteristics of the container corner fittings to measure container distance, the present invention can accurately determine the distance the vehicle needs to travel to load the second container and ensure that the second container is accurately loaded onto the trailer.

Description

Technical Field

[0001] The present invention relates to the field of automatic control technology, and in particular to a container distance measurement method, system and storage medium. Background Art

[0002] When an autonomous port vehicle is loading and unloading containers, it must travel to the yard or under the gantry crane of a quay crane. The gantry crane places the grabbed container onto the autonomous port vehicle's trailer or grabs the container on the trailer. To ensure that the gantry crane can accurately place or grab the container, the existing autonomous port vehicles mostly use sensors to identify some local features of the gantry crane to achieve precise parking of the vehicle, ensuring that the vehicle's trailer is aligned with the container grabbed by the gantry crane, or the gantry crane is aligned with the container on the trailer. However, the aforementioned methods for achieving precise parking of vehicles are mostly aimed at the loading and unloading of a large container. During the loading and unloading operation, if two small containers need to be placed on the autonomous port vehicle, relying solely on precise parking cannot accurately provide the parking position for the vehicle for the second loading, and thus may not be able to complete the loading and unloading operation, and may even cause friction between the two containers or the inability to load the latter container. Summary of the Invention

[0003] This specification provides a container distance measurement method, system, and storage medium to overcome at least one technical problem in the prior art.

[0004] In a first aspect, according to an embodiment of this specification, a container distance measurement method is provided, comprising:

[0005] Using a calibrated camera to obtain contour information of a first container, and detecting four corner fittings of the first container to obtain the coordinates of the center points of the four corner fittings; wherein the camera is mounted on a side of the front of a vehicle used to load the container, close to a trailer of the vehicle;

[0006] Calculating a first distance based on the center point coordinates of the four corner pieces; wherein the first distance is the distance between a side of the first container close to the camera and the camera;

[0007] Obtaining vehicle information and container information of the vehicle, and determining whether the trailer of the vehicle can be loaded with a second container based on the vehicle information, the container information, and the first distance; wherein the vehicle information includes the standard length of the trailer of the vehicle and the distance between the camera and the front of the trailer of the vehicle, and the container information includes size information of the first container and the second container;

[0008] When the trailer of the vehicle can load the second container, the distance that the vehicle needs to travel is calculated based on the vehicle information, the container information, and the first distance so that the spreader can completely place the second container on the trailer of the vehicle without interfering with the first container.

[0009] Optionally, also include:

[0010] The camera is mounted on a side of the front of the vehicle close to the trailer of the vehicle, with the optical axis of the camera parallel to the trailer of the vehicle; wherein the camera is a monocular camera;

[0011] The camera is calibrated to obtain the camera's intrinsic parameter matrix, distortion parameters, and extrinsic parameter matrix.

[0012] Further optionally, calibrating the camera to obtain the intrinsic parameter matrix, distortion parameters, and extrinsic parameter matrix of the camera includes:

[0013] vertically placing a calibration plate on the trailer of the vehicle, and making the calibration plate perpendicular to the optical axis of the camera;

[0014] Using the camera to take a plurality of calibration plate images at various angles of the calibration plate;

[0015] The intrinsic parameters and distortion of the camera are calibrated according to the multiple calibration plate images to obtain the intrinsic parameter matrix and distortion parameters of the camera, and the extrinsic parameters of the camera are calibrated according to the multiple calibration plate images to obtain the extrinsic parameter matrix of the camera.

[0016] Further optionally, the using the calibrated camera to obtain the contour information of the first container and detecting four corner fittings of the first container to obtain the center point coordinates of the four corner fittings includes:

[0017] Using the camera, capturing a container image of the first container;

[0018] Dedistorting the container image according to the camera's intrinsic parameter matrix and distortion parameters to obtain a dedistorted rectangular image, and acquiring contour information of the first container; the contour information of the first container includes contour pixel values of the first container;

[0019] Using the four corner points of the rectangular image as the center points of the ROIs of the four corner fittings, and setting the size of the ROIs to four times the pixel size of the corner fittings in the container image;

[0020] The four ROIs are sequentially used as inputs to the trained lightweight semantic segmentation network BiseNetV2 to obtain the center point coordinates of the four corner pieces.

[0021] Further optionally, the trained lightweight semantic segmentation network BiseNetV2 is formed through the following training steps:

[0022] Build a lightweight semantic segmentation network BiseNetV2;

[0023] Using the camera to acquire training images; the training images are corner piece images of the four corner pieces at various angles taken by the camera;

[0024] The center point coordinates of the corner piece targets in the corner piece image are marked to obtain training data, and the constructed lightweight semantic segmentation network BiseNetV2 is trained to obtain a trained lightweight semantic segmentation network BiseNetV2.

[0025] Optionally, calculating the first distance according to the center point coordinates of the four corner pieces includes:

[0026] Calculating the coordinates of the center point of a side of the first container close to the camera based on the coordinates of the center points of the four corner pieces;

[0027] Obtain the container information and the focal length of the camera, and obtain the first distance l by using the PNP algorithm based on the container information, the focal length of the camera, and the coordinates of the center point of the side of the first container close to the camera. cam .

[0028] Further optionally, obtaining the vehicle information and container information of the vehicle, and determining whether the trailer of the vehicle can be loaded with a second container based on the vehicle information, the container information, and the first distance, includes:

[0029] Obtain and calculate the distance l between the camera and the front of the trailer of the vehicle head , the standard length l of the first container box1 and the first distance l cam , calculate the second distance l first , wherein the second distance is the distance between the side of the first container away from the camera and the front of the trailer of the vehicle, and the second distance l first The calculation formula is:

[0030] l first =l box1 +(l cam -l head )

[0031] Get the standard length l of the vehicle's trailer trailer, according to the second distance l first and the standard length l of the vehicle's trailer trailer Calculate the third distance l after , wherein the third distance is the distance between the side of the first container away from the camera and the rear end of the trailer of the vehicle, and the third distance l after The calculation formula is:

[0032] l after =l trailer -l first

[0033] Get the standard length l of the second container box2 , the third distance l after The standard length of the second container is l box2 The difference between the preset remaining space l threshold A comparison is made to determine whether the trailer of the vehicle can be loaded with the second container.

[0034] Further optionally, when the trailer of the vehicle can be loaded with the second container, calculating the distance that the vehicle needs to travel based on the vehicle information, the container information, and the first distance includes:

[0035] When the third distance l after The standard length of the second container is l box2 The difference is greater than the preset remaining space l threshold , it is determined that the trailer of the vehicle can be loaded with the second container;

[0036] Obtain and according to the standard length l of the first container box1 And the third distance l after , calculate the distance l that the vehicle needs to travel drive , the distance l that the vehicle needs to travel drive The calculation formula is:

[0037]

[0038] In a second aspect, according to an embodiment of this specification, a container ranging system is provided, which is implemented based on the container ranging method described in the first aspect and includes:

[0039] a first container information acquisition module, configured to acquire contour information of the first container using a calibrated camera, and detect four corner fittings of the first container to obtain center point coordinates of the four corner fittings;

[0040] a first distance calculation module, configured to calculate a first distance based on the coordinates of the center points of the four corner pieces; wherein the first distance is the distance between a side of the first container close to the camera and the camera;

[0041] a judgment module, configured to obtain vehicle information and container information of the vehicle, and judge whether the trailer of the vehicle can be loaded with a second container based on the vehicle information, the container information and the first distance;

[0042] a travel distance calculation module for calculating, when the trailer of the vehicle can carry the second container, the distance that the vehicle needs to travel based on the vehicle information, the container information, and the first distance, so that the spreader can completely place the second container on the trailer of the vehicle without interfering with the first container.

[0043] In a third aspect, according to an embodiment of this specification, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the container distance measurement method described in the first aspect is implemented.

[0044] The beneficial effects of the embodiments of this specification are as follows:

[0045] An inexpensive monocular camera is used, and the features of container corner fittings are utilized in combination with the lightweight semantic segmentation network BiseNetV2 to detect the four corner fittings of the first container and obtain their center point coordinates. The position of the first container is then measured using the PNP algorithm, the four center point coordinates, and the known container dimensions. This is used to determine whether the second container can be loaded, and to accurately determine the distance the vehicle needs to move to load the second container, to ensure that the second container is accurately loaded onto the trailer. This method greatly reduces the manual operation process, making the double-container loading process no longer dependent on human judgment, and no longer requiring the vehicle to repeatedly adjust its parking position. Only one parking position adjustment is required to quickly complete the double-container loading operation, resulting in higher loading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flow chart of a container distance measurement method provided in an embodiment of this specification;

[0048] Figure 2 This is a schematic diagram of distance calculation for the container distance measurement method provided in the embodiments of this specification. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of this specification and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to the process, method, product, or apparatus.

[0051] This specification discloses a container distance measurement method, primarily used to determine the distance a vehicle must travel between loading containers. This method combines semantic segmentation and the PNP algorithm, utilizing container corner fitting features to accurately determine the distance required for loading a second container. This ensures the second container is accurately loaded onto a trailer without interfering with the first. Detailed descriptions are provided below.

[0052] Figure 1 A container distance measurement method according to an embodiment of this specification is shown. Figure 1 As shown, the container distance measurement method includes the following steps:

[0053] Step 100: Use the calibrated camera to obtain the outline information of the first container, detect the four corner fittings of the first container, and obtain the center point coordinates of the four corner fittings.

[0054] In existing technology, most of the sensors used by port vehicles are LiDARs. While LiDARs offer high accuracy, they are also expensive. Furthermore, due to the harsh port environment, with frequent rain, snow, and windy weather, raindrops, snowflakes, and windblown dust can significantly weaken the radar's light waves, reducing the signal's return rate and affecting the LiDAR's accuracy. Furthermore, the number of LiDARs installed on vehicles is large, and the amount of data generated per second is enormous, placing high demands on the computing power and bandwidth of industrial computers. Furthermore, during ship loading and unloading operations, multiple port vehicles operate simultaneously, and the distances between them are relatively close, which can lead to interference between radar signals, further reducing radar accuracy.

[0055] To address the above problems, the container ranging method in the embodiments of this specification uses a monocular camera to measure the distance of the container. Only a monocular camera needs to be installed on the rear side of the front of the vehicle (that is, the side of the front of the vehicle close to the trailer of the vehicle). This not only greatly reduces the cost, but also solves the problem that the detection accuracy of the laser radar is affected by harsh environmental factors and the mutual interference of radar signals between vehicles affects the detection accuracy. In addition, the monocular camera has high detection accuracy, occupies less bandwidth, and has a fast detection speed.

[0056] In one embodiment, if Figure 2 As shown, the camera 1 is installed on the side of the vehicle's front 2 close to the vehicle's trailer 3. It should be noted that the vehicle described in this article refers to a vehicle used to load containers, for example, an autonomous port vehicle. The vehicle includes a front 2 and a trailer 3. The container is loaded on the trailer 3. The front 2 is a container tractor, which is used to tow the trailer 3 carrying the container.

[0057] At the same time, this camera refers to the camera that has been calibrated after installation. The following describes the camera installation and calibration process:

[0058] (A) Install the camera on the front of the vehicle, close to the side of the vehicle's trailer, with the camera's optical axis parallel to the vehicle's trailer.

[0059] In one embodiment, if Figure 2 As shown, the camera 1 is placed on the rear side of the vehicle head 2 (i.e., the side of the vehicle head 2 close to the trailer 3). During the installation of the camera 1, it is necessary to ensure that the optical axis of the camera 1 is parallel to the trailer 3 after installation, and the installation height of the camera 1 is as high as possible to the container loaded on the trailer 3 (in the Figure 2 The middle refers to the height adaptation of the first container 4) to ensure that the entire container can appear in the image captured by the camera 1.

[0060] (B) Calibrate the camera to obtain the camera's intrinsic parameter matrix, distortion parameters, and extrinsic parameter matrix.

[0061] In one embodiment, a calibration target is placed vertically on the vehicle's trailer, perpendicular to the camera's optical axis. The target's surface must be flat and free of curvature to ensure accurate calibration results. It should be noted that the calibration target mentioned above refers to a flat plate with a fixed-pitch pattern array. The camera captures the flat plate with the fixed-pitch pattern array, and the calibration algorithm calculates the camera's geometric model and determines the camera's internal and external parameters. This completes the calibration process, enabling the camera to achieve high-precision measurement and image reconstruction results.

[0062] Specifically, a camera is used to capture multiple images of the calibration plate at various angles of the calibration plate, that is, the camera is used to capture images of the calibration plate at various angles so that the calibration plate can cover all positions in the image captured by the camera. The intrinsic parameters and distortion of the camera are calibrated according to the multiple images of the calibration plate captured at different angles to obtain the intrinsic parameter matrix and distortion parameters of the camera. The extrinsic parameters of the camera are calibrated according to the multiple images of the calibration plate captured at different angles to obtain the extrinsic parameter matrix of the camera.

[0063] In a specific implementation process, the world coordinate system O is established. W _X W Y W Z W , as the reference coordinate system to describe the position of the camera in three-dimensional space, and use it to describe the position of any other objects placed in this three-dimensional space environment, where Z W The axis is perpendicular to the plane of the calibration plate, X W The Y axis is horizontal to the left (ie, toward the left side of the image taken by the camera), W The axis is pointed vertically downward (ie perpendicular to the ground).

[0064] Establish camera coordinate system O C _X C Y C Z C , to describe the object position from the perspective of the camera, where the optical center of the camera is the coordinate origin, Z C The X axis is the optical axis of the camera. C Axis horizontal to the right, Y C Axis vertically downward; establish the image coordinate system O_xy, the coordinate origin is the intersection of the camera's optical axis and the camera imaging plane, the x-axis is the horizontal right of the camera image, and the y-axis is the vertical downward of the camera image. C The axis is parallel to the x-axis, and the Y C The u-axis is parallel to the y-axis of the image coordinate system; establish the pixel coordinate system O_uv, with the coordinate origin being the upper left corner of the image taken by the camera, the u-axis being parallel to the x-axis of the image coordinate system, and the v-axis being parallel to the y-axis of the image coordinate system.

[0065] The conversion relationship between the pixel coordinate system O_uv and the image coordinate system O_xy is:

[0066]

[0067] Using homogeneous coordinates and matrix form, the above formula can be expressed as:

[0068]

[0069] In the above two formulas, (u0, v0) is the coordinate of the origin of the image coordinate system in the pixel coordinate system, and dx and dy are the physical sizes of each pixel in the x-axis and x-axis directions of the image plane, respectively.

[0070] Assuming that there is a point M in the camera coordinate system, the coordinates of the imaging point P in the ideal image coordinate system (without distortion) (which can be obtained by the principle of similar triangles) are:

[0071]

[0072] Wherein, in the above formula, f is the focal length of the camera.

[0073] In matrix form, the above formula can be expressed as:

[0074]

[0075] From the above formula, the intrinsic parameter matrix M of the camera is:

[0076]

[0077] Among them, (u0, v0) is the coordinate of the origin of the image coordinate system in the pixel coordinate system, the unit is pixel, f x 、f y The unit is piece (ie number of pixels).

[0078] Due to manufacturing and assembly errors in actual camera optical systems, lenses cannot satisfy the ideal object-image relationship of similar triangles. Therefore, there is distortion between the actual image formed on the camera image plane and the ideal image, including radial distortion and tangential distortion. The first few terms of the Taylor series expansion at r = 0 are used to approximately describe radial distortion, requiring three distortion parameters (k1, k2, k3) to solve. If tangential distortion is present, a rectangle projected onto the imaging plane is likely to become a trapezoid, requiring two additional distortion parameters (p1, p2) to solve. That is, the camera's distortion parameters are k1, k2, k3, p1, and p2.

[0079] Since the world coordinate system and the camera coordinate system generally do not coincide, if a point P in the world coordinate system is to be projected onto the image plane, the coordinates of the point P must first be converted to the camera coordinate system. Let the coordinates of the point P in the world coordinate system be (X W ′,Y W ′,Z W ′), which is converted to the camera coordinate system as (X C ′,Y C ′,Z C ′), the relative rotation between the world coordinate system and the camera coordinate system is the matrix R (a rotation matrix with three rows and three columns), and the relative displacement is the vector T (a translation vector with three rows and one column). The transformation matrix is represented by a homogeneous coordinate matrix composed of a rotation matrix and a translation vector:

[0080]

[0081] This is the camera's extrinsic parameter matrix.

[0082] From the above process, we get the camera's intrinsic parameter matrix M, the camera's distortion parameters are k1, k2, k3, p1, p2, and the camera's extrinsic parameter matrix This completes the camera calibration.

[0083] Since the port crane needs to accurately place the container on the vehicle trailer, the parking position of the vehicle is required to be relatively precise. In the embodiment of this specification, Figure 2 As shown, the vehicle needs to load two small containers (i.e., a first container 4 and a second container 5). After the first container 4 is loaded, the position of the second vehicle needs to be further determined to ensure the accurate loading of the second container 5. Therefore, a camera 1 installed on the rear side of the vehicle head 2 and calibrated is required to measure the distance of the first container 4.

[0084] In one embodiment, a calibrated camera mounted on the rear side of a vehicle is used to capture a first container loaded on a trailer to obtain a container image of the first container. The container image is then dedistorted based on the camera's intrinsic parameter matrix M and distortion parameters k1, k2, k3, p1, and p2, resulting in a dedistorted rectangular image. Specifically, the image of the first container captured by the camera is binarized, and the findContours function in the OpenCV library is used to locate contours within the current container image. Because prior information about the container is pre-determined, the pixel size of the first container in the container image can be determined. Therefore, upper and lower thresholds for the circumscribed rectangle of the contour are set, and the detected contours are filtered. The remaining contours are sorted, and the pixel values closest to the pre-determined container pixel values are considered the first container contour pixel values. This results in the first container contour information. Based on this information, a circumscribed rectangular shape is fitted to obtain the dedistorted rectangular image. Among them, the container ranging method pre-sets the initial range of the image according to the actual size of the container, that is, prior information, to reduce the time spent on online container ranging and make it easier to find the outline of the container.

[0085] Each corner of a container is fitted with a corner fitting. The standard corner fittings measure 178×162×118mm, and each has a hole. Based on the inherent characteristics of these corner fittings, a lightweight semantic segmentation network (BiseNetV2) is used to locate the corner fittings. Images of the four corner fittings of the container, taken from various angles, are trained using a camera. In a specific implementation, a lightweight semantic segmentation network (BiseNetV2) is constructed. Images of the four corner fittings of a container loaded on a trailer, taken from various angles, are used as training images for the lightweight semantic segmentation network (BiseNetV2). The center coordinates of the corner fitting targets in the corner fitting images are annotated to obtain training data. The constructed lightweight semantic segmentation network (BiseNetV2) is trained based on the obtained training data, resulting in a trained lightweight semantic segmentation network (BiseNetV2). The trained lightweight semantic segmentation network (BiseNetV2) is then used to calculate the specific locations of the container's four corner fittings.

[0086] In this embodiment, the four corner points of the first container are roughly determined using its outline information. Based on this, the ROIs (Regions of Interest) for the four corner fittings are then determined. Subsequently, the four corner fittings are detected using the trained lightweight semantic segmentation network BiseNetV2, and the coordinates of their center points are obtained. Specifically, the four corner points of the rectangular image obtained above are used as the center points of the ROIs for the four corner fittings of the first container. The ROI size is set to four times the pixel size of the corner fittings in the container image. The four ROIs are then sequentially fed into the trained lightweight semantic segmentation network BiseNetV2 to obtain the coordinates of the four corner fittings' center points. Due to the small size of the ROIs, the network runs quickly, reduces CPU usage, and significantly improves detection speed.

[0087] Step 200: Calculate a first distance based on the center point coordinates of the four corner pieces.

[0088] Among them, such as Figure 2 As shown, the first distance l cam It refers to the distance between the side of the first container 4 close to the camera 1 and the camera 1.

[0089] Specifically, based on the center point coordinates of the four corner pieces obtained in step 100, the center point coordinates of the side of the first container close to the camera can be calculated, and the container information and the focal length f of the camera can be obtained. The container information includes the size information of the first container and the second container. Since the size information of the first container, the focal length of the camera, and the center point coordinates of the side of the first container close to the camera are known, the distance between the side of the first container close to the camera and the camera can be obtained through the PNP algorithm, that is, the first distance l is obtained. cam .

[0090] It should be noted that the first container and the second container may both be small standard containers with the same size information, or two small containers with different size information may be selected, and this embodiment of the present specification does not limit this.

[0091] Step 300: Acquire vehicle information and container information of the vehicle, and determine whether the trailer of the vehicle can be loaded with the second container based on the vehicle information, container information and the first distance.

[0092] By calculating the distance of the empty space after the vehicle trailer is loaded with the first container, it is determined whether the vehicle trailer can still load the second container. The vehicle information includes the standard length l of the vehicle trailer. trailer , the distance between the camera and the front of the vehicle's trailer l headThe container information includes the size information of the first container and the second container, that is, the standard length l of the first container box1 And the standard length of the second container l box2 It should be noted that when the first container and the second container are small containers of the same size, box1 With l box2 equal.

[0093] In one embodiment, the distance l between the camera and the front of the trailer of the vehicle is obtained. head 、The standard length of the first container l box1 and the first distance l obtained in step 200 cam , according to the distance l between the camera and the front of the vehicle's trailer head 、The standard length of the first container l box1 and the first distance l cam , calculate the distance between the side of the first container away from the camera and the front of the vehicle trailer, and record it as the second distance l first , specifically, Figure 2 As shown, the second distance l first The calculation formula is:

[0094] l first =l box1 +(l cam -l head )

[0095] Then, get the standard length l of the vehicle's trailer trailer , according to the second distance l first and the standard length of the vehicle's trailer trailer The distance between the side of the first container away from the camera and the rear of the vehicle trailer is calculated, that is, the distance of the empty space after the vehicle trailer is loaded with the first container, which is recorded as the third distance l after , specifically, Figure 2 As shown, the third distance l after The calculation formula is:

[0096] l after =l trailer -l first

[0097] Then obtain the standard length l of the second container box2 , the third distance l after The standard length of the second container l box2 Difference, i.e. l after -l box2 , according to the third distance l after The standard length of the second container l box2The difference between the vehicle's trailer and the vehicle's trailer determines whether the vehicle's trailer can be loaded with the second container. In a specific embodiment, the third distance l after The standard length of the second container l box2 The difference between the preset remaining space l threshold For comparison, when the third distance l after The standard length of the second container l box2 The difference is not greater than the preset remaining space l threshold , that is, l after -l box2 ≤l threshold When the third distance l is reached, it is determined that the vehicle's trailer cannot be loaded with the second container, and the process goes from step 300 to step 400, where the vehicle leaves the quay crane. after The standard length of the second container l box2 The difference is greater than the preset remaining space l threshold , that is, l after -l box2 >l threshold When the vehicle trailer is determined to be able to load the second container, the process proceeds from step 300 to step 500 to calculate the distance the vehicle needs to travel to load the second container. threshold It is artificially set, for example, threshold Can be set to 20cm.

[0098] Step 400: The vehicle leaves the quay crane.

[0099] Step 500: Calculate the distance the vehicle needs to travel based on the vehicle information, the container information, and the first distance so that the spreader can completely place the second container on the trailer of the vehicle without interfering with the first container.

[0100] In one embodiment, the standard length l of the first container is obtained. box1 and the third distance l obtained in step 300 after , according to the standard length l of the first container box1 and the third distance l after , calculate the distance l that the vehicle needs to travel drive , specifically, Figure 2 As shown, the distance the vehicle needs to travel is l drive The calculation formula is:

[0101]

[0102] The vehicle continues to move forward driveThe distance between the lifting device and the first container is set, so that the center of the spreader currently loading the first container moves from the center position of the first container to the center position of the vacant space after the trailer is loaded with the first container, so that the second container can be accurately placed at the center of the vacant space after the trailer is loaded with the first container by using the spreader, ensuring the stable placement of the second container and avoiding collision between the second container and the first container during the placement process. At the same time, friction between the vehicles during driving due to the small distance between the second container and the first container is avoided.

[0103] In summary, this specification discloses a container ranging method, which adopts an inexpensive monocular camera, utilizes the container corner fitting features, and combines with the lightweight semantic segmentation network BiseNetV2 to detect the four corner fittings of the first container and obtain their center point coordinates. Then, the position of the first container is measured in combination with the PNP algorithm and the four center point coordinates, known container dimensions, etc., so as to determine whether the second container can be loaded, and accurately determine the distance the vehicle needs to move to load the second container, so as to ensure that the second container is accurately loaded onto the trailer. This method greatly reduces the manual operation process, so that the process of double-box packing operation no longer relies on human judgment, and the vehicle no longer needs to repeatedly adjust its parking position. Only one adjustment of the parking position is required to quickly complete the double-box packing operation, and the packing efficiency is higher.

[0104] Corresponding to the aforementioned container distance measurement method embodiment, an embodiment of the present invention further provides a container distance measurement system for executing the steps of the container distance measurement method in the aforementioned embodiment. The container distance measurement system includes a first container information acquisition module, a first distance calculation module, a judgment module, and a travel distance calculation module.

[0105] Specifically, the first container information acquisition module is used to acquire the contour information of the first container using the calibrated camera, and detect the four corner fittings of the first container to obtain the center point coordinates of the four corner fittings.

[0106] The first distance calculation module is used to calculate a first distance based on the center point coordinates of the four corner pieces, wherein the first distance is the distance between a side of the first container close to the camera and the camera.

[0107] The judgment module is used to obtain vehicle information and container information of the vehicle, and judge whether the trailer of the vehicle can be loaded with the second container according to the vehicle information, container information and the first distance.

[0108] The driving distance calculation module is used to calculate the distance the vehicle needs to travel based on the vehicle information, container information and the first distance when the vehicle trailer can be loaded with the second container, so that the spreader can completely place the second container on the vehicle trailer without interfering with the first container.

[0109] It should be noted that the container ranging system provided in the embodiment of the present invention is based on the same concept as the container ranging method of the present invention, and the technical effects it brings are the same as those of the container ranging method embodiment of the present invention. For matters not mentioned in this embodiment, please refer to the description in the container ranging method embodiment of the present invention, and no further details will be given here.

[0110] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, the container distance measurement method of the aforementioned embodiment is implemented.

[0111] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0112] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A container distance measurement method, characterized in that: include: Using a calibrated camera to obtain contour information of a first container, and detecting four corner fittings of the first container to obtain the coordinates of the center points of the four corner fittings; wherein the camera is mounted on a side of the front of a vehicle used to load the container, close to a trailer of the vehicle; Calculating a first distance based on the center point coordinates of the four corner pieces; wherein the first distance is the distance between a side of the first container close to the camera and the camera; Obtaining vehicle information and container information of the vehicle, and determining whether the trailer of the vehicle can be loaded with a second container based on the vehicle information, the container information, and the first distance; wherein the vehicle information includes the standard length of the trailer of the vehicle and the distance between the camera and the front of the trailer of the vehicle, and the container information includes size information of the first container and the second container; When the trailer of the vehicle can load the second container, the distance that the vehicle needs to travel is calculated based on the vehicle information, the container information, and the first distance so that the spreader can completely place the second container on the trailer of the vehicle without interfering with the first container.

2. The container distance measurement method according to claim 1, characterized in that: Also includes: The camera is mounted on a side of the front of the vehicle close to the trailer of the vehicle, with the optical axis of the camera parallel to the trailer of the vehicle; wherein the camera is a monocular camera; The camera is calibrated to obtain the camera's intrinsic parameter matrix, distortion parameters, and extrinsic parameter matrix.

3. The container distance measurement method according to claim 2, characterized in that: The step of calibrating the camera to obtain the camera's intrinsic parameter matrix, distortion parameters, and extrinsic parameter matrix includes: vertically placing a calibration plate on the trailer of the vehicle, and making the calibration plate perpendicular to the optical axis of the camera; Using the camera to take a plurality of calibration plate images at various angles of the calibration plate; The intrinsic parameters and distortion of the camera are calibrated according to the multiple calibration plate images to obtain the intrinsic parameter matrix and distortion parameters of the camera, and the extrinsic parameters of the camera are calibrated according to the multiple calibration plate images to obtain the extrinsic parameter matrix of the camera.

4. The container distance measurement method according to claim 2, characterized in that: The step of acquiring the contour information of the first container using the calibrated camera and detecting four corner fittings of the first container to obtain the center point coordinates of the four corner fittings includes: Using the camera, capturing a container image of the first container; Dedistorting the container image according to the camera's intrinsic parameter matrix and distortion parameters to obtain a dedistorted rectangular image, and acquiring contour information of the first container; the contour information of the first container includes contour pixel values of the first container; Using the four corner points of the rectangular image as the center points of the ROIs of the four corner fittings, and setting the size of the ROIs to four times the pixel size of the corner fittings in the container image; The four ROIs are sequentially used as inputs to the trained lightweight semantic segmentation network BiseNetV2 to obtain the center point coordinates of the four corner pieces.

5. The container distance measurement method according to claim 4, characterized in that: The trained lightweight semantic segmentation network BiseNetV2 is formed through the following training steps: Build a lightweight semantic segmentation network BiseNetV2; Using the camera to acquire training images; the training images are corner piece images of the four corner pieces at various angles taken by the camera; The center point coordinates of the corner piece targets in the corner piece image are marked to obtain training data, and the constructed lightweight semantic segmentation network BiseNetV2 is trained to obtain a trained lightweight semantic segmentation network BiseNetV2.

6. The container distance measurement method according to claim 1, characterized in that: The calculating the first distance according to the center point coordinates of the four corner pieces includes: Calculating the coordinates of the center point of a side of the first container close to the camera based on the coordinates of the center points of the four corner pieces; Obtain the container information and the focal length of the camera, and obtain the first distance l by using the PNP algorithm based on the container information, the focal length of the camera, and the coordinates of the center point of the side of the first container close to the camera. cam .

7. The container distance measurement method according to claim 6, characterized in that: The acquiring the vehicle information and the container information of the vehicle, and determining whether the trailer of the vehicle can be loaded with the second container according to the vehicle information, the container information and the first distance, includes: Obtain and calculate the distance l between the camera and the front of the trailer of the vehicle head , the standard length l of the first container box1 and the first distance l cam , calculate the second distance l first , wherein the second distance is the distance between the side of the first container away from the camera and the front of the trailer of the vehicle, and the second distance l first The calculation formula is: l first ＝l box1 +(l cam -l head ) Get the standard length l of the vehicle's trailer trailer , according to the second distance l first and the standard length l of the vehicle's trailer trailer Calculate the third distance l after , wherein the third distance is the distance between the side of the first container away from the camera and the rear end of the trailer of the vehicle, and the third distance l after The calculation formula is: l after ＝l trailer -l first Get the standard length l of the second container box2 , the third distance l after The standard length of the second container is l box2 The difference between the preset remaining space l threshold A comparison is made to determine whether the trailer of the vehicle can be loaded with the second container.

8. The container distance measurement method according to claim 7, characterized in that: When the trailer of the vehicle can be loaded with the second container, calculating the distance that the vehicle needs to travel based on the vehicle information, the container information, and the first distance includes: When the third distance l after The standard length of the second container is l box2 The difference is greater than the preset remaining space l threshold , it is determined that the trailer of the vehicle can be loaded with the second container; Obtain and according to the standard length l of the first container box1 And the third distance l after , calculate the distance l that the vehicle needs to travel drive , the distance l that the vehicle needs to travel drive The calculation formula is: 。 9. A container distance measurement system, characterized in that: The container distance measurement method according to any one of claims 1 to 8 is implemented, comprising: a first container information acquisition module, configured to acquire contour information of the first container using a calibrated camera, and detect four corner fittings of the first container to obtain center point coordinates of the four corner fittings; a first distance calculation module, configured to calculate a first distance based on the coordinates of the center points of the four corner pieces; wherein the first distance is the distance between a side of the first container close to the camera and the camera; a judgment module, configured to obtain vehicle information and container information of the vehicle, and judge whether the trailer of the vehicle can be loaded with a second container based on the vehicle information, the container information and the first distance; a travel distance calculation module for calculating, when the trailer of the vehicle can carry the second container, the distance that the vehicle needs to travel based on the vehicle information, the container information, and the first distance, so that the spreader can completely place the second container on the trailer of the vehicle without interfering with the first container.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the container distance measurement method according to any one of claims 1 to 8 is implemented.

Images