Trailer positioning system and method based on vision
Through the vision-based trailer positioning system, using high-definition cameras to build three-dimensional maps and combined with AR navigation, the traditional system's lack of flexibility in bad weather and achieves more accurate and flexible trailer positioning.
Patent Information
- Application Number
- CN202411632969.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-04-22
AI Technical Summary
In the prior art, traditional trailer intelligent positioning system can easily lead to inaccurate alignment information when inclement weather or damaged reflector strips, and has low flexibility, and only supports 40-foot and 20-foot container alignment.
A vision-based trailer positioning system is adopted, including an LED outdoor display, a high-definition camera, a terminal server and a decoder, and a high-definition three-dimensional map is built through a high-definition camera, a positioning device is used to import the trailer position, and a real-time display of the alignment line on the LED display through AR navigation.
It realizes more accurate trailer positioning in bad weather conditions, supports multiple container size alignment, improves the flexibility and accuracy of the system, and reduces the number of repeated operations by shore and bridge drivers.
Smart Images
Figure CN119863577A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vision positioning technology, and in particular to a vision-based trailer positioning system and method. Background Art
[0002] At present, there are few trailer intelligent positioning systems on the market that use visual methods. In the traditional trailer intelligent positioning system, four alignment lines are pasted on the quay crane, and the alignment lines are aligned through four camera systems and displayed on the LED screen. The traditional method displays the alignment lines by physically pasting reflective strips on the quay crane. Bad weather or damaged reflective strips can easily cause inaccurate alignment information, and only supports 40-foot and 20-foot container alignment, with low flexibility. Summary of the invention
[0003] The object of the present invention is to provide a vision-based trailer positioning system and method to solve the problems raised in the above background technology.
[0004] To achieve the above object, the present invention provides the following technical solutions: a trailer positioning system based on vision, comprising an LED outdoor display screen, a high-definition camera, a terminal server and a decoder, characterized in that: the high-definition camera is installed on the connecting beam of the quay crane, and can virtually display the alignment line; the LED outdoor display screen is installed on the lower cross beam on the land side so that the truck driver can observe it easily; the terminal server and the decoder are installed in the power distribution room of the quay crane; The high-definition camera includes two RGB cameras and a far-infrared camera, one of the RGB camera and the far-infrared camera is in the same direction and placed side by side, and the other RGB camera is used to capture images of the entire quayside bridge; The terminal server and decoder are electrically connected to the high-definition camera and the LED outdoor display screen respectively. After the terminal server processes the panoramic image of the quay crane taken by an RGB camera, it builds a high-definition three-dimensional map inside the server, uses the positioning device to import the position of the trailer, locates it in the high-precision three-dimensional map, and transmits the real-time traffic image to the LED outdoor display screen according to the change of the trailer's position to realize AR navigation.
[0005] Preferably, the construction of the high-definition three-dimensional map also includes using a single or a few images to reconstruct a three-dimensional stereo model of the target object; using these matching points to describe and obtain the coordinate information points of the three-dimensional target object in space, thereby achieving three-dimensional reconstruction.
[0006] Preferably, in order to ensure the integrity of the imaging, the captured image needs to be subjected to denoising processing when being transmitted to the terminal server, and after the denoising process, the image can be stored as simple data.
[0007] Preferably, when receiving image data, the terminal server resets the image size at the current position to a standard image size of m×n in order to eliminate the influence of the image size on the result.
[0008] Preferably, the vision-based trailer positioning method is characterized by comprising the following steps: S1: A panoramic image of the quay crane is captured by an RGB camera, which quickly acquires the image and point cloud information of the quay crane; S2: After the acquisition is completed, the data is transmitted to the processor via wireless signals. The terminal server dynamically adjusts the communication link parameters of the collected data and then reconstructs the 3D model to restore the 3D scene information and realize the alignment mapping of the point cloud and the image. S3: On the computer supporting the terminal server, data is lightweighted by compressing textures and merging root nodes. At the map loading and rendering level, artificial modeling technology is used, that is, artificial models are used to replace real-scene 3D models. Artificial modeling is performed using modeling software. By importing data of different sizes and texture mapping relationship tables, different textures or models are assigned to different elements to generate high-precision 3D maps. S4: The position of the trailer is located by a positioning device, and the computer performs AR recognition on the captured scene information to obtain a recognition result, obtains the location information of the scene, and locates it in a three-dimensional high-precision map; S5: According to the change of user location information, the terminal server is connected to the quay crane PLC to obtain information such as the operation lane, operation type, and container type. According to different situations, the front-end camera is controlled to switch to different display modes. The camera should display the alignment line in the screen through the virtual frame line according to the pre-set mode, and mark the alignment conditions of "double 20 feet", "single 20 feet", and "40 feet" respectively; S6: The terminal displays the alignment status of the current mode on the LED screen through the decoder. The truck driver can check whether the box column is aligned with the alignment line on the display screen, and then determine whether the position is properly parked.
[0009] Preferably, the modeling software in S3 is 3DSMax.
[0010] Compared with the prior art, the present invention has the following beneficial effects: This vision-based trailer positioning system and method displays the appropriate parking position of the current operating lane through the linkage of the quay crane PLC, camera and display screen, guides the trailer to park the vehicle disc within the alignment line, and enables the truck driver to find the parking position more accurately by observing the display screen, thereby reducing the number of repeated operations by the quay crane driver. At the same time, the real-time status of the grabbing and releasing box operation is displayed on the display screen, so that the trailer driver can clearly understand the real-time status of the spreader, adjust the trailer position by himself, and avoid the situation of dragging the box. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0012] Figure 1 It is a composition diagram of the trailer positioning system of the present invention. DETAILED DESCRIPTION
[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0014] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0015] See also Figure 1 , the present invention provides a technical solution: Embodiment 1:
[0016] The trailer positioning system based on vision includes an LED outdoor display, a high-definition camera, a terminal server and a decoder. The high-definition camera is installed on the connecting beam of the quay crane, which can virtually display the alignment line; the LED outdoor display is installed on the lower beam on the land side for easy observation by the truck driver; the terminal server and decoder are installed in the power distribution room of the quay crane; The high-definition camera includes two RGB cameras and a far-infrared camera. One RGB camera and the far-infrared camera are in the same direction and are placed side by side. The other RGB camera is used to capture images of the entire shore bridge. Steering structures are set at the bottom of the two RGB cameras and the far-infrared camera.
[0017] The terminal server and decoder are electrically connected to the high-definition camera and the LED outdoor display screen respectively. After the terminal server processes the panoramic image of the quay crane taken by an RGB camera, it builds a high-definition three-dimensional map inside the server, uses the positioning device to import the position of the trailer, locates it in the high-precision three-dimensional map, and transmits the real-time traffic image to the LED outdoor display screen according to the changes in the trailer's position to realize AR navigation.
[0018] The construction of high-definition three-dimensional maps also includes using a single or a few images to reconstruct a three-dimensional stereo model of the target object; using these matching points to describe and obtain the coordinate information points of the three-dimensional target object in space, thereby achieving three-dimensional reconstruction.
[0019] To ensure the integrity of the image, the captured image needs to be denoised when transmitted to the terminal server. After the denoising process, the image can be stored as simple data.
[0020] When receiving image data, the terminal server resets the current image size to the standard image size m×n in order to eliminate the influence of image size on the result. .
[0021] A PLC system is configured in the quay crane power distribution room, which is connected to the terminal server and electrically connected to the steering structure at the bottom of the two RGB cameras and the far-infrared camera.
[0022] The vision-based trailer positioning method includes the following steps: S1: A panoramic image of the quay crane is captured by an RGB camera, which quickly acquires the image and point cloud information of the quay crane; S2: After the acquisition is completed, the data is transmitted to the processor via wireless signals. The terminal server dynamically adjusts the communication link parameters of the collected data and then reconstructs the 3D model to restore the 3D scene information and realize the alignment mapping of the point cloud and the image. Due to the limitations of the equipment and the existence of scanning errors, some accurate point pairs may be lost during the calculation process. By calculating the distance from point to surface, such errors can be reduced to a certain extent. This search method establishes a reference plane for the nearest point to be searched, and uses the nearest point on the plane as the nearest point pair to match it.
[0023] The point cloud image to be registered is recorded as a set
[0024] Source point cloud Any point in In the target point cloud Search distance The three nearest points, i.e. , the coordinates are , j = 1, 2, 3 connect these three points into a plane triangle. The vertical foot When it falls inside the triangle, as the closest point, and Together they form a pair of corresponding points; when If it falls outside the triangle, the distance The nearest point is considered as the nearest point, and Together they form corresponding point pairs.
[0025] When obtaining point clouds at different angles of an object model, the coordinate positions of the point cloud data of the same area of the object model at different angles may not be completely consistent. In the process of finding the closest point pair, the closest point pair that matches it may also be the closest point of its adjacent position. In order to make the effect of the closest point iteration closer to reality, it is necessary to screen the initially formed closest point pairs and remove the wrong points. The closest point pairs with a deviation greater than the threshold are deleted, and the point pairs within the threshold range are retained.
[0026] The closest point to the formation , To calculate the distance, set is the maximum distance threshold for the closest point pair.
[0027]
[0028] After removing the wrong closest point pairs from the point cloud, the singular value decomposition method is used to effectively solve the transformation matrix and translation matrix for the retained valid closest point pairs. According to the formula Obtaining the source point cloud and target point cloud Center of gravity , after finding the centroid of the source point cloud and the target point cloud, and The point set in the point cloud is translated to form a new point cloud set. and ,
[0029] According to the source point cloud and target point cloud obtained by centroid translation, a third-order covariance matrix H is constructed.
[0030]
[0031] Further singular value decomposition is performed on the third-order covariance matrix obtained in the previous step, where D is a diagonal matrix, And the three elements in the diagonal matrix satisfy
[0032]
[0033] The rank of the covariance matrix H is judged. When the rank is not less than 2, the rotation matrix R and the translation matrix T can be obtained.
[0034]
[0035] S3: On the computer supporting the terminal server, data is lightweighted by compressing textures and merging root nodes. At the map loading and rendering level, artificial modeling technology is used, that is, artificial models are used to replace real-life 3D models. Artificial modeling is done using modeling software. By importing data of different sizes and texture mapping relationship tables, different elements are given different textures or models to generate high-precision 3D maps. The modeling software is 3DSMax.
[0036] S4: The position of the trailer is located by a positioning device, and the computer performs AR recognition on the captured scene information to obtain a recognition result, obtains the location information of the scene, and locates it in a three-dimensional high-precision map; S5: According to the change of user location information, the terminal server is connected to the quay crane PLC to obtain information such as the operation lane, operation type, and container type. The front-end camera is controlled to switch to different display modes according to different situations. The camera should display the alignment line in the screen through a virtual frame line according to the pre-set mode, and mark the alignment conditions of "double 20 feet", "single 20 feet", and "40 feet" respectively.
[0037] S6: The terminal displays the alignment status of the current mode on the LED screen through the decoder. The truck driver can check whether the box column is aligned with the alignment line on the display screen, and then determine whether the position is properly parked.
[0038] Embodiment 2: Different from the first embodiment, the method for producing the high-precision three-dimensional map is different in that after obtaining the image and point cloud information of the quay crane, a three-dimensional model is established: Introduce the solid equation:
[0039] Where: Represents the intensity coefficient of light source reflected by the surface of an object; is the object's normal vector; is the light source refraction vector; is the image pixel; It represents the two-dimensional parameter transformation value, which reflects the value of the collected data and can ensure the integrity of the data during the data analysis process; The stereo equation method uses multiple non-conjugate light sources to obtain multiple images of the same object. By combining the conjugate equations of different images, the kinetic energy normal vector of the object can be solved. , and the light source refraction vector , and finally the three-dimensional shape of the object can be restored. Therefore, the current methods basically use multiple light sources for three-dimensional reconstruction, which can be expressed as a matrix:
[0040] The GH matrix represents the collected data storage and calculation mode. Indicates the number of transformable simplified conditions of the light source. By transforming the pixels of the simplified conditions, the clarity of the image can be guaranteed during the data transformation process, and it is more suitable for the transformation of multi-object acquisition data under different conditions.
[0041] Identify the data:
[0042] Where: Represents the high-order attribute distortion vector after data transformation, which can reflect the effectiveness of the transformation process; Indicates the variable parameters that can be effectively retained during the transformation process; Indicates the conversion success rate.
[0043] The RGB camera and the far infrared camera are aligned in the same direction to obtain images, and the directions of the two cameras are as parallel as possible. This is conducive to reducing the amount of calculation during camera calibration, and obtaining RGB images and depth images respectively. In order to solve the problem of image center deviation caused by the different positions of the two cameras, the RGB image is first transformed by coordinates to align with the depth image coordinates, and then the x and y coordinate positions of the spatial points are calculated. Finally, the three-dimensional coordinate point cloud data (x, y, z, R, G, B) is displayed through a hierarchical model:
[0044] Where: is the difference prime number of the one-dimensional coordinate of the model; Verify the energy level variables for the two-dimensional coordinate data of the model.
[0045] After the display, the collected data can be verified in stereo, and the process is as follows:
[0046] Where: are the effective values of the three-dimensional coordinates of the points in the stereo calibration process; S is the optimal stereo calibration parameter value. The coordinates of the points that can obtain the image in the three-dimensional map can be intuitively obtained.
[0047] The reconstructed three-dimensional dynamic scene must be denoised to ensure the integrity of the imaging.
[0048]
[0049] Where: There is no data loss or omission in the normal denoising process; is the value of data mutation. After the denoising process, the data can be easily calculated and stored. The process is as follows:
[0050] Where: The effective completion of the data calculation and storage process; It is the data operation and storage rate parameter; is the superposition limit storage capacity of the storage process. After the above process, the denoising process is completed and the reconstruction is completed at the same time.
Claims
1. A vision-based trailer positioning system, including an LED outdoor display, a high-definition camera, a terminal server and a decoder, characterized by: The high-definition camera is installed on the quay crane connecting beam and can virtually display the alignment line; The LED outdoor display screen is installed on the lower beam on the land side, where it is easy for the truck driver to observe; The terminal server and decoder are installed in the quay crane power distribution room; The high-definition camera includes two RGB cameras and a far-infrared camera, one of the RGB camera and the far-infrared camera is in the same direction and is placed side by side, and the other RGB camera is used to capture images of the entire quay bridge, and the bottoms of the two RGB cameras and the far-infrared camera are both provided with a steering structure; The terminal server and decoder are electrically connected to the high-definition camera and the LED outdoor display screen respectively. After the terminal server processes the panoramic image of the quay crane taken by an RGB camera, it builds a high-definition three-dimensional map inside the server, uses the positioning device to import the position of the trailer, locates it in the high-precision three-dimensional map, and transmits the real-time traffic image to the LED outdoor display screen according to the change of the trailer's position to realize AR navigation.
2. The vision-based trailer positioning system according to claim 1, characterized in that: The construction of the high-definition three-dimensional map also includes using a single or a few images to reconstruct a three-dimensional stereo model of the target object; using these matching points to describe and obtain the coordinate information points of the three-dimensional target object in space, thereby achieving three-dimensional reconstruction.
3. The vision-based trailer positioning system according to claim 1, characterized in that: In order to ensure the integrity of the imaging, the captured image needs to be denoised when transmitted to the terminal server. After the denoising process, the image can be stored as simple data.
4. The vision-based trailer positioning system according to claim 1, characterized in that: When receiving image data, the terminal server resets the image size at the current position to the standard image size m×n in order to eliminate the influence of the image size on the result.
5. The vision-based trailer positioning system of claim 1, wherein: A PLC system is configured in the quay crane power distribution room, the PLC system is connected to the terminal server, and the PLC system is electrically connected to the two RGB cameras and the steering structure at the bottom of the far-infrared camera.
6. The method for trailer positioning based on vision according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1: A panoramic image of the quay crane is captured by an RGB camera, which quickly acquires the image and point cloud information of the quay crane; S2: After the acquisition is completed, the data is transmitted to the processor via wireless signals. The terminal server dynamically adjusts the communication link parameters of the collected data and then reconstructs the 3D model to restore the 3D scene information and realize the alignment mapping of the point cloud and the image. S3: On the computer supporting the terminal server, data is lightweighted by compressing textures and merging root nodes. At the map loading and rendering level, artificial modeling technology is used, that is, artificial models are used to replace real-scene 3D models. Artificial modeling is performed using modeling software. By importing data of different sizes and texture mapping relationship tables, different textures or models are assigned to different elements to generate high-precision 3D maps. S4: The position of the trailer is located by a positioning device, and the computer performs AR recognition on the captured scene information to obtain a recognition result, obtains the location information of the scene, and locates it in a three-dimensional high-precision map; S5: According to the change of user location information, the terminal server is connected to the quay crane PLC to obtain information such as the operation lane, operation type, and container type. According to different situations, the front-end camera is controlled to switch to different display modes. The camera should display the alignment line in the screen through the virtual frame line according to the pre-set mode, and mark the alignment conditions of "double 20 feet", "single 20 feet", and "40 feet" respectively. S6: The terminal displays the alignment status of the current mode on the LED screen through the decoder. The truck driver can check whether the box column is aligned with the alignment line on the display screen, and then determine whether the position is properly parked.
7. The method for trailer positioning based on vision according to claim 6, characterized in that: The modeling software in S3 is 3DSMax.