System and method for identifying container number of railway track-mounted gantry crane based on binocular vision

Abstract

The invention relates to the technical field of container loading and unloading operation automation, in particular to a system and a method for identifying container numbers of railway track-mounted cranes based on binocular vision, comprising a binocular vision acquisition module, a data processing and control module, a distance measuring and triggering unit, a container number identification unit and a system interaction interface; the binocular vision acquisition module is electrically connected with the data processing and control module, the binocular vision acquisition module is used for synchronously acquiring a left image and a right image of a small surface of a container, and the data processing and control module is used for processing and controlling data acquired by the binocular vision acquisition module; the data processing and control module comprises a distance measuring and triggering unit and a container number recognition unit, and the distance measuring and triggering unit calculates the accurate distance between the lifting appliance and the small surface of the target container through a stereo matching algorithm according to a depth image given by the data processing and control module; and the container number identification unit carries out container number identification on the acquired image under a triggering condition.

Description

Technical Field

[0001] This invention relates to the field of automated container loading and unloading technology, specifically to a system and method for identifying container numbers using a railway track crane based on binocular vision. Background Technology

[0002] Currently, in railway freight yards and container terminals, rail-mounted container gantry cranes (rail-mounted gantry cranes) are the core equipment for container loading, unloading, and stacking operations. For example... Figure 1 and Figure 2 As shown, existing automatic container number recognition systems typically mount a monocular camera fixedly on the four legs or crossbeam of a rail-mounted crane, rather than moving with the spreader. This fixed installation method limits its field of view, preventing it from effectively covering the container number area before the spreader locks, resulting in a delay in recognition. Specific drawbacks are as follows: (1) Delay in recognition timing: Due to the limitations of the camera's installation angle and field of view, the system often needs to capture an image of the container number on the side (large side) of the container for recognition only after the spreader has grabbed the container and lifted it to a certain height. At this time, the container has already been lifted and its physical state has changed.

[0003] (2) Waste of energy and time: After the container is lifted, if the container number is found to be inconsistent with the container number in the operation plan issued by the operation management system, the operator must put the container back in its original position and then find the correct container. This inefficient lifting process significantly increases the operation time (usually taking an extra 5-10 minutes) and energy consumption, and reduces the efficiency of operation.

[0004] (3) Significant interference in identification: Containers are densely stacked in the yard, and the monocular camera cannot effectively perceive depth information. It is easy to capture the container number of non-target containers due to the long shooting distance and large field of view, resulting in misidentification.

[0005] (4) Poor image quality: Container numbers are usually printed on the surface of the container, which has wrinkles and unevenness. When shooting at close range, the monocular camera has difficulty overcoming the perspective distortion and light obstruction caused by wrinkles due to the lack of stereo information, resulting in a decrease in the recognition rate of the OCR (Optical Character Recognition) algorithm.

[0006] Therefore, there is an urgent need for a technical solution that can accurately and quickly identify container numbers before the spreader grabs the container, in order to fundamentally avoid ineffective lifting and achieve energy saving and efficiency improvement. Summary of the Invention

[0007] The present invention aims to provide a system for identifying container numbers on railway track cranes based on binocular vision. This system can accurately identify container numbers before the spreader grabs the container, avoiding repeated lifting operations caused by mismatched container numbers, thereby saving energy, improving work efficiency, and solving problems such as poor image quality and large recognition interference in the prior art.

[0008] The technical solution for achieving the objective of this invention is as follows: A system for identifying container numbers on railway rail gantry cranes based on binocular vision, characterized in that it includes a binocular vision acquisition module, a data processing and control module, a ranging and triggering unit, a container number identification unit, and a system interaction interface; The binocular vision acquisition module and the data processing and control module are electrically connected. The binocular vision acquisition module is used to simultaneously acquire two images, the left and right, of the container front. The data processing and control module processes and controls the data acquired by the binocular vision acquisition module. The data processing and control module includes a ranging and triggering unit and a container number recognition unit. The ranging and triggering unit calculates the precise distance between the spreader and the small face of the target container based on the depth image provided by the data processing and control module using a stereo matching algorithm. The container number recognition unit identifies the container number in the acquired image under trigger conditions. The system's interactive interface communicates with the main control system of the rail-mounted gantry crane and the upper-level operation management information system, receives the operation plan (target box number), and reports the identification results.

[0009] Preferably, the binocular vision acquisition module includes two parallel cameras mounted on the spreader of the rail-mounted gantry crane, preferably mounted on the small side of the spreader so that its field of view can cover the small area of ​​the container directly below the spreader.

[0010] Preferably, the camera is an industrial-grade CMOS camera with a resolution of 5 megapixels or higher, such as... Figure 3 As shown, it is fixedly installed on both sides of the lifting frame with a baseline distance of 150-200mm, ensuring a distance measurement accuracy of ±5mm within a working distance of 0.5-1.5 meters.

[0011] The data processing and control module is installed in the hoist or crane machine room and includes a computing unit (such as an industrial computer or embedded GPU processor) and a storage unit.

[0012] A method for identifying container numbers using a railway track crane based on binocular vision, characterized in that, as Figure 4 As shown, it includes the following steps: Step 1, Initialization Phase: The system starts up, loads camera calibration parameters, and starts the binocular vision acquisition module to begin acquiring images; Step Two: Image Acquisition and Processing Stage: The data processing and control module performs preprocessing such as denoising, enhancement, and distortion correction on the continuously and synchronously acquired images from the left and right cameras. Then, using the principle of binocular vision, the container depth is calculated through the disparity map of the left and right images. The specific calculation method is as follows: The optical axes of the two cameras (left and right) are parallel, their focal lengths are the same, the baseline (the distance between the optical centers of the two cameras) is B, and the focal length is f.

[0013] For a point P in space, the imaging positions in the left figure are x_L, y_L, and the imaging positions in the right figure are x_R, y_R.

[0014] Disparity is defined as: d = x_L - x_R. Based on the principle of similar triangles, the distance (depth) Z from point P to the camera plane can be derived as: Z = (B × f) / d. Additionally: z / f = x / x_L = (xB) / f = y / y_L = y / y_R, therefore: x=x_L*z / f or B+x_R*z / f, y=y_L*z / f or y_R*z / f, Where: z: distance (depth) from the target point to the camera plane; x: x-coordinate of the target point; y: y-coordinate of the target point, with the axis perpendicular to the page; B: baseline distance (distance between the optical centers of the two cameras); f: camera focal length; d: parallax (pixel difference). Step 3, Distance Detection Stage: The ranging and triggering unit calculates the distance D between the spreader and the container in real time based on the depth image. The system presets a recognition trigger distance threshold D_th (e.g., 0.5 meters - 1.5 meters). When the real-time distance D ≤ D_th, the container number recognition process is triggered. The core function of this step is to ensure that the camera is at a sufficiently close distance to obtain a high-definition image. At the same time, because this distance is very close, the field of view is very small, which can effectively eliminate interference from other surrounding containers and accurately focus on the target container.

[0015] Step 4: Image Correction and Fusion Based on Stereo Vision: After the ranging and triggering unit issues the start recognition command, the image processing and control module begins geometric correction based on depth information. Specifically, based on the depth information, geometric correction is performed on image deformations caused by wrinkles and unevenness on the box surface to reconstruct a box number image area that is as "flat" as possible. This step is crucial for overcoming the influence of wrinkles and improving the recognition rate.

[0016] The specific binocular image processing flow is as follows: 1) Obtain point cloud image (point_cloud) and color image (color_img) 2) The YOLO object detection method is used to locate the container number area in the color image.

[0017] 3) The RGB values ​​of the box number area image are fused with the point cloud image to form a point cloud image with color information.

[0018] 4) Fit the local optimal plane of the box number region using RANSAC. 5) Project the 3D points with color information orthogonally onto the plane and perform 2D resampling to obtain a flattened two-dimensional image.

[0019] 6) Process the two-dimensional image (fill in missing parts, image enhancement, etc.), and use the processed image for box number recognition.

[0020] Step 5, Container Number Recognition Stage: The clear image after correction in Step 4 is used to detect the container number using an AI algorithm. The detected container number area is filtered by the configured ROI area to obtain the real container number string area. Then, the container number is recognized by the AI ​​algorithm to finally obtain the container number string.

[0021] Step Six: Result Comparison and Decision Feedback Stage: The identification results are compared with the work plan. If they match, a "Correct Container Number" signal is sent to the main control system, which then notifies the administrator to continue operation. If they do not match, an "Incorrect Container Number" alarm signal is immediately sent to the main control system. The main control system immediately notifies the operator to intervene, thus preventing the incorrect container from being lifted.

[0022] The beneficial effects of this invention are: This invention can accurately identify the container number before the spreader grabs the container, avoiding repeated lifting operations caused by mismatched container numbers, thereby saving energy, improving work efficiency, and solving problems such as poor image quality and large recognition interference in the prior art. Attached Figure Description

[0023] Figure 1 In existing technologies, the container number image cannot be obtained when the spreader is locked due to obstructed view. Figure 2 This is an example of aerial identification of the container number after hoisting, as described in existing technologies. Figure 3 This is a system component block diagram of the present invention; Figure 4 This is a schematic diagram of the hardware installation of the present invention; Figure 5 This is a schematic diagram illustrating the principle of the troublesome depth generation of this invention; Figure 6 This is a flowchart of the workflow of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1

[0026] A system for identifying container numbers using a railway track crane based on binocular vision, such as Figure 3 As shown, it includes a binocular vision acquisition module, a data processing and control module, a ranging and triggering unit, a box number recognition unit, and a system interaction interface; The binocular vision acquisition module and the data processing and control module are electrically connected. The binocular vision acquisition module is used to simultaneously acquire two images, the left and right, of the container front. The data processing and control module processes and controls the data acquired by the binocular vision acquisition module. The data processing and control module includes a ranging and triggering unit and a container number recognition unit. The ranging and triggering unit calculates the precise distance between the spreader and the small face of the target container based on the depth image provided by the data processing and control module using a stereo matching algorithm. The container number recognition unit identifies the container number in the acquired image under trigger conditions. The system's interactive interface communicates with the main control system of the rail-mounted gantry crane and the upper-level operation management information system, receives the operation plan (target box number), and reports the identification results.

[0027] Preferably, the binocular vision acquisition module includes two parallel cameras mounted on the spreader of the rail-mounted gantry crane, preferably mounted on the small side of the spreader so that its field of view can cover the small area of ​​the container directly below the spreader.

[0028] Preferably, the camera is an industrial-grade CMOS camera with a resolution of 5 megapixels or higher, such as... Figure 4 As shown, it is fixedly installed on both sides of the lifting frame with a baseline distance of 150-200mm, ensuring a distance measurement accuracy of ±5mm within a working distance of 0.5-1.5 meters.

[0029] The data processing and control module is installed in the hoist or crane machine room and includes a computing unit (such as an industrial computer or embedded GPU processor) and a storage unit.

[0030] Example 2

[0031] A method for identifying container numbers using a railway track crane based on binocular vision, characterized in that, as Figure 6 As shown, it includes the following steps: Step 1, Initialization Phase: The system starts up, loads camera calibration parameters, and starts the binocular vision acquisition module to begin acquiring images; Step Two: Image Acquisition and Processing Stage: The data processing and control module performs preprocessing such as denoising, enhancement, and distortion correction on the continuously and synchronously acquired images from the left and right cameras. Then, using the principle of binocular vision, it calculates the container depth through the disparity map of the left and right images; for example... Figure 5 As shown, the specific calculation method is as follows: The optical axes of the two cameras (left and right) are parallel, their focal lengths are the same, the baseline (the distance between the optical centers of the two cameras) is B, and the focal length is f.

[0032] For a point P in space, the imaging positions in the left figure are x_L, y_L, and the imaging positions in the right figure are x_R, y_R.

[0033] Disparity is defined as: d = x_L - x_R. Based on the principle of similar triangles, the distance (depth) Z from point P to the camera plane can be derived as: Z = (B × f) / d. Additionally: z / f = x / x_L = (xB) / f = y / y_L = y / y_R, therefore: x=x_L*z / f or B+x_R*z / f, y=y_L*z / f or y_R*z / f, Where: z: distance (depth) from the target point to the camera plane; x: x-coordinate of the target point; y: y-coordinate of the target point, with the axis perpendicular to the page; B: baseline distance (distance between the optical centers of the two cameras); f: camera focal length; d: parallax (pixel difference). Step 3, Distance Detection Stage: The ranging and triggering unit calculates the distance D between the spreader and the container in real time based on the depth image. The system presets a recognition trigger distance threshold D_th (e.g., 0.5 meters - 1.5 meters). When the real-time distance D ≤ D_th, the container number recognition process is triggered. The core function of this step is to ensure that the camera is at a sufficiently close distance to obtain a high-definition image. At the same time, because this distance is very close, the field of view is very small, which can effectively eliminate interference from other surrounding containers and accurately focus on the target container.

[0034] Step 4: Image Correction and Fusion Based on Stereo Vision: After the ranging and triggering unit issues the start recognition command, the image processing and control module begins geometric correction based on depth information. Specifically, based on the depth information, geometric correction is performed on image deformations caused by wrinkles and unevenness on the box surface to reconstruct a box number image area that is as "flat" as possible. This step is crucial for overcoming the influence of wrinkles and improving the recognition rate.

[0035] The specific binocular image processing flow is as follows: 1) Obtain point cloud image (point_cloud) and color image (color_img) 2) The YOLO object detection method is used to locate the container number area in the color image.

[0036] 3) The RGB values ​​of the box number area image are fused with the point cloud image to form a point cloud image with color information.

[0037] 4) Fit the local optimal plane of the box number region using RANSAC. 5) Project the 3D points with color information orthogonally onto the plane and perform 2D resampling to obtain a flattened two-dimensional image.

[0038] 6) Process the two-dimensional image (fill in missing parts, image enhancement, etc.), and use the processed image for box number recognition.

[0039] Step 5, Container Number Recognition Stage: The clear image after correction in Step 4 is used to detect the container number using an AI algorithm. The detected container number area is filtered by the configured ROI area to obtain the real container number string area. Then, the container number is recognized by the AI ​​algorithm to finally obtain the container number string.

[0040] Step Six: Result Comparison and Decision Feedback Stage: The identification results are compared with the work plan. If they match, a "Correct Container Number" signal is sent to the main control system, which then notifies the administrator to continue operation. If they do not match, an "Incorrect Container Number" alarm signal is immediately sent to the main control system. The main control system immediately notifies the operator to intervene, thus preventing the incorrect container from being lifted.

[0041] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A system for identifying container numbers using a railway track crane based on binocular vision, characterized in that, It includes a binocular vision acquisition module, a data processing and control module, and a system interaction interface; The binocular vision acquisition module and the data processing and control module are electrically connected. The binocular vision acquisition module is used to simultaneously acquire two images, the left and right, of the container front. The data processing and control module processes and controls the data acquired by the binocular vision acquisition module. The data processing and control module includes a ranging and triggering unit and a container number recognition unit. The ranging and triggering unit calculates the precise distance between the spreader and the small face of the target container based on the depth image provided by the data processing and control module using a stereo matching algorithm. The container number recognition unit identifies the container number in the acquired image under trigger conditions. The system's interactive interface communicates with the main control system of the rail-mounted gantry crane and the upper-level operation management information system, receives operation plans, and reports identification results.

2. The system for identifying container numbers on railway track cranes based on binocular vision according to claim 1, characterized in that, The binocular vision acquisition module includes two parallel cameras mounted on the lifting device of the rail-mounted gantry crane.

3. The system for identifying container numbers on railway rail cranes based on binocular vision according to claim 2, characterized in that, The camera is mounted on the small side of the spreader so that its field of view can cover the small area of ​​the container directly below the spreader.

4. The system for identifying container numbers on railway rail cranes based on binocular vision according to claim 3, characterized in that, The camera is an industrial-grade CMOS camera with a resolution of 5 megapixels or higher, which is fixedly installed on both sides of the hoist frame with a baseline distance of 150-200mm, ensuring a ranging accuracy of ±5mm within a working distance of 0.5-1.5 meters.

5. The system for identifying container numbers on railway track cranes based on binocular vision according to claim 4, characterized in that, The data processing and control module is installed in the machine room of the hoist or crane and includes a computing unit and a storage unit. The computing unit includes an industrial control computer and an embedded GPU processor.

6. A method for identifying container numbers using a railway track crane based on binocular vision, characterized in that, Includes the following steps, Step 1: The system starts up, loads the camera calibration parameters, and starts the binocular vision acquisition module to begin acquiring images; Step 2: The data processing and control module performs preprocessing such as denoising, enhancement, and distortion correction on the continuously and synchronously acquired images from the left and right cameras. Then, using the principle of binocular vision, the container depth is calculated through the disparity map of the left and right images. Step 3: The ranging and triggering unit calculates the distance D between the spreader and the container in real time based on the depth image. The system presets a recognition trigger distance threshold D_th. When the real-time distance D ≤ D_th, the container number recognition process is triggered. Step 4: After the ranging and triggering unit gives the start recognition command, the image processing and control module begins to perform geometric correction based on depth information. The specific method is as follows: based on the depth information, geometric correction is performed on the image deformation caused by the wrinkles and unevenness of the box surface to reconstruct a box number image area that is as "flat" as possible. Step 5: For the clear image after correction in Step 4, use AI algorithm to detect the container number. Filter the detected container number area using the configured ROI area to obtain the actual container number string area. Then, use AI algorithm to recognize the container number and finally obtain the container number string. Step Six: Compare the identification results with the work plan. If they match, send a "Correct Container Number" signal to the main control system, which will then notify the administrator to continue the operation. If they do not match, immediately send an "Incorrect Container Number" alarm signal to the main control system, which will then immediately notify the operator to intervene, thus preventing the wrong container from being lifted.

7. The method for identifying container numbers using a railway track crane based on binocular vision according to claim 1, characterized in that, In step two, the binocular image processing flow is as follows: Step 1: Obtain the point cloud image (point_cloud) and the color image (color_img); Step 2: Use the object detection method (YOLO) to locate the container number region in the color image; Step 3: Fuse the RGB values ​​of the box number area image with the point cloud image to form a point cloud image with color information; Step 4: Fit the local optimal plane of the box number region using RANSAC; Step 5: Orthogonally project the 3D points with color information onto the plane and perform 2D resampling to obtain a flattened 2D image; Step 6: Process the two-dimensional image. The processed image is used for box number recognition.

Images