A carriage displacement detection method based on structured light vision
By setting vertical steel bars on the surface of the carriage and combining structured light vision and binocular vision technology, the problem of poor environmental adaptability of existing carriage dynamic detection equipment has been solved, and stable and convenient detection of carriage displacement has been achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGMEI KEGONG INTELLIGENT STORAGE TECH CO LTD
- Filing Date
- 2023-08-17
- Publication Date
- 2026-08-04
AI Technical Summary
Existing dynamic detection equipment for carriages has poor environmental adaptability. The grating detection equipment is easily affected by sprayed liquids, natural light angles and low temperatures, while the lidar group is easily affected by smog and water vapor. The equipment is large in size, has complicated wiring, is difficult to maintain, and troubleshooting and debugging are complicated.
By employing structured light vision technology, vertical steel bars with continuous intervals are set on the surface of the carriage. Structured light images of the carriage surface are acquired using laser projection equipment and cameras. Combined with binocular vision technology, the movement of the vertical steel bars is analyzed to detect the displacement of the carriage.
It improves the environmental adaptability of the equipment, reduces the number of devices and maintenance complexity, avoids detection failures caused by environmental factors, and achieves stable tracking of carriage displacement.
Smart Images
Figure CN117091508B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for detecting carriage displacement based on structured light vision. Background Technology
[0002] During the exploratory phase of intelligent, unmanned, rapid, and quantitative loading technology, various manufacturers experimented with numerous techniques for dynamic detection of the loading compartments, including beacon positioning at key locations and integral positioning through speed detection. After market testing and consolidation, the most widely used and prevalent technologies now are dynamic detection methods based on grating groups and LiDAR arrays. These two methods primarily detect the gaps between loading compartments, and by tracking these gaps, the position of the loading compartments is determined.
[0003] However, during routine operation of the intelligent, unmanned, rapid, and quantitative loading system, it was found that the current dynamic detection equipment for the freight car has poor environmental adaptability. The existing dynamic equipment spans a large area, especially the grating detection equipment, which is over 15 meters in total and distributed on both sides of the freight car. This makes it susceptible to environmental factors such as sprayed liquids, natural light angles, and low temperatures. The large size of the grating groups also increases the likelihood of being affected, making troubleshooting difficult, especially during loading. Therefore, the grating groups are large in size, numerous in components, and have complex wiring, making troubleshooting difficult. Daily maintenance requirements are high, requiring dedicated personnel to wipe the gratings, and the wiping areas are quite large. Replacing and adjusting the grating sensors after a problem occurs is also cumbersome and requires specialized personnel.
[0004] The dynamic detection method of lidar arrays involves using lidar sensors on the side of the truck bed to detect the distance to the truck bed, and then converting the data to obtain the relative displacement between the vehicle and the loading / unloading chute. Two-dimensional lidar emits a ring of laser beams around a central point on a plane. Each laser beam corresponds to an angle, and the distance from that point to the object is measured using the laser echo. The lidar is then rotated to follow the captured feature point, and the movement of the truck bed is calculated based on the rotation angle and the changed distance. In real-world scenarios, due to factors such as lighting, weather changes, and shadows cast by moving objects, the background needs to be updated promptly, and the influence of shadows needs to be eliminated. Therefore, lidar arrays are susceptible to smog, and especially in winter, they may stop working due to moisture from coal washing. Summary of the Invention
[0005] To address the aforementioned problems, the present invention aims to provide a carriage displacement detection method based on structured light vision. Unlike the two methods mentioned above, the present invention mimics the binocular characteristics of the human eye, focusing on observing the local features of the carriage. The observation direction does not change with the movement of the carriage. It alternately acquires target features in the image and obtains the carriage's movement state by analyzing and judging the movement of the alternating target features, thereby achieving carriage tracking.
[0006] To achieve the above objectives, the solution of the present invention is as follows:
[0007] A method for detecting train carriage displacement based on structured light vision includes a train carriage with continuously spaced vertical steel bars on its surface. These steel bars are used to reinforce the carriage side panels. The method includes a carriage displacement detection device installed beside the passing carriage. This device includes a laser projection device with cameras positioned on either side. The laser projection device projects a laser beam across the passing carriage surface with a specific width and height. The cameras, acting as binocular vision, synchronously acquire structured light images of the carriage surface across the laser beam width as the carriage moves, recording the time difference between frames. The method then identifies the vertical steel bars to be tracked within the structured light images. The magnitude, direction, and speed of the carriage displacement are determined based on the laser beam width and the time difference between the entry and exit of the vertical steel bars into and out of the structured light image area.
[0008] A further step in the plan is to install at least two sets of carriage displacement detection devices within the length of the carriage.
[0009] A further step in the solution is to place a sensing sensor next to the laser projection equipment, which is used to detect when the moving carriage has entered the laser projection area of the laser projection equipment.
[0010] A further aspect of the solution is that the structured light image is a visual difference image of the vertical steel bar illumination obtained by processing images synchronously acquired by cameras on both sides.
[0011] A further aspect of the solution is that the laser irradiation width is at least the width of two vertical steel bars, and the magnitude, direction, and speed of the carriage displacement distance are continuously obtained by measuring the time difference between the continuous entry and exit of the vertical steel bars into the structured light image area.
[0012] The solution further includes the following steps for acquiring the vertical steel bar to be tracked in the structured light image:
[0013] Step 1: Perform laser illumination width and height coordinate region correction on the images simultaneously acquired by the left and right cameras to determine the image processing area;
[0014] Step 2: Preprocess the corrected left and right images, and convert them into left and right grayscale images respectively. The conversion is performed using a weighted average method based on the importance of each channel of the RGB image, according to Formula 1.
[0015] f(x,y)=K1R(x,y)+K2G(x,y)+K3B(x,y) Formula 1
[0016] in:
[0017] K1, K2, and K3 are weight ratio values;
[0018] Step 3: Perform horizontal scans on the left and right grayscale images within the processing area, and calculate the window energy at each scan point using Formula 2:
[0019]
[0020] in:
[0021] (x,y) represents the coordinates of the scan point, which is also the center coordinate of the calculation window;
[0022] n represents the distance from the center of the selected window of the left grayscale image to the edge;
[0023] I(x+i, y+j) represents the image grayscale value at image coordinates (x+i, y+j);
[0024] The maximum value of E(x,y) in each scan line is the imaging point of the line laser. Since the number of line lasers is 1, the ordinate is fixed. The maximum value of E(x,y) in each scan line is the imaging point of the line laser. The x-coordinates are sorted from left to right and denoted as (x,y)_k, k=1,2,…. When in a plane, the y-coordinate value remains consistent. When a vertical rebar appears, its y-coordinate is significantly greater than the coordinates of other points. Then its x-coordinate is the vertical rebar point that needs to be marked and tracked.
[0025] The solution further states that the weight ratio values K1, K2, and K3 are parameters that are optimally obtained through on-site environmental debugging.
[0026] Compared with existing technologies, the advantages of this invention are as follows: The structured light binocular vision measurement technology used in this invention combines structured light and binocular measurement techniques. A laser projection device is used to project a laser beam onto the surface of the moving vehicle body. A camera captures the surface of the vehicle body to be measured, generating a light stripe distortion image containing spatial object surface information. This image is then processed to obtain information about the corresponding spatial points of the light stripes. By tracking the light stripes in the image, the movement of the vehicle body can be tracked. Because this method tracks the movement of the vehicle body by continuously acquiring and analyzing local images of the vehicle body surface, it avoids the problems of large grating groups, numerous devices, complicated wiring, and difficulty in troubleshooting issues associated with dynamic detection methods using grating groups, as well as the susceptibility of lidar groups to smog, especially in winter when they stop working due to moisture from coal washing. This invention uses fewer devices and is convenient to install, operate, and maintain.
[0027] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description
[0028] Figure 1 This is a schematic diagram of the structured light binocular measurement model of the present invention;
[0029] Figure 2 This is a schematic diagram of the binocular linear structured light imaging results;
[0030] Figure 3 This is a schematic diagram of the structured light tracking process. Detailed Implementation
[0031] A method for detecting carriage displacement based on structured light vision, such as... Figure 1 and Figure 2 As shown, the carriage displacement detection method includes a train carriage 1. The surface of the carriage 1 has continuously spaced vertical steel bars 101 for reinforcing the side panels of the carriage. The method also includes a carriage displacement detection device installed next to the passing carriage. The carriage displacement detection device includes a laser projection device 2 installed next to the passing carriage 1. A left camera 3 and a right camera 4 are respectively installed on the left and right sides of the laser projection device. The laser projection device 2 and the left and right cameras 3 and 4 on the left and right sides of the laser projection device 2 are called a set of carriage displacement detection devices. The laser projection device 2 projects a laser beam onto the surface of the passing carriage with a certain laser irradiation width and height. The cameras on both sides act as binocular vision and move with the carriage, synchronously acquiring the structured light image of the carriage surface with a laser irradiation width of 5 in a continuous frame manner, and recording the time difference between frames. The vertical steel bars to be tracked in the structured light image are obtained. The magnitude, direction, and speed of the carriage displacement distance are obtained based on the laser irradiation width and the time difference between the vertical steel bars entering and leaving the structured light image area.
[0032] Specifically: at least two sets of carriage displacement detection devices are installed within the length of the carriage; and a sensing sensor is installed next to the laser projection equipment, side by side, to detect when the moving carriage has entered the laser projection area of the laser projection equipment; the interval between the two sets of carriage displacement detection devices is determined according to the actual situation, and a preferred option is that the interval is set according to half the length of the carriage.
[0033] Wherein: the structured light image is a visual difference image of the vertical steel bar illumination obtained by processing the images synchronously acquired by cameras on both sides.
[0034] In order to continuously obtain the magnitude, direction and speed of the displacement distance of the moving carriage: the laser irradiation width 5 is at least the width of two vertical steel bars, and the magnitude, direction and speed of the carriage displacement distance are continuously obtained by the time difference of the vertical steel bars entering and leaving the structured light image area.
[0035] Wherein: the step of acquiring the vertical steel bar to be tracked in the structured light image includes:
[0036] Step 1: Perform laser illumination width and height coordinate region correction on the images simultaneously acquired by the left and right cameras to determine the image processing area;
[0037] Step 2: Preprocess the corrected left and right images, converting them into left and right grayscale images respectively. Then, use a weighted average method based on the importance of each channel of the RGB image, applying Formula 1 for the conversion.
[0038] f(x,y)=K1R(x,y)+K2G(x,y)+K3B(x,y) Formula 1
[0039] in:
[0040] K1, K2, and K3 are weight ratio values;
[0041] Step 3: Perform horizontal scans on the left and right grayscale images within the processing area, and calculate the window energy at each scan point using Formula 2:
[0042]
[0043] in:
[0044] (x,y) represents the coordinates of the scan point, which is also the center coordinate of the calculation window;
[0045] n represents the distance from the center of the selected window of the left grayscale image to the edge;
[0046] I(x+i, y+j) represents the image grayscale value at image coordinates (x+i, y+j);
[0047] The maximum value of E(x,y) in each scan line is the imaging point of the line laser. Since the number of line lasers is 1, the ordinate is fixed. The maximum value of E(x,y) in each scan line is the imaging point of the line laser. The x-coordinates are sorted from left to right and denoted as (x,y)_k, k=1,2,…. When in a plane, the y-coordinate value remains consistent. When a vertical rebar appears, its y-coordinate is significantly greater than the coordinates of other points. Then its x-coordinate is the vertical rebar point that needs to be marked and tracked.
[0048] The weight ratio values K1, K2, and K3 are parameters obtained by selecting the optimal values through on-site environmental debugging.
[0049] This embodiment mimics the characteristics of human binocular vision, focusing on observing the features of the carriage. By judging the movement of these features, it achieves carriage tracking. Simple binocular stereo vision is generally used for visible light scene reconstruction, but it has high environmental requirements and is prone to failure in scenes where the colors and textures of scattered carriages are relatively monotonous. Binocular structured light technology introduces structured light into a binocular vision system to highlight the surface of the carriage, achieving the acquisition of its three-dimensional shape. The structured light binocular vision measurement technology used in this embodiment combines structured light and binocular measurement techniques. Two cameras acquire light stripe distortion images containing spatial object surface information as structured light strikes the surface of the carriage to be measured. These images are then processed to obtain information on the corresponding spatial points of the light stripes. Finally, by combining the intrinsic system structural parameters of the two cameras in the measurement system, the three-dimensional spatial information of the surface of the object projected by the structured light is recovered.
[0050] Structured light binocular vision measurement model, such as Figure 1 and Figure 2 As shown, the structured light stripe images captured by the left and right cameras are processed by a combination of image difference and threshold segmentation to segment the light stripe images from the background image. Then, the processed light stripe images are converted to grayscale. The processed image is scanned along the projection to obtain the light stripe pixels of each row. The peak position of the curve of each row is obtained and recorded. This position is the position of the center of each row of structured light stripes, and the set of structural center points of the entire light stripe can be obtained.
[0051] Conventionally, stereo correction is performed on the imaging results. For the left and right images corrected as described above, matching point pairs need to be searched in the same row. This mainly includes the following:
[0052] 1) Determine the Region of Interest (ROI) for image processing based on the location of the detection equipment and the carriage;
[0053] 2) The corrected left and right images are preprocessed and converted into left and right grayscale images respectively. In this embodiment, the weighted average method is used to convert according to the importance of each channel of the RGB image. The conversion formula is as shown in Formula 1 above.
[0054] 3) Perform horizontal scanning on the portions of the left and right grayscale images located within the ROI, and calculate the window energy at each scanning point, as shown in Formula 2 above;
[0055] For the car body, the light strip primarily measures the distortion pattern of the vertical steel bars (101) that act as reinforcing ribs. The position of the car body is measured by tracking and detecting these reinforcing ribs. The tracking principle is mainly based on a high-frequency scanning and recognition process: during loading, the vehicle's speed is extremely slow, maintained at 0-1.5 km / h; during high-frequency scanning, the distortion closest to the previous time point is identified as the current distortion portion. The tracking process is achieved by monitoring the distortion displacement change within the time difference between two scans. Figure 3 As shown: time t1 is the scan result of the previous time, time t2 is the scan result of the next time, and 2# is the tracking reinforcement rib. After confirmation at time t1, tracking is performed at time t2. Since △X1 is less than △X2, 2#' is the target to be tracked at the next time.
[0056] To ensure continuous tracking, there must always be a distorted portion within the field of view that can be tracked. Therefore, at least two distorted portions must be present to achieve continuous object tracking. The installation position and angle of the optical camera and structured light source need to be adjusted and debugged to consider the detection accuracy.
[0057] This embodiment achieves dynamic detection through high-frequency data acquisition. Due to the obstruction of the loading station's steel structure, the requirements of train clearance, and the need for detection accuracy, the initial plan is to maintain a distance of approximately 2 meters between the device and the train car. However, a single device, being too close to the car, cannot measure a sufficient length, and its detection will stop when a gap occurs. Therefore, a sensor group is required, specifically a car entry sensor placed alongside the laser projection equipment to detect when a car has entered the measurement area. Based on the required car length, the estimated number of sensors is generally 2-3. The exact number needs to be determined through model building and testing to avoid the device simultaneously detecting gaps. When multiple sets of laser projection and left / right side camera detection devices are used simultaneously, the coordination of data from multiple devices needs to be considered to achieve mutual redundancy without mutual interference.
[0058] Algorithm design is required for the collaboration of multiple monitoring devices. If two sets of devices meet the usage requirements, in actual use, when there are carriages in front of both sets of devices simultaneously, both sets of devices will perform detection and data transmission concurrently. Because the direction of carriage movement can be detected, and the occurrence of gaps in the two sets of devices follows a pattern and a sequential relationship, the data conversion between the two sets of devices can be implemented based on this pattern, ensuring the continuity and stability of data for the loading system.
Claims
1. A method for detecting train carriage displacement based on structured light vision, comprising a train carriage, wherein the carriage surface has continuously spaced vertical steel bars, the steel bars being used to reinforce the side panels of the carriage, characterized in that, The method includes setting up a carriage displacement detection device next to the passing carriage. The carriage displacement detection device includes a laser projection device and cameras set on the left and right sides of the laser projection device. The laser projection device projects a certain laser irradiation width and height onto the surface of the passing carriage. The cameras on both sides act as binocular vision and move with the carriage, synchronously acquiring structured light images of the carriage surface with the laser irradiation width in a continuous frame manner, and recording the time difference between frames. The vertical steel bars to be tracked in the structured light images are obtained. The magnitude, direction and speed of the carriage displacement distance are obtained based on the laser irradiation width and the time difference between the vertical steel bars entering and leaving the structured light image area. The steps for acquiring the vertical steel bar to be tracked in the structured light image include: Step 1: Perform laser illumination width and height coordinate region correction on the images simultaneously acquired by the left and right cameras to determine the image processing area; Step 2: Preprocess the corrected left and right images, and convert them into left and right grayscale images respectively. The conversion is performed using a weighted average method based on the importance of each channel of the RGB image, according to Formula 1. , Formula 1; in: K1, K2, and K3 are weight ratio values; Step 3: Perform horizontal scans on the left and right grayscale images within the processing area, and calculate the window energy at each scan point using Formula 2: , Official 2; in: (x,y) represents the coordinates of the scan point, which is also the center coordinate of the calculation window; n represents the distance from the center of the selected window of the left grayscale image to the edge; Represents image coordinates The grayscale value of the image at that location; The maximum value of E(x,y) in each scan line represents the imaging point of the line laser. Since the number of line lasers is 1, the ordinate is fixed. The maximum values of E(x,y) in each scan line are sorted from left to right by their x-coordinates and denoted as (x,y)_k, k=1,2. When on a plane, the Y-coordinate value remains consistent. When a vertical rebar appears, its Y-coordinate is significantly greater than the coordinates of other points. Then its X-coordinate is the vertical rebar point that needs to be marked and tracked.
2. The method for detecting carriage displacement according to claim 1, characterized in that, At least two sets of carriage displacement detection devices shall be installed within the length of the carriage.
3. The method for detecting carriage displacement according to claim 1, characterized in that, Next to the laser projection equipment, a sensor is set up side by side. The sensor is used to detect when the moving carriage has entered the laser projection area of the laser projection equipment.
4. The method for detecting carriage displacement according to claim 1, characterized in that, The structured light image is a visual difference image of the vertical steel bar illumination obtained by processing images synchronously acquired by cameras on both sides.
5. The method for detecting carriage displacement according to claim 1, characterized in that, The laser irradiation width is at least the width of two vertical steel bars. The magnitude, direction, and speed of the carriage displacement distance are continuously obtained by measuring the time difference between the continuous entry and exit of the vertical steel bars into the structured light image area.
6. The method for detecting carriage displacement according to claim 1, characterized in that, The weight ratio values K1, K2, and K3 are parameters that are optimally selected through on-site environmental debugging.