A machine vision-based automatic railway container yard train leading car positioning method

Abstract

The application discloses a kind of automatic railway container yard train first vehicle positioning method based on machine vision, comprising: S1, through high-definition vision camera collection real-time picture information obtains the identification information of each container of train passing through corresponding track crane, calculates the real-time car number of train passing through each track crane;S2, the absolute coordinate of container train first vehicle is calculated by data processing module;S3, the real-time picture information collected by high-definition vision camera obtains the train skin image information passing through the track crane directly below, whether train is completed by data processing module and judges whether parking;S4, the absolute coordinate of container train first vehicle calculated by calculation module and the absolute coordinate of each track crane, and then the absolute position of each car that track crane needs to park when loading and unloading container operation is calculated by information fusion.According to the application, the complex situation of container train parking is adapted while ensuring the automation operation precision, and the maintenance cost and expansion complexity are greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent mechanical equipment, and particularly relates to a train head positioning method for an automated railway container yard based on machine vision. BACKGROUND

[0002] In an automated railway container yard, a train enters from an entrance throat, drives from a track crane until it is stable. Then the track crane carries out loading and unloading operations. Since the track crane in the automated railway container yard implements unmanned operation, an automatic device for positioning the head of a train in the automated railway container yard is needed to realize the autonomous positioning of the track crane for loading and unloading operations.

[0003] At present, the positioning mode of the track crane for railway container automation mainly uses a photoelectric encoder installed on a driven wheel on one side to position the track crane. However, this technology cannot meet the demand for position accuracy of automation, and the encoder error is accumulated due to wheel sliding when the train suddenly stops and starts. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application aims to provide a train head positioning method for an automated railway container yard based on machine vision, which can adapt to complex situations of container train parking while ensuring the automation accuracy, and greatly reduces the maintenance cost and expansion complexity. In order to achieve the above-mentioned purposes and other advantages according to the present application, a train head positioning method for an automated railway container yard based on machine vision is provided, comprising:

[0005] a plurality of track crane girders, a high-definition vision camera fixed on the track crane, a train located below the track crane, a complete car loaded on the train, and a container train head located at the head of the train;

[0006] The container yard train head positioning method comprises the following steps:

[0007] S1, acquiring the real-time picture information of the train driving through each container of the corresponding track crane by the high-definition vision camera to obtain the identification information of the train, and calculating the real-time car number of the train passing through each track crane;

[0008] S2, calculating the absolute coordinates of the head of the container train by the data processing module;

[0009] S3, acquiring the image information of the train skin driving through the track crane directly below by the high-definition vision camera, and determining whether the train has completed parking by the data processing module;

[0010] S4, the absolute coordinates of the first car of the container train and the absolute coordinates of each rail-mounted crane are calculated by the calculation module, and then the absolute position of each car where the rail-mounted crane needs to stop during the loading and unloading of containers is calculated through information fusion.

[0011] Preferably, in step S2, the length of each car of the train is obtained from the car RFID information provided when the train arrives, the absolute coordinates of each rail-mounted crane are obtained from the BMS / TOS system of the central station, and then the absolute coordinates of the first car of the container train are calculated by the data processing module.

[0012] Preferably, the processing steps of the high-definition image in step S1 include:

[0013] S11, real-time sharpening and noise reduction image processing is performed on the real-time high-definition container train car aerial view to obtain high-quality samples;

[0014] S12, the obtained high-quality samples are input into a neural network for target detection, and the detected container samples trigger a counter;

[0015] Preferably, step S2 further includes the following steps:

[0016] S21, the length of each car of the train is obtained from the car RFID information provided when the train arrives;

[0017] S22, the absolute coordinate information of each rail-mounted crane is obtained from the BMS / TOS system of the central station;

[0018] S23, the real-time relative distance of the first car from the rail-mounted crane is calculated according to the number of train cars passing through the rail-mounted crane and the obtained length information of each car;

[0019] S24, the real-time absolute coordinates of the first car of the container train are calculated according to the real-time relative distance of the first car of the container train from the rail-mounted crane and the obtained absolute coordinate information of each rail-mounted crane.

[0020] Preferably, step S3 further includes the following steps:

[0021] S31, the train car image information passing through the rail-mounted crane directly below is obtained from the real-time picture information collected by the high-definition visual camera at the top of each rail-mounted crane;

[0022] S32, the data processing module calculates and judges whether the train has completed stopping;

[0023] S33, if the data processing module still determines that the train has not completed stopping after a set time, the intelligent alarm module is started to provide relevant operators and limit the action of the rail-mounted crane;

[0024] Preferably, step S4 further comprises the following steps:

[0025] S41, according to the calculation information obtained by each rail-mounted crane, the precise absolute coordinates of the first car of the container train are calculated through information fusion;

[0026] S42, according to the precise coordinates of the first car of the container train after information fusion and the absolute coordinates of each rail-mounted crane itself, the absolute coordinates of each car on the track are obtained by inputting the calculation module.

[0027] Preferably, step S5 further comprises the following steps:

[0028] S51, the sample features are enhanced by performing multi-scale color restoration on the high-definition vision camera to obtain samples;

[0029] S52, the sample features are further enhanced by performing rectangular fitting on the multi-scale color restored image;

[0030] S53, the sample real-time input neural network is strengthened to identify the train car passing directly below the rail-mounted crane;

[0031] S54, the information fusion is performed by identifying the train car passing below through multiple rail-mounted cranes, and the counter is triggered to calculate the number of containers passing through the rail-mounted crane when the container translates forward with the train.

[0032] Compared with the prior art, the present application has the beneficial effects that: the high-definition image collected by each rail-mounted crane is transmitted into the neural network in real time to identify the container train car in real time, and the counter is triggered to obtain the number of container train cars passing directly below each rail-mounted crane. According to the RFID information obtained when the railway container train enters, the length-changing data of each car and the absolute coordinate information of each rail-mounted crane obtained from the BMS / TOS system of the central station are input into the data processing module to calculate the coordinates of the first car of the container train in real time. After the container train stops, the data processing module distributes the absolute coordinates of each car to the rail-mounted crane for loading and unloading operations. The first layer of this method is the real-time tracking method of the first car coordinates when the container train enters, and the second layer operation is performed after the train is completely stopped. The second layer is the positioning of the car corresponding to the rail-mounted crane loading and unloading operation. The present application can adapt to the complex situation of container train stopping while ensuring the automation operation precision, and the maintenance cost and expansion complexity are greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 It is a front view of the automatic railway container yard train first car positioning method based on machine vision according to the present application;

[0034] Figure 2A top view of a method for positioning a first car of a container train in an automated railway container yard based on machine vision according to the present application;

[0035] Figure 3 A container train car recognition trigger counter schematic diagram of a method for positioning a first car of a container train in an automated railway container yard based on machine vision according to the present application;

[0036] Figure 4 A schematic diagram of a container yard train first car positioning method of a method for positioning a first car of a container train in an automated railway container yard based on machine vision according to the present application;

[0037] Figure 5 A container train parking determination method schematic diagram of a method for positioning a first car of a container train in an automated railway container yard based on machine vision according to the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0039] REFERENCE Figures 1-5 A method for positioning a first car of a container train in an automated railway container yard based on machine vision, comprising: a plurality of rail-mounted gantries 1, a high-definition vision camera 2 fixed to the rail-mounted gantry 1, a train 3 located below the rail-mounted gantry 1, a complete car 4 loaded on the train 3, and a container train first car 7 located at the front of the train 3, as shown in Figures 1-2 The high-definition vision camera 2 installed above the trackway at the bottom of the rail-mounted gantry 1 ensures that its working range 6 covers at least one complete car 4 of the train 3 passing directly below the rail-mounted gantry, for collecting image data of the container car 4 of the train, and the information of the container car 4 on the train 3 passing directly below the rail-mounted gantry 1 obtained by the high-definition vision camera 2, in combination with the RFID information obtained when the railway container train enters the yard, obtains the length change data of each car and the absolute coordinate information of each rail-mounted gantry 1 obtained from the BMS / TOS system of the central station, to calculate the coordinates of the container train first car 7 in real time, and to assign the absolute coordinates of each car to the rail-mounted gantry 1 for loading and unloading operations after the train is parked. When the data processing module information of the multiple rail-mounted gantries 1 is abnormal, the alarm module issues a warning to provide relevant operators and restricts the action of the rail-mounted gantry 1.

[0040] The data acquisition module includes a high-definition vision camera 2, a train approach RFID identifier, and a BMS / TOS system of a central station. The high-definition vision camera 2 is installed at the bottom of the girder of the track crane and is used to collect image data of the container train car. The train approach RFID identifier is connected to the data processing module and is used to obtain the length change data of each car. The BMS / TOS system of the central station is connected to the data processing module and is used to obtain the absolute coordinate information of each track crane 1.

[0041] The data processing module is arranged in the electrical room / electrical cabinet of each track crane 1 and is connected to the data acquisition module. The data processing module is used to process the image data of the railway container train car collected by the data acquisition module. The data processing module is used to obtain the length change data of each car according to the RFID information obtained when the railway container train approaches and to obtain the absolute coordinate information of each track crane from the BMS / TOS system of the central station. The data processing module is used to calculate the coordinates of the first car 7 of the container train in real time and to assign the absolute coordinates of each car to the track crane for loading and unloading operations after the train is stopped. The data processing module uses an intelligent computing board, which is connected to the data acquisition module and is used to process the collected visual images and to identify the container train car 4 passing under the track crane to calculate the accurate coordinates of the first car 7 of the container train and to assign the absolute coordinates of each car to the track crane 1 for loading and unloading operations after the information is fused.

[0042] The intelligent alarm module is arranged in the electrical room or command center of the track crane 1 and is connected to the data processing module and the electrical control system of the track crane 1. The intelligent alarm module is used to issue a warning when the information fusion of the data processing module of the track crane 1 is abnormal, to provide relevant operators, and to limit the movement of the track crane 1. The intelligent alarm module includes an IO module, which is connected to the electrical control system of the track crane 1 and is used to output alarm information.

[0043] As shown in Figure 3 , in the specific implementation process, the high-definition vision camera 2 obtains samples from the container train car 4 passing below the train. The coverage range 6 of the high-definition vision camera 2 needs to ensure that its working range covers at least one complete car 4 of the train 3 passing below the track crane. In the process of identifying the container train passing below, there are Figure 3 two cases:

[0044] In the first case, as shown in Figure 3 , a container train car 4 completely covers the working range of the high-definition vision camera 2. At this moment, the image is input into the neural network for recognition. If the container train car 4 is successfully recognized, the counter is triggered, that is, the number of container train cars 4 passing below the track crane 1 is accumulated.

[0045] The second case, as shown in Figure 3 When the second container train car 4 is not completely out of the working range of the high-definition vision camera 2, and the other container train car 4 is not completely into the working range of the high-definition vision camera 2, the data processing module does not trigger the counter until the next container train car 4 is completely into the working range of the high-definition vision camera 2 is identified.

[0046] As shown in Figure 4 The absolute coordinate information of each rail-mounted gantry crane 1 obtained in the BMS / TOS system of the central station can be represented as:

[0047] The absolute coordinate of the first rail-mounted gantry crane 1(1) is (X RMG1 , Y RMG1 );

[0048] The absolute coordinate of the second rail-mounted gantry crane 1(2) is (X RMG2 , Y RMG2 );

[0049] And the absolute coordinate of the i-th rail-mounted gantry crane is (X RMGi , Y RMGi );

[0050] As shown in Figure 4 The RFID information obtained when the railway container train arrives can obtain the length change data of each car, which can be represented as:

[0051] The length change of the first car 4(1) is CL1

[0052] The length change of the second car 4(1) is CL2

[0053] The length change of the third car 4(1) is CL3

[0054] And the length change of the i-th car 4(1) is CL i

[0055] As shown in Figure 4 The absolute coordinate of the first car 7 of the container train can be represented as (X Loco , Y Loco )

[0056] As shown in Figure 4 The relative distance 8(1) between the first car 7 of the container train and the first rail-mounted gantry crane can be represented as L1, the relative distance 8(2) between the first car and the second rail-mounted gantry crane can be represented as L2, and the relative distance between the first car and the i-th rail-mounted gantry crane can be represented as L i

[0057] According to the above information, the system uses the above information to calculate the relative distance Li The calculation formula is as shown in equation (1):

[0058]

[0059] like Figure 4 As shown, if the angle between the yard coordinate system and the axis of the main view of the rail-mounted crane is α, then each rail-mounted crane 1 is determined according to its own absolute coordinates (X... RMGi Y RMGi The relative distance L between the first container train (7) and the i-th rail-mounted crane (1). i The input system can calculate the absolute coordinates (X) of the first vehicle. Locoi Y Locoi The formulas are as shown in equations (2) and (3):

[0060]

[0061]

[0062] like Figure 5 As shown, during the docking process of the container train, the visual camera will identify the carriage 4 and perform rectangular fitting, and detect whether it has come to a complete stop by the displacement 9 of its center of gravity.

[0063] Furthermore, the thresholds set by the detection system are all externally input, thus allowing for adaptation to local conditions. The basic steps are as follows: The data processing module identifies the corner points of the train container wagons in real time and fits a rectangle. After determining the center position of the rectangle, it continuously observes the offset of the center position at equal intervals. If the offset of the center position within a set time is less than the set value, it is determined to be in a stopped state, and the data processing module sends loading and unloading instructions to each rail-mounted crane.

[0064] If the center position offset exceeds the set value within the set time, it is determined to be in running state. If the running time exceeds the set value, the alarm module is triggered and outputs alarm information.

[0065] The method for positioning the first train in a container yard includes the following steps:

[0066] S1. Collect real-time image information through high-definition vision camera 2 to obtain the identification information of each container that the train 3 passes by the corresponding rail crane, and calculate the real-time number of carriages 4 that the train passes by each rail crane.

[0067] S2. Calculate the absolute coordinates of the first container train 7 using the data processing module;

[0068] S3, obtain the image information of the train car body directly below the track crane girder 1 through the real-time picture information collected by the high-definition visual camera 2, and determine whether the train has completed parking through the data processing module, wherein the railway container train parking determination method comprises the following steps:

[0069] The data processing module identifies the train container train car body corner point in real time and fits a rectangle, determines the center position of the rectangle, and continuously observes the center position offset in the same interval. If the center position offset in the set time is less than the set value, it is determined that it is in the parking state, and the data processing module sends the loading and unloading instructions to each track crane 1.

[0070] If the center position offset in the set time is greater than the set value, it is determined that it is in the running state, and the running time exceeds the set value. The alarm module is triggered, and the alarm information is output;

[0071] S4, calculate the absolute coordinates of the first car 7 of the container train and the absolute coordinates of each track crane 1 through the calculation module, and then calculate the absolute position of each car 4 that needs to be parked by the track crane 1 during the loading and unloading of the container through information fusion.

[0072] Further, in step S2, the length of each car of the train is obtained from the car body RFID information provided when the railway container train enters, and the absolute coordinates of each track crane 1 are obtained from the BMS / TOS system of the central station. Then the data processing module calculates the absolute coordinates of the first car 7 of the container train.

[0073] Further, the processing steps of the high-definition image in step S1 include:

[0074] S11, real-time sharpening and noise reduction image processing is performed on the real-time obtained high-definition container train car body aerial photograph to obtain high-quality samples;

[0075] S12, input the obtained high-quality samples into the neural network for target detection, and the detected container samples trigger the counter;

[0076] Further, step S2 further comprises the following steps:

[0077] S21, obtain the length of each car of the train from the car body RFID information provided when the container train enters the database;

[0078] S22, obtain the absolute coordinate information of each track crane 1 from the BMS / TOS system of the central station;

[0079] S23, calculate the real-time relative distance of the first car from the track crane 1 according to the number of train car bodies of the track crane 1 and the obtained length information of each car;

[0080] S24, calculate the real-time absolute coordinate of the container train head car 7 according to the real-time relative distance between the container train head car 7 and the track crane 1 and the obtained absolute coordinate information of each track crane 1.

[0081] Further, step S3 further includes the following steps:

[0082] S31, obtain the train car image information passing through the track crane 1 according to the real-time picture information collected by the high-definition vision camera 2 at the top of each track crane 1;

[0083] S32, calculate and determine whether the train has completed parking through the data processing module;

[0084] S33, if the data processing module still determines that the train has not completed parking after a set time, start the intelligent alarm module to provide relevant operators and limit the action of the track crane 1;

[0085] Further, step S4 further includes the following steps:

[0086] S41, calculate the accurate absolute coordinate of the container train head car 7 according to the calculation information obtained by each track crane 1;

[0087] S42, input the accurate coordinate of the container train head car 7 after information fusion and the absolute coordinate of each track crane 1 into the calculation module to obtain the absolute coordinate of each car track crane work.

[0088] Further, step S5 further includes the following steps:

[0089] S51, enhance the sample features by multi-scale color restoration of the samples obtained by the high-definition vision camera 2;

[0090] S52, further enhance the sample features by rectangular fitting of the multi-scale color restored image;

[0091] S53, strengthen the sample real-time input neural network to identify the train car passing through the track crane;

[0092] S54, identify the passing train car through information fusion of multiple track cranes 1, and trigger the counter to calculate the number of containers passing through the track crane 1 when the container moves forward with the train.

[0093] The container train carriage recognition method based on machine vision comprises the following steps: a data acquisition module, at least one high-definition vision camera 2 is provided for each rail-mounted gantry. The high-definition vision camera 2 is installed at the bottom of the rail-mounted gantry 1 to take pictures downward, and the image data of the train 3 passing below the rail-mounted gantry is input into a neural network to recognize the container train carriage after image processing, wherein the image processing process comprises synthesizing an enhanced image after processing each color channel; converting the picture in the RGB color space into the HSV color space and performing threshold processing in the V space; binarizing the image and performing Gaussian blur again to eliminate noise pixels. The above operations will effectively eliminate irrelevant information in the image and help improve the measurement accuracy.

[0094] The present application provides two container recognition methods: corner hole recognition method and container train carriage rectangle fitting method.

[0095] The corner hole recognition method completes the recognition of the container train carriage by recognizing the four corner holes of the container train carriage and fitting a rectangle through the lock hole positioning.

[0096] The container train carriage rectangle fitting method completes the recognition of the container train carriage by directly fitting a rectangle to the processed image.

[0097] The corner hole recognition method alone has high recognition accuracy, but may cause the phenomenon of "connection" of the container train carriage. The container train carriage rectangle fitting method alone has high recognition speed, but may cause the phenomenon of inaccurate recognition. Therefore, the two methods are used together to supplement the details of the container train carriage rectangle fitting method through the corner hole recognition method to ensure its accuracy and efficiency at the same time.

[0098] The present application also provides an auxiliary container method based on machine vision, comprising the following steps:

[0099] The data processing module obtains the real-time position of the container train carriage from the real-time image data collected by the rail-mounted gantry 1. When the loading and unloading operation is performed, the deviation value between the rail-mounted gantry 1 and the standard position of the container operation is calculated according to the visual image, the rail-mounted gantry 1 is fine-tuned to move to the standard position, and the spreader automatic guidance is performed.

[0100] The number of devices and the scale of processing described herein are used to simplify the description of the present application, and the application, modification and change of the present application are obvious to those skilled in the art.

[0101] While embodiments of the application have been disclosed in connection with the above specification, it will be evident to those skilled in the art that many modifications, substitutions, and alterations to the embodiments of the application can be made and that many specifically protected details shown can be substituted by other specifically protected details. Accordingly, it is intended that the application not be limited to the specific details shown and described, but that it be given broadest scope indicated by the claims and their equivalents.

Claims

1. A method for positioning the first car of a train in a railway container yard based on machine vision, characterized in that, The application relates to a container yard train head positioning method and device. The device comprises the following: a plurality of girder track cranes (1), high-definition vision cameras (2) fixed on the track cranes (1), a train (3) located below the track cranes (1), a complete car (4) loaded on the train (3), and a container train head car (7) located at the head of the train (3). The container yard train head positioning method comprises the following steps: S1, obtaining the identification information of each container of the train (3) passing through the corresponding track crane through the high-definition vision camera (2) to collect real-time picture information, and calculating the real-time number of carriages (4) of the train passing through each track crane; S2, calculating the absolute coordinates of the container train head car (7) through a data processing module; The step S2 further comprises the following steps: S21, obtaining the length of each carriage of the train according to the carriage RFID information provided when the container train enters the yard; S22, obtaining the absolute coordinate information of each track crane (1) from the BMS / TOS system of the central station; S23, calculating the real-time relative distance of the train head car (7) from the track crane (1) according to the number of carriages of the train passing through the track crane (1) and the length information of each carriage obtained; S24, calculating the real-time absolute coordinates of the container train head car (7) according to the real-time relative distance of the train head car (7) from the track crane (1) and the absolute coordinate information of each track crane (1) obtained; S3, obtaining the train carriage image information passing through the track crane (1) directly below through the real-time picture information collected by the high-definition vision camera (2), and judging whether the train has completed parking through a data processing module; S4, calculating the absolute position of each carriage (4) required for the track crane (1) to stop during the loading and unloading of containers through information fusion according to the absolute coordinates of the container train head car (7) and the absolute coordinates of each track crane (1) calculated by the calculation module.

2. The method of claim 1, wherein the method comprises: In step S2, the length of each carriage of the train is obtained from the carriage RFID information provided when the container train enters the yard, the absolute coordinates of each track crane (1) are obtained from the BMS / TOS system of the central station, and then the absolute coordinates of the container train head car (7) are calculated by the data processing module.

3. A machine vision based automated railway container yard train lead car positioning method according to claim 2, characterized in that, The processing steps of the high-definition image in step S1 include: S11, performing real-time sharpening and noise reduction image processing on the high-definition container train carriage overhead image obtained in real time to obtain a high-quality sample; S12, inputting the obtained high-quality sample into a neural network for target detection, and triggering a counter when a container sample is detected.

4. The automated railway container yard train first car positioning method based on machine vision as described in claim 1, characterized in that, Step S3 further comprises the following steps: S31, obtaining the train carriage image information passing through the track crane (1) directly below according to the real-time picture information collected by the high-definition vision camera (2) at the top of each track crane (1); S32, calculating and judging whether the train has completed parking through a data processing module; S33, if the train has not completed parking after a set time, starting an intelligent alarm module, providing relevant operators, and limiting the action of the track crane (1).

5. The automated railway container yard train first car positioning method based on machine vision as described in claim 1, characterized in that, Step S4 further comprises the following steps: S41, according to the calculation information obtained by each rail-mounted gantry (1), information fusion is performed to calculate the accurate absolute coordinates of the first car (7) of the container train; S42, according to the accurate coordinates of the first car (7) of the container train after information fusion and the absolute coordinates of each rail-mounted gantry (1) itself, the absolute coordinates of each car on the track are input into the calculation module.

6. A machine vision based automated railway container yard train lead car positioning method as claimed in claim 1, wherein, In step S5 The method further includes the following steps: S51, the sample obtained by the high-definition vision camera (2) is subjected to multi-scale color restoration to enhance its sample features; S52, the image subjected to multi-scale color restoration is subjected to rectangular fitting to further enhance its sample features; S53, the sample is input into a neural network in real time to identify the train car passing directly below the rail-mounted gantry; S54, the information of the train car passing below is identified by multiple rail-mounted gantries (1) to perform information fusion, and a counter is triggered to calculate the number of containers passing through the rail-mounted gantry (1) when the containers are translated forward with the train.

Images