Anti-collision method for rail-mounted gantry crane during automated operation, and rail-mounted gantry crane

Abstract

The present invention belongs to the technical field of obstacle detection. Disclosed are an anti-collision method for a rail-mounted gantry crane during automated operation, and a rail-mounted gantry crane. The method comprises: using a camera to acquire pixel points of left and right lane lines, and generating two-dimensional plane equations for the left and right lane lines; respectively extracting two coordinate points from the two-dimensional plane equations of the left and right lane lines, and performing transformation by means of a first transformation matrix and a second transformation matrix to then obtain coordinate points under a laser coordinate system, so as to generate left and right lane line equations under the laser coordinate system; by means of the second transformation matrix, transforming point cloud data collected by a LiDAR, so as to form three-dimensional point cloud coordinates under the laser coordinate system, and filtering out, from the three-dimensional point cloud coordinates, ground point cloud coordinates, and point cloud coordinates outside the left and right lane lines under the laser coordinate system; and on the basis of the remaining obstacle point clouds, determining whether there is an obstacle. The method in the present invention is simple and effective, and can quickly output a determination result, thereby allowing sufficient response time for a rail-mounted gantry crane to stop and avoid obstacles, and thus ensuring the travel safety of the rail-mounted gantry crane.

Description

Rail-mounted crane automatic operation anti-collision method and rail-mounted crane TECHNICAL FIELD

[0001] The present application belongs to the technical field of obstacle detection, and particularly relates to a rail-mounted crane automatic operation anti-collision method and a rail-mounted crane. BACKGROUND

[0002] The rail-mounted crane is a portal crane, which is mainly used to complete the lifting, stacking and loading and unloading of containers between the container yard and the container truck (hereinafter referred to as the container truck) outside the yard.

[0003] In the traditional container terminal, when the container loading and unloading operation is performed, the rail-mounted crane driver finds the container truck arriving at the operation position, and then manually controls the rail-mounted crane to move to the operation position to start the loading and unloading operation. During the movement of the rail-mounted crane, if there is an obstacle in the running track area, the driver needs to manually stop the rail-mounted crane when the obstacle is found. However, due to the limited field of view, the driver is difficult to quickly determine whether there is an obstacle in the running area of the rail-mounted crane and the accurate position of the obstacle, and thus there is a safety hazard. To solve this problem, an automatic anti-collision function needs to be realized on the rail-mounted crane.

[0004] At present, the commonly used rail-mounted crane automatic anti-collision method in the industry mainly includes infrared ranging and ultrasonic ranging, but these methods have the disadvantages of short detection distance, low precision, high false detection rate, etc. In recent years, with the rapid development of laser radar scanning technology and image recognition technology, technologies applying laser radar and image sensors (such as cameras, etc.) to the field of obstacle detection have emerged, such as the Chinese invention patent application “Obstacle detection method and obstacle detection device applied to vehicle” with the publication number CN 113985405 A and the Chinese invention patent application “Rail transit active obstacle detection device based on laser and image data fusion” with the publication number CN 113568002 A. Both of them determine the target point cloud data according to the point cloud data obtained by the laser radar and the image information collected by the image sensor; then perform obstacle recognition on the target point cloud data and the image information respectively to obtain the first obstacle information corresponding to the target point cloud data and the second obstacle information corresponding to the image information; and perform obstacle fusion on the first obstacle information and the second obstacle information to determine the target obstacle.

[0005] Although this detection method can improve the accuracy of obstacle recognition, the recognition process requires the use of complex machine learning algorithms, such as for point cloud data obtained by a laser radar, a high-efficiency clustering algorithm is required to classify non-ground point clouds by features, analyze various target objects in the point cloud, and extract the target object contour; and for image information collected by an image sensor, an artificial intelligence convolutional neural network algorithm is required to detect obstacles in the image and identify the type of obstacle. The use of machine learning algorithms will occupy a large amount of resources of the processor, and the running time is long, which affects the detection speed and is not conducive to the emergency obstacle avoidance of the rail-mounted gantry crane. TECHNICAL PROBLEM

[0006] The purpose of the present application is to provide a rail-mounted gantry crane automatic operation anti-collision method and a rail-mounted gantry crane, so as to solve the problem that the existing obstacle detection method has a short detection distance and low precision, or a complex algorithm that affects the detection speed. TECHNICAL SOLUTION

[0007] To solve the above technical problems, the present application adopts the following technical solutions:

[0008] In one aspect, the present application provides a rail-mounted gantry crane automatic operation anti-collision method, comprising:

[0009] A first transformation matrix is established for conversion from a camera coordinate system to a laser coordinate system;

[0010] A second transformation matrix is established for converting point cloud coordinates collected by a laser radar into point cloud coordinates in the laser coordinate system;

[0011] A two-dimensional image in a set region in front of the rail-mounted gantry crane walking direction is captured by a camera;

[0012] Three-dimensional point cloud data in the set region in front of the rail-mounted gantry crane walking direction is collected by a laser radar;

[0013] Pixel points of left and right lane lines are extracted from the two-dimensional image captured by the camera;

[0014] Two-dimensional plane equations of the left and right lane lines are generated by using the extracted pixel points of the left and right lane lines;

[0015] Two coordinate points are extracted from the two-dimensional plane equations of the left and right lane lines, and after conversion by the first transformation matrix and the second transformation matrix, coordinate points in the laser coordinate system are obtained;

[0016] Left and right lane line equations in the laser coordinate system are generated by using the coordinate points in the laser coordinate system;

[0017] After conversion by the second transformation matrix, three-dimensional point cloud coordinates in the laser coordinate system are formed from the point cloud data collected by the laser radar;

[0018] Filtering out ground point cloud coordinates from the three-dimensional point cloud coordinates;

[0019] From the non-ground point cloud coordinates, the point cloud coordinates whose x coordinates and y coordinates are located between the left and right lane lines under the laser coordinate system are selected as obstacle point cloud, and the non-obstacle point cloud is deleted;

[0020] When the number of obstacle point clouds exceeds a set threshold, it is determined that there is an obstacle;

[0021] When it is determined that there is an obstacle, the track crane is parked.

[0022] In some embodiments of the present application, the two-dimensional image captured by the camera can be segmented using a lane line segmentation model to extract the pixel points of the left and right lane lines of the lane where the track crane is located; then, the RANSAC algorithm is used to fit the pixel points of the left and right lane lines respectively to obtain the two-dimensional plane equation of the left and right lane lines.

[0023] In some embodiments of the present application, in order to realize the automatic obstacle avoidance function on the track crane, the x axis of the camera coordinate system and the laser coordinate system can be first configured to face the walking direction of the track crane; then, when it is determined that there is an obstacle, the minimum x coordinate value in the obstacle point cloud is taken as the distance between the obstacle and the track crane; when the distance between the obstacle and the track crane is less than a set safety threshold, the track crane is controlled to stop. Here, the safety threshold is set to avoid the influence of moving obstacles on the normal driving of the track crane and to avoid unnecessary stopping of the track crane, which affects the work efficiency.

[0024] In some embodiments of the present application, in order to realize accurate conversion from the camera coordinate system to the laser coordinate system, the establishment process of the first transformation matrix can include:

[0025] The camera and the laser radar are calibrated using the same calibration board;

[0026] The camera and the laser radar scan the calibration board and obtain the x coordinates and y coordinates of a plurality of corner points on the calibration board under the camera coordinate system and the laser coordinate system respectively;

[0027] According to the conversion relationship between the x coordinates and y coordinates of the same corner points under the camera coordinate system and the laser coordinate system, a first transformation matrix is established, so that the coordinate points under the camera coordinate system are converted to corresponding coordinate points under the laser coordinate system after the first transformation matrix, and the z axis coordinate is ignored.

[0028] In some embodiments of the present application, in order to convert the point cloud coordinates collected by the laser radar to the established laser coordinate system, the establishment process of the second transformation matrix can include:

[0029] selecting a plurality of point cloud coordinates from the ground laser point cloud data collected by the laser radar, and finding corresponding points of the plurality of point cloud coordinates on an xoy plane of a laser coordinate system;

[0030] using an svd decomposition method to construct a linear equation set, to obtain parameters of a transformation matrix, and to form a second transformation matrix.

[0031] In some embodiments of the present application, since the lane lines are formed on the ground and do not have z-axis coordinates, after two coordinate points are extracted from the two-dimensional plane equations of the left and right lane lines respectively and converted through the first transformation matrix, only the x-axis and y-axis coordinates can be converted using the second transformation matrix, and the z-axis coordinate can be directly assigned a value of 0, thereby obtaining four coordinate points in the laser coordinate system; using the x-axis and y-axis coordinates of the four coordinate points in the laser coordinate system, the left lane line equation and the right lane line equation in the laser coordinate system can be generated, to distinguish between obstacles located between the left and right lane lines and affecting the normal walking of the track crane, and obstacles located outside the left and right lane lines, which do not affect the normal walking of the track crane, can be ignored.

[0032] In some embodiments of the present application, the screening process of the obstacle point cloud can include:

[0033] After the point cloud data collected by the laser radar is converted through the second transformation matrix, three-dimensional point cloud coordinates in the laser coordinate system are formed, and three-dimensional point cloud coordinates with a z-coordinate value less than or equal to 0.1 are selected as ground point cloud coordinates for preliminary filtering; then, the x-coordinate value of each non-ground point cloud coordinate is substituted into the left and right lane line equations in the laser coordinate system, respectively, to calculate two y values.

[0034] If the y-coordinate value of the non-ground point cloud coordinate is between the two calculated y values, it indicates that the point cloud is located between the left and right lane lines and should be retained as an obstacle point cloud; otherwise, the point cloud coordinate is filtered out and ignored.

[0035] In some embodiments of the present application, for the retained obstacle point cloud, a set threshold value can be set in advance to determine whether it is an obstacle. When the number of obstacle point clouds is greater than the set threshold value 10, it indicates that it is an obstacle, otherwise, it indicates that it is not an obstacle. In order to improve the accuracy of the obstacle judgment result, the specific value of the set threshold value needs to be considered specially. If the set threshold value is set too small, noise points may be mistaken for obstacles; if the set threshold value is set too large, small obstacles may be missed. Therefore, the present application sets the set threshold value to 10 to achieve accurate identification of obstacles.

[0036] In another aspect, the present application also provides a rail-mounted crane, comprising a trolley mechanism for driving the whole rail-mounted crane to walk, a camera and a laser radar mounted on the trolley mechanism, and a control system in communication with the camera and the laser radar; the control system is configured to run a computer program to execute the above-mentioned anti-collision method for automated operation of the rail-mounted crane.

[0037] In some embodiments of the present application, the rail-mounted crane reciprocates between the left and right lane lines demarcated in the field area; in order to effectively collect obstacles in the area in front of the rail-mounted crane during walking in two directions, a set of cameras and laser radars are respectively installed on the front and rear legs of the trolley mechanism, and the naming of the front and rear legs should be adjusted according to the change of the walking direction of the rail-mounted crane; then, a set of cameras and laser radars mounted on the front legs are turned on according to the walking direction of the rail-mounted crane, and can scan the obstacles in the area 1m-15m in front of the walking direction of the rail-mounted crane, so as to leave enough time for the rail-mounted crane to slow down and stop. Advantages

[0038] Compared with the prior art, the advantages and positive effects of the present application mainly lie in:

[0039] 1. The present application uses camera imaging and image recognition technology to obtain the coordinates of the left and right lane lines in the field area where the rail-mounted crane walks, and maps them to the laser coordinate system to construct the left and right lane line equations in the laser coordinate system. The point cloud coordinates of the area in front of the walking direction of the rail-mounted crane are collected by using laser radar scanning technology, and after filtering out the ground point cloud coordinates and the point cloud coordinates outside the left and right lane lines, the remaining point cloud coordinates are the obstacle point cloud. According to the number of obstacle point clouds, it can be judged whether there is an obstacle in front of the rail-mounted crane that affects its normal walking, and the judgment result is accurate and reliable.

[0040] 2. The obstacle recognition method of the present application is simple and effective, does not need to use complex machine learning algorithms, the program is simple to write and runs fast, and can output the judgment result in real time, leaving enough reaction time for the rail-mounted crane to avoid obstacles and stop.

[0041] 3. According to the coordinate values of the obstacle point cloud, the present application can accurately judge the distance between the rail-mounted crane and the obstacle, and by setting a safety threshold, the influence of moving obstacles on the normal driving of the rail-mounted crane can be avoided. In the case of ensuring the safe driving of the rail-mounted crane, the rail-mounted crane does not stop effectively due to the existence of moving obstacles, thereby ensuring the operation efficiency of the rail-mounted crane.

[0042] 4. The obstacle detection and anti-collision stopping function of the present application can be automatically completed by the rail-mounted crane without the participation of personnel, and the degree of automation is high, which ensures the safety of the rail-mounted crane operation.

[0043] Other features and advantages of the present application will become more apparent from the following detailed description of an embodiment thereof, from the claims as well as from the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] Fig. 1 is a structural schematic diagram of an embodiment of a rail-mounted crane;

[0045] Fig. 2 is a structural schematic diagram of an embodiment of the installation position and installation mode of a camera and a laser radar on a rail-mounted crane;

[0046] Fig. 3 is a flowchart of an embodiment of the automatic anti-collision method for rail-mounted crane operation proposed by the present application;

[0047] Fig. 4 is an example diagram of pixel points of left and right lane lines extracted from a two-dimensional image taken by a camera;

[0048] Fig. 5 is an example diagram of left and right lane lines generated by fitting the pixel points of left and right lane lines shown in Fig. 4 in a camera coordinate system;

[0049] Fig. 6 is a distribution diagram of laser point clouds in a laser coordinate system collected by a laser radar. Embodiment of the present application

[0050] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0051] The current rail-mounted crane, as shown in Fig. 1, includes main components such as a car mechanism 1, a trolley mechanism 2, a spreader 3, a power house 4, a driver's cabin 5, etc. Among them, the car mechanism 1 is used to drive the overall movement of the rail-mounted crane, including front and rear two groups of legs 11, 12, each group of legs 11 / 12 includes two vertical columns, the top of the two vertical columns is connected through a cross beam 13 / 14 to form a portal structure. The trolley mechanism 2 is installed between the cross beams 13, 14 of the two portals and moves in the cross beam direction. The spreader 3 is installed on the trolley mechanism and is used to grab and hoist containers. The power house 4 can be installed on the bottom beam 15 connecting the front and rear two groups of legs 11, 12, and is used to provide energy for the power system on the rail-mounted crane. The driver's cabin 5 can be installed on the trolley mechanism 2 and is provided with an operation table for the driver to operate the rail-mounted crane to perform work tasks.

[0052] In order to realize the automatic anti-collision function on the rail-mounted crane, the camera cooperates with the laser radar in this embodiment to collect and identify the obstacles in the front area of the rail-mounted crane.

[0053] Since the rail-mounted gantry crane is walking along the left and right lane lines drawn on the ground when it is working in the container yard, if the front and rear legs 11, 12 are defined according to the walking direction of the rail-mounted gantry crane, the naming mode of the two legs 11, 12 needs to be adjusted accordingly with the change of the walking direction of the rail-mounted gantry crane. For example, if the rail-mounted gantry crane drives eastward, the eastward leg is the front leg and the westward leg is the rear leg; conversely, if the rail-mounted gantry crane drives westward, the westward leg is the front leg and the eastward leg is the rear leg.

[0054] In order to accurately collect the obstacles in the front area of the rail-mounted gantry crane after adjusting the walking direction, the embodiment installs a set of cameras 6 and laser radars 7 on the two sets of legs 11, 12 of the rail-mounted gantry crane. FIG. 2 shows the position and angle relationship of the cameras 6 and laser radars 7 installed on one of the legs 11 / 12. Preferably, the cameras 6 and laser radars 7 are slightly downwardly inclined and installed on the legs 11, 12 of the rail-mounted gantry crane, so as to ensure that the area at least 1m-15m in front of the rail-mounted gantry crane is the scanning area.

[0055] In some embodiments, the cameras 6 can be 2D cameras, and the laser radars 7 can be 3D laser radars. The rail-mounted gantry crane is configured to automatically start the cameras 6 and laser radars 7 installed on the front legs 11 / 12 according to the driving direction, so as to collect the positions of the left and right lane lines 8, 9 and obstacles 10 in the scanning area.

[0056] The specific working process of the automatic anti-collision method of the rail-mounted gantry crane of the embodiment will be described in detail below in combination with FIG. 3.

[0057] S301, a first transformation matrix for converting a coordinate point in a camera coordinate system to a laser coordinate system is established.

[0058] Since the pixel point coordinates collected by the cameras 6 and the point cloud coordinates collected by the laser radars 7 are in different coordinate systems, the conversion relationship between the coordinate systems needs to be determined in advance, so as to correspond the coordinate points in different coordinate systems to the same coordinate system, so as to facilitate the identification of obstacles.

[0059] Therefore, the embodiment first establishes a first transformation matrix for the conversion from the camera coordinate system to the laser coordinate system. The specific process is as follows:

[0060] (1) The same calibration board is used to calibrate the internal parameters of the cameras 6 and the laser radars 7.

[0061] The method of calibrating the internal parameters of the cameras and the laser radars using the calibration board is prior art, which will not be described herein.

[0062] (2) The same point calibration is performed on the camera coordinate system and the laser coordinate system using the calibration board.

[0063] In the present embodiment, the calibration board can be scanned by the camera 6 and the laser radar 7, and the coordinates of the same corner point on the calibration board in the camera coordinate system and the laser coordinate system, i.e., the x coordinate and the y coordinate, are obtained. A plurality of corner point coordinates, preferably more than 9 corner points, are collected for calculating the first transformation matrix.

[0064] (3) According to the coordinates of the same point in the camera coordinate system and the laser coordinate system, the conversion relationship matrix is calculated to form the first transformation matrix.

[0065] In the present embodiment, the first transformation matrix can be established according to the conversion relationship between the x coordinates and the y coordinates of the same corner point on the calibration board in the camera coordinate system and the laser coordinate system, so that the coordinate point in the camera coordinate system is converted into the coordinate of the corresponding point in the laser coordinate system after the first transformation matrix. Since the camera coordinate point is two-dimensional and the laser coordinate point is three-dimensional, the z-axis coordinate does not need to be considered in the conversion process.

[0066] In the present embodiment, the origins of the camera coordinate system and the laser coordinate system can be set at the lower left corner of the scanning area, as shown in FIG. 5, the x-axis is forward, the y-axis is right, and the z-axis of the laser coordinate system is perpendicular to the ground and upward.

[0067] S302, a second transformation matrix for converting the point cloud coordinates collected by the laser radar into the point cloud coordinates in the laser coordinate system is established.

[0068] The ground laser point cloud is a collection of reflection points formed by laser irradiation to the ground, and exists in the form of a plane. Since the laser radar 7 of the present embodiment is installed obliquely downward on the front support leg of the track suspension trolley mechanism, the scanning direction is not vertically downward, and the plane formed by the collected ground laser point cloud will form a certain angle with the xoy plane of the laser coordinate system, so a second transformation matrix needs to be established for converting the point cloud coordinates collected by the laser radar into the point cloud coordinates in the laser coordinate system. The specific process is as follows:

[0069] (1) A plurality of point cloud coordinates are selected from the ground laser point cloud data collected by the laser radar, and the corresponding points of the plurality of point cloud coordinates on the xoy plane of the laser coordinate system are found.

[0070] In the present embodiment, 6 laser points can be selected from the ground laser point cloud data, and the corresponding points of the 6 laser points on the xoy plane of the laser coordinate system are found, and then six groups of corresponding coordinate values are obtained.

[0071] (2) The svd decomposition method is used to construct a linear equation group to obtain the parameters of the transformation matrix to form the second transformation matrix.

[0072] Using the obtained multiple sets of corresponding coordinate values, for example, the six sets of corresponding coordinate values, a linear equation set is constructed using an svd decomposition method, so as to obtain the parameters of the transformation matrix, and a second transformation matrix is formed, so that the plane formed by the ground laser point cloud collected by the laser radar 7 coincides with the xoy plane of the laser coordinate system after being converted by the second transformation matrix.

[0073] The svd decomposition method is prior art, which is not described in detail in this embodiment.

[0074] S303, a two-dimensional image in a set region in front of the walking direction of the track crane is photographed by the camera.

[0075] The established first transformation matrix and second transformation matrix are saved in the control system of the track crane, for automatic calling by the track crane during operation.

[0076] During normal operation of the track crane, the control system in the track crane can be configured to automatically start the camera 6 and laser radar 7 installed on the front support leg of the walking mechanism to operate, so as to scan the region 1m-15m in front of the walking direction of the track crane.

[0077] The camera 6 can collect a two-dimensional image in the scanned region in front of the walking direction of the track crane.

[0078] S304, three-dimensional point cloud data in a set region in front of the walking direction of the track crane is collected by the laser radar.

[0079] The laser radar 7 can collect three-dimensional point cloud data in the scanned region in front of the walking direction of the track crane.

[0080] S305, pixel points of left and right lane lines are extracted from the two-dimensional image photographed by the camera.

[0081] In this embodiment, the two-dimensional image photographed by the camera 6 can be segmented using an existing lane line segmentation model to extract pixel points of left and right lane lines of the lane where the track crane is located.

[0082] As shown in FIG. 4, the extracted pixel points are distributed in the form of two straight lines. If the middle of the two-dimensional image picture is taken as a dividing line, the pixel points on the left side of the middle dividing line are left lane pixel points, and the pixel points on the right side of the middle dividing line are right lane pixel points.

[0083] S306, two-dimensional plane equations of left and right lane lines are generated using the extracted pixel points of left and right lane lines.

[0084] In this embodiment, the pixel points of left and right lane lines can be fitted using an existing RANSAC algorithm to obtain a two-dimensional plane equation y=kleft x+b left and the two-dimensional plane equation of the right lane line is y=k right x+b right As shown in FIG. 5.

[0085] S307, two coordinate points are extracted from the two-dimensional plane equations of the left and right lane lines respectively, and after conversion via the first conversion matrix and the second conversion matrix, coordinate points in the laser coordinate system are obtained.

[0086] In order to correspond the left and right lane lines in the camera coordinate system to the laser coordinate system, the embodiment first extracts two coordinate points P1(x1 camera , y1 camera ), P2(x2 camera , y2 camera ) from the two-dimensional plane equation of the left lane line, and then extracts two coordinate points P3(x3 camera , y3 camera ), P4(x4 camera , y4 camera ) from the two-dimensional plane equation of the right lane line.

[0087] After converting the four coordinate points P1, P2, P3, P4 via the first conversion matrix, the x-axis coordinate and the y-axis coordinate (because there is no z-axis coordinate) are converted by using the second conversion matrix. Since the left and right lane lines are formed on the ground, the ground laser point cloud is formed in the laser coordinate system, and therefore the z-axis coordinate can be assigned as 0, so that four three-dimensional coordinate points P1 ’ (x1 laser , y1 laser , 0), P2 ’ (x2 laser , y2 laser , 0), P3 ’ (x3 laser , y3 laser , 0), and P4 ’ (x4 laser , y4 laser , 0) in the laser coordinate system are obtained.

[0088] S308, the coordinate points in the laser coordinate system are used to generate the left and right lane line equations in the laser coordinate system.

[0089] Since the left and right lane lines also appear as straight lines in the laser coordinate system, the x-axis coordinate and the y-axis coordinate in the coordinate points P1 ’ , P2 ’ are substituted into the straight line equation formula y=kx+b respectively, and the constants k and b are solved, and then the left lane line equation in the laser coordinate system is obtained.

[0090] Similarly, the x-axis coordinate and y-axis coordinate in the coordinate point P3 ’ , P4 ’ can be substituted into the linear equation formula y=kx+b respectively to obtain the right lane line equation in the laser coordinate system.

[0091] S309, after the point cloud data collected by the laser radar is converted through the second transformation matrix, three-dimensional point cloud coordinates in the laser coordinate system are formed.

[0092] In this embodiment, the control system in the track crane can be configured to convert all the point cloud data (three-dimensional coordinate points) collected by the laser radar 7 through the second transformation matrix respectively to form three-dimensional point cloud coordinates in the laser coordinate system, as shown in FIG. 6. The two parallel straight lines in FIG. 6 are the left and right lane lines, which are basically parallel to the x-axis of the laser coordinate system, and the points in FIG. 6 are laser point clouds.

[0093] S310, ground point cloud coordinates are filtered out from the three-dimensional point cloud coordinates in the laser coordinate system.

[0094] Since the obstacles affecting the track crane are all convex ground, the ground laser point cloud scanned by the laser radar 7 can be filtered out first.

[0095] Since the z-axis coordinate value of the ground laser point cloud scanned by the laser radar 7 should be 0 after the second transformation matrix conversion, but considering the scanning error problem, the z-axis coordinate of some ground laser points after the second transformation matrix conversion is not necessarily 0. Therefore, for fault tolerance consideration, the laser point cloud with z-axis coordinate value less than 0.1 in the laser coordinate system (i.e. the laser point formed by the object within 0.1 cm height from the ground) is considered as ground laser point cloud and filtered out.

[0096] S311, from the non-ground point cloud coordinates, the point cloud coordinates whose x and y coordinates are located between the left and right lane lines in the laser coordinate system are selected as obstacle point cloud, and the non-obstacle point cloud is deleted.

[0097] After filtering out the ground laser point cloud, the laser point cloud located outside the left and right lane lines can be filtered out. Since the track crane only travels within the left and right lane lines, even if there are obstacles outside the left and right lane lines, it will not affect the normal travel of the track crane, so it can be directly filtered out and not considered.

[0098] The following method is used in this embodiment to select the point cloud coordinates located between the left and right lane lines in the laser coordinate system:

[0099] First, the x-coordinate value of each non-ground point cloud coordinate is substituted into the left and right lane line equations in the laser coordinate system to calculate two y values;

[0100] Then, it is judged whether the y coordinate value of the non-ground point cloud coordinate is between the two calculated y values. If the y coordinate value of the non-ground point cloud coordinate is between the two calculated y values, it is considered to be an obstacle point cloud and is retained; otherwise, it is considered to be a non-obstacle point cloud outside the left and right lane lines and is directly filtered out.

[0101] S312, it is judged whether the number of obstacle point clouds exceeds a set threshold value. If yes, it is judged that there is an obstacle; otherwise, it is judged that there is no obstacle, and the process returns to S303 to continue scanning.

[0102] After the above processes S310 and S311 of filtering, only obstacle point clouds are retained. If the number of obstacle point clouds is greater than a set threshold value, it is considered that there is an obstacle; otherwise, it is considered that there is no obstacle.

[0103] In some embodiments, the set threshold value can be set to 10. Too small may cause noise to be mistaken for an obstacle, and too large may miss identifying small obstacles. Of course, the set threshold value can also be specifically selected as other values according to actual conditions.

[0104] S313, the minimum x coordinate value in the obstacle point cloud is taken as the distance between the obstacle and the track crane.

[0105] When it is judged that there is an obstacle, since the x axis direction of the laser coordinate system is consistent with the walking direction of the track crane, the minimum x coordinate value in the identified obstacle point cloud can be taken as the distance between the obstacle and the track crane.

[0106] S314, it is judged whether the distance between the obstacle and the track crane is less than a set safety threshold value. If yes, the track crane is controlled to stop; otherwise, the track crane continues to walk.

[0107] When the judgment result indicates that there is an obstacle on the driving lane of the track crane and the distance between the obstacle and the track crane is less than the set safety threshold value, the track crane mechanism is controlled to stop, and the automatic anti-collision function can be realized. After the obstacle is cleaned, the track crane can be controlled to resume operation.

[0108] The role of setting the safety threshold value in this embodiment is to avoid the influence of moving obstacles on the normal driving of the track crane. That is, if the track crane detects that there is an obstacle in front of its driving lane, but the distance between them exceeds the safety threshold value, the track crane can continue to drive, because if the obstacle is a moving obstacle, it may have left the driving lane before the track crane approaches, and will not affect the normal driving of the track crane. In this case, the track crane does not stop due to the presence of the moving obstacle, and the working efficiency of the track crane can be ensured.

[0109] In some embodiments, the safety threshold can be determined according to the running speed of the track crane and the brake time, for example, can be reasonably valued in the range of 5m~10m, as long as the track crane can be completely stopped before reaching the location of the obstacle.

[0110] A corresponding computer program can be written according to the above track crane automatic operation anti-collision method and stored in the control system of the track crane. The computer program is run through the control system to automatically control the track crane to stop when encountering obstacles and realize the automatic anti-collision function.

[0111] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Changes, modifications, additions or substitutions made by ordinary skilled in the art within the essential scope of the present application should also be within the protection scope of the present application.

Claims

1. A rail-mounted automated job collision avoidance method, characterized by, The method comprises: establishing a first transformation matrix for converting the camera coordinate system to the laser coordinate system; establishing a second transformation matrix for converting the point cloud coordinates collected by the laser radar to the point cloud coordinates in the laser coordinate system; capturing a two-dimensional image of a designated area in front of the track-bound vehicle walking direction by using the camera; collecting three-dimensional point cloud data of the designated area in front of the track-bound vehicle walking direction by using the laser radar; extracting pixel points of the left and right lane lines from the two-dimensional image captured by the camera; generating two-dimensional plane equations of the left and right lane lines by using the extracted pixel points of the left and right lane lines; extracting two coordinate points from the two-dimensional plane equations of the left and right lane lines respectively, and converting the two coordinate points by using the first transformation matrix and the second transformation matrix to obtain coordinate points in the laser coordinate system; generating left and right lane line equations in the laser coordinate system by using the coordinate points in the laser coordinate system; converting the point cloud data collected by the laser radar by using the second transformation matrix to form three-dimensional point cloud coordinates in the laser coordinate system; filtering out ground point cloud coordinates from the three-dimensional point cloud coordinates; selecting, from the non-ground point cloud coordinates, point cloud coordinates whose x and y coordinates are located between the left and right lane lines in the laser coordinate system as obstacle point cloud, and deleting non-obstacle point cloud; when the number of obstacle point cloud exceeds a set threshold, it is determined that there is an obstacle; when it is determined that there is an obstacle, the track-bound vehicle stops.

2. The track-bound vehicle automatic operation anti-collision method according to claim 1, wherein: the x-axes of the camera coordinate system and the laser coordinate system are oriented towards the track-bound vehicle walking direction; when it is determined that there is an obstacle, the minimum x-coordinate value in the obstacle point cloud is taken as the distance between the obstacle and the track-bound vehicle; when the distance between the obstacle and the track-bound vehicle is less than a set safety threshold, the track-bound vehicle stops.

3. The railcar automated job collision avoidance method of claim 1 or 2, wherein, The establishment process of the first transformation matrix comprises: using the same calibration board to calibrate the internal parameters of the camera and the laser radar; scanning the calibration board by using the camera and the laser radar, and obtaining the x and y coordinates of a plurality of corner points on the calibration board in the camera coordinate system and the laser coordinate system respectively; establishing the first transformation matrix according to the conversion relationship between the x and y coordinates of the same corner points in the camera coordinate system and the laser coordinate system, so that the coordinate points in the camera coordinate system are converted to the coordinates of the corresponding points in the laser coordinate system after the first transformation matrix conversion, and the z-axis coordinate is ignored.

4. The rail automated job collision avoidance method of claim 1 or 2, wherein, The establishment process of the second transformation matrix comprises: selecting a plurality of point cloud coordinates from the ground laser point cloud data collected by the laser radar, and finding the corresponding points of the plurality of point cloud coordinates in the xoy plane of the laser coordinate system; using the svd decomposition method to construct a linear equation group to obtain the parameters of the transformation matrix, thereby constituting the second transformation matrix.

5. The track-bound vehicle automatic operation anti-collision method according to claim 3, wherein: after extracting two coordinate points from the two-dimensional plane equations of the left and right lane lines respectively and converting by using the first transformation matrix, only converting the x-axis coordinate and the y-axis coordinate by using the second transformation matrix, and assigning 0 to the z-axis coordinate value, to obtain four coordinate points in the laser coordinate system; The x-axis coordinate and the y-axis coordinate in the four coordinate points in the laser coordinate system are used to generate a left lane line equation and a right lane line equation in the laser coordinate system.

6. The rail-mounted crane automatic operation anti-collision method of claim 1 or 2, wherein, a lane line segmentation model is used to segment the two-dimensional image captured by the camera, and pixel points of the left and right lane lines of the lane where the rail-mounted crane is located are extracted; a RANSAC algorithm is used to fit the pixel points of the left and right lane lines respectively, and a two-dimensional plane equation of the left and right lane lines is obtained.

7. The rail automated job collision avoidance method of claim 1 or 2, wherein, The screening process of the obstacle point cloud includes: The x-coordinate value of each non-ground point cloud coordinate is substituted into the left and right lane line equations in the laser coordinate system respectively, and two y values are calculated; if the y-coordinate value of the non-ground point cloud coordinate is between the two calculated y values, it is retained as an obstacle point cloud; otherwise, the point cloud coordinate is filtered out.

8. The rail-mounted crane automatic operation anti-collision method of claim 1 or 2, wherein, three-dimensional point cloud coordinates with a z-coordinate value less than or equal to 0.1 in the three-dimensional point cloud coordinates are filtered out as ground point cloud coordinates; the threshold value is set to 10.

9. A rail-mounted drone comprising a trolley mechanism driving the rail-mounted drone to walk as a whole, a camera and a lidar mounted on the trolley mechanism, a control system in communication with the camera and the lidar; characterized in that, The control system runs a computer program to execute the rail-mounted crane automatic operation anti-collision method of any one of claims 1 to 8.

10. The rail-mounted crane of claim 9, wherein, the rail-mounted crane reciprocates between the left and right lane lines drawn in the field area; a set of cameras and laser radars are respectively installed on the front legs and rear legs of the walking mechanism, and the front legs and rear legs adjust their naming mode according to the change of the walking direction of the rail-mounted crane; according to the walking direction of the rail-mounted crane, a set of cameras and laser radars installed on the front legs are turned on to scan the obstacles in the area 1m-15m in front of the walking direction of the rail-mounted crane.

Images