A kind of rail crane automatic operation anti-collision method and rail crane
Through the combination of camera and lidar, the problems of low accuracy and slow speed of obstacle detection in track cranes are solved, and fast and accurate obstacle detection and automatic collision prevention functions are achieved, ensuring the safe and efficient operation of track cranes.
Patent Information
- Application Number
- CN202411102772.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The existing obstacle detection methods have problems in track cranes with close detection distance, low accuracy or complex algorithms, which affect the detection speed.
Using a combination of camera and lidar, by establishing a transformation matrix from the camera coordinate system to the laser coordinate system, converting two-dimensional images and three-dimensional point cloud data into laser coordinate system, extracting lane line equations, filtering ground point clouds, and filtering obstacles Point clouds, and judge whether there are obstacles based on the number of obstacles, and control track crane parking.
Accurate and fast obstacle detection is achieved, algorithms are simplified, detection speed is improved, and track crane safety and operating efficiency are ensured.
Smart Images

Figure CN118666159B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of obstacle detection, and in particular relates to an automatic operation anti-collision method for a rail crane and a rail crane. Background Art
[0002] Rail mounted crane is a gantry crane, which is mainly used to complete the tasks of lifting, stacking and loading and unloading containers between the yard and the container trucks (referred to as container trucks) outside the yard.
[0003] In traditional container terminals, when loading and unloading containers, the rail crane driver manually controls the rail crane to move to the working position to start loading and unloading after discovering that the container truck has arrived at the working position. During the process of controlling the movement of the rail crane, if there is an obstacle in the driving track area, the driver needs to manually brake the rail crane when the obstacle is found. However, due to the limited field of vision, it is difficult for the driver to quickly determine whether there is an obstacle in the driving area of the rail crane and the exact location of the obstacle, so there is a safety hazard. To address this problem, it is necessary to implement an automatic collision avoidance function on the rail crane.
[0004] At present, the commonly used automatic collision avoidance methods for rail cranes in the industry mainly include infrared ranging, ultrasonic ranging, etc., but these methods have the disadvantages of short detection distance, low accuracy, high false detection rate, etc. In recent years, with the rapid development of laser radar scanning technology and image recognition technology, technologies that apply laser radar and image sensors (such as cameras, etc.) to the field of obstacle detection have emerged continuously, such as the Chinese invention patent application with publication number CN 113985405 A "Obstacle detection method, obstacle detection device applied to vehicles" and the Chinese invention patent application with publication number CN 113568002 A "Rail transportation active obstacle detection device based on laser and image data fusion", both of which determine the target point cloud data based on the point cloud data obtained by the laser radar and the image information collected by the image sensor; then, obstacle recognition is performed on the target point cloud data and image information respectively, and the first obstacle information corresponding to the target point cloud data and the second obstacle information corresponding to the image information are obtained; the target obstacle is determined by performing obstacle fusion of the first obstacle information and the second obstacle information.
[0005] Although this detection method can improve the accuracy of obstacle recognition, the recognition process requires the use of complex machine learning algorithms. For example, for the point cloud data obtained by the laser radar, it is necessary to use efficient clustering algorithms to classify the non-ground point cloud by features, parse the various target objects in the point cloud and extract the contours of the target objects; and for the image information collected by the image sensor, it is necessary to use artificial intelligence convolutional neural network algorithms to detect obstacles in the image and identify the types of obstacles. The use of machine learning algorithms will occupy a lot of processor resources and have a long running time, which will affect the detection speed and is not conducive to emergency obstacle avoidance of rail cranes. Summary of the invention
[0006] The object of the present invention is to provide a rail crane automated operation anti-collision method and a rail crane, so as to solve the problems that the existing obstacle detection methods either have a short detection distance and low accuracy, or have a complex algorithm and affect the detection speed.
[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:
[0008] In one aspect, the present invention provides a method for preventing collisions during automated operation of a rail crane, comprising:
[0009] Establishing a first transformation matrix for converting the camera coordinate system to the laser coordinate system;
[0010] A second transformation matrix is established to convert the point cloud coordinates collected by the laser radar into point cloud coordinates in the laser coordinate system;
[0011] A camera is used to capture a two-dimensional image of a set area in front of the traveling direction of the rail crane;
[0012] Use laser radar to collect three-dimensional point cloud data in the set area in front of the rail crane's travel direction;
[0013] Extracting pixel points of left and right lane lines from the two-dimensional image captured by the camera;
[0014] Generate the two-dimensional plane equations of the left and right lane lines using the extracted pixel points of the left and right lane lines;
[0015] Extract two coordinate points from the two-dimensional plane equations of the left and right lane lines respectively, and obtain the coordinate points in the laser coordinate system after conversion through the first transformation matrix and the second transformation matrix;
[0016] Generate left and right lane line equations in the laser coordinate system using the coordinate points in the laser coordinate system;
[0017] The point cloud data collected by the laser radar is transformed by a second transformation matrix to form three-dimensional point cloud coordinates in a laser coordinate system;
[0018] Filtering out ground point cloud coordinates from the three-dimensional point cloud coordinates;
[0019] Filter out the point cloud coordinates whose x-coordinates and y-coordinates are between the left and right lane lines in the laser coordinate system from the non-ground point cloud coordinates, retain them as obstacle point clouds, and delete the non-obstacle point clouds;
[0020] When the number of obstacle point clouds exceeds the set threshold, it is judged that there is an obstacle;
[0021] When it is determined that there is an obstacle, the rail crane stops.
[0022] In some embodiments of the present application, a lane line segmentation model can be used to segment the two-dimensional image captured by the camera to extract the pixel points of the left and right lane lines of the lane where the rail crane is located; then, the RANSAC algorithm is used to fit the pixel points of the left and right lane lines respectively to obtain the two-dimensional plane equations of the left and right lane lines.
[0023] In some embodiments of the present application, in order to realize the automatic obstacle avoidance function on the rail crane, the x-axis of the camera coordinate system and the laser coordinate system can be first configured to face the travel direction of the rail crane; then, when it is determined that there is an obstacle, the minimum x-coordinate value in the obstacle point cloud is used as the distance between the obstacle and the rail crane; when the distance between the obstacle and the rail crane is less than the set safety threshold, the rail crane is controlled to stop. Setting the safety threshold here can avoid the impact of moving obstacles on the normal driving of the rail crane, avoid unnecessary parking of the rail crane, and affect the operation efficiency.
[0024] In some embodiments of the present application, in order to achieve accurate conversion from the camera coordinate system to the laser coordinate system, the process of establishing the first transformation matrix may be configured to include:
[0025] Use the same calibration board to perform intrinsic calibration on the camera and lidar;
[0026] Scanning the calibration plate using the camera and the laser radar, and obtaining x-coordinates and y-coordinates of multiple corner points on the calibration plate in the camera coordinate system and the laser coordinate system respectively;
[0027] According to the conversion relationship between the x-coordinate and y-coordinate of the same corner point in the camera coordinate system and the laser coordinate system, a first transformation matrix is established so that the coordinate point in the camera coordinate system becomes the corresponding coordinate point in the laser coordinate system after being transformed by the first transformation matrix, and the z-axis coordinate is ignored.
[0028] In some embodiments of the present application, in order to convert the point cloud coordinates collected by the laser radar into the established laser coordinate system, the process of establishing the second transformation matrix may be configured to include:
[0029] Select multiple point cloud coordinates from the ground laser point cloud data collected by the laser radar, and find corresponding points of the multiple point cloud coordinates on the xoy plane of the laser coordinate system;
[0030] Use the SVD decomposition method to construct a system of linear equations, find the parameters of the transformation matrix, and form the second transformation matrix.
[0031] In some embodiments of the present application, since the lane lines are formed on the ground and do not have z-axis coordinates, after two coordinate points are respectively extracted from the two-dimensional plane equations of the left and right lane lines and converted through the first transformation matrix, only the x-axis coordinates and the y-axis coordinates can be converted using the second transformation matrix, and the z-axis coordinate can be directly assigned a value of 0, thereby obtaining four coordinate points in the laser coordinate system; using the x-axis coordinates and the y-axis coordinates of the four coordinate points in the laser coordinate system, the left lane line equation and the right lane line equation in the laser coordinate system can be generated to distinguish obstacles located between the left and right lane lines that affect the normal movement of the rail crane, and obstacles located outside the left and right lane lines do not affect the normal movement of the rail crane and can be ignored.
[0032] In some embodiments of the present application, the obstacle point cloud screening process may be configured to include:
[0033] After the point cloud data collected by the laser radar is converted by the second transformation matrix, the three-dimensional point cloud coordinates in the laser coordinate system are formed, and the three-dimensional point cloud coordinates with z coordinate values less than or equal to 0.1 are selected as the ground point cloud coordinates to be filtered out first; then, the x coordinate value of each non-ground point cloud coordinate is substituted into the left and right lane line equations in the laser coordinate system to calculate two y values;
[0034] If the y-coordinate value of the non-ground point cloud coordinate is between the two calculated y-values, it means that the point cloud is located between the left and right lane lines and should be retained as an obstacle point cloud; otherwise, the point cloud coordinate is filtered out and not considered.
[0035] In some embodiments of the present application, a threshold value can be pre-set for the retained obstacle point cloud to determine whether it is an obstacle. When the number of obstacle point clouds is greater than the set threshold value of 10, it indicates that it is an obstacle, otherwise, it indicates that it is not an obstacle. In order to improve the accuracy of the obstacle judgment result, it is necessary to give special consideration to the specific value of the set threshold value. If the set threshold value is set too small, noise points may be mistaken for obstacles; if the set threshold value is set too large, small obstacles may be missed. For this reason, the present application sets the set threshold value to 10 to achieve accurate identification of obstacles.
[0036] In another aspect, the present invention also proposes a rail crane, including a trolley mechanism that drives the rail crane to move as a whole, a camera and a laser radar installed on the trolley mechanism, and a control system that communicates with the camera and the laser radar; the control system is configured to run a computer program to execute the above-mentioned rail crane automated operation anti-collision method.
[0037] In some embodiments of the present application, a rail crane is configured to travel back and forth between left and right lane lines designated within the site; in order to effectively collect obstacles in the area in front of the rail crane while it is traveling in both directions, a group of cameras and laser radars can be installed on the front legs and rear legs of the trolley mechanism at an angle downward, respectively, and the naming method of the front legs and rear legs should be adjusted according to the change in the traveling direction of the rail crane; then, a group of cameras and laser radars installed on the front legs are turned on according to the traveling direction of the rail crane, and can scan obstacles in the area 1m to 15m in front of the traveling direction of the rail crane, so as to leave enough time for the rail crane to decelerate and stop.
[0038] Compared with the prior art, the advantages and positive effects of the present invention are mainly reflected in:
[0039] 1. The present invention uses camera imaging and image recognition technology to obtain the coordinates of the left and right lane lines in the track crane's travel area, and maps them to the laser coordinate system to construct the left and right lane line equations in the laser coordinate system. The laser radar scanning technology is used to collect the point cloud coordinates of the area in front of the track crane's travel direction, and after filtering out the ground point cloud coordinates and the point cloud coordinates outside the left and right lane lines, the retained point cloud coordinates are the obstacle point clouds. According to the number of obstacle point clouds, it can be judged whether there are obstacles in front of the track crane that affect its normal travel, and the judgment result is accurate and reliable.
[0040] 2. The obstacle recognition method of the present invention is simple and effective, does not require the use of complex machine learning algorithms, is easy to program, runs quickly, and can output judgment results instantly, leaving enough reaction time for the rail crane to avoid obstacles and stop.
[0041] 3. The present invention can accurately determine the distance between the rail crane and the obstacle based on the coordinate value of the obstacle point cloud. By setting a safety threshold, the influence of the moving obstacles on the normal operation of the rail crane can be avoided. While ensuring the safe operation of the rail crane, the crane will not stop ineffectively due to the presence of moving obstacles, thereby ensuring the operating efficiency of the rail crane.
[0042] 4. The obstacle detection and anti-collision parking functions of the present invention can be automatically completed by the rail crane without the need for human involvement, with a high degree of automation, thus ensuring the safety of the rail crane operation.
[0043] Other features and advantages of the present invention will become more apparent after reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a structural schematic diagram of an embodiment of a rail crane;
[0045] Figure 2 It is a structural schematic diagram of an embodiment of the installation position and installation method of a camera and a laser radar on a rail crane;
[0046] Figure 3 It is a flow chart of an embodiment of the anti-collision method for automated operation of a rail crane proposed by the present invention;
[0047] Figure 4 This is an example of extracting the pixels of the left and right lane lines from the two-dimensional image captured by the camera;
[0048] Figure 5 is to use Figure 4 The example diagram of the left and right lane lines shown is generated by fitting the pixel points of the left and right lane lines in the camera coordinate system;
[0049] Figure 6 It is the distribution map of the laser point cloud collected by the laser radar in the laser coordinate system. DETAILED DESCRIPTION
[0050] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.
[0051] Current rail cranes, such as Figure 1 As shown, it includes main components such as a trolley mechanism 1, a trolley mechanism 2, a hoist 3, a power house 4, and a driver's cab 5. Among them, the trolley mechanism 1 is used to drive the overall movement of the rail crane, including two groups of front and rear legs 11 and 12. Each group of legs 11 / 12 includes two left and right columns. The tops of the two columns are connected by a crossbeam 13 / 14 to form a portal structure. The trolley mechanism 2 is installed between the crossbeams 13 and 14 of the two gantries and moves along the direction of the crossbeams. The hoist 3 is installed on the trolley mechanism and is used to grab and place containers. The power house 4 can be installed on the bottom beam 15 connecting the front and rear groups of legs 11 and 12 to provide energy for the power system on the rail crane. The driver's cab 5 can be installed on the trolley mechanism 2, and is equipped with an operating table for the driver to operate the rail crane to perform work tasks.
[0052] In order to realize the automatic collision avoidance function on the rail crane, this embodiment uses a camera in conjunction with a laser radar to collect and identify obstacles in the area in front of the rail crane.
[0053] When the rail crane is operating in the container yard, it moves back and forth along the left and right lanes marked on the ground of the yard. Therefore, if the front legs and the rear legs 11 and 12 are defined according to the moving direction of the rail crane, the naming of the two legs 11 and 12 needs to be adjusted accordingly as the moving direction of the rail crane changes. For example, if the rail crane is moving eastward, the legs in the east direction are the front legs, and the legs in the west direction are the rear legs; conversely, if the rail crane is moving westward, the legs in the west direction are the front legs, and the legs in the east direction are the rear legs.
[0054] In order to accurately detect obstacles in the area in front of the rail crane after adjusting the traveling direction, in this embodiment, a set of cameras 6 and a laser radar 7 are respectively installed on the two sets of legs 11 and 12 of the trolley mechanism 1 of the rail crane. Figure 2 The position and angle relationship of the camera 6 and the laser radar 7 installed on one of the legs 11 / 12 are shown. The camera 6 and the laser radar 7 are preferably installed on the legs 11 and 12 of the trolley mechanism 1 at a slight downward tilt to ensure that the area at least 1m to 15m in front of the traveling direction of the rail crane is the scanning area.
[0055] In some embodiments, the camera 6 may be a 2D camera, and the laser radar 7 may be a 3D laser radar. The configuration track crane automatically starts the camera 6 and the laser radar 7 installed on the front legs 11 / 12 according to its driving direction to collect the positions of the left and right lane lines 8, 9 and obstacles 10 in the scanning area.
[0056] Combine the following Figure 3 , the specific work flow of the rail crane automated operation anti-collision method of this embodiment is elaborated in detail.
[0057] S301, establishing a first transformation matrix for converting coordinate points in a camera coordinate system to a laser coordinate system.
[0058] Since the coordinate system of the pixel point coordinates collected by the camera 6 is different from the coordinate system of the point cloud coordinates collected by the laser radar 7, it is necessary to predetermine the conversion relationship between the coordinate systems so as to correspond the coordinate points in different coordinate systems to the same coordinate system to facilitate the identification of obstacles.
[0059] To this end, this embodiment first establishes a first transformation matrix for converting the camera coordinate system to the laser coordinate system. The specific process is as follows:
[0060] (1) Use the same calibration board to perform intrinsic calibration on the camera 6 and the lidar 7.
[0061] The method of using a calibration plate to calibrate the internal parameters of a camera and a lidar is a prior art and will not be described in detail in this embodiment.
[0062] (2) Use the calibration plate to calibrate the camera coordinate system and the laser coordinate system at the same point.
[0063] In this embodiment, the calibration plate can be scanned by using the camera 6 and the laser radar 7, and the coordinates of the same corner point on the calibration plate in the camera coordinate system and the laser coordinate system are obtained, which are the x coordinate and the y coordinate. The coordinates of multiple corner points, preferably more than 9 corner points, are collected for calculating the first transformation matrix.
[0064] (3) Based on the coordinates of the same point in the camera coordinate system and the laser coordinate system, the transformation relationship matrix is calculated to form the first transformation matrix.
[0065] In this embodiment, a first transformation matrix can be established based on the conversion relationship between the x-coordinate and the y-coordinate of the same corner point on the calibration plate in the camera coordinate system and the laser coordinate system, so that the coordinate point in the camera coordinate system becomes the coordinate of the corresponding point in the laser coordinate system after being converted by the first transformation matrix. Since the camera coordinate point is two-dimensional and the laser coordinate point is three-dimensional, there is no need to consider the z-axis coordinate during the conversion process.
[0066] In this embodiment, the origin of the camera coordinate system and the laser coordinate system can be set at the lower left corner of the scanning area, such as Figure 5 As shown, the x-axis is forward, the y-axis is to the right, and the z-axis of the laser coordinate system is perpendicular to the ground and upward.
[0067] S302: Establish a second transformation matrix for converting the point cloud coordinates collected by the laser radar into point cloud coordinates in the laser coordinate system.
[0068] The ground laser point cloud is a collection of reflection points formed by the laser irradiating the ground, and exists in the form of a plane. Since the laser radar 7 of this embodiment is installed on the front leg of the rail crane trolley mechanism at an angle downward, its scanning direction is not vertically downward. The plane formed by the collected ground laser point cloud will form a certain angle with the xoy plane of the laser coordinate system. Therefore, it is necessary to establish a second transformation matrix to convert the point cloud coordinates collected by the laser radar into point cloud coordinates in the laser coordinate system. The specific process is as follows:
[0069] (1) Select multiple point cloud coordinates from the ground laser point cloud data collected by the laser radar, and find the corresponding points of the multiple point cloud coordinates on the xoy plane of the laser coordinate system.
[0070] In this embodiment, six laser points may be selected from the ground laser point cloud data, and corresponding points of the six laser points on the xoy plane of the laser coordinate system may be found respectively, and then six sets of corresponding coordinate values may be obtained.
[0071] (2) Use the SVD decomposition method to construct a system of linear equations, find the parameters of the transformation matrix, and form the second transformation matrix.
[0072] Using the acquired multiple sets of corresponding coordinate values, such as the six sets of corresponding coordinate values, the SVD decomposition method is used to construct a set of linear equations to obtain the parameters of the transformation matrix and form a second transformation matrix so that the plane formed by the ground laser point cloud collected by the laser radar 7 coincides with the Xoy plane of the laser coordinate system after being converted by the second transformation matrix.
[0073] The SVD decomposition method is a prior art and will not be described in detail in this embodiment.
[0074] S303, using a camera to capture a two-dimensional image of a set area in front of the travel direction of the rail crane.
[0075] The established first transformation matrix and the second transformation matrix are stored in the control system of the rail crane so as to be automatically called by the rail crane during operation.
[0076] During normal operation of the rail crane, the control system in the rail crane can be configured to automatically start the camera 6 and the laser radar 7 installed on the front legs of the trolley mechanism according to the travel direction of the rail crane, and scan the area 1m to 15m ahead of the travel direction of the rail crane.
[0077] The camera 6 can collect a two-dimensional image of the front scanning area in the travel direction of the rail crane.
[0078] S304, using laser radar to collect three-dimensional point cloud data in a set area in front of the travel direction of the rail crane.
[0079] The laser radar 7 can collect three-dimensional point cloud data in the front scanning area in the direction of travel of the rail crane.
[0080] S305 , extracting pixel points of the left and right lane lines from the two-dimensional image captured by the camera.
[0081] In this embodiment, the existing lane line segmentation model can be used to segment the two-dimensional image captured by the camera 6 to extract the pixel points of the left and right lane lines of the lane where the rail crane is located.
[0082] like Figure 4 As shown, the extracted pixel points will be distributed in the form of two straight lines. If the middle of the two-dimensional image is used as the dividing line, the pixels on the left side of the middle dividing line are the left lane pixels, and the pixels on the right side of the middle dividing line are the right lane pixels.
[0083] S306: Generate two-dimensional plane equations of the left and right lane lines using the extracted pixel points of the left and right lane lines.
[0084] In this embodiment, the existing RANSAC algorithm can be used to fit the extracted pixel points of the left and right lane lines respectively to obtain the two-dimensional plane equation y=k left x+b left and the two-dimensional plane equation of the right lane line y=k right x+b right ,like Figure 5 shown.
[0085] S307 , extract two coordinate points from the two-dimensional plane equations of the left and right lane lines respectively, and obtain the coordinate points in the laser coordinate system after conversion through the first transformation matrix and the second transformation matrix.
[0086] In order to correspond the left and right lane lines in the camera coordinate system to the laser coordinate system, this embodiment first extracts two coordinate points P1 (x1 camera ,y1 camera )、P2(x2 camera , y2 camera ), and then extract two coordinate points P3(x3 camera ,y3 camera )、P4(x4 camera ,y4 camera ).
[0087] After the four coordinate points P1, P2, P3, and P4 are transformed by the first transformation matrix, the x-axis coordinates and y-axis coordinates (because there is no z-axis coordinate) are transformed by the second transformation matrix. Since the left and right lane lines are formed on the ground, the ground laser point cloud is formed in the laser coordinate system, so the z-axis coordinate can be assigned 0, thus obtaining the four three-dimensional coordinate points P1 in the laser coordinate system. ’ (x1 laser ,y1 laser ,0),P2 ’ (x2 laser , y2 laser ,0) 、P3 ’ (x3 laser ,y3 laser ,0) 、P4 ’ (x4 laser ,y4 laser ,0).
[0088] S308. Generate left and right lane line equations in the laser coordinate system using the coordinate points in the laser coordinate system.
[0089] Since the left and right lane lines are also straight lines in the laser coordinate system, the coordinate point P1 can be ’ 、P2’ Substitute the x-axis coordinate and y-axis coordinate into the straight line equation y=kx+b, solve for the constants k and b, and then obtain the left lane line equation in the laser coordinate system.
[0090] Similarly, we can use the coordinate point P3 ’ 、P4 ’ Substitute the x-axis coordinate and y-axis coordinate into the straight line equation y=kx+b to obtain the right lane line equation in the laser coordinate system.
[0091] S309: The point cloud data collected by the laser radar is transformed by a second transformation matrix to form three-dimensional point cloud coordinates in a laser coordinate system.
[0092] In this embodiment, the control system in the rail crane can be configured to transform all point cloud data (three-dimensional coordinate points) collected by the laser radar 7 through the second transformation matrix to form three-dimensional point cloud coordinates in the laser coordinate system, such as Figure 6 shown. Figure 6 The two parallel straight lines in are the left and right lane lines, which are basically parallel to the x-axis of the laser coordinate system. Figure 6 The points in the image are laser point clouds.
[0093] S310 , filtering out the ground point cloud coordinates from the three-dimensional point cloud coordinates in the laser coordinate system.
[0094] Since the obstacles that affect the movement of the rail crane are all protruding from the ground, the ground laser point cloud scanned by the laser radar 7 can be filtered out first.
[0095] Since the z-axis coordinate value of the ground laser point cloud scanned by the laser radar 7 should be 0 after being transformed by the second transformation matrix, but considering the problem of scanning error, the z-axis coordinate value of some ground laser points after being transformed by the second transformation matrix may not be 0. Therefore, for fault tolerance considerations, laser point clouds with z-axis coordinate values less than 0.1 in the laser coordinate system (i.e., laser points formed by objects within 0.1 cm height from the ground) can be considered as ground laser point clouds and filtered out.
[0096] S311. Filter out the point cloud coordinates whose x-coordinate and y-coordinate are between the left and right lane lines in the laser coordinate system from the non-ground point cloud coordinates, retain them as obstacle point clouds, and delete the non-obstacle point clouds.
[0097] After filtering out the ground laser point cloud, the laser point cloud outside the left and right lane lines can be filtered out. Since the rail crane only moves within the left and right lane lines, even if there are obstacles outside the left and right lane lines, it will not affect the normal movement of the rail crane, so it can be directly filtered out and not considered.
[0098] This embodiment uses the following method to select the point cloud coordinates between the left and right lane lines in the laser coordinate system:
[0099] First, substitute the x-coordinate value of each non-ground point cloud coordinate into the left and right lane line equations in the laser coordinate system to calculate two y-values;
[0100] Then, it is determined whether the y coordinate value of the non-ground point cloud coordinates is between the two calculated y values. If the y coordinate value of the non-ground point cloud coordinates is between the two calculated y values, it is considered to be an obstacle point cloud and is retained; otherwise, it is considered to be a non-obstacle point cloud located outside the left and right lane lines and is directly filtered out.
[0101] S312: Determine whether the number of obstacle point clouds exceeds a set threshold. If so, determine that there is an obstacle; otherwise, determine that there is no obstacle and return to process S303 to continue scanning.
[0102] After the above-mentioned filtering process S310 and S311, only the obstacle point cloud is retained. If the number of obstacle point clouds is greater than the set threshold, it is considered that there is an obstacle; otherwise, it is considered that there is no obstacle.
[0103] In some embodiments, the threshold value may be set to 10. If it is too small, noise points may be mistaken for obstacles, and if it is too large, small obstacles may be missed. Of course, the threshold value may also be set to other values according to actual conditions.
[0104] S313. The minimum x-coordinate value in the obstacle point cloud is used as the distance between the obstacle and the rail crane.
[0105] When determining that there is an obstacle, since the x-axis direction of the laser coordinate system is consistent with the travel direction of the rail crane, the minimum x-coordinate value in the identified obstacle point cloud can be used as the distance between the obstacle and the rail crane.
[0106] S314, determining whether the distance between the obstacle and the rail crane is less than a set safety threshold, if so, controlling the rail crane to stop; otherwise, maintaining the rail crane to continue moving.
[0107] When the judgment result indicates that there is an obstacle on the driving lane of the rail crane and the distance between the obstacle and the rail crane is less than the set safety threshold, the trolley mechanism of the rail crane is controlled to stop, which can realize the automatic collision avoidance function. After the obstacle is cleared, the rail crane can be controlled to resume operation.
[0108] The purpose of setting the safety threshold in this embodiment is to avoid the impact of moving obstacles on the normal driving of the rail crane. That is, if the rail crane detects that there is an obstacle in front of its driving lane, but the distance between the two exceeds the safety threshold, the rail crane can continue to drive, because if the obstacle is a moving obstacle, the moving obstacle may have left the driving lane before the rail crane approaches, and will not affect the normal driving of the rail crane. In this case, the rail crane does not stop due to the presence of a moving obstacle, which can ensure the operating efficiency of the rail crane.
[0109] In some embodiments, the safety threshold can be determined comprehensively based on the travel speed and braking time of the rail crane. For example, a reasonable value can be taken within the range of 5m to 10m, as long as the rail crane can be completely stopped before reaching the location of the obstacle.
[0110] A corresponding computer program can be written based on the above-mentioned rail crane automated operation anti-collision method and stored in the control system of the rail crane. The computer program can be run through the control system to automatically control the rail crane to stop when encountering obstacles, thereby realizing the automatic anti-collision function.
[0111] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.
Claims
1. A method for preventing collisions during automated operation of a rail crane, characterized in that: include: The first transformation matrix is established for converting the camera coordinate system to the laser coordinate system, which includes: Using the same calibration plate to perform internal calibration on the camera and the laser radar; Scanning the calibration plate using the camera and the laser radar, and obtaining x-coordinates and y-coordinates of multiple corner points on the calibration plate in the camera coordinate system and the laser coordinate system respectively; According to the conversion relationship between the x-coordinate and the y-coordinate of the same corner point in the camera coordinate system and the laser coordinate system, a first transformation matrix is established, so that the coordinate point in the camera coordinate system becomes the coordinate of the corresponding point in the laser coordinate system after being transformed by the first transformation matrix, and the z-axis coordinate is ignored; A second transformation matrix is established to convert the point cloud coordinates collected by the laser radar into point cloud coordinates in the laser coordinate system, which includes: Select multiple point cloud coordinates from the ground laser point cloud data collected by the laser radar, and find corresponding points of the multiple point cloud coordinates on the xoy plane of the laser coordinate system; Use the SVD decomposition method to construct a system of linear equations, find the parameters of the transformation matrix, and form the second transformation matrix; A camera is used to capture a two-dimensional image of a set area in front of the traveling direction of the rail crane; Use laser radar to collect three-dimensional point cloud data in the set area in front of the rail crane's travel direction; Extracting pixel points of left and right lane lines from the two-dimensional image captured by the camera; Generate the two-dimensional plane equations of the left and right lane lines using the extracted pixel points of the left and right lane lines; Two coordinate points are extracted from the two-dimensional plane equations of the left and right lane lines respectively and transformed by the first transformation matrix, and then only the x-axis coordinate and the y-axis coordinate are transformed by the second transformation matrix, and the z-axis coordinate value is assigned to 0, so as to obtain four coordinate points in the laser coordinate system; Generate a left lane line equation and a right lane line equation in the laser coordinate system using the x-axis coordinates and the y-axis coordinates of the four coordinate points in the laser coordinate system; The point cloud data collected by the laser radar is transformed by a second transformation matrix to form three-dimensional point cloud coordinates in a laser coordinate system; Filtering out ground point cloud coordinates from the three-dimensional point cloud coordinates; Substitute the x-coordinate value of each non-ground point cloud coordinate into the left and right lane line equations in the laser coordinate system to calculate two y-values; If the y coordinate value of the non-ground point cloud coordinate is between the two calculated y values, it is retained as an obstacle point cloud; otherwise, the point cloud coordinate is filtered out; When the number of obstacle point clouds exceeds the set threshold, it is judged that there is an obstacle; When it is determined that there is an obstacle, the rail crane stops.
2. The anti-collision method for automated operation of a rail crane according to claim 1, characterized in that: The x-axes of the camera coordinate system and the laser coordinate system are oriented toward the travel direction of the rail crane; When it is determined that there is an obstacle, the minimum x-coordinate value in the obstacle point cloud is used as the distance between the obstacle and the track crane; When the distance between the obstacle and the rail crane is less than the set safety threshold, the rail crane stops.
3. The anti-collision method for automated operation of a rail crane according to claim 1 or 2, characterized in that: The lane line segmentation model is used to segment the two-dimensional image captured by the camera, and the pixel points of the left and right lane lines of the lane where the rail crane is located are extracted; The RANSAC algorithm is used to fit the pixel points of the left and right lane lines respectively to obtain the two-dimensional plane equations of the left and right lane lines.
4. The anti-collision method for automated operation of a rail crane according to claim 1 or 2, characterized in that: Filter out the three-dimensional point cloud coordinates whose z coordinate values are less than or equal to 0.1 as ground point cloud coordinates; The set threshold is 10.
5. A rail crane, comprising a trolley mechanism for driving the rail crane as a whole, a camera and a laser radar installed on the trolley mechanism, and a control system communicating with the camera and the laser radar; characterized in that: The control system runs a computer program to execute the rail crane automated operation collision avoidance method according to any one of claims 1 to 4.
6. The rail crane according to claim 5, characterized in that: The rail crane moves back and forth between the left and right lane lines delineated within the site; A group of cameras and laser radars are respectively installed on the front legs and rear legs of the trolley mechanism in an inclined downward direction, and the naming of the front legs and rear legs is adjusted according to the change of the traveling direction of the rail crane; According to the travel direction of the rail crane, a group of cameras and lidars installed on the front legs are turned on to scan obstacles within the area 1m to 15m in front of the travel direction of the rail crane.
Citation Information
Patent Citations
Rail transit active obstacle detection device based on laser and image data fusion
CN113568002A
Obstacle detection method and obstacle detection equipment applied to vehicle
CN113985405A
Obstacle detection method and device for automatic driving scene of port
CN110764108A
Method and system for detecting spatial obstacle and engineering machine
CN110950250A
Obstacle detection method, vehicle, equipment and computer storage medium
CN113536883A
