A port tire crane lane keeping control method and system

Abstract

The present invention belongs to the field of port machinery automation technology and image recognition, and relates to a control method and system, including: S1: an image acquisition module acquires a driving lane image of a port crane; S2: an image recognition processing module processes the acquired image and obtains a processed lane image; S3: the image recognition processing module identifies the lane line in the processed lane image and calculates the error to obtain an error result; S4: the image recognition processing module sends the error result to a programmable logic controller for processing. Beneficial effects: the automated transformation of the tire crane is realized, the automated control of the trolley operation is realized, the correction performance and effect of the tire crane are optimized, the transformation cost is significantly reduced, and the promotion of the lane keeping technology of the port tire crane is promoted.

Description

Technical Field

[0001] The invention belongs to the field of port machinery automation technology and image recognition, and in particular relates to a control method and system. Background Art

[0002] At present, the automatic control technology of tire crane trolley travel is constantly emerging and developing. The operation mode of automated container terminals has put forward higher requirements for system planning, scientific layout, and efficient safety. The importance of safe and intelligent equipment and automation systems is increasing day by day. The operation of tire cranes is relatively flexible and can operate across the yard. Therefore, tire cranes are widely used in container terminals at home and abroad. At present, tire cranes can generally meet the production requirements of "stacking five and passing six" for containers, and their trolleys have a faster running speed, generally up to 110 m / min. Since there is no fixed track, tire cranes are flexible in turning and have a large range of activities, but at the same time, they also put forward higher requirements on the operating driver.

[0003] Current problems in this field: During long-distance truck operation, the driver must concentrate on keeping the truck safe. As tire cranes become faster and larger, it becomes more challenging for the driver to keep the truck in the lane. In addition, tire cranes also need to undergo a series of automation modifications. It has become more urgent to achieve automated control of the truck's movement. Current mainstream lane keeping methods are not effective in correcting tire crane deviations, have high requirements on environmental factors, and have high modification costs, all of which hinder the promotion of lane keeping technology for port tire cranes. Summary of the invention

[0004] The present invention provides a port tire crane lane keeping control method and system, aiming to solve the above-mentioned technical problems.

[0005] A port tire crane lane keeping control method is characterized in that it includes the following steps: S1: an image acquisition module acquires a driving lane image of a port crane; S2: an image recognition processing module processes the acquired image and obtains a processed lane image; S3: the image recognition processing module identifies the lane lines in the processed lane image and calculates the error to obtain an error result; S4: the image recognition processing module sends the error result to a programmable logic controller for processing.

[0006] Preferably, step S1 includes: S11: collecting parameters of the image and calculating a transformation matrix of the center position of a wheel of the port crane; S12: calibrating the parameters and performing a coordinate transformation on the transformation matrix to obtain a coordinate transformed transformation matrix; S13: processing the image.

[0007] Preferably, step S2 includes: S21: dividing the image into a region of interest; S22: processing the region of interest; S23: searching for the real-time operating position of the port crane on the craneway; S24: judging whether step S23 is completed: if so, executing step S25; if not, refitting the lane line of the craneway, and then repeating step S23; S25: outputting the fitting equation of the lane line.

[0008] Preferably, step S3 includes: S31: calculating the position of the center line of the lane line; S32: performing fitting calculation on a position signal and obtaining a center line fitting equation; S33: performing a bird's-eye view transformation based on the center line fitting equation; S34: calculating the error of the port crane based on the result of the bird's-eye view transformation and obtaining an error result.

[0009] Preferably, step S4 includes: S41: transmitting the error result to the programmable logic controller; S42: calculating the control time of the port crane operation through a built-in lane keeping algorithm, and controlling a tire crane of the port crane to correct the on-off time of a relay; S43: performing a correction action on the port crane by calling a control program of the programmable logic controller, and judging whether an error value recognized by an image system is less than a trigger value of the correction action: if so, the correction action stops; if not, the correction action continues.

[0010] Preferably, in step S4, when the error result received by the programmable logic controller is greater than a warning value, the tire crane is controlled to brake the relay and a motor relay, and the tire crane stops moving and issues an alarm.

[0011] Preferably, in step S4, when the distance between the port crane and the obstacle is less than a preset value, the programmable logic controller sends an alarm to a cab and the port crane stops running.

[0012] Furthermore, a port tire crane lane keeping control system is used for the port tire crane lane keeping control method, comprising: an image acquisition module (1), the image acquisition module (1) being used to acquire a driving lane image of the tire crane; an image recognition processing module (2), the image recognition processing module (2) being connected to the image acquisition module (1), processing the distortion of the acquired image and obtaining the processed lane image, identifying the lane line in the corrected lane image and calculating the error; and a driving control module (3), the driving control module (3) being connected to the image recognition processing module (2), calculating a correction value and an alarm threshold, and generating a driving control signal.

[0013] Furthermore, the image recognition processing module (2) performs a noise reduction process on the lane image and intercepts the region of interest, calculates the error of the lane line to obtain the error result, and transmits the error result to the programmable logic controller.

[0014] Furthermore, the driving control module (3) comprises the programmable logic controller, the ultrasonic radar and at least one relay, and the programmable logic controller is used to receive data from the ultrasonic radar and the image recognition processing module (2), and to control the on and off of the relay through the driving control signal, thereby controlling the operating posture of the port crane.

[0015] Beneficial effects: The lane keeping system provided by the present invention adopts an external connection method, which basically does not change the circuit structure of the original vehicle system, greatly reduces the modification time and cost, avoids the impact on the stability of the original vehicle system, and realizes the automated modification of the tire crane. Through a control method provided by the present invention, the automation of the control of the trolley operation is realized, the correction performance and effect of the tire crane are optimized, and the promotion of the lane keeping technology of the port tire crane is promoted. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained according to the structures shown in these drawings without creative work.

[0017] Figure 1 It is a schematic diagram of the overall flow of a method for realizing automatic control of a port craneway according to the present invention;

[0018] Figure 2 A schematic diagram of a calibration process of an image collected by a camera of a port crane track used in the present invention;

[0019] Figure 3 It is a flowchart of a camera calibration in step S1 of a method for realizing automatic control of a port crane track of the present invention;

[0020] Figure 4 It is a schematic diagram of a camera calibration and correction process in step S2 in a method for realizing automatic control of a port craneway of the present invention;

[0021] Figure 5 It is a flow chart of image processing and lane line detection of a port lane in step S3 of a method for realizing automatic control of a port crane lane of the present invention;

[0022] Figure 6 It is a flow chart of the center line fitting and correction calculation of a lane in step S4 in a method for realizing the automatic control of a port crane lane of the present invention;

[0023] Figure 7 It is a schematic diagram of the motion state control of the tire crane in an embodiment of the present invention;

[0024] Figure 8 A schematic diagram of a programmable logic controller (PLC) lane keeping control process applied to the present invention;

[0025] Fig. 9 The figure is a schematic diagram of the hardware structure of an automatic control system for a port craneway according to the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0029] like Figure 1 As shown, the overall process of realizing automatic control of a port crane lane of the present invention includes: S1: an image acquisition module acquires a driving lane image of a port crane; S2: an image recognition processing module processes the acquired image and obtains a processed lane image; S3: the image recognition processing module recognizes the lane lines in the processed lane image and calculates the error to obtain an error result; S4: the image recognition processing module sends the error result to a programmable logic controller for processing.

[0030] like Figure 2As shown, in this embodiment, a calibration process of an image collected by a camera of a port crane track adopted by the present invention includes: S101: making a 1m*1m (length and width are 1m) black and white chessboard; S102: photographing the black and white chessboard from different angles by a camera to obtain a chessboard image; S103: detecting feature points of the chessboard image; S104: calculating an internal and external parameter of the camera based on these feature points; S105: further calculating a radial distortion parameter based on the internal and external parameter; S106: combining the internal and external parameters and the distortion parameter obtained in steps S104 and S105 to improve the estimation accuracy; S107: based on the estimation accuracy, further calculating more accurate internal and external parameters and distortion parameters.

[0031] Specifically, according to the above situation description, the present invention uses a parameter calibration method. This method refers to the hand-eye calibration method in robot arm control, and the entire system includes the following four coordinate systems: base coordinate system B, vehicle coordinate system R, camera coordinate system C and calibration object coordinate system O. For the calibration process, the positional relationship between the vehicle coordinate system R and the base coordinate system B can be calculated by the encoder of the cart itself. The positional relationship between the camera coordinate system C and the robot coordinate system R is fixed. The positional relationship between the calibration object coordinate system O and the camera coordinate system C can be obtained by the chessboard calibration method. The positional relationship between the calibration object coordinate system O and the base coordinate system B is fixed, so by moving the position of the vehicle multiple times, the positional relationship between C and R can be obtained based on the known positional relationship between R and B and the obtainable positional relationship between O and C.

[0032] like Figure 3 As shown, as a preference, in this embodiment, a camera calibration process of step S1 includes: S11: collecting an internal and external parameter and a distortion parameter of the camera, and calculating a transformation matrix of the center position of the camera and a wheel of the port crane; S12: calibrating the parameters obtained from the camera, and performing a coordinate transformation on the transformation matrix; S13: performing a coordinate calibration on the image;

[0033] S14: Calculate an angle error and a position error of the port crane.

[0034] like Figure 4 As shown, as a preferred embodiment, a camera calibration and correction process in step S2 includes: S21: dividing the image into a region of interest; S22: performing a color segmentation and a morphological segmentation on the obtained region of interest; S23: searching for the real-time operating position of the port crane on the craneway; S24: judging whether step S23 is completed: if so, executing step S25; if not, refitting the lane line of the craneway, and then repeating step S23; S25: outputting the fitting equation of the lane line.

[0035] Specifically, the region of interest (ROI) calculated in step S21 is used to eliminate the influence of the background information of the surrounding environment of the port crane on subsequent detection. In S22, an HSV color detection method (Hue, Saturation, Value) is used to detect color features, and the Sobel operator (first-order gradient operator) is used to detect the direction of straight lines in the image. Then, the pixels that meet the two feature threshold conditions are segmented.

[0036] like Figure 5 As shown, as a preferred embodiment, a process of image processing and lane line detection of a port lane in step S3 includes: S31: calculating the position of the center line of the lane line; S32: performing fitting calculation on the obtained position signal and deriving a center line fitting equation; S33: performing a bird's-eye view transformation based on the obtained center line fitting equation; S34: calculating the angle error and position error of the port crane based on the result of the obtained bird's-eye view transformation.

[0037] Specifically, the segmentation map is accumulated and calculated in the horizontal direction, and the calculation results are analyzed by histogram statistics. The center position of the image is used as the dividing point to distinguish the lane lines on the left and right sides, and the maximum value of the histogram on both sides is calculated as the starting position of the lane line search. With a fixed window and step size, a grid search is performed in the vertical direction. If the number of valid pixels (mask is 1) in the window exceeds a certain threshold, it is determined that there is a lane line in the window, and the mean of the valid pixel position is calculated as the lane line position corresponding to the window. After obtaining the lane line positions corresponding to all windows, a binomial fitting calculation is performed to obtain the fitted lane line equation. According to the fitted equation, the position of the search point is updated and iterative calculation is performed again, and smoothing is performed to obtain the final lane line fitting equation.

[0038] Therefore, after completing the detection of the lane line position and obtaining the lane line fitting equation, the position and shape information of the lane line has been basically determined, and the center line position and angle and position deviation can be further calculated based on this information.

[0039] like Figure 6 As shown, as a preferred embodiment, a process of centerline fitting and correction calculation of a lane in step S4 includes: S41: transmitting the angle error and the position error to the programmable logic controller; S42: calculating the control time of the port crane operation through a built-in lane keeping algorithm, and at the same time controlling a tire crane of the port crane to correct the on-off time of a relay; S43: performing a correction action on the port crane by calling a control program of the programmable logic controller, and judging whether the error value recognized by an image system is less than the trigger value of the correction action: if so, the correction action stops; if not, the correction action continues.

[0040] Preferably, in step S4, when an error signal received by the programmable logic controller is greater than a warning value, the programmable logic controller controls a tire crane braking relay and a motor relay, and the tire crane stops moving and issues an alarm.

[0041] Preferably, in step S4, when the distance between the port crane and an obstacle is less than a preset value, the programmable logic controller issues an alarm to a cab, and the port crane stops operating.

[0042] As Figure 7 shown, preferably, in this embodiment, among the movement states of the tire crane of this vehicle, ① shows the normal movement of the tire crane; ② shows that the diesel engine room on the left side of the tire crane is ahead, and the angle deviation △α < 0; ③ shows that the electrical room on the right side of the tire crane is ahead, and the angle deviation △α > 0; ④ shows that the position of the tire crane deviates towards the diesel engine room, and the position deviation △d > 0; ⑤ shows that the position of the tire crane deviates towards the electrical room, and the position deviation △d < 0.

[0043] When the set position deviation d > 20 cm and the angle deviation α > 1°, the PLC executes a shutdown program to stop the movement of the tire crane, and alarms the driver's cab and the main control terminal until the cause is查明 and the deviation is manually repaired before the tire crane can be run again. When the set position deviation 10 cm < d < 20 cm and the angle deviation 0.5° < α < 1°, the PLC executes a protection program to control the speeds of the motors on both sides to drop to one-fourth of the original speed, and issues a warning to the driver's cab. At the same time, when the set position deviation d < 2 cm and the angle deviation α < 0.1°, the PLC does not execute a correction action.

[0044] When the second situation in the figure occurs, it is necessary to control the motor on the right side to accelerate and the motor on the left side to decelerate; when the third situation in the figure occurs, it is necessary to control the motor on the left side to accelerate and the motor on the right side to decelerate; when the fourth situation in the figure occurs, it is necessary to control the motor on the left side to accelerate and the motor on the right side to decelerate; when the fifth situation in the figure occurs, it is necessary to control the motor on the right side to accelerate and the motor on the left side to decelerate.

[0045] As Figure 8 shown, in this embodiment, based on the above movement states of the tire crane Figure 6The control process shown can be implemented by PLC. The PLC module used in the present invention includes the following process for maintaining the lane of the port vehicle: a camera is installed at the front and rear positions of the diesel engine side and the electrical room side, and the distance between the two cameras on each side is about 620 cm (the specific value needs to be determined according to the tire crane model on site, etc.), and a floodlight is installed under the camera of the camera. When the light becomes dark, the floodlight automatically turns on to ensure the image effect. If a camera supporting infrared camera function is used, an infrared searchlight needs to be installed. The real-time data collected by the camera is transmitted to the electrical room through the communication module after image processing, and the receiving module of the electrical room then transmits the data to an IPC module. The image data is processed and analyzed in real time by the image recognition processing program of this IPC module. The error results obtained by the processing and analysis are transmitted to the PLC through the communication module. At the same time, the ultrasonic radar also sends the obtained data to the PLC. The two types of data are processed in the PLC, and then the control amount is output to the two frequency converters through the controller. The frequency converters then drive the two asynchronous motors to move respectively. When different control amounts are given to the two motors, the corresponding correction effect is achieved.

[0046] The lane keeping technology of tire crane is the basic technology of remote automatic control of tire crane. Its safety must comply with the requirements of the whole system. This lane keeping system solves the problem of control safety through test operation and sets up 2-level safety protection measures to prevent accidents caused by the deviation of the truck and collision with the box.

[0047] Level 1 safety protection measures: When the lane keeping system encounters the following situations, the tire crane truck's travel speed is reduced to 1 / 4 of the original speed: the lane keeping system fails to collect image data for 2 consecutive seconds; the position error collected by the lane keeping system exceeds 10cm, or the deflection angle is greater than 0.5°.

[0048] Level 2 safety protection measures: When the lane keeping system encounters the following situations, a system fault signal is sent, the tire crane truck stops urgently, finds out the cause, and continues to operate after troubleshooting: The lane keeping system fails to collect image data for 3 consecutive seconds; The position error collected by the lane keeping system exceeds 20 cm, or the deflection angle is greater than 1°.

[0049] In addition, the ultrasonic radar detects obstacle information and sends this signal to the PLC. When the detection distance is less than 3 meters, the PLC alarms the cab; when the detection distance is less than 1 meter, the PLC module controls the truck to stop running.

[0050] like Fig. 9As shown, in the practical application example of the present invention, a lane keeping control system for a port tire crane is used for the above-mentioned lane keeping control method for a port tire crane, comprising: an image acquisition module 1, used to acquire the driving lane image of the tire crane; an image recognition processing module 2, connected to the image acquisition module 1, correcting the distortion of an image acquired by the camera and obtaining a corrected lane image, identifying the lane lines in the corrected lane image and calculating the error; a driving control module 3, connected to the image recognition processing module 2, calculating a correction amount and an alarm threshold according to a control program, and generating a driving control signal.

[0051] Furthermore, the image recognition processing module 2 includes an industrial computer and an image recognition software. The image recognition processing module performs a noise reduction process on the lane image and captures the area of ​​interest, calculates the angle error and position error of the lane line, and transmits the error signal to the programmable logic controller.

[0052] Furthermore, the driving control module 3 includes a programmable logic controller, an ultrasonic radar and at least one relay. The programmable logic controller is used to receive data from the ultrasonic radar and the image recognition processing module, and control the on and off of the relay through the driving control signal to control the operating posture of the port crane.

[0053] Compared with the prior art, the present invention has the following outstanding features:

[0054] 1. Image technology has a wide range of applications: it is applicable to all tire crane trolley lane keeping scenarios, such as tire crane trolley lane lines are clear, and there are no special requirements for docks and equipment; if the original tire crane is already equipped with CCTV trolley cameras, after confirmation, the original cameras can be used for transformation;

[0055] 2. High reliability of image technology: Because the camera shoots obliquely downward and is located at a low position, the system can still operate normally even in bad weather such as rainy and foggy days. The surrounding buildings and equipment will not interfere with the system;

[0056] 3. Image technology has low economic cost: it only needs to install image recognition processing system, monitoring camera and PLC for lane keeping control. No additional modification of port infrastructure is required, the original environment will not be damaged, and the modification speed of tire crane truck is fast.

[0057] 4. The image recognition system is combined with ultrasonic radar, and the system reliability is high: In addition to the image recognition system, ultrasonic radars are installed at the front and rear of the truck to assist the image recognition system. When the image system has a blind spot and cannot detect obstacles, the ultrasonic radar can provide distance information of the obstacles, increasing the reliability and safety of the system.

[0058] It should be noted that, in this article, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only the above elements, but also other elements that have not been explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0059] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A port tire crane lane keeping control method, characterized in that: The following steps are involved: S1: an image acquisition module acquires a lane image of a tire crane; S2: an image recognition processing module processes the acquired image and obtains a processed lane image; S3: the image recognition processing module identifies the lane line in the processed lane image and calculates the error to obtain an error result; S4: the image recognition processing module sends the error result to a programmable logic controller for processing; The camera calibration process in step S1 includes: S11: collecting internal and external parameters and distortion parameters of the camera, and calculating a transformation matrix of the center position of a wheel of the tire crane; S12: calibrating the parameters obtained from the camera, and performing a coordinate transformation on the transformation matrix to obtain a coordinate transformation transformation matrix; S13: performing coordinate calibration on the image; Step S2 includes: S21: dividing the image into an area of ​​interest; S22: processing the area of ​​interest; S23: searching for the real-time operating position of the tire crane; S24: judging whether step S23 is completed: if so, executing step S25; if not, refitting the lane line, and then repeating step S23; S25: outputting the fitting equation of the lane line; Step S3 includes: S31: calculating the position of the center line of the lane line; S32: performing fitting calculation on a position signal and obtaining a center line fitting equation; S33: performing a bird's-eye view transformation based on the center line fitting equation; S34: calculating the error of the tire crane based on the result of the bird's-eye view transformation and obtaining an error result; Step S4 includes: S41: transmitting the error result to the programmable logic controller; S42: calculating the control time of the tire crane operation through a built-in lane keeping algorithm, and controlling the tire crane to correct the on-off time of the relay; S43: performing a correction action on the tire crane by calling a control program of the programmable logic controller, and judging whether the error result recognized by the image recognition processing module is less than the trigger value of the correction action: if so, the correction action stops; if not, the correction action continues.

2. A port tire crane lane keeping control system, used to implement the port tire crane lane keeping control method according to claim 1, comprising: An image acquisition module (1), the image acquisition module (1) being used to acquire an image of a driving lane of the tire crane; An image recognition processing module (2), the image recognition processing module (2) being connected to the image acquisition module (1), processing the distortion of the acquired image and obtaining the processed lane image, identifying the lane lines in the processed lane image and calculating the error; and a driving control module (3), the driving control module (3) being connected to the image recognition processing module (2), calculating a correction amount and an alarm threshold, and generating a driving control signal.

3. A port tire crane lane keeping control system as claimed in claim 2, characterized in that: The image recognition processing module (2) performs a noise reduction process on the lane image and intercepts the region of interest, calculates the error of the lane line to obtain the error result, and transmits the error result to the programmable logic controller.

4. A harbor tire crane lane keeping control system as claimed in claim 2, characterized in that: The driving control module (3) comprises a programmable logic controller, an ultrasonic radar and at least one relay, wherein the programmable logic controller is used to receive data from the ultrasonic radar and the image recognition processing module (2), and to control the on and off of the relay through the driving control signal, thereby controlling the operating posture of the tire crane.

Images