A RGV visual positioning system and method based on laser recognition

Abstract

The invention discloses an RGV visual positioning system and method based on laser recognition. A main control unit is respectively connected with a left laser generator, a right laser generator, a rear laser generator, an industrial camera and an RVG trolley; a part of laser light emitted by the left laser generator irradiates a calibration block of the left trolley, and another part of laser light emitted by the left laser generator irradiates a gravity brick in front of the RGV trolley; a part of laser light emitted by the right laser generator irradiates the calibration block of the right trolley, and another part of laser light emitted by the right laser generator irradiates the gravity brick in front of the RGV trolley; a part of laser light emitted by the rear laser generator irradiates the calibration block of the rear trolley, and another part of laser light emitted by the rear laser generator irradiates the rear of the RGV trolley; a rearview mirror is used to reflect the laser light emitted by the rear laser generator, and the industrial camera shoots the laser light emitted by the left laser generator, the right laser generator and the rear laser generator.

Description

Technical Field

[0001] The present invention relates to the technical field of RGV positioning control, and more specifically, to an RGV visual positioning system and method based on laser recognition. Background Art

[0002] Rail Guided Vehicle (RGV) is used in various high-density storage warehouses. At present, the coordination between RGV and assembly line is becoming more and more frequent. RGV docks with assembly line to load and unload materials, which requires frequent control of RGV to quickly, accurately and stably locate to the docking position with assembly line to complete loading and unloading, thus putting forward higher requirements on the positioning control method of RGV.

[0003] Traditionally, there are many ways to position the RGV, including encoder feedback positioning, laser sensor distance measurement positioning, and sensor fixed position proximity sensing positioning. Once the wheel slips, it is difficult to ensure accuracy in encoder feedback positioning; laser distance measurement positioning is difficult to meet the harsh on-site environment, such as dust, and cannot guarantee the accuracy of the measured distance; sensor fixed position sensing positioning, if the sensor is damaged or the position is inaccurate, the work of recalibrating the position is also extremely cumbersome. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides an RGV visual positioning system and method based on laser recognition, which uses a single camera to replace the traditional solution that requires the deployment of multiple sensors to complete the tasks of guiding the RGV trolley to move forward and backward with high precision on the track, material identification, obstacle avoidance detection, alignment of the trolley and the material, material picking and placing, and material lifting or lowering status detection, thereby realizing the functions of repeated high-precision positioning of the RGV trolley and automatic correction of material position in the gravity energy storage project.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: an RGV visual positioning system based on laser recognition, comprising: a main control unit and a left laser generator, a right laser generator, a rear laser generator, a left trolley calibration block, a right trolley calibration block, a rear trolley calibration block, an industrial camera, and a rearview mirror, the main control unit is respectively connected to the left laser generator, the right laser generator, the rear laser generator, the industrial camera, and the RVG trolley; a part of the laser emitted by the left laser generator is irradiated on the left trolley calibration block, and the left laser generator Another part of the laser is irradiated on the gravity brick in front of the RGV trolley; a part of the laser emitted by the right laser generator is irradiated on the calibration block of the right trolley, and another part of the laser emitted by the right laser generator is irradiated on the gravity brick in front of the RGV trolley; a part of the laser emitted by the rear laser generator is irradiated on the calibration block of the rear trolley, and another part of the laser emitted by the rear laser generator is irradiated on the rear of the RGV trolley; the rearview mirror is used to reflect the laser emitted by the rear laser generator, and the industrial camera shoots the lasers emitted by the left laser generator, the right laser generator, and the rear laser generator.

[0006] Furthermore, the laser includes: a laser straight line of the left trolley calibration block, a laser straight line of the right trolley calibration block, a laser straight line on the left side of the brick in front of the RGV trolley, a laser straight line on the right side of the brick in front of the RGV trolley and a laser straight line behind the RGV trolley reflected by the rearview mirror.

[0007] Furthermore, the RGV visual positioning system is set on a floor track of a certain floor of the gravity energy storage structure. One end of the track is the storage area for gravity bricks, and the other end of the track is the transportation area for gravity bricks. The RGV trolley runs between the storage area and the transportation area for gravity bricks.

[0008] The present invention also provides a positioning method of an RGV visual positioning system based on laser recognition, including: a clearing positioning method and a storage positioning method. The clearing positioning method is: the RGV trolley runs from a standby position to below the gravity bricks to be cleared in the gravity brick storage area, and the RGV trolley lifts the gravity bricks to be cleared, and then moves them to the elevator position, puts down the gravity bricks to be cleared, and then returns to the standby position; the storage positioning method is: the RGV trolley runs from a standby position to below the gravity bricks to be moved on the elevator side, and the RGV trolley lifts the gravity bricks to be cleared, and then moves them to the corresponding gravity brick placement position in the gravity brick storage area, and then puts down the gravity bricks to be cleared, and then returns to the standby position.

[0009] Furthermore, the clearing positioning method specifically includes the following steps:

[0010] Step S0: When the RGV trolley moves from the standby position to the bottom of the gravity bricks to be transported in the gravity brick storage area, the main control unit controls the industrial camera to capture images in real time;

[0011] Step S1: setting the left laser line and the right laser line ROI area as identification frames on the real-time image, and when the left laser line and the right laser line appear in the corresponding identification frames on the image, the RGV trolley decelerates;

[0012] Step S2: In the image captured by the industrial camera, when the left laser line and the left trolley calibration block laser line change from parallel lines to a straight line and the right laser line and the right trolley calibration block laser line change from parallel lines to a straight line, the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L1 are recorded;

[0013] Step S3: lift the gravity brick to be transported by the hydraulic lifting platform on the RGV trolley, and use the industrial camera to shoot and record the position of the left laser line and the position L2 of the right laser line. When L2 is higher than L1, it is determined that the gravity brick to be transported is lifted;

[0014] Step S4: Move the RGV trolley toward the elevator side, and when the special mark of the elevator side material discharging position in the rearview mirror is photographed by an industrial camera and begins to appear in the corresponding identification frame, the RGV trolley decelerates;

[0015] Step S5: In the image captured by the industrial camera, when the special mark in the special mark information box of the elevator side material placement position is at the specified position of the ROI area, the RGV trolley reaches the elevator side material removal and brick placement position, stops running, and records the position of the left laser line and the position of the right laser line L3;

[0016] Step S6: lower the hydraulic lifting platform on the RGV trolley so that the gravity bricks to be transported fall on the track, and use the industrial camera to shoot and record the position of the left laser line and the position of the right laser line L4. When L4 is lower than L3, it is determined that the gravity bricks to be transported are put down, and the RGV trolley returns to the standby position;

[0017] Step S7: Repeat steps S0 to S6 until all gravity bricks in the gravity brick storage area are cleared.

[0018] Furthermore, the storage positioning method specifically comprises the following steps:

[0019] Step A0: When the RGV trolley moves from the standby position to the bottom of the gravity brick to be transported on the elevator side, the main control unit controls the industrial camera to capture images in real time;

[0020] Step A1: Set the ROI area of ​​the rear laser line as the rear identification frame on the real-time image. When the rear laser line is within the rear identification frame, the RGV trolley decelerates.

[0021] Step A2: In the image captured by the industrial camera, when the rear laser line and the rear trolley calibration block laser line change from parallel lines to a straight line, the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L5 are recorded;

[0022] Step A3: Lift the gravity brick to be transported by the hydraulic lifting platform on the RGV trolley, and use the industrial camera to shoot and record the position of the left laser line and the position of the right laser line L6. When L6 is higher than L5, it is determined that the gravity brick to be transported is lifted;

[0023] Step A4: Move the RGV trolley to the storage area of ​​the gravity bricks, turn on the industrial camera to take real-time images, and determine whether the gravity brick to be transported is the first brick. If not, execute step A5; otherwise, execute A8;

[0024] Step A5: When the left laser line and the right laser line are observed to appear in the corresponding identification frames on the image, the RGV trolley decelerates;

[0025] Step A6: Calculate the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the car calibration block, and the pixel interval D2 between the laser line on the right side of the brick and the laser line on the right side of the car calibration block in real time, and calculate the average interval If the average spacing reaches the set threshold, the RGV trolley stops running and records the position of the left laser line and the position of the right laser line L7;

[0026] Step A7: Lower the hydraulic lifting platform on the RGV trolley so that the gravity block to be transported falls on the track, and use the industrial camera to shoot and record the position of the left laser line and the position of the right laser line L8. When L8 is lower than L7, it is determined that the gravity brick to be transported is put down, and the RGV trolley returns to the standby position;

[0027] Step A8: When the special mark of the first brick position photographed by the industrial camera begins to appear in the corresponding identification frame, the RGV trolley slows down; in the image photographed by the industrial camera, when the special mark in the special identification information frame of the first brick position in the gravity brick storage area is at the specified position of the ROI area, the RGV trolley has reached the first brick position in the gravity brick storage area, the RGV trolley stops running, and the position of the left laser line and the position of the right laser line L9 are recorded; the hydraulic lifting platform on the RGV trolley is lowered so that the gravity block to be transported falls on the track, and the position of the left laser line and the position of the right laser line L10 are photographed and recorded by the industrial camera. When L10 is lower than L9, it is determined that the gravity brick to be transported is put down, and the RGV trolley returns to the standby position;

[0028] Step A9: Repeat steps A0-A8 until all the weight blocks on the elevator side are stored.

[0029] Furthermore, the calculation process of the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the trolley calibration block is specifically as follows:

[0030] (a) Filtering the images taken by the industrial camera through Gaussian transformation;

[0031] (b) Performing Canny edge detection on the filtered image, calculating the gradient and direction of each pixel in each image, applying the double threshold method to distinguish strong edges from weak edges, extracting the edge information of the identification box corresponding to the left laser line, and obtaining an edge information image;

[0032] (c) performing dilation processing on the obtained edge information image to obtain a dilated image;

[0033] (d) extracting the outer contour of each dilated image based on the edge information;

[0034] (e) The outer contour is combined with the corresponding edge information image, and two laser lines are simulated by least squares linear fitting. The upper laser line is used as the laser line on the left side of the brick, and the lower laser line is used as the laser line on the left side of the trolley calibration block;

[0035] (f) Traverse and calculate the distance of all points on the laser straight line of the left trolley calibration block to the laser straight line on the left side of the brick, and take the maximum distance as the pixel interval D1 between the laser straight line on the left side of the brick and the laser straight line on the left trolley calibration block.

[0036] Furthermore, the Canny edge detection process in step (b) is specifically as follows: using the Sobel horizontal operator Sx to calculate the horizontal pixel gradient matrix Gx of image x, using the Sobel vertical operator Sy to calculate the horizontal pixel gradient matrix Gy of image y, and calculating the gradient and direction of each pixel point in each filtered image.

[0037] Furthermore, the process of using the double threshold method to distinguish strong edges from weak edges is as follows:

[0038]

[0039] Among them, g is the image pixel, maxVal is the high threshold, and minVal is the low threshold.

[0040] Furthermore, the expansion process in step (c) is as follows: a kernel of any shape is selected to traverse all pixel points on the edge information image, and the maximum pixel value on the edge information image covered by the kernel is extracted to replace the pixel of the anchor point in the kernel.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The RGV visual positioning system and method based on laser recognition in the present invention has low positioning accuracy compared to traditional RGVs. The industrial camera's vision can capture the position of the first brick, the pick-up and placement position in the handling area, and the actual relative position of the RGV trolley and the previous gravity brick in real time with high precision. The RGV trolley can accurately position all positions in the track, effectively control the positioning error within 1 mm, and realize high-repeat precision positioning of the RGV trolley in gravity energy storage.

[0043] (2) The RGV visual positioning system and method based on laser recognition of the present invention uses an industrial camera to shoot the lasers emitted by the left laser generator, the right laser generator, and the rear laser generator, thereby realizing the positioning of the RGV trolley carrying the gravity bricks through image recognition and judgment, improving the operating speed of the positioning system, and through the positioning and recognition of the laser straight line on the image, the reliability is strong, ensuring the stability of the positioning system;

[0044] The RGV visual positioning system and method based on laser recognition of the present invention realizes multiple functions such as movement, recognition, alignment, and pick-and-place judgment by combining laser with vision, and has the characteristics of less manual intervention, high degree of automation, and strong reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural schematic diagram of the RGV visual positioning system based on laser recognition of the present invention;

[0046] Figure 2 It is a schematic diagram of the application of the RGV visual positioning system of the present invention on the gravity energy storage structure;

[0047] Figure 3 It is a flow chart of the RGV visual positioning system clearing positioning method based on laser recognition of the present invention;

[0048] Figure 4 It is a flow chart of the storage positioning method of the RGV visual positioning system based on laser recognition of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0050] like Figure 1 The present invention is a structural schematic diagram of the RGV visual positioning system based on laser recognition, the RGV visual positioning system comprises: a main control unit and a left laser generator 1, a right laser generator 2, a rear laser generator 3, a left trolley calibration block 4, a right trolley calibration block 5, a rear trolley calibration block 6, an industrial camera 7, and a rearview mirror 8 arranged on the RVG trolley. The main control unit is respectively connected to the left laser generator 1, the right laser generator 2, the rear laser generator 3, the industrial camera 7, and the RVG trolley; a part of the laser emitted by the left laser generator 1 is irradiated on the left trolley calibration block 4, and the position of the part of the laser straight line remains unchanged; another part of the laser emitted by the left laser generator 1 is irradiated on the gravity brick in front of the RGV trolley, and through the continuous advancement or retreat of the RGV trolley, the other part of the laser straight line gradually approaches or moves away from the laser straight line on the left trolley calibration block 4; a part of the laser emitted by the right laser generator 2 is irradiated on the right trolley calibration block 5, and the position of the part of the laser straight line remains unchanged. Another part of the laser light emitted by the right laser generator 2 is irradiated on the gravity brick in front of the RGV trolley. As the RGV trolley continuously moves forward or backward, another part of the laser straight line gradually approaches or moves away from the laser straight line on the calibration block 5 of the right trolley; a part of the laser light emitted by the rear laser generator 3 is irradiated on the calibration block 6 of the rear trolley, and the position of this part of the laser straight line remains unchanged. Another part of the laser light emitted by the rear laser generator 3 is irradiated on the rear of the RGV trolley. As the RGV trolley continuously moves forward or backward, another part of the laser straight line gradually moves away from or approaches the laser straight line on the calibration block 6 of the rear trolley; the rearview mirror 8 is used to reflect the laser light emitted by the rear laser generator 3, and the industrial camera 7 shoots the laser light emitted by the left laser generator 1, the right laser generator 2, and the rear laser generator 3, so as to realize the positioning of the gravity brick transported by the RGV trolley through image recognition and judgment, thereby improving the operating speed of the positioning system, and through the positioning and recognition of the laser straight line on the image, the reliability is strong, and the stability of the positioning system is guaranteed.

[0051] The laser in the present invention includes: a laser straight line of the left trolley calibration block, a laser straight line of the right trolley calibration block, a laser straight line on the left side of the brick in front of the RGV trolley, a laser straight line on the right side of the brick in front of the RGV trolley and a laser straight line behind the RGV trolley reflected by the rearview mirror 8.

[0052] like Figure 2The RGV visual positioning system is set on a floor track of a gravity energy storage structure. One end of the track is the storage area for gravity bricks, and the other end of the track is the transportation area for gravity bricks. The RGV trolley runs between the storage area and the transportation area for gravity bricks.

[0053] The present invention provides a positioning method of the RGV visual positioning system based on laser recognition, including: a clearing positioning method and a storage positioning method. The clearing positioning method is: the RGV trolley runs from the standby position to the bottom of the gravity brick to be cleared in the gravity brick storage area, the RGV trolley lifts the gravity brick to be cleared, carries it to the elevator position, puts down the gravity brick to be cleared, and returns to the standby position. Figure 3 The clearing positioning method in the present invention specifically includes the following steps:

[0054] Step S0: When the RGV trolley moves from the standby position to the bottom of the gravity bricks to be removed in the gravity brick storage area, the main control unit controls the industrial camera 7 to capture images in real time;

[0055] Step S1: setting the left laser line and the right laser line ROI area as identification frames on the real-time image, and when the left laser line and the right laser line appear in the corresponding identification frames on the image, the RGV trolley decelerates;

[0056] Step S2: In the image captured by the industrial camera 7, when the left laser line and the left trolley calibration block laser line change from parallel lines to a straight line and the right laser line and the right trolley calibration block laser line change from parallel lines to a straight line, it means that the RGV trolley is flush with the gravity bricks to be cleared, that is, the RGV trolley reaches the bottom of the gravity bricks to be cleared, and the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L1 are recorded;

[0057] Step S3: lift the gravity brick to be transported by the hydraulic lifting platform on the RGV trolley, and shoot and record the position of the left laser line and the position of the right laser line L2 through the industrial camera 7. When L2 is higher than L1, it is determined that the gravity brick to be transported is lifted, and the material picking process of the RGV trolley is completed;

[0058] Step S4: Move the RGV trolley to the elevator side, and when the special mark of the elevator side material discharging position in the rearview mirror 8 is photographed by the industrial camera 7 and begins to appear in the corresponding identification frame, the RGV trolley slows down;

[0059] Step S5: In the image captured by the industrial camera 7, when the special mark in the special mark information box of the elevator side material placement position is at the specified position of the ROI area, the RGV trolley reaches the elevator side material removal and brick placement position, stops running, and records the position of the left laser line and the position of the right laser line L3;

[0060] Step S6: lower the hydraulic lifting platform on the RGV trolley so that the gravity bricks to be transported fall on the track, and use the industrial camera 7 to shoot and record the position of the left laser line and the position of the right laser line L4. When L4 is lower than L3, it is determined that the gravity bricks to be transported are put down, the RGV trolley completes the unloading process, and the RGV trolley returns to the standby position;

[0061] Step S7: Repeat steps S0 to S6 until all gravity bricks in the gravity brick storage area are cleared.

[0062] The storage positioning method of the present invention is as follows: the RGV trolley runs from the standby position to the elevator side below the gravity bricks to be transported, the RGV trolley lifts the gravity bricks to be transported, and then transports them to the corresponding gravity brick placement position in the gravity brick storage area, and then puts down the gravity bricks to be transported, and then returns to the standby position. Figure 4 The storage and positioning method of the present invention specifically includes the following steps:

[0063] Step A0: When the RGV trolley moves from the standby position to the bottom of the gravity brick to be transported on the elevator side, the main control unit controls the industrial camera 7 to capture images in real time;

[0064] Step A1: Set the ROI area of ​​the rear laser line as the rear identification frame on the real-time image. When the rear laser line is within the rear identification frame, the RGV trolley decelerates.

[0065] Step A2: In the image captured by the industrial camera 7, when the rear laser line and the rear trolley calibration block laser line change from parallel lines to a straight line, the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L5 are recorded;

[0066] Step A3: lift the gravity brick to be transported by the hydraulic lifting platform on the RGV trolley, and use the industrial camera 7 to shoot and record the position of the left laser line and the position of the right laser line L6. When L6 is higher than L5, it is determined that the gravity brick to be transported is lifted;

[0067] Step A4: Move the RGV trolley to the storage area of ​​the gravity bricks, turn on the industrial camera 7 to take real-time images, and determine whether the gravity brick to be transported is the first brick. If not, execute step A5; otherwise, execute A8;

[0068] Step A5: When the left laser line and the right laser line are observed to appear in the corresponding identification frames on the image, the RGV trolley decelerates;

[0069] Step A6: Calculate the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the car calibration block, and the pixel interval D2 between the laser line on the right side of the brick and the laser line on the right side of the car calibration block in real time, and calculate the average interval If the average spacing reaches the set threshold, the RGV trolley stops running. When placing the gravity bricks at the average spacing, it prevents the gravity bricks in front from having a slight angle on the left and right sides. The average spacing can reduce the angle error of the gravity bricks, so that the gravity bricks on the RGV trolley and the gravity bricks in front are placed as evenly as possible. And record the position of the left laser line and the position of the right laser line L7;

[0070] Step A7: lower the hydraulic lifting platform on the RGV trolley so that the gravity block to be transported falls on the track, and use the industrial camera 7 to shoot and record the position of the left laser line and the position of the right laser line L8. When L8 is lower than L7, it is determined that the gravity brick to be transported is put down, and the RGV trolley returns to the standby position;

[0071] The calculation process of the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the trolley calibration block in the present invention is specifically as follows:

[0072] (a) filtering the image taken by the industrial camera 7 by Gaussian transformation to reduce the influence of noise in the image on the extraction and fitting of the image laser line;

[0073] (b) Perform Canny edge detection on the filtered image, calculate the gradient and direction of each pixel in each image, specifically, use the Sobel horizontal operator Sx to calculate the pixel gradient matrix Gx in the horizontal direction of image x, use the Sobel vertical operator Sy to calculate the pixel gradient matrix Gy in the horizontal direction of image y, and calculate the gradient and direction of each pixel in each filtered image. Use the double threshold method to distinguish strong edges from weak edges, extract the edge information of the identification box corresponding to the left laser line, and obtain an edge information image; the process of using the double threshold method to distinguish strong edges from weak edges in the present invention is specifically as follows:

[0074]

[0075] Among them, g is the gradient value of the image pixel, maxVal is the high threshold, and minVal is the low threshold.

[0076] By setting the high threshold maxVal and the low threshold minVal, the image pixel with a gradient value higher than maxVal is represented as a strong edge point, which represents a real edge point; the image pixel with a gradient value lower than maxVal but higher than minVal is a weak edge point, which needs to be retained for additional processing. Specifically, if there is a strong edge point in the neighborhood of the weak edge point, the gradient value of the image pixel is a real edge point, otherwise it is suppressed and removed; the image pixel with a gradient value lower than minVal is not regarded as an edge point and is deleted. By applying double thresholds, the noise in the image can be effectively filtered out, which can improve the image quality to a certain extent.

[0077] (c) dilating the obtained edge information image to obtain a dilated image; specifically, selecting a kernel of any shape to traverse all pixel points on the edge information image, extracting the maximum pixel value on the edge information image covered by the kernel, and replacing the pixel of the anchor point in the kernel.

[0078] (d) extracting the outer contour of each dilated image based on the edge information;

[0079] (e) The outer contour is combined with the corresponding edge information image, and two laser lines are simulated by least squares linear fitting. The upper laser line is used as the laser line on the left side of the brick, and the lower laser line is used as the laser line on the left side of the trolley calibration block;

[0080] (f) Traverse and calculate the distance of all points on the laser straight line of the left trolley calibration block to the laser straight line on the left side of the brick, and take the maximum distance as the pixel interval D1 between the laser straight line on the left side of the brick and the laser straight line on the left trolley calibration block.

[0081] The calculation process of the pixel interval D2 between the laser line on the right side of the brick and the laser line on the right side of the trolley calibration block in the present invention is the same as the calculation process of the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the trolley calibration block, and will not be repeated here.

[0082] Step A8: When the special mark of the first brick position photographed by the industrial camera 7 begins to appear in the corresponding identification frame, the RGV trolley slows down; in the image photographed by the industrial camera 7, when the special mark in the special identification information frame of the first brick position in the gravity brick storage area is at the specified position of the ROI area, the RGV trolley has reached the first brick position in the gravity brick storage area, the RGV trolley stops running, and the position of the left laser line and the position of the right laser line L9 are recorded; the hydraulic lifting platform on the RGV trolley is lowered so that the gravity block to be transported falls on the track, and the position of the left laser line and the position of the right laser line L10 are photographed and recorded by the industrial camera 7. When L10 is lower than L9, it is determined that the gravity brick to be transported is put down, and the RGV trolley returns to the standby position;

[0083] Step A9: Repeat steps A0-A8 until all the weight blocks on the elevator side are stored.

[0084] The RGV visual positioning method based on laser recognition of the present invention uses the vision of an industrial camera to determine whether the RGV trolley has reached the specified position. Compared with traditional position feedback or laser ranging methods, it has the characteristics of accurate positioning with high precision and high efficiency. The present invention uses a single camera to replace the traditional solution that requires the arrangement of multiple sensors to complete the tasks of guiding the RGV trolley to move forward and backward on the track with high precision, material recognition, obstacle avoidance detection, alignment of the trolley and the material, material picking and placing, and material lifting or putting down status detection, and has strong stability. At the same time, through visual methods, all positions in the track can be automatically corrected, achieving the characteristics of less manual intervention and high degree of automation.

[0085] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A RGV visual positioning method based on laser recognition, characterized in that: include: The emptying positioning method and the storage positioning method are as follows: the RGV trolley runs from the standby position to the bottom of the gravity bricks to be cleared in the gravity brick storage area, the RGV trolley lifts the gravity bricks to be cleared, carries them to the elevator position, puts down the gravity bricks to be cleared, and returns to the standby position; The storage positioning method is as follows: the RGV trolley runs from the standby position to the bottom of the gravity brick to be transported on the elevator side, the RGV trolley lifts the gravity brick to be cleared, and then transports it to the corresponding gravity brick placement position in the gravity brick storage area, and then puts down the gravity brick to be cleared, and then returns to the standby position; The clearing positioning method specifically comprises the following steps: Step S0: When the RGV trolley is moved from the standby position to the bottom of the gravity brick storage area where the gravity bricks are to be removed, the main control unit controls the industrial camera (7) to capture images in real time; Step S1: setting the left laser line and the right laser line ROI area as identification frames on the real-time image, and when the left laser line and the right laser line appear in the corresponding identification frames on the image, the RGV trolley decelerates; Step S2: In the image captured by the industrial camera (7), when the left laser line and the left trolley calibration block laser line change from parallel lines to a straight line and the right laser line and the right trolley calibration block laser line change from parallel lines to a straight line, the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L1 are recorded; Step S3: The hydraulic lifting platform on the RGV trolley lifts the gravity brick to be transported, and the industrial camera (7) photographs and records the position of the left laser line and the position L2 of the right laser line. When L2 is higher than L1, it is determined that the gravity brick to be transported is lifted; Step S4: Move the RGV trolley toward the elevator side, and when the special mark of the elevator side material discharging position in the rearview mirror (8) is photographed by the industrial camera (7) and begins to appear in the corresponding identification frame, the RGV trolley decelerates; Step S5: In the image captured by the industrial camera (7), when the special mark in the special mark information box of the elevator side material placement position is at the specified position of the ROI area, the RGV trolley reaches the elevator side material removal and brick placement position, stops running, and records the position of the left laser line and the position of the right laser line L3; Step S6: lower the hydraulic lifting platform on the RGV trolley so that the gravity brick to be transported falls on the track, and use the industrial camera (7) to shoot and record the position of the left laser line and the position of the right laser line L4. When L4 is lower than L3, it is determined that the gravity brick to be transported is put down, and the RGV trolley returns to the standby position; Step S7: Repeat steps S0 to S6 until all the gravity bricks in the gravity brick storage area are cleared away.

2. The RGV visual positioning method based on laser recognition according to claim 1 is characterized in that: The storage positioning method specifically comprises the following steps: Step A0: When the RGV trolley moves from the standby position to the bottom of the gravity brick to be transported on the elevator side, the main control unit controls the industrial camera (7) to capture images in real time; Step A1: Set the ROI area of ​​the rear laser line as the rear identification frame on the real-time image. When the rear laser line is within the rear identification frame, the RGV trolley decelerates. Step A2: In the image captured by the industrial camera (7), when the rear laser line and the rear trolley calibration block laser line change from parallel lines to a straight line, the RGV trolley stops running. At this time, the position of the left laser line and the position of the right laser line L5 are recorded; Step A3: The hydraulic lifting platform on the RGV trolley lifts the gravity brick to be transported, and the industrial camera (7) photographs and records the position of the left laser line and the position of the right laser line L6. When L6 is higher than L5, it is determined that the gravity brick to be transported is lifted; Step A4: Move the RGV trolley to the storage area of ​​the gravity bricks, turn on the industrial camera (7) to take real-time images, and determine whether the gravity brick to be transported is the first brick. If not, execute step A5; Otherwise, execute A8; Step A5: When the left laser line and the right laser line are observed to appear in the corresponding identification frames on the image, the RGV trolley decelerates; Step A6: Calculate the pixel interval D1 between the laser line on the left side of the brick and the laser line on the left side of the car calibration block, and the pixel interval D2 between the laser line on the right side of the brick and the laser line on the right side of the car calibration block in real time, and calculate the average interval ,If the average spacing reaches the set threshold, the RGV trolley stops running and records the position of the left laser line and the position of the right laser line L7; Step A7: lower the hydraulic lifting platform on the RGV trolley so that the weight block to be transported falls on the track, and use the industrial camera (7) to shoot and record the position of the left laser line and the position of the right laser line L8. When L8 is lower than L7, it is determined that the weight block to be transported is put down, and the RGV trolley returns to the standby position; Step A8: When the special mark of the first brick position photographed by the industrial camera (7) begins to appear in the corresponding identification frame, the RGV trolley slows down; in the image photographed by the industrial camera (7), when the special mark in the special identification information frame of the first brick position in the gravity brick storage area is at the specified position of the ROI area, the RGV trolley has reached the first brick position in the gravity brick storage area, the RGV trolley stops running, and the position of the left laser line and the position of the right laser line L9 are recorded; the hydraulic lifting platform on the RGV trolley is lowered so that the gravity block to be transported falls on the track, and the position of the left laser line and the position of the right laser line L10 are photographed and recorded by the industrial camera (7). When L10 is lower than L9, it is determined that the gravity brick to be transported has been put down, and the RGV trolley returns to the standby position; Step A9: Repeat steps A0-A8 until all the weight blocks on the elevator side are stored.

3. The RGV visual positioning method based on laser recognition according to claim 2 is characterized in that: The calculation process of the pixel interval D1 between the laser straight line on the left side of the brick and the laser straight line on the left side of the trolley calibration block is specifically as follows: (a) Filtering the image taken by the industrial camera (7) through Gaussian transformation; (b) Perform Canny edge detection on the filtered image, calculate the gradient and direction of each pixel in each image, apply the double threshold method to distinguish strong edges from weak edges, extract the edge information of the identification box corresponding to the left laser line, and obtain the edge information image; (c) performing dilation processing on the obtained edge information image to obtain a dilated image; (d) Extract the outer contour of each dilated image based on the edge information; (e) Combine the outer contour with the corresponding edge information image and use the least squares method to linearly fit and simulate two laser lines. The upper laser line is used as the laser line on the left side of the brick, and the lower laser line is used as the laser line on the left side of the trolley calibration block. (f) Traverse all points on the laser straight line of the left trolley calibration block and calculate the distance to the laser straight line on the left side of the brick, and take the maximum distance as the pixel interval D1 between the laser straight line on the left side of the brick and the laser straight line on the left trolley calibration block.

4. The RGV visual positioning method based on laser recognition according to claim 3 is characterized in that: The specific process of Canny edge detection processing in step (b) is as follows: using the Sobel horizontal operator Sx to calculate the horizontal pixel gradient matrix Gx of image x, using the Sobel vertical operator Sy to calculate the horizontal pixel gradient matrix Gy of image y, and calculating the gradient and direction of each pixel point in each filtered image.

5. The RGV visual positioning method based on laser recognition according to claim 4 is characterized in that: The specific process of using the double threshold method to distinguish strong edges from weak edges is: ； Among them, g is the image pixel, max Val is the high threshold, min Val is the low threshold.

6. The RGV visual positioning method based on laser recognition according to claim 3 is characterized in that: The process of the expansion processing in step (c) is: select a kernel of any shape to traverse all pixel points on the edge information image, extract the maximum pixel value on the edge information image covered by the kernel, and replace the pixel of the anchor point in the kernel.

7. An RGV visual positioning system based on laser recognition, characterized in that: The RGV visual positioning method according to any one of claims 1 to 6 comprises: a main control unit and a left laser generator (1), a right laser generator (2), a rear laser generator (3), a left trolley calibration block (4), a right trolley calibration block (5), a rear trolley calibration block (6), an industrial camera (7), and a rearview mirror (8) arranged on the RVG trolley, wherein the main control unit is respectively connected to the left laser generator (1), the right laser generator (2), the rear laser generator (3), the industrial camera (7), and the RVG trolley; a part of the laser light emitted by the left laser generator (1) is irradiated on the left trolley calibration block (4), and another part of the laser light emitted by the left laser generator (1) is irradiated on the left trolley calibration block (4), and another part of the laser light emitted by the left laser generator (1) is irradiated on the left trolley calibration block (4). The laser is irradiated onto the gravity brick in front of the RGV trolley; a portion of the laser light emitted by the right laser generator (2) is irradiated onto the right trolley calibration block (5), and another portion of the laser light emitted by the right laser generator (2) is irradiated onto the gravity brick in front of the RGV trolley; a portion of the laser light emitted by the rear laser generator (3) is irradiated onto the rear trolley calibration block (6), and another portion of the laser light emitted by the rear laser generator (3) is irradiated onto the rear of the RGV trolley; the rearview mirror (8) is used to reflect the laser light emitted by the rear laser generator (3), and the industrial camera (7) photographs the laser light emitted by the left laser generator (1), the right laser generator (2), and the rear laser generator (3).

8. The RGV visual positioning system based on laser recognition according to claim 7 is characterized in that: The laser includes: a laser straight line of the left trolley calibration block, a laser straight line of the right trolley calibration block, a laser straight line of the left side of the brick in front of the RGV trolley, a laser straight line of the right side of the brick in front of the RGV trolley, and a laser straight line behind the RGV trolley reflected by the rearview mirror (8).

9. The RGV visual positioning system based on laser recognition according to claim 7 is characterized in that: The RGV visual positioning system is set on a floor track of a gravity energy storage structure. One end of the track is the storage area for gravity bricks, and the other end of the track is the transportation area for gravity bricks. The RGV trolley runs between the storage area and the transportation area for gravity bricks.

Images