Forklift goods forking method and system based on visual identification of tray position

Abstract

The invention relates to the technical field of industrial automation and robots, in particular to a forklift goods forking method and system based on tray position visual identification, and the method comprises the steps that an AGV body travels to a fan-shaped area, point cloud information is collected through a radar to be matched with tray characteristics, and the initial position of a tray is calculated; planning a retreating curve, driving to a visual identification position, and controlling the AGV body to retreat and fork after primary visual identification, error adjustment and secondary visual identification accurate positioning; the forking state is detected through a tray in-place switch and a fork tip photoelectric sensor, and the result is judged. The system comprises an AGV body, a recognizable tray and a scheduling module, and the AGV body carries a controller, a radar, a camera, an in-place switch and a photoelectric sensor. According to the scheme, SLAM accumulated errors are eliminated through laser-vision double recognition, tray deviation is adapted, recognition reliability is improved in a fan-shaped area, a narrow channel strategy widens the range, intervention is reduced, and the forking success rate and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the fields of industrial automation and robotics, and in particular to a forklift loading method and system based on visual recognition of pallet position. Background Technology

[0002] Currently, SLAM navigation technology has become one of the core technologies in the field of robot autonomous navigation. It can support robots to achieve self-localization and environmental map construction in unknown environments, providing basic support for automated handling in industrial scenarios. However, in the specific scenario of forklift pallet picking, existing technical solutions have the following limitations: First, they are highly dependent on the pallet placement position, requiring the pallet to be fixed in place for accurate picking. If the initial placement of the pallet by the personnel is off-center in position or angle, it is very easy to cause picking failure. Second, pure SLAM navigation inevitably suffers from cumulative error during continuous picking and placing of goods. As the number of operations increases, the pallet will gradually deviate from its original position, eventually causing the forks to collide with the pallet, resulting in interruption of continuous operation. Third, existing solutions lack adaptation strategies for special operating environments such as narrow aisles. Since forklifts cannot rotate in place, the conventional picking path is prone to collisions with obstacles on both sides, further limiting the operating range. Fourth, single recognition modes (such as relying solely on laser or vision) have weak anti-interference capabilities. Laser is easily affected by obstruction, and vision is prone to insufficient matching. Moreover, there is a lack of clear error adjustment standards, resulting in poor operational stability and frequent manual intervention adjustments, which significantly reduces the overall handling efficiency and cannot meet the high precision and high reliability requirements of industrial scenarios for automated operations.

[0003] Therefore, there is an urgent need for a forklift loading method and system that can eliminate cumulative errors, adapt to pallet placement deviations, and meet the needs of narrow aisle operations, in order to overcome the shortcomings of existing technologies. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a forklift loading method and system based on visual recognition of pallet position. This method addresses the technical problems of cumulative errors, reliance on fixed pallet placement, difficulties in narrow aisle operations, and low operational efficiency in the prior art, thereby increasing the success rate of tasks, reducing manual intervention, and improving handling efficiency.

[0005] The above objectives are achieved through the following technical solutions: A forklift loading method based on visual recognition of pallet position includes the following steps: S1: The controller combines SLAM positioning information and the destination location information issued by the scheduler to control the AGV to travel to the preset fan-shaped area; S2: The AGV vehicle body collects point cloud information within the fan-shaped area through radar, fits a straight line, matches the left support width, middle support width, right support width and span characteristics between adjacent supports of the pallet, and calculates the initial position of the pallet. S3: The controller calculates the Euclidean distance between the current position of the AGV vehicle and the initial position of the pallet, plans the backward curve, and controls the AGV vehicle to perform the first backward movement to reach the initial visual recognition position. S4: Turn on the camera to perform initial visual recognition, calculate the lateral and angular errors between the AGV body and the tray, and determine whether to advance a preset distance based on the magnitude of the error to initially eliminate the error; S5: Initiate secondary visual recognition to accurately update the pallet position information. The controller combines the SLAM positioning information to calculate the speed and angular velocity of the second backward movement and control the AGV body to move backward. S6: Real-time detection of fork-picking status via pallet position switch and fork tip photoelectric sensor; determination of successful picking based on detection results and reporting.

[0006] As an optimization of this method, the radius of the sector area in step S1 is 5 meters and the central angle is 90°. The radius and central angle are adjusted through a configuration file according to the radar model and the field environment.

[0007] As an optimization of this method, in step S2, when there is an obstruction within the fan-shaped area, at least 50% of one fork hole of the tray must be identified, and the position of the fork hole is determined by the feature that the width of the middle support is less than the width of the two side supports, and then the initial position of the tray is calculated, specifically including: 1. When performing a forklift task, the dispatcher will issue the cargo location. However, due to reasons such as manual placement, the cargo may deviate from the location issued by the dispatcher. The placement location of the cargo will be marked with a frame line. Manual placement may deviate, but it will still be within the frame. If there is an obstruction in the fan-shaped area, when the pallet cannot be identified initially, it will first move according to the target location issued by the dispatcher, thereby ensuring that a forklift hole range can be identified when moving towards the target location issued by the dispatcher.

[0008] 2. The width of the middle support should be smaller than that of the two sides. This condition can avoid a misidentification situation. When the width of the middle support is the same as that of the two sides, if the distance between the other pallet and the current pallet is exactly the width of a forkhole, the pallet position may be misidentified. When the width of the middle support is smaller than that of the two side supports, this misidentification situation can be avoided.

[0009] Note: Using scheduling points can avoid this misidentification. If the identified pallet position exceeds the deviation range of the scheduling points, it is not considered a pallet. For example, if the scheduling points are x, y, t, and the allowable placement deviation is ±20cm and 10°, then if the identified pallet position exceeds this range, it is considered not a pallet. If this condition is turned off, it can be identified even beyond a larger range, and having the middle support smaller than the side supports can avoid misidentification.

[0010] As an optimization of this method, the formula for calculating the Euclidean distance in step S3 is as follows: , in, ps The vehicle's current location. pt For the target pallet location, pt x For the target pallet position x value, ps y The vehicle's current location y value, ps x The vehicle's current location x value, pt y For the target pallet position y value.

[0011] As an optimization of this method, when the lateral error is greater than 10cm or the angular error is greater than 5° in step S4, the forward distance of the AGV vehicle body shall not exceed 30cm; the forward distance is set according to the optimal recognition distance of the camera.

[0012] As an optimization of this method, when the matching degree of the initial visual recognition in step S4 is less than 0.6, the visual recognition result is fused with the center position information of the middle support width detected by the rear radar in the middle of the fork to obtain the corrected pallet position.

[0013] As an optimization of this method, the formula for calculating the velocity in step S5 is as follows: V x =K p * offset_y (k) +k d *(offset_y (k) - offset_y (k-1) )， The formula for calculating angular velocity is: W = K p * offset_y (k)+k d *(offset_y (k) - offset_y (k-1) )， in, offset_y This represents the lateral deviation of the vehicle's current position relative to the projection point. K p , K d These are the proportional coefficient and the differential coefficient.

[0014] As an optimization of this method, when the AGV is operating in a narrow channel, it is prohibited to perform rotational movements. Except for the forward movement required for error adjustment, it can only move backward along the channel direction, and the maximum steering angle is constrained. At the same time, obstacle avoidance is achieved based on the radar information on both sides.

[0015] A forklift loading and unloading system based on visual recognition of pallet position includes an AGV body, a pallet identifiable component, and a scheduling module; the AGV body includes a controller, radar, forks, a pallet position switch, a fork tip photoelectric sensor, and a camera; wherein: The radar is used to collect point cloud information within a fan-shaped area and feed back the position information of the AGV vehicle body; The camera is used to identify tray position information; The pallet positioning switch is used to detect whether the fork has been properly positioned. The fork tip photoelectric sensor is used to detect whether the fork is tilted or colliding. The controller is communicatively connected to the radar, the camera, the pallet positioning switch, the fork tip photoelectric sensor, and the scheduling module, respectively, and is used to receive position information and adjust the movement trajectory of the AGV vehicle.

[0016] As an optimization of this system, the triggering determination area of ​​the pallet positioning switch includes a normal judgment triggering area and an abnormal triggering area. The normal judgment triggering area is a range of 0.2 meters on both sides of the fork-taking point, and the abnormal triggering area is outside the normal judgment triggering area. When both pallet positioning switches on both sides are triggered in the normal judgment triggering area, the fork-taking is determined to be successful.

[0017] This invention provides a forklift loading method and system based on visual recognition of pallet position. It eliminates the cumulative error of pure SLAM navigation through laser-vision dual recognition, adapting to pallet placement deviations; a fan-shaped area setting improves radar recognition reliability; and a narrow-channel-specific strategy expands the operating range; combining formulaic calculations, dual detection, and quantitative adjustments reduces manual intervention, significantly improving the success rate of forklift loading tasks and handling efficiency. Specific beneficial effects are as follows: 1. This invention effectively eliminates the cumulative error caused by pure SLAM navigation by combining laser coarse positioning with visual dual recognition. At the same time, it adapts to the initial position deviation of personnel placing pallets, eliminating the need to strictly fix the pallet placement position and significantly increasing the success rate of forklift tasks.

[0018] 2. For scenarios where narrow passages prevent forklifts from rotating in place, a dedicated operating procedure was designed. By constraining the vehicle's movement mode, using laser obstacle avoidance on both sides, and scheduling-assisted positioning, safe and efficient forklift picking within narrow passages was achieved, expanding the forklift's operating range.

[0019] 3. The fan-shaped area setting focuses the radar's recognition range, ensuring the effectiveness and recognition accuracy of point cloud information. At the same time, through the occlusion adaptation strategy, tray positioning can still be completed in partially occluded scenarios, improving the system's anti-interference capability.

[0020] 4. Quantitative error judgment standards and clear adjustment procedures, combined with precise formulaic calculations, avoid blind operations and ensure cross-cutting accuracy; the fusion mechanism of visual recognition and rear radar further improves the reliability of positioning.

[0021] 5. The dual detection of pallet position switch and fork tip photoelectric sensor, along with clear rules for successful fork pickup, reduces misjudgment during the fork pickup process, lowers the risk of forks rubbing against the pallet, reduces manual intervention, and improves handling efficiency. Attached Figure Description

[0022] Figure 1 This is a task flowchart of a forklift loading method based on visual recognition of pallet position as described in this invention; Figure 2 This is a schematic diagram of the sector-shaped area identified in the forklift loading method based on visual recognition of pallet position according to the present invention; Figure 3 This is a view of the pallet picking direction in a forklift loading method based on visual recognition of pallet position as described in this invention. Figure 4 This is a top view of the narrow aisle forklift picking up goods in a forklift picking up goods method based on visual recognition of pallet position as described in this invention; Figure 5 This is a schematic diagram of the position switch trigger judgment area in a forklift loading method based on visual recognition of pallet position as described in this invention. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. The described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this solution provides a forklift loading method based on visual recognition of pallet position, including the following steps: Step (1) Driving to the fan-shaped recognition area: The controller combines SLAM positioning information and the destination position information issued by the dispatcher to control the AGV to drive to the preset fan-shaped area. The fan-shaped area is set according to the pallet position to ensure the radar recognition effect. Its radius is 5 meters and the angle is set at 90°. These two values ​​can be changed according to the configuration file. Different radars may have different effects. The radius and central angle can be adjusted according to the test. The setting of the fan-shaped area is based on the radar performance and resolution. The radar currently used can detect a relatively good point cloud effect within 5m, and the narrowest support width of the pallet is 10cm. If the radar detection distance is longer, the number of point clouds scanned on the support width is too small, and the calculated value may not match the support width, resulting in matching failure. Therefore, a fan-shaped area with a radius of 5m is set.

[0025] Step (2) Initial Position of the Pallet Based on Laser Coarse Positioning: The AGV body collects point cloud information within a fan-shaped area using radars on the left, right, and rear. Based on the parking position, a general detection area is locked. The lengths of each straight line are calculated by fitting the radar point cloud. Using the set lengths of the left and right supports, the length of the middle support, and the span between adjacent supports, a straight line shape meeting the conditions is calculated and matched with the set data. If consecutive left support lengths, spans, middle support lengths, spans, and right support lengths appear, it is considered that a complete pallet has been detected. Then, the initial position of the pallet is calculated based on the angle and the straight line position of the pallet edge. When there is an obstruction within the fan-shaped area, it is necessary to ensure that the range of one fork hole can be identified, that is, 50%. Simultaneously, the width of the middle support must be less than the widths of the two side supports to determine whether the left or right fork hole is identified when only one fork hole is identified, and the initial position of the pallet is calculated accordingly.

[0026] Step (3) First backward movement to the visual recognition position: The controller plans the backward movement curve based on the current positioning position ps of the AGV body and the received initial position pt of the pallet. First, it calculates the Euclidean distance from the current point to the target point. The calculation formula is as follows: , In the formula: ps The vehicle's current location. pt For the target pallet location, pt xLet x be the x-value of the target pallet position. pt y Here is the y-value of the vehicle's current position. ps y Let x be the current position of the vehicle. pt y y is the y-value of the target pallet position, and dis2goal is the Euclidean distance from the vehicle's current position to the target pallet position.

[0027] Based on the distance from the current point to the target point and two set values x_offset_pos_ and x_offset_neg_ Determine the offset distance of the target position and calculate the endpoint of the curve. The endpoint of the curve is calculated as follows: if dis2goal Greater than x_offset_pos_ + x_offset_neg_ ,but px_offset = x_offset_ pos_ ; otherwise px_offset = dis2goal - x_offset_neg_; in: x_offset_pos_ and x_offset_neg_ Two set values ​​are used to calculate the control point offset distance. px_offset This is the base value for the offset distance in the endpoint direction.

[0028] x_offset_pos_ and x_offset_neg_ The settings are intended to ensure the smoothness of the curve's end. These settings need to be measured and configured based on the vehicle's dimensions and to ensure the accuracy of the end-joining. The current vehicle needs to ensure... x_offset_neg_ A depth of 1.0m is needed to ensure a sufficiently smooth curve at the end, allowing the tracking and forking effect to achieve the required accuracy. x_offset_pos_ It refers to the smoothness of the first half of the curve. Ideally, this should be achieved; if not, the first half of the curve must be sacrificed to allow for a better connection in the latter half. x_offset_pos_ The set value is also 1.0m; when both cannot be guaranteed simultaneously, it is reduced. x_offset_pos_ .

[0029] If reversing, the distance will be set to a negative value: dis2goal = - dis2goal px_offset = - px_offset, The coordinates of the curve's endpoint are calculated as follows: p3 x = pt x - px_offset * cos(pt theta ) p3 y = pt y - px_offset * sin(pttheta ) ; In the formula: p3 x The x-coordinate of the endpoint p3 y The y-coordinate of the endpoint; pt theta The angle value at the endpoint, in rad; dis2goal = - dis2goal and px_offset = - px_offset The purpose of the assignment statement in the code is to change the sign of these two values; when reversing, a negative value is needed, while when moving forward, a positive value is needed.

[0030] Calculate the other two control points of the curve: First, calculate the offset distance of the first control point along the starting direction: ps_offset = (dis2goal - px_offset) * 0.3, In the formula, ps_offset The offset distance of the control point closer to the starting point; control point p1 The calculation is as follows: p1 x = ps x + ps_offset * cos(ps theta ) p1 y = ps y + ps_offset * sin(ps theta ) , In the formula: p1 x The x-coordinate of the control point closest to the starting point. p1 y The y-coordinate of the control point closest to the starting point.

[0031] Calculate the offset distance of the second control point along the endpoint direction: pt_offset = (dis2goal - px_offset) * 0.5, In the formula, pt_offset This represents the offset distance of the control point closer to the endpoint. Control point p2 is calculated as follows: p2 x = pt x - pt_offset * cos(pt theta ) p2 y = pt y - pt_offset * sin(pt theta) , In the formula: p2 x The x-coordinate of the control point closest to the endpoint. p2 y The y-coordinate of the control point near the endpoint.

[0032] The controller performs tracking control based on the planned curve and the remaining straight line, controlling the AGV to perform the first straight backward movement or a backward movement including turning, to reach the appropriate initial visual recognition position.

[0033] Step (4) Initial Visual Recognition and Error Adjustment: At the initial visual recognition position, activate the camera to identify the pallet position. Based on the detected position information, calculate the lateral and angular errors of the vehicle's current position as it moves towards the pallet pickup position. If the angle of the initial recognition position is greater than 5° and the lateral deviation is greater than 10cm, directly reversing the forklift to pick it up may not completely eliminate the error to the target accuracy requirement. A forward movement is needed to initially eliminate the angular and lateral errors. When the lateral error is less than 10cm and the angular error is less than 5 degrees, no forward adjustment is required. The forward distance should not exceed 30cm, and this distance is set based on the camera's optimal recognition distance.

[0034] During visual inspection, if the visual recognition matching degree is less than 0.6, the detection result is fused with the center position information calculated by the rear radar detection of the middle support width in the middle of the fork, and the fusion result is given before proceeding with subsequent actions.

[0035] The fusion calculation is based on the fusion of visual recognition results and laser recognition results. When the visual matching degree is lower than 0.6, it is considered that the recognition may not be accurate enough, and the laser recognition position needs to be fused. The specific method is to calculate the deviation based on the endpoint position of the two recognitions. If the deviation is too large, exceeding ±5cm, or the angle exceeds 2°, it is considered that the recognition has failed and the recognition conditions are not met. If the conditions are met, the weights are calculated based on the matching degree, and the recognition results are fused (the camera recognition weight is p, which is the matching degree, 0-1.0, and the laser weight is 1-p).

[0036] Step (5) Secondary visual recognition and second backward movement: Based on the position adjustment results after the initial visual recognition, secondary visual recognition is activated to accurately determine the pallet position again and update the pallet target position information. The controller combines the pallet position information obtained from visual recognition with SLAM positioning information to perform a second backward movement. The speed and angular velocity of the second backward movement are calculated based on the pallet position and SLAM position to reduce the lateral and longitudinal positional deviation between the AGV forks and the pallet.

[0037] The calculation process for velocity and angular velocity is as follows: The controller calculates the speed based on the longitudinal distance of the route, first calculating the projected length of the current point on the path: proj_len = (ps x - pt x ) * cos(pt theta ) + (ps y - pt y ) * sin(pt theta ) ; In the formula: proj_len is the projected length of the vehicle's current position on the route.

[0038] The route has a starting point and an ending point. The projection point is calculated as the projection length of the current point on the tangent direction of the ending point.

[0039] The controller calculates the tracking angular velocity based on the lateral distance of the current position from the route, and first determines the projection point: proj_x = pt x + proj_len * cos(pt theta ) proj_y = pt y + proj_len * sin(pt theta ) In the formula, proj_x is the x-coordinate of the vehicle's current position projected onto the route, and proj_y is the y-coordinate of the vehicle's current position projected onto the route.

[0040] Calculate the lateral deviation: , In the formula, offset_y This represents the lateral deviation of the vehicle's current position relative to the projection point.

[0041] Calculate the angle deviation: offset_theta = ps theta - pt theta , In the formula, offset_theta This represents the angular deviation of the vehicle's current position relative to the projection point.

[0042] The formula for calculating speed is as follows: V x =K pv * offset_y (k) +K dv *(offset_y (k) - offset_y(k-1) ) , In the formula, V x To determine the vehicle's speed.

[0043] The formula for calculating angular velocity is as follows: W = K pw * offset_y (k) + K dw *(offset_y (k) - offset_y (k-1) ) , In the formula, W is the calculated angular velocity of the vehicle.

[0044] offset_y (k) At time k, the lateral deviation offset_y The formula above has already been given. offset_y The calculation method; offset_y (k-1) This is the calculated value at time k-1, which is the value from the previous cycle; The formula is a PID formula, except that it does not use the I term, or the K1 coefficient is 0; K p and K d The value was obtained through continuous debugging during actual use testing. This value may vary slightly for different vehicle models, and the specific value will not be given in the article.

[0045] Step (6) Forklift Detection and Result Judgment: The AGV body reverses at the calculated expected speed. When the forks are fully inserted into the pallet, they will touch the positioning switch. When the distance reaches the positioning switch trigger judgment area, a comprehensive judgment is made based on the remaining distance to the destination and the positioning switch trigger, determining the stopping time for this task and reporting whether the task was successful. Figure 5 The area shown is the reporting area triggered by the position switch.

[0046] Triggering the position switch in the abnormal trigger zone is considered an abnormal trigger, and the task is reported as failed. If the position switch is not triggered within 0.2m beyond the final identification point, the forklift is considered to have failed, and the task is reported as failed. In the normal judgment trigger zone, if the position switch on one side is triggered, it will continue to retreat and attempt to trigger the other side. If the other side is not triggered, the forklift is considered to have failed, and the task is reported as failed. In the normal judgment trigger zone, if both sides are triggered, the task is successful, and the controller controls the forks to rise.

[0047] Step (7) Narrow Channel Operation Adaptation Process: For narrow channel paths, such as Figure 4As shown, the vehicle must not rotate, and should not move forward before successfully picking up the pallet, except for adjustments to prevent errors. Obstacle avoidance is performed within the narrow passage based on laser information from both sides to prevent collisions with walls or objects. When reversing in a straight line, the maximum steering angle is constrained to prevent the vehicle from tilting and touching the pallet. The vehicle reverses along the passage direction, continuously detecting the pallet. If the pallet cannot be detected due to obstructions on both sides, the target position is assisted by dispatch. Once the pallet position can be identified after the vehicle reverses and turns, the initial laser recognition position is switched, and the above logical process is repeated.

[0048] like Figures 1 - 5 As shown, this solution also provides a forklift forking system based on visual recognition of pallet position. The system includes an AGV body, pallets for recognition, and a scheduling module. Specifically: The AGV body includes a controller, radar, forks, a pallet positioning switch, fork tip photoelectric sensors, and a camera. The controller is the core of the AGV body's control system, receiving feedback information from various components and outputting control commands. The radar is installed on the left and right sides and rear of the AGV body, collecting point cloud information within a fan-shaped area and simultaneously providing feedback on the AGV's own position. The forks are used to insert into the pallet to pick up goods, and their structure is adapted to the pallet's fork hole size. The pallet positioning switch is installed at the end of the forks, detecting whether the forks are fully inserted into the pallet and providing real-time feedback on the picking position. The fork tip photoelectric sensors are installed at the tips of the forks, detecting whether the fork tilts or collides with the pallet during the picking process and providing feedback. The camera is installed directly behind the AGV body, identifying the pallet's position information, and its recognition range is adapted to the fan-shaped area and visual recognition position.

[0049] like Figure 3 As shown, the tray has clearly defined left support width, middle support width, right support width, and span characteristics between adjacent supports. The middle support width is smaller than the widths of the two side supports, which facilitates radar and camera identification and positioning.

[0050] The scheduling module is used to send out destination location information, provide target location assistance when the narrow channel tray is completely blocked, and communicate with the controller of the AGV vehicle body.

[0051] The controller is connected to the radar, camera, pallet position switch, fork tip photoelectric sensor and scheduling module via wired or wireless means. The controller adjusts the movement trajectory of the AGV body according to the pallet position information identified by the radar and camera and the AGV position information fed back by the radar, so that the forks can pick up the pallet.

[0052] This system also includes a starting position for the forklift task. The AGV body also includes a center position of the vehicle body and a center position of the pallet. When the center position of the vehicle body is close to or coincides with the starting position of the forklift task, the extension line of the AGV pallet direction is perpendicular to or in the same direction as the straight-line movement direction of the vehicle body. The vehicle body determines and identifies the target position 1 of the pallet based on radar information within the fan-shaped area 1 or 2. Figure 2 As shown, sector 1 and sector 2 are located on both sides of the tray, ensuring that the AGV can enter the effective recognition area when it approaches from different directions.

[0053] The above description is merely illustrative of the embodiments of the present invention and is not intended to limit the present invention. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A forklift loading method based on visual recognition of pallet position, characterized in that, Includes the following steps: S1: The controller combines SLAM positioning information and the destination location information issued by the scheduler to control the AGV to travel to the preset fan-shaped area; S2: The AGV vehicle body collects point cloud information within the fan-shaped area through radar, fits a straight line, matches the left support width, middle support width, right support width and span characteristics between adjacent supports of the pallet, and calculates the initial position of the pallet. S3: The controller calculates the Euclidean distance between the current position of the AGV vehicle and the initial position of the pallet, plans the backward curve, and controls the AGV vehicle to perform the first backward movement to reach the initial visual recognition position. S4: Turn on the camera to perform initial visual recognition, calculate the lateral and angular errors between the AGV body and the tray, and determine whether to advance a preset distance based on the magnitude of the error to initially eliminate the error; S5: Initiate secondary visual recognition to accurately update the pallet position information. The controller combines the SLAM positioning information to calculate the speed and angular velocity of the second backward movement and control the AGV body to move backward. S6: Real-time detection of fork-picking status via pallet position switch and fork tip photoelectric sensor; determination of successful picking based on detection results and reporting.

2. The forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, The radius of the sector area in step S1 is 5 meters and the central angle is 90°. The radius and central angle are adjusted through a configuration file according to the radar model and the field environment.

3. The forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, In step S2, when there is an obstruction in the fan-shaped area, at least 50% of one fork hole of the tray must be identified, and the position of the fork hole is determined by the feature that the width of the middle support is less than the width of the two side supports, and then the initial position of the tray is calculated.

4. The forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, The formula for calculating the Euclidean distance in step S3 is as follows: , in, ps The vehicle's current location. pt For the target pallet position, pt x For the target pallet position x value, ps y The vehicle's current location y value, ps x The vehicle's current location x value, pt y For the target pallet position y value.

5. A forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, When the lateral error is greater than 10cm or the angular error is greater than 5° in step S4, the forward distance of the AGV vehicle body shall not exceed 30cm; the forward distance is set according to the optimal recognition distance of the camera.

6. A forklift loading method based on visual recognition of pallet position according to claim 1 or 5, characterized in that, When the matching degree of the initial visual recognition in step S4 is less than 0.6, the visual recognition result is fused with the center position information of the middle support width detected by the rear radar in the middle of the fork to obtain the corrected pallet position.

7. A forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, The formula for calculating the speed mentioned in step S5 is: V x =K p *offset_y (k) +k d *(offset_y (k) -offset_y (k-1) )， The formula for calculating angular velocity is: W = K p *offset_y (k) +k d *(offset_y (k) -offset_y (k-1) )， in, offset_y This represents the lateral deviation of the vehicle's current position relative to the projection point. K p , K d These are the proportional coefficient and the differential coefficient.

8. A forklift loading method based on visual recognition of pallet position according to claim 1, characterized in that, When the AGV is operating in a narrow passage, it is prohibited to perform rotation. Except for the forward movement required for error adjustment, it can only move backward along the passage direction and the maximum turning angle is constrained. At the same time, it can avoid obstacles based on the radar information on both sides.

9. A forklift loading system based on visual recognition of pallet position, characterized in that, It includes an AGV body, an identifiable pallet, and a scheduling module; the AGV body includes a controller, radar, forks, pallet position switch, fork tip photoelectric sensor, and camera; The radar is used to collect point cloud information within a fan-shaped area and feed back the position information of the AGV vehicle body; The camera is used to identify tray position information; The pallet positioning switch is used to detect whether the fork has been properly positioned. The fork tip photoelectric sensor is used to detect whether the fork is tilted or colliding. The controller is communicatively connected to the radar, the camera, the pallet positioning switch, the fork tip photoelectric sensor, and the scheduling module, respectively, and is used to receive position information and adjust the movement trajectory of the AGV vehicle.

10. A forklift loading system based on visual recognition of pallet position according to claim 9, characterized in that, The triggering determination area of ​​the pallet positioning switch includes a normal judgment triggering area and an abnormal triggering area. The normal judgment triggering area is a range of 0.2 meters on both sides of the fork-taking point, and the abnormal triggering area is outside the normal judgment triggering area. When both pallet positioning switches on both sides are triggered in the normal judgment triggering area, the fork-taking is determined to be successful.

Images