A positioning and navigation method and system for AGV forklifts based on vision recognition

Abstract

This application relates to a positioning and navigation method and system for AGV forklifts based on visual recognition, belonging to the field of forklift intelligent control technology. The method includes acquiring a visual recognition image; determining whether external obstacle features exist based on the visual recognition image; if external obstacle features exist, determining the external obstacle point based on the external obstacle features, and determining the required travel segment based on the forklift's current position and end position; determining whether the external obstacle point is on the required travel segment; if the external obstacle point is on the required travel segment, determining the required avoidance point based on the external obstacle features, and constructing a path replacement segment based on the forklift's current position, the required avoidance point, and the end position; and updating the required travel segment with the path replacement segment to control the forklift to move along the updated forklift travel path. This application facilitates the forklift's execution of transportation tasks.

Description

Technical Field

[0001] This application relates to the field of intelligent control technology for forklifts, and in particular to a positioning and navigation method and system for AGV forklifts based on vision recognition. Background Technology

[0002] AGV forklifts are the application of Automated Guided Vehicle (AGV) technology in the forklift field. They are forklift vehicles capable of autonomous driving, requiring no human operator. Instead, they complete material handling tasks through pre-programmed procedures, navigation systems, and sensors, thus finding widespread application in logistics warehousing, manufacturing workshops, and other fields. Among these applications, visual recognition technology, with its advantages of low cost, high adaptability, and rich information capabilities, provides effective technical support for AGV forklifts.

[0003] In related technologies, AGV forklifts generally apply visual recognition technology to obstacle avoidance. They use visual sensors to detect obstacles on the driving path, and when a corresponding obstacle is detected, they take corresponding avoidance measures to bypass the obstacle.

[0004] In the aforementioned technologies, when the AGV forklift is close to an obstacle to avoid it, the forklift needs to adjust its movement significantly. This not only requires waiting for the forklift to adjust its movement, but also makes the overall movement of the forklift difficult, thus making it inconvenient for the forklift to perform transportation tasks. There is still room for improvement. Summary of the Invention

[0005] To facilitate forklifts in performing transportation tasks, this application provides a positioning and navigation method and system for AGV forklifts based on vision recognition.

[0006] Firstly, this application provides a positioning and navigation method for AGV forklifts based on visual recognition, employing the following technical solution:

[0007] A vision-based positioning and navigation method for AGV forklifts includes:

[0008] Acquire visual recognition images, forklift travel path, and forklift current position;

[0009] According to the preset transportation space map, the forklift travel path is divided into several straight travel segments, and the straight travel segment containing the current position of the forklift is defined as the current segment, and the end position point and the direction of movement of the segment are determined on the current segment.

[0010] Based on the forklift's current position and the direction of movement of the road segment, determine the image of the unobstructed road segment from the preset image record library;

[0011] The images of unobstructed road sections are compared with visual recognition images to determine whether there are external obstacles.

[0012] If there are no external obstacles, control the forklift to continue moving along the current road segment;

[0013] If there are external obstacle features, the external obstacle point is determined based on the external obstacle features, and the required travel route is determined based on the current position and the end position of the forklift.

[0014] Determine whether the external obstacle is located on the required driving section of the road;

[0015] If the external obstacle is not on the required travel segment, control the forklift to continue moving along the current segment.

[0016] If an external obstacle is located on the required travel segment, the required avoidance point is determined based on the characteristics of the external obstacle. A path replacement segment is constructed based on the current position of the forklift, the required avoidance point, and the end position. The path replacement segment is then used to replace and update the required travel segment to control the forklift to move along the updated forklift travel path.

[0017] Optionally, if the external obstacle is located on the required travel section, the vision-based AGV forklift positioning and navigation method also includes:

[0018] Establish a unit interval with a preset unit duration on the preset timeline, with the current time point as the starting point;

[0019] Control the forklift to continue moving along the current road segment within the unit area and continuously acquire visual recognition images, and determine the external obstacle points corresponding to the external obstacle features based on the visual recognition images;

[0020] Determine the obstacle movement distance based on any two external obstacle points within the unit interval;

[0021] Determine if there is a situation where the obstacle's movement distance exceeds the preset action distance;

[0022] If there is no obstacle movement distance greater than the action distance, then determine the required avoidance point based on the characteristics of the external obstacle after the unit interval, and update the forklift travel path;

[0023] If there is an obstacle that needs to be moved further than the movement distance, then control the forklift to continue moving along the current road segment.

[0024] Optionally, if there are situations where the obstacle movement distance is greater than the movement distance, the positioning and navigation methods for AGV forklifts based on vision recognition may include:

[0025] Determine whether the external obstacle features are the preset transport forklift features;

[0026] If the external obstacle features are not those of a transport forklift, then control the current forklift to continue moving along the current road segment.

[0027] If the external obstacle feature is a transport forklift feature, then the forklift location of each transport forklift is obtained from the preset task scheduling library;

[0028] The distance between points is determined by calculating the distance between the points based on the location of the forklift and the external obstacle points. The distance between the points with the smallest value is determined according to the preset sorting rules, and the transport forklift corresponding to the distance between the points with the smallest value is defined as the obstructing forklift.

[0029] The task scheduling library determines the obstructing task path based on the obstructing forklift, and determines the real-time location of the obstructing forklift at each time point based on the obstructing task path.

[0030] Determine the simulated location of the forklift at each time point based on the forklift's travel path and current position.

[0031] Determine if there is a situation where real-time existing points and simulated existing points overlap in the current road segment;

[0032] If there is no situation where the real-time existing point and the simulated existing point overlap on the current road segment, then control the current forklift to continue moving along the current road segment;

[0033] If there is a situation where the real-time existing point and the simulated existing point overlap on the current road segment, the required avoidance point is determined based on the characteristics of the transport forklift.

[0034] Optionally, after the distance between points is determined, the vision-based AGV forklift positioning and navigation method also includes:

[0035] Determine if there exist at least two forklift locations that are equidistant from each other and have the smallest possible distance.

[0036] If there are no two forklift locations that are equidistant from each other and have the smallest possible distance, then the forklift location corresponding to the smallest possible distance is used to determine the obstructing forklift.

[0037] If there are at least two forklift locations that are equidistant from each other and have the smallest possible distance, then the forklift execution posture of each forklift location is obtained from the task scheduling library, and the forklift obstruction posture is determined based on the characteristics of the transport forklift.

[0038] The forklift's executing posture and its obstructing posture are compared to determine the posture similarity ratio;

[0039] The maximum posture similarity ratio is determined according to the sorting rules, and the obstructing forklift is determined based on the forklift presence point corresponding to the maximum posture similarity ratio.

[0040] Optionally, the steps for determining the required avoidance point based on the characteristics of the transport forklift include:

[0041] The point where the real-time existing point and the simulated existing point overlap on the current road segment is defined as the meeting point;

[0042] Obtain the model and specifications of the forklift that is obstructing the forklift;

[0043] The required clearance width corresponding to the forklift model and specifications is determined based on the preset parameter matching relationship;

[0044] Based on the current position of the forklift and the meeting point, construct a virtual moving line segment, and at the meeting point, construct a straight line perpendicular to the virtual moving line segment;

[0045] The required avoidance width is determined by moving the meeting point in any direction along the avoidance line.

[0046] Optionally, after the avoidance line is determined, the vision recognition-based positioning and navigation method for AGV forklifts also includes:

[0047] Determine whether the ending position coincides with the path endpoint on the forklift's travel path;

[0048] If the ending point coincides with the endpoint of the forklift's travel path, the meeting point will be moved in any direction along the avoidance line by the required avoidance width to determine the required avoidance point.

[0049] If the ending point does not coincide with the endpoint of the forklift's travel path, the straight travel segment following the current segment on the forklift's travel path is defined as the subsequent travel segment, and the subsequent travel direction is determined based on the subsequent travel segment.

[0050] The required avoidance width is moved in two directions from the meeting point to the avoidance line to determine the simulated avoidance point, and the direction of the subsequent avoidance is determined based on the simulated avoidance point and the ending position point;

[0051] The directional angle is determined based on the subsequent driving direction and the direction of the avoidance section. The directional angle with the smallest value is determined according to the sorting rules, and the simulated avoidance point corresponding to the directional angle with the smallest value is defined as the required avoidance point.

[0052] Secondly, this application provides a positioning and navigation system for AGV forklifts based on vision recognition, employing the following technical solution:

[0053] A positioning and navigation system for AGV forklifts based on vision recognition, comprising:

[0054] The acquisition module is used to acquire visual recognition images, forklift travel paths, and the current position of the forklift;

[0055] The processing module, connected to the acquisition and judgment modules, is used for information storage and processing;

[0056] The judgment module, connected to the acquisition and processing modules, is used for judging information.

[0057] The processing module divides the forklift travel path into several straight travel segments according to the preset transportation space map, defines the straight travel segment containing the current position of the forklift as the current segment, and determines the end position point and the direction of movement of the segment on the current segment.

[0058] The processing module determines the image of the unobstructed road section from the preset image recording library based on the current position of the forklift and the direction of road movement;

[0059] The processing module compares the features of the unobstructed road segment image with the visual recognition image so that the judgment module can determine whether there are external obstacles.

[0060] If the judgment module determines that there are no external obstacles, the processing module controls the current forklift to continue moving along the current road segment.

[0061] If the judgment module determines that there are external obstacle features, the processing module determines the external obstacle point based on the external obstacle features, and determines the required travel route based on the forklift's current position and end position.

[0062] The judgment module determines whether the external obstacle point is located on the required driving section;

[0063] If the judgment module determines that the external obstacle is not on the required driving section, the processing module controls the current forklift to continue moving along the current road section.

[0064] If the judgment module determines that the external obstacle is on the required driving section, the processing module determines the required avoidance point based on the characteristics of the external obstacle, and constructs a path replacement section based on the current position of the forklift, the required avoidance point, and the end position point. The path replacement section is then used to replace and update the required driving section to control the forklift to move along the updated forklift driving path.

[0065] Thirdly, this application provides a computer storage medium capable of storing corresponding programs, which facilitates forklifts in performing transportation tasks, and adopts the following technical solution:

[0066] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the above-described vision-based AGV forklift positioning and navigation methods.

[0067] In summary, this application includes at least one of the following beneficial technical effects:

[0068] 1. During the movement of the forklift, the situation of obstacles in front can be determined through visual recognition technology. When an obstacle is detected at a distance, the forklift can immediately adjust its movement angle. At this time, the forklift can avoid the obstacle more easily, which facilitates the forklift to perform the transportation task.

[0069] 2. When analyzing obstacles, the situation of two forklifts meeting can be comprehensively analyzed to enable forklift avoidance control in special situations. Attached Figure Description

[0070] Figure 1 This is a flowchart of a vision-based AGV forklift positioning and navigation method.

[0071] Figure 2 This is a flowchart of the module for a vision-based AGV forklift positioning and navigation method. Detailed Implementation

[0072] To make the purpose, technical solution, and advantages of this application clearer, the following is combined with Figures 1-2 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0073] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0074] This application discloses a positioning and navigation method for AGV forklifts based on vision recognition, referring to... Figure 1 The method flow of the vision recognition-based AGV forklift positioning and navigation method includes the following steps:

[0075] Step S100: Acquire the visual recognition image, the forklift travel path, and the current position of the forklift.

[0076] The visual recognition image is the image acquired by a visual sensor installed on the forklift to acquire images of the road in front of the forklift. The forklift travel path is the travel path that the forklift needs to move when performing the current task. It can be determined by analyzing the forklift task. The forklift's current position is the position point of the forklift when acquiring the visual recognition image, which can be obtained by installing a positioning device on the forklift.

[0077] Step S101: Divide the forklift travel path into several straight travel segments according to the preset transportation space map, define the straight travel segment containing the current position of the forklift as the current segment, and determine the end position point and the direction of movement of the segment on the current segment.

[0078] The transport space map is a spatial diagram of the space where the forklift is currently located, such as a warehouse distribution map. This transport space map records the relative positions of each rack and road. The straight travel segment is the straight segment in the forklift's travel path. The current segment is defined to determine the segment the forklift is currently moving on, which is convenient for analyzing obstacle situations. The end position point is the endpoint on the current segment that the forklift has not yet moved to. The segment movement direction is the direction in which the forklift moves when it is on the current segment.

[0079] Step S102: Determine the unobstructed road segment image from the preset image recording library based on the current position of the forklift and the direction of road movement.

[0080] The image recording library is a database that stores images of various locations. The images include those that can be acquired when the forklift is at any position and facing any direction, and there are no obstacles in front of it in the space corresponding to the transport space map. The images of unobstructed road sections are those that can be acquired when the forklift is at its current position and moving in the direction of the road section, and there are no obstacles in front of it.

[0081] Step S103: Compare the features of the unobstructed road section image with the visual recognition image to determine whether there are external obstacle features.

[0082] The feature comparison method is to visually identify whether there are features in the image that are not present in the image of a smooth road section. These features are external obstacle features. The purpose of the judgment is to determine whether there are obstacles that may obstruct the current forklift.

[0083] Step S1031: If there are no external obstacles, control the current forklift to continue moving along the current road segment.

[0084] When there are no external obstacles, it means there are no obstacles in front of the forklift, and you can continue to move the forklift.

[0085] Step S1032: If there are external obstacle features, determine the external obstacle point based on the external obstacle features, and determine the required travel route based on the current position and end position of the forklift.

[0086] When external obstacle features are present, it indicates that there may be obstacles that will hinder the current forklift, and further analysis is required. The external obstacle point is the location of the external obstacle feature. The distance and relative position between the external obstacle feature and the forklift can be determined by the acquisition of the external obstacle feature in the image, so as to effectively identify the external obstacle point. The required travel segment is the route that the current forklift needs to travel from the current position to the end position.

[0087] Step S104: Determine whether the external obstacle is located on the required driving section.

[0088] The purpose of the judgment is to determine whether the obstacle ahead will obstruct the forklift.

[0089] Step S1041: If the external obstacle is not on the required travel segment, control the current forklift to continue moving along the current travel segment.

[0090] When the external obstacle is not located on the required driving section, it means that the obstacle in front will not obstruct the current forklift. At this time, you can control the forklift to move normally.

[0091] Step S1042: If the external obstacle is located on the required travel segment, the required avoidance point is determined based on the characteristics of the external obstacle. A path replacement segment is constructed based on the current position of the forklift, the required avoidance point, and the end position. The path replacement segment is then used to replace and update the required travel segment to control the forklift to move along the updated forklift travel path.

[0092] When an external obstacle is located on the required travel segment, it indicates that the obstacle will obstruct the forklift, requiring a avoidance maneuver. The required avoidance point is the position the forklift needs to move to avoid the obstacle. This can be determined by analyzing the external obstacle's width in the current road segment and moving the obstacle point by the corresponding width value in the width direction. The method for determining this avoidance point is similar to steps S600-S602 and will not be elaborated here. The path replacement segment is the segment that allows the forklift to avoid external obstacle features. This segment involves moving the forklift from its current position to the required avoidance point and then to the final position. All movements between points are linear. Updating the forklift's travel path using the path replacement segment allows the forklift to move normally. When the forklift detects an obstacle at a relatively far distance, it only needs to adjust a small movement angle to avoid the obstacle, making adjustments easier and facilitating the forklift's execution of transport tasks.

[0093] If the external obstacle is located on the required travel section, the vision recognition-based positioning and navigation method for AGV forklifts also includes:

[0094] Step S200: On the preset timeline, establish a unit interval with a preset unit duration, with the current time point as the starting point.

[0095] When an external obstacle is located on the required driving section of the road, it may be a worker. In this case, the detected obstruction is caused by the worker's movement and will not cause continuous obstruction. Therefore, no avoidance operation is required in this situation, but analysis is needed. The time axis is a coordinate axis formed by the combination of various time points. In this coordinate axis, the left side is defined as the front and the right side is defined as the rear. The unit time is a fixed value set by the worker. Establishing unit intervals can facilitate the determination of the situation of external obstacles within a unit time, which is convenient for subsequent analysis.

[0096] Step S201: Control the forklift to continue moving along the current road segment within the unit interval and continuously acquire visual recognition images, and determine the external obstacle points corresponding to the external obstacle features based on the visual recognition images.

[0097] Controlling the forklift to continue moving and using visual recognition can acquire information about external obstacles, facilitating subsequent analysis.

[0098] Step S202: Determine the obstacle movement distance based on any two external obstacle points within the unit interval.

[0099] The obstacle movement distance is the straight-line distance between two external obstacle points.

[0100] Step S203: Determine if there is a situation where the obstacle movement distance is greater than the preset action distance.

[0101] The action distance is the minimum obstacle movement distance that must be reached when the obstacle is identified and determined by the staff. The purpose of the judgment is to know whether there is an obstacle movement, so as to determine whether the current obstacle will continue to obstruct the current forklift.

[0102] Step S2031: If there is no obstacle movement distance greater than the action distance, then determine the required avoidance point based on the characteristics of the external obstacle after the unit interval, and update the forklift travel path.

[0103] If there is no situation where the obstacle moves a distance greater than the action distance, it means that there is no situation where the obstacle will move away on its own. In this case, the forklift can perform the normal avoidance operation.

[0104] Step S2032: If there is an obstacle that moves a distance greater than the action distance, then control the current forklift to continue moving along the current road segment.

[0105] When the obstacle's movement distance is greater than the action distance, it indicates that the obstacle ahead is movable. In this case, the obstacle may move on its own when the forklift approaches it. Therefore, normal control of the forklift movement without performing an avoidance operation is sufficient. Note that this forklift also has a forced avoidance mechanism. When the distance between the forklift and an external obstacle is less than a preset approximate distance value, the forklift will immediately perform an avoidance operation. The specific measures for forced avoidance are set by the staff according to the actual situation and are not a point of this application, so they will not be elaborated here.

[0106] If there are situations where the obstacle movement distance is greater than the action distance, visual recognition-based positioning and navigation methods for AGV forklifts may include:

[0107] Step S300: Determine whether the external obstacle features are preset transport forklift features.

[0108] The transport forklift features are the features that can be detected when the forklift acts as an obstacle. The purpose of the determination is to determine whether the currently movable external obstacle is another forklift being transported within the transport space.

[0109] Step S3001: If the external obstacle feature is not a transport forklift feature, then control the current forklift to continue moving along the current road segment.

[0110] When the external obstacle features are not those of a transport forklift, it means that the external obstacle is not a forklift. In this case, you can control the current forklift to move normally.

[0111] Step S3002: If the external obstacle feature is a transport forklift feature, then obtain the forklift location of each transport forklift from the preset task scheduling library.

[0112] When the external obstacle feature is a forklift feature, it means that the external obstacle is a forklift. In this case, there may be a situation where the subsequent movement of the forklift will still hinder the current movement of the forklift, which requires further analysis. The forklift existence point is the theoretical position that each forklift performing its corresponding task should be at the current time. The task scheduling database is a database that records the forklifts and their corresponding tasks. The tasks to be executed include the path to be moved and the corresponding movement rules, including driving speed, turning speed, etc.

[0113] Step S301: Calculate the distance between points based on the location of the forklift and the external obstacle points, and determine the distance between points with the smallest value according to the preset sorting rules. Define the transport forklift corresponding to the distance between points with the smallest value as the obstructing forklift.

[0114] The distance between points is the distance between the point where the forklift is located and the external obstacle point. The sorting rule is a method set by the staff to sort the values, such as the bubble sort method. The sorting rule can determine the distance between points with the smallest value, which means that the forklift at the point where the forklift is located is the forklift that is obstructing the current forklift. It is defined as the obstructing forklift to distinguish different transport forklifts and facilitate subsequent analysis.

[0115] Step S302: Determine the obstructing task path in the task scheduling library based on the obstructing forklift, and determine the real-time location of the obstructing forklift at each time point based on the obstructing task path.

[0116] The obstruction path is the path that prevents the forklift from traveling while performing the current task. The real-time point is the location that the obstruction forklift will reach at each point in time while performing the current task.

[0117] Step S303: Determine the simulated location of the forklift at each time point based on the forklift's travel path and current position.

[0118] The simulated point of existence is the position that the forklift will be at at each time point after it departs from its current position.

[0119] Step S304: Determine whether there is a situation where the real-time existing point and the simulated existing point overlap on the current road segment.

[0120] The purpose of the judgment is to determine whether the forklifts will meet, and thus whether it is necessary to take evasive action in advance.

[0121] Step S3041: If there is no situation where the real-time existing point and the simulated existing point overlap on the current road segment, then control the current forklift to continue moving along the current road segment.

[0122] When there is no overlap between the real-time and simulated points on the current road segment, it means that the forklift will not meet on the current road segment. In this case, the forklift can be controlled to move normally.

[0123] Step S3042: If there is a situation where the real-time existing point and the simulated existing point overlap on the current road segment, then the required avoidance point is determined based on the characteristics of the transport forklift.

[0124] When a real-time location point and a simulated location point overlap on the current road segment, it indicates that forklifts will encounter each other on the current road segment. In this case, advance avoidance is required. Therefore, the corresponding required avoidance point can be determined based on the characteristics of the transport forklift for subsequent avoidance operations. The method for determining the required avoidance point is the method described in steps S600-S602.

[0125] Once the distance between points is determined, the positioning and navigation method for AGV forklifts based on vision recognition also includes:

[0126] Step S400: Determine whether there exist at least two forklift locations that are equidistant from each other and have the smallest possible distance.

[0127] The purpose of the judgment is to determine whether there are multiple forklifts that roughly match the current obstacle, so as to more accurately identify the single obstructing forklift.

[0128] Step S4001: If there are no two forklift locations that are equidistant from each other and have the smallest possible distance, then determine the forklift location that is obstructing the forklift based on the forklift location corresponding to the smallest possible distance.

[0129] When there are no two forklift locations that are equidistant from each other and have the smallest possible distance, it means that there is only one forklift that meets the requirements. In this case, the normal process of identifying the obstructing forklift can be performed.

[0130] Step S4002: If there are at least two forklift locations that are equidistant from each other and have the smallest possible distance, then obtain the forklift execution posture of the forklift corresponding to each forklift location from the task scheduling library, and determine the forklift obstruction posture based on the characteristics of the transport forklift.

[0131] When there are at least two forklift locations that are equidistant from each other and have the smallest possible distance, it indicates that multiple forklifts are close to the external obstacle point and could all be forklifts obstructing the current forklift, requiring further analysis. The forklift execution posture is the movement posture of the forklift at the determined forklift location, which includes the direction of movement, the height of the goods lifted by the forklift, etc. The forklift obstruction posture is the movement posture of the acquired transport forklift characteristics, which also includes the direction of movement, the height of the goods lifted by the forklift, etc.

[0132] Step S401: Compare the forklift's executing posture and the forklift's obstructing posture to determine the posture similarity ratio.

[0133] The posture similarity ratio is the degree of similarity between two forklifts. The higher the overlap between the forklift's executing posture and its obstructing posture, the greater the corresponding posture similarity ratio.

[0134] Step S402: Determine the posture similarity ratio with the largest value according to the sorting rules, and determine the obstructing forklift based on the forklift presence point corresponding to the posture similarity ratio with the largest value.

[0135] The sorting rules can be used to determine the posture similarity ratio with the highest value. That is, the forklift with the highest posture similarity ratio is most similar to the external obstacle features, which means that the forklift is the obstructing forklift.

[0136] The steps to determine the required avoidance point based on the characteristics of transport forklifts include:

[0137] Step S500: Define the point where the real-time existing point and the simulated existing point overlap on the current road segment as the meeting point.

[0138] Define meeting points to mark the locations where forklifts may meet, facilitating subsequent analysis.

[0139] Step S501: Obtain the model and specifications of the forklift that is obstructing the forklift.

[0140] The forklift model and specifications refer to the forklift model that obstructs the forklift.

[0141] Step S502: Determine the required clearance width corresponding to the forklift model and specifications based on the preset parameter matching relationship.

[0142] The required clearance width is the width value that needs to be moved in the width direction when a forklift of a certain model and specification needs to make way for another forklift. Different forklift models and specifications have different forklift parameters, so the corresponding required clearance width is also different. The parameter matching relationship between the two is entered and stored in advance by the staff.

[0143] Step S503: Construct a virtual moving line segment based on the current position of the forklift and the meeting point, and construct an avoidance line perpendicular to the virtual moving line segment at the meeting point.

[0144] The virtual moving line segment is a line segment formed with the current position of the forklift and the meeting point as its two endpoints. The avoidance line is the straight line along the path that the forklift needs to move when performing an avoidance operation.

[0145] Step S504: Move the meeting point in any direction along the avoidance line by the required avoidance width to determine the required avoidance point.

[0146] By moving the meeting point along the avoidance line and the required avoidance width, the position point where the forklift can effectively avoid obstacles can be determined. The required avoidance point determined at this time can meet the forklift avoidance requirements.

[0147] After the avoidance line is determined, the vision recognition-based positioning and navigation method for AGV forklifts also includes:

[0148] Step S600: Determine whether the end position point coincides with the path endpoint on the forklift's travel path.

[0149] The purpose of the judgment is to determine whether the current road segment is the last straight section, that is, to determine whether the forklift needs to make a turning operation.

[0150] Step S6001: If the ending position coincides with the endpoint of the forklift travel path, the meeting point is moved in any direction along the avoidance line by the required avoidance width to determine the required avoidance point.

[0151] When the ending point coincides with the endpoint of the forklift's travel path, it means that the subsequent forklift will not be turning. At this time, you can randomly select a direction to determine the required avoidance point.

[0152] Step S6002: If the end point does not coincide with the endpoint of the forklift's travel path, the straight travel segment following the current segment on the forklift's travel path is defined as the subsequent travel segment, and the subsequent travel direction is determined based on the subsequent travel segment.

[0153] When the end point does not coincide with the endpoint of the forklift's travel path, it means that the forklift still needs to move to the next straight travel segment. At this time, the forklift still needs to turn after reaching the end point. Therefore, how to facilitate the forklift to turn at the end point is the problem that needs to be solved, which requires further analysis. Define the subsequent travel segment to determine the execution travel segment that needs to be turned later. The subsequent travel direction is the direction of movement of the forklift when it moves on the subsequent travel segment.

[0154] Step S601: Move the meeting point in both directions of the avoidance line by the required avoidance width to determine the simulated avoidance point, and determine the direction of the subsequent avoidance section based on the simulated avoidance point and the ending position point.

[0155] The simulated avoidance point is determined to mark the locations that may be required to avoid the forklift, which will facilitate subsequent analysis. The avoidance rear direction is the direction in which the forklift moves after reaching the simulated avoidance point, that is, the direction from the simulated avoidance point to the end position point.

[0156] Step S602: Determine the directional angle based on the subsequent driving direction and the direction of the avoidance section, and determine the directional angle with the smallest value according to the sorting rules, and define the simulated avoidance point corresponding to the directional angle with the smallest value as the required avoidance point.

[0157] The directional angle is the angle between the subsequent driving direction and the direction of the avoidance section. The directional angle with the smallest value can be determined by the sorting rules. That is, the angle that the forklift needs to adjust when it reaches the end position is the smallest, which means that the adjustment difficulty of the forklift is the smallest. At this time, the corresponding simulated avoidance point is defined as the required avoidance point so that the forklift can perform subsequent movement control.

[0158] Reference Figure 2 Based on the same inventive concept, embodiments of the present invention provide a positioning and navigation system for AGV forklifts based on visual recognition, comprising:

[0159] The acquisition module is used to acquire visual recognition images, forklift travel paths, and the current position of the forklift;

[0160] The processing module, connected to the acquisition and judgment modules, is used for information storage and processing;

[0161] The judgment module, connected to the acquisition and processing modules, is used for judging information.

[0162] The processing module divides the forklift travel path into several straight travel segments according to the preset transportation space map, defines the straight travel segment containing the current position of the forklift as the current segment, and determines the end position point and the direction of movement of the segment on the current segment.

[0163] The processing module determines the image of the unobstructed road section from the preset image recording library based on the current position of the forklift and the direction of road movement;

[0164] The processing module compares the features of the unobstructed road segment image with the visual recognition image so that the judgment module can determine whether there are external obstacles.

[0165] If the judgment module determines that there are no external obstacles, the processing module controls the current forklift to continue moving along the current road segment.

[0166] If the judgment module determines that there are external obstacle features, the processing module determines the external obstacle point based on the external obstacle features, and determines the required travel route based on the forklift's current position and end position.

[0167] The judgment module determines whether the external obstacle point is located on the required driving section;

[0168] If the judgment module determines that the external obstacle is not on the required driving section, the processing module controls the current forklift to continue moving along the current road section.

[0169] If the judgment module determines that the external obstacle point is on the required driving section, the processing module determines the required avoidance point based on the characteristics of the external obstacle, and constructs a path replacement section based on the current position of the forklift, the required avoidance point and the end position point. The path replacement section replaces and updates the required driving section to control the forklift to move along the updated forklift driving path.

[0170] The obstacle analysis module is used to analyze and process the specific situation of the obstacles in front.

[0171] The forklift encounter analysis module is used to analyze and process situations where the obstacle ahead is another forklift.

[0172] The obstructing forklift screening module is used to screen multiple forklifts that meet the requirements to determine the unique obstructing forklift;

[0173] The demand avoidance point determination module is used to determine the demand avoidance point so that the forklift can effectively perform avoidance handling.

[0174] The demand avoidance point precision module is used to determine a more suitable demand avoidance point so that the forklift can be moved and controlled more easily afterward.

[0175] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0176] This invention provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as a positioning and navigation method for a vision-based AGV forklift.

[0177] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

Claims

1. A positioning and navigation method for an AGV forklift based on vision recognition, characterized in that, include: Acquire visual recognition images, forklift travel path, and forklift current position; According to the preset transportation space map, the forklift travel path is divided into several straight travel segments, and the straight travel segment containing the current position of the forklift is defined as the current segment, and the end position point and the direction of movement of the segment are determined on the current segment. Based on the forklift's current position and the direction of movement of the road segment, determine the image of the unobstructed road segment from the preset image record library; The images of unobstructed road sections are compared with visual recognition images to determine whether there are external obstacles. If there are no external obstacles, control the forklift to continue moving along the current road segment; If there are external obstacle features, the external obstacle point is determined based on the external obstacle features, and the required travel route is determined based on the current position and the end position of the forklift. Determine whether the external obstacle is located on the required driving section of the road; If the external obstacle is not on the required travel segment, control the forklift to continue moving along the current segment. If the external obstacle is located on the required travel segment, the required avoidance point is determined based on the characteristics of the external obstacle. A path replacement segment is constructed based on the current position of the forklift, the required avoidance point, and the end position. The path replacement segment is then used to replace and update the required travel segment to control the forklift to move along the updated forklift travel path. If the external obstacle is located on the required travel section, the vision recognition-based positioning and navigation method for AGV forklifts also includes: Establish a unit interval with a preset unit duration on the preset timeline, with the current time point as the starting point; Control the forklift to continue moving along the current road segment within the unit area and continuously acquire visual recognition images, and determine the external obstacle points corresponding to the external obstacle features based on the visual recognition images; Determine the obstacle movement distance based on any two external obstacle points within the unit interval; Determine if there is a situation where the obstacle's movement distance exceeds the preset action distance; If there is no obstacle movement distance greater than the action distance, then determine the required avoidance point based on the characteristics of the external obstacle after the unit interval, and update the forklift travel path; If there is an obstacle that needs to be moved further than the action distance, then control the forklift to continue moving along the current road segment. If the obstacle movement distance is greater than the movement distance, the positioning and navigation method for AGV forklifts based on vision recognition also includes: Determine whether the external obstacle features are the preset transport forklift features; If the external obstacle features are not those of a transport forklift, then control the current forklift to continue moving along the current road segment. If the external obstacle feature is a transport forklift feature, then the forklift location of each transport forklift is obtained from the preset task scheduling library; The distance between points is determined by calculating the distance between the points based on the location of the forklift and the external obstacle points. The distance between the points with the smallest value is determined according to the preset sorting rules, and the transport forklift corresponding to the distance between the points with the smallest value is defined as the obstructing forklift. The task scheduling library determines the obstructing task path based on the obstructing forklift, and determines the real-time location of the obstructing forklift at each time point based on the obstructing task path. Determine the simulated location of the forklift at each time point based on the forklift's travel path and current position. Determine if there is a situation where real-time existing points and simulated existing points overlap in the current road segment; If there is no situation where the real-time existing point and the simulated existing point overlap on the current road segment, then control the current forklift to continue moving along the current road segment; If there is a situation where the real-time existing point and the simulated existing point overlap on the current road segment, the required avoidance point is determined based on the characteristics of the transport forklift; Once the distance between points is determined, the positioning and navigation method for AGV forklifts based on vision recognition also includes: Determine if there exist at least two forklift locations that are equidistant from each other and have the smallest possible distance. If there are no two forklift locations that are equidistant from each other and have the smallest possible distance, then the forklift location corresponding to the smallest possible distance is used to determine the obstructing forklift. If there are at least two forklift locations that are equidistant from each other and have the smallest possible distance, then the forklift execution posture of each forklift location is obtained from the task scheduling library, and the forklift obstruction posture is determined based on the characteristics of the transport forklift. The forklift's executing posture and its obstructing posture are compared to determine the posture similarity ratio; The maximum pose similarity ratio is determined according to the sorting rules, and the obstructing forklift is determined according to the forklift presence point corresponding to the maximum pose similarity ratio. The steps to determine the required avoidance point based on the characteristics of transport forklifts include: The point where the real-time existing point and the simulated existing point overlap on the current road segment is defined as the meeting point; Obtain the model and specifications of the forklift that is obstructing the forklift; The required clearance width corresponding to the forklift model and specifications is determined based on the preset parameter matching relationship; Based on the current position of the forklift and the meeting point, construct a virtual moving line segment, and at the meeting point, construct a straight line perpendicular to the virtual moving line segment; The required avoidance width is determined by moving the meeting point in any direction along the avoidance line.

2. The positioning and navigation method for AGV forklifts based on vision recognition according to claim 1, characterized in that, After the avoidance line is determined, the vision recognition-based positioning and navigation method for AGV forklifts also includes: Determine whether the ending position coincides with the path endpoint on the forklift's travel path; If the ending point coincides with the endpoint of the forklift's travel path, the meeting point will be moved in any direction along the avoidance line by the required avoidance width to determine the required avoidance point. If the ending point does not coincide with the endpoint of the forklift's travel path, the straight travel segment following the current segment on the forklift's travel path is defined as the subsequent travel segment, and the subsequent travel direction is determined based on the subsequent travel segment. The required avoidance width is moved in two directions from the meeting point to the avoidance line to determine the simulated avoidance point, and the direction of the subsequent avoidance is determined based on the simulated avoidance point and the ending position point; The directional angle is determined based on the subsequent driving direction and the direction of the avoidance section. The directional angle with the smallest value is determined according to the sorting rules, and the simulated avoidance point corresponding to the directional angle with the smallest value is defined as the required avoidance point.

3. A positioning and navigation system for an AGV forklift based on vision recognition, characterized in that, include: The acquisition module is used to acquire visual recognition images, forklift travel paths, and the current position of the forklift; The processing module, connected to the acquisition and judgment modules, is used for information storage and processing; The judgment module, connected to the acquisition and processing modules, is used for judging information. The processing module divides the forklift travel path into several straight travel segments according to the preset transportation space map, defines the straight travel segment containing the current position of the forklift as the current segment, and determines the end position point and the direction of movement of the segment on the current segment. The processing module determines the image of the unobstructed road section from the preset image recording library based on the current position of the forklift and the direction of road movement; The processing module compares the features of the unobstructed road segment image with the visual recognition image so that the judgment module can determine whether there are external obstacles. If the judgment module determines that there are no external obstacles, the processing module controls the current forklift to continue moving along the current road segment. If the judgment module determines that there are external obstacle features, the processing module determines the external obstacle point based on the external obstacle features, and determines the required travel route based on the forklift's current position and end position. The judgment module determines whether the external obstacle point is located on the required driving section; If the judgment module determines that the external obstacle is not on the required driving section, the processing module controls the current forklift to continue moving along the current road section. If the judgment module determines that the external obstacle point is on the required driving section, the processing module determines the required avoidance point based on the characteristics of the external obstacle, and constructs a path replacement section based on the current position of the forklift, the required avoidance point and the end position point. The path replacement section replaces and updates the required driving section to control the forklift to move along the updated forklift driving path. If the external obstacle is located on the required driving section, the processing module establishes a unit interval with a preset unit duration on the preset time axis, with the current time point as the leading point; The processing module controls the forklift to continue moving along the current road segment within the unit area and continuously acquires visual recognition images, and determines the external obstacle points corresponding to the external obstacle features based on the visual recognition images; The processing module determines the obstacle movement distance based on any two external obstacle points within the unit interval; The judgment module determines whether there is a situation where the obstacle's movement distance exceeds the preset action distance; If the judgment module determines that there is no obstacle movement distance greater than the action distance, the processing module determines the required avoidance point based on the characteristics of the external obstacle after the unit interval and updates the forklift travel path. If the judgment module determines that there is an obstacle that can be moved further than the action distance, the processing module will control the forklift to continue moving along the current road segment. If the obstacle movement distance is greater than the action distance, the judgment module determines whether the external obstacle features are preset transport forklift features; If the judgment module determines that the external obstacle features are not those of a transport forklift, the processing module controls the current forklift to continue moving along the current road segment. If the judgment module determines that the external obstacle feature is a transport forklift feature, the acquisition module obtains the forklift location of each transport forklift from the preset task scheduling library; The processing module calculates the distance between points based on the location of the forklift and the external obstacle points, and determines the distance between points with the smallest value according to the preset sorting rules. The transport forklift corresponding to the distance between points with the smallest value is defined as the obstructing forklift. The processing module determines the obstruction task path based on the obstructing forklift in the task scheduling library, and determines the real-time existence point of the obstructing forklift at each time point based on the obstruction task path. The processing module determines the simulated location of the forklift at each time point based on the forklift's travel path and current position. The judgment module determines whether there is a situation where a real-time existing point and a simulated existing point overlap in the current road segment; If the judgment module determines that there is no real-time existing point and simulated existing point overlapping on the current road segment, the processing module controls the current forklift to continue moving along the current road segment. If the judgment module determines that there is a situation where a real-time existing point and a simulated existing point overlap on the current road segment, the processing module determines the required avoidance point based on the characteristics of the transport forklift. After the distance between the points is determined, the judgment module determines whether there are at least two forklift points with the same and smallest distance between them. If the judgment module determines that there are no forklift locations with at least two points that are equidistant from each other and have the smallest possible distance, the processing module determines the forklift location that is obstructing the forklift based on the forklift location corresponding to the smallest possible distance. If the judgment module determines that there are at least two forklift locations with the same and smallest distance between them, the processing module obtains the forklift execution posture of the forklift corresponding to each forklift location from the task scheduling library, and determines the forklift obstruction posture based on the characteristics of the transport forklift. The processing module compares the forklift's execution posture and its obstruction posture to determine the posture similarity ratio; The processing module determines the pose similarity ratio with the largest value according to the sorting rules, and determines the obstructing forklift based on the forklift presence point corresponding to the pose similarity ratio with the largest value. The steps to determine the required avoidance point based on the characteristics of transport forklifts include: The processing module defines the point where the real-time existing point and the simulated existing point overlap on the current road segment as the meeting point; The module retrieves the model and specifications of the forklift that is obstructing the forklift. The processing module determines the required clearance width corresponding to the forklift model and specifications based on the preset parameter matching relationship; The processing module constructs a virtual moving line segment based on the current position of the forklift and the meeting point, and constructs a straight line perpendicular to the virtual moving line segment at the meeting point; The processing module moves the encounter point in any direction along the avoidance line by the required avoidance width to determine the required avoidance point.

4. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed according to any one of claims 1 to 2.

Images