Path planning method and system of AGV (Automatic Guided Vehicle) based on machine vision

Abstract

The invention discloses a path planning method and system for an AGV trolley based on machine vision. The method comprises the following steps: acquiring environment information and destination information when the AGV trolley runs; according to the environment information, carrying out obstacle identification operation to obtain obstacle information; according to the obstacle information and the destination information, coincidence time calculation operation is carried out to obtain coincidence time; according to the coincidence time, the obstacle information and the destination information, performing path planning operation to obtain an optimal path and an alternative path; and carrying out movement control operation according to the optimal path and the alternative path so as to enable the AGV to run along the optimal path. According to the method, the AGV can efficiently perform path planning in a dynamic environment, and meanwhile, the method has relatively high accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of AGV trolley path planning, and in particular to a machine vision-based AGV trolley path planning method and system. Background Art

[0002] At present, in the field of modern industrial automation and intelligent manufacturing, automatic guided vehicles (AGV) play an increasingly important role. AGVs significantly improve production efficiency and flexibility of material management through automated material handling. With the development of technology, the application of AGVs has expanded from traditional manufacturing to logistics warehousing, e-commerce, medical and other fields. Visual guidance technology is a key component of the AGV navigation system. It uses machine vision to obtain environmental information, and through image processing and pattern recognition algorithms, it realizes precise control and path planning of the vehicle. This technology can not only improve the positioning accuracy of AGVs, but also enable them to navigate flexibly in complex environments. With the development of artificial intelligence technologies such as deep learning, the visual guidance system has significantly improved the recognition speed and accuracy, providing more possibilities for the application of AGVs.

[0003] In the prior art, the three main technical paths are mainly model-based planning methods, sensor-based planning methods, and rule-based planning methods. The model-based method pre-calculates the optimal path by constructing a mathematical model of the environment, which is effective in scenarios where the environmental structure is relatively fixed and there are fewer obstacles. However, in a dynamic environment, the path planned by it is no longer applicable because the environmental model is difficult to update in real time. Sensor-based methods, such as laser SLAM and visual SLAM, can obtain environmental information in real time and dynamically build maps to perform path planning. Although these technologies are well adapted to environmental changes, it is difficult to process a large amount of sensor data in a complex environment and has a large demand for computing resources. The rule-based planning method plans the route of the AGV car by pre-setting rules, such as the shortest path or the safest path. This solution is easy to implement, but in a dynamic environment, due to the lack of adaptability to environmental changes, the planned path is also not applicable.

[0004] In summary, the existing technology has the problem of being unable to perform path planning efficiently and accurately in a dynamic environment with dense crowds and frequently changing obstacles. Summary of the invention

[0005] The present invention provides a machine vision-based AGV path planning method and system, so as to enable the AGV to perform path planning efficiently and accurately in a dynamic environment with dense human traffic and frequently changing obstacles.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a path planning method for an AGV based on machine vision, comprising: Obtain the environment information and destination information of the AGV when it is running; Performing an obstacle recognition operation according to the environmental information to obtain obstacle information; Performing a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; Performing a path planning operation according to the overlap time, the obstacle information and the destination information to obtain an optimal path and alternative paths; According to the optimal path and the alternative paths, a movement control operation is performed to make the AGV vehicle travel along the optimal path.

[0007] Preferably, in an optional implementation, the path planning method for the AGV based on machine vision is characterized in that the environmental information and destination information of the AGV during operation are obtained, including: Use sensors to obtain the surrounding environment information during the operation of the AGV, and use input devices to obtain the destination information of the AGV.

[0008] Preferably, in an optional implementation, the path planning method for the AGV based on machine vision is characterized in that, according to the environmental information, an obstacle recognition operation is performed to obtain obstacle information, including: Performing a convolution operation according to the environmental information to obtain an image feature map; Performing downsampling layer processing according to the image feature map to obtain a reduced dimension feature map; Performing full connection layer processing according to the dimension reduction feature map to obtain output features; According to the output features, normalization and classification processing operations are performed to obtain obstacle information; The convolution operation formula is as follows:

[0009] in, Indicates Local linear units on convolutional layers, represents the feature map of the previous layer, Represents the convolution kernel of the current layer.

[0010] Preferably, in an optional implementation, the path planning method for the AGV based on machine vision is characterized in that, according to the obstacle information and the destination information, a coincidence time calculation operation is performed to obtain the coincidence time, including: According to the obstacle information and the destination information, a speed calculation operation is performed to obtain the AGV vehicle speed, the road speed and the obstacle speed; Performing a straight line projection operation according to the obstacle information and the destination information to obtain a path coincidence point; The overlap time calculation operation is performed according to the path overlap point, the AGV vehicle speed, the road speed and the obstacle speed to obtain the overlap time.

[0011] Preferably, in an optional implementation, the path planning method for the AGV based on machine vision is characterized in that, according to the path coincidence point, the AGV speed, the road speed and the obstacle speed, a coincidence time calculation operation is performed to obtain the coincidence time, including: When the distance between the obstacle and two adjacent points on the path is within the collision range, the first coincidence time calculation formula is used to obtain the coincidence time; When the distance between the obstacle and two adjacent points on the path is outside the collision range, the second coincidence time calculation formula is used to obtain the coincidence time; The first coincidence time calculation formula is as follows:

[0012] in, represents the first coincidence time, represents the obstacle speed, Indicates the size of the obstacle. represents the road speed, Indicates the speed of the AGV. Indicates the time it takes for the obstacle to reach the previous point on the path; The second coincidence time calculation formula is as follows:

[0013] in, represents the second coincidence time, Indicates the location of the obstacle. Represents the distance between two adjacent points on the path. represents the road speed, Indicates the speed of the AGV.

[0014] Preferably, in an optional implementation, the path planning method for the AGV based on machine vision is characterized in that, according to the overlap time, the obstacle information and the destination information, a path planning operation is performed to obtain an optimal path and an alternative path, including: Obtaining a first path using a heuristic search algorithm according to the overlap time, the obstacle information and the destination information; According to the first path, a depth-first search algorithm is used to obtain a simplified path; A path screening operation is performed according to the simplified path, the overlap time and the obstacle information to obtain an optimal path and alternative paths.

[0015] Preferably, in an optional embodiment, the path planning method of the AGV based on machine vision is characterized in that, according to the optimal path and the alternative paths, a mobile control operation is performed to make the AGV travel along the optimal path, including: Performing data conversion operations according to the optimal path and the alternative paths to obtain navigation data; According to the navigation data, mobile control operations are performed to make the AGV vehicle travel along the optimal path.

[0016] In a second aspect, the present invention provides a path planning system for an AGV based on machine vision, comprising: Information acquisition module, used to obtain the environment information and destination information of the AGV when it is running; An obstacle recognition module, used to perform an obstacle recognition operation according to the environmental information to obtain obstacle information; A coincidence time calculation module, used to perform a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; A path planning module, used to perform path planning operations according to the overlap time, the obstacle information and the destination information, so as to obtain an optimal path and alternative paths; The mobile control module is used to perform mobile control operations according to the optimal path and the alternative paths, so that the AGV car travels along the optimal path.

[0017] In a third aspect, the present invention further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements any one of the above-mentioned path planning methods for an AGV vehicle based on machine vision.

[0018] In a fourth aspect, the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned machine vision-based AGV path planning methods.

[0019] Compared with the prior art, the present invention has the following beneficial effects: the present invention obtains the environmental information and destination information of the AGV during operation, uses machine vision to identify obstacles, and improves the accuracy of identification through artificial intelligence technologies such as deep learning. According to the obstacle information and destination information, the overlap time is calculated to predict the potential conflict points between the obstacle and the AGV path. Based on the overlap time and obstacle information, a heuristic search algorithm is used for path planning to obtain the optimal path and alternative paths. According to the optimal path and the alternative paths, mobile control operations are performed to make the AGV travel along the optimal path, thereby achieving efficient and accurate path planning in a dynamic environment. By combining machine vision and deep learning technology, the path planning efficiency and accuracy of the AGV in a dynamic environment are significantly improved, and the adaptability and robustness of the system are enhanced, thereby achieving efficient and safe operation of the AGV in an environment with dense traffic and frequent changes in obstacles.

[0020] In summary, the present invention discloses a path planning method and system for an AGV trolley based on machine vision, including: obtaining environmental information and destination information when the AGV trolley is running; performing obstacle recognition operations according to the environmental information to obtain obstacle information; performing overlap time calculation operations according to the obstacle information and the destination information to obtain overlap time; performing path planning operations according to the overlap time, the obstacle information and the destination information to obtain an optimal path and an alternative path; performing mobile control operations according to the optimal path and the alternative path to make the AGV trolley travel along the optimal path. The present invention combines machine vision and deep learning technology to improve the path planning efficiency and accuracy of the AGV trolley in a dynamic environment, enhance the adaptability and robustness of the system, thereby realizing the efficient and safe operation of the AGV trolley in an environment with dense flow of people and frequent changes in obstacles. It solves the problem that the prior art cannot perform path planning efficiently and accurately in a dynamic environment with dense flow of people and frequent changes in obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic flow chart of a path planning method for an AGV based on machine vision provided in the first embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of a path planning system for an AGV based on machine vision provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Reference Figure 1 The first embodiment of the present invention provides a path planning method for an AGV based on machine vision, comprising the following steps: Obtain the environment information and destination information of the AGV when it is running; Performing an obstacle recognition operation according to the environmental information to obtain obstacle information; Performing a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; Performing a path planning operation according to the overlap time, the obstacle information and the destination information to obtain an optimal path and alternative paths; According to the optimal path and the alternative paths, a movement control operation is performed to make the AGV vehicle travel along the optimal path.

[0024] It should be noted that the existing technology faces many challenges and limitations when planning the path of the AGV. In a dynamic environment with dense traffic and frequent changes in obstacles, traditional methods often find it difficult to update the environmental model in real time, resulting in the planned path becoming outdated quickly. In addition, although the sensor-based method can obtain environmental information in real time, it consumes a lot of computing resources when processing a large amount of sensor data, and has limited adaptability in complex environments. The rule-based planning method often cannot provide the optimal path solution due to the lack of flexible response to environmental changes. These problems not only reduce the operating efficiency of the AGV, but may also bring safety risks. In order to solve these problems, the existing technology has tried a variety of improvement measures, including optimizing algorithms, enhancing sensor fusion technology, and improving computing power, etc., to improve the navigation ability of the AGV in complex environments. However, these improvement measures often require high hardware costs and complex software adjustments, and the effect in practical applications is still limited.

[0025] It is worth noting that the present invention provides an innovative path planning method and system by combining machine vision and deep learning technology. The present invention first obtains the environmental information and destination information of the AGV car during operation, then uses machine vision to identify obstacles, and combines deep learning to improve the accuracy of identification. Then, according to the obstacle information and destination information, the overlap time is calculated to predict the potential conflict points between the obstacle and the AGV car path. Based on the overlap time and obstacle information, a heuristic search algorithm is used for path planning to obtain the optimal path and alternative paths. Finally, according to the optimal path and the alternative path, a mobile control operation is performed to make the AGV car travel along the optimal path. This method can not only accurately and efficiently adapt to the path planning requirements in a dynamic environment, but also improve the navigation efficiency and accuracy of the AGV car in a complex environment. Through intelligent path planning and mobile control, the present invention effectively solves the problem that the prior art cannot efficiently and accurately perform path planning in an environment with dense traffic and frequent changes in obstacles, and provides a strong guarantee for the safe and efficient operation of the AGV car.

[0026] In step S11, the environment information and destination information of the AGV during operation are obtained, including: Use sensors to obtain the surrounding environment information during the operation of the AGV, and use input devices to obtain the destination information of the AGV.

[0027] It should be noted that obtaining environmental information and destination information when the AGV is running is a key step to ensure that the AGV can accurately perceive the surrounding environment and know the target location. The acquisition of environmental information relies on advanced sensor technology, such as high-definition cameras, laser radar (LIDAR), ultrasonic sensors, etc., which can capture the physical environment around the AGV in real time, including the location, size and shape of obstacles, as well as the flatness of the ground, traffic conditions, etc. These data are crucial to the accuracy and feasibility of path planning. The acquisition of destination information is completed through input devices, such as operator command input or preset goals of the automation system, to guide the AGV to plan the path in the right direction.

[0028] It is worth noting that the acquisition of environmental information relies on a series of advanced sensor technologies. High-definition cameras can capture visual information of the surrounding environment, including details such as the color, texture and contour of obstacles, which is very helpful for identifying and classifying obstacles. By emitting laser pulses and measuring the time it takes to reflect back, laser radar (LIDAR) sensors can accurately measure the distance between the AGV and surrounding objects, as well as their relative position and speed, which is essential for accurate positioning and obstacle avoidance in three-dimensional space. Ultrasonic sensors measure distance by emitting ultrasonic waves and receiving echoes. They are used for close-range obstacle detection, especially at close ranges where cameras and LIDAR cannot work effectively. The data collected by these sensors includes the location, size and shape of obstacles, which is essential for determining the free space and possible paths around the AGV. In addition, sensor data also includes the flatness of the ground, which is very important for the driving stability and safety of the AGV, because uneven ground can cause the car to bump or deviate from the intended path. Traffic condition information, such as whether there are other vehicles or pedestrians nearby, is also an important factor to consider when planning the path to ensure that the AGV can navigate safely in a busy environment.

[0029] It is worth noting that the acquisition of destination information involves the target position of the AGV. This is done through input devices, such as operator command input, which can be manually entered coordinates or positions on a map selected through a graphical interface. In the automation system, the destination information is already preset in the system, and the AGV needs to reach these target points along the preset path. This information is crucial for the path planning algorithm because it defines the goal and direction of the path planning.

[0030] It should be noted that the acquired environmental information and destination information will be used in subsequent key steps. In the obstacle recognition step, machine vision technology and deep learning algorithms will analyze sensor data to identify and classify obstacles in the environment to ensure that the AGV avoids collisions. In the overlap time calculation step, this information will be used to predict potential conflict points and calculate overlap time, providing a time-dimensional decision-making basis for path planning. The path planning step will combine obstacle information and destination information, and use a heuristic search algorithm to determine the optimal path and alternative paths to the destination to ensure the rationality and safety of the planned path. Finally, the mobile control step will perform data conversion operations based on the determined optimal path to obtain navigation data to guide the mobile control of the AGV, so that the AGV can travel efficiently and safely along the optimal path.

[0031] In step S12, an obstacle recognition operation is performed according to the environmental information to obtain obstacle information, including: Performing a convolution operation according to the environmental information to obtain an image feature map; Performing downsampling layer processing according to the image feature map to obtain a reduced dimension feature map; Performing full connection layer processing according to the dimension reduction feature map to obtain output features; According to the output features, normalization and classification processing operations are performed to obtain obstacle information; The convolution operation formula is as follows:

[0032] in, Indicates Local linear units on convolutional layers, represents the feature map of the previous layer, Represents the convolution kernel of the current layer.

[0033] It should be noted that deep learning technology, especially convolutional neural network (CNN), is used to process environmental information obtained from sensors to achieve accurate identification of obstacles around the AGV. This process involves several key calculation steps, each of which plays a vital role in the final obstacle identification result.

[0034] It is worth noting that the image data in the environmental information is sent to the convolution layer of CNN for convolution operation. In this step, the image data is convolved with the predefined convolution kernel. Perform mathematical operations, which include a series of multiplication and addition operations. These convolution kernels slide over the image, calculate the dot product between the convolution kernel and the local area of ​​the image, and generate a new feature map. This feature map represents the spatial hierarchy in the input image, capturing basic features such as edges and textures. The formula for the convolution operation is This process is described in represents the output of the mth convolutional layer, represents the feature map of the previous layer, is the convolution kernel of the current layer. In this way, the convolution operation of each layer is trying to extract higher-level features and provide richer information for subsequent recognition tasks.

[0035] It is worth mentioning that the downsampling layer processing is applied to the output of the convolutional layer to reduce the dimensionality of the feature map. This downsampling is achieved through maximum pooling or average pooling, which reduces the spatial size of the data while retaining the most important features. This step helps to reduce the computational burden of the model while preventing overfitting and improving the generalization ability of the model. The fully connected layer processing then further analyzes the downsampled feature map. In the fully connected layer, each neuron is connected to all neurons in the previous layer, which enables the network to learn and recognize more complex feature combinations. The output of the fully connected layer is a set of features that integrate all the information extracted from the image and provide a basis for the final obstacle classification. Finally, the output features are normalized and classified to obtain specific information about the obstacles. The normalization step ensures that the feature values ​​are in a suitable range, which helps improve the performance of the classifier. The classification process uses the trained deep learning model to identify and classify obstacles, such as distinguishing between pedestrians, vehicles, fixed obstacles, etc.

[0036] It should be noted that the present invention also takes into account the complexity of the environment, such as different lighting conditions, the diversity and dynamic changes of obstacles, etc., during the obstacle recognition process. By using data enhancement technology and multi-scale feature fusion, the model can better adapt to these changes and improve the robustness of recognition. Data enhancement simulates different environmental conditions by creating variants of the image, while multi-scale feature fusion allows the model to understand the image from different levels, thereby improving the ability to recognize obstacles.

[0037] It is worth noting that in the subsequent path planning steps, this precise obstacle information will be used to calculate potential conflict points and overlap times, and predict the possible interactions between the AGV and obstacles. This prediction is crucial for path planning in dynamic environments because it enables the AGV to make decisions in advance, avoid potential obstacles, and choose a safe and efficient path.

[0038] In step S13, a coincidence time calculation operation is performed according to the obstacle information and the destination information to obtain the coincidence time, including: According to the obstacle information and the destination information, a speed calculation operation is performed to obtain the AGV vehicle speed, the road speed and the obstacle speed; Performing a straight line projection operation according to the obstacle information and the destination information to obtain a path coincidence point; The overlap time calculation operation is performed according to the path overlap point, the AGV vehicle speed, the road speed and the obstacle speed to obtain the overlap time.

[0039] It should be noted that a key calculation process in step S13, namely the calculation of the overlap time, is to predict the time point at which the AGV may interact with obstacles in the environment. This step is crucial for path planning in a dynamic environment because it can help the AGV make decisions in advance, avoid potential obstacles, and thus choose the safest and most efficient path. The calculation of the overlap time begins with a comprehensive analysis of obstacle information and destination information. First, the system needs to determine the relative speed of the AGV, obstacles, and the surrounding environment. This involves further processing of sensor data, such as using laser radar (LIDAR) and cameras to track the movement of obstacles and calculate their velocity vectors. At the same time, the current speed and direction of the AGV are also included in the judgment to determine its dynamic position in the environment. Next, the system performs a straight line projection operation to determine the potential intersection between the AGV path and the obstacle, that is, the path overlap point. This step requires simulating the expected path of the AGV and comparing this path with the current position and movement trend of the obstacle to predict possible future interaction points. Once the path overlap points are known, the system will calculate the overlap time based on these points, the speed of the AGV, the speed of the road conditions, and the speed of the obstacles. The overlap time is the time it takes for the AGV to reach each potential conflict point from the current moment. This calculation is critical to evaluating the safety of different paths because it helps the system determine which paths will become unsafe or infeasible within a certain time.

[0040] It should be noted that the overlap time calculation operation can be performed by the following steps: when the AGV car follows the current path (denoted as Include Point ) When driving, the system will detect obstacles ahead on the road , calculate the speed of the AGV and road speed For example, the speed of the car For 5 , road speed For 2 Of course, the speed of the car will vary depending on the actual application and user needs. For 10 , road speed is 0 The present invention does not limit this. These speed parameters are very important for the subsequent calculation of the overlap time. Determine the forward speed of the obstacle , which is the speed of the obstacle along its direction of motion. Calculate the obstacle to reach the current path The time (denoted as This is done by summing the time it takes the obstacle to reach each point on the path. in, is the predicted overlap time. Different formulas are used to predict the obstacle arrival path based on the distance between the obstacle and two adjacent points on the path. Time , if the distance between the obstacle and two adjacent points on the path is within the collision range ( ), using the first coincidence time calculation formula to obtain the coincidence time ( ), otherwise the second coincidence time calculation formula is used to obtain the coincidence time ( ). Calculate the time difference ( ), if the time difference Less than time difference , then use the coincidence time calculated by the first coincidence time calculation formula plus the total time calculated previously, Otherwise, add the coincidence time calculated using the second coincidence time calculation formula to the total time calculated previously. For example, the time difference Set to 2 Of course, the time difference varies according to the actual application and user needs. Set to 3 The present invention does not limit this. Through this precise prediction method, the AGV can identify potential obstacle conflicts in advance and adjust its path planning accordingly. This method significantly improves the navigation efficiency and accuracy of the AGV in complex dynamic environments, ensuring its safe and efficient operation.

[0041] In step S14, a path planning operation is performed according to the overlap time, the obstacle information and the destination information to obtain an optimal path and alternative paths, including: Obtaining a first path using a heuristic search algorithm according to the overlap time, the obstacle information and the destination information; According to the first path, a depth-first search algorithm is used to obtain a simplified path; A path screening operation is performed according to the simplified path, the overlap time and the obstacle information to obtain an optimal path and alternative paths.

[0042] It should be noted that the optimal path and alternative paths of the AGV car are determined based on the overlap time, obstacle information and destination information obtained in the previous step. This step is the core of the entire path planning process because it directly affects the driving efficiency and safety of the AGV car. First, the system uses a heuristic search algorithm to generate a first path in combination with the overlap time and obstacle information. The heuristic search algorithm is an effective path search method that uses heuristic information in the problem space to guide the search process, thereby quickly finding a path from the starting point to the destination. In this process, the algorithm evaluates the feasibility and safety of each path, while considering the location of obstacles and the predicted overlap time to avoid potential conflicts. Exemplarily, the heuristic search algorithm uses the A* algorithm. Of course, according to different actual applications and user needs, the heuristic search algorithm can also use the Dijkstra algorithm, etc., which is not limited by the present invention. Then, the system uses a depth-first search algorithm to simplify the first path. The depth-first search algorithm is an algorithm for traversing or searching a tree or graph. It goes deep along the path until it can no longer go deep, and then backtracks and tries other paths. This step helps to extract the simplest and most direct path from the path generated by the heuristic search algorithm while maintaining avoidance of environmental obstacles. Finally, the system screens the path based on the simplified path, overlap time, and obstacle information. This step involves a comprehensive evaluation of the safety and efficiency of all feasible paths to determine the optimal path and alternative paths. The optimal path refers to the path that enables the AGV to reach the destination with the highest efficiency and safety under the current environmental conditions. The alternative path provides a flexible response plan when faced with emergencies, such as the sudden appearance of obstacles or blocked paths.

[0043] It is worth noting that during the entire path planning process, the system continuously receives environmental information updates from sensors, as well as changes in destination information, and adjusts and optimizes path planning in real time. This dynamic path planning method enables the AGV trolley to adapt to complex and changeable industrial environments, and effectively cope with situations with dense flow of people and frequent changes in obstacles. Through this intelligent path planning method, the present invention not only improves the accuracy and adaptability of path planning, but also significantly improves the navigation efficiency and safety of the AGV trolley in complex environments. This method effectively solves the problem that the prior art cannot efficiently and accurately plan paths in dynamic environments, and provides a strong guarantee for the safe and efficient operation of the AGV trolley. In addition, by providing alternative paths, the present invention also enhances the robustness of the system, ensuring that the AGV trolley can quickly resume operation and continue to move toward the destination when faced with unexpected situations.

[0044] In step S15, according to the optimal path and the alternative paths, a movement control operation is performed to make the AGV travel along the optimal path, including: Performing data conversion operations according to the optimal path and the alternative paths to obtain navigation data; According to the navigation data, mobile control operations are performed to make the AGV vehicle travel along the optimal path.

[0045] It should be noted that the results of path planning are converted into specific mobile control instructions to guide the AGV trolley to travel along the optimal path. This step is the final link in the path planning process, ensuring a seamless connection from path planning to actual driving. First, the system performs data conversion operations based on the optimal path and alternative paths to generate navigation data. These navigation data include the various waypoints on the path, the expected speed configuration, and the specific actions that need to be performed at specific waypoints (such as turning, slowing down, or stopping). This step involves complex algorithms that convert the results of path planning into a format suitable for the AGV trolley control system to understand. Next, the system performs mobile control operations based on the generated navigation data. This involves precise control of the AGV trolley's drive motor and steering system to ensure that the trolley can travel smoothly and accurately along the predetermined path. Mobile control operations also require real-time monitoring of the AGV trolley's status, such as position, speed, and direction, as well as changes in the environment, such as sudden obstacles or unexpected situations on the path. If the system detects any deviation from the optimal path or environmental changes, it will immediately recalculate the path and update the navigation data to ensure that the AGV trolley can quickly adapt to these changes. This real-time feedback and adjustment mechanism is the key to ensure the safe and efficient operation of the AGV in a dynamic environment. In addition, the present invention also takes into account the robustness of the system by providing alternative paths to deal with possible emergencies. If the optimal path becomes unfeasible for some reason, the system can quickly switch to an alternative path and recalculate the navigation data to minimize delays and ensure the continuity of the task.

[0046] It is worth noting that the generation of navigation data takes into account a variety of factors, including the dynamic characteristics of the AGV (such as acceleration and maximum speed limit), battery life, and obstacle information on the path. In this way, the system can ensure that the generated navigation data is both safe and efficient.

[0047] It is worth noting that, through this fine mobile control operation, the present invention not only improves the navigation efficiency and accuracy of the AGV in complex environments, but also improves its adaptability and robustness in the face of emergencies. This method effectively solves the problem that the existing technology cannot efficiently and accurately plan and execute paths in dynamic environments, provides a strong guarantee for the safe and efficient operation of the AGV, and ensures stable operation in environments with dense traffic and frequently changing obstacles.

[0048] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0049] In a busy manufacturing plant, AGVs are responsible for transporting raw materials and finished products between different workstations.

[0050] Step 1: Acquisition of environmental information and destination information. The AGV first uses its onboard sensor suite, including high-definition cameras, LIDAR, and ultrasonic sensors, to scan the surrounding environment in real time. Through these sensors, the AGV can identify the location, size, and shape of obstacles, sense the flatness of the ground, and monitor traffic conditions, such as nearby people or other vehicles. At the same time, the AGV receives the destination coordinates from the central control system or operator.

[0051] Step 2: Obstacle Identification. The collected environmental information is fed into a deep learning model that is trained to identify and classify a variety of obstacles. Using a convolutional neural network (CNN), the system extracts image features through a series of convolutional and pooling layers, and then outputs specific information about the obstacle, such as obstacle type and obstacle location, through a fully connected layer.

[0052] Step 3: Calculate the overlap time. Based on the speed of the obstacle and the predicted path, the AGV calculates the potential conflict points and the overlap time to reach those points. This calculation takes into account the relative speed of the AGV and the obstacle, as well as the time it takes for the obstacle to reach the AGV's current path.

[0053] Step 4: Path planning. Combining the overlap time and obstacle information, the AGV uses a heuristic search algorithm, such as the A* algorithm, to determine the optimal path and alternative paths to the destination. The system evaluates the safety and efficiency of each path and selects the best route.

[0054] Step 5: Mobile control. After determining the optimal path, the AGV's control system guides the vehicle along this path based on the navigation data. This includes precisely controlling the vehicle's speed and direction to avoid obstacles and reach the destination smoothly.

[0055] Throughout the process, the AGV continuously receives sensor data updates and adjusts the path planning in real time to adapt to environmental changes. This intelligent path planning and mobile control method not only improves the navigation efficiency and accuracy of the AGV in complex dynamic environments, but also improves its adaptability and robustness in the face of emergencies. With this approach, the AGV can operate safely and efficiently in busy manufacturing plants, reducing delays and potential safety risks caused by obstacles.

[0056] In summary, the present invention provides an innovative path planning method and system by combining machine vision and deep learning technology, which is particularly suitable for dynamic and complex industrial environments. The method first uses advanced sensor technologies such as high-definition cameras, laser radar (LIDAR) and ultrasonic sensors to obtain environmental information and destination information of the AGV. This information includes the location, size, shape and ground flatness of obstacles, which is crucial for subsequent path planning. Next, the environmental information is analyzed using a deep learning model, image features are extracted through a convolutional neural network (CNN), and obstacles are identified. Then, based on the obstacle information and destination information, the system calculates the overlap time of potential conflict points and predicts the possible interaction between obstacles and the AGV path. In the path planning stage, a heuristic search algorithm, such as the A* algorithm, is used to determine the optimal path and alternative paths to the destination in combination with the overlap time and obstacle information. This step ensures the safety and efficiency of the path, while providing a flexible solution to deal with emergencies. Finally, in the mobile control stage, the system generates navigation data based on the optimal path and alternative paths, and accurately controls the speed and direction of the AGV to ensure that it travels efficiently and safely along the optimal path. Throughout the process, the system continuously receives sensor data updates and adjusts the path planning in real time to adapt to environmental changes.

[0057] The advantage of the present invention is that it can accurately and efficiently adapt to the path planning requirements in dynamic environments, significantly improving the navigation efficiency and accuracy of AGVs in complex environments. Through intelligent path planning and mobile control, the present invention effectively solves the problem that the existing technology cannot efficiently and accurately plan paths in environments with dense crowds and frequently changing obstacles, providing a strong guarantee for the safe and efficient operation of AGVs. In addition, the robustness of the system has been enhanced, ensuring that it can quickly resume operation and continue to move toward the destination in the face of unexpected situations.

[0058] Reference Figure 2 The second embodiment of the present invention provides a path planning system for an AGV based on machine vision, comprising: Information acquisition module, used to obtain the environment information and destination information of the AGV when it is running; An obstacle recognition module, used to perform an obstacle recognition operation according to the environmental information to obtain obstacle information; A coincidence time calculation module, used to perform a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; A path planning module, used to perform path planning operations according to the overlap time, the obstacle information and the destination information, so as to obtain an optimal path and alternative paths; The mobile control module is used to perform mobile control operations according to the optimal path and the alternative paths, so that the AGV car travels along the optimal path.

[0059] It should be noted that the path planning system for an AGV car based on machine vision provided in an embodiment of the present invention is used to execute all the process steps of the path planning method for an AGV car based on machine vision in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0060] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a path planning program for an AGV based on machine vision. When the processor executes the computer program, the steps in the above-mentioned path planning method for an AGV based on machine vision are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized, such as the movement control module.

[0061] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0062] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0063] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0064] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0065] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0066] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0067] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A path planning method for an AGV based on machine vision, characterized in that: Executed by a computer, including: Obtain the environment information and destination information of the AGV when it is running; Performing an obstacle recognition operation according to the environmental information to obtain obstacle information; Performing a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; Performing a path planning operation according to the overlap time, the obstacle information and the destination information to obtain an optimal path and alternative paths; According to the optimal path and the alternative paths, a movement control operation is performed to make the AGV vehicle travel along the optimal path.

2. The path planning method of the AGV based on machine vision according to claim 1 is characterized in that: Obtain the environment information and destination information of the AGV during operation, including: Use sensors to obtain the surrounding environment information during the operation of the AGV, and use input devices to obtain the destination information of the AGV.

3. The path planning method of the AGV based on machine vision according to claim 1 is characterized in that: According to the environmental information, an obstacle recognition operation is performed to obtain obstacle information, including: Performing a convolution operation according to the environmental information to obtain an image feature map; Performing downsampling layer processing according to the image feature map to obtain a reduced dimension feature map; Performing full connection layer processing according to the dimension reduction feature map to obtain output features; According to the output features, normalization and classification processing operations are performed to obtain obstacle information; The convolution operation formula is as follows: in, Indicates Local linear units on convolutional layers, represents the feature map of the previous layer, Represents the convolution kernel of the current layer.

4. The path planning method of the AGV based on machine vision according to claim 1 is characterized in that: Performing a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time includes: According to the obstacle information and the destination information, a speed calculation operation is performed to obtain the AGV vehicle speed, the road speed and the obstacle speed; Performing a straight line projection operation according to the obstacle information and the destination information to obtain a path coincidence point; The overlap time calculation operation is performed according to the path overlap point, the AGV vehicle speed, the road speed and the obstacle speed to obtain the overlap time.

5. The path planning method of the AGV based on machine vision according to claim 4 is characterized in that: The overlap time calculation operation is performed according to the path overlap point, the AGV vehicle speed, the road speed and the obstacle speed to obtain the overlap time, including: When the distance between the obstacle and two adjacent points on the path is within the collision range, the first coincidence time calculation formula is used to obtain the coincidence time; When the distance between the obstacle and two adjacent points on the path is outside the collision range, the second coincidence time calculation formula is used to obtain the coincidence time; The first coincidence time calculation formula is as follows: in, represents the first coincidence time, represents the obstacle speed, Indicates the size of the obstacle. represents the road speed, Indicates the speed of the AGV. Indicates the time it takes for the obstacle to reach the previous point on the path; The second coincidence time calculation formula is as follows: in, represents the second coincidence time, Indicates the location of the obstacle. Represents the distance between two adjacent points on the path. represents the road speed, Indicates the speed of the AGV.

6. The path planning method of the AGV based on machine vision according to claim 1 is characterized in that: Performing a path planning operation according to the overlap time, the obstacle information and the destination information to obtain an optimal path and an alternative path includes: Obtaining a first path using a heuristic search algorithm according to the overlap time, the obstacle information and the destination information; According to the first path, a depth-first search algorithm is used to obtain a simplified path; A path screening operation is performed according to the simplified path, the overlap time and the obstacle information to obtain an optimal path and alternative paths.

7. The path planning method of the AGV based on machine vision according to claim 1 is characterized in that: According to the optimal path and the alternative paths, a mobile control operation is performed to make the AGV vehicle travel along the optimal path, including: Performing data conversion operations according to the optimal path and the alternative paths to obtain navigation data; According to the navigation data, mobile control operations are performed to make the AGV vehicle travel along the optimal path.

8. A path planning system for an AGV based on machine vision, characterized in that: include: Information acquisition module, used to obtain the environment information and destination information of the AGV when it is running; An obstacle recognition module, used to perform an obstacle recognition operation according to the environmental information to obtain obstacle information; A coincidence time calculation module, used to perform a coincidence time calculation operation according to the obstacle information and the destination information to obtain the coincidence time; A path planning module, used to perform path planning operations according to the overlap time, the obstacle information and the destination information, so as to obtain an optimal path and alternative paths; The mobile control module is used to perform mobile control operations according to the optimal path and the alternative paths, so that the AGV car travels along the optimal path.

9. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the path planning method of the AGV car based on machine vision as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the machine vision-based AGV vehicle path planning method as described in any one of claims 1 to 7.

Images