AGV obstacle avoidance method and device

Abstract

The invention discloses an AGV obstacle avoidance method and device. The method comprises the steps that environment information corresponding to an AGV is acquired in real time through a laser radar and a UWB positioning system; generating a space-time diagram model based on the environment information, analyzing the space-time diagram model, and determining a space characteristic matrix and a time characteristic matrix; performing multi-modal feature fusion on the spatial feature matrix and the time feature matrix to obtain a space-time feature matrix; and according to the space-time characteristic matrix, an obstacle avoidance strategy of the AGV is optimized by using SAC algorithm planning, and a driving path is adjusted based on an optimization result. According to the invention, the space-time characteristic modeling and the multi-modal characteristic fusion are carried out, and finally the driving path is adjusted through the SAC algorithm, so that compared with the prior art, the space distribution and time sequence characteristics of the obstacles are considered at the same time, and the obstacle avoidance precision of the AGV and the stability of the AGV are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an obstacle avoidance method and device for an AGV cart. Background Art

[0002] In today's highly automated logistics and warehousing industries, automated guided vehicle (AGV) carts play a crucial role. They shuttle within warehouses, carry goods, and perform repetitive and precise tasks. However, to ensure that these carts can accurately avoid obstacles while executing tasks and maintain a stable operating state involves a series of complex technical challenges.

[0003] In the current technical field, when an automated guided vehicle (AGV) cart executes an obstacle avoidance task, it often faces a complex challenge: it needs to consider the distribution of obstacles in three-dimensional space and the dynamic changes of these obstacles over time when identifying and responding to obstacles. However, the existing obstacle avoidance of AGV carts cannot simultaneously consider the spatial distribution and time series characteristics of obstacles, resulting in low obstacle avoidance accuracy and stability of AGV carts.

[0004] Therefore, there is an urgent need for an AGV cart obstacle avoidance method that can simultaneously consider the spatial distribution and time series characteristics of obstacles, thereby improving the obstacle avoidance accuracy and stability of AGV carts. Summary of the Invention

[0005] The main objective of the present invention is to provide an obstacle avoidance method and device for an AGV cart, aiming to solve the technical problem in the prior art that due to the inability of the obstacle avoidance of AGV carts to simultaneously consider the spatial distribution and time series characteristics of obstacles, the obstacle avoidance accuracy and stability of AGV carts are low.

[0006] To achieve the above objective, the present invention provides an obstacle avoidance method for an AGV cart, and the method includes the following steps:

[0007] Obtain the environmental information corresponding to the AGV cart in real time through a lidar and a UWB positioning system;

[0008] Generate a spatio-temporal graph model based on the environmental information, and analyze the spatio-temporal graph model to determine a spatial characteristic matrix and a time characteristic matrix;

[0009] Perform multi-modal feature fusion on the spatial characteristic matrix and the time characteristic matrix to obtain a spatio-temporal characteristic matrix;

[0010] According to the spatio-temporal characteristic matrix, use the SAC algorithm to optimize the obstacle avoidance strategy of the AGV cart and adjust the driving path based on the optimization result.

[0011] Optionally, the environmental information includes obstacle information and positioning information. The step of obtaining the environmental information corresponding to the AGV cart in real time through the lidar and the UWB positioning system includes:

[0012] Scanning the current environment with the lidar to determine the obstacle information of the current environment;

[0013] Determining the first positioning information of the AGV cart according to the obstacle information;

[0014] Determining the distance information between multiple base stations through the UWB positioning system, and determining the second positioning information of the AGV cart based on the distance information between the multiple base stations;

[0015] Combining the first positioning information and the second positioning information to obtain positioning information.

[0016] Optionally, the step of combining the first positioning information and the second positioning information to obtain positioning information includes:

[0017] Predicting the position information of the AGV cart according to the environmental information and the status information of the AGV cart;

[0018] Based on the position information, the first positioning information and the second positioning information, obtaining initial positioning information according to the Kalman filtering algorithm;

[0019] Smoothing the initial positioning information to obtain positioning information.

[0020] Optionally, the step of combining the first positioning information and the second positioning information to obtain positioning information further includes:

[0021] Determining the weight data corresponding to the first positioning information and the second positioning information respectively;

[0022] Based on each weight data, combining the first positioning information and the second positioning information by using the weighted average method to obtain initial positioning information;

[0023] Smoothing the initial positioning information to obtain positioning information.

[0024] Optionally, the step of generating a spatio-temporal graph model based on the environmental information and analyzing the spatio-temporal graph model to determine the spatial characteristic matrix and the temporal characteristic matrix further includes:

[0025] Determining the morphological information of the obstacle and the relationship between the obstacle and the AGV cart according to the environmental information;

[0026] Build a geometric model based on the morphological information, and generate a spatio-temporal graph model based on the geometric model and the relationship between the obstacle and the AGV vehicle;

[0027] Perform spatial analysis and temporal analysis on the spatio-temporal graph model to obtain a spatial characteristic matrix and a temporal characteristic matrix.

[0028] Optionally, the step of performing multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatio-temporal characteristic matrix further includes:

[0029] Concatenate the spatial characteristic matrix and the temporal characteristic matrix to obtain a concatenated feature matrix;

[0030] Generate target input information by passing the concatenated feature matrix through the fully connected layer of the multi-modal transformer;

[0031] Input the target input information into the multi-head attention mechanism of the multi-modal transformer to obtain an initial spatio-temporal characteristic matrix;

[0032] Input the initial spatio-temporal characteristic matrix into the residual connection and the feed-forward neural network of the multi-modal transformer to generate a spatio-temporal characteristic matrix.

[0033] Optionally, after the step of optimizing the obstacle avoidance strategy of the AGV vehicle according to the spatio-temporal characteristic matrix by using the SAC algorithm and adjusting the driving path based on the optimization result, it further includes:

[0034] Evaluate the adjusted driving path based on a preset obstacle avoidance evaluation index to obtain a path evaluation result;

[0035] Determine whether the adjusted driving path meets the preset optimization goal according to the path evaluation result;

[0036] When the adjusted driving path meets the preset optimization goal, output the adjusted driving path;

[0037] When the adjusted driving path does not meet the preset optimization goal, return to execute the step of optimizing the obstacle avoidance strategy of the AGV vehicle according to the spatio-temporal characteristic matrix by using the SAC algorithm and adjusting the driving path based on the optimization result.

[0038] In addition, to achieve the above object, the present invention also proposes an AGV vehicle obstacle avoidance device, and the device includes:

[0039] An information acquisition module, configured to acquire the environmental information corresponding to the AGV vehicle in real time through a lidar and a UWB positioning system;

[0040] A model analysis module, configured to generate a spatio-temporal graph model based on the environmental information, and analyze the spatio-temporal graph model to determine a spatial feature matrix and a temporal feature matrix;

[0041] A feature fusion module, configured to perform multi-modal feature fusion on the spatial feature matrix and the temporal feature matrix to obtain a spatio-temporal feature matrix;

[0042] A strategy optimization module, configured to optimize the obstacle avoidance strategy for the AGV cart according to the spatio-temporal feature matrix by using the SAC algorithm, and adjust the driving path based on the optimization result.

[0043] Optionally, the device further includes:

[0044] An environment scanning module, configured to scan the current environment through a lidar to determine the obstacle information of the current environment;

[0045] A first positioning module, configured to determine the first positioning information of the AGV cart according to the obstacle information;

[0046] A second positioning module, configured to determine the distance information between multiple base stations through a UWB positioning system, and determine the second positioning information of the AGV cart based on the distance information between the multiple base stations;

[0047] An information combination module, configured to combine the first positioning information and the second positioning information to obtain positioning information.

[0048] Optionally, the device further includes:

[0049] A path evaluation module, configured to evaluate the adjusted driving path based on a preset obstacle avoidance evaluation index to obtain a path evaluation result;

[0050] A path judgment module, configured to determine whether the adjusted driving path meets a preset optimization goal according to the path evaluation result;

[0051] A result output module, configured to output the adjusted driving path when the adjusted driving path meets the preset optimization goal;

[0052] A strategy adjustment module, configured to, when the adjusted driving path does not meet the preset optimization goal, return to execute the step of optimizing the obstacle avoidance strategy for the AGV cart according to the spatio-temporal feature matrix by using the SAC algorithm and adjusting the driving path based on the optimization result.

[0053] In addition, to achieve the above object, the present invention further provides an obstacle avoidance device for an AGV cart, the device comprising: a memory, a processor, and an AGV cart obstacle avoidance program stored on the memory and operable on the processor, the AGV cart obstacle avoidance program being configured to implement the steps of the AGV cart obstacle avoidance method as described above.

[0054] In addition, to achieve the above object, the present invention further provides a storage medium having an AGV cart obstacle avoidance program stored thereon, the AGV cart obstacle avoidance program, when executed by a processor, implementing the steps of the AGV cart obstacle avoidance method as described above.

[0055] The present invention discloses a method for obtaining environmental information corresponding to an AGV cart in real time through a lidar and a UWB positioning system; generating a spatio-temporal graph model based on the environmental information, and analyzing the spatio-temporal graph model to determine a spatial characteristic matrix and a temporal characteristic matrix; performing multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatio-temporal characteristic matrix; and according to the spatio-temporal characteristic matrix, using the SAC algorithm to optimize the obstacle avoidance strategy for the AGV cart and adjusting the driving path based on the optimization result. Since the present invention generates a spatio-temporal graph model based on environmental information, determines a spatial characteristic matrix and a temporal characteristic matrix, then performs multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix, and finally adjusts the driving path through the SAC algorithm, compared with the prior art, the present invention simultaneously considers the spatial distribution and temporal sequence characteristics of obstacles, thereby improving the obstacle avoidance accuracy of the AGV cart and the stability of the AGV cart. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic flowchart of the first embodiment of the AGV cart obstacle avoidance method of the present invention;

[0057] Figure 2 It is a schematic flowchart of the second embodiment of the AGV cart obstacle avoidance method of the present invention;

[0058] Figure 3 It is a schematic flowchart of the third embodiment of the AGV cart obstacle avoidance method of the present invention;

[0059] Figure 4 It is a structural block diagram of the first embodiment of the AGV cart obstacle avoidance device of the present invention;

[0060] Figure 5 It is a schematic structural diagram of an AGV cart obstacle avoidance device which is a hardware operating environment related to the embodiment solution of the present invention.

[0061] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] It should be understood that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.

[0063] An embodiment of the present invention provides an obstacle avoidance method for an AGV vehicle. Refer to Figure 1 , Figure 1 which is a schematic flow chart of the first embodiment of the obstacle avoidance method for the AGV vehicle of the present invention.

[0064] In this embodiment, the obstacle avoidance method for the AGV vehicle includes steps S10 to S40:

[0065] Step S10: Real-time obtain the environmental information corresponding to the AGV vehicle through a lidar and a UWB positioning system.

[0066] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication, and program running functions applied to the driving scenario of the AGV vehicle, such as a server, a tablet computer, a smart phone, a smart watch, etc., or an electronic device, an AGV vehicle obstacle avoidance device, etc. that can implement the above functions. Hereinafter, a system including an AGV vehicle obstacle avoidance device (hereinafter referred to as the system) is taken as an example to illustrate this embodiment and the following embodiments.

[0067] It should be understood that lidar (Laser Radar or LiDAR) is an advanced sensor technology that detects the position, speed, and other characteristic quantities of a target by emitting laser beams. The positioning principle of the UWB positioning system is to calculate the position of an object by measuring the time delay difference of the signal in space propagation. When a UWB signal is sent from a base station to the object to be measured and reflected back to the base station on the object, the base station can measure the time delay of the signal.

[0068] In specific implementation, in order to achieve high-precision positioning, usually multiple base stations can be used to locate an object. These base stations are distributed in space and communicate with the object. By measuring the time delay difference between the object and each base station and using methods such as triangulation or the Doppler effect, the precise position of the object can be calculated.

[0069] It should be explained that the above environmental information may include obstacle information and positioning information. The obstacle information may include the position information of the obstacle and the distance information between the obstacle and the AGV vehicle. The positioning information may be the position information of the AGV vehicle.

[0070] In specific implementation, a UWB tag can be installed on the AGV vehicle, and multiple UWB base stations are arranged in the factory. The distance between the UWB tag and the base station is measured through a wireless signal. Then, based on the distance information of multiple base stations, the precise position of the AGV is calculated by triangulation. In order to further ensure high-precision real-time positioning, the position can be updated once per second.

[0071] Step S20: Generate a spatio-temporal graph model based on the environmental information, and analyze the spatio-temporal graph model to determine a spatial feature matrix and a temporal feature matrix.

[0072] It should be understood that a spatio-temporal graph model is a graph structure model used to describe and analyze objects that are interrelated in time and space. In the graph, each obstacle can be represented as a node, and the relative distance or other relevant information between obstacles can be used as the edges between nodes (such as the relative distance, direction, etc. between obstacles).

[0073] In a specific implementation, the morphological information of the obstacle and the relationship between the obstacle and the AGV vehicle can be determined according to the environmental information; a geometric model can be established according to the morphological information, and a spatio-temporal graph model can be generated based on the geometric model and the relationship between the obstacle and the AGV vehicle; spatial analysis and temporal analysis are performed on the spatio-temporal graph model to obtain a spatial feature matrix and a temporal feature matrix.

[0074] It should be noted that the spatial feature matrix H S : represents the spatial distribution of obstacles and their relative positions. The temporal feature matrix H T : represents the relative motion between the agent (i.e., the AGV vehicle) and the obstacle or the temporal sequence information of the path.

[0075] Step S30: Perform multi-modal feature fusion on the spatial feature matrix and the temporal feature matrix to obtain a spatio-temporal feature matrix.

[0076] It should be noted that the spatial feature matrix H S and the temporal feature matrix H T are fused to generate a spatio-temporal feature matrix H ST . This process is implemented by a multi-modal transformer, and its key purpose is to fuse spatial and temporal information through a multi-head attention mechanism to capture spatio-temporal dependencies. The multi-modal transformer can include a feature concatenation function, a fully connected layer, a multi-head attention mechanism, a residual connection, and a feed-forward neural network.

[0077] It should be explained that the spatial feature matrix and the temporal feature matrix are concatenated to obtain a concatenated feature matrix; the concatenated feature matrix is passed through the fully connected layer of the multi-modal transformer to generate target input information; the target input information is input into the multi-head attention mechanism of the multi-modal transformer to obtain an initial spatio-temporal feature matrix; the initial spatio-temporal feature matrix is input into the residual connection and the feed-forward neural network of the multi-modal transformer to generate a spatio-temporal feature matrix.

[0078] In a specific implementation, first, the spatial feature matrix and the temporal feature matrix are concatenated to obtain a concatenated feature matrix

[0079]

[0080] Then, the concatenated fused feature matrix generates Query, Key, and Value through a fully connected layer as the input to the multi-head attention mechanism.

[0081]

[0082] In the formula, Q U , K F , V F represent the matrices corresponding to Query, Key, and Value respectively, serving as the input to the multi-head attention mechanism. represents the concatenated feature matrix. represent the weight matrices of Query, Key, and Value respectively, which are learnable parameters.

[0083] Furthermore, the multi-head attention mechanism is used to calculate Query, Key, and Value, thereby obtaining the initial spatio-temporal feature matrix

[0084]

[0085] In the formula, Y headj represents the output result of the j-th attention head, represents the processing result of the cross-modal attention mechanism on the input represents the standard attention calculation mechanism, whose input is the Query, Key, and Value matrices. Where g ∈ {S, T} represents the spatial and temporal modalities, and Atten(Q, K, V) is the standard attention mechanism, which calculates the attention scores by computing the dot product of the Query and the Key, and then performs weighted summation on the Value.

[0086]

[0087] In the formula, Atten(Q, K, V) represents the attention result, that is, the fused importance weight. d k represents the dimension of the Key matrix, which is used to scale the dot product value to avoid excessive numerical values.

[0088] The outputs of multiple heads are calculated through the multi-head attention mechanism and finally aggregated through a fully connected layer to obtain the target input information ​

[0089]

[0090] In the formula, represents the attention result of each head. FFC represents a fully connected layer, which is used to integrate the outputs of multiple attention heads.

[0091] Finally, in the output calculated by the multi-head attention mechanism, a residual connection and a feed-forward neural network are added to further optimize the initial spatio-temporal feature matrix:

[0092]

[0093] Among them, represents the output result after optimizing the initial spatio-temporal feature matrix, which combines the feature information of space and time.

[0094] The feature expression ability can be further improved through the feed-forward neural network:

[0095]

[0096] H ST = Y M .

[0097] It should be noted that represents the output after cross-modal feature fusion. represents the input feature matrix (such as a spatio-temporal feature matrix). represents the output of the multi-head attention mechanism. LayerNorm represents layer normalization, which is used to stabilize training. FFN represents a feed-forward neural network, which is used to improve the feature expression ability. H ST combines spatial features and temporal features, and can provide a comprehensive environmental understanding for the agent. Especially in obstacle avoidance decision-making, it considers the spatial distribution of obstacles and the temporal behavior of the agent.

[0098] Step S40: According to the spatio-temporal feature matrix, use the SAC algorithm to optimize the obstacle avoidance strategy of the AGV vehicle and adjust the driving path based on the optimization result.

[0099] It should be understood that the SAC algorithm combines the ideas of policy iteration and value iteration. By introducing an entropy regularization term, the algorithm achieves a good balance between exploration and exploitation. It is an algorithm based on the maximum entropy reinforcement learning framework. By introducing an entropy regularization term, it encourages the algorithm to explore, thereby improving the performance of the algorithm.

[0100] In a specific implementation, through the SAC (Soft Actor-Critic) algorithm, based on the spatio-temporal feature matrix H ST, the obstacle avoidance strategy of the AGV vehicle. Combining real-time perception data and the spatio-temporal characteristic matrix, the SAC algorithm is used to adjust the path to ensure that the agent can avoid obstacles in real time.

[0101] This embodiment discloses obtaining the environmental information corresponding to the AGV vehicle in real time through a lidar and a UWB positioning system; generating a spatio-temporal graph model based on the environmental information, and analyzing the spatio-temporal graph model to determine a spatial characteristic matrix and a temporal characteristic matrix; performing multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatio-temporal characteristic matrix; according to the spatio-temporal characteristic matrix, using the SAC algorithm to optimize the obstacle avoidance strategy of the AGV vehicle, and adjusting the driving path based on the optimization result. Since this embodiment generates a spatio-temporal graph model based on the environmental information, determines a spatial characteristic matrix and a temporal characteristic matrix, then performs multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix, and finally adjusts the driving path through the SAC algorithm. Compared with the prior art, this embodiment simultaneously considers the spatial distribution and time series characteristics of obstacles, thereby improving the obstacle avoidance accuracy of the AGV vehicle and the stability of the AGV vehicle.

[0102] Reference Figure 2 , Figure 2 is a schematic flowchart of the second embodiment of the AGV vehicle obstacle avoidance method of the present invention.

[0103] Based on the above first embodiment, in this embodiment, the step S10 includes steps S101 to S104:

[0104] Step S101: Scan the current environment through a lidar to determine the obstacle information of the current environment.

[0105] Step S102: Determine the first positioning information of the AGV vehicle according to the obstacle information.

[0106] Step S103: Determine the distance information between multiple base stations through a UWB positioning system, and determine the second positioning information of the AGV vehicle based on the distance information between the multiple base stations.

[0107] Step S104: Combine the first positioning information and the second positioning information to obtain positioning information.

[0108] It should be noted that before step S101, the initial position of the AGV can be calculated by matching the lidar data with a known map through a pre-constructed map or SLAM (Simultaneous Localization and Mapping) technology.

[0109] In a specific implementation, to obtain accurate positioning information, the position information of the AGV vehicle can be predicted based on environmental information and the status information of the AGV vehicle; based on the position information, the first positioning information, and the second positioning information, initial positioning information is obtained according to the Kalman filter algorithm; the initial positioning information is smoothed to obtain positioning information.

[0110] It should be noted that by fusing laser and UWB data through Kalman Filter (KF) or Extended Kalman Filter (EKF), high-precision positioning information can be obtained.

[0111] It should be added that the weight data corresponding to the first positioning information and the second positioning information can also be determined; based on each weight data, the first positioning information and the second positioning information are combined using the weighted average method to obtain initial positioning information; the initial positioning information is smoothed to obtain positioning information.

[0112] It should be explained that the fused initial positioning information is smoothed to reduce noise and ensure stability.

[0113] This embodiment discloses scanning the current environment through a lidar to determine the obstacle information of the current environment; determining the first positioning information of the AGV vehicle according to the obstacle information; determining the distance information between multiple base stations through a UWB positioning system, and determining the second positioning information of the AGV vehicle based on the distance information between the multiple base stations; combining the first positioning information and the second positioning information to obtain positioning information. Since this embodiment combines the first positioning information determined based on the lidar with the second positioning information determined based on the UWB positioning system to obtain positioning information, compared with the prior art, the accuracy of the positioning information is effectively improved in this embodiment.

[0114] Reference Figure 3 , Figure 3 is a schematic flowchart of the third embodiment of the AGV vehicle obstacle avoidance method of the present invention.

[0115] Based on the above embodiments, in this embodiment, after step S40, steps S50 to S80 are further included:

[0116] Step S50: Evaluate the adjusted driving path based on a preset obstacle avoidance evaluation index to obtain a path evaluation result.

[0117] Step S60: Determine whether the adjusted driving path meets a preset optimization goal according to the path evaluation result.

[0118] Step S70: When the adjusted driving path meets the preset optimization goal, output the adjusted driving path.

[0119] Step S80: When the adjusted driving path does not meet the preset optimization goal, return to execute the step of optimizing the obstacle avoidance strategy of the AGV cart according to the spatio-temporal characteristic matrix by using the SAC algorithm and adjusting the driving path based on the optimization result.

[0120] It should be noted that the obstacle avoidance evaluation indicators can include success rate: whether the task can be completed (reach the target point); collision rate: whether a collision with an obstacle occurs; efficiency: the time cost of the planned path; sociality: whether it conforms to social norms (such as safety distance, naturalness of behavior); robustness: whether it can work properly under noise or environmental changes.

[0121] In a specific implementation, the adjusted driving path can be quantitatively evaluated based on preset obstacle avoidance evaluation indicators to obtain a path evaluation result.

[0122] This embodiment discloses evaluating the adjusted driving path based on preset obstacle avoidance evaluation indicators to obtain a path evaluation result; determining whether the adjusted driving path meets the preset optimization goal according to the path evaluation result; when the adjusted driving path meets the preset optimization goal, output the adjusted driving path; when the adjusted driving path does not meet the preset optimization goal, return to execute the step of optimizing the obstacle avoidance strategy of the AGV cart according to the spatio-temporal characteristic matrix by using the SAC algorithm and adjusting the driving path based on the optimization result. Compared with the prior art, this embodiment evaluates the adjusted driving path based on preset obstacle avoidance evaluation indicators to obtain a path evaluation result, effectively ensuring the reliability of the AGV cart's obstacle avoidance strategy.

[0123] In addition, an embodiment of the present invention also proposes a storage medium, on which an AGV cart obstacle avoidance program is stored. When the AGV cart obstacle avoidance program is executed by a processor, the steps of the AGV cart obstacle avoidance method described above are implemented.

[0124] Refer to Figure 4 , Figure 4 It is the structural block diagram of the first embodiment of the AGV cart obstacle avoidance device of the present invention.

[0125] As Figure 4 shown, the AGV cart obstacle avoidance device proposed by the embodiment of the present invention includes: an information acquisition module 401, a model analysis module 402, a feature fusion module 403, and a strategy optimization module 404.

[0126] The information acquisition module 401 is used to obtain the environmental information corresponding to the AGV vehicle in real time through a lidar and a UWB positioning system.

[0127] The model analysis module 402 is used to generate a spatio-temporal graph model based on the environmental information, and analyze the spatio-temporal graph model to determine a spatial characteristic matrix and a temporal characteristic matrix.

[0128] The feature fusion module 403 is used to perform multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatio-temporal characteristic matrix.

[0129] The strategy optimization module 404 is used to optimize the obstacle avoidance strategy of the AGV vehicle according to the spatio-temporal characteristic matrix by using the SAC algorithm, and adjust the driving path based on the optimization result.

[0130] The embodiment of the present device discloses obtaining the environmental information corresponding to the AGV vehicle in real time through a lidar and a UWB positioning system; generating a spatio-temporal graph model based on the environmental information, and analyzing the spatio-temporal graph model to determine a spatial characteristic matrix and a temporal characteristic matrix; performing multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatio-temporal characteristic matrix; optimizing the obstacle avoidance strategy of the AGV vehicle according to the spatio-temporal characteristic matrix by using the SAC algorithm, and adjusting the driving path based on the optimization result. Since the embodiment of the present device generates a spatio-temporal graph model based on the environmental information, determines a spatial characteristic matrix and a temporal characteristic matrix, then performs multi-modal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix, and finally adjusts the driving path through the SAC algorithm, compared with the prior art, the embodiment of the present device simultaneously considers the spatial distribution and time series characteristics of obstacles, thereby improving the obstacle avoidance accuracy of the AGV vehicle and the stability of the AGV vehicle.

[0131] Based on the first embodiment of the AGV vehicle obstacle avoidance device of the present invention, a second embodiment of the AGV vehicle obstacle avoidance device of the present invention is proposed.

[0132] In this embodiment, the device further includes:

[0133] An environment scanning module, which is used to scan the current environment through a lidar to determine the obstacle information of the current environment.

[0134] A first positioning module, which is used to determine the first positioning information of the AGV vehicle according to the obstacle information.

[0135] A second positioning module, which is used to determine the distance information between multiple base stations through a UWB positioning system, and determine the second positioning information of the AGV vehicle based on the distance information between the multiple base stations.

[0136] An information combination module is configured to combine the first positioning information and the second positioning information to obtain positioning information.

[0137] For other embodiments or specific implementation manners of the AGV car obstacle avoidance device of the present invention, reference may be made to the above method embodiments, which will not be elaborated herein.

[0138] This application provides an AGV car obstacle avoidance device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the AGV car obstacle avoidance method in Embodiment 1 above.

[0139] Next, with reference to Figure 5 , which shows a schematic structural diagram of an AGV car obstacle avoidance device suitable for implementing the embodiments of the present application. The AGV car obstacle avoidance device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The AGV car obstacle avoidance device shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0140] As Figure 5As shown, the obstacle avoidance device of the AGV vehicle may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the AGV vehicle obstacle avoidance device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the AGV vehicle obstacle avoidance device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an AGV vehicle obstacle avoidance device with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0141] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0142] The AGV vehicle obstacle avoidance device provided by the present application adopts the AGV vehicle obstacle avoidance method in the above-mentioned embodiment, and can solve the technical problem in the prior art that the obstacle avoidance of the AGV vehicle cannot consider the spatial distribution and time series characteristics of obstacles at the same time, thereby resulting in low obstacle avoidance accuracy and low stability of the AGV vehicle. Compared with the prior art, the beneficial effects of the AGV vehicle obstacle avoidance device provided by the present application are the same as those of the AGV vehicle obstacle avoidance method provided by the above-mentioned embodiment, and other technical features in the AGV vehicle obstacle avoidance device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0143] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0144] As described above, the above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all of them should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0145] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or system including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or system including that element.

[0146] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0147] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory / random access memory, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0148] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An AGV vehicle obstacle avoidance method, characterized in that: The method comprises: Obtain the corresponding environmental information of the AGV car in real time through the laser radar and UWB positioning system; Generate a space-time graph model based on the environmental information, and analyze the space-time graph model to determine a spatial characteristic matrix and a temporal characteristic matrix; Perform multimodal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatiotemporal characteristic matrix; According to the spatiotemporal characteristic matrix, the obstacle avoidance strategy of the AGV is optimized using the SAC algorithm, and the driving path is adjusted based on the optimization result.

2. The AGV obstacle avoidance method according to claim 1, characterized in that: The environmental information includes obstacle information and positioning information. The step of obtaining the environmental information corresponding to the AGV in real time through the laser radar and UWB positioning system includes: Scanning the current environment by laser radar to determine obstacle information of the current environment; Determine the first positioning information of the AGV according to the obstacle information; Determine the distance information between multiple base stations through the UWB positioning system, and determine the second positioning information of the AGV based on the distance information between the multiple base stations; The first positioning information and the second positioning information are combined to obtain positioning information.

3. The AGV obstacle avoidance method according to claim 2, characterized in that: The step of combining the first positioning information and the second positioning information to obtain positioning information includes: Predicting the position information of the AGV according to the environmental information and the status information of the AGV; Based on the position information, the first positioning information and the second positioning information, obtaining initial positioning information according to a Kalman filter algorithm; The initial positioning information is smoothed to obtain positioning information.

4. The AGV obstacle avoidance method according to claim 2, characterized in that: The step of combining the first positioning information and the second positioning information to obtain positioning information further includes: Determine weight data corresponding to the first positioning information and the second positioning information respectively; Based on each of the weight data, combining the first positioning information and the second positioning information by using a weighted average method to obtain initial positioning information; The initial positioning information is smoothed to obtain positioning information.

5. The AGV obstacle avoidance method according to claim 1, characterized in that: The step of generating a spatiotemporal graph model based on the environmental information, analyzing the spatiotemporal graph model, and determining a spatial characteristic matrix and a temporal characteristic matrix further includes: Determine the shape information of the obstacle and the relationship between the obstacle and the AGV according to the environmental information; Establishing a geometric model according to the morphological information, and generating a spatiotemporal graph model based on the geometric model and the relationship between the obstacle and the AGV; The space-time graph model is subjected to spatial analysis and time series analysis to obtain a space characteristic matrix and a time characteristic matrix.

6. The AGV obstacle avoidance method according to claim 1, characterized in that: The step of performing multimodal feature fusion on the spatial characteristic matrix and the temporal characteristic matrix to obtain a spatiotemporal characteristic matrix further includes: Concatenate the spatial characteristic matrix and the temporal characteristic matrix to obtain a concatenated characteristic matrix; Passing the concatenated feature matrix through a fully connected layer of a multimodal transformer to generate target input information; Inputting the target input information into the multi-head attention mechanism of the multimodal transformer to obtain an initial spatiotemporal feature matrix; The initial spatiotemporal characteristic matrix is ​​input into the residual link and feedforward neural network of the multimodal converter to generate a spatiotemporal characteristic matrix.

7. The AGV obstacle avoidance method according to claim 1, characterized in that: After the step of optimizing the obstacle avoidance strategy of the AGV using the SAC algorithm according to the spatiotemporal characteristic matrix and adjusting the driving path based on the optimization result, the method further includes: Evaluate the adjusted driving path based on the preset obstacle avoidance evaluation index to obtain a path evaluation result; Determining whether the adjusted driving path meets a preset optimization target according to the path evaluation result; When the adjusted driving path meets the preset optimization target, outputting the adjusted driving path; When the adjusted driving path does not meet the preset optimization target, the process returns to the step of optimizing the obstacle avoidance strategy of the AGV vehicle using the SAC algorithm according to the spatiotemporal characteristic matrix, and adjusting the driving path based on the optimization result.

8. An AGV vehicle obstacle avoidance device, characterized in that: The device comprises: The information acquisition module is used to obtain the corresponding environmental information of the AGV in real time through the laser radar and UWB positioning system; A model analysis module, used to generate a space-time graph model based on the environmental information, and analyze the space-time graph model to determine a spatial characteristic matrix and a temporal characteristic matrix; A feature fusion module, used for performing multimodal feature fusion on the spatial feature matrix and the temporal feature matrix to obtain a spatiotemporal feature matrix; The strategy optimization module is used to optimize the obstacle avoidance strategy of the AGV car according to the spatiotemporal characteristic matrix using the SAC algorithm planning, and adjust the driving path based on the optimization result.

9. The AGV obstacle avoidance device according to claim 8, characterized in that: The device further comprises: An environment scanning module is used to scan the current environment through a laser radar to determine obstacle information in the current environment; A first positioning module, used to determine the first positioning information of the AGV according to the obstacle information; A second positioning module is used to determine the distance information between multiple base stations through a UWB positioning system, and determine the second positioning information of the AGV based on the distance information between the multiple base stations; The information combining module is used to combine the first positioning information and the second positioning information to obtain positioning information.

10. The AGV obstacle avoidance device according to claim 8, characterized in that: The device further comprises: A path evaluation module is used to evaluate the adjusted driving path based on preset obstacle avoidance evaluation indicators to obtain a path evaluation result; A path judgment module, used to determine whether the adjusted driving path meets the preset optimization target according to the path evaluation result; A result output module, configured to output the adjusted driving path when the adjusted driving path meets a preset optimization target; The strategy adjustment module is used to return to the step of optimizing the obstacle avoidance strategy of the AGV vehicle using the SAC algorithm planning according to the spatiotemporal characteristic matrix and adjusting the driving path based on the optimization result when the adjusted driving path does not meet the preset optimization target.

Images