Agv vehicle control and navigation system based on visual large model

Abstract

The invention discloses an agv vehicle control and navigation system based on a visual large model, and relates to the technical field of agv vehicle control. The agv vehicle control and navigation system based on the visual large model comprises an initialization and data acquisition unit, a path planning and guiding module, an image data processing module, a path deviation analysis and selection module, a path adjustment module, an agv speed regulation and control module and a position positioning and exception handling module. Electromagnetic and visual navigation systems are fused, and an LSTM algorithm and a multi-visual sensor fusion technology are utilized, so that the path planning accuracy is improved; extracting fixed object features to update a database, and enhancing system adaptability; the path is preferentially selected, and the navigation reliability is improved; a navigation system can be automatically switched to cope with an interference area; dynamic speed regulation can be achieved, and the driving requirement is met; the method can position and predict a path in real time, detect abnormity, adjust direction and speed in time, give an alarm when processing cannot be carried out, and guarantee safe and stable operation of agv.

Description

Technical Field

[0001] The present invention relates to the technical field of AGV vehicle control, and in particular to an AGV vehicle control and navigation system based on a large visual model. Background Art

[0002] AGVs (Automated Guided Vehicles) are a common form of automated equipment used in logistics and warehousing. Their purpose is to replace manual labor, enabling the automated transportation and handling of goods. To achieve automated movement, AGVs are equipped with automatic guidance devices, such as electromagnetic or optical ones. After production, they are equipped with intelligent guidance systems, enabling them to follow a prescribed path. These transport vehicles feature safety features and various transfer functions. AGVs, characterized by wheeled mobility, offer advantages over walking, crawling, and other non-wheeled mobile robots, such as speed, high efficiency, simple structure, strong controllability, and improved safety. Compared to other equipment commonly used in material handling, AGVs require no fixed fixtures, such as tracks or supports, and are not restricted by site, roads, or space. Therefore, within automated logistics systems, AGVs best demonstrate their automation and flexibility, enabling efficient, cost-effective, and flexible unmanned production.

[0003] Publication No. CN112180929A discloses a guidance and control system for automated guided vehicles (AGVs), comprising a main control system, a ground control system, an onboard control system, an AGV guidance system, and a positioning system. The central control system is unidirectionally connected to the ground control system, the onboard control system, and the AGV guidance system, respectively, while the main control system is bidirectionally connected to the positioning system. The ground control system includes a task management unit, a vehicle management unit, a traffic management unit, a communication management unit, and a vehicle drive unit, while the onboard control system includes an AGV navigation system. By combining electromagnetic guidance and visual navigation, the system analyzes the AGV's travel path in real time, accurately locating and guiding the AGV's spatial position. This ensures the AGV's ability to continuously transport objects over a large area and its high-precision assembly capabilities.

[0004] However, both electromagnetic guidance and visual navigation have corresponding defects. Although the visual navigation system can provide higher positioning accuracy, its recognition ability will be significantly affected in complex environments with insufficient light, severe reflections or the presence of obstructions, which may lead to navigation failure or inaccurate positioning. The electromagnetic guidance system relies on electromagnetic signals for navigation. If there is a strong electromagnetic interference source in the working environment, it may interfere with the transmission and reception of electromagnetic signals, thereby affecting the navigation accuracy and stability of the AGV. Even if the two systems are used in combination to avoid the defects of the two systems, using the two systems for navigation at the same time will interfere with each other. How to correctly choose the navigation of different systems in different environments is an urgent problem that needs to be solved. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides an AGV vehicle control and navigation system based on a large visual model, which solves the problems of the existing technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an AGV vehicle control and navigation system based on a large visual model, comprising:

[0007] The initialization and data acquisition unit is used to initialize the system, specifically loading the electromagnetic navigation system parameters and the visual navigation system parameters; collect environmental maps, mark electromagnetic interference areas and light reflection areas, and establish an environmental feature database; record the AGV's running path, speed, steering angle, and environmental feature data;

[0008] The path planning and guidance module calculates the path based on electromagnetic signals for electromagnetic navigation systems and uses the LSTM algorithm to predict and compensate for electromagnetic signals. For visual navigation systems, it uses a large visual model to process images and introduces multi-visual sensor fusion technology to calculate the path.

[0009] The image data processing module is used to collect and pre-process the image data of the AGV's surrounding environment, extract the fixed object features in the image as reference points, compare them with the historical environmental features, use the unchanged objects as new reference points, and update the historical database;

[0010] The path deviation analysis and selection module compares the path planning results of the electromagnetic navigation system and the visual navigation system, and selects the path planned by the system with the smallest deviation as the driving path of the AGV;

[0011] A path adjustment module automatically selects an alternative navigation system in areas with high electromagnetic or optical interference;

[0012] The AGV speed control module introduces an adaptive speed control algorithm to dynamically adjust the AGV's operating speed based on the image processing speed and the AGV's driving needs;

[0013] The position positioning and exception handling module receives positioning data and combines the predictive control algorithm and anomaly detection technology to perform real-time prediction and anomaly detection on the AGV's driving path.

[0014] Preferably, the initialization and data acquisition unit operations specifically include:

[0015] A1 system initialization

[0016] Load electromagnetic guidance system parameters: electromagnetic sensor sensitivity S em , electromagnetic signal frequency range (f min , f max ), electromagnetic signal strength threshold I th ;

[0017] Load the visual navigation system parameters: visual large model input image resolution R in , model inference time threshold T th ;

[0018] A2 Collect factory environment maps: Use lidar or high-precision mapping tools to mark areas with high electromagnetic interference. em and the area A with large light reflection light ; Establish an environmental feature database to store the feature vector F of each area i , where F i ={f i1 , f i2 ,...,f in}, f ij represents the jth feature of the i-th region;

[0019] A3 establishes an AGV operation history database: records the AGV operation path P = {p1, p2, ..., p m}, where p k Indicates the coordinates of the kth point on the path; records the speed v k , steering angle θ k And environmental characteristic data E k .

[0020] Preferably, the path planning and guidance module operations are divided into navigation system and visual navigation system, specifically including:

[0021] Electromagnetic guidance system: calculates the path based on the electromagnetic signal and uses electromagnetic sensors to measure the electromagnetic signal strength I. When I>I th When , it is considered that there is a path in this direction;

[0022] Combine deep learning algorithms to predict and compensate electromagnetic signals:

[0023] Prediction formula: Where L is the LSTM model, is the predicted value of electromagnetic signal strength at the next moment;

[0024] Compensation formula: Among them I comp is the electromagnetic signal strength after compensation;

[0025] Visual navigation system: uses a large visual model to process road images and identify paths and fixed objects;

[0026] Introducing multi-vision sensor fusion technology to calculate the fused path:

[0027] Assume that the path calculation result of the x-th sensor is Px , with weight w x , then the fused path Where N is the number of sensors,

[0028] Preferably, the image data processing module operations specifically include:

[0029] B1 Image Acquisition: Use a high-resolution camera to continuously capture the surrounding environment of the AGV and obtain real-time image data. raw ; Perform pre-processing operations such as denoising and enhancement on the collected images to improve image quality;

[0030] B2 Feature extraction: Extract the features of fixed objects in the image as reference points; use feature extraction algorithms to extract feature points E points ={e1, e2, ..., e q};

[0031] B3 Environmental matching: Compare the extracted features with the environmental features in the historical database and identify unchanged objects as reference points; calculate the feature similarity S(E points , E db ), where E db When it is detected that the objects around the same location have changed and are different from the stored data, the unchanged objects for reference are extracted and marked as new reference points, and the environmental feature data in the historical database is updated;

[0032] B4 Path Calculation: Calculate the AGV's driving path based on the extracted reference points and visual model; use the path planning algorithm to calculate the optimal path P from the current position to the target position opt .

[0033] Preferably, the path deviation analysis and selection module operations specifically include:

[0034] The main control system receives the path planning results P of the electromagnetic navigation system and the visual navigation system. em and P vis ;

[0035] Calculate the paths planned by the two systems and the paths P at the previous time point prev Deviation:

[0036] Electromagnetic navigation system path deviation

[0037] Path deviation of visual navigation system

[0038] Compare the deviations of the two systems. The one with the larger deviation is considered abnormal data. The path planned by the system with the smaller deviation is selected as the driving path P of the AGV. final .

[0039] Preferably, the path adjustment module operation specifically includes:

[0040] When the AGV drives to area A with large electromagnetic interference em When the path planned by the visual navigation system is used as the main method, the frequency of image data processing f is appropriately increased. img ;

[0041] When the AGV drives to area A with large light reflection light When performing the navigation, the path planned by the electromagnetic navigation system is mainly used, and the camera parameters are adjusted to reduce the impact of light reflection on image processing.

[0042] Preferably, the path adjustment module combines the predictive control algorithm to predict in advance the electromagnetic interference or light reflection area that the AGV is about to enter, and adjusts the driving parameters of the AGV according to the prediction results. The prediction formula is expressed as:

[0043]

[0044] Where Y is the prediction model, is the regional prediction value at the next moment, A current It is the region-related information at the current moment, including the feature data related to the region at the current moment, used to describe the state of the current region; current is the driving speed of the AGV at the current moment, θ current is the current driving direction angle of the AGV, which is the deflection angle relative to the specified reference direction.

[0045] Preferably, the operation of the AGV speed control module specifically includes:

[0046] Introducing the adaptive speed control algorithm, during the image processing process, according to the image processing speed T process Dynamically adjust the AGV's operating speed v according to the AGV's driving needs:

[0047] When T process >T th When the AGV running speed is reduced to v=v prev -Δv;

[0048] When T process Close to T th When the AGV is running, the speed v is kept at v prev ;

[0049] When T process <T thWhen there is a certain margin, increase the AGV running speed v=v prev +Δv;

[0050] Among them, T th is the set image processing speed threshold, Δv is the speed adjustment value, v prev Indicates the running speed of the AGV at the previous moment;

[0051] When the image processing speed cannot keep up with the AGV's running speed, the AGV's running speed should be adjusted or stopped to avoid the AGV's deviation due to insufficient image processing.

[0052] Preferably, the operations of the position positioning and exception handling module specifically include:

[0053] The location positioning system receives the data of the AGV navigation system in real time, performs location positioning, and transmits the located location data P loc Transmitted to the main control system, and the planned path P final To compare:

[0054] Deviation calculation formula: D loc =||P loc -P final ||;

[0055] Combining predictive control algorithms and anomaly detection technology, real-time prediction and anomaly detection of AGV driving paths are performed:

[0056] Prediction formula: Where Y is the prediction model, is the predicted position value at the next moment, P current is the current location information;

[0057] Anomaly detection formula: When D loc >D th When it is judged as an abnormal situation, D th is the deviation threshold.

[0058] Preferably, when the AGV is detected to have deviated from the planned path, the main control system immediately performs abnormal processing and adjusts the driving direction and speed of the AGV:

[0059] Direction adjustment formula: θ new =θ current +Δθ, where Δθ is the angle adjustment value, θ new is the adjusted angle;

[0060] Speed ​​adjustment formula: v new =v current -Δv, where Δv is the speed adjustment value, v new is adjusted after adjustment;

[0061] When an abnormal situation cannot be handled in time, the system sends an alarm signal to notify the operator to intervene.

[0062] The present invention provides an AGV vehicle control and navigation system based on a large visual model. Compared with the existing technology, it has the following advantages:

[0063] 1. This AGV vehicle control and navigation system, based on a large-scale visual model, features a comprehensive and detailed initialization and data acquisition unit. It loads multiple parameters, collects environmental maps, and establishes a historical database, providing a rich data foundation for subsequent navigation. The path planning and guidance module integrates electromagnetic and visual navigation systems, utilizing the LSTM algorithm and multi-visual sensor fusion technology to improve path planning accuracy. The image data processing module extracts features of fixed objects and updates the database, enhancing system adaptability. The path deviation analysis and selection module selects the optimal path to improve navigation reliability. The path adjustment module automatically switches the navigation system to address interference areas. The AGV speed control module implements dynamic speed regulation to meet driving requirements. The position positioning and exception handling module locates and predicts paths in real time, detects anomalies, and promptly adjusts direction and speed. It issues an alarm if the process cannot be handled, ensuring the safe and stable operation of the AGV, effectively improving navigation accuracy, reliability, and safety, and reducing the need for human intervention.

[0064] 2. This AGV vehicle control and navigation system, based on a large-scale visual model, uses an electromagnetic guidance system that combines deep learning algorithms to predict and compensate for electromagnetic signals, enabling more accurate route planning based on these signals and minimizing the impact of interference. The visual navigation system utilizes a large-scale visual model and multi-sensor fusion technology to integrate multi-sensor information to plan more reliable routes. The image data processing module utilizes high-quality image acquisition, precise feature extraction, effective environmental matching, and path calculation based on reference points and the large-scale visual model. This enables real-time update of reference points, adapting to environmental changes, and accurately calculating the optimal path. These improvements enhance navigation accuracy and stability, strengthen the system's adaptability to complex environments, and prevent difficulty identifying and positioning the vehicle during subsequent passes due to environmental changes recorded by the system.

[0065] 3. This AGV vehicle control and navigation system, based on a large-scale visual model, receives path planning results from both the electromagnetic and visual navigation systems, calculates deviations from the previous path, and compares the two. It accurately identifies abnormal data with large deviations and automatically selects the system-planned path with smaller deviations as the driving path. This effectively improves the accuracy and reliability of path selection, reduces the risk of AGV driving anomalies caused by errors in a single navigation system, and ensures stable AGV operation.

[0066] 4. This AGV vehicle control and navigation system, based on a large-scale visual model, intelligently switches the dominant navigation system when the AGV enters different interference zones. For example, in areas with significant electromagnetic interference, it prioritizes visual navigation and increases the frequency of image data processing. In areas with significant light reflection, it prioritizes electromagnetic navigation and adjusts camera parameters, effectively reducing the impact of interference. Furthermore, combined with a predictive control algorithm, it can predict the interference zone the AGV is about to enter and promptly adjust driving parameters such as speed and steering angle based on the predicted results. This approach of early prediction and dynamic adjustment improves the AGV's adaptability to complex environments and ensures stable and safe driving.

[0067] 5. This large-scale visual model-based AGV control and navigation system incorporates an adaptive speed control algorithm, dynamically adjusting the speed based on image processing speed and AGV travel requirements. The system reduces speed when image processing speed exceeds a threshold, maintains speed when approaching the threshold, and increases speed when below the threshold with a margin. The system can also control or stop the AGV if image processing cannot keep up with travel speed. This effectively prevents AGV deviations due to untimely image processing, ensures operational stability and safety, and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a system module block diagram of the present invention;

[0069] Figure 2 It is a schematic flow chart of the main steps of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] See Figure 1-Figure 2 , the present invention provides the following five technical solutions:

[0072] The first embodiment: an AGV vehicle control and navigation system based on a large visual model, comprising:

[0073] Initialization and data acquisition unit, specifically including:

[0074] A1 system initialization

[0075] Load electromagnetic guidance system parameters: electromagnetic sensor sensitivity S em , electromagnetic signal frequency range (f min , f max), electromagnetic signal strength threshold I th ;

[0076] Load the visual navigation system parameters: visual large model input image resolution R in (such as 640×480), model inference time threshold T th ;

[0077] A2 Collect factory environment maps: Use lidar or high-precision mapping tools to mark areas with high electromagnetic interference. em and the area A with large light reflection light ; Establish an environmental feature database to store the feature vector F of each area i , where F i ={f i1 , f i2 ,...,f in}, f ij Represents the jth feature of the i-th region (such as electromagnetic signal intensity, light reflectivity, etc.);

[0078] A3 establishes an AGV operation history database: records the AGV operation path P = {p1, p2, ..., p m}, where p k Indicates the coordinates of the kth point on the path; records the speed v k , steering angle θ k And environmental characteristic data E k (such as ground marker type, fixed object location coordinates, etc.);

[0079] The path planning and guidance module calculates the path based on electromagnetic signals for electromagnetic navigation systems and uses the LSTM algorithm to predict and compensate for electromagnetic signals. For visual navigation systems, it uses a large visual model to process images and introduces multi-visual sensor fusion technology to calculate the path.

[0080] The image data processing module is used to collect and pre-process the image data of the AGV's surrounding environment, extract the fixed object features in the image as reference points, compare them with the historical environmental features, use the unchanged objects as new reference points, and update the historical database;

[0081] The path deviation analysis and selection module compares the path planning results of the electromagnetic navigation system and the visual navigation system, and selects the path planned by the system with the smallest deviation as the driving path of the AGV;

[0082] A path adjustment module automatically selects an alternative navigation system in areas with high electromagnetic or optical interference;

[0083] The AGV speed control module introduces an adaptive speed control algorithm to dynamically adjust the AGV's operating speed based on the image processing speed and the AGV's driving needs;

[0084] Positioning and exception handling module, specifically including:

[0085] The location positioning system receives the data of the AGV navigation system in real time, performs location positioning, and transmits the located location data P loc Transmitted to the main control system, and the planned path P final To compare:

[0086] Deviation calculation formula: D loc =||P loc -P final ||;

[0087] Combining predictive control algorithms and anomaly detection technology, real-time prediction and anomaly detection of AGV driving paths are performed:

[0088] Prediction formula: Where Y is the prediction model, is the predicted position value at the next moment, P current is the current location information;

[0089] Anomaly detection formula: When D loc >D th When it is judged as an abnormal situation, D th is the deviation threshold.

[0090] When the AGV is detected to have deviated from the planned path, the main control system immediately handles the exception and adjusts the AGV's driving direction and speed:

[0091] Direction adjustment formula: θ new =θ current +Δθ, where Δθ is the angle adjustment value, θ new is the adjusted angle;

[0092] Speed ​​adjustment formula: v new =v current -Δv, where Δv is the speed adjustment value, v new is adjusted after adjustment;

[0093] When an abnormal situation cannot be handled in time, the system sends an alarm signal to notify the operator to intervene.

[0094] The initialization and data acquisition unit is comprehensive and detailed, loading multiple parameters, collecting environmental maps, and establishing a historical database, providing a rich data foundation for subsequent navigation. The path planning and guidance module integrates electromagnetic and visual navigation systems, using the LSTM algorithm and multi-visual sensor fusion technology to improve path planning accuracy. The image data processing module extracts the features of fixed objects and updates the database, enhancing system adaptability. The path deviation analysis and selection module selects the best path to improve navigation reliability. The path adjustment module can automatically switch the navigation system to deal with interference areas. The AGV speed control module implements dynamic speed regulation to meet driving needs. The position positioning and exception handling module locates and predicts the path in real time, detects anomalies, adjusts the direction and speed in a timely manner, and issues alarms when the process cannot be handled, ensuring the safe and stable operation of the AGV, effectively improving navigation accuracy, reliability, and safety, and reducing the need for manual intervention.

[0095] The second embodiment differs from the first embodiment mainly in that the path planning and guidance module operates according to the navigation system and the visual navigation system, specifically including:

[0096] Electromagnetic guidance system: calculates the path based on the electromagnetic signal and uses electromagnetic sensors to measure the electromagnetic signal strength I. When I>I th When , it is considered that there is a path in this direction;

[0097] Combine deep learning algorithms (such as LSTM) to predict and compensate electromagnetic signals:

[0098] Prediction formula: Where L is the LSTM model, is the predicted value of electromagnetic signal strength at the next moment;

[0099] Compensation formula: Among them I comp is the electromagnetic signal strength after compensation;

[0100] Visual navigation system: uses a large visual model to process road images and identify paths and fixed objects;

[0101] Introducing multi-vision sensor fusion technology to calculate the fused path:

[0102] Assume that the path calculation result of the x-th sensor is P x , with weight w x , then the fused path Where N is the number of sensors,

[0103] The image data processing module operations specifically include:

[0104] B1 Image Acquisition: Use a high-resolution camera to continuously capture the surrounding environment of the AGV and obtain real-time image data. raw ; Perform pre-processing operations such as denoising and enhancement on the collected images to improve image quality;

[0105] B2 Feature Extraction: Extract fixed object features (such as walls, pillars, fixed equipment, etc.) in the image as reference points; use feature extraction algorithms (such as SIFT, SURF, etc.) to extract feature points E points ={e1, e2, ..., e q};

[0106] B3 Environmental matching: Compare the extracted features with the environmental features in the historical database and identify unchanged objects as reference points; calculate the feature similarity S(E points , E db ), where E db When it is detected that the objects around the same location have changed and are different from the stored data, the unchanged objects for reference are extracted and marked as new reference points, and the environmental feature data in the historical database is updated;

[0107] B4 Path Calculation: Calculate the AGV's driving path based on the extracted reference points and the visual model; use a path planning algorithm (such as the A* algorithm) to calculate the optimal path P from the current position to the target position. opt .

[0108] The electromagnetic guidance system, combined with deep learning algorithms to predict and compensate for electromagnetic signals, can more accurately plan routes based on electromagnetic signals and reduce the impact of interference. The visual navigation system utilizes a large visual model and multi-sensor fusion technology to integrate multi-sensor information to plan more reliable routes. The image data processing module uses high-quality image acquisition, precise feature extraction, effective environmental matching, and path calculation based on reference points and a large visual model. This allows for real-time update of reference points, adaptation to environmental changes, and accurate calculation of the optimal path. These improvements enhance navigation accuracy and stability, strengthen the system's adaptability to complex environments, and prevent difficulty identifying and positioning during subsequent passes due to environmental changes recorded by the system.

[0109] The third embodiment differs from the first embodiment mainly in that the path deviation analysis and selection module operation specifically includes:

[0110] The main control system receives the path planning results P of the electromagnetic navigation system and the visual navigation system. em and P vis ;

[0111] Calculate the paths planned by the two systems and the paths P at the previous time point prev Deviation:

[0112] Electromagnetic navigation system path deviation

[0113] Path deviation of visual navigation system

[0114] Compare the deviations of the two systems. The one with the larger deviation is considered abnormal data. The path planned by the system with the smaller deviation is selected as the driving path P of the AGV. final .

[0115] By receiving the path planning results from both the electromagnetic and visual navigation systems, the system calculates the deviation from the previous path and compares the two. It accurately identifies abnormal data with large deviations and automatically selects the system-planned path with the smallest deviation as the driving path. This effectively improves the accuracy and reliability of path selection, reduces the risk of AGV driving anomalies caused by errors in a single navigation system, and ensures stable AGV operation.

[0116] The fourth implementation manner mainly differs from the first implementation manner in that the path adjustment module operation specifically includes:

[0117] When the AGV drives to area A with large electromagnetic interference em When the path planned by the visual navigation system is used as the main method, the frequency of image data processing f is appropriately increased. img ;

[0118] When the AGV drives to area A with large light reflection light When the path planned by the electromagnetic navigation system is used as the main method, the camera parameters (such as exposure time t exp , gain g, etc.) to reduce the impact of light reflection on image processing.

[0119] The path adjustment module combines the predictive control algorithm to predict in advance the electromagnetic interference or light reflection area that the AGV is about to enter, and adjusts the AGV's driving parameters (such as speed, steering angle, etc.) according to the prediction results. The prediction formula is expressed as:

[0120]

[0121] Where Y is the prediction model, is the regional prediction value at the next moment, A current v is the area-related information at the current moment, which includes the feature data related to the area at the current moment. For example, in the AGV application scenario, it may be the electromagnetic signal characteristics and visual image characteristics of the area where the AGV is currently located, which are used to describe the status of the current area. current is the driving speed of the AGV at the current moment, θ currentis the current driving direction angle of the AGV, which is the deflection angle relative to the specified reference direction.

[0122] When an AGV enters different interference zones, it intelligently switches the dominant navigation system. For example, in areas with significant electromagnetic interference, it prioritizes visual navigation and increases the frequency of image data processing. In areas with significant light reflection, it prioritizes electromagnetic navigation and adjusts camera parameters, effectively reducing the impact of interference. Furthermore, combined with a predictive control algorithm, it can predict the interference zone the AGV is about to enter and adjust driving parameters such as speed and steering angle accordingly. This approach of early prediction and dynamic adjustment improves the AGV's adaptability to complex environments and ensures stable and safe operation.

[0123] The fifth embodiment differs from the first embodiment mainly in that the operation of the AGV speed control module specifically includes:

[0124] Introducing the adaptive speed control algorithm, during the image processing process, according to the image processing speed T process Dynamically adjust the AGV's operating speed v according to the AGV's driving needs:

[0125] When T process >T th When the AGV running speed is reduced to v=v prev -Δv;

[0126] When T process Close to T th When the AGV is running, keep the running speed v=v prev ;

[0127] When T process <T th When there is a certain margin, increase the AGV running speed v=v prev +Δv;

[0128] Among them, T th is the set image processing speed threshold, Δv is the speed adjustment value, v prev Indicates the running speed of the AGV at the previous moment;

[0129] When the image processing speed cannot keep up with the AGV's running speed, the AGV's running speed should be adjusted or stopped to avoid the AGV's deviation due to insufficient image processing.

[0130] An adaptive speed control algorithm was introduced to dynamically adjust the AGV's speed based on image processing speed and driving requirements. The algorithm reduces speed when image processing speed exceeds a threshold, maintains speed when approaching the threshold, and increases speed when below the threshold with a margin. The algorithm can also control or stop the AGV if image processing cannot keep up with driving speed. This effectively prevents AGV deviations caused by untimely image processing, ensures stable and safe operation, and improves work efficiency.

[0131] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0132] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0133] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An AGV vehicle control and navigation system based on a large visual model, characterized in that: include: The initialization and data acquisition unit is used to initialize the system, specifically loading the electromagnetic navigation system parameters and the visual navigation system parameters; collect environmental maps, mark electromagnetic interference areas and light reflection areas, and establish an environmental feature database; record the AGV's running path, speed, steering angle, and environmental feature data; The path planning and guidance module calculates the path based on electromagnetic signals for electromagnetic navigation systems and uses the LSTM algorithm to predict and compensate for electromagnetic signals. For visual navigation systems, it uses a large visual model to process images and introduces multi-visual sensor fusion technology to calculate the path. The image data processing module is used to collect and pre-process the image data of the AGV's surrounding environment, extract the fixed object features in the image as reference points, compare them with the historical environmental features, use the unchanged objects as new reference points, and update the historical database; The path deviation analysis and selection module compares the path planning results of the electromagnetic navigation system and the visual navigation system, and selects the path planned by the system with the smallest deviation as the driving path of the AGV; A path adjustment module automatically selects an alternative navigation system in areas with high electromagnetic or optical interference; The AGV speed control module introduces an adaptive speed control algorithm to dynamically adjust the AGV's operating speed based on the image processing speed and the AGV's driving needs; The position positioning and exception handling module receives positioning data and combines the predictive control algorithm and anomaly detection technology to perform real-time prediction and anomaly detection on the AGV's driving path.

2. The AGV vehicle control and navigation system based on a visual large model according to claim 1 is characterized by: The operations of the initialization and data acquisition unit specifically include: A1 system initialization Load electromagnetic guidance system parameters: electromagnetic sensor sensitivity S em , electromagnetic signal frequency range (f min , f max ), electromagnetic signal strength threshold I th ; Load the visual navigation system parameters: visual large model input image resolution R in , model inference time threshold T th ; A2 Collect factory environment maps: Use lidar or high-precision mapping tools to mark areas with high electromagnetic interference. em and the area A with large light reflection light ; Establish an environmental feature database to store the feature vector F of each area i , where F i ={f i1 , f i2 ,...,f in }, f ij represents the jth feature of the i-th region; A3 establishes an AGV operation history database: records the AGV operation path P = {p1, p2, ..., p m }, where p k Indicates the coordinates of the kth point on the path; records the speed v k , steering angle θ k And environmental characteristic data E k .

3. The AGV vehicle control and navigation system based on a visual large model according to claim 1 is characterized by: The path planning and guidance module operations are divided into the navigation system and the visual navigation system, specifically including: Electromagnetic guidance system: calculates the path based on the electromagnetic signal and uses electromagnetic sensors to measure the electromagnetic signal strength I. When I>I th When , it is considered that there is a path in this direction; Combine deep learning algorithms to predict and compensate electromagnetic signals: Prediction formula: Where L is the LSTM model, is the predicted value of electromagnetic signal strength at the next moment; Compensation formula: Among them I comp is the electromagnetic signal strength after compensation; Visual navigation system: uses a large visual model to process road images and identify paths and fixed objects; Introducing multi-vision sensor fusion technology to calculate the fused path: Assume that the path calculation result of the x-th sensor is P x , with weight w x , then the fused path Where N is the number of sensors, 4. The AGV vehicle control and navigation system based on a visual large model according to claim 1 is characterized by: The image data processing module operations specifically include: B1 Image Acquisition: Use a high-resolution camera to continuously capture the surrounding environment of the AGV and obtain real-time image data. raw ; Perform denoising and enhancement preprocessing operations on the collected images to improve image quality; B2 Feature extraction: Extract the features of fixed objects in the image as reference points; use feature extraction algorithms to extract feature points E points ={e1, e2, ..., e q }; B3 Environmental matching: Compare the extracted features with the environmental features in the historical database and identify unchanged objects as reference points; calculate the feature similarity S(E points , E db ), where E db When it is detected that the objects around the same location have changed and are different from the stored data, the unchanged objects for reference are extracted and marked as new reference points, and the environmental feature data in the historical database is updated; B4 Path Calculation: Calculate the AGV's driving path based on the extracted reference points and visual model; use the path planning algorithm to calculate the optimal path P from the current position to the target position opt .

5. The AGV vehicle control and navigation system based on a visual large model according to claim 2 is characterized by: The path deviation analysis and selection module operations specifically include: The main control system receives the path planning results P of the electromagnetic navigation system and the visual navigation system. em and P vis ; Calculate the paths planned by the two systems and the paths P at the previous time point prev Deviation: Electromagnetic navigation system path deviation Path deviation of visual navigation system Compare the deviations of the two systems. The one with the larger deviation is considered abnormal data. The path planned by the system with the smaller deviation is selected as the driving path P of the AGV. final .

6. The AGV vehicle control and navigation system based on a visual large model according to claim 2 is characterized by: The path adjustment module operations specifically include: When the AGV drives to area A with large electromagnetic interference em When the path planned by the visual navigation system is used as the main method, the frequency of image data processing f is appropriately increased. img ; When the AGV drives to area A with large light reflection light When performing the navigation, the path planned by the electromagnetic navigation system is mainly used, and the camera parameters are adjusted to reduce the impact of light reflection on image processing.

7. The AGV vehicle control and navigation system based on a visual large model according to claim 6 is characterized by: The path adjustment module combines the predictive control algorithm to predict in advance the electromagnetic interference or light reflection area that the AGV is about to enter, and adjusts the driving parameters of the AGV according to the prediction results. The prediction formula is expressed as: Where Y is the prediction model, is the regional prediction value at the next moment, A current It is the region-related information at the current moment, including the feature data related to the region at the current moment, used to describe the state of the current region; current is the driving speed of the AGV at the current moment, θ current is the current driving direction angle of the AGV, which is the deflection angle relative to the specified reference direction.

8. The AGV vehicle control and navigation system based on a large visual model according to claim 2 is characterized by: The operation of the AGV speed control module specifically includes: Introducing the adaptive speed control algorithm, during the image processing process, according to the image processing speed T process Dynamically adjust the AGV's operating speed v according to the AGV's driving needs: When T process >T th When the AGV running speed is reduced to v=v prev -Δv; When T process Close to T th When the AGV is running, keep the running speed v=v prev ; When T process <T th When there is a certain margin, increase the AGV running speed v=v prev +Δv; Among them, T th is the set image processing speed threshold, Δv is the speed adjustment value, v prev Indicates the running speed of the AGV at the previous moment; When the image processing speed cannot keep up with the AGV's running speed, the AGV's running speed should be adjusted or stopped to avoid the AGV's deviation due to insufficient image processing.

9. The AGV vehicle control and navigation system based on a visual large model according to claim 7 is characterized in that: The operations of the position positioning and exception handling module specifically include: The location positioning system receives the data of the AGV navigation system in real time, performs location positioning, and transmits the located location data P loc Transmitted to the main control system, and the planned path P final To compare: Deviation calculation formula: D loc =||P loc -P final ||; Combining predictive control algorithms and anomaly detection technology, real-time prediction and anomaly detection of AGV driving paths are performed: Prediction formula: Where Y is the prediction model, is the predicted position value at the next moment, P current is the current location information; Anomaly detection formula: When D loc >D th When it is judged as an abnormal situation, D th is the deviation threshold.

10. The AGV vehicle control and navigation system based on a visual large model according to claim 9 is characterized in that: When the AGV is detected to have deviated from the planned path, the main control system immediately handles the exception and adjusts the AGV's driving direction and speed: Direction adjustment formula: θ new =θ current +Δθ, where Δθ is the angle adjustment value, θ new is the adjusted angle; Speed ​​adjustment formula: v new =v current -Δv, where Δv is the speed adjustment value, v new is adjusted after adjustment; When an abnormal situation cannot be handled in time, the system sends an alarm signal to notify the operator to intervene.

Images