A kind of intelligent warehouse automatic inspection system and method

Abstract

The application provides a kind of intelligent warehouse automatic inspection system and method, it is related to intelligent inspection technical field, including inspection robot, 5G communication transmission module and remote computer, inspection robot includes: environmental perception module, generates and updates environmental map, and identifies obstacle information;Autonomous navigation control module includes: path planning unit, generates inspection path, and carries out local path adjustment;Obstacle avoidance unit, generates obstacle avoidance strategy;Motion control unit, controls the movement of inspection robot;Data acquisition module, acquires information in warehouse;Data processing and analysis module are used for processing and analysis to the collected multi-source data, output monitoring report and early warning information;Wherein, 5G communication transmission module connects inspection robot and remote computer, for transmitting the data output in inspection robot each module, and receiving the instruction sent by remote computer.The application realizes the real-time perception of warehouse environment, intelligent obstacle avoidance and autonomous inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent inspection, and in particular to an intelligent warehouse automatic inspection system and method. BACKGROUND

[0002] Warehouse inspection plays a crucial role in grain storage and management. Effective inspection not only helps to timely detect and handle problems such as temperature and humidity abnormalities, insect infestation, and mold during grain storage, ensuring grain quality and safety, but also prevents potential safety hazards, ensuring the overall operational efficiency and economic benefits of the warehouse. With the increase in grain reserves and the complexity of storage conditions, automated and intelligent inspection methods are particularly important to improve the coverage and accuracy of inspection, reduce labor costs, and improve management levels.

[0003] Currently, warehouse inspection mainly relies on manual inspection, unmanned aerial vehicle inspection, and traditional inspection robots. Although manual inspection is intuitive, it has low efficiency, limited coverage, and low detection accuracy. Unmanned aerial vehicle inspection improves the coverage and efficiency of inspection to some extent and is suitable for large-area warehouse environmental monitoring. However, there are still some technical challenges in the operation of unmanned aerial vehicles in complex terrain. Traditional inspection robots achieve automatic inspection by carrying various sensors, but further improvements are needed in autonomous navigation, obstacle recognition, and environmental adaptability to meet the diverse and complex inspection needs of large warehouses.

[0004] The existing technology still faces many deficiencies in the process of warehouse inspection. First, many inspection devices have limited adaptability to environmental changes and are difficult to maintain stable operation in complex and variable warehouse environments. Second, the degree of automation of data collection and processing is not high, which limits the real-time and accuracy of the inspection results. In addition, the existing inspection system lacks the ability to manage energy and autonomously charge, making it impossible to achieve long-term and continuous inspection tasks. These defects not only affect the efficiency and effectiveness of inspection, but also increase the cost of maintenance and operation. SUMMARY

[0005] Therefore, the present application proposes an intelligent warehouse automatic inspection system and method, which integrates environmental perception, autonomous navigation, multi-source data collection, and other functional modules, combines deep learning and reinforcement learning algorithms, realizes real-time perception, intelligent obstacle avoidance, and autonomous inspection of the warehouse environment, and uses edge computing technology to process and analyze the collected data, ultimately achieving the purpose of improving inspection efficiency, ensuring job safety, and reducing labor costs.

[0006] The technical solution of the present application is as follows:

[0007] In one aspect, the application provides a smart granary automatic inspection system, comprising an inspection robot, a 5G communication transmission module and a remote computer, the inspection robot comprising:

[0008] An environment perception module is provided with a laser radar and a depth camera, which is used to obtain three-dimensional data of the granary environment in real time, generate and update an environment map, and identify obstacle information on the inspection path;

[0009] An autonomous navigation control module is connected with the environment perception module, comprising:

[0010] A path planning unit is based on a deep learning algorithm to analyze the environment map in real time, generate an inspection path, and receive an obstacle avoidance strategy from the obstacle avoidance unit for local path adjustment;

[0011] An obstacle avoidance unit adopts a reinforcement learning algorithm to generate an obstacle avoidance strategy based on the obstacle information identified by the environment perception module, and feeds back the obstacle avoidance strategy to the path planning unit to trigger local path planning;

[0012] A motion control unit receives and integrates the inspection path and the obstacle avoidance strategy to control the movement of the inspection robot;

[0013] A data acquisition module is provided with a temperature and humidity sensor, a high-definition camera and a video acquisition device, which is used to collect temperature, humidity, image and video information in the granary;

[0014] A data processing and analysis module adopts edge computing technology and is connected with the data acquisition module to process and analyze the collected multi-source data, output monitoring reports and early warning information;

[0015] The 5G communication transmission module connects the inspection robot with the remote computer, and is used to transmit the data output by each module in the inspection robot, and receive the instructions sent by the remote computer.

[0016] Preferably, the generation process of the environment map comprises:

[0017] A1、The laser radar performs all-around scanning at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud data set, divides the two-dimensional space into g×g grid units, wherein g is the grid size, each grid unit is identified by its center coordinate, and the occupancy probability P(O j ) of each grid unit is calculated by using the Bayesian update formula, a first occupancy probability threshold θ1 is set, and the grid with P(O j ) greater than the first occupancy probability threshold θ1 is marked as an occupied grid, and vice versa as an idle grid;

[0018] A2, the depth camera collects three-dimensional point cloud data of the surrounding environment at a higher frequency, for each occupancy grid, according to the height data provided by the depth camera, adjust the height information of the occupancy grid, form a three-dimensional occupancy grid; using the voxel grid method, the three-dimensional occupancy grid and the depth camera data are locally fused, the noise points are eliminated and the data gaps are filled, and the environment map extended to three-dimensional space is generated;

[0019] A3, set the parameters of grid cell subdivision, including curvature threshold κ th and occupancy difference threshold ΔP th ; according to the curvature of the grid cell and the curvature threshold, the grid cell is further subdivided into smaller sub-grid cells; calculate the occupancy probability change of the grid cell, respectively representing the occupancy probability of the grid cell j in the new and old states, if ΔP> ΔP th , continue to subdivide the grid cell, if ΔP<- ΔP th , a plurality of low change sub-grid cells are combined into a larger grid cell;

[0020] A4, after the grid cell subdivision is finished, the generated environment map is output.

[0021] Preferably, in step 3, the curvature threshold κ th is divided into high curvature threshold κ high and low curvature threshold κ low , the curvature κ of each grid cell is calculated, and the subdivision intensity is adjusted based on the adaptive weighting coefficient ω(κ):

[0022]

[0023] When κ≤κ low , ω(κ)=1, keep the current grid size, do not subdivide;

[0024] When κ low <κ≤κ high , ω(κ) increases from 0 to 1 in a linear manner, gradually increasing the subdivision intensity;

[0025] When κ>κ high , ω(κ)=1, the high curvature grid cell is subdivided with maximum intensity.

[0026] Preferably, the update process of the environment map is:

[0027] Using the IMU and odometer equipped on the inspection robot, real-time dynamic data is obtained;

[0028] Using Kalman filtering algorithm, fuse IMU and odometer data, estimate the current pose of the inspection robot, and regularly correct the pose using fixed feature points in the surrounding environment;

[0029] Based on the current pose and moving direction of the inspection robot, define the environment map area A that needs to be updated, and dynamically adjust the range and shape of area A according to the moving speed of the inspection robot and the rate of environmental change;

[0030] During the inspection process, the inspection robot continuously collects data of the laser radar and depth camera, obtains new three-dimensional data, and performs filtering, denoising and registration on the new three-dimensional data to complete preprocessing;

[0031] Fuse the preprocessed new three-dimensional data into the area A of the environment map, analyze the occupancy probability and terrain curvature of the updated area A, detect whether the grain pile form has changed significantly, and if the grain pile form changes, call the grid subdivision algorithm to re-subdivide or merge the grid cells to complete the environment map update.

[0032] Preferably, the generation process of the inspection path is as follows:

[0033] B1, input the environment map to the path planning unit, the environment map is three-dimensional grid data, each grid cell contains position information (x, y, z), occupancy probability P(O j ) and curvature κ;

[0034] B2, set a second occupancy probability threshold θ2, according to the second occupancy probability threshold θ2 and the occupancy probability of the grid cell, screen out the passable grid cells for marking, and normalize the curvature to complete the preprocessing of the environment map;

[0035] B3, input the preprocessed environment map and the current state information of the inspection robot into the neural network model, extract local features and global features, and the model outputs the risk score U j of each grid cell, reflecting the potential obstacle density and dynamic change risk of the grid cell, wherein the neural network model adopts a hybrid model based on convolutional neural network and graph neural network, and its structure includes multiple 3D convolution layers, graph convolution layers and fully connected layers; the calculation formula of the risk score is:

[0036] U j =f CNN-GNN (I j ,Q)

[0037] In the formula, I j represents the feature set of the grid cell j, f CNN-GNN represents the trained neural network model, and Q represents the state information of the inspection robot;

[0038] B4, generate an optimized heuristic function h *(n), the deep learning model comprises a spatial attention layer, a time attention layer and a position encoding module, the heuristic function h * (n) is defined as:

[0039] h * (n)=f Heuristic (n,G,Q)

[0040] wherein n is a current search node, G is an environment map, f Heuristic is a deep learning model;

[0041] B6, under the guidance of the heuristic function, performing path search through the improved A* algorithm to obtain a preliminary inspection path;

[0042] B7, performing smoothing processing on the preliminary inspection path using a spline curve to obtain a smoothed inspection path;

[0043] B8, detecting obstacles or environmental changes on the path according to the latest environment map and obstacle avoidance strategy, and triggering local path planning according to the detection result, and fine-tuning the inspection path in combination with the heuristic function.

[0044] Preferably, the improved A* algorithm process is as follows:

[0045] C1, setting a starting point S and a target point T, initializing an open list and a closed list, the open list being used for storing nodes to be explored and initially containing only the starting point, the closed list being used for storing nodes that have been explored and initially being empty; calculating an initial cost g(S) of the starting point as 0, calculating a heuristic estimated cost h * (S) of the starting point as f Heuristic (S,G,Q), and calculating a total cost f(S) of the starting point as g(S)+h * (S), creating starting point node information and adding the starting point node information to the open list;

[0046] C2, when the open list is not empty and the target point is not found, performing the following steps:

[0047] C21, selecting a node n with the lowest f(n)=g(n)+h * (n) from the open list and moving to the closed list, wherein f(n) is a cost function and g(n) is an actual cost from the starting point to the node n;

[0048] C22, if the node n is the target point, constructing a final path and terminating the algorithm;

[0049] C23, otherwise, expanding adjacent nodes of the node n, and performing the following operations on each adjacent node m:

[0050] C231, if the node m is already in the closed list, skipping the node;

[0051] C232, the actual cost g(m) of the computing node:

[0052] g(m) = g(n) + cost(n,m) + aU m

[0053] wherein cost(n,m) is the moving cost of node n to node m, U m is the risk score of node m, and a is the risk weight coefficient;

[0054] C233, if node m is not in the open list, add node m to the open list, record its parent node as n, calculate its heuristic estimated cost h * (m), and further calculate its total cost f(m) = g(m) + h * (m);

[0055] C234, if node m is already in the open list and the new g(m) is lower, update g(m) of node m, update its parent node as n, and recalculate the total cost f(m) = g(m) + h * (m);

[0056] C24, repeat steps C21-C23, select the node with the lowest f(n) in the open list for expansion, for each expanded node, update g(m), h * (m) and f(m) of its neighbor nodes until the target point is found;

[0057] C3, starting from the target point, trace the parent nodes of the nodes in reverse, and construct a complete path from the starting point to the target point, i.e. the preliminary inspection path.

[0058] Preferably, the identification process of the obstacle information is:

[0059] D1, obtain the surround scan data of the laser radar, eliminate noise points by median filtering, convert the polar coordinate data to point cloud data in the Cartesian coordinate system, collect the depth image stream of the depth camera, remove invalid points and outliers, and convert the depth data to three-dimensional point cloud format;

[0060] D2, apply DBSCAN algorithm to the laser radar point cloud data for dynamic clustering; analyze the characteristics of each cluster, and preliminarily screen the possible obstacle regions; establish a preliminary obstacle candidate region list, and record the position and basic feature information thereof;

[0061] D3, for the preliminary identified obstacle candidate region, extract the corresponding depth image region; calculate the depth gradient information, extract the obstacle boundary feature; combine the depth information and the boundary feature, verify and refine the three-dimensional geometric feature of the obstacle;

[0062] D4, establish an obstacle feature database to record the position, size and feature information of the identified obstacles; associate the current frame detection result with the historical record in the feature database through a feature matching method; use a Kalman filter to track the motion state of the obstacle and predict its position change; when the obstacle is stably detected in multiple consecutive frames, confirm its validity and update the environment map.

[0063] Preferably, the obstacle avoidance unit makes an obstacle avoidance decision through a reinforcement learning algorithm, specifically including:

[0064] E1, extract the relative position information of the obstacle, including the distance and azimuth angle of the obstacle to the inspection robot; obtain the current motion state of the inspection robot, including the position, speed and attitude angle; calculate the local terrain features, including the ground inclination and the surface undulation of the grain pile; integrate the above information to form a state vector as the input of the obstacle avoidance decision;

[0065] E2, define a basic motion instruction set; specify the corresponding linear and angular velocity parameters for each basic action; establish safety constraints for action execution, including maximum speed and acceleration limits;

[0066] E3, design a reward function to give reward feedback to the action of the inspection robot;

[0067] E4, based on the current state, use Q-learning to calculate the expected return of each possible action; combine the ε-greedy strategy to balance between exploration and utilization; select the optimal action to generate the obstacle avoidance strategy.

[0068] Preferably, the reward function is calculated as follows:

[0069] R = β1·r1-β2·r2+β3·r3

[0070] In the formula, r1 is the basic obstacle avoidance reward value:

[0071]

[0072] r2 is the path deviation penalty term:

[0073] r2 = d 偏离 × γ

[0074] Where d 偏离 is the shortest distance between the current pose of the inspection robot and the predetermined inspection path; γ is the penalty coefficient of path deviation;

[0075] r3 is the task completion reward value:

[0076]

[0077] β1, β2, β3 are weight coefficients, and β1> β2> β3 is set.

[0078] In another aspect, the present application also provides an intelligent warehouse automatic inspection method, which is executed in the system described in any of the above, and the method comprises:

[0079] S1, start the inspection robot, initialize the environment perception module, autonomous navigation control module and 5G communication transmission module, and establish a communication connection with the remote computer;

[0080] S2, synchronously start the laser radar and the depth camera, collect two-dimensional and three-dimensional point cloud data of the surrounding environment according to a predetermined frequency, and simultaneously start the temperature and humidity sensor and the high-definition camera to collect related environmental parameters;

[0081] S3, convert the laser radar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, perform data fusion through the voxel grid method, and generate and subdivide a three-dimensional environment map;

[0082] S4, use IMU and odometry data, combine Kalman filter algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environment map;

[0083] S5, input the preprocessed environment map into the path planning unit, extract environmental features based on the deep learning model, generate and smooth the preliminary inspection path using the improved A* algorithm, and adjust the path according to real-time environmental changes;

[0084] S6, apply DBSCAN algorithm to cluster analysis of laser radar point cloud, combine boundary feature extraction of depth camera and Kalman filter for dynamic tracking and confirmation of obstacles, and update obstacle information in the environment map;

[0085] S7, extract the relative position of the obstacle, the motion state of the inspection robot and the local terrain features, integrate the state vectors required for obstacle avoidance decision, based on the constructed state vectors, use Q-learning in the reinforcement learning algorithm to calculate the expected return of each action, combine the ε-greedy strategy to select the optimal obstacle avoidance action, and generate the corresponding obstacle avoidance strategy;

[0086] S8, according to the obstacle avoidance strategy, perform segmented speed planning and attitude adjustment, and use a closed-loop control system to execute the motion instructions to control the movement of the inspection robot;

[0087] S9, real-time monitor the environmental changes and obstacle avoidance effect in the inspection process, if an abnormality or significant environmental change is detected, update the environment map and adjust the inspection path in time, and feedback the inspection state and early warning information to the remote computer through the 5G communication transmission module.

[0088] The present application has the following beneficial effects relative to the prior art:

[0089] (1) The intelligent warehouse automatic inspection system provided by the present application realizes real-time perception, intelligent obstacle avoidance and autonomous inspection of the warehouse environment by integrating an environment perception module, an autonomous navigation control module and a data processing and analysis module, combining 5G communication transmission technology, improving the automation level and efficiency of inspection operations, and ensuring the safety and reliability of inspection operations;

[0090] (2) The present application adopts an environment perception scheme that fuses laser radar and depth camera, dynamically updates the environment map through an adaptive grid subdivision algorithm, and combines a curvature-based subdivision intensity adjustment mechanism to accurately depict the geometric features of complex terrain in the warehouse, improving the accuracy and real-time performance of environment modeling;

[0091] (3) The path planning scheme designed by the present application generates an optimized heuristic function through a deep learning model, and combines an improved A* algorithm for path search, not only considering the path length, but also incorporating environmental risk scores into the cost function, so that the generated inspection path not only meets the shortest path requirement, but also effectively avoids high-risk areas, improving the intelligence and practicality of path planning;

[0092] (4) The obstacle recognition method based on DBSCAN clustering and multi-sensor fusion proposed by the present application can accurately recognize and predict the position and motion state of obstacles through comprehensive analysis of depth information and boundary features, combined with a Kalman filter for dynamic tracking;

[0093] (5) The present application adopts an obstacle avoidance decision mechanism based on Q-learning, which designs a multi-dimensional reward function, and comprehensively considers obstacle avoidance effect, path deviation and task completion, so that the obstacle avoidance strategy can maintain high inspection efficiency while ensuring safety, effectively solving the obstacle avoidance problem in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0094] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0095] Figure 1 The system framework diagram of the present application

[0096] Figure 2 The method flowchart of the present application. DETAILED DESCRIPTION

[0097] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0098] As shown in Figure 1 The present application provides an automatic inspection system for intelligent granary, which comprises an inspection robot, a 5G communication transmission module and a remote computer. The inspection robot comprises:

[0099] An environment perception module is provided with a laser radar and a depth camera, which is used to obtain three-dimensional data of the granary environment in real time, generate and update the environment map, and identify obstacle information on the inspection path;

[0100] An autonomous navigation control module is connected with the environment perception module, which comprises:

[0101] A path planning unit is based on a deep learning algorithm to analyze the environment map in real time, generate an inspection path, and receive an obstacle avoidance strategy from the obstacle avoidance unit for local path adjustment;

[0102] An obstacle avoidance unit adopts a reinforcement learning algorithm to generate an obstacle avoidance strategy based on the obstacle information identified by the environment perception module, and feeds back the obstacle avoidance strategy to the path planning unit to trigger local path planning;

[0103] A motion control unit receives and integrates the inspection path and the obstacle avoidance strategy to control the movement of the inspection robot;

[0104] A data acquisition module is provided with a temperature and humidity sensor, a high-definition camera and a video acquisition device, which is used to collect temperature, humidity, image and video information in the granary;

[0105] A data processing and analysis module adopts edge computing technology and is connected with the data acquisition module, which is used to process and analyze the collected multi-source data, and output monitoring reports and early warning information;

[0106] The 5G communication transmission module connects the inspection robot with the remote computer, which is used to transmit the data output by each module in the inspection robot, and receive the instructions sent by the remote computer.

[0107] Specifically, the inspection robot is the core execution unit of the system, responsible for autonomous inspection and data collection within the warehouse. The main components include: chassis platform: provides mechanical structure support, ensures the stable operation of the robot on complex terrain. Battery pack: provides long-term energy support for the robot, with high energy density and fast charging capability. Servo motor: drives the movement of the robot and the action of the mechanical arm, realizes autonomous navigation and object operation. Laser radar: for environment perception and map construction, real-time scanning of surrounding obstacles, supporting obstacle avoidance and path planning. Jetson Nano: embedded computing platform, responsible for processing sensor data, running AI algorithms, and making real-time decisions. TN 7-inch display screen: provides a human-machine interaction interface, displays robot status information and inspection data. Temperature and humidity sensor: collects temperature and humidity data in the warehouse, monitors environmental conditions. Depth camera: obtains three-dimensional image data, detects grain quality such as mold and insect damage. Infrared sensor: measures ambient temperature, assists in temperature and humidity monitoring. SF6 harmful gas detection module: monitors the concentration of harmful gases in the environment to ensure the safety of the warehouse. Mechanical claw: used to clamp the grain on the grain pile to assist in quality detection and handling. Obstacle avoidance system: combines laser radar and depth camera data to adjust the path in real time to avoid collisions.

[0108] 5G communication module realizes high-speed and low-latency data transmission, ensuring real-time communication between the robot and the remote computer. It supports large bandwidth and high reliability wireless connection, ensuring timely uploading of inspection data and rapid issuance of instructions.

[0109] The remote computer as the control and data processing center of the system undertakes the following functions: data processing and analysis: store, process and analyze the data collected by the robot such as temperature, humidity, image, gas concentration, etc., and generate inspection report. Map construction and path planning: based on the data of laser radar and depth camera, real-time update the warehouse environment map, optimize the inspection path. Remote control and monitoring: through the host terminal, the management personnel can issue inspection instructions, and monitor the status and work progress of the inspection robot in real time. System management: maintain the software and hardware of the system, ensure the collaborative operation of each component.

[0110] The local area network deployed inside the warehouse is used to connect the inspection robot, remote computer and other network equipment, ensuring efficient data transmission and stable operation of the system.

[0111] The inspection system of the present application includes the following functions:

[0112] Environment perception and map construction: through laser radar and depth camera, the inspection robot can generate a high-precision three-dimensional environment map. Laser radar is responsible for scanning surrounding obstacles, and depth camera provides detailed terrain and object information. The system uses voxel grid method to fuse data, eliminate noise, and ensure the accuracy and real-time updating ability of the map.

[0113] Autonomous navigation and path planning: The inspection robot uses deep learning algorithms and reinforcement learning algorithms on Jetson Nano to achieve autonomous navigation and path planning. Combined with the obstacle avoidance system, the robot can intelligently adjust the path in the complex and variable warehouse environment to ensure efficient completion of the inspection task.

[0114] Data collection and transmission: The robot is equipped with various sensors to collect real-time data such as temperature and humidity, environmental images, and gas concentration. Through the 5G communication module, the collected data is quickly transmitted to the remote computer, supporting real-time monitoring and data analysis.

[0115] Autonomous obstacle avoidance and motion control: The obstacle avoidance system combines the data of laser radar and depth camera to identify and track obstacles in real time, and uses Q-learning algorithm for obstacle avoidance decision. The robot realizes precise motion control through servo motor and closed-loop control system, ensuring stable operation on complex terrain.

[0116] Object operation and quality detection: The mechanical claw module enables the robot to pick up and handle grains on the grain pile, assisting in quality detection. The collected data is analyzed by the host computer to help management personnel timely discover and handle grain quality problems.

[0117] In an embodiment of the present application, the generation process of the environment map includes:

[0118] A1, the laser radar performs full-range scanning at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud data set, divides the two-dimensional space into g×g grid units, wherein g is the grid size, each grid unit is identified by the center coordinates, and the occupancy probability P(O j ) of each grid unit is calculated using the Bayesian update formula, and a first occupancy probability threshold θ1 is set. The grid with P(O j ) greater than the first occupancy probability threshold θ1 is marked as an occupied grid, and vice versa as an idle grid.

[0119] Specifically, the laser radar performs 360° full-range scanning at a fixed scanning frequency of 30Hz; 1080 distance data points are obtained each time to form a two-dimensional point cloud data set; the scanning angle resolution is 0.33°, and the measurement range is 0.1m to 30m. The two-dimensional space is divided into g×g grid units, g=0.1m, and each grid unit is uniquely identified by the center coordinates (x c ,y c ), and a coordinate system conversion matrix is established to convert the data in the laser radar coordinate system to the global coordinate system.

[0120] The calculation method of the occupancy probability P(O j ) is as follows:

[0121]

[0122] where P(Z|O j ) is the probability of the cell j being occupied given the measurement Z; P(O j ) is the prior occupancy probability; P(Z) is the total probability of the measurement data; set the first occupancy probability threshold θ1=0.65.

[0123] A2, the depth camera collects the three-dimensional point cloud data of the surrounding environment at a higher frequency, for each occupancy grid, adjusts the height information of the occupancy grid according to the height data provided by the depth camera, forms a three-dimensional occupancy grid; using the voxel grid method, the three-dimensional occupancy grid and the depth camera data are locally fused, the noise points are eliminated and the data gaps are filled, and the environment map extended to three-dimensional space is generated;

[0124] Specifically, the depth camera collects three-dimensional point cloud data of the environment at a frequency of 60Hz; the field of view angle is 87° horizontally and 58° vertically; the depth measurement range is 0.5m to 6m. The voxel grid method is used for downsampling and fusion of the data; the voxel size is set to 0.05m*0.05m*0.05m; the statistical characteristics of the point cloud data in each voxel are calculated; the abnormal points and noise data are removed; the nearest neighbor interpolation method is used to fill the data gaps.

[0125] A3, set the parameters of grid cell subdivision, including curvature threshold κ th and occupancy difference threshold ΔP th ; according to the curvature of the grid cell and the curvature threshold, the grid cell is further subdivided into smaller sub-grid cells; calculate the occupancy probability change of the grid cell, respectively representing the occupancy probability of the grid cell j in the new and old states, if ΔP>ΔP th , continue to subdivide the grid cell, if ΔP<-ΔP th , combine multiple low change sub-grid cells into a larger grid cell;

[0126] where the curvature threshold κ th is divided into high curvature threshold κ high and low curvature threshold κ low , the curvature κ of each grid cell is calculated, and the subdivision intensity is adjusted based on the adaptive weighting coefficient ω(κ):

[0127]

[0128] where ω(κ)=1 when κ≤κ low , the current grid size is maintained and no subdivision is performed;

[0129] when κlow <κ≤κ high When κ < κ

[0130] When κ > κ high , ω(κ) = 1, the maximum degree of subdivision is performed on the high-curvature grid cell.

[0131] The κ calculation formula is as follows:

[0132]

[0133] Wherein, h j is the height of the current grid, h j-1 , h j+1 is the height of the adjacent grid, and Δx is the grid spacing.

[0134] The weighting coefficient ω(κ) is calculated according to the curvature κ, the subdivision level is determined according to the weighting coefficient, the grid to be subdivided is divided into four sub-grids, the new coordinate identifier of the sub-grid is assigned, the occupancy probability of the parent grid is inherited, and the local data is updated.

[0135] The occupancy probability change ΔP of the adjacent grid cell is calculated, if ΔP > ΔP th , the grid cell is continuously subdivided, if ΔP < -ΔP th , the adjacent sub-grid cell is checked, if the sub-grid cell has similar features, it is merged into a larger grid cell.

[0136] A4、After the grid cell subdivision is completed, the generated environment map is output.

[0137] A grid index table is established to record the attribute information of each grid: position coordinates (x, y, z), occupancy probability P(O j ), curvature value κ, and subdivision level.

[0138] An environment map data file is generated, which contains grid attribute information and topological relationship; a map update timestamp is provided; and map visualization data is output.

[0139] In an embodiment of the present application, the update process of the environment map is as follows:

[0140] The IMU and odometer equipped on the inspection robot are used to obtain dynamic data in real time; the IMU is used to obtain acceleration and angular velocity data of the inspection robot in real time. The odometer measures the travel distance and direction change of the inspection robot.

[0141] The Kalman filter algorithm is adopted to fuse the IMU and odometry data to estimate the current pose of the inspection robot, and fixed feature points in the surrounding environment are regularly used for pose correction; the steps of Kalman filtering include: a prediction step of predicting the current position and attitude according to the IMU data. An updating step of combining the odometry data to correct the predicted value and improve the estimation accuracy. A pose correction process: feature point extraction: extracting stable environmental feature points from the laser radar and depth camera data. Matching and correction: matching the real-time detected feature points with the feature points in the environmental map to adjust the pose estimation.

[0142] Based on the current pose and moving direction of the inspection robot, the environmental map area A that needs to be updated is defined, and the range and shape of the area A are dynamically adjusted according to the moving speed of the inspection robot and the environmental change rate; the definition of the area A considers the following factors: the moving speed of the robot: a higher moving speed may require a larger or more frequent update area. Environmental change rate: if the environment changes rapidly, the area A should be larger to cover the potential change area. Range adjustment: increase or decrease the radius of the area A to adapt to different moving speeds. Shape adjustment: adjust the shape of the area A according to the moving direction and environmental complexity, such as adjusting from a circular shape to an elliptical shape to improve efficiency.

[0143] During the inspection process, the inspection robot continuously collects data from the laser radar and depth camera to obtain new three-dimensional data, and performs filtering, denoising and registration on the new three-dimensional data to complete preprocessing.

[0144] The preprocessed new three-dimensional data is fused into the area A of the environmental map, and the occupancy probability and terrain curvature of the updated area A are analyzed to detect whether the grain pile morphology has changed significantly, and if the grain pile morphology changes are detected, the grid subdivision algorithm is called to re-subdivide or merge the grid cells to complete the environmental map update.

[0145] Specifically, the grain pile morphology change detection identifies significant differences by comparing the terrain curvature and occupancy probability before and after updating, and sets threshold values for curvature and occupancy probability changes to determine whether to trigger further processing.

[0146] If significant changes in the grain pile morphology are detected, the grid subdivision algorithm is called to re-subdivide or merge the grid cells.

[0147] The updated map data is stored in the database of the system for subsequent path planning and obstacle avoidance decision-making. Through the 5G communication module, the updated map information is transmitted to the remote computer in real time for monitoring and management personnel to view.

[0148] In an embodiment of the present application, the generation process of the inspection path is:

[0149] B1, input an environment map to a path planning unit, the environment map being three-dimensional grid data, each grid cell containing position information (x, y, z), an occupancy probability P(O j ) and a curvature K;

[0150] B2, set a second occupancy probability threshold θ2, filter out passable grid cells according to the second occupancy probability threshold θ2 and the occupancy probability of the grid cells, mark the passable grid cells, normalize the curvature, and complete preprocessing of the environment map;

[0151] B3, input the preprocessed environment map and state information of a current inspection robot into a neural network model, extract local features and global features, and output a risk score U j of each grid cell, reflecting potential obstacle density and dynamic change risk of the grid cell, wherein the neural network model adopts a hybrid model based on a convolutional neural network and a graph neural network, and the structure of the hybrid model includes a plurality of 3D convolutional layers, a graph convolutional layer and a fully connected layer; the 3D convolutional layers process spatial information and extract spatial features in the environment. The graph convolutional layer models the relationship between the grid cells and captures global environmental features. Multi-stage residual connection enhances the expression ability of the model and captures feature information of different scales. The calculation formula of the risk score is:

[0152] U j =f CNN-GNN (I j ,Q)

[0153] In the formula, I j represents a feature set of the grid cell j, f CNN-GNN represents a trained neural network model, and Q represents state information of the inspection robot;

[0154] The neural network model includes a training process, and the training data is a large amount of inspection path data, including path selection and obstacle avoidance records in different environments. The data labeling includes successful obstacle avoidance paths and collision paths. The training process adopts a supervised learning method, and is trained by minimizing a loss function between a predicted path score and an actual path effect. Cross-validation and early stopping mechanism are used to prevent overfitting and ensure the generalization ability of the model.

[0155] B4, generate an optimized heuristic function h * (n) through a deep learning model, the deep learning model including a spatial attention layer, a temporal attention layer and a position encoding module, the spatial attention layer focusing on the edges of the grain pile and high-risk areas to improve the perception ability of the model to key areas. The temporal attention layer considers the changes of dynamic obstacles to improve the adaptability of the model to dynamic changes of the environment. The position encoding module encodes the three-dimensional position information of the nodes into high-dimensional features to help the model understand spatial relationships and relative distances. The heuristic function h *(n) is defined as:

[0156] h * (n)=f Heuristic (n,G,Q)

[0157] where n is the current search node, G is the environment map, f Heuristic is the deep learning model;

[0158] The deep learning model includes a training process, which collects diverse three-dimensional environment maps and corresponding optimal path data as a training set. The optimal path can be generated by expert annotation or using the traditional A* algorithm. The training process inputs the position information of node n, the environment map G, and the robot state information Q. The output is the heuristic estimate value h * (n) of node n. The loss function uses mean square error (MSE) to minimize the difference between h * (n) and the true path cost. The optimization algorithm uses the Adam optimizer to train the model, adjusting parameters to improve prediction accuracy.

[0159] B6. Under the guidance of the heuristic function, the improved A* algorithm is used for path search to obtain the preliminary inspection path.

[0160] B7. The preliminary inspection path is smoothed using a spline curve to obtain the smoothed inspection path. The preliminary generated inspection path may have sharp turns and be not smooth. To improve the running efficiency and stability of the inspection robot, a spline curve is introduced for path smoothing.

[0161] Specifically, a cubic spline curve is used for smoothing.

[0162] B8. According to the latest environment map and obstacle avoidance strategy, the obstacles or environmental changes on the path are detected, and according to the detection results, local path planning is triggered, combined with the heuristic function, to fine-tune the inspection path.

[0163] The improved A* algorithm process is as follows:

[0164] C1. Set the starting point S and the target point T, initialize the open list and the closed list, the open list is used to store the nodes to be explored, initially containing only the starting point; the closed list is used to store the nodes that have been explored, initially empty; calculate the initial cost of the starting point g(S)=0, calculate the heuristic estimate cost of the starting point h * (S)=f Heuristic (S,G,Q), the total cost f(S)=g(S)+h * (S), create the starting point node information and add it to the open list;

[0165] C2. When the open list is not empty and the target point is not found, the following steps are executed:

[0166] C21, select a node n with the lowest f(n) = g(n) + h * (n) from the open list and move to the closed list, wherein f(n) is a cost function, g(n) is an actual cost from the start point to the node n;

[0167] C22, if n is the target point, construct the final path, and the algorithm terminates;

[0168] C23, otherwise, expand the adjacent nodes of the node n, and for each adjacent node m, perform the following operations:

[0169] C231, if the node m is already in the closed list, skip the node;

[0170] C232, calculate the actual cost g(m) of the node:

[0171] g(m) = g(n) + cost(n, m) + aU m

[0172] wherein cost(n, m) is the moving cost from the node n to the node m, U m is the risk score of the node m, and a is a risk weight coefficient;

[0173] C233, if the node m is not in the open list, add the node m to the open list, record its parent node as n, calculate its heuristic estimated cost h * (m), and further calculate its total cost f(m) = g(m) + h * (m);

[0174] C234, if the node m is already in the open list and the new g(m) is lower, update the g(m) of the node m, update its parent node as n, and recalculate the total cost f(m) = g(m) + h * (m);

[0175] C24, repeat steps C21-C23 to select the node with the lowest f(n) in the open list for expansion, and for each expanded node, update the g(m), h * (m) and f(m) of its neighbor nodes until the target point is found;

[0176] C3, start from the target point, trace the parent nodes of the nodes in reverse, and construct a complete path from the start point to the target point, i.e., the preliminary inspection path.

[0177] Specifically, in each iteration, the algorithm selects the node with the smallest total cost f(n) in the current open list for expansion, and updates the open list according to the expanded adjacent nodes. By introducing the risk score U m, the algorithm considers not only the path length but also the safety of the path when selecting a path. The iteration process continues until the target point is found or the open list is empty (indicating no feasible path).

[0178] After the target point is found, an empty list Path is created. The target point is traced back from the target point: set the current node as the target point T. Add the position of the current node to the path list. Set the current node as its parent node. Repeat this process until it returns to the starting point S. Reverse the path list: reverse the path list so that it is arranged in the order from the starting point to the target point. Output the final path: return the reversed path list as the final output of the algorithm.

[0179] In the present application, the environment perception module and the path planning unit work collaboratively through an efficient data interaction mechanism. Specifically, after the path planning unit generates the inspection path, the environment perception module will automatically analyze the path and identify the possible obstacle information on the path in real time. By fusing real-time data from sensors such as laser radar and depth camera, the environment perception module can dynamically update the position and nature of obstacles in the path.

[0180] The identification process of obstacle information is as follows:

[0181] D1, obtain the surround scan data of the laser radar, and eliminate noise points by median filtering, convert the polar coordinate data to point cloud data in the Cartesian coordinate system; collect the depth image stream of the depth camera, remove invalid points and outliers, and convert the depth data to three-dimensional point cloud format.

[0182] Specifically, laser radar data processing: median filtering: apply a median filter to the surround scan data of the laser radar to remove isolated abnormal points. Coordinate conversion: convert the filtered polar coordinate data to point cloud data in the Cartesian coordinate system. Depth camera data processing: invalid point removal: remove invalid points and obvious outliers in the depth image. Point cloud generation: convert the valid depth data to three-dimensional point cloud format to provide a basis for obstacle geometric feature extraction.

[0183] D2, apply DBSCAN algorithm to the laser radar point cloud data for dynamic clustering; analyze the characteristics of each cluster, and preliminarily screen the possible obstacle area; establish a preliminary obstacle candidate area list to record its position and basic feature information.

[0184] Application of DBSCAN algorithm:

[0185] Algorithm parameter setting: neighborhood radius (ε): set according to the complexity of the environment and the resolution of the laser radar. Minimum sample size (MinPts): set as the minimum sample size of clustering to ensure the effectiveness of each cluster.

[0186] Dynamic clustering process: Apply DBSCAN algorithm to the converted Cartesian point cloud data of the laser radar for dynamic clustering analysis. Each cluster represents a potential obstacle region. By analyzing the density and shape characteristics of the cluster, the possible obstacle regions are preliminarily screened out.

[0187] Record information: Position: the center coordinates of the cluster. Size: the range and volume of the cluster. Features: basic feature information such as the geometric shape and density of the cluster.

[0188] Candidate region screening: According to the preset size and shape threshold, the obstacle candidate regions that meet the conditions are screened out, and false obstacles are excluded.

[0189] D3, for the preliminarily identified obstacle candidate regions, extract the corresponding region from the depth image of the depth camera corresponding to the obstacle candidate region identified by the laser radar, obtain more detailed depth information, and extract the corresponding depth image region; calculate the depth gradient information, identify the edge region with large depth change, and extract the obstacle boundary feature; extract the contour line of the obstacle through the depth gradient information, and determine its three-dimensional geometric boundary; combine the boundary feature extracted by the depth camera with the point cloud data of the laser radar, verify the three-dimensional geometric features of the obstacle. Use a geometric matching algorithm to match the extracted features with the predefined obstacle model to confirm the validity and category of the obstacle.

[0190] D4, establish an obstacle feature database to record the position, size and feature information of the identified obstacles; through feature matching method, associate the current frame detection result with the historical record in the feature database; use Kalman filter to track the motion state of the obstacle and predict its position change; when the obstacle is stably detected in continuous multiple frames, confirm its validity and update the environment map.

[0191] Record the position, size, shape feature and other information of each identified obstacle. Through the feature matching method, associate the obstacle detected in the current frame with the historical record in the database to ensure the continuous tracking of the same obstacle in different time frames. For each associated obstacle, use Kalman filter to track its motion state, including position, velocity and acceleration. Based on the prediction function of Kalman filter, estimate the position change of the obstacle at the next moment to provide reference for dynamic obstacle avoidance. Update the verified and tracked obstacle information to the environment map. Regularly clean up the obstacle records in the database that have not been updated for a long time or move slowly to keep the environment map up to date. When an obstacle is stably detected in continuous multiple frames, confirm its validity and perform the final environment map update. For detected abnormal or fast-moving obstacles, trigger the emergency obstacle avoidance strategy to ensure the safe operation of the inspection robot.

[0192] In this invention, the environmental perception module first identifies and locates obstacle information on the inspection path, including the type, location, and dynamic state of the obstacles. Then, this detailed obstacle data is transmitted to the obstacle avoidance unit in real time. Based on the received information, the obstacle avoidance unit formulates corresponding obstacle avoidance strategies, such as adjusting the path, slowing down, or stopping. The obstacle avoidance unit makes obstacle avoidance decisions using a reinforcement learning algorithm, specifically including:

[0193] E1. Extract the relative position information of obstacles, including the distance and azimuth angle from the obstacle to the inspection robot; obtain the current motion state of the inspection robot, including position, velocity, and attitude angle; calculate local terrain features, including ground slope and surface undulation of the grain pile; integrate the above information to form a state vector, which serves as the input for obstacle avoidance decision-making; the structure of the state vector S is as follows:

[0194] S=[d1,θ1,d2,θ2,...,d n ,θ n ,v,ω,φ,ψ]

[0195] In the formula, d i and θ i Let v and w represent the distance and azimuth of the i-th obstacle, respectively; v and w represent the linear velocity and angular velocity of the inspection robot, respectively; and φ and ψ represent the heading angle and tilt angle of the inspection robot, respectively.

[0196] E2. Define the basic motion instruction set; specify the corresponding linear velocity and angular velocity parameters for each basic motion; establish safety constraints for motion execution, including maximum speed and acceleration limits.

[0197] The basic motion command set includes, but is not limited to, the following actions: Forward: Move forward at a preset linear velocity. Backward: Move backward at a preset linear velocity. Left Turn: Rotate left at a preset angular velocity. Right Turn: Rotate right at a preset angular velocity. Stop: Immediately stop all movement.

[0198] Specify the corresponding linear velocity v and angular velocity ω parameters for each basic movement. For example: forward: v = 1.0 m / s, ω = 0; backward: v = -1.0 m / s, ω = 0; left turn: v = 0, ω = 30° / s; right turn: v = 0, ω = -30° / s; stop: v = 0, ω = 0.

[0199] Establish safety constraints for action execution, including:

[0200] Maximum linear velocity: not exceeding the maximum walking speed v of the inspection robot. max .

[0201] Maximum angular velocity: not exceeding the maximum rotational speed ω of the inspection robot. max .

[0202] Acceleration limitation: control acceleration a and angular acceleration α, prevent sharp changes in the process of movement.

[0203] E3, design reward function, give reward feedback to the action of the inspection robot; the calculation formula of the reward function is as follows:

[0204] R=β1·r1-β2·r2+β3·r3

[0205] In the formula, r1 is a basic obstacle avoidance reward value:

[0206]

[0207] r1 calculates the distance between the inspection robot and the target path in the current state, and the closer the distance, the higher the reward.

[0208] r2 is a path deviation penalty term:

[0209] r2=d 偏离 ×γ

[0210] Wherein, d 偏离 is the shortest distance between the current pose of the inspection robot and the predetermined inspection path;γ is the penalty coefficient of path deviation;r2 calculates the distance between the inspection robot and the nearest obstacle, and the closer the distance, the greater the penalty.

[0211] r3 is a task completion reward value:

[0212]

[0213] β1, β2, β3 are weight coefficients, and β1> β2> β3 is set.

[0214] E4, based on the current state, use Q-learning to calculate the expected return of each possible action;Combine the ε-greedy strategy to balance between exploration and utilization;Select the optimal action to generate an obstacle avoidance strategy.

[0215] Q-learning algorithm and ε-greedy strategy are existing algorithm contents, and the present application does not make specific description, and the Q-learning algorithm and ε-greedy strategy are specifically adopted to update Q value, select the optimal action for each state, and form an obstacle avoidance strategy. That is, based on the current state S, select the action with the highest Q value as the obstacle avoidance strategy.

[0216] In an embodiment of the present application, the motion control unit comprehensively analyzes and accurately controls the movement of the inspection robot by receiving the newly generated inspection path and obstacle avoidance strategy. Specifically, the path management module first parses the inspection path data provided by the path planning unit to determine the route points that the robot should follow. At the same time, the obstacle avoidance strategy module processes the obstacle avoidance instructions transmitted by the obstacle avoidance unit, such as emergency stop, turning or deceleration, etc. The motion instruction generation module prioritizes the path and obstacle avoidance instructions to ensure that the obstacle avoidance action is executed first when a potential obstacle is detected. Subsequently, the execution control module converts the generated motion instructions into specific motor drive signals, which are sent to the robot's drive motors through the communication interface, achieving precise forward movement, turning and speed adjustment. In addition, the feedback monitoring module collects real-time information such as robot position, speed and attitude, and uses sensor feedback data for dynamic adjustment to ensure that the execution of the motion instructions meets the expected path and obstacle avoidance requirements.

[0217] In an embodiment of the present application, after the final inspection path is formed, the inspection robot inspects the grain depot according to the inspection path, and the autonomous navigation control module adjusts the obstacle avoidance strategy and the inspection path according to real-time data to ensure the stability and safety of the robot during the inspection process. During the robot inspection process, the data acquisition module is responsible for collecting multi-source data in the grain depot in real time, including environmental temperature and humidity measured by the temperature and humidity sensor, images captured by the high-definition camera, and video information recorded by the video acquisition device. The collected data is transmitted to the data processing and analysis module through the 5G communication transmission module. This module uses edge computing technology to process and analyze data in real time. Specifically, the data processing and analysis module performs image recognition and analysis on image and video data to monitor abnormal conditions in the grain depot; real-time monitoring of temperature and humidity data to identify abnormal fluctuations in environmental parameters. After processing, the module generates a monitoring report containing the current temperature and humidity status of the environment, real-time image and video recording in the grain depot, and environmental health status assessment obtained through data analysis. At the same time, the module will also generate warning information such as high temperature, abnormal humidity, suspicious activity detection, etc. according to the preset threshold and anomaly detection algorithm, to ensure that potential risks can be responded to and handled in a timely manner, and the safety and management efficiency of the grain depot are ensured.

[0218] In addition, as shown in Figure 2 The present application also provides an intelligent grain depot automatic inspection method, which is executed in the system described in any of the above embodiments. The method comprises:

[0219] S1, start the inspection robot, initialize the environment perception module, autonomous navigation control module and 5G communication transmission module, and establish communication connection with the remote computer;

[0220] S2, synchronously start the laser radar and the depth camera, collect two-dimensional and three-dimensional point cloud data of the surrounding environment at a predetermined frequency, simultaneously start the temperature and humidity sensor and the high-definition camera to collect related environmental parameters;

[0221] S3, convert the laser radar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, perform data fusion through the voxel grid method, and generate and subdivide a three-dimensional environmental map;

[0222] S4, utilize the IMU and odometry data, combine the Kalman filter algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environmental map;

[0223] S5, input the preprocessed environmental map into the path planning unit, extract environmental features based on a deep learning model, generate and smooth a preliminary inspection path using an improved A* algorithm, and adjust the path according to real-time environmental changes;

[0224] S6, apply the DBSCAN algorithm to cluster analysis of the laser radar point cloud, combine the boundary feature extraction of the depth camera and the Kalman filter to dynamically track and confirm obstacles, and update the obstacle information in the environmental map;

[0225] S7, extract the relative position of the obstacle, the motion state of the inspection robot, and the local terrain features, integrate the state vectors required for obstacle avoidance decision-making, based on the constructed state vectors, use the Q-learning in the reinforcement learning algorithm to calculate the expected return of each action, combine the epsilon-greedy strategy to select the optimal obstacle avoidance action, and generate the corresponding obstacle avoidance strategy;

[0226] S8, according to the obstacle avoidance strategy, perform segmented speed planning and attitude adjustment, and use a closed-loop control system to execute motion instructions to control the movement of the inspection robot;

[0227] S9, monitor the environmental changes and obstacle avoidance effects in real time during the inspection process, if an abnormality or significant environmental change is detected, update the environmental map and adjust the inspection path in time, and feedback the inspection state and early warning information to the remote computer through the 5G communication transmission module.

[0228] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A smart granary automatic inspection system, characterized in that, The inspection robot, the 5G communication transmission module and the remote computer are connected, and the 5G communication transmission module is used for transmitting data output by each module in the inspection robot and receiving instructions sent by the remote computer. The generation process of the environment map comprises: A2, the depth camera collects three-dimensional point cloud data of the surrounding environment at a high frequency, adjusts height information of an occupancy grid according to height data provided by the depth camera for each occupancy grid, and forms a three-dimensional occupancy grid; the three-dimensional occupancy grid and the depth camera data are locally fused by using a voxel grid method, noise points are eliminated, data gaps are filled, and an environment map extending to a three-dimensional space is generated; A4, after the grid unit subdivision is completed, the generated environment map is output; The updating process of the environment map comprises: IMU and an odometer are used to obtain dynamic data in real time; Kalman filtering algorithm is used to fuse IMU and odometer data, estimate the current pose of the inspection robot, and regularly correct the pose by using fixed feature points in the surrounding environment; Based on the current pose and the moving direction of the inspection robot, an environment map area A to be updated is defined, and the range and shape of the area A are dynamically adjusted according to the moving speed of the inspection robot and the environmental change rate; In the inspection process, the inspection robot continuously collects data of the laser radar and the depth camera, obtains new three-dimensional data, and completes preprocessing by filtering, denoising and registration of the new three-dimensional data; The preprocessed new three-dimensional data is fused into the area A of the environment map, the occupancy probability and the terrain curvature of the updated area A are analyzed, whether the grain pile form changes significantly is detected, if the grain pile form changes, the grid subdivision algorithm is called to re-subdivide or merge the grid units, and the environment map is updated. A1, the laser radar performs all-around scanning at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud data set, divides a two-dimensional space into grid units with a size of , where g is a grid size, each grid unit is identified by a center coordinate, an occupancy probability of each grid unit is calculated by using a Bayesian update formula , a first occupancy probability threshold is set, and grid units greater than the first occupancy probability threshold are marked as occupied grid units, and otherwise as idle grid units; The generation process of the inspection path comprises: A3, setting parameters of grid cell subdivision, including curvature threshold and occupancy difference threshold ; further subdividing the grid cell into smaller sub-grid cells according to the curvature of the grid cell and the curvature threshold; calculating the occupancy probability change of the grid cell , respectively represent the occupancy probability of the grid cell j in the new and old states, if , continue to subdivide the grid cell, if , merge multiple low-change sub-grid cells into a larger grid cell; B6, under the guidance of the heuristic function, the improved A* algorithm is used for path search, and a preliminary inspection path is obtained. ​ ​ ​ ​ ​ ​ 2. The system of claim 1, wherein the system is configured to: determine a location of the mobile device; and determine a location of the at least one object based on the location of the mobile device. In step 3, curvature threshold is divided into high curvature threshold and low curvature threshold , the curvature of each grid cell is calculated , and the subdivision intensity is adjusted based on the adaptive weighting coefficient : wherein, when , , the current grid size is maintained without subdivision; When , , the subdivision level is gradually increased in a linear manner from 0 to 1; when , , the maximum subdivision level is performed for high-curvature grid cells.

3. The intelligent grain warehouse automatic inspection system as described in claim 1, characterized in that, ​ B1. inputting an environment map to a path planning unit, the environment map being three-dimensional grid data, each grid cell containing position information (x, y, z), occupancy probability and curvature ; B2, set a second occupancy probability threshold , according to the second occupancy probability threshold and the occupancy probability of the grid cell, filter out the passable grid cells for marking, normalize the curvature, and complete the preprocessing of the environment map; B3. Input the preprocessed environmental map and the current status information of the inspection robot into the neural network model, extract local and global features, and output the risk score for each grid cell. This reflects the potential obstacle density and dynamic change risk of grid cells. The neural network model employs a hybrid model based on convolutional neural networks and graph neural networks, with a structure including multiple 3D convolutional layers, graph convolutional layers, and fully connected layers. The risk score is calculated using the following formula: In the formula, represents a feature set of the grid unit j, represents a trained neural network model, and Q represents state information of the inspection robot. B4. Generating an optimized heuristic function by a deep learning model The deep learning model comprises a spatial attention layer, a temporal attention layer and a position encoding module, and the heuristic function is defined as: In the formula, n is a current search node, G is an environment map, is a deep learning model; ​ B7, the initial inspection path is smoothed using a spline curve to obtain a smoothed inspection path; B8, according to the latest environment map and obstacle avoidance strategy, the obstacles or environmental changes on the path are detected, and according to the detection result, local path planning is triggered to fine-tune the inspection path in combination with the heuristic function.

4. The intelligent grain warehouse automatic inspection system as described in claim 3, characterized in that, The improved A* algorithm process is as follows: C1, set the starting point S and the target point T, initialize the open list and the closed list, the open list is used to store the nodes to be explored, and the initial contains only the starting point; the closed list is used to store the nodes that have been explored, and the initial is empty; calculate the initial cost of the starting point , calculate the heuristic estimated cost of the starting point , total cost , create the starting point node information and add it to the open list; C2, when the open list is not empty and the target point is not found, the following steps are performed: C21. Selecting from the open list the node n with the lowest cost, moving to the closed list, wherein, is the cost function, is the actual cost from the start to node n; C22, if n is the target point, the final path is constructed, and the algorithm terminates; C23, otherwise, the adjacent nodes of the node n are expanded, and for each adjacent node m, the following operations are performed: C231, if the node m is in the closed list, skip the node; C232. Real cost of compute node : wherein, is the movement cost from node n to node m, is the risk score of node m, is the risk weight coefficient; C233、if node m is not in the open list, add node m to the open list and record its parent as n, and compute its heuristic estimate cost , further compute its total cost ; C234. If node m is already in the open list, and the new cost is lower, update node m's cost, and update its parent to n, recompute the total cost ; C24, repeat steps C21-C23, select the open list in The lowest node is expanded, for each expanded node, update its neighbor nodes , and until the target point is found; C3, starting from the target point, the parent nodes of the nodes are traced back in reverse to construct a complete path from the starting point to the target point, that is, the initial inspection path.

5. The intelligent grain warehouse automatic inspection system as described in claim 1, characterized in that, The identification process of the obstacle information is as follows: D1, obtain the ring scanning data of the laser radar, remove the noise points by median filtering, and convert the polar coordinate data into point cloud data in the Cartesian coordinate system; collect the depth image stream of the depth camera, remove invalid points and outliers, and convert the depth data into a three-dimensional point cloud format; D2, apply the DBSCAN algorithm to the laser radar point cloud data for dynamic clustering; analyze the characteristics of each cluster, and preliminarily screen the possible obstacle regions; establish a preliminary obstacle candidate region list to record the position and basic feature information thereof; D3, for the preliminary identified obstacle candidate region, extract the corresponding depth image region; calculate the depth gradient information and extract the obstacle boundary features; Combine the depth information and boundary features to verify and refine the three-dimensional geometric features of the obstacle; D4, establish an obstacle feature database to record the position, size and feature information of the identified obstacle; By a feature matching method, the current frame detection result is associated with the historical record in the feature database; a Kalman filter is used to track the motion state of the obstacle to predict the position change thereof; when the obstacle is stably detected in continuous multiple frames, the validity thereof is confirmed and the environment map is updated.

6. The system of claim 1, wherein the system is configured to: determine a location of the mobile device; and determine a location of the at least one object based on the location of the mobile device. The obstacle avoidance unit makes obstacle avoidance decisions through a reinforcement learning algorithm, specifically including: E1, extract the relative position information of the obstacle, including the distance and azimuth angle of the obstacle to the inspection robot; obtain the current motion state of the inspection robot, including the position, speed and attitude angle; calculate the local terrain features, including the ground inclination and the surface undulation of the grain pile; integrate the above information to form a state vector as the input of the obstacle avoidance decision; E2, define a basic motion instruction set; specify the corresponding linear velocity and angular velocity parameters for each basic action; establish safety constraint conditions for action execution, including maximum speed and acceleration limits; E3, design a reward function to give reward feedback to the action of the inspection robot; E4, based on the current state, use Q-learning to calculate the expected return of each possible action; balance between exploration and utilization in combination with an ε-greedy strategy; select the optimal action to generate an obstacle avoidance strategy.

7. The system of claim 6, wherein the system is configured to: determine a location of the mobile device; and determine a location of the at least one object based on the location of the mobile device. The calculation formula of the reward function is as follows: In the formula, base obstacle avoidance reward value: path deviation penalty term: wherein, is the shortest distance between the current pose of the inspection robot and the predetermined inspection path; is a penalty coefficient for path deviation; reward value for task completion: , , are weight coefficients, set .

8. A method for intelligent automatic inspection of grain warehouses, characterized in that, The method is executed in the system of any one of claims 1-7, and the method comprises: S1, start the inspection robot, initialize the environment perception module, autonomous navigation control module and 5G communication transmission module, and establish communication connection with the remote computer; S2, start the laser radar and depth camera synchronously, collect two-dimensional and three-dimensional point cloud data of the surrounding environment according to the predetermined frequency, start the temperature and humidity sensor and high-definition camera to collect related environmental parameters at the same time; S3, convert the laser radar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, and perform data fusion through the voxel grid method to generate and subdivide the three-dimensional environment map; S4, use IMU and odometry data, combine Kalman filter algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environment map; S5, input the preprocessed environment map into the path planning unit, extract environmental features based on the deep learning model, generate and smooth the preliminary inspection path using the improved A* algorithm, and adjust the path according to the real-time environmental changes; S6, apply DBSCAN algorithm to cluster analysis of laser radar point cloud, combine boundary feature extraction of depth camera and Kalman filter for dynamic tracking and confirmation of obstacles, update obstacle information in the environment map; S7, extract the relative position of the obstacle, the motion state of the inspection robot and the local terrain features, integrate the state vector required for obstacle avoidance decision, based on the constructed state vector, use Q-learning in reinforcement learning algorithm to calculate the expected return of each action, combine the ε-greedy strategy to select the optimal obstacle avoidance action, and generate the corresponding obstacle avoidance strategy; S8, according to the obstacle avoidance strategy, carry out segmented speed planning and attitude adjustment, and use closed-loop control system to execute motion instructions to control the movement of the inspection robot; S9, real-time monitoring of environmental changes and obstacle avoidance effect in the inspection process, if abnormal or significant environmental changes are detected, update the environment map and adjust the inspection path in time, and feedback the inspection state and early warning information to the remote computer through the 5G communication transmission module.

Images