Intelligent grain bin automatic inspection system and method

Abstract

The invention provides an intelligent barn automatic inspection system and method, and relates to the technical field of intelligent inspection, the system comprises an inspection robot, a 5G communication transmission module and a remote computer, the inspection robot comprises an environment sensing module for generating and updating an environment map and identifying obstacle information; the autonomous navigation control module comprises a path planning unit for generating an inspection path and performing local path adjustment; the obstacle avoidance unit is used for generating an obstacle avoidance strategy; the motion control unit controls the movement of the inspection robot; the data acquisition module is used for acquiring information in the granary; the data processing and analyzing module is used for processing and analyzing the collected multi-source data and outputting a monitoring report and early warning information; wherein the 5G communication transmission module is connected with the inspection robot and the remote computer, and is used for transmitting data output by each module in the inspection robot and receiving an instruction sent by the remote computer. According to the invention, real-time sensing, intelligent obstacle avoidance and autonomous inspection of the granary environment are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection, and in particular to an intelligent control granary automatic inspection system and method. Background Art

[0002] Granary inspection plays a crucial role in grain storage and management. Effective inspection can not only timely detect and handle problems such as abnormal temperature and humidity, pests, and mildew during grain storage, ensure grain quality and safety, but also prevent potential safety hazards, and guarantee the overall operation efficiency and economic benefits of the granary. With the increase in grain storage and the complexity of storage conditions, automated and intelligent inspection means are particularly important to improve the inspection coverage and accuracy, reduce labor costs, and enhance management levels.

[0003] Currently, granary inspection mainly relies on technical means such as manual inspection, drone inspection, and traditional inspection robots. Although manual inspection is intuitive, it has problems such as low efficiency, limited coverage, and low detection accuracy. Drone inspection has improved the inspection coverage and efficiency to a certain extent and is suitable for environmental monitoring of large-scale granaries. However, there are still certain technical challenges in the operation of drones in complex terrains. Traditional inspection robots achieve automatic inspection by carrying various sensors, but they still need to be further improved in terms of autonomous navigation, obstacle recognition, and environmental adaptability to meet the diverse and complex inspection needs of large granaries.

[0004] The existing technologies still face many deficiencies in the process of granary inspection. First, the environmental adaptability of many inspection devices is limited, and it is difficult to maintain stable operation in the complex and changeable granary environment. Second, the degree of automation of data collection and processing is not high, resulting in limited real-time and accuracy of inspection results. In addition, the existing inspection systems have insufficient capabilities in energy management and autonomous charging, and cannot achieve long-term and continuous inspection tasks. These defects not only affect the inspection efficiency and effect, but also increase the maintenance and operation costs. Summary of the Invention

[0005] In view of this, the present invention proposes an intelligent control granary automatic inspection system and method. By integrating functional modules such as environmental perception, autonomous navigation, and multi-source data collection, and combining deep learning and reinforcement learning algorithms, it realizes the real-time perception of the granary environment, intelligent obstacle avoidance, and autonomous inspection. At the same time, edge computing technology is used to process and analyze the collected data, and finally achieves the purposes of improving inspection efficiency, ensuring operation safety, and reducing labor costs.

[0006] The technical solution of the present invention is realized as follows:

[0007] On the one hand, the present invention provides an intelligent control granary automatic inspection system, including an inspection robot, a 5G communication transmission module, and a remote computer. The inspection robot includes:

[0008] An environmental perception module, equipped with a lidar and a depth camera, for real-time acquisition of three-dimensional data of the granary environment, generating and updating an environmental map, and identifying obstacle information on the inspection path;

[0009] An autonomous navigation control module, connected to the environmental perception module, including:

[0010] A path planning unit, which performs real-time analysis on the environmental map based on a deep learning algorithm, generates an inspection path, and receives an obstacle avoidance strategy from the obstacle avoidance unit for local path adjustment;

[0011] An obstacle avoidance unit, which uses a reinforcement learning algorithm to generate an obstacle avoidance strategy based on the obstacle information identified by the environmental perception module, and feeds back the obstacle avoidance strategy to the path planning unit to trigger local path planning;

[0012] A motion control unit, which receives and synthesizes the inspection path and the obstacle avoidance strategy to control the movement of the inspection robot;

[0013] A data acquisition module, equipped with a temperature and humidity sensor, a high-definition camera, and a video acquisition device, for collecting temperature, humidity, image, and video information in the granary;

[0014] A data processing and analysis module, which uses edge computing technology and is connected to the data acquisition module, for processing and analyzing the collected multi-source data and outputting a monitoring report and warning information;

[0015] Among them, the 5G communication transmission module connects the inspection robot and the remote computer, for transmitting the data output by each module in the inspection robot, and receiving instructions sent by the remote computer.

[0016] Preferably, the process of generating the environmental map includes:

[0017] A1. The lidar performs an omnidirectional scan at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud data set, divides the two-dimensional space into grid cells of size g×g, where g is the grid size, and each grid cell is identified by its central coordinates. The occupancy probability P(O j ) of each grid cell is calculated using the Bayesian update formula. A first occupancy probability threshold θ1 is set, and the grid cells with P(O j ) greater than the first occupancy probability threshold θ1 are marked as occupied grid cells, and the rest are free grid cells;

[0018] A2. The depth camera collects 3D point cloud data of the surrounding environment at a high frequency. For each occupied grid, according to the height data provided by the depth camera, the height information of the occupied grid is adjusted to form a 3D occupied grid. The voxel grid method is used to locally fuse the 3D occupied grid and the depth camera data, eliminate noise points and fill data gaps, and generate an environmental map extended to the 3D space.

[0019] A3. Set the parameters for grid cell subdivision, including the curvature threshold κ th and the occupancy difference threshold ΔP th ; according to the curvature of the grid cell and the curvature threshold, the grid cell is further subdivided into smaller sub-grid cells; calculate the change in the occupancy probability of the grid cell respectively represent the occupancy probabilities of grid cell j in the old and new states. If ΔP > ΔP th , then continue to subdivide this grid cell. If ΔP < -ΔP th , then merge multiple sub-grid cells with low changes into a larger grid cell.

[0020] A4. After the grid cell subdivision is completed, the generated environmental map is output.

[0021] Preferably, in step 3, the curvature threshold κ th is divided into a high curvature threshold κ high and a low curvature threshold κ low , calculate the curvature κ of each grid cell, and adjust the subdivision intensity based on the adaptive weighting coefficient ω(κ):

[0022]

[0023] where, when κ ≤ κ low , ω(κ) = 1, keep the current grid size and do not perform subdivision;

[0024] When κ low < κ ≤ κ high , ω(κ) increases linearly from 0 to 1, gradually increasing the subdivision intensity;

[0025] When κ > κ high , ω(κ) = 1, and the high-curvature grid cells are subdivided with the maximum intensity.

[0026] Preferably, the update process of the environmental map is as follows:

[0027] Use the IMU and odometer equipped on the inspection robot to obtain dynamic data in real time;

[0028] Adopt the Kalman filter algorithm to fuse the IMU and odometer data, estimate the current pose of the inspection robot, and regularly correct the pose using fixed feature points in the surrounding environment.

[0029] Based on the current pose and moving direction of the inspection robot, define the area A of the environmental map that needs to be updated, and dynamically adjust the range and shape of area A according to the moving speed of the inspection robot and the environmental change rate;

[0030] During the inspection process, the inspection robot continuously collects data from the lidar and depth camera, obtains new three-dimensional data, and performs filtering, denoising, and registration on the new three-dimensional data to complete preprocessing;

[0031] Fuse the preprocessed new three-dimensional data into area A of the environmental map, analyze the occupancy probability and terrain curvature of the updated area A, detect whether there is a significant change in the grain pile shape. If a change in the grain pile shape is detected, call the grid subdivision algorithm to re-subdivide or merge the grid cells to complete the update of the environmental map.

[0032] Preferably, the generation process of the inspection path is as follows:

[0033] B1. Input the environmental map into the path planning unit. The environmental map is three-dimensional grid data, and each grid cell contains position information (x, y, z), occupancy probability P(O j ) and curvature κ;

[0034] B2. Set the second occupancy probability threshold θ2. According to the second occupancy probability threshold θ2 and the occupancy probability of the grid cells, screen out the passable grid cells for marking, and perform normalization processing on the curvature to complete the preprocessing of the environmental map;

[0035] B3. Input the preprocessed environmental map and the current state information of the inspection robot into the neural network model to extract local features and global features. The model outputs the risk score U j of each grid cell, which reflects the density of potential obstacles and the risk of dynamic changes in the grid cell. Among them, the neural network model adopts a hybrid model based on convolutional neural network and graph neural network, and its structure includes multiple 3D convolutional layers, graph convolutional layers, and fully connected layers; the calculation formula of the risk score is:

[0036] U j = f CNN-GNN (I j , Q)

[0037] In the formula, I j represents the feature set of grid cell j, f CNN-GNN represents the trained neural network model, and Q represents the state information of the inspection robot;

[0038] B4. Generate an optimized heuristic function h through the deep learning model *(n), the deep learning model includes a spatial attention layer, a temporal attention layer, and a position encoding module, and the heuristic function h * (n) is defined as:

[0039] h * (n) = f Heuristic (n, G, Q)

[0040] where n is the current search node, G is the environmental map, and f Heuristic is the deep learning model;

[0041] B6. Under the guidance of the heuristic function, perform path search through the improved A* algorithm to obtain a preliminary inspection path;

[0042] B7. Use a spline curve to smooth the preliminary inspection path to obtain a smoothed inspection path;

[0043] B8. According to the latest environmental map and obstacle avoidance strategy, detect obstacles or environmental changes on the path. According to the detection results, trigger local path planning and, in combination with the heuristic function, fine-tune the inspection path.

[0044] Preferably, the process of the improved A* algorithm is as follows:

[0045] C1. Set the starting point S and the target point T, initialize the open list and the closed list. The open list is used to store nodes to be explored, initially containing only the starting point; the closed list is used to store nodes that have been explored, initially empty; calculate the initial cost of the starting point g(S) = 0, calculate the heuristic estimated cost of the starting point h * (S) = f Heuristic (S, G, Q), and the total cost f(S) = g(S) + h * (S), create the starting point node information and add it to the open list;

[0046] C2. When the open list is not empty and the target point has not been found, perform the following steps:

[0047] C21. Select the node n with the lowest f(n) = g(n) + h * (n) from the open list, move it to the closed list, where f(n) is the cost function and g(n) is the actual cost from the starting point to node n;

[0048] C22. If n is the target point, construct the final path and terminate the algorithm;

[0049] C23. Otherwise, expand the adjacent nodes of node n, and for each adjacent node m, perform the following operations:

[0050] C231. If node m is already in the closed list, skip this node;

[0051] C232. Calculate the actual cost g(m) of the computing node:

[0052] g(m) = g(n) + cost(n,m) + αU m

[0053] Where cost(n,m) is the movement cost from node n to node m, U m is the risk score of node m, and α is the risk weight coefficient;

[0054] C233. If node m is not in the open list, add node m to the open list, record its parent node as n, and calculate its heuristic estimated cost h * (m), and further calculate its total cost f(m) = g(m) + h * (m);

[0055] C234. If node m is already in the open list and the new g(m) is lower, update g(m) of node m, update its parent node as n, and recalculate the total cost f(m) = g(m) + h * (m);

[0056] C24. Repeat steps C21 - C23, select the node with the lowest f(n) in the open list for expansion, and for each expanded node, update the g(m), h * (m) and f(m) of its neighbor nodes until the target point is found;

[0057] C3. Starting from the target point, trace back the parent nodes of the nodes in reverse to construct a complete path from the starting point to the target point, that is, the preliminary inspection path.

[0058] Preferably, the identification process of the obstacle information is as follows:

[0059] D1. Obtain the surrounding scan data of the lidar, eliminate noise through median filtering, and convert the polar coordinate data into point cloud data in the Cartesian coordinate system; collect the depth image stream of the depth camera, remove invalid points and outliers, and convert the depth data into a three - dimensional point cloud format;

[0060] D2. Apply the DBSCAN algorithm to the lidar point cloud data for dynamic clustering; analyze the characteristics of each clustering cluster, preliminarily screen the possible obstacle areas; establish a preliminary obstacle candidate area list, and record its position and basic feature information;

[0061] D3. For the preliminarily identified obstacle candidate areas, extract the corresponding depth image areas; calculate the depth gradient information and extract the obstacle boundary features; combine the depth information and boundary features to verify and refine the three - dimensional geometric features of the obstacles;

[0062] D4. Establish an obstacle feature database to record the positions, sizes, and feature information of the identified obstacles; associate the current frame detection results with the historical records in the feature database through feature matching methods; use a Kalman filter to track the motion states of the obstacles and predict their position changes; when an obstacle is stably detected in multiple consecutive frames, confirm its validity and update the environmental map.

[0063] Preferably, the obstacle avoidance unit makes obstacle avoidance decisions through a reinforcement learning algorithm, specifically including:

[0064] E1. Extract the relative position information of the obstacles, including the distance and azimuth angle from the obstacles to the inspection robot; obtain the current motion states of the inspection robot, including position, speed, and attitude angle; calculate the local terrain features, including the ground inclination and the undulation of the grain pile surface; integrate the above information to form a state vector as the input for obstacle avoidance decisions.

[0065] E2. Define a basic motion instruction set; specify the corresponding linear velocity and angular velocity parameters for each basic action; establish the safety constraint conditions for action execution, including maximum speed and acceleration limits.

[0066] E3. Design a reward function to give reward feedback to the actions of the inspection robot.

[0067] E4. Based on the current state, use Q-learning to calculate the expected rewards of each possible action; balance between exploration and exploitation by combining the ε-greedy strategy; select the optimal action to generate an obstacle avoidance strategy.

[0068] Preferably, the calculation formula of the reward function is as follows:

[0069] R = β1·r1 - β2·r2 + β3·r3

[0070] In the formula, r1 is the basic obstacle avoidance reward value:

[0071]

[0072] r2 is the path deviation penalty term:

[0073] r2 = d 偏离 ×γ

[0074] Where d 偏离 is the shortest distance between the current pose of the inspection robot and the predetermined inspection path; γ is the penalty coefficient for path deviation;

[0075] r3 is the task completion reward value:

[0076]

[0077] β1, β2, and β3 are weight coefficients, and it is set that β1 > β2 > β3.

[0078] On the other hand, the present invention also provides an intelligent control granary automatic inspection method, which is executed in the system described in any one of the above, and the method includes:

[0079] S1. Start the inspection robot, initialize the environment perception module, autonomous navigation control module, and 5G communication transmission module, and establish a communication connection with the remote computer;

[0080] S2. Synchronously start the lidar and depth camera, collect two-dimensional and three-dimensional point cloud data of the surrounding environment at a predetermined frequency, and at the same time start the temperature and humidity sensor and high-definition camera to collect relevant environmental parameters;

[0081] S3. Convert the lidar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, and perform data fusion through the voxel grid method to generate and subdivide the three-dimensional environment map;

[0082] S4. Use the IMU and odometer data, combine the Kalman filter algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environment map;

[0083] S5. Input the preprocessed environment map into the path planning unit, extract environmental features based on the deep learning model, generate and smooth the preliminary inspection path using the improved A* algorithm, and adjust the path according to the real-time environmental changes;

[0084] S6. Apply the DBSCAN algorithm to perform clustering analysis on the lidar point cloud, combine the boundary feature extraction of the depth camera and the Kalman filter to perform dynamic tracking and confirmation of obstacles, and update the obstacle information in the environment map;

[0085] S7. Extract the relative position of the obstacle, the motion state of the inspection robot, and the local terrain features, integrate them to form a state vector required for obstacle avoidance decision-making, based on the constructed state vector, use the Q-learning in the reinforcement learning algorithm to calculate the expected rewards of each action, and combine the ε-greedy strategy to select the optimal obstacle avoidance action to generate the corresponding obstacle avoidance strategy;

[0086] S8. According to the obstacle avoidance strategy, perform segmented speed planning and attitude adjustment, and use a closed-loop control system to execute the motion instruction to control the movement of the inspection robot;

[0087] S9. Real-time monitor the environmental changes and obstacle avoidance effects during the inspection process. If anomalies or significant environmental changes are detected, update the environment map and adjust the inspection path in a timely manner, and at the same time feedback the inspection status and warning information to the remote computer through the 5G communication transmission module.

[0088] The present invention has the following beneficial effects compared with the prior art:

[0089] (1) The intelligent control grain depot automatic inspection system provided by the present invention integrates an environment perception module, an autonomous navigation control module, and a data processing and analysis module, and combines 5G communication transmission technology to achieve real-time perception of the grain depot environment, intelligent obstacle avoidance, and autonomous inspection, improving the automation level and efficiency of the inspection operation, and at the same time ensuring the safety and reliability of the inspection operation;

[0090] (2) The present invention adopts an environment perception scheme that fuses lidar and depth cameras. By using an adaptive grid subdivision algorithm to dynamically update the environmental map and combining a subdivision strength adjustment mechanism based on curvature, it can accurately depict the geometric features of the complex terrain inside the grain depot, improving the accuracy and real-time performance of environmental modeling;

[0091] (3) The path planning scheme designed by the present invention generates an optimized heuristic function through a deep learning model and combines an improved A* algorithm for path search. It not only considers the path length but also incorporates the environmental risk score into the cost function, making the generated inspection path meet the shortest path requirement and effectively avoid high-risk areas, enhancing the intelligence and practicality of path planning;

[0092] (4) The obstacle recognition method based on DBSCAN clustering and multi-sensor fusion proposed by the present invention can accurately identify and predict the position and motion state of obstacles through comprehensive analysis of depth information and boundary features and combining a Kalman filter for dynamic tracking;

[0093] (5) The present invention adopts an obstacle avoidance decision-making mechanism based on Q-learning. By designing a multi-dimensional reward function and comprehensively considering the obstacle avoidance effect, path deviation degree, and task completion situation, the obstacle avoidance strategy can maintain a high inspection efficiency while ensuring safety, effectively solving the obstacle avoidance problem in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0095] Figure 1 is the system framework diagram of the present invention

[0096] Figure 2 is the method flow diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0097] Next, in combination with the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0098] As Figure 1 shown, the present invention provides an intelligent control grain depot automatic inspection system, which includes an inspection robot, a 5G communication transmission module, and a remote computer. The inspection robot includes:

[0099] An environment perception module, which is provided with a lidar and a depth camera, is used to obtain three-dimensional data of the grain depot environment in real time, generate and update an environment map, and identify obstacle information on the inspection path;

[0100] An autonomous navigation control module, which is connected to the environment perception module, includes:

[0101] A path planning unit, which performs real-time analysis on the environment map based on a deep learning algorithm, generates an inspection path, and receives an obstacle avoidance strategy from the obstacle avoidance unit to perform local path adjustment;

[0102] An obstacle avoidance unit, which uses a reinforcement learning algorithm to generate an obstacle avoidance strategy based on the obstacle information identified by the environment perception module, and feeds back the obstacle avoidance strategy to the path planning unit to trigger local path planning;

[0103] A motion control unit, which receives and synthesizes the inspection path and the obstacle avoidance strategy to control the movement of the inspection robot;

[0104] A data acquisition module, which is provided with a temperature and humidity sensor, a high-definition camera, and a video acquisition device, is used to collect temperature, humidity, image, and video information in the grain depot;

[0105] A data processing and analysis module, which uses edge computing technology and is connected to the data acquisition module, is used to process and analyze the multi-source data collected, and output a monitoring report and early warning information;

[0106] Among them, the 5G communication transmission module connects the inspection robot and the remote computer, and is used to transmit the data output by each module in the inspection robot, and receive the instructions sent by the remote computer.

[0107] Specifically, the inspection robot is the core execution unit of the system, responsible for autonomous inspection and data collection in the granary. The main components include: Frame chassis: Provide mechanical structure support to ensure the stable operation of the robot on complex terrain. Battery pack: Provide long-term energy support for the robot, with high energy density and fast charging capabilities. Servo motor: Drive the movement of the robot and the movement of the robotic arm to achieve autonomous navigation and object manipulation. LiDAR: Used for environmental perception and map construction, real-time scanning of surrounding obstacles, and support obstacle avoidance and path planning. Jetson Nano: Embedded computing platform, responsible for processing sensor data, running AI algorithms, and achieving real-time decision-making. TN7-inch display: Provides a human-computer interaction interface to display robot status information and inspection data. Temperature and humidity sensor: Collect temperature and humidity data in the granary to monitor environmental conditions. Depth camera: Obtain three-dimensional image data to detect grain quality, such as mold and pests. Infrared sensor: Measure ambient temperature and assist in temperature and humidity monitoring. SF6 harmful gas detection module: Monitor the concentration of harmful gases in the environment to ensure the safety of the granary. Mechanical claw: used to grab grain from the grain pile to assist in quality inspection and processing. Obstacle avoidance system: combined with lidar and depth camera data, it makes real-time path adjustments to avoid collisions.

[0108] The 5G communication module enables high-speed, low-latency data transmission, ensuring real-time communication between the robot and the remote computer. It supports high-bandwidth and high-reliability wireless connections, ensuring timely upload of inspection data and rapid issuance of instructions.

[0109] As the control and data processing center of the system, the remote computer undertakes the following functions: Data processing and analysis: Store, process and analyze the temperature, humidity, image, gas concentration and other data collected by the robot to generate inspection reports. Map construction and path planning: Based on the data of the laser radar and depth camera, the granary environment map is updated in real time to optimize the inspection path. Remote control and monitoring: Through the host computer terminal, managers can issue inspection instructions and monitor the status and work progress of the inspection robot in real time. System management: Maintain the system's software and hardware to ensure the coordinated operation of each component.

[0110] The local area network deployed in the granary is used to connect inspection robots, remote computers and other network devices to ensure efficient data transmission and stable operation of the system.

[0111] The inspection system of the present invention includes the following functions:

[0112] Environmental perception and map construction: Through LiDAR and depth cameras, the inspection robot can generate high-precision three-dimensional environmental maps. LiDAR is responsible for scanning surrounding obstacles, and the depth camera provides detailed terrain and object information. The system uses voxel grid method to fuse data and eliminate noise to ensure the accuracy of the map and the ability to update in real time.

[0113] Autonomous Navigation and Path Planning: The inspection robot uses deep learning algorithms and reinforcement learning algorithms on Jetson Nano to achieve autonomous navigation and path planning. Combined with the obstacle avoidance system, the robot can intelligently adjust its path in the complex and changeable granary environment to ensure the efficient completion of the inspection task.

[0114] Data Acquisition and Transmission: The robot is equipped with a variety of sensors to collect data such as temperature and humidity, environmental images, and gas concentration in real time. The collected data is quickly transmitted to a remote computer through a 5G communication module to support real-time monitoring and data analysis.

[0115] Autonomous Obstacle Avoidance and Motion Control: The obstacle avoidance system combines data from lidar and depth cameras to identify and track obstacles in real time, and uses the Q-learning algorithm to make obstacle avoidance decisions. The robot achieves precise motion control through servo motors and a closed-loop control system to ensure stable operation on complex terrains.

[0116] Item Manipulation and Quality Inspection: The robotic arm module enables the robot to pick up and process grains on the grain pile and assist in quality inspection. The collected data is analyzed by the host computer to help managers detect and handle grain quality problems in a timely manner.

[0117] In one embodiment of the present invention, the process of generating the environmental map includes:

[0118] A1. The lidar performs an omnidirectional scan at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud dataset, divides the two-dimensional space into grid cells of size g×g, where g is the grid size, and each grid cell is identified by its central coordinates. The occupancy probability P(O j ) of each grid cell is calculated using the Bayesian update formula. A first occupancy probability threshold θ1 is set, and the grid cells with P(O j ) greater than the first occupancy probability threshold θ1 are marked as occupied grid cells, and the rest are free grid cells;

[0119] Specifically, the lidar performs a 360° omnidirectional scan at a fixed scan frequency of 30Hz; 1080 distance data points are obtained for each scan to form a two-dimensional point cloud dataset; the scan angular resolution is 0.33°, and the measurement range is from 0.1m to 30m. The two-dimensional space is divided into grid cells of size g×g, where g = 0.1m, and each grid cell is uniquely identified by its central coordinates (x c , y c ). A coordinate system transformation matrix is established to convert the data in the lidar coordinate system to the global coordinate system.

[0120] The calculation method of the occupancy probability P(O j ) is as follows:

[0121]

[0122] Among them, P(Z|O j ) is the probability that grid j is occupied under the given measurement Z; P(O j ) is the prior occupancy probability; P(Z) is the total probability of the measurement data; the first occupancy probability threshold θ1 = 0.65 is set.

[0123] A2. The depth camera collects three-dimensional point cloud data of the surrounding environment at a high frequency. For each occupied grid, according to the height data provided by the depth camera, the height information of the occupied grid is adjusted to form a three-dimensional occupied grid; the voxel grid method is used to locally fuse the three-dimensional occupied grid and the depth camera data to eliminate noise points and fill data gaps, generating an environmental map extended to three-dimensional space;

[0124] Specifically, the depth camera collects three-dimensional point cloud data of the environment at a frequency of 60 Hz; the field of view angle: 87° horizontally and 58° vertically; the depth measurement range: 0.5 m to 6 m. The voxel grid method is used to downsample and fuse the data; the voxel size is set to 0.05 m × 0.05 m × 0.05 m; the statistical characteristics of the point cloud data within each voxel are calculated; the outlier points and noise data are removed; the nearest neighbor interpolation method is used to fill the data gaps.

[0125] A3. Set the parameters for grid cell subdivision, including the curvature threshold κ th and the occupancy difference threshold ΔP th ; according to the curvature of the grid cell and the curvature threshold, the grid cell is further subdivided into smaller sub-grid cells; calculate the change in the occupancy probability of the grid cell respectively represent the occupancy probabilities of grid cell j in the old and new states. If ΔP > ΔP th , then continue to subdivide the grid cell. If ΔP < -ΔP th , then merge multiple sub-grid cells with low changes into a larger grid cell;

[0126] Among them, the curvature threshold κ th is divided into a high curvature threshold κ high and a low curvature threshold κ low . Calculate the curvature κ of each grid cell and adjust the subdivision intensity based on the adaptive weighting coefficient ω(κ):

[0127]

[0128] Among them, when κ ≤ κ low , ω(κ) = 1, keep the current grid size and do not perform subdivision;

[0129] When κlow <κ ≤ κ high When, ω(κ) increases linearly from 0 to 1, gradually increasing the subdivision strength;

[0130] When κ > κ high When, ω(κ) = 1, and the grid cells with high curvature are subdivided with the maximum strength.

[0131] The calculation formula of κ is as follows:

[0132]

[0133] Among them, h j is the height of the current grid, h j-1 , h j+1 are the heights of adjacent grids, and Δx is the grid spacing.

[0134] Calculate the weighted coefficient ω(κ) according to the curvature κ, determine the subdivision level according to the weighted coefficient, for the grids that need to be subdivided, divide them into 4 sub-grids, assign new coordinate identifiers to the sub-grids, inherit the occupancy probability of the parent grid, and update according to the local data.

[0135] Calculate the change in occupancy probability ΔP of adjacent grid cells. If ΔP > ΔP th , then continue to subdivide this grid cell. If ΔP < -ΔP th , check the adjacent sub-grid cells. If the sub-grid cells have similar characteristics, merge them into larger grid cells.

[0136] A4. After the subdivision of grid cells is completed, the generated environmental map is output.

[0137] Establish a grid index table to record the attribute information of each grid: position coordinates (x, y, z), occupancy probability P(O j ), curvature value κ, subdivision level.

[0138] Generate an environmental map data file, including grid attribute information and topological relationships; provide a map update timestamp; output map visualization data.

[0139] In an embodiment of the present invention, the update process of the environmental map is as follows:

[0140] Use the IMU and odometer equipped on the inspection robot to obtain dynamic data in real time; the IMU is used to obtain the acceleration and angular velocity data of the inspection robot in real time. The odometer measures the driving distance and direction change of the inspection robot.

[0141] The Kalman filtering algorithm is adopted to fuse IMU and odometer data, estimate the current pose of the inspection robot, and regularly correct the pose using fixed feature points in the surrounding environment; the steps of the Kalman filter include: Prediction step: Predict the current position and attitude based on IMU data. Update step: Combine odometer data to correct the predicted value and improve the estimation accuracy. Pose correction process: Feature point extraction: Extract stable environmental feature points from lidar and depth camera data. Matching and correction: Match the feature points detected in real time with the feature points in the environmental map and adjust the pose estimation.

[0142] Based on the current pose and moving direction of the inspection robot, define the area A of the environmental map that needs to be updated, and dynamically adjust the range and shape of area A according to the moving speed of the inspection robot and the environmental change rate; the definition of area A considers the following factors: The moving speed of the robot: A higher moving speed may require a larger or more frequent update area. The environmental change rate: If the environment changes rapidly, area A should be larger to cover potential change areas. Range adjustment: Increase or decrease the radius of area A to adapt to different moving speeds. Shape adjustment: Adjust the shape of area A according to the moving direction and environmental complexity, such as adjusting from a circle to an ellipse to improve efficiency.

[0143] During the inspection process, the inspection robot continuously collects lidar and depth camera data, obtains new three-dimensional data, and performs filtering, denoising, and registration on the new three-dimensional data to complete preprocessing.

[0144] Fuse the preprocessed new three-dimensional data into area A of the environmental map, analyze the occupancy probability and terrain curvature of the updated area A, detect whether there is a significant change in the grain pile shape. If a change in the grain pile shape is detected, call the grid subdivision algorithm to re-subdivide or merge the grid cells to complete the update of the environmental map.

[0145] Specifically, the detection of changes in the grain pile shape is achieved by comparing the terrain curvature and occupancy probability before and after the update, identifying significant differences, and setting thresholds for changes in curvature and occupancy probability to determine whether to trigger further processing.

[0146] If a significant change in the grain pile shape is detected, call the grid subdivision algorithm to re-subdivide or merge the grid cells.

[0147] Store the updated map data in the system database for subsequent path planning and obstacle avoidance decision-making. Through the 5G communication module, transmit the updated map information to the remote computer in real time for monitoring and management personnel to view.

[0148] In an embodiment of the present invention, the process of generating the inspection path is as follows:

[0149] B1. Input the environmental map into the path planning unit. The environmental map is three-dimensional grid data, and each grid cell contains position information (x, y, z), occupancy probability P(O j ) and curvature κ;

[0150] B2. Set the second occupancy probability threshold θ2. According to the second occupancy probability threshold θ2 and the occupancy probability of the grid cells, filter out the passable grid cells for marking, and perform normalization processing on the curvature to complete the preprocessing of the environmental map;

[0151] B3. Input the preprocessed environmental map and the status information of the current inspection robot into the neural network model to extract local features and global features. The model outputs the risk score U of each grid cell j , which reflects the potential obstacle density and dynamic change risk of the grid cell. Among them, the neural network model adopts a hybrid model based on convolutional neural network and graph neural network, and its structure includes multiple 3D convolutional layers, graph convolutional layers and fully connected layers; the 3D convolutional layer processes spatial information and extracts spatial features in the environment. The graph convolutional layer models the relationship between grid cells and captures global environmental features. The multi-stage residual connection enhances the expressive ability of the model and captures feature information at different scales. The calculation formula of the risk score is:

[0152] U j = f CNN-GNN (I j , Q)

[0153] In the formula, I j represents the feature set of grid cell j, f CNN-GNN represents the trained neural network model, and Q represents the status information of the inspection robot;

[0154] Among them, the neural network model includes a training process. The training data is a large amount of inspection path data, including path selection and obstacle avoidance records in different environments. The data annotation includes paths with successful obstacle avoidance and collision paths. The training process adopts a supervised learning method and is trained by minimizing the loss function between the predicted path score and the actual path effect. The cross-validation and early stopping mechanisms are used to prevent overfitting and ensure the generalization ability of the model.

[0155] B4. Generate an optimized heuristic function h * (n) through the deep learning model. The deep learning model includes a spatial attention layer, a temporal attention layer and a position encoding module. The spatial attention layer focuses on the edge of the grain pile and high-risk areas, improving the model's perception ability of key areas. The temporal attention layer considers the changes of dynamic obstacles and enhances the model's adaptability to environmental dynamic changes. The position encoding encodes the three-dimensional position information of the nodes into high-dimensional features to help the model understand spatial relationships and relative distances. The heuristic function h *(n) is defined as:

[0156] h * (n) = f Heuristic (n, G, Q)

[0157] Where n is the current search node, G is the environmental map, and f Heuristic is a deep learning model;

[0158] The deep learning model includes a training process that collects diverse 3D environmental maps and corresponding optimal path data as the training set. The optimal path can be generated by expert annotation or using the traditional A* algorithm. The training process is as follows: Input: the position information of node n, the environmental map G, and the robot state information Q. Output: the heuristic estimate value h * (n). Loss function: The mean squared error (MSE) is adopted to minimize the difference between h * (n) and the true path cost. Optimization algorithm: The Adam optimizer is used to train the model and adjust the parameters to improve the prediction accuracy.

[0159] B6. Under the guidance of the heuristic function, perform path search through the improved A* algorithm to obtain a preliminary inspection path;

[0160] B7. Use a spline curve to smooth the preliminary inspection path to obtain a smoothed inspection path; The initially generated inspection path may have sharp turns and unevenness. To improve the operation efficiency and stability of the inspection robot, a spline curve is introduced for path smoothing.

[0161] Specifically, a cubic spline curve is used for smoothing.

[0162] B8. According to the latest environmental map and obstacle avoidance strategy, detect obstacles or environmental changes on the path. According to the detection results, trigger local path planning and, in combination with the heuristic function, fine-tune the inspection path.

[0163] The process of the improved A* algorithm is as follows:

[0164] C1. Set the starting point S and the target point T, initialize the open list and the closed list. The open list is used to store the nodes to be explored, initially containing only the starting point; the closed list is used to store the explored nodes, initially empty; calculate the initial cost g(S) = 0 of the starting point, calculate the heuristic estimated cost h * (S) = f Heuristic (S, G, Q), and the total cost f(S) = g(S) + h * (S), create the starting point node information and add it to the open list;

[0165] C2. When the open list is not empty and the target point has not been found, perform the following steps:

[0166] C21. Select the node n with the lowest f(n) = g(n) + h * (n) from the open list, and move it to the closed list, where f(n) is the cost function and g(n) is the actual cost from the starting point to node n;

[0167] C22. If n is the target point, construct the final path and terminate the algorithm;

[0168] C23. Otherwise, expand the adjacent nodes of node n, and for each adjacent node m, perform the following operations:

[0169] C231. If node m is already in the closed list, skip this node;

[0170] C232. Calculate the actual cost g(m) of the node:

[0171] g(m) = g(n) + cost(n, m) + αU m

[0172] where cost(n, m) is the movement cost from node n to node m, and U m is the risk score of node m, and α is the risk weight coefficient;

[0173] C233. If node m is not in the open list, add node m to the open list, record its parent node as n, calculate its heuristic estimated cost h * (m), and further calculate its total cost f(m) = g(m) + h * (m);

[0174] C234. If node m is already in the open list and the new g(m) is lower, update g(m) of node m, update its parent node as n, and recalculate the total cost f(m) = g(m) + h * (m);

[0175] C24. Repeat steps C21 - C23, select the node with the lowest f(n) in the open list for expansion, and for each expanded node, update g(m), h * (m), and f(m) of its neighbor nodes until the target point is found;

[0176] C3. Starting from the target point, trace back the parent nodes of the nodes in reverse to construct a complete path from the starting point to the target point, which is the preliminary inspection path.

[0177] Specifically, in each iteration, the algorithm selects the node with the minimum total cost f(n) in the current open list for expansion and updates the open list according to the expanded adjacent nodes. By introducing the risk score U m, when the algorithm selects a path, it not only considers the path length but also the path safety. The iterative process continues until the target point is found or the open list is empty (indicating no feasible path).

[0178] After finding the target point, create an empty list Path. Trace back from the target point: Set the current node as the target point T. Add the position of the current node to the path list. Set the current node as its parent node. Repeat this process until reaching the starting point S. Reverse the path list: Reverse the path list so that it is arranged in the order from the starting point to the target point. Output the final path: Return the reversed path list as the final output of the algorithm.

[0179] In the present invention, the environmental perception module and the path planning unit achieve collaborative work through an efficient data interaction mechanism. Specifically, after the path planning unit generates the inspection path, the environmental perception module will automatically parse the path and real-time identify the obstacle information that may exist on the path. By fusing the real-time data from sensors such as lidar and depth cameras, the environmental perception module can dynamically update the positions and properties of the obstacles in the path.

[0180] The process of identifying obstacle information is as follows:

[0181] D1. Obtain the circumferential scan data of the lidar, eliminate the noise through median filtering, and convert the polar coordinate data into point cloud data in the Cartesian coordinate system; collect the depth image stream of the depth camera, remove the invalid points and outliers, and convert the depth data into a three-dimensional point cloud format.

[0182] Specifically, lidar data processing: Median filtering: Apply a median filter to the circumferential scan data of the lidar to eliminate noise and remove isolated abnormal points. Coordinate conversion: Convert the filtered polar coordinate data into point cloud data in the Cartesian coordinate system. Depth camera data processing: Invalid point removal: Eliminate the invalid points and obvious outliers in the depth image. Point cloud generation: Convert the valid depth data into a three-dimensional point cloud format to provide a basis for extracting the geometric features of obstacles.

[0183] D2. Apply the DBSCAN algorithm to the lidar point cloud data for dynamic clustering; analyze the characteristics of each clustering cluster, preliminarily screen the possible obstacle areas; establish a preliminary obstacle candidate area list and record its position and basic feature information.

[0184] Application of the DBSCAN algorithm:

[0185] Algorithm parameter setting: Neighborhood radius (ε): Set according to the environmental complexity and the resolution of the lidar. Minimum number of samples (MinPts): Set as the minimum number of samples for clustering to ensure the effectiveness of each clustering cluster.

[0186] Dynamic clustering process: Apply the DBSCAN algorithm to the Cartesian point cloud data after lidar conversion for dynamic clustering analysis. Each clustering cluster represents a potential obstacle area. By analyzing the density and shape characteristics of the clustering clusters, possible obstacle areas are initially screened out.

[0187] Record information: Location: The central coordinates of the clustering cluster. Size: The range and volume of the clustering cluster. Features: Basic feature information such as the geometric shape and density of the clustering cluster.

[0188] Candidate area screening: According to the preset size and shape thresholds, screen out candidate obstacle areas that meet the conditions and exclude falsely identified false obstacles.

[0189] D3. For the initially identified candidate obstacle areas, extract the areas corresponding to the candidate obstacle areas identified by the lidar from the depth image of the depth camera to obtain more detailed depth information, and extract the corresponding depth image areas; calculate the depth gradient information, identify the edge areas with large depth changes, and extract the obstacle boundary features; through the depth gradient information, extract the contour line of the obstacle to clarify its three-dimensional geometric boundary; combine the boundary features extracted by the depth camera with the point cloud data of the lidar to verify the three-dimensional geometric features of the obstacle. Use the geometric matching algorithm to match the extracted features with the predefined obstacle models to confirm the effectiveness and category of the obstacle.

[0190] D4. Establish an obstacle feature database to record the location, size, and feature information of the identified obstacles; through the feature matching method, associate the detection results of the current frame with the historical records in the feature database; use the Kalman filter to track the motion state of the obstacle and predict its position change; when the obstacle is stably detected in multiple consecutive frames, confirm its effectiveness and update the environmental map.

[0191] Record the location, size, shape characteristics, etc. of each identified obstacle. Through the feature matching method, associate the obstacles detected in the current frame with the historical records in the database to ensure continuous tracking of the same obstacle in different time frames. For each associated obstacle, use the Kalman filter to track its motion state, including position, speed, and acceleration. Based on the prediction function of the Kalman filter, estimate the position change of the obstacle at the next moment to provide a reference for dynamic obstacle avoidance. Update the information of the verified and tracked obstacles to the environmental map. Regularly clean the obstacle records in the database that have not been updated for a long time or move slowly to keep the environmental map up-to-date. When an obstacle is stably detected in multiple consecutive frames, confirm its effectiveness and perform the final update of the environmental map. For detected abnormal or fast-moving obstacles, trigger an emergency obstacle avoidance strategy to ensure the safe operation of the inspection robot.

[0192] In the present invention, the environmental perception module first identifies and locates obstacle information on the inspection path, including the type, location, and dynamic state of the obstacles. Then, this detailed obstacle data is transmitted to the obstacle avoidance unit in real time, and the obstacle avoidance unit formulates corresponding obstacle avoidance strategies based on the received information, such as adjusting the path, decelerating, or stopping. The obstacle avoidance unit makes obstacle avoidance decisions through a reinforcement learning algorithm, specifically including:

[0193] E1. Extract the relative position information of the obstacles, including the distance and azimuth angle from the obstacles to the inspection robot; obtain the current motion state of the inspection robot, including position, speed, and attitude angle; calculate the local terrain features, including the ground inclination and the undulation of the grain pile surface; integrate the above information to form a state vector as the input for obstacle avoidance decision-making. The structure of the state vector S is as follows:

[0194] S = [d1, θ1, d2, θ2,..., d n , θ n , v, ω, φ, ψ]

[0195] In the formula, d i and θ i respectively represent the distance and azimuth angle of the i-th obstacle; v and w respectively represent the linear velocity and angular velocity of the inspection robot; φ and ψ respectively represent the heading angle and tilt angle of the inspection robot.

[0196] E2. Define a basic motion instruction set; specify the corresponding linear velocity and angular velocity parameters for each basic action; establish safety constraints for action execution, including maximum speed and acceleration limits.

[0197] The basic motion instruction set includes but is not limited to the following actions: Forward: Move forward at a preset linear velocity. Backward: Move backward at a preset linear velocity. Left turn: Rotate left at a preset angular velocity. Right turn: Rotate right at a preset angular velocity. Stop: Immediately stop all movements.

[0198] Specify the corresponding linear velocity v and angular velocity ω parameters for each basic action. For example: Forward: v = 1.0 m / s, ω = 0; Backward: v = -1.0 m / s, ω = 0; Left turn: v = 0, ω = 30° / s; Right turn: v = 0, ω = -30° / s; Stop: v = 0, ω = 0.

[0199] Establish safety constraints for action execution, including:

[0200] Maximum linear velocity: Not exceeding the maximum feasible walking speed v max .

[0201] Maximum angular velocity: Not exceeding the maximum rotation speed ω max .

[0202] Acceleration limit: Control the acceleration a and angular acceleration α to prevent drastic changes during movement.

[0203] E3. Design a reward function to give reward feedback to the actions of the inspection robot; the calculation formula of the reward function is as follows:

[0204] R = β1·r1 - β2·r2 + β3·r3

[0205] In the formula, r1 is the basic obstacle avoidance reward value:

[0206]

[0207] r1 calculates the distance between the inspection robot and the target path in the current state. The closer the distance, the higher the reward.

[0208] r2 is the path deviation penalty term:

[0209] r2 = d 偏离 ×γ

[0210] Where d 偏离 is the shortest distance between the current pose of the inspection robot and the predetermined inspection path; γ is the penalty coefficient for path deviation; r2 calculates the distance between the inspection robot and the nearest obstacle. The closer the distance, the greater the penalty.

[0211] r3 is the task completion reward value:

[0212]

[0213] β1, β2, β3 are weight coefficients, and it is set that β1 > β2 > β3.

[0214] E4. Based on the current state, use Q-learning to calculate the expected rewards of each possible action; combine the ε-greedy strategy to balance exploration and exploitation; select the optimal action to generate an obstacle avoidance strategy.

[0215] The Q-learning algorithm and the ε-greedy strategy are existing algorithm contents, and the present invention does not elaborate specifically. Specifically, the Q-learning algorithm and the ε-greedy strategy are used to update the Q value, select the optimal action for each state, and form an obstacle avoidance strategy. That is, based on the current state S, select the action with the highest Q value as the obstacle avoidance strategy.

[0216] In one embodiment of the present invention, the motion control unit receives the latest generated inspection path and obstacle avoidance strategy, comprehensively analyzes and precisely controls the movement of the inspection robot. Specifically, the path management module first parses the inspection path data provided by the path planning unit to determine the route points that the robot should follow. At the same time, the obstacle avoidance strategy module processes the obstacle avoidance instructions transmitted by the obstacle avoidance unit, such as emergency stop, steering, or deceleration. The motion instruction generation module prioritizes the path and obstacle avoidance instructions to ensure that the obstacle avoidance action is preferentially executed when a potential obstacle is detected. Subsequently, the execution control module converts the generated motion instructions into specific motor drive signals and sends them to the respective drive motors of the robot through the communication interface to achieve precise forward movement, steering, and speed adjustment. In addition, the feedback monitoring module real-time collects information such as the position, speed, and attitude of the robot, and uses the sensor feedback data for dynamic adjustment to ensure that the execution of the motion instructions conforms to the expected path and obstacle avoidance requirements.

[0217] In one embodiment of the present invention, after the final inspection path is formed, the inspection robot patrols the granary according to the inspection path. The autonomous navigation control module will adjust the obstacle avoidance strategy and inspection path according to the real-time data to ensure the stability and safety of the robot during the inspection process. During the robot's inspection process, the data acquisition module is responsible for real-time collecting multi-source data in the granary, including the environmental temperature and humidity measured by the temperature and humidity sensors, the images captured by the high-definition cameras, and the video information recorded by the video acquisition devices. The collected data is transmitted to the data processing and analysis module through the 5G communication transmission module. This module uses edge computing technology to perform real-time processing and analysis on the data. Specifically, the data processing and analysis module performs image recognition and analysis on the image and video data to monitor abnormal situations in the granary; it performs real-time monitoring on the temperature and humidity data to identify abnormal fluctuations in the environmental parameters. After processing, the module generates a monitoring report, the content of which includes the temperature and humidity status of the current environment, the real-time images and video records in the granary, and the environmental health status assessment obtained through data analysis. At the same time, the module will also generate warning information, such as too high temperature, abnormal humidity, detection of suspicious activities, etc., according to the preset thresholds and anomaly detection algorithms to ensure that potential risks can be promptly responded to and processed, and to guarantee the safety and management efficiency of the granary.

[0218] In addition, as Figure 2 shown, the present invention also provides an intelligent control granary automatic inspection method, which is executed in the system described in any one of the above. The method includes:

[0219] S1. Start the inspection robot, initialize the environment perception module, the autonomous navigation control module, and the 5G communication transmission module, and establish a communication connection with the remote computer;

[0220] S2. Synchronously start the lidar and the depth camera, collect two-dimensional and three-dimensional point cloud data of the surrounding environment at a predetermined frequency, and at the same time start the temperature and humidity sensor and the high-definition camera to collect relevant environmental parameters;

[0221] S3. Convert the lidar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, and perform data fusion through the voxel grid method to generate and subdivide the three-dimensional environment map;

[0222] S4. Utilize the IMU and odometer data, combine the Kalman filtering algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environment map;

[0223] S5. Input the preprocessed environment map into the path planning unit, extract environmental features based on the deep learning model, generate and smooth the preliminary inspection path using the improved A* algorithm, and adjust the path according to the real-time environmental changes;

[0224] S6. Apply the DBSCAN algorithm to perform clustering analysis on the lidar point cloud, combine the boundary feature extraction of the depth camera and the Kalman filter to perform dynamic tracking and confirmation of obstacles, and update the obstacle information in the environment map;

[0225] S7. Extract the relative positions of the obstacles, the motion state of the inspection robot, and the local terrain features, integrate them to form the state vector required for obstacle avoidance decision-making. Based on the constructed state vector, use the Q-learning in the reinforcement learning algorithm to calculate the expected rewards of each action, and combine the ε-greedy strategy to select the optimal obstacle avoidance action to generate the corresponding obstacle avoidance strategy;

[0226] S8. According to the obstacle avoidance strategy, perform segmented speed planning and attitude adjustment, and use a closed-loop control system to execute the motion instructions to control the movement of the inspection robot;

[0227] S9. Real-time monitor the environmental changes and obstacle avoidance effects during the inspection process. If anomalies or significant environmental changes are detected, update the environment map and adjust the inspection path in a timely manner, and at the same time feedback the inspection status and warning information to the remote computer through the 5G communication transmission module.

[0228] The above is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent control granary automatic inspection system, characterized in that, Including an inspection robot, a 5G communication transmission module, and a remote computer, the inspection robot includes: An environment perception module, equipped with a lidar and a depth camera, for obtaining three-dimensional data of the granary environment in real time, generating and updating an environment map, and identifying obstacle information on the inspection path; An autonomous navigation control module, connected to the environment perception module, including: A path planning unit, which performs real-time analysis on the environment map based on a deep learning algorithm, generates an inspection path, and receives an obstacle avoidance strategy from the obstacle avoidance unit to perform local path adjustment; An obstacle avoidance unit, using a reinforcement learning algorithm, generates an obstacle avoidance strategy based on the obstacle information identified by the environment perception module, and feeds back the obstacle avoidance strategy to the path planning unit to trigger local path planning; A motion control unit, which receives and synthesizes the inspection path and the obstacle avoidance strategy to control the movement of the inspection robot; A data collection module, equipped with a temperature and humidity sensor, a high-definition camera, and a video collection device, for collecting temperature, humidity, image, and video information inside the granary; A data processing and analysis module, using edge computing technology, connected to the data collection module, for processing and analyzing the multi-source data collected, and outputting a monitoring report and a warning message; Among them, the 5G communication transmission module connects the inspection robot and the remote computer, for transmitting the data output by each module in the inspection robot, and receiving instructions sent by the remote computer.

2. The intelligent control grain bin automatic inspection system according to claim 1, characterized in that, The generation process of the environment map includes: A1. The lidar performs omnidirectional scanning at a fixed frequency to obtain distance data of the surrounding environment, generates a two-dimensional point cloud dataset, divides the two-dimensional space into grid cells of size g×g, where g is the grid size, and each grid cell is identified by its central coordinates. The occupancy probability P(O j ) of each grid cell is calculated using the Bayesian update formula. A first occupancy probability threshold θ1 is set, and the grid cells with P(O j ) greater than the first occupancy probability threshold θ1 are marked as occupied grid cells, and the others are marked as free grid cells; A2. The depth camera collects three-dimensional point cloud data of the surrounding environment at a high frequency. For each occupied grid, according to the height data provided by the depth camera, adjust the height information of the occupied grid to form a three-dimensional occupied grid; use the voxel grid method to perform local fusion on the three-dimensional occupied grid and the depth camera data, eliminate noise points and fill data gaps to generate an environment map extended to three-dimensional space; A3. Set the parameters for raster cell subdivision, including the curvature threshold κ th and the occupancy difference threshold ΔP th ; further subdivide the raster cell into smaller sub-raster cells according to the curvature of the raster cell and the curvature threshold; calculate the change in the occupancy probability of the raster cell respectively represent the occupancy probabilities of raster cell j in the old and new states. If ΔP > ΔP th , then continue to subdivide this raster cell. If ΔP < -ΔP th , then merge multiple sub-raster cells with low changes into a larger raster cell; A4. After the grid cell subdivision is completed, the generated environment map is output.

3. The intelligent control grain bin automatic inspection system according to claim 2, characterized in that, In step 3, the curvature threshold κ th is divided into a high curvature threshold κ high and a low curvature threshold κ low . Calculate the curvature κ of each grid cell, and adjust the subdivision strength based on the adaptive weighting coefficient ω(κ): where, when κ ≤ κ low , ω(κ) = 1, maintaining the current grid size without subdivision; When κ low < κ ≤ κ high ω(κ) increases linearly from 0 to 1, gradually increasing the subdivision strength; When κ > κ high , ω(κ) = 1, and the high-curvature grid cells are subdivided with the greatest intensity.

4. The intelligent control grain depot automatic inspection system according to claim 2, characterized in that, The update process of the environment map is: Use the IMU and odometer equipped on the inspection robot to obtain dynamic data in real time; Adopt the Kalman filtering algorithm to fuse the IMU and odometer data, estimate the current pose of the inspection robot, and regularly perform pose correction using fixed feature points in the surrounding environment; Based on the current pose and moving direction of the inspection robot, define the area A of the environment map that needs to be updated, and dynamically adjust the range and shape of area A according to the moving speed of the inspection robot and the environmental change rate; During the inspection process, the inspection robot continuously collects data from the lidar and the depth camera, obtains new three-dimensional data, and performs filtering, denoising, and registration on the new three-dimensional data to complete preprocessing; Fuse the preprocessed new three-dimensional data into area A of the environment map, analyze the occupancy probability and terrain curvature of the updated area A, detect whether there is a significant change in the grain pile shape. If a change in the grain pile shape is detected, call the grid subdivision algorithm to re-subdivide or merge the grid cells to complete the update of the environment map.

5. The automatic inspection system for intelligent grain storage as claimed in claim 1, wherein, The generation process of the inspection path is: B1. Input an environmental map into the path planning unit. The environmental map is three-dimensional raster data, and each raster cell contains position information (x, y, z), occupancy probability P(O j ), and curvature κ; B2. Set the second occupancy probability threshold θ2. According to the second occupancy probability threshold θ2 and the occupancy probability of the grid cells, filter out the passable grid cells for marking, perform normalization processing on the curvature, and complete the preprocessing of the environmental map; B3. Input the preprocessed environmental map and the status information of the current inspection robot into the neural network model to extract local features and global features, and the model outputs the risk score U of each grid cell j , which reflects the potential obstacle density and dynamic change risk of the grid cell. Among them, the neural network model adopts a hybrid model based on convolutional neural network and graph neural network, and its structure includes multiple layers of 3D convolutional layers, graph convolutional layers and fully connected layers; the calculation formula of the risk score is: U j = f CNN-GNN (I j , Q) Where, I j represents the feature set of raster cell j, f CNN-GNN represents the trained neural network model, and Q represents the status information of the inspection robot; B4. Generate an optimized heuristic function h(n) through a deep learning model. The deep learning model includes a spatial attention layer, a temporal attention layer, and a position encoding module. The heuristic function h(n) is defined as: * (n), where the deep learning model includes a spatial attention layer, a temporal attention layer, and a position encoding module, and the heuristic function h * (n) is defined as: h * (n) = f Heuristic (n, G, Q) where n is the current search node, G is the environmental map, and f Heuristic is the deep learning model; B6. Under the guidance of the heuristic function, perform path search through the improved A* algorithm to obtain a preliminary inspection path; B7. Use the spline curve to smooth the preliminary inspection path to obtain a smoothed inspection path; B8. According to the latest environmental map and the obstacle avoidance strategy, detect obstacles or environmental changes on the path. According to the detection results, trigger local path planning, and combine with the heuristic function to fine-tune the inspection path.

6. The intelligent control grain bin automatic inspection system according to claim 5, characterized in that, The process of the improved A* algorithm is as follows: C1. Set the starting point S and the target point T, initialize the open list and the closed list. The open list is used to store the nodes to be explored, initially only containing the starting point; the closed list is used to store the explored nodes, initially empty; The initial cost of the starting point g(S) = 0, and the heuristic estimated cost h * (S) = f Heuristic (S, G, Q), and the total cost f(S) = g(S) + h * (S). Create the starting point node information and add it to the open list; C2. When the open list is not empty and the target point has not been found, perform the following steps: C21. Select the node n with the lowest f(n) = g(n) + h * (n) from the open list and move it to the closed list, where f(n) is the cost function and g(n) is the actual cost from the start point to node n; C22. If n is the target point, construct the final path and terminate the algorithm; C23. Otherwise, expand the adjacent nodes of node n. For each adjacent node m, perform the following operations: C231. If node m is already in the closed list, skip this node; C232. Calculate the actual cost g(m) of the node: g(m) = g(n) + cost(n, m) + αU m where cost(n,m) is the moving cost from node n to node m, and U m is the risk score of node m, and α is the risk weight coefficient; C233. If node m is not in the open list, add node m to the open list, record its parent node as n, calculate its heuristic estimated cost h * (m), and further calculate its total cost f(m) = g(m) + h * (m); C234. If node m is already in the open list and the new g(m) is lower, then update g(m) of node m, update its parent node to n, and recalculate the total cost f(m) = g(m) + h * (m); C24. Repeat steps C21 - C23, select the node with the lowest f(n) in the open list for expansion, and for each expanded node, update the g(m), h * (m) and f(m) until the target point is found; C3. Starting from the target point, trace back the parent nodes of the nodes in reverse to construct a complete path from the starting point to the target point, that is, the preliminary inspection path.

7. The automatic inspection system for intelligent control of grain silos according to claim 1, characterized in that, The process of identifying obstacle information is as follows: D1. Obtain the circumferential scan data of the lidar, eliminate noise through median filtering, and convert the polar coordinate data into point cloud data in the Cartesian coordinate system; collect the depth image stream of the depth camera, remove invalid points and outliers, and convert the depth data into a three-dimensional point cloud format; D2. Apply the DBSCAN algorithm to the lidar point cloud data for dynamic clustering; analyze the characteristics of each clustering cluster, initially screen the possible obstacle areas; establish a preliminary obstacle candidate area list, and record its position and basic feature information; D3. For the preliminarily identified obstacle candidate areas, extract the corresponding depth image areas; calculate the depth gradient information and extract the obstacle boundary features; Combine the depth information and the boundary features to verify and refine the three-dimensional geometric features of the obstacles; D4. Establish an obstacle feature database to record the position, size, and feature information of the identified obstacles; Through the feature matching method, associate the detection results of the current frame with the historical records in the feature database; use the Kalman filter to track the motion state of the obstacles and predict their position changes; when the obstacles are stably detected in multiple consecutive frames, confirm their effectiveness and update the environmental map.

8. The intelligent control grain bin automatic inspection system according to claim 1, characterized in that, The obstacle avoidance unit makes obstacle avoidance decisions through the reinforcement learning algorithm, specifically including: E1. Extract the relative position information of the obstacles, including the distance and azimuth angle from the obstacles to the inspection robot; obtain the current motion state of the inspection robot, including position, speed, and attitude angle; calculate the local terrain features, including the ground inclination and the undulation of the grain pile surface; integrate the above information to form a state vector as the input of the obstacle avoidance decision; E2. Define a basic motion instruction set; specify corresponding linear velocity and angular velocity parameters for each basic action; establish safety constraints for action execution, including maximum speed and acceleration limits; E3. Design a reward function to give reward feedback to the actions of the inspection robot; E4. Based on the current state, use Q-learning to calculate the expected rewards of each possible action; balance exploration and exploitation by combining the ε-greedy strategy; select the optimal action to generate an obstacle avoidance strategy.

9. The intelligent control grain bin automatic inspection system according to claim 8, characterized in that, The calculation formula of the reward function is as follows: R＝β1·r1-β2·r2+β3·r3 In the formula, r1 is the basic obstacle avoidance reward value: r2 is the path deviation penalty term: r2 = d 偏离 ×γ where d 偏离 is the shortest distance between the current pose of the inspection robot and the predetermined inspection path; γ is the penalty coefficient for path deviation; r3 is the task completion reward value: β1, β2, and β3 are weight coefficients, and it is set that β1 > β2 > β3.

10. An intelligent control granary automatic inspection method, characterized in that, The method is executed in the system according to any one of claims 1-9, and the method includes: S1. Start the inspection robot, initialize the environment perception module, the autonomous navigation control module, and the 5G communication transmission module, and establish a communication connection with the remote computer; S2. Synchronously start the lidar and the depth camera, collect two-dimensional and three-dimensional point cloud data of the surrounding environment at a predetermined frequency, and at the same time start the temperature and humidity sensor and the high-definition camera to collect relevant environmental parameters; S3. Convert the lidar data into point cloud data in the Cartesian coordinate system, combine the height information of the depth camera, and perform data fusion through the voxel grid method to generate and subdivide the three-dimensional environmental map; S4. Utilize the IMU and odometer data, combine with the Kalman filter algorithm, estimate and correct the current pose of the inspection robot in real time, and dynamically adjust the update area of the environmental map; S5. Input the preprocessed environmental map into the path planning unit, extract environmental features based on the deep learning model, generate and smooth the preliminary inspection path using the improved A* algorithm, and adjust the path according to the real-time environmental changes; S6. Apply the DBSCAN algorithm to perform clustering analysis on the lidar point cloud, combine with the boundary feature extraction of the depth camera and the Kalman filter to perform dynamic tracking and confirmation of obstacles, and update the obstacle information in the environmental map; S7. Extract the relative positions of the obstacles, the motion state of the inspection robot, and the local terrain features, integrate them to form a state vector required for obstacle avoidance decision-making. Based on the constructed state vector, use Q-learning in the reinforcement learning algorithm to calculate the expected rewards of each action, and select the optimal obstacle avoidance action by combining the ε-greedy strategy to generate the corresponding obstacle avoidance strategy; S8. According to the obstacle avoidance strategy, perform segmented speed planning and attitude adjustment, and use a closed-loop control system to execute motion instructions to control the movement of the inspection robot; S9. Real-time monitor the environmental changes and obstacle avoidance effects during the inspection process. If anomalies or significant environmental changes are detected, update the environmental map and adjust the inspection path in a timely manner, and at the same time feedback the inspection status and warning information to the remote computer through the 5G communication transmission module.

Images