Mobile robot efficient inspection method based on AI large model online decision

Abstract

The invention relates to the technical field of intelligent inspection, in particular to a mobile robot efficient inspection method based on AI large model online decision. The method comprises the following steps: constructing a motion joint coordinate system of the mobile robot and carrying out kinematics analysis, constructing an external sensor coordinate system of the mobile robot and carrying out calibration, setting a bottom layer control communication mode of the mobile robot, setting a speedometer calibration mode of the mobile robot, and setting a speedometer calibration mode of the mobile robot. The optimal moving speed and the optimal moving track of the mobile robot are determined based on a navigation algorithm, a YOLOV4-tiny model and a semantic segmentation model are determined based on an AI large model, and an inspection strategy is executed to complete a detection task. Through kinematics analysis, sensor calibration, underlying control communication, a navigation algorithm and the decision scheduling capability of the AI large model, efficient adaptation and flexible response to a complex and changeable inspection environment are realized, the inspection efficiency is improved, the inspection resources are saved, and the man-machine interaction experience is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection technology, and in particular to an efficient inspection method for a mobile robot based on online decision-making of an AI large model. Background Art

[0002] Inspection is a periodic inspection of public facilities, traffic conditions and manufacturing processes, aimed at discovering and solving potential or existing problems. Intelligent inspection, as a product of scientific and technological progress, uses computer-controlled machines to conduct automated inspections of manufacturing enterprises, marking a significant improvement in my country's industrial level and is crucial to enterprise production. Compared with manual inspections, intelligent inspections can more quickly identify and deal with safety hazards, present the problem site in a graphic and textual manner, assist management personnel in timely dispatch, and optimize inspection strategies in combination with map data to prevent hidden dangers.

[0003] However, in the construction of smart factories, traditional manual or semi-automatic inspection methods are gradually being eliminated because they are time-consuming and labor-intensive, have limited accuracy, and are susceptible to environmental interference. Although there are mobile robot motion planning and obstacle avoidance methods proposed in Chinese patent CN116540747A, the existing ROS-based mobile robot inspection still has limitations, especially when different inspection strategies need to be adopted for different industrial equipment, and conventional mobile robots are difficult to switch autonomously. Therefore, how to overcome the problems of single method, low efficiency, and excessive human intervention in the execution of inspection mobile robot tasks has become a technical problem that needs to be overcome in this field. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide an efficient inspection method for mobile robots based on online decision-making of an AI large model.

[0005] The technical solution adopted by the present invention is as follows: An efficient inspection method for a mobile robot based on online decision-making of an AI large model includes the following steps: S1. Construct the motion joint coordinate system of the mobile robot and perform kinematic analysis; S2, construct the external sensor coordinate system of the mobile robot and calibrate it; S3, setting the underlying control communication mode of the mobile robot; S4, setting the odometer calibration mode of the mobile robot; S5. Determine the optimal moving speed and motion trajectory of the mobile robot based on the navigation algorithm; S6. Determine the YOLOV4-tiny model and semantic segmentation model based on the AI ​​big model, and execute the inspection strategy to complete the detection task.

[0006] This technical solution achieves efficient adaptation and flexible response to complex and changeable inspection environments through kinematic analysis, sensor calibration, underlying control communication, navigation algorithms, and the decision-making and scheduling capabilities of large AI models. It not only improves inspection efficiency and saves inspection resources, but also enhances the human-computer interaction experience. Specifically, kinematic analysis ensures that the mobile robot accurately understands its own motion state, while sensor calibration ensures that the mobile robot accurately perceives the external environment. Through precise mathematical models, the mobile robot calculates its own position, speed and acceleration. At the same time, through the precise calibration of sensors (such as cameras and lidar), the mobile robot accurately obtains environmental information such as distance, shape and color. The underlying control communication is the key to the mobile robot's execution of instructions, which determines how the mobile robot receives and executes control commands from the upper layer. The odometer calibration is an important step to ensure that the mobile robot accurately records its own moving distance and direction. These two steps together ensure the stability and accuracy of the mobile robot when performing tasks. The navigation algorithm is the core of the mobile robot's path planning, which calculates an optimal path based on the mobile robot's current position, target position and surrounding environment information. The AI ​​big model plays a core role in decision-making and scheduling here. Through the self-attention mechanism during the training process, the AI ​​big model learns the correlation and priority between different inspection tasks, thereby formulating a more efficient inspection strategy. The YOLOV4-tiny model and the semantic segmentation model are used to inspect the inspection environment. Real-time perception and analysis provide decision-making basis for AI big models; through the online decision-making ability of AI big models, mobile robots can perceive environmental changes in real time during the inspection process, and automatically adjust the inspection strategy according to environmental changes, so as to maintain efficient and stable inspection results in the face of complex and changeable inspection environments; AI big models learn the relationship between the advantages and disadvantages of different inspection strategies through the self-attention mechanism during the training process, so as to automatically adjust the inspection strategy according to environmental changes, and flexibly switch the inspection strategy when facing different inspection tasks to improve inspection efficiency; through the decision-making and scheduling ability of AI big models, mobile robots can While ensuring the inspection effect, it reduces the number of unnecessary inspections and inspection time, thereby saving inspection resources; the AI ​​big model automatically adjusts the inspection strategy according to environmental changes, so that the mobile robot can maintain an efficient and stable inspection speed when facing complex and changeable inspection environments, thereby improving inspection efficiency; through the online decision-making ability of the AI ​​big model, the mobile robot interacts with the user in real time during the inspection process, provides inspection progress and inspection result information, and enhances the user experience; at the same time, the user can also fine-tune the inspection strategy of the mobile robot through visual prompts to achieve more flexible human-computer interaction.

[0007] In addition, the mobile robot efficient inspection method based on online decision-making of AI large model proposed above according to the present invention also has the following additional technical features: According to one embodiment of the present invention, the motion joint coordinate system of the mobile robot in step S1 is constructed as follows: S11, establish the left front wheel coordinate system front_left_wheel; S12, establishing a right front wheel coordinate system front_right_wheel; S13, establishing the left rear wheel coordinate system back_left_wheel; S14, establishing a right rear wheel coordinate system back_right_wheel; S15. Establish a coordinate system base_link with the mobile robot as the geometric center.

[0008] This technical solution establishes coordinate systems (front_left_wheel, front_right_wheel, back_left_wheel, back_right_wheel) for the left front wheel, right front wheel, left rear wheel and right rear wheel respectively, accurately describing the motion state of each wheel, including position, speed and acceleration; at the same time, the geometric center coordinate system (base_link) of the mobile robot is established as a reference point for the overall motion of the mobile robot, which helps to achieve accurate control and navigation of the mobile robot as a whole. In practical applications, mobile robots are usually also equipped with multiple sensors such as odometers, gyroscopes, and cameras. By combining sensor data with the motion state of the wheel coordinate system, the accuracy and robustness of the mobile robot's positioning and navigation are improved.

[0009] According to one embodiment of the present invention, the kinematic analysis in step S1 includes the following steps: S16. Given the speed of each wheel, find the speed of the mobile robot in the Cartesian coordinate system. The kinematic solution is: V5=(V1+V2+V3+V4) / 4; V6=(V3-V1+V2-V4) / 4; V7=(-V3-V1+V2+V4) / (2H+2W); S17. Given the velocity of the mobile robot in the Cartesian coordinate system, the velocity of each wheel is solved, and the inverse kinematic solution is: V1=V5-V6-V7*(H / 2+W / 2); V2=V5+V6+V7*(H / 2+W / 2); V3=V5+V6-V7*(H / 2+W / 2); V4=V5-V6+V7*(H / 2+W / 2); Among them, V1, V2, V3, and V4 represent the speeds of the left front wheel, right front wheel, left rear wheel, and right rear wheel of the mobile robot, respectively, in m / s; V5 and V6 represent the linear speed of the mobile robot, and V7 represents the angular speed of the mobile robot; in the Cartesian coordinate system, the front of the car is the X direction, the right is the Y direction, and the Z direction is perpendicular to the plane of the car; W and H are the wheelbase and track width of the mobile robot, respectively, in meters.

[0010] In the technical solution, the linear velocity V5 represents that the speeds of the four wheels have a common contribution in the horizontal horizontal direction, that is, the average speed of the four wheels represents the overall linear velocity of the mobile robot. The linear velocity V6 represents the difference between the left and right wheel speeds, and reflects the velocity component of the mobile robot perpendicular to the forward direction (i.e., lateral direction) by calculating the difference in the diagonal wheel speeds (left front and right rear, right front and left rear); the angular velocity V7 reflects the angular velocity of the mobile robot rotating around its center point by calculating the difference in the speeds of the left and right wheels (taking into account the wheelbase H and track W); due to the layout of the wheels of the mobile robot (assuming a rectangle or approximate rectangle), the difference in the speeds of the left and right wheels is related to the wheelbase and track, so the angular velocity is calculated; the speed of each wheel is calculated by combining the linear velocity V5, the lateral velocity V6 and the angular velocity V7 with the geometric parameters of the mobile robot (wheelbase H and track W). Here we consider that when the mobile robot rotates around its center point, each wheel has a different linear velocity due to its different position; the angular velocity V7 multiplied by (2H+2W) is to convert the angular velocity into the linear velocity component of each wheel relative to the center point; the lateral velocity V6 also reflects the lateral speed difference between the left and right wheels through the change of positive and negative signs.

[0011] According to one embodiment of the present invention, the step S2 of constructing and calibrating the external sensor coordinate system of the mobile robot includes the following steps: S21, establish camera coordinate system camera_link; S22, establish laser radar coordinate system laser_link; S23. In the ROS launch file, fix the original coordinates of the camera in the camer_link coordinate system, fix the original coordinates of the lidar in the laser_link coordinate system, start the ROS TF2 coordinate transformation, and convert the spatial coordinate information perceived by the camera and lidar into the information in the base_link coordinate system.

[0012] This technical solution achieves the fusion and unified processing of different sensor data by establishing the coordinate system of the camera and the laser radar and using the ROS and TF2 libraries for coordinate transformation. Specifically, the camera coordinate system camera_link is established to convert the pixel coordinates in the image captured by the camera into coordinates in three-dimensional space, so as to realize the perception of the object's position and distance information; the laser radar coordinate system laser_link is established to accurately convert the point cloud data into coordinates in three-dimensional space, which is then used for environmental perception, obstacle avoidance, and path planning tasks; ROS launch file: ROS (Robot Operating System) ROS is an open source operating system that provides hardware abstraction, device drivers, function libraries, visualization tools, message communication, and package management for mobile robots. The launch file is used to define and start multiple nodes and parameters in the ROS system. TF2 is a library for processing coordinate transformations in ROS, which allows coordinates to be converted between multiple coordinate systems and supports timestamps to handle coordinate transformations in dynamic environments. The base_link coordinate system is the base coordinate system of the mobile robot, that is, the reference coordinate system of the mobile robot itself. The coordinates of all other sensors and components are transformed relative to this coordinate system. The coordinate transformation fixes the original coordinates of the camera and lidar in camera_link and las respectively. After being converted to the er_link coordinate system, the coordinate transformation is performed through the TF2 library, and the spatial coordinate information perceived by these sensors is uniformly converted to the base_link coordinate system, so that the data perceived by different sensors can be fused and compared in the same reference coordinate system, thereby achieving more accurate environmental perception; the data perceived by different sensors (such as cameras and lidars) are fused together to improve the accuracy and reliability of the mobile robot's perception of the environment; through coordinate transformation, the data perceived by different sensors are also checked for consistency to detect sensor failures; ROS and TF2 libraries provide an efficient communication mechanism, so that different nodes can share and update coordinate transformation information in real time, thereby supporting real-time perception and decision-making of mobile robots.

[0013] According to one embodiment of the present invention, the step S4 of setting the odometer calibration mode of the mobile robot comprises the following steps: S41, start the ROS keyboard control node; S42, the mobile robot moves forward 1m in the X direction and checks whether the ROS odometer record is 1m; S43, the mobile robot moves forward 1m in the Y direction and checks whether the ROS odometer record is 1m; S44, the mobile robot rotates clockwise for one circle to check whether the odometer is 3.14 rad; S45. An error of 3% is allowed. If the error exceeds the allowable value, the parameters are corrected in the single chip microcomputer, and the process from step S41 to step S44 is repeated until it is within the allowable error range.

[0014] This technical solution realizes the calibration of the mobile robot odometer by starting the ROS keyboard control node, moving the robot on a predetermined path, verifying the consistency between the odometer record and the actual movement, and performing parameter correction when necessary. Specifically, starting the keyboard control node is to manually control the movement of the mobile robot so that it can move along the predetermined path in the subsequent calibration process; the odometer is an important sensor in the navigation and positioning of the mobile robot. The displacement and direction change of the mobile robot are estimated by measuring the rotation of the wheels of the mobile robot. The purpose is to verify the accuracy of the odometer in the X direction; the accuracy of the odometer is evaluated by allowing the mobile robot to move a known distance (1m) in the X direction and comparing the recorded value of the odometer with the actual value; by changing the direction of movement, ensure that the odometer can accurately record the displacement in all major directions; S44: The mobile robot rotates one circle clockwise to check whether the odometer is 3.14rad; the mobile robot rotates one circle (360 degrees or 2π radians, approximately 3.14 rad), the odometer should record the corresponding angle change; the actual odometer record of one rotation is slightly greater than or less than 3.14 rad, because the wheel slippage and uneven ground factors cause the actual rotation angle to deviate from the theoretical value; an error of 3% is allowed. If the error exceeds the allowable value, the parameters are corrected in the microcontroller, and the process from step S41 to step S44 is repeated until it is within the allowable error range; due to various factors (such as wheel size, ground conditions, sensor noise) causing errors in the odometer data, correction is required; by repeatedly executing the calibration process (S41 to S44) and correcting the parameters in the microcontroller according to the allowable error value, the error is gradually reduced and the accuracy of the odometer is improved.

[0015] According to one embodiment of the present invention, the navigation algorithm of step S5 relies on a global map, self-positioning, path planning, motion control, and environmental perception, wherein: The global map is constructed using SLAM and relies on environmental perception; Self-positioning, using the amcl adaptive Monte Carlo algorithm, relies on environmental perception and global maps; Path planning, including local path planning and global path planning. The local path planning obtains the linear velocity in the X direction, the linear velocity in the Y direction and the angular velocity in the Z direction of the mobile robot, which are transmitted to the single-chip microcomputer through step S3 to complete motion control; Environmental perception, using the depth camera and lidar in step S2.

[0016] In this technical solution, SLAM updates the map in real time by combining the motion information of the mobile robot and the environmental perception data from the sensor; AMCL reflects the uncertainty of the mobile robot's position by continuously updating the position and weight of the particles; global path planning is to plan an optimal path from the starting point to the end point for the mobile robot on a known map; local path planning is to adjust the motion trajectory of the mobile robot according to the real-time environmental perception data during the actual movement of the mobile robot to avoid obstacles or respond to emergencies; local path planning usually outputs the linear velocity and angular velocity of the mobile robot; environmental perception is to collect information about the mobile robot's surrounding environment through sensors; the depth camera provides depth information of the scene, and the lidar provides high-precision distance measurement and point cloud data.

[0017] According to one embodiment of the present invention, the step S5 of determining the optimal moving speed and motion trajectory of the mobile robot based on the navigation algorithm includes the following steps: S51, the global map, self-positioning, path planning, motion control, and environmental perception in the navigation algorithm belong to the overall internal closed loop, and the input is the current X-direction linear velocity and Y-direction linear velocity of the mobile robot and the Z-direction angular velocity, laser radar information, depth camera information, and various coordinate information; the output is the X-direction linear velocity and Y-direction linear velocity and Z-direction angular velocity of the mobile robot at the next moment; S52, the single chip microcomputer obtains the linear velocity in the X direction and the Y direction and the angular velocity in the Z direction of the mobile robot, and then uses the kinematic inverse solution of step S1 to obtain the speed of each wheel, and quickly reaches the target speed through the incremental PID; The actual PID output is: u(k)=Kp*e(k-1)+Ki*e(k)+Kd*(e(k)-2e(k-1)+e(k-2))+u(k-1) Among them, u(k) represents the actual output of this time; u(k-1) represents the output of the previous time; △u(k) represents the output change; Kp represents the proportional coefficient; Ki represents the integral coefficient; Kd represents the differential coefficient; e(k-1) represents the error value between the previous target and the actual value; e(k) represents the error value between the current target and the actual value; e(k-2) represents the error value between the previous target and the actual value; S53, global path planning uses A* algorithm: The evaluation function is: f(n)=g(n)+h(n) Among them, g(n) represents the actual path cost from the starting point to any node n; h(n) represents the estimated cost from node n to the end point; S54, local path planning uses DWA algorithm; The DWA algorithm samples multiple sets of velocities in the velocity space (v, ω) and simulates the motion trajectory under the velocity. The evaluation function is used to select the optimal trajectory, which includes the following factors: Approaching the target point Target is evaluated by calculating the distance between the end point of the trajectory and the target point; Stay away from obstacles and use the distance to the nearest obstacle as the evaluation criterion; Path smoothness is evaluated by calculating the curvature or acceleration change of the trajectory; Score=α*DistanceToTarget+β*DistanceToObstacle+γ*Smoothness Among them, α, β, and γ all represent weight coefficients, which are used to adjust the importance of different factors.

[0018] This technical solution determines the optimal moving speed and motion trajectory of the mobile robot by constructing a closed-loop system, using kinematic inverse solution and PID control to achieve speed control, using the A* algorithm for global path planning, and using the DWA algorithm for local path planning strategy; specifically, by continuously receiving the current motion state of the mobile robot (linear velocity in the X and Y directions, angular velocity in the Z direction) and sensor information (lidar, depth camera), and outputting the motion command at the next moment, real-time control and adjustment of the mobile robot's motion are achieved; the single-chip microcomputer first obtains the motion state information of the mobile robot, and then calculates the speed of each wheel through the inverse kinematic solution; an incremental PID control algorithm is used to calculate the speed of each wheel according to the target speed and the actual The output is adjusted according to the speed error, so that the mobile robot can reach the target speed quickly and accurately; PID control comprehensively considers the past, present and future of the error through three links: proportional, integral and differential, so as to achieve precise control of the speed; the A* algorithm reduces the search space and improves the search efficiency while ensuring the optimal solution; the DWA algorithm samples multiple groups of speeds in the speed space (v, ω) and simulates the motion trajectories at these speeds; then, the optimal trajectory is selected through the evaluation function; the evaluation function comprehensively considers the factors of approaching the target point, staying away from obstacles and path smoothness, and adjusts the importance of different factors through weight coefficients α, β, γ. The DWA algorithm plans a safe and efficient motion trajectory in real time in a complex environment.

[0019] According to one embodiment of the present invention, the AI ​​big model of step S6 reads the background information and strategy analysis plan of steps S1 to S5, and Paddleocr reads the text information and transmits it to the AI ​​big model, and the AI ​​big model executes the inspection strategy using the read information.

[0020] This technical solution deeply integrates and analyzes the detailed background information and strategy analysis framework provided by steps S1 to S5. It first uses PaddleOCR technology to accurately identify and extract text information, and then efficiently transmits this information to the AI ​​big model. Based on this comprehensive information, the AI ​​big model uses its powerful data processing and decision-making capabilities to accurately execute the preset inspection strategy.

[0021] According to an embodiment of the present invention, the execution of the inspection strategy in step S6 includes the following steps: S61, based on steps S1 to S5, complete the construction of a conventional inspection AGV; S62. Use Python language to import the ERNIE Bot library of the AI ​​big model and set instructions in the AI ​​big model background settings; S63. When the AI ​​big model reads instructions through PaddleOCR, it uses Python language to extract key points, and then publishes topics through the topic of the ROS control system.

[0022] In this technical solution, the specific steps of executing the inspection strategy include building a regular inspection AGV, setting AI big model instructions, extracting key points and publishing topics, thus realizing autonomous inspection of the mobile robot under the guidance of the AI ​​big model.

[0023] According to one embodiment of the present invention, the method further comprises the following subsequent steps: S7. The mobile robot uses the opencv library to capture images from the camera and publishes the topic name to the ROS system, including the following steps: S71. After the mobile robot is started, it automatically uses the opencv library to capture images from the camera and publishes them to the ROS system with the topic name "image". S72. After the mobile robot reaches the target position through step S5, it receives the image content of the topic "image" through a program based on pytorch framework reasoning, and brings the image into the yolov4-tiny model. After the model detects the instrument, it automatically publishes the topic "coordinate". The content of the topic is five coordinate points, the upper left border coordinates, the upper right border coordinates, the lower left border coordinates, the lower right border coordinates, and the frame center coordinates. ROS TF2 is started to convert the coordinates based on the camera space into the coordinates based on base_link through steps S1 and S2. The mobile robot chassis adjusts its posture based on the five coordinate points, that is, the base_link coordinates are aligned with the five coordinate points in the x-axis direction; S73. After the posture is adjusted, the instrument is segmented using the semantic segmentation model for reading, and the "threshold" topic is received at the same time, and the read data is compared with the threshold information of "threshold": If it is within the threshold range, the mobile robot goes to the next target point for inspection; If it is outside the threshold range, the mobile robot will issue an alarm message to the system.

[0024] This technical solution uses the opencv library to capture images through a mobile robot and publishes them to the ROS system, and uses the yolov4-tiny model and semantic segmentation model to detect and read instruments, thereby achieving accurate inspection and abnormality detection of target equipment and improving inspection efficiency and accuracy; by receiving the threshold information of the "threshold" topic for comparison, intelligent judgment and decision-making of inspection results are achieved. Compared with the prior art, the present invention has the following beneficial effects: (1) Through kinematic analysis, sensor calibration, underlying control communication, navigation algorithms, and the decision-making and scheduling capabilities of AI large models, mobile robots can efficiently adapt to and flexibly respond to complex and changing inspection environments, improve inspection efficiency, and save inspection resources; (2) The AI ​​large model perceives environmental changes in real time during the inspection process, and automatically adjusts the inspection strategy according to environmental changes to maintain efficient and stable inspection results; it uses PaddleOCR technology to accurately identify and extract text information, and combines the YOLOV4-tiny model and semantic segmentation model for real-time perception and analysis to improve the accuracy of inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the control system of a mobile robot under the decision-making and scheduling of an AI large model; Figure 2 It is a schematic diagram of the mechanical structure of the mobile robot and the connecting rod and joint coordinates of the external sensors; Figure 3 It is a schematic diagram of the underlying motion control of the mobile robot; Figure 4 It is a schematic diagram of the navigation algorithm of the upper system of the mobile robot; Figure 5 This is a schematic diagram of the network structure of the YOLOV4-Tiny model based on PyTorch computer vision used by mobile robots in inspections; Figure 6 This is a schematic diagram of the YOLOV4-Tiny model reasoning example; Figure 7 This is a schematic diagram of the network structure of the PaddleSeg semantic segmentation model for industrial instruments used by mobile robots in target detection and positioning; Figure 8 This is a schematic diagram of a semantic segmentation model reasoning instance; Fig. 9 This is a schematic diagram of the Gazebo simulation of a mobile robot in the ROS environment; Fig.10 This is a physical picture of a mobile robot based on AI decision-making and scheduling. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Example 1 like Figure 1 As shown, this embodiment provides an efficient inspection method for a mobile robot based on online decision-making of an AI large model, comprising the following steps: S1. Construct the motion joint coordinate system of the mobile robot and perform kinematic analysis; S2, construct the external sensor coordinate system of the mobile robot and calibrate it; S3, setting the underlying control communication mode of the mobile robot; S4, setting the odometer calibration mode of the mobile robot; S5. Determine the optimal moving speed and motion trajectory of the mobile robot based on the navigation algorithm; S6. Determine the YOLOV4-tiny model and semantic segmentation model based on the AI ​​big model, and execute the inspection strategy to complete the detection task.

[0028] This technical solution achieves efficient adaptation and flexible response to complex and changeable inspection environments through kinematic analysis, sensor calibration, underlying control communication, navigation algorithms, and the decision-making and scheduling capabilities of large AI models. It not only improves inspection efficiency and saves inspection resources, but also enhances the human-computer interaction experience. Specifically, kinematic analysis ensures that the mobile robot accurately understands its own motion state, while sensor calibration ensures that the mobile robot accurately perceives the external environment. Through precise mathematical models, the mobile robot calculates its own position, speed and acceleration. At the same time, through the precise calibration of sensors (such as cameras and lidar), the mobile robot accurately obtains environmental information such as distance, shape and color. The underlying control communication is the key to the mobile robot's execution of instructions, which determines how the mobile robot receives and executes control commands from the upper layer. The odometer calibration is an important step to ensure that the mobile robot accurately records its own moving distance and direction. These two steps together ensure the stability and accuracy of the mobile robot when performing tasks. The navigation algorithm is the core of the mobile robot's path planning, which calculates an optimal path based on the mobile robot's current position, target position and surrounding environment information. The AI ​​big model plays a core role in decision-making and scheduling here. Through the self-attention mechanism during the training process, the AI ​​big model learns the correlation and priority between different inspection tasks, thereby formulating a more efficient inspection strategy. The YOLOV4-tiny model and the semantic segmentation model are used to inspect the inspection environment. Real-time perception and analysis provide decision-making basis for AI big models; through the online decision-making ability of AI big models, mobile robots can perceive environmental changes in real time during the inspection process, and automatically adjust the inspection strategy according to environmental changes, so as to maintain efficient and stable inspection results in the face of complex and changeable inspection environments; AI big models learn the relationship between the advantages and disadvantages of different inspection strategies through the self-attention mechanism during the training process, so as to automatically adjust the inspection strategy according to environmental changes, and flexibly switch the inspection strategy when facing different inspection tasks to improve inspection efficiency; through the decision-making and scheduling ability of AI big models, mobile robots can While ensuring the inspection effect, it reduces the number of unnecessary inspections and inspection time, thereby saving inspection resources; the AI ​​big model automatically adjusts the inspection strategy according to environmental changes, so that the mobile robot can maintain an efficient and stable inspection speed when facing complex and changeable inspection environments, thereby improving inspection efficiency; through the online decision-making ability of the AI ​​big model, the mobile robot interacts with the user in real time during the inspection process, provides inspection progress and inspection result information, and enhances the user experience; at the same time, the user can also fine-tune the inspection strategy of the mobile robot through visual prompts to achieve more flexible human-computer interaction.

[0029] S1: Establish the motion joint coordinate system of the mobile robot for kinematic analysis; First, establish the front_left_wheel coordinate system for the left front wheel of the mobile robot, the front_right_wheel coordinate system for the right front wheel, the back_left_wheel coordinate system for the left rear wheel, the back_right_wheel coordinate system for the left rear wheel, and the base_link coordinate system for the geometric center of the mobile robot.

[0030] The specific process of kinematic analysis is as follows: The speeds of the left front wheel, right front wheel, left rear wheel and right rear wheel of the mobile robot are represented by V1, V2, V3 and V4 in units of m / s respectively; the wheelbase and wheelbase of the mobile robot are represented by W and H in units of m respectively; it is stipulated that in the Cartesian coordinate system, the front of the car is the X direction, the right is the Y direction, and the Z direction is perpendicular to the plane of the car; the corresponding speeds are V5, V6 and V7, where V5 and V6 are linear speeds and V7 is angular speed; Among them, the speed of each wheel is known to solve the speed of the mobile robot in the Cartesian coordinate system, and the kinematic solution is: V5=(V1+V2+V3+V4) / 4; V6=(V3-V1+V2-V4) / 4; V7=(-V3-V1+V2+V4) / (2H+2W); Given the velocity of the mobile robot in the Cartesian coordinate system, the velocity of each wheel can be solved, and the inverse kinematic solution is: V1=V5-V6-V7*(H / 2+W / 2); V2=V5+V6+V7*(H / 2+W / 2); V3=V5+V6-V7*(H / 2+W / 2); V4=V5-V6+V7*(H / 2+W / 2).

[0031] S2: Establishing a sensor coordinate system for the external sensor of the mobile robot for calibration; In this step, the Intel RealSense D435 camera coordinate system camera_link is established, the laser radar coordinate system laser_link is established, the camera original coordinates are fixed in the camer_link coordinate system in the ROS launch file, the laser radar original coordinates are fixed in the laser_link coordinate system, and the ROS TF2 coordinate transformation is started to convert the spatial coordinate information perceived by the camera and the laser radar into the information in the base_link coordinate system. The mechanical structure of the mobile robot and the connecting rod and joint coordinate states of the external sensors are as follows: Figure 2 shown.

[0032] S3: Set the underlying control communication mode, and make the underlying STM32F407 microcontroller communicate with Jetson orin nano via the USB data cable; the specific contents of the serial communication between the STM32F407 microcontroller and Jetson orin nano are as follows: Table 1: STM32 sends mobile robot status information table

[0033] Among them, taking the command [7B00000064FF9C012C517D] of sending 11 bytes of control to STM32 through serial port 1, that is, the command of X-axis speed 0.1m / s, Y-axis speed -0.1m / s, and Z-axis speed 0.3rad / s, STM32 sends 24 bytes of data [7B000065FF9B0126FCD801E03EFC0024009CFFF92E16DC7D] outward through serial port 1.

[0034] Now let's interpret these 24 bytes of data one by one.

[0035] Frame header and disabled flag: The first byte: frame header, fixed value, 0X7B.

[0036] The second byte: the disabled flag, 1 represents that the mobile robot is in a disabled state, at which point the mobile robot stops moving and does not accept control. There are generally three reasons for the disabled state: the battery voltage is lower than 10V, and the battery is about to run out of power; the enable switch is turned to the OFF end; and the robot is within 10 seconds of powering on.

[0037] 0X00, indicating that the mobile robot is in an enabled (non-disabled) state and is under control.

[0038] Mobile robot XYZ three-axis real-time speed: Bytes 3 to 8: Real-time speed of the mobile robot's X, Y, and Z axes. Odd bytes are the high 8-bit data of the corresponding speed, and even bytes are the low 8-bit data of the corresponding speed. The speed units of the X and Y axes are 0.001 m / s, and the speed unit of the Z axis is 0.001 rad / s. The speed is the real-time speed of the wheel obtained by the motor encoder and is calculated through kinematics forward solution.

[0039] The third and fourth bytes: the real-time speed of the X axis of the mobile robot. 0X00, 0X65, you can see that the highest bit of the high 8-bit data is 0, which means that the speed is positive and does not need to be complemented.

[0040] Then the real-time speed of the mobile robot's X axis = 0X00*2^8+0X65=0*256+6*16+5=101mm / s. It can be seen that it is very close to the target value of 100mm / s of the control command.

[0041] The 5th and 6th bytes: the real-time speed of the Y axis of the mobile robot. 0XFF = 11111111 (binary), 0X9B. You can see that the highest bit of the high 8-bit data is 1, which means that the speed is negative and needs to be complemented. Then the real-time speed of the Y axis of the mobile robot = -[2^16-(0XFF*2^8+0X9B)] = -[(0XFF*2^8+0XFF+1)-(0XFF*2^8+0X9B)] = -(0XFF-0X9B+1) = -(0X64+1) = -(6*16+4+1) = -101mm / s. It can be seen that it is very close to the target value of -101mm / s of the control command.

[0042] The 7th and 8th bytes: the real-time speed of the Z axis of the mobile robot. 0X01, 0X26, we can see that the highest bit of the high 8-bit data is 0, which means that the speed is positive and does not need to be complemented.

[0043] Then the real-time speed of the mobile robot's Z axis = 0X01*2^8+0X26=1*256+2*16+6=294*0.001rad / s=0.294rad / s. It can be seen that it is very close to the target value of 0.3rad / s of the control command.

[0044] S4: Set odometer calibration; the specific process is as follows: start the ROS keyboard control node, move the robot forward 1m in the X direction, check whether the ROS odometer record is 1m, move the robot forward 1m in the Y direction, check whether the ROS odometer record is 1m, the mobile robot rotates clockwise for one circle, and check whether the odometer is 3.14rad; an error of 3% is allowed. If the error exceeds the allowable value, the mechanical structure parameters are corrected in the STM32F407 microcontroller, and the above process is repeated until it is within the allowable error range.

[0045] S5: Determine the optimal moving speed and motion trajectory of the mobile robot according to the navigation algorithm, such as Figure 3 , 4 shown.

[0046] The navigation algorithm relies on the global map, self-positioning, path planning, motion control, and environmental perception; among which environmental perception relies on the depth camera and laser radar in step S2, and the global map is SLAM mapping, which relies on environmental perception; self-positioning uses the amcl adaptive Monte Carlo algorithm, which relies on environmental perception and the global map; path planning is local path planning and global path planning; the mobile robot's X-direction linear velocity, Y-direction linear velocity, and Z-direction angular velocity obtained by local path planning are transmitted to an STM32F407 microcontroller through step S3 to complete motion control. The global map, self-positioning, path planning, motion control, and environmental perception in the navigation algorithm belong to the overall internal closed loop, and the input is the current mobile robot's X-direction linear velocity, Y-direction linear velocity, and Z-direction angular velocity, laser radar information, depth camera information, and various coordinate information; the output is the mobile robot's X-direction linear velocity, Y-direction linear velocity, and Z-direction angular velocity at the next moment; The single-chip microcomputer obtains the linear velocity in the X direction, the linear velocity in the Y direction, and the angular velocity in the Z direction of the mobile robot, and then uses the kinematic inverse solution of step S1 to obtain the speed of each wheel, and quickly reaches the target speed through the incremental PID; Actual PID output: u(k)=Kp*e(k-1)+Ki*e(k)+Kd*(e(k)-2e(k-1)+e(k-2))+u(k-1); u(k): the actual output of this time; u(k-1): the last output; △u(k): output change; Kp: proportionality coefficient; Ki: integration coefficient; Kd: differential coefficient; e(k-1): the error between the last target and the actual value; e(k): The error between the target and the actual value this time; e(k-2): the error between the previous target and the actual value; Global path planning uses the A* algorithm: Evaluation function: f(n)=g(n)+h(n); g(n): the actual path cost from the starting point to any node n; h(n): estimated cost from node n to the end point; The local path planning adopts the DWA algorithm: The DWA algorithm samples multiple sets of speeds in the speed space (v, ω) and simulates the motion trajectories under these speeds. The evaluation function is used to select the optimal trajectory, which includes the following factors: approaching the target point Target: evaluated by calculating the distance between the end point of the trajectory and the target point, and staying away from the obstacle Obstacle: using the distance to the nearest obstacle as the evaluation criterion. Path smoothness Smoothness: evaluated by calculating the curvature of the trajectory or the change in acceleration; Score=α*DistanceToTarget+β*DistanceToObstacle+γ*Smoothness; Among them, α, β, and γ are weight coefficients used to adjust the importance of different factors.

[0047] Step S6: Setting the Wenxinyiyan model to participate in the mobile robot control system and execution strategy through PaddleOCR; The Wenxinyiyan model is used to inform the mobile robot of steps S1, S2, S3, S4, S5, background information, and strategy analysis solutions in the big model. Paddleocr reads the text information and transmits it to the big model, which uses the information to determine the inspection strategy. The specific process of determining the inspection strategy is as follows: Based on step S1, step S2, step S3, step S4, and step S5, the construction of the conventional inspection AGV is completed; Use Python to import the ERNIE Bot library, and set the background setting of the large model as "I am an AGV mobile robot in a modern intelligent manufacturing factory. My task is to detect instruments at the navigation target point, and detect fire, garbage and workers who stray into the workspace during the rest of the cruise. If I detect fire or garbage during the execution of my task, I will send the plane coordinates to the background of the intelligent manufacturing system in time for relevant personnel to handle. In the process of checking the instrument, the recognition threshold of the instrument is dynamically divided according to the visual text prompts around the instrument."; When the Wenxinyiyan model reads the input "The normal value of the instrument during the day is 100-200, and the normal value at night is 130-240" through PaddleOCR, the key points "daytime", "100-200", "nighttime", and "130-240" are extracted through the Python language, and a topic named "threshold" is published through the topic of the ROS control system. The topic content is "daytime", "100-200", "nighttime", and "130-240".

[0048] S7: The YOLOV4-tiny model and the semantic segmentation model rely on Wenxinyiyan to obtain the execution strategy to complete the detection task, such as Figure 5 , 6, 7, and 8.

[0049] After the mobile robot starts, it automatically uses the opencv library to capture images from the camera and publishes them to the ROS system with the topic name "image"; After the mobile robot reaches the target position through step S5, it receives the image content of the "image" topic through a program based on pytorch framework reasoning, and brings the image into the yolov4-tiny model. After the model detects the instrument, it automatically publishes the "coordinate" topic. The content of the topic is five coordinate points, the upper left border coordinates, the upper right border coordinates, the lower left border coordinates, the lower right border coordinates, and the frame center coordinates. ROS TF2 is started to convert the coordinates based on the camera space into the coordinates based on base_link through steps S1 and S2. The mobile robot chassis adjusts its posture based on the five coordinate points, that is, the base_link coordinates are aligned with the five coordinate points in the x-axis direction; After the posture is adjusted, the semantic segmentation model is used to segment the instrument for reading. At the same time, the "threshold" topic is received, and the read data is compared with the threshold information of "threshold". If it is within the threshold range, the mobile robot will move to the next target point for inspection. If it is outside the threshold, the mobile robot will issue an alarm to the intelligent manufacturing system.

[0050] Embodiment 2: In order to improve the inspection efficiency of mobile robots and enhance the human-computer interaction experience, Baidu's self-developed large model Wenxin Yiyan is combined with the navigation control and computer vision detection of the ROS mobile robot, and unified scheduling is performed through Wenxin Yiyan.

[0051] like Fig. 9 , Fig.10 As shown, at 9:00 am, the mobile robot is started, and the keyboard control node roslaunchteleop_twist_keyboard teleop_twist_keyboard.py and the point cloud slam mapping node roslaunchcartographer_ros demo_revo_lds.launch are used to control the mobile robot to build a 3D salm map of the factory. rosrun map_server map_server -f my_map is used to save the map, and five target points are marked in the map, which are published to the system by creating the topic "total".

[0052] The mobile robot starts the autonomous navigation node rosrun nav .Launch and accepts the topic "total". It starts cruising based on steps S1, S2, S3, S4, and S5.

[0053] No fire, garbage, or people who stray into the workspace are found on the way to the first navigation point. After arriving at the first target point, the normal range of the whole day "550-600" is displayed next to the instrument. After passing steps 5 and S6, the reading is 556, and the mobile robot goes to the next target point.

[0054] When heading to the second target point, garbage is found through the target detection model in step S7, and the location of the point is released to the staff in the central control room, who then deal with it. When arriving at the second target point, the normal range for daytime is "120-360" and the normal range for nighttime is "80-160". If the number read in step S6 and step 7 is 450, an alarm is raised, the information is transmitted to the staff in the central control room, and the next target point is headed.

[0055] When heading to the third target point, the target detection model in step S7 finds that the worker has entered the workspace by mistake and raises the alarm, then the location of the point is released to the staff in the central control room, and the staff notifies the worker who has entered by mistake, which provides double protection. After reaching the second target point, the normal range for daytime is "200-400" and the normal range for nighttime is "300-500". The number read in step S6 and step 7 is 280, so it goes to the next target point.

[0056] When heading to the fourth target point, the target detection model in step S7 finds a fire and sounds an alarm, then the location of the point is released to the staff in the central control room, and the intelligent manufacturing system is connected to the network to cut off the power supply to prevent more serious dangers. After arriving at the second target point, the normal range for daytime is "150-400" and the normal range for nighttime is "300-460". The number read in steps S6 and S7 is 330, so it goes to the next target point.

[0057] No fire, garbage, or people who stray into the workspace are found on the way to the last navigation point. After reaching the first target point, the normal range of the whole day "90-140" is displayed next to the meter. After passing steps S5 and S6, the reading is 100, and the mobile robot returns to the starting point for automatic charging.

[0058] In the system of the present invention, the Jetson orin nano system board equipped with the ROS noetic version is connected to the Internet through a network card to call the Wenxin Yiyan model. The Jetson orin nano system board based on the ROS noetic is connected to the STM32F407 series single-chip microcomputer, the Yushu 4D radar, and the Intel D435 real-sensing camera through the serial port. The STM32F407 series single-chip microcomputer is connected to the GMR encoder motor through the encoding terminal and the encoding line, and the output shaft of the GMR encoder motor is connected to the Mecanum wheel. The above systems are all prior art and can be called as needed, so the specific content is not described in detail.

[0059] The present invention uses Jetson orin nano to perform SLAM_3D mapping of the smart factory through Yushu 4D laser radar under the decision scheduling based on Wenxin Yiyan model, and marks the SLAM map coordinates of the instruments, trash can positions, and charging pile positions that need to be inspected in the smart factory. The mobile robot navigates at multiple points through the parameter server communication of the ROS system, and transmits the motion posture of the mobile robot under path planning to the STM32F407 single-chip microcomputer through serial communication. The single-chip microcomputer controls the GMR encoding motor to make the mobile robot reach the target position. At the detection position in different areas, practical text prompts are used. After the mobile robot uses PaddleOCR to read through the Intel D435 real-sense camera, the Wenxin Yiyan model is analyzed to obtain the inspection strategy of the position. Under the premise of determining the inspection strategy, the mobile robot performs YOLOV4-Tiny target detection and PaddleSeg speech segmentation through the video stream input by the Intel D435 real-sense camera to obtain the correct detection result of the task point. At the same time, relying on the self-attention mechanism of the Wenxin Yiyan model, key information during the inspection is autonomously recorded in the industrial Internet of Things for human-computer interaction with staff.

[0060] Although the present invention is described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An efficient inspection method for mobile robots based on online decision-making of AI large models, characterized in that: The steps include: S1. Construct the motion joint coordinate system of the mobile robot and perform kinematic analysis; S2, construct the external sensor coordinate system of the mobile robot and calibrate it; S3, setting the underlying control communication mode of the mobile robot; S4, setting the odometer calibration mode of the mobile robot; S5. Determine the optimal moving speed and motion trajectory of the mobile robot based on the navigation algorithm; S6. Determine the YOLOV4-tiny model and semantic segmentation model based on the AI ​​big model, and execute the inspection strategy to complete the detection task.

2. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 1 is characterized in that: The step S1 constructs the motion joint coordinate system of the mobile robot as follows: S11, establish the left front wheel coordinate system front_left_wheel; S12, establishing a right front wheel coordinate system front_right_wheel; S13, establishing the left rear wheel coordinate system back_left_wheel; S14, establishing a right rear wheel coordinate system back_right_wheel; S15. Establish a coordinate system base_link with the mobile robot as the geometric center.

3. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 2 is characterized in that: The kinematic analysis of step S1 includes the following steps: S16. Given the speed of each wheel, find the speed of the mobile robot in the Cartesian coordinate system. The kinematic solution is: V5=(V1+V2+V3+V4) / 4; V6=(V3-V1+V2-V4) / 4; V7=(-V3-V1+V2+V4) / (2H+2W); S17. Given the velocity of the mobile robot in the Cartesian coordinate system, the velocity of each wheel is solved, and the inverse kinematic solution is: V1=V5-V6-V7*(H / 2+W / 2); V2=V5+V6+V7*(H / 2+W / 2); V3=V5+V6-V7*(H / 2+W / 2); V4=V5-V6+V7*(H / 2+W / 2); Among them, V1, V2, V3, and V4 represent the speeds of the left front wheel, right front wheel, left rear wheel, and right rear wheel of the mobile robot, respectively, in m / s; V5 and V6 represent the linear speed of the mobile robot, and V7 represents the angular speed of the mobile robot; in the Cartesian coordinate system, the front of the car is the X direction, the right is the Y direction, and the Z direction is perpendicular to the plane of the car; W and H are the wheelbase and track width of the mobile robot, respectively, in meters.

4. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 3 is characterized in that: The step S2 of constructing the external sensor coordinate system of the mobile robot and calibrating it includes the following steps: S21, establish camera coordinate system camera_link; S22, establish laser radar coordinate system laser_link; S23. In the ROS launch file, fix the original coordinates of the camera in the camer_link coordinate system, fix the original coordinates of the lidar in the laser_link coordinate system, start the ROS TF2 coordinate transformation, and convert the spatial coordinate information perceived by the camera and lidar into the information in the base_link coordinate system.

5. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 4 is characterized in that: The step S4 of setting the odometer calibration mode of the mobile robot includes the following steps: S41, start the ROS keyboard control node; S42, the mobile robot moves forward 1m in the X direction and checks whether the ROS odometer record is 1m; S43, the mobile robot moves forward 1m in the Y direction and checks whether the ROS odometer record is 1m; S44, the mobile robot rotates clockwise for one circle to check whether the odometer is 3.14 rad; S45. An error of 3% is allowed. If the error exceeds the allowable value, the parameters are corrected in the single chip microcomputer, and the process from step S41 to step S44 is repeated until it is within the allowable error range.

6. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 5 is characterized in that: The navigation algorithm of step S5 relies on global map, self-positioning, path planning, motion control, and environmental perception, among which: The global map is constructed using SLAM and relies on environmental perception; Self-positioning, using the amcl adaptive Monte Carlo algorithm, relies on environmental perception and global maps; Path planning, including local path planning and global path planning. The local path planning obtains the linear velocity in the X direction, the linear velocity in the Y direction and the angular velocity in the Z direction of the mobile robot, which are transmitted to the single-chip microcomputer through step S3 to complete motion control; Environmental perception, using the depth camera and lidar in step S2.

7. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 6 is characterized in that: The step S5 of determining the optimal moving speed and motion trajectory of the mobile robot based on the navigation algorithm includes the following steps: S51, the global map, self-positioning, path planning, motion control, and environmental perception in the navigation algorithm belong to the overall internal closed loop, and the input is the current X-direction linear velocity and Y-direction linear velocity of the mobile robot and the Z-direction angular velocity, laser radar information, depth camera information, and various coordinate information; the output is the X-direction linear velocity and Y-direction linear velocity and Z-direction angular velocity of the mobile robot at the next moment; S52, the single chip microcomputer obtains the linear velocity in the X direction and the Y direction and the angular velocity in the Z direction of the mobile robot, and then uses the kinematic inverse solution of step S1 to obtain the speed of each wheel, and quickly reaches the target speed through the incremental PID; The actual PID output is: u(k)=Kp*e(k-1)+Ki*e(k)+Kd*(e(k)-2e(k-1)+e(k-2))+u(k-1) Among them, u(k) represents the actual output of this time; u(k-1) represents the output of the previous time; △u(k) represents the output change; Kp represents the proportional coefficient; Ki represents the integral coefficient; Kd represents the differential coefficient; e(k-1) represents the error value between the previous target and the actual value; e(k) represents the error value between the current target and the actual value; e(k-2) represents the error value between the previous target and the actual value; S53, global path planning uses A* algorithm: The evaluation function is: f(n)=g(n)+h(n) Among them, g(n) represents the actual path cost from the starting point to any node n; h(n) represents the estimated cost from node n to the end point; S54, local path planning uses DWA algorithm; The DWA algorithm samples multiple sets of velocities in the velocity space (v, ω) and simulates the motion trajectory under the velocity. The evaluation function is used to select the optimal trajectory, which includes the following factors: Approaching the target point Target is evaluated by calculating the distance between the end point of the trajectory and the target point; Stay away from obstacles and use the distance to the nearest obstacle as the evaluation criterion; Path smoothness is evaluated by calculating the curvature or acceleration change of the trajectory; Score=α*DistanceToTarget+β*DistanceToObstacle+γ*Smoothness Among them, α, β, and γ all represent weight coefficients, which are used to adjust the importance of different factors.

8. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 7 is characterized in that: The AI ​​big model in step S6 reads the background information and strategy analysis plan from step S1 to step S5. Paddleocr reads the text information and transmits it to the AI ​​big model. The AI ​​big model uses the read information to execute the inspection strategy.

9. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 8 is characterized in that: The execution inspection strategy of step S6 includes the following steps: S61, based on steps S1 to S5, complete the construction of a conventional inspection AGV; S62. Use Python language to import the ERNIE Bot library of the AI ​​big model and set instructions in the AI ​​big model background settings; S63. When the AI ​​big model reads instructions through PaddleOCR, it uses Python language to extract key points, and then publishes topics through the topic of the ROS control system.

10. The mobile robot efficient inspection method based on AI large model online decision-making as claimed in claim 9 is characterized in that: The method further comprises the following subsequent steps: S7. The mobile robot uses the opencv library to capture images from the camera and publishes the topic name to the ROS system, including the following steps: S71. After the mobile robot is started, it automatically uses the opencv library to capture images from the camera and publishes them to the ROS system with the topic name "image". S72, after the mobile robot reaches the target position through step S5, it receives the image content of the "image" topic through the program based on the pytorch framework reasoning, and brings the image into the yolov4-tiny model. After the model detects the instrument, it automatically publishes the "coordinate" topic. The content of the topic is five coordinate points, the upper left border coordinates, the upper right border coordinates, the lower left border coordinates, the lower right border coordinates, and the frame center coordinates. ROS TF2 is started to convert the coordinates based on the camera space into the coordinates based on the base_link through steps S1 and S2. The mobile robot chassis adjusts its posture based on the five coordinate points, that is, the base_link coordinates are aligned with the five coordinate points in the x-axis direction; S73. After the posture is adjusted, the instrument is segmented using the semantic segmentation model for reading, and the "threshold" topic is received at the same time, and the read data is compared with the threshold information of "threshold": If it is within the threshold range, the mobile robot goes to the next target point for inspection; If it is outside the threshold range, the mobile robot will issue an alarm message to the system.

Images