Robot autonomous response decision-making system and method based on Internet of all things

Abstract

The invention discloses a robot autonomous response decision-making system and method based on everything interconnection, and belongs to the technical field of robots. The method comprises the steps that a robot is interconnected with a material management system of a factory, related information of a material order is obtained from the material management system, environment multi-source data is collected through Bluetooth, a microphone, a camera and multiple sensors, and the environment multi-source data is sent to the robot; storing the data in a corresponding database; extracting data from each database, respectively performing validity analysis, constructing a plurality of corresponding model libraries, and associating the model libraries to form a database chain large model; the robot determines a carrying task of a current material in combination with order verification information, and makes a decision-making scheme of material carrying by using a database chain large model; and in the process of executing the material carrying task, the carrying process of the robot, the material state and the environment change are monitored in real time, the task execution result and all data and feedback information in the operation process are sent to each model library, and related data information is updated.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a robot autonomous response decision-making system and method based on the Internet of Everything. Background Art

[0002] In today's industrial production field, with the rapid development of science and technology, the intelligent transformation of factories has become a general trend, and the rise of Internet of Everything technology has brought new opportunities and challenges to the efficient operation of factories. In the material handling process of factories, the traditional handling method mainly relies on manual operation, which is not only inefficient and costly, but also prone to errors and safety hazards.

[0003] With the continuous advancement of robot technology, robots are increasingly used in factory material handling. However, existing robot material handling systems often have some problems: on the one hand, although robots can improve handling efficiency to a certain extent, most of them operate independently and lack effective interconnection with other systems and equipment in the factory, which makes it impossible for robots to obtain accurate material order information in a timely manner and to fully understand the factory's production progress and demand changes; at the same time, due to the lack of interaction with other equipment, robots find it difficult to make optimal decisions when faced with a complex factory environment, resulting in unreasonable handling paths, waste of resources and other problems; on the other hand, the existing robot response decision-making mechanism is not intelligent and flexible enough. When faced with emergencies such as changes in material requirements, equipment failures, environmental changes, etc., robots often cannot make adjustments quickly, affecting the factory's production efficiency and stability. Summary of the invention

[0004] The purpose of the present invention is to provide a robot autonomous response decision-making system and method based on the Internet of Everything to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a robot autonomous response decision-making method based on the Internet of Everything, the method comprising: Step S100: The robot is connected to the material management system of the factory to obtain relevant information of the material order, and collects multi-source data of the surrounding environment through Bluetooth, microphone, camera and various sensors, and stores the obtained data in the corresponding database; Step S200: extracting data from each database and performing validity analysis respectively, including verifying material order information and analyzing and processing multi-source data, constructing multiple corresponding model libraries according to the process and results of validity analysis, and performing association operations on each model library, and finally constructing a database chain model; Step S300: The robot combines the order verification information to clarify the specific content and requirements of the current material handling task, and formulates a decision plan for material handling using the database chain model based on the task analysis results; Step S400: The robot performs the material handling task according to the decision plan, monitors the robot's handling process, the state of the material, and the changes in the environment in real time, and sends the task execution results and all data and feedback information during the operation to each model library to update relevant data information.

[0006] Furthermore, the step S100 includes: Step S101: The robot connects to the factory's material management system through TCP / IP to obtain material order information, including material number MID, material name MN, material type ML, material quantity MQ, material weight MW, delivery time DT, starting position coordinates (x s ,y s ,z s ) and the target position coordinates (x t ,y t ,z t ), where x s ,y s ,z s They represent the starting position coordinates in three-dimensional space, x t ,y t ,z t Respectively represent the target position coordinates in the three-dimensional space, build a material order database based on the acquired material information, and store the data in the database; Step S102: The robot connects to the sensor on the material handling tool via Bluetooth to obtain the status information of the handling tool in real time, including the remaining power E of the tool, the usage time T, and the measurement sequence W=[w1,w2,...,w n ] and the measurement sequence of maximum carrying capacity C=[c1,c2,...,c m ], where w n represents the load-bearing weight measured at the nth time, c m represents the maximum carrying capacity measured at the mth time, constructs a handling tool status database based on the acquired status information, and stores the data in the database; Step S103: The robot collects the original signal s(t) of the voice command through the microphone, which represents the voice signal at time t; and constructs a voice database to store the voice signal; Step S104: capturing image data in real time through a camera, including material images and environment images; and constructing an image database for storing image data; Step S105: The robot collects the material temperature T in real time through the built-in temperature sensor, humidity sensor and pressure sensor. s 、Humidity s And pressure data P s ,Build a sensor database to store the acquired sensor-related data.

[0007] Furthermore, the step S200 includes: Step S201: Perform integrity check, format check and logic check on the received material data. The integrity check is used to check whether the received data contains all the information of the material order. The format check is used to check whether the data meets the predetermined format requirements. The logic check is used to access the material management system to check whether the material number and material name exist in the material master data, whether the material quantity and delivery time are reasonable, and whether the starting location and the target location are different. According to the above process, a material order model library is constructed to process and analyze order data. Step S202: Processing multi-source data: For the load-bearing weight data, the sliding average filtering method is used to remove noise interference and obtain the final load-bearing weight W avg ; For the maximum load capacity data, take the average value of multiple measurements and get the maximum load weight as C w , combined with the remaining power E and the usage time data T, the state information set of the material handling tool Tols={W avg , E, T, C w}; According to the above process, a tool status model library is constructed to monitor the status information of each handling tool in real time, and a safety threshold is set. When each data exceeds the corresponding safety threshold, a risk warning signal is issued; Perform Fourier transform on the collected speech signal to extract the frequency and amplitude of the sound, expressed as NF={(f1,A1),(f2,A2),...,(f n ,A n )}, where f n represents the frequency of the nth sound, A n Indicates that f n Corresponding amplitude; set the sound amplitude threshold A t , the frequency exceeding the threshold is determined as an abnormal environmental sound a(t), which represents the abnormal environmental sound signal at time t; according to the above process, a speech recognition model library is constructed to identify abnormal environmental sounds in speech signals, and the type of abnormal sound is determined by extracting frequency components and amplitude data; The material image acquired by the camera is recognized, and the type, quantity and location information of the material in the image are identified by the YOLO detection algorithm; the environmental image acquired by the camera is converted into a grayscale image, and denoised and contrast enhanced. The segmentation threshold is set according to the grayscale distribution and contrast of the image, and the image grayscale is compared with the set threshold to generate a binary image; the area with a pixel value greater than the threshold is determined as an obstacle area, and the image is divided into a areas through contour detection, the number of pixels in the area is calculated to obtain the obstacle area, and the location information of the obstacle is obtained by calculating the centroid of the obstacle area; according to the above process, an image recognition model library is constructed to identify the material information in the acquired image and the obstacle information during the handling process; Filter the data from the temperature sensor, humidity sensor, and pressure sensor to remove noise, and determine whether the material is in a suitable environmental condition based on a preset threshold. Based on the above process, a sensor monitoring model library is constructed to monitor the environmental conditions of the material in real time; Step S203: perform association operations on each model library. The material quantity and weight information in the material order model library is associated with the tool status model library, so as to screen out tools with sufficient carrying capacity according to the order requirements; the voice recognition model library is associated with the image recognition model library. When the voice recognition model library detects abnormal environmental sounds, it identifies the abnormal sound intensity and transmits it to the image recognition model library, triggering the image recognition model library to make corresponding adjustments to the image; the sensor monitoring model library is associated with the environmental conditions of the material order execution time and method, and the material order task schedule is adjusted through the triggers in the database according to the environmental data and the suitable conditions of different materials. Through the association operations between the above model libraries, a large database chain model is constructed.

[0008] Furthermore, the step S300 includes: Step S301: construct a path planning map according to the obstacle location information provided by the image recognition model library in step S200; divide the factory map into n grids, each grid represents a node, and if there is an obstacle in the grid, it is marked as an inaccessible node; perform path planning through the ant colony algorithm, first set the number of ants m, the pheromone importance factor α, the heuristic function importance factor β, and the pheromone volatility coefficient ρ; set the initial pheromone concentration on all paths to a constant g0; Step S302: Each ant obtains the starting position from the material order database and selects the next node according to probability until it reaches the target position. According to the probability formula: ; Where k is an index used to identify each ant in the ant colony, k=1,2,...,m; i represents the node where the current ant is located, and j represents the node that the ant will choose next; represents the probability that ant k chooses node j from node i at time t, represents the pheromone concentration on the path from node i to node j at time t; represents the inverse of the distance from node i to node j, , where d ij represents the actual distance from node i to node j, Ak represents the node set selected by ant k in the next step, and the node set is the nodes without obstacles and within the ant's moving range; Step S303: When all ants have completed a path construction, the pheromone concentration on the path is updated according to the following formula: ; in represents the pheromone concentration on the path from node i to node j at time t+1, represents the total amount of pheromone released by all ants on the path from node i to node j in this iteration; represents the amount of pheromone released by ant k on the path from node i to node j in this iteration, , where Q represents the total amount of pheromones released by ants, L k represents the length of the path traversed by ant k in this iteration; Repeat step S301 and step S302 until the preset number of iterations is reached; after all iterations are completed, select the path with the highest pheromone concentration as the final material transport path output; Step S304: Obtain the remaining power E and the maximum carrying capacity C of the transport tool according to the tool status model library in step S202 w data, dynamically adjust the probability of ants choosing paths, and set , where E low is a preset power threshold. When the remaining power E is lower than E low By increasing The value of increases the probability of selecting a short path; at the same time, the frequency and corresponding amplitude data of the sound obtained in step S202 are used to dynamically adjust the pheromone volatility coefficient ρ. When the abnormal sound intensity exceeds the set threshold, the value of ρ is increased. By increasing the volatilization speed of the pheromone, the ant's stay time at the abnormal sound location is reduced. According to the adjustment formula: ρ n =ρ+K s ×(A n -A t ), where ρ n represents the adjusted pheromone volatility coefficient, K s Represents a preset adjustment factor, K s∈[0,0.5], which is used to control the increase in the pheromone volatility coefficient caused by abnormal sounds; combined with the obstacle position information obtained from the image recognition model library, if there is an obstacle between two nodes, the distance between them is set to infinity, so that the ants can automatically avoid the identified obstacles during path planning.

[0009] Furthermore, in step S400, the robot performs the material handling task according to the decided material handling path and material handling tools, and monitors the handling process in real time; when the robot transports the material to the target location and completes the placement, it interacts with the sensor at the target location to obtain material delivery feedback information to confirm whether the task is completed; the task execution results, including the handling time, path length, and the use of the material handling tools and all data and feedback information during the operation, including abnormal sounds, image recognition results, and sensor data, are collected and organized, and sent to each model database to update the relevant data information.

[0010] A robot autonomous response decision system based on the Internet of Everything, the system comprising a data acquisition module, a data analysis module, an autonomous decision module and a decision response module; The data acquisition module is used to connect the robot to the factory's material management system to obtain relevant information about material orders, and to collect multi-source data of the surrounding environment through Bluetooth, microphones, cameras and various sensors, and store the acquired data in the corresponding database; The data analysis module extracts data from each database and performs validity analysis, including verifying material order information and analyzing and processing multi-source data. According to the process and results of the validity analysis, multiple corresponding model libraries are constructed, and each model library is associated, and finally a database chain model is constructed; The autonomous decision-making module is used to combine the order verification information, clarify the specific content and requirements of the current material handling task, and formulate a decision-making plan for material handling using the database chain large model based on the task analysis results; The decision response module is used to execute material handling tasks according to the decision plan, monitor the robot's handling process, material status and environmental changes in real time, and send the task execution results and all data and feedback information during the operation process to each model library to update relevant data information.

[0011] The data acquisition module includes a material information unit and a multi-source data unit. The material information unit connects the robot to the material management system of the factory through TCP / IP to obtain material order information, and builds a material order database based on the obtained material information, and stores the data in the database; The multi-source data unit robot is connected to the sensor on the material handling tool via Bluetooth, obtains the status information of the handling tool in real time, builds a handling tool status database based on the acquired status information, and stores the data in the database; The robot collects the original signal of the voice command through the microphone and builds a voice database to store the voice signal; Capture image data in real time through the camera, including material images and environment images, and build an image database for storing image data; The robot uses built-in temperature sensors, humidity sensors and pressure sensors to collect the material temperature T in real time. s 、Humidity s And pressure data P s , and build a sensor database to store the acquired sensor-related data.

[0012] The data analysis module includes a data verification unit, a data processing unit and a data association unit; the data verification unit performs integrity verification, format verification and logic verification on the received material data inspection data, and builds a material order model library for processing and analyzing order data; The data preprocessing unit analyzes and processes the status information of the handling tool, the voice signal data, the image data, and the sensor data and constructs a model library respectively: The data association unit performs association operations on each model library, and finally forms a database chain large model.

[0013] The autonomous decision-making module includes a path planning module and a path decision module; the path planning module is used to plan the material handling path using the ant colony algorithm; the path decision module optimizes the handling path according to the data association of each model library through the database chain large model, and formulates the final decision plan for material handling; The decision response module is used to execute the decision plan and monitor the handling process in real time; collect all data and feedback information during the operation process, send them to each model database, and update relevant data information.

[0014] Compared with the prior art, the beneficial effects achieved by the present invention are: The present invention uses the Internet of Everything technology to closely connect the robot system with the factory's material management system, multiple sensors, handling tools, etc., to achieve real-time data sharing and interaction; The present invention constructs multiple small model libraries by performing validity analysis and modeling on material orders, tool status, voice signals, image data and sensor data, and performs association operations to form a complete database chain model; the database chain model is constructed based on this information, and material handling decision plans are independently formulated and executed, thereby improving the accuracy and efficiency of decision-making; at the same time, this modeling processing method not only improves the data processing efficiency, but also can discover potential problems and risks through association analysis, providing strong support for the robot's autonomous decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a schematic diagram of a method for autonomous response decision-making of robots based on the Internet of Everything; Figure 2 It is a method flow chart of a robot autonomous response decision-making method based on the Internet of Everything. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1-Figure 2 The present invention provides a technical solution: a robot autonomous response decision-making method based on the Internet of Everything, the method comprising: Step S100: The robot is connected to the material management system of the factory to obtain relevant information of the material order, and collects multi-source data of the surrounding environment through Bluetooth, microphone, camera and various sensors, and stores the obtained data in the corresponding database; Step S200: extracting data from each database and performing validity analysis respectively, including verifying material order information and analyzing and processing multi-source data, constructing multiple corresponding model libraries according to the process and results of validity analysis, and performing association operations on each model library, and finally constructing a database chain model; Step S300: The robot combines the order verification information to clarify the specific content and requirements of the current material handling task, and formulates a decision plan for material handling using the database chain model based on the task analysis results; Step S400: The robot performs the material handling task according to the decision plan, monitors the robot's handling process, the state of the material, and the changes in the environment in real time, and sends the task execution results and all data and feedback information during the operation to each model library to update relevant data information.

[0018] The step S100 includes: Step S101: The robot connects to the factory's material management system through TCP / IP to obtain material order information, including material number MID, material name MN, material type ML, material quantity MQ, material weight MW, delivery time DT, starting position coordinates (x s ,y s ,z s ) and the target position coordinates (x t ,y t ,z t ), where x s ,y s ,z s They represent the starting position coordinates in three-dimensional space, x t ,y t ,z t Respectively represent the target position coordinates in the three-dimensional space, build a material order database based on the acquired material information, and store the data in the database; Step S102: The robot connects to the sensor on the material handling tool via Bluetooth to obtain the status information of the handling tool in real time, including the remaining power E of the tool, the usage time T, and the measurement sequence W=[w1,w2,...,w n ] and the measurement sequence of maximum carrying capacity C=[c1,c2,...,c m ], where w n represents the load-bearing weight measured at the nth time, c m represents the maximum carrying capacity measured at the mth time, constructs a handling tool status database based on the acquired status information, and stores the data in the database; Step S103: The robot collects the original signal s(t) of the voice command through the microphone, which represents the voice signal at time t; and constructs a voice database to store the voice signal; Step S104: capturing image data in real time through a camera, including material images and environment images; and constructing an image database for storing image data; Step S105: The robot collects the material temperature T in real time through the built-in temperature sensor, humidity sensor and pressure sensor. s 、Humidity s And pressure data P s ,Build a sensor database to store the acquired sensor-related data.

[0019] The step S200 includes: Step S201: Perform integrity check, format check and logic check on the received material data. The integrity check is used to check whether the received data contains all the information of the material order. The format check is used to check whether the data meets the predetermined format requirements. The logic check is used to access the material management system to check whether the material number and material name exist in the material master data, whether the material quantity and delivery time are reasonable, and whether the starting location and the target location are different. According to the above process, a material order model library is constructed to process and analyze order data. Step S202: Processing multi-source data: For the load-bearing weight data, the sliding average filtering method is used to remove noise interference and obtain the final load-bearing weight W avg ; For the maximum load capacity data, take the average value of multiple measurements and get the maximum load weight as C w , combined with the remaining power E and the usage time data T, the state information set of the material handling tool Tols={W avg , E, T, C w}; According to the above process, a tool status model library is constructed to monitor the status information of each handling tool in real time, and a safety threshold is set. When each data exceeds the corresponding safety threshold, a risk warning signal is issued; Perform Fourier transform on the collected speech signal to extract the frequency and amplitude of the sound, expressed as NF={(f1,A1),(f2,A2),...,(f n ,A n )}, where f n represents the frequency of the nth sound, A n Indicates that f n Corresponding amplitude; set the sound amplitude threshold A t , the frequency exceeding the threshold is determined as an abnormal environmental sound a(t), which represents the abnormal environmental sound signal at time t; according to the above process, a speech recognition model library is constructed to identify abnormal environmental sounds in speech signals, and the type of abnormal sound is determined by extracting frequency components and amplitude data; The material image acquired by the camera is recognized, and the type, quantity and location information of the material in the image are identified by the YOLO detection algorithm; the environmental image acquired by the camera is converted into a grayscale image, and denoised and contrast enhanced. The segmentation threshold is set according to the grayscale distribution and contrast of the image, and the image grayscale is compared with the set threshold to generate a binary image; the area with a pixel value greater than the threshold is determined as an obstacle area, and the image is divided into a areas through contour detection, the number of pixels in the area is calculated to obtain the obstacle area, and the location information of the obstacle is obtained by calculating the centroid of the obstacle area; according to the above process, an image recognition model library is constructed to identify the material information in the acquired image and the obstacle information during the handling process; Filter the data from the temperature sensor, humidity sensor, and pressure sensor to remove noise, and determine whether the material is in a suitable environmental condition based on a preset threshold. Based on the above process, a sensor monitoring model library is constructed to monitor the environmental conditions of the material in real time; Step S203: perform association operations on each model library. The material quantity and weight information in the material order model library is associated with the tool status model library, so as to screen out tools with sufficient carrying capacity according to the order requirements; the voice recognition model library is associated with the image recognition model library. When the voice recognition model library detects abnormal environmental sounds, it identifies the abnormal sound intensity and transmits it to the image recognition model library, triggering the image recognition model library to make corresponding adjustments to the image; the sensor monitoring model library is associated with the environmental conditions of the material order execution time and method, and the material order task schedule is adjusted through the triggers in the database according to the environmental data and the suitable conditions of different materials. Through the association operations between the above model libraries, a large database chain model is constructed.

[0020] The step S300 includes: Step S301: construct a path planning map according to the obstacle location information provided by the image recognition model library in step S200; divide the factory map into n grids, each grid represents a node, and if there is an obstacle in the grid, it is marked as an inaccessible node; perform path planning through the ant colony algorithm, first set the number of ants m, the pheromone importance factor α, the heuristic function importance factor β, and the pheromone volatility coefficient ρ; set the initial pheromone concentration on all paths to a constant g0; Step S302: Each ant obtains the starting position from the material order database and selects the next node according to probability until it reaches the target position. According to the probability formula: ; Where k is an index used to identify each ant in the ant colony, k=1,2,...,m; i represents the node where the current ant is located, and j represents the node that the ant will choose next; represents the probability that ant k chooses node j from node i at time t, represents the pheromone concentration on the path from node i to node j at time t; represents the inverse of the distance from node i to node j, , where d ij represents the actual distance from node i to node j, Ak represents the node set selected by ant k in the next step, and the node set is the nodes without obstacles and within the ant's moving range; Step S303: When all ants have completed a path construction, the pheromone concentration on the path is updated according to the following formula: ; in represents the pheromone concentration on the path from node i to node j at time t+1, represents the total amount of pheromone released by all ants on the path from node i to node j in this iteration; represents the amount of pheromone released by ant k on the path from node i to node j in this iteration, , where Q represents the total amount of pheromones released by ants, L k represents the length of the path traversed by ant k in this iteration; Repeat step S301 and step S302 until the preset number of iterations is reached; after all iterations are completed, select the path with the highest pheromone concentration as the final material transport path output; Step S304: Obtain the remaining power E and the maximum carrying capacity C of the transport tool according to the tool status model library in step S202 w data, dynamically adjust the probability of ants choosing paths, and set , where E low is a preset power threshold. When the remaining power E is lower than E low By increasing The value of increases the probability of selecting a short path; at the same time, the frequency and corresponding amplitude data of the sound obtained in step S202 are used to dynamically adjust the pheromone volatility coefficient ρ. When the abnormal sound intensity exceeds the set threshold, the value of ρ is increased. By increasing the volatilization speed of the pheromone, the ant's stay time at the abnormal sound location is reduced. According to the adjustment formula: ρ n =ρ+K s ×(A n -A t ), where ρ n represents the adjusted pheromone volatility coefficient, K s Represents a preset adjustment factor, K s∈[0,0.5], which is used to control the increase in the pheromone volatility coefficient caused by abnormal sounds; combined with the obstacle position information obtained from the image recognition model library, if there is an obstacle between two nodes, the distance between them is set to infinity, so that the ants can automatically avoid the identified obstacles during path planning.

[0021] In step S400, the robot performs the material handling task according to the decided material handling path and material handling tools, and monitors the handling process in real time; when the robot transports the material to the target location and completes the placement, it interacts with the sensor at the target location to obtain material delivery feedback information to confirm whether the task is completed; the task execution results, including the handling time, path length, and the use of the material handling tools and all data and feedback information during the operation, including abnormal sounds, image recognition results, and sensor data, are collected and organized, and sent to each model database to update the relevant data information.

[0022] A robot autonomous response decision method based on the Internet of Everything, the system includes a data acquisition module, a data analysis module, an autonomous decision module and a decision response module; The data acquisition module is used to connect the robot to the factory's material management system to obtain relevant information about material orders, and to collect multi-source data of the surrounding environment through Bluetooth, microphones, cameras and various sensors, and store the acquired data in the corresponding database; The data analysis module extracts data from each database and performs validity analysis, including verifying material order information and analyzing and processing multi-source data. According to the process and results of the validity analysis, multiple corresponding model libraries are constructed, and each model library is associated, and finally a database chain model is constructed; The autonomous decision-making module is used to combine the order verification information, clarify the specific content and requirements of the current material handling task, and formulate a decision-making plan for material handling using the database chain large model based on the task analysis results; The decision response module is used to execute material handling tasks according to the decision plan, monitor the robot's handling process, material status and environmental changes in real time, and send the task execution results and all data and feedback information during the operation process to each model library to update relevant data information.

[0023] The data acquisition module includes a material information unit and a multi-source data unit. The material information unit connects the robot to the material management system of the factory through TCP / IP to obtain material order information, and builds a material order database based on the obtained material information, and stores the data in the database; The multi-source data unit robot is connected to the sensor on the material handling tool via Bluetooth, obtains the status information of the handling tool in real time, builds a handling tool status database based on the acquired status information, and stores the data in the database; The robot collects the original signal of the voice command through the microphone and builds a voice database to store the voice signal; Capture image data in real time through the camera, including material images and environment images, and build an image database for storing image data; The robot uses built-in temperature sensors, humidity sensors and pressure sensors to collect the material temperature T in real time. s 、Humidity s And pressure data P s , and build a sensor database to store the acquired sensor-related data.

[0024] The data analysis module includes a data verification unit, a data processing unit and a data association unit; the data verification unit performs integrity verification, format verification and logic verification on the received material data inspection data, and builds a material order model library for processing and analyzing order data; The data preprocessing unit analyzes and processes the status information of the handling tool, the voice signal data, the image data, and the sensor data and constructs a model library respectively: The data association unit performs association operations on each model library, and finally forms a database chain large model.

[0025] The autonomous decision-making module includes a path planning module and a path decision module; the path planning module is used to plan the material handling path using the ant colony algorithm; the path decision module optimizes the handling path according to the data association of each model library through the database chain large model, and formulates the final decision plan for material handling; The decision response module is used to execute the decision plan and monitor the handling process in real time; collect all data and feedback information during the operation process, send them to each model database, and update relevant data information.

[0026] Embodiment of the present invention: In the material handling process of a factory, step S100: the robot connects to the material management system of the factory via TCP / IP to obtain the following material order information: Material number MID = "M001"; Material name MN = "Part A"; Material type ML = "Metal Parts"; Material quantity MQ=10; Material weight MW = 1kg; Delivery time DT = "2024-12-31 14:00:00"; Starting position coordinates (x s , y s , z s )=(10,20,5); Target position coordinates (x t , y t , z t )=(30,40,10); The robot builds a material order database based on the acquired material information and stores the data in the database; The robot connects to the sensor on the material handling tool via Bluetooth and obtains the status information of the handling tool in real time as follows: remaining power E=80%; usage time T=2h; measurement sequence of load weight W=[9,10,11,9.5,10.5]kg; measurement sequence of maximum load capacity C=[20,21,20.5,19.5,20]kg; the robot builds a handling tool status database based on the status information obtained and stores the data in the database; The robot collects the original signal s(t) of the voice command through the microphone. It is a numerical sequence of length 100, representing the sound signal strength at different time points, and is stored in the voice database. The robot captures image data in real time through the camera, including material images and environment images, and stores them in the image database. The robot collects data through built-in temperature sensors, humidity sensors and pressure sensors: Temperature T s =25℃; humidity H s =50%; pressure data P s =101.3kPa; the robot builds a sensor database to store the acquired sensor-related data; Step S200: Build a material order model library: Check whether the received material order data contains all the above information, the result is complete, and the order information conforms to the predetermined JSON format; confirm that the material number "M001" and the material name "Part A" exist in the material master data, the material quantity and delivery time are reasonable, and the starting location and target location are different; Build a tool status model library: Use the sliding average filter method (set the window size to 3) to process the load weight data: For the load weight, calculate the sliding average to get the final load weight W avg =10kg; take the average value of the maximum load capacity data to get C w =20.2kg; Finally, the state information set Tols of the material handling tool is obtained: {W avg =10,E=20,T=50,C w=20.2}; Set safety thresholds: power safety threshold is 20%, usage time threshold is 100h, carrying weight safety threshold is 22, and maximum carrying capacity safety threshold is 25; Build a speech recognition model library, perform Fourier transform on the collected speech signal s(t), extract the main frequency components and their amplitudes, and obtain that the main frequency components of the sound are [500Hz, 1000Hz], and the corresponding amplitudes are [0.5, 0.8]. Set the sound amplitude threshold to 0.6, and identify the frequencies exceeding the threshold as abnormal environmental sounds; Build an image recognition model library. For material images, use the YOLO detection algorithm to identify the type, quantity, and location information of the materials in the image. The material type is identified as metal parts, the quantity is 10, and the location is at the starting position. For environmental images, convert them into grayscale images, perform denoising and contrast enhancement, and detect three obstacle areas in the generated binary images, and calculate their centroid positions and areas. Build a sensor monitoring model library, filter the temperature, humidity and pressure data using moving average filtering to remove noise, and set thresholds to determine whether the current data is abnormal; Each model library is associated with each other to build a database chain model. The material quantity and weight information in the material order model library is associated with the tool status model library to screen out tools with sufficient carrying capacity (C w >MQ*MW); The abnormal sound information in the speech recognition model library is transmitted to the image recognition model library. When an abnormal sound is detected, the image recognition model library can focus on detecting the corresponding area; The environmental conditions in the sensor monitoring model library meet the requirements; Step S300: construct a path planning map based on the obstacle location information provided by the image recognition model library, divide the factory environment into 100 grids (n=100), each grid represents a node, and if there is an obstacle (such as a machine or shelf) in the grid, it is marked as an inaccessible node; Set the number of ants m=50, the pheromone importance factor α=1, the heuristic function importance factor β=2, the pheromone volatility coefficient ρ=0.5, and set the initial pheromone concentration on all paths to a constant g0=1; each ant obtains the starting position (10,20,5) from the material order database, and selects the next node according to probability until it reaches the target position (30,40,10); calculates the probability of selecting a node according to the probability formula, and continuously updates the pheromone concentration on the path; repeats the path planning process until the set number of iterations is reached; after all iterations are completed, select the path with the highest pheromone concentration as the final material handling path output; obtain the remaining power E=80% and the maximum carrying capacity C of the handling tool according to the tool status model library w= 100kg data, dynamically adjust the probability of ants choosing a path, for example, set a preset power threshold E low =50%, when the remaining power E is lower than E low When adjusting (E low / E) to increase the probability of choosing a shorter path; use the speech recognition model library to obtain the frequency and corresponding amplitude data of the sound, and dynamically adjust the pheromone volatility coefficient ρ. For example, when the abnormal sound intensity exceeds the set threshold, increase the value of the pheromone volatility coefficient ρ. According to the formula, K is set to 0.5, then ρ n =0.5+0.5*(0.8-0.6)=0.6; Combined with the obstacle location information obtained from the image recognition model library, the distance between them is set to infinity, so that the ants can automatically avoid the identified obstacles during path planning; Step S400: Task execution. The robot performs the material handling task according to the decided material handling path and material handling tools. The robot carries 10 parts A from the starting position (10, 20, 5) to the target position (30, 40, 10) along the planned path, with a total weight of 10 kg. The robot monitors the handling process in real time, including the material handling status, the operating status of the handling tools, and environmental changes. When the robot carries the material to the target position and completes the placement, it interacts with the sensor at the target position to obtain material delivery feedback information, including the sensor confirming that the material quantity is correct, the position is accurate, and there is no damage. The robot confirms whether the task is completed, and collects and organizes the task execution results (including handling time, path length, and material handling tool usage) and all data and feedback information during the operation (including abnormal sounds, image recognition results, and sensor data); the robot sends the task execution results and feedback information to each model database for updating, such as updating the material status in the material order database to "delivered"; and updating the usage time and remaining power in the handling tool status database.

[0027] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A robot autonomous response decision-making method based on the Internet of Everything, characterized by: The method comprises: Step S100: The robot is connected to the material management system of the factory to obtain relevant information of the material order, and collects multi-source data of the surrounding environment through Bluetooth, microphone, camera and various sensors, and stores the obtained data in the corresponding database; Step S200: extracting data from each database and performing validity analysis respectively, including verifying material order information and analyzing and processing multi-source data, constructing multiple corresponding model libraries according to the process and results of validity analysis, and performing association operations on each model library, and finally constructing a database chain model; Step S300: The robot combines the order verification information to clarify the specific content and requirements of the current material handling task, and formulates a decision plan for material handling using the database chain model based on the task analysis results; Step S400: The robot performs the material handling task according to the decision plan, monitors the robot's handling process, the state of the material, and the changes in the environment in real time, and sends the task execution results and all data and feedback information during the operation to each model library to update relevant data information.

2. The robot autonomous response decision-making method based on the Internet of Everything according to claim 1 is characterized in that: The step S100 includes: Step S101: The robot connects to the factory's material management system through TCP / IP to obtain material order information, including material number MID, material name MN, material type ML, material quantity MQ, material weight MW, delivery time DT, starting position coordinates (x s ,y s ,z s ) and the target position coordinates (x t ,y t ,z t ), where x s ,y s ,z s They represent the starting position coordinates in three-dimensional space, x t ,y t ,z t Respectively represent the target position coordinates in the three-dimensional space, build a material order database based on the acquired material information, and store the data in the database; Step S102: The robot connects to the sensor on the material handling tool via Bluetooth to obtain the status information of the handling tool in real time, including the remaining power E of the tool, the usage time T, and the measurement sequence W=[w1,w2,...,w n ] and the measurement sequence of maximum carrying capacity C=[c1,c2,...,c m ], where w n represents the load-bearing weight measured at the nth time, c m represents the maximum carrying capacity measured at the mth time, constructs a handling tool status database based on the acquired status information, and stores the data in the database; Step S103: The robot collects the original signal s(t) of the voice command through the microphone, where s(t) represents the voice signal at time t; and constructs a voice database to store the voice signal; Step S104: capturing image data in real time through a camera, including material images and environment images; and constructing an image database for storing the image data; Step S105: The robot collects the material temperature T in real time through the built-in temperature sensor, humidity sensor and pressure sensor. s 、Humidity s And pressure data P s ,Build a sensor database to store the acquired sensor-related data.

3. The robot autonomous response decision-making method based on the Internet of Everything according to claim 1 is characterized in that: The step S200 includes: Step S201: Perform integrity check, format check and logic check on the received material data. The integrity check is used to check whether the received data contains all the information of the material order. The format check is used to check whether the data meets the predetermined format requirements. The logic check is used to access the material management system to check whether the material number and material name exist in the material master data, whether the material quantity and delivery time are reasonable, and whether the starting location and the target location are different. According to the above process, a material order model library is constructed to process and analyze order data. Step S202: Processing multi-source data: For the load-bearing weight data, the sliding average filtering method is used to remove noise interference and obtain the final load-bearing weight W avg ; For the maximum load capacity data, take the average value of multiple measurements and get the maximum load weight as C w , combined with the remaining power E and the usage time data T, the state information set of the material handling tool Tols={W avg , E, T, C w }; According to the above process, a tool status model library is constructed to monitor the status information of each handling tool in real time, and a safety threshold is set. When each data exceeds the corresponding safety threshold, a risk warning signal is issued; Perform Fourier transform on the collected speech signal to extract the frequency and amplitude of the sound, expressed as NF={(f1,A1),(f2,A2),...,(f n ,A n )}, where f n represents the frequency of the nth sound, A n Indicates that f n Corresponding amplitude; set the sound amplitude threshold A t , the frequency exceeding the threshold is determined as an abnormal environmental sound a(t), where a(t) represents the abnormal environmental sound signal at time t; according to the above process, a speech recognition model library is constructed to identify abnormal environmental sounds in speech signals, and the type of abnormal sound is determined by extracting frequency components and amplitude data; The material image acquired by the camera is recognized, and the type, quantity and location information of the material in the image are identified by the YOLO detection algorithm; the environmental image acquired by the camera is converted into a grayscale image, and denoised and contrast enhanced. The segmentation threshold is set according to the grayscale distribution and contrast of the image, and the image grayscale is compared with the set threshold to generate a binary image; the area with a pixel value greater than the threshold is determined as an obstacle area, and the image is divided into a areas through contour detection, the number of pixels in the area is calculated to obtain the obstacle area, and the location information of the obstacle is obtained by calculating the centroid of the obstacle area; according to the above process, an image recognition model library is constructed to identify the material information in the acquired image and the obstacle information during the handling process; Filter the data from the temperature sensor, humidity sensor, and pressure sensor to remove noise, and determine whether the material is in a suitable environmental condition based on a preset threshold. Based on the above process, a sensor monitoring model library is constructed to monitor the environmental conditions of the material in real time; Step S203: perform association operations on each model library. The material quantity and weight information in the material order model library is associated with the tool status model library, so as to screen out tools with sufficient carrying capacity according to the order requirements; the voice recognition model library is associated with the image recognition model library. When the voice recognition model library detects abnormal environmental sounds, it identifies the abnormal sound intensity and transmits it to the image recognition model library, triggering the image recognition model library to make corresponding adjustments to the image; the sensor monitoring model library is associated with the environmental conditions of the material order execution time and method, and the material order task schedule is adjusted through the triggers in the database according to the environmental data and the suitable conditions of different materials. Through the association operations between the above model libraries, a large database chain model is constructed.

4. The robot autonomous response decision-making method based on the Internet of Everything according to claim 1 is characterized in that: The step S300 includes: Step S301: construct a path planning map according to the obstacle location information provided by the image recognition model library in step S200; divide the factory map into n grids, each grid represents a node, and if there is an obstacle in the grid, it is marked as an inaccessible node; perform path planning through the ant colony algorithm, first set the number of ants m, the pheromone importance factor α, the heuristic function importance factor β, and the pheromone volatility coefficient ρ; set the initial pheromone concentration on all paths to a constant g0; Step S302: Each ant obtains the starting position from the material order database and selects the next node according to probability until it reaches the target position. According to the probability formula: ； Where k is an index used to identify each ant in the ant colony, k=1,2,...,m; i represents the node where the current ant is located, and j represents the node that the ant will choose next; represents the probability that ant k chooses node j from node i at time t, represents the pheromone concentration on the path from node i to node j at time t; represents the inverse of the distance from node i to node j, , where d ij represents the actual distance from node i to node j, Ak represents the node set selected by ant k in the next step, and the node set is the nodes without obstacles and within the ant's moving range; Step S303: When all ants have completed a path construction, the pheromone concentration on the path is updated according to the following formula: ； in represents the pheromone concentration on the path from node i to node j at time t+1, represents the total amount of pheromone released by all ants on the path from node i to node j in this iteration; represents the amount of pheromone released by ant k on the path from node i to node j in this iteration, , where Q represents the total amount of pheromones released by ants, L k represents the length of the path traversed by ant k in this iteration; Repeat step S301 and step S302 until the preset number of iterations is reached; after all iterations are completed, select the path with the highest pheromone concentration as the final material transport path output; Step S304: Obtain the remaining power E and the maximum carrying capacity C of the transport tool according to the tool status model library in step S202 w data, dynamically adjust the probability of ants choosing paths, and set , where E low is a preset power threshold. When the remaining power E is lower than E low By increasing The value of increases the probability of selecting a short path; at the same time, the frequency and corresponding amplitude data of the sound obtained in step S202 are used to dynamically adjust the pheromone volatility coefficient ρ. When the abnormal sound intensity exceeds the set threshold, the value of ρ is increased. By increasing the volatilization speed of the pheromone, the ant's stay time at the abnormal sound location is reduced. According to the adjustment formula: ρ n =ρ+K s ×(A n -A t ), where ρ n represents the adjusted pheromone volatility coefficient, K s Indicates a preset adjustment factor, K s ∈[0,0.5], which is used to control the increase in the pheromone volatility coefficient caused by abnormal sounds; combined with the obstacle position information obtained from the image recognition model library, if there is an obstacle between two nodes, the distance between them is set to infinity, so that the ants can automatically avoid the identified obstacles during path planning.

5. The robot autonomous response decision-making method based on the Internet of Everything according to claim 1 is characterized in that: In step S400, the robot performs the material handling task according to the decided material handling path and material handling tools, and monitors the handling process in real time; when the robot transports the material to the target location and completes the placement, it interacts with the sensor at the target location to obtain material delivery feedback information to confirm whether the task is completed; the task execution results, including the handling time, path length, and the use of the material handling tools and all data and feedback information during the operation, including abnormal sounds, image recognition results, and sensor data, are collected and organized, and sent to each model database to update the relevant data information.

6. A robot autonomous response decision-making system based on the Internet of Everything, characterized by: The system includes a data acquisition module, a data analysis module, an autonomous decision-making module and a decision response module; The data acquisition module is used to connect the robot to the factory's material management system to obtain relevant information about material orders, and to collect multi-source data of the surrounding environment through Bluetooth, microphones, cameras and various sensors, and store the acquired data in the corresponding database; The data analysis module extracts data from each database and performs validity analysis, including verifying material order information and analyzing and processing multi-source data. According to the process and results of the validity analysis, multiple corresponding model libraries are constructed, and each model library is associated, and finally a database chain model is constructed; The autonomous decision-making module is used to combine the order verification information, clarify the specific content and requirements of the current material handling task, and formulate a decision-making plan for material handling using the database chain large model based on the task analysis results; The decision response module is used to execute material handling tasks according to the decision plan, monitor the robot's handling process, material status and environmental changes in real time, and send the task execution results and all data and feedback information during the operation process to each model library to update relevant data information.

7. The robot autonomous response decision-making system based on the Internet of Everything according to claim 6 is characterized by: The data acquisition module includes a material information unit and a multi-source data unit. The material information unit connects the robot to the material management system of the factory through TCP / IP to obtain material order information, and builds a material order database based on the obtained material information, and stores the data in the database; The multi-source data unit robot is connected to the sensor on the material handling tool via Bluetooth, obtains the status information of the handling tool in real time, builds a handling tool status database based on the acquired status information, and stores the data in the database; The robot collects the original signal of the voice command through the microphone and builds a voice database to store the voice signal; Capture image data in real time through the camera, including material images and environment images, and build an image database for storing image data; The robot uses built-in temperature sensors, humidity sensors and pressure sensors to collect the material temperature T in real time. s 、Humidity s And pressure data P s , and build a sensor database to store the acquired sensor-related data.

8. The robot autonomous response decision-making system based on the Internet of Everything according to claim 6 is characterized by: The data analysis module includes a data verification unit, a data processing unit and a data association unit; The data verification unit performs integrity verification, format verification and logic verification on the received material data inspection data, and builds a material order model library for processing and analyzing order data; The data preprocessing unit analyzes and processes the status information of the handling tool, the voice signal data, the image data, and the sensor data and constructs a model library respectively: The data association unit performs association operations on each model library, and finally forms a database chain large model.

9. The robot autonomous response decision-making system based on the Internet of Everything according to claim 6, characterized in that: The autonomous decision-making module includes a path planning module and a path decision module; the path planning module is used to plan the material handling path using the ant colony algorithm; the path decision module optimizes the handling path according to the data association of each model library through the database chain model, and formulates the final material handling decision plan.

10. The robot autonomous response decision-making system based on the Internet of Everything according to claim 6, characterized in that: The decision response module is used to execute the decision plan and monitor the handling process in real time; collect all data and feedback information during the operation process, send them to each model database, and update relevant data information.

Images