Robot scheduling decision simulation agent platform

Abstract

The invention discloses a robot scheduling decision simulation agent platform, belongs to the field of robot scheduling, belongs to the field of advanced planning and scheduling, solves the problem of deep fusion of robot scheduling and production planning in industrial production, and solves the problem of intelligent robot scheduling based on production tasks in a full-automatic unmanned factory. A mechanical simulation analysis module and a large model module are fused to form an intelligent agent platform for intelligently distributing robot clusters and intelligently adjusting actions and postures of robots based on operation tasks.

Description

TECHNICAL FIELD

[0001] Robot scheduling decision simulation agent platform, SIAP, has wider and more dynamic application fields than traditional APS, especially suitable for scenarios that require high adaptability, real-time response and multi-agent collaboration. The main application fields include:

[0002] Scheduling, material requirement planning (MRP) and capacity optimization for production lines in the field of mechatronics manufacturing, robot scheduling field;

[0003] LLMs in SIAP can analyze massive production data and predict failures, while robot scheduling can command robotic arms, embodied robots (such as embodied robot driving AGV automatic guided vehicle), humanoid robots in real time. Simulation rehearsal can simulate the entire production line changes. For example: in an automobile factory, the system can use LLMs to generate an optimized production plan, schedule robots to assemble parts, and simulate supply chain disruption scenarios to minimize downtime, improve production efficiency, and support personalized customization;

[0004] Automated logistics and supply chain, the system can use large models to process natural language orders (such as voice input), schedule robot warehouse picking and transportation, and simulate peak traffic to avoid congestion;

[0005] Energy and utilities, the integrated system can use large models to predict energy demand, schedule maintenance robots (such as inspection robots), and simulate power grid load changes. For example, in a wind farm, SIAP can schedule robots to inspect turbines, optimize maintenance plans based on weather data, and simulate "storm coming" energy distribution scenarios, which can help achieve sustainable energy management and reduce energy waste;

[0006] Transportation and smart cities, large models process traffic data to generate routing plans, schedule autonomous vehicles or drones, and simulate traffic congestion or accident responses. BACKGROUND

[0007] Traditional APS (Advanced Planning and Scheduling) systems are widely used in manufacturing and supply chain management, but they have inherent limitations that limit their effectiveness in complex and dynamic environments. These limitations include: insufficient real-time processing capabilities, traditional APS systems often rely on static data and predefined models, which cannot effectively respond to real-time changes such as sudden demand fluctuations or supply chain disruptions, resulting in slow system response and inability to dynamically adjust; integration and compatibility issues, traditional APS systems are often designed as isolated modules, making it difficult to seamlessly integrate with other enterprise systems such as ERP or MES, especially in a distributed environment, traditional APS has inherent limitations in modeling distributed context, unable to capture important business dynamics, leading to data silos and collaboration difficulties; weak handling of uncertainty and complexity, traditional APS is good at solving well-defined optimization problems, but it performs poorly when faced with uncertain factors such as market fluctuations or supply chain risks, it lacks the ability to adapt to new situations and cannot handle ambiguous or emerging problems like modern AI systems, resulting in inaccurate or ineffective planning; limitations of optimization algorithms and modeling, the system has quantitative challenges in defining and solving optimization problems, such as being unable to effectively handle multi-objective optimization or large-scale data, making it difficult to achieve efficient multi-task coordination;

[0008] Traditional APS systems play a crucial role in automotive manufacturing plants, optimizing production planning, resource allocation, and supply chain coordination. However, in the complex environment of the automotive industry, such as highly automated assembly lines, fluctuating demand, and global supply chains, these systems expose several inherent shortcomings. The problem lies in their static, rule-based architecture, which cannot fully adapt to the dynamics and uncertainties of modern automotive manufacturing. In future scenarios, such as assembly line operations, component supply, and robot scheduling, specific issues include: inadequate real-time response and dynamic adjustment capabilities, traditional APS relies on batch processing and pre-set models, unable to handle sudden events such as sudden order changes, equipment failures, or supply chain disruptions, resulting in production plan lag, unable to optimize in real-time, while automotive assembly lines require continuous, high-speed production, if components are delayed (such as the chip shortage event), APS is difficult to quickly reschedule, causing production line to stop, for example, during the COVID-19 period, many automobile enterprises faced supply chain disruptions, traditional APS cannot adjust in real-time, leading to inventory accumulation or idle capacity, for example, certain regions' tariff policies affect market channels, and production manufacturers cannot adjust in time based on traditional APS systems; integration and data compatibility issues, traditional APS often acts as an isolated system, difficult to seamlessly integrate with other factory systems (such as MES-Manufacturing Execution System, ERP or IoT sensors), especially in a multi-supplier environment; performance in automotive factories, automotive manufacturing involves numerous subsystems such as robot arms, AGVs (automatic guided vehicles) and humanoid robot scheduling, traditional APS is difficult to integrate real-time data (such as robot position or sensor feedback), causing data silos, resulting in low collaboration efficiency, unable to achieve end-to-end visibility, especially after the introduction of embodied robots or humanoid robots, tighter integration is needed to coordinate human-robot hybrid environments; traditional APS is weak in handling uncertainty and complex scenarios, traditional APS systems are good at handling deterministic optimization, but lack robustness for uncertain factors (such as market demand fluctuations or external risks), unable to effectively simulate multi-variable scenarios; the demand in the automotive industry is highly volatile (such as electric vehicle transformation or seasonal sales peaks), traditional APS is difficult to predict and adapt.For example, when switching vehicle models (e.g., from a gas-powered car to an EV), the system cannot quickly reconfigure the assembly line, ignoring variables such as labor shortages or robot malfunctions, any delays can amplify into systemic issues, causing lower planning accuracy, leading to excess inventory or shortages, increased costs; limitations of optimization algorithms and scalability, traditional APS algorithms (e.g., linear programming) are inefficient in handling large-scale, multi-objective optimization, cannot be extended to complex networks, automotive factories often involve thousands of components and multiple production lines, traditional APS cannot achieve multi-robot scheduling (e.g., body-mounted robots handling or humanoid robots assembling), let alone fine-grained scheduling, limiting the potential of flexible manufacturing, in high-load scenarios, the calculation time is too long to support real-time decision-making; traditional APS systems lack in adapting to emerging technologies and sustainability, lack built-in support for AI, IoT, or sustainability factors, cannot integrate modern industrial trends, as electrification and automation advance (e.g., introducing robots), APS struggles to integrate AI predictions or environmental constraints (e.g., carbon emission optimization, resource consumption optimization), cannot support predictive maintenance or green manufacturing. Overall, the limitations of traditional APS are particularly pronounced in automotive manufacturing, as the industry emphasizes speed, precision, and global collaboration. Studies show that the limitations of traditional APS result in an average capacity utilization rate of only 70-80%, if body-mounted or humanoid robots are introduced, these problems will be further amplified, requiring a more advanced scheduling framework.

[0009] If large models are integrated into APS systems, it is essentially to equip this highly data and algorithm dependent system with a "brain" or "intelligent co-pilot" that can understand and efficiently collaborate with humans. This is expected to transform APS from a complex computational tool into a more intelligent, adaptable, and user-friendly partner. The fundamental advantage is that large models bridge the gap between complex systems, algorithms, and human planners. Large models are good at understanding natural language, processing unstructured information, and generating explanations that align with human thought, thus creating a revolutionary human-machine interaction experience. First, natural language interaction is the most direct and impactful advantage. The planner's operation mode can be transformed from "clicking on a cumbersome menu" to "talking to the system." In the past, the planner needed to manually search for a specific order, navigate to all its associated items, run a simulation, and then interpret a bunch of raw numerical results. In the future, simply input or speak instructions, such as: "Help me find all delayed orders for customer X, and what are the main bottlenecks? Simulate advancing the arrival time of raw materials by 3 days, and tell me the new estimated completion date." Second, processing unstructured data, traditional APS relies heavily on structured data (such as BOM, process route, inventory quantity). However, real-world decision-making often requires a large amount of unstructured information, which is where large models come in. In a typical scenario, the system can automatically read relevant news about customs policies and understand equipment maintenance logs to more accurately predict downtime risks. Third, enhanced simulation and scenario division, large models can help planners more creatively build and evaluate various "possibilities." In the past, planners needed to manually modify ten parameters to build a simulation scenario. In the future, ask an open-ended question, such as: "For our highest priority product, what methods can we use to alleviate the impact of chip shortages? Help me compare the costs and risks of various solutions." Large models can assist in building these scenarios, instructing the APS core engine to run, and finally summarizing the results in a clear comparison report. Finally, automated anomaly detection and explanation, APS systems can flag anomalies, but often cannot explain "why" in a way that humans can easily understand. In the past, when a "material shortage" alert for material number MS123 appeared, the planner needed to manually trace the demand source, check the in-transit materials, and review the production plan to locate the problem. In the future, the system not only pops up an alert, but also provides a summary: "Alert: Material MS123 has a potential shortage in week 32, the main reason is that order AB's demand unexpectedly increased by 30%. In summary, the application of large models in APS systems is promising and will greatly enhance the capabilities of human planners. Large models will make the system more user-friendly, more business-oriented, and more forward-looking, thus transforming planners from "data processors" to true "strategic decision-makers."

[0010] The large language model is very good at understanding and generating language, but it sometimes fabricates facts out of thin air, or is not good enough for scenarios that require accurate and real-time information. The knowledge graph and time series data fusion method in the invention can make up for the shortcomings of excessive divergence or excessive reliance on training data, thereby improving factual accuracy, reducing illusions, enhancing the real-time nature of knowledge, improving the explainability and traceability of answers, achieving more complex reasoning capabilities, providing a reliable, dynamic and structured industrial decision-making brain, making large models not only good at speaking, but also knowledgeable, accurate and credible. It is a key step towards more powerful and reliable industrial intelligent decision-making. The existing fine-tuning is based on a general large model (base model) that has been trained, and then it is retrained with a specific size of data set. The process is to adapt the model to new tasks in a specific application field. By training with text data in a specific field (such as professional papers, documents), the model learns the professional terminology, writing style and implicit knowledge of this field. It is a process of internalizing knowledge. Although fine-tuning on high-quality and accurate data sets can guide the model to reduce factual errors and generate more expected answers, fine-tuning the model into a specific role or personality customizes the model's behavior pattern. By learning specific domain cases and files, the model gradually masters the professional terminology, workflow and way of thinking in a specific field, and the reasoning ability of the large model is improved. However, for the various complex and fuzzy relationships existing in specific business fields, the fine-tuning efficiency is low, and in the case of dynamic changes in application scenarios and resources, the fine-tuning effect is obviously insufficient.

[0011] Embodied robots (typically refers to intelligent robots with physical bodies and the ability to interact with the environment) and humanoid robots (robots with a form similar to humans) have very broad application prospects in industrial scenarios, especially in the context of automation and intelligent transformation. With the advancement of AI, sensors and mechanical technology, these robots are moving from the laboratory to the actual production environment, helping enterprises cope with labor shortages, improve efficiency and safety. Embodied robots emphasize real-time interaction with the environment and are often used for dynamic and unstructured tasks. For example, robots equipped with AI vision and grasping systems can handle irregular object handling or complex assembly, making them suitable for applications in factory internal logistics and warehousing. Humanoid robots simulate human form, with the ability to walk on two legs, multi-joint arms and fine manipulation, making them suitable for tasks that require human-like flexibility, such as assembly lines. In the future, humanoid robots may replace repetitive and high-intensity human work in manufacturing, driving production efficiency, and automotive OEMs are a typical scenario for robot applications, as assembly lines are highly standardized and repetitive. For example, robots replace human labor to transport parts, embodied robots use AGVs or wheeled, tracked devices to transport parts, and robots use AI path planning to flow between warehouses or factories. Humanoid robots use dual-arm grasping and balanced walking to perform tasks such as screw tightening, part assembly or quality inspection on assembly lines, mimicking human actions to pick up and install components, or coexisting as collaborative robots with humans to provide assistance, and may completely replace some positions, such as engine assembly or interior installation. The advantage is high flexibility, which can quickly switch between vehicle production lines. For example, in electric vehicle battery assembly, humanoid robots can handle fine operations and reduce defects caused by human fatigue. Robots have broad application prospects in manufacturing, but the problem is how to schedule these robots? Scheduling is the key to ensuring efficient collaboration among multiple robots. SUMMARY

[0012] To solve the problem of APS system and robot collaborative scheduling, a robot scheduling decision simulation agent platform, SIAP, is designed, which realizes automatic and intelligent advanced planning and scheduling by designing a large model LLMs and AI model set, and realizes the function of large model, i.e. large language model, through the LLMs module of SIAP;

[0013] Due to the differences in actual resources and production rules of each factory, different enterprises in the mechatronics manufacturing industry also have great differences in equipment types and production processes. Even if they belong to the same automobile manufacturing enterprise, there are differences in the types and quantities of equipment tools. A method for LLMs to understand production resources and rules is designed, which maps the input production resources into high-level structured data, reflects the resource rule relationship through the association relationship of the containing layer and the sequence layer resource nodes, and reflects the number of resource nodes in each layer. Each node is given the attributes of function, performance (or effect), purpose, and association with production progress. This method is implemented through the ODMG module of SIAP. The ODMG data structure defines ordered nodes and unordered nodes to reflect the resource classification relationship, quantity relationship, and rule relationship. The fixed classification and rule relationship is reflected by defining ordered nodes, the containing and sequence relationship of equipment is reflected by the equipment data in the ODMG data structure, and the purpose, function, and performance in the attributes reflect the equipment classification and corresponding quantity relationship. The process data in the ODMG data structure reflects the process path and rules through the containing relationship and sequence relationship of the process. The conditional and changeable characteristic relationship of resource classification and rules is reflected by defining unordered nodes, and then the large model LLMs is further implemented in a training manner to define the classification and rule relationship based on specific scenarios.

[0014] To realize intelligent scheduling and dispatching of large models, a large model real-time inference scheduling decision method is designed. Highly structured production progress data is obtained by highly integrating production progress time series data and knowledge graph data, which is achieved by the fusion of the TSDB module and the ODMG module of SIAP. The TSDB module obtains the production progress data of each time node of the manufacturing enterprise, and the TSDB module is associated with the ODMG module to obtain highly structured progress data of production resources. Finally, the TSDB data and ODMG data are fused and converted into tensor data, and the large model LLMs fuses this tensor data and trains and reasons in a highly microscopic way of production resource progress data.

[0015] To further realize the accuracy of LLMs training and inference, an enhanced training and inference method is designed. In the first step, a high-level planning and scheduling quantification method and algorithm are designed based on the characteristics of the ODMG data structure, or the existing high-level planning and scheduling algorithm is optimized and extended based on the resource logical relationship of the ODMG data structure to achieve a highly structured high-level planning and scheduling quantification method and algorithm. The high-level planning and scheduling quantification method and algorithm cover problem analysis, solution ideas, index parameters, and mathematical algorithms, and are stored in any one of the following forms: text, abstract syntax tree (based on tree structure), markup language (XML text), and reverse polish (suffix notation). The module for implementing the first step method is the auxiliary enhanced input module, namely AAS. The AAS module mainly includes a resource planning method and algorithm model specially designed based on the production resource situation and business demand of a manufacturing enterprise, and supports storage of the data in the form of a text file. The robot scheduling pre-training module and the production scheduling pre-training module in the present application are examples of the pre-training module. In the second step, when a complex calculation model is involved, the algorithm model in the first step needs to be designed into an algorithm program to be called by LLMs in the form of an algorithm tool program module. The module for implementing the second step method is the algorithm tool module, namely ATP (or algorithm tool enhanced module), and each algorithm tool function is named, i.e., the name is defined, and a function description, i.e., a description written in natural language, is added to explain what the function does. The description corresponds to the problem analysis and solution ideas covered in the first step high-level planning and scheduling quantification method and algorithm. This is the most critical step because the model will determine when to call it based on this description. At the same time, it is necessary to define which parameters need to be input for the algorithm tool function, with each parameter having a parameter name, parameter type, parameter description (explaining what the parameter represents), and whether it is required (indicating whether the parameter is mandatory or optional). In the third step, training and calling, the association between problem analysis and algorithm program modules is established, i.e., the large model LLMs learns what problem to call what algorithm tool. The calling module of SIAP realizes the calling and interaction functions in the third step, including natural language interaction with schedulers and natural language association interaction with ERP systems. In addition to the algorithm program designed in the first step of the enhanced training and inference method, the algorithm tool module ATP also has algorithm programs directly designed and developed. Algorithm tools refer to algorithm tools called by LLMs to enhance the inference and calculation ability of LLMs. Algorithm tool data refers to data describing algorithm tools, including algorithm tool names, what situations require calling the algorithm tool, how to call it, and what parameters need to be input.

[0016] To solve the problem of the disconnection between the APS system and the production and manufacturing enterprise multi-system, and to adapt to the future application prospect of robots (embodied robots and humanoid robots) in manufacturing factories, a production scheduling system is designed to schedule the mechanical arms and robots of the production line. Through the robot scheduling module (MPS) of SIAP, the MPS module is associated with the API interface of the mechanical arms and robots, and the production scheduling plan based on SIAP directly schedules the mechanical arms and robots, realizes direct scheduling and scheduling, and divides the cluster scheduling of robots into groups according to the purpose (work type), function, performance, and quantity, i.e., based on the data structure division of the ODMG module, and associates the time sequence progress data of the TSDB module. Through the LLMs module of SIAP, the training and reasoning are carried out, and in an enhanced manner, i.e., associated with AAS and ATP modules, combined with LLMs reasoning and algorithm tool calling, the quantitative scheduling of robots and mechanical arms is realized.

[0017] To solve the problem of lack of scheduling rehearsal and result simulation in traditional advanced planning and scheduling systems, resulting in high uncertainty in the consequences of planning and scheduling, i.e., decision-making presents high risk, a scheduling and dispatching rehearsal method is designed, and a simulation and rehearsal module (VGM module) is implemented. The VGM module generates real-time simulation animation and scheduling and dispatching rehearsal simulation. In addition to real-time simulation of production progress, it also simulates all consequences brought by the unexecuted scheduling scheme after execution, and demonstrates the production impact process and final consequences in the form of simulation animation video. Animation generation uses the video generation model (such as Diffusion model) in the VGM module in SIAP as the implementation method, and the video generated by the Diffusion model is used to simulate device operation and to rehearse robot scheduling or production process. The steps are as follows: first, training and fine-tuning, collecting real factory videos, such as recording real device normal working videos with a camera, including multi-angle, different speed and condition fragments, training the Diffusion model, using the collected videos as a data set, the model learns the motion pattern of the device, and then fine-tuning the pre-trained model; second, inference and animation generation, input prompt, or through LLMs generated scheduling plan, the model generates a segment of animation video, simulates device work, adds simulation elements such as parameter control speed, adds virtual failure as a rehearsal tool, and generates robot scheduling simulation video. For example, the scheduling and dispatching scheme is "scheduling 5 robots to assemble cars", and the VGM module generates a video of the execution of this scheduling scheme, i.e., the model outputs a video showing the process of 5 robots assembling cars, and engineers or schedulers can foresee the scenes in the execution of the scheduling plan through the video;

[0018] To solve the problem of training and reasoning difficulty caused by different enterprise resource conditions, a method for understanding resource data is designed, and is realized through the resource input module of SIAP. The enterprise production related resources are divided into tangible resources and intangible resources, and SIAP is constructed in the form of application software. On the basis of this software, the enterprise resource framework, i.e. the resource input module, is simulated and constructed. Based on the enterprise resource framework, the classification and quantitative description of tangible resources, the classification and quantitative description of process technology are added according to the actual situation of the enterprise. Tangible resources include equipment tool resources (equipment resources) and labor resources. Labor resources are divided into real people and robots, and robots include embodied robots and humanoid robots. Equipment tools (referred to as equipment) resources are further subdivided into multi-level nested relationships based on function, purpose and performance. At the same time, the system combination of the equipment includes the relationship in the form of multi-level nested relationships, for example, A tool (tool system or tool set) contains a, b, c tools. If A belongs to the first level, a, b, c belong to the second level. a, b, c are further divided into layers. a, b, c tools are further divided according to function, purpose and performance. Each level of classification of equipment tool resources has a quantity description. Performance is the quantification of a certain equipment in the implementation of a number of subdivided targets within the function category in the specified purpose. Function is the category of actions that a certain equipment can achieve. Purpose is the application range of the equipment. One equipment can contain one or more functions, one function corresponds to one or more purposes, and one purpose corresponds to one or more performances. Labor resources are subdivided into multi-level nested relationships based on responsibility and skill. Each level of classification of labor resources has a quantity description. Intangible resources refer to process technology resources, which are intangible knowledge resources. They need to be realized by equipment tool resources or through the cooperation of labor resources and equipment resources. Process technology resource classification and quantitative description method: process is divided into multi-level nested relationships, for example, A process (process group or process set) contains a, b, c processes. If A belongs to the first level, a, b, c belong to the second level. a, b, c processes are further divided according to function, purpose and effect. Effect is the quantification of a certain process in the implementation of a certain function target in the specified purpose. Function is the category of actions. Purpose is the constraint on the application range. One process can contain one or more functions, one function corresponds to one or more purposes, and one purpose corresponds to one or more effects. Process is the smallest unit in the process nested relationship. One or more processes constitute a minimum level of process. The minimum level is the most microscopic level in the process nested relationship.

[0019] To solve the problem of large models understanding enterprise resources and rules, a method for understanding resources and rules is designed and implemented through the ODMG module. ODMG is a data structure that divides the association relationship of each resource node according to the inclusion layer and sequence relationship, and assigns attributes to the corresponding progress data. The ODMG data structure defines ordered nodes and unordered nodes to reflect the resource classification relationship, quantity relationship, and rule relationship. The ordered nodes are defined to reflect the fixed classification and rule relationship, and the equipment data in the ODMG data structure is used to reflect the inclusion and sequence relationship of the equipment, as well as the purpose, function, and performance of the attributes to reflect the classification and corresponding quantity relationship of the equipment. The process data in the ODMG data structure reflects the process path and rules through the inclusion relationship and sequence relationship of the process. The unordered nodes are defined to reflect the conditional and changeable characteristics of the resource classification and rule relationship, and then the large model LLMs are used to further implement the classification and rule relationship based on specific scenarios through training. The ODMG module mainly includes knowledge graph data based on the ODMG data structure, including resource input data and algorithm tool data, which are used for LLMs training and reasoning.

[0020] The TSDB module mainly includes real-time progress data and historical progress data based on the TSDB time series data structure, such as the progress data of each process flow in a car manufacturing plant or the progress data of each device. The ODMG module data is used for LLMs training and reasoning.

[0021] The knowledge base module, also known as the expert knowledge base or expert library, mainly includes external professional data for training and reasoning. In the training or reasoning stage of the large model, the knowledge base provides external knowledge, including training purposes and reasoning purposes. During training, the knowledge base provides data to fine-tune the model and help the model learn specific patterns. During reasoning, the large model queries the knowledge base to enhance the accuracy of reasoning during runtime.

[0022] The perception module perceives and understands the environment and user input, which may include text understanding, image recognition, and voice recognition. In addition, it includes the collection and acquisition of news policy information. Specifically, it obtains the latest news policy data from the Internet. The perception module needs data preprocessing and cleaning to remove HTML tags and irrelevant information, unify the format, and extract key information from the original text. The LLMs in SIAP use natural language processing (NLP) techniques to analyze semantic information, store the obtained information in a database or knowledge base, and establish an index for quick retrieval and use. The update mechanism ensures regular updates to ensure that the latest information is obtained.

[0023] The memory module stores and retrieves information, knowledge, and learned experiences from past interactions. This can be short-term memory (for current conversations) and long-term memory (for persistent knowledge).

[0024] Action modules, executing procurement plans, inventory plans, order delivery plans, production scheduling plans; performing actions or using external tools to interact with their environment or complete specific tasks, calling APIs, executing code, controlling robotic arms and robot software or hardware;

[0025] Mechanical simulation and analysis modules, providing robot pose and weight mechanics simulation analysis, motion flow analysis, providing support for robot environment perception and interaction, motion control and collaboration.

[0026] The features of the robot scheduling decision simulation intelligent agent platform, SIAP, are as follows: mainly including a large language model, LLMs, module, a resource input module, an ODMG module, a TSDB module, an algorithm tool module, ATP, an auxiliary augmented input module, AAS, a simulation and preview module, VGM, a robot scheduling module, MPS, a knowledge base module, a perception module, a memory module, an action module, and a mechanical simulation and analysis module; the function of a large model, i.e., a large language model, is realized through the LLMs module of the SIAP; the method for the LLMs module of the SIAP to understand production resources and rules is realized by mapping the input production resources into high-level structured data, reflecting the resource rule relationship through the association relationship of the resource nodes in the layer and sequence layer, reflecting the number of resource nodes in each layer, and assigning each node with functions, performance (or effect), purposes, and attribute information associated with production progress; the method is realized through the ODMG module of the SIAP; the TSDB module is designed to reflect real-time progress data and historical progress data of time series data structure, and the TSDB data is associated with the ODMG to realize the fusion and tensorization of high-level structured resource data and time series progress data for large model training and inference; the method of designing a production scheduling system to schedule production line manipulators and robots is realized through the robot scheduling module, MPS, of the SIAP, the MPS module is associated with the API interface of the manipulator and robot, and the manipulator and robot are directly scheduled based on the production scheduling plan of the SIAP; the APS scheduling system is integrated with the large model LLMs and the AI model set to realize automatic and intelligent high-level planning and scheduling, and the robot posture and action flow are accurately controlled based on environmental perception; the features of the robot are as follows: first, the API interface is open to the LLMs scheduling instruction system; second, the LLMs have obtained the mechanical structure simulation and analysis data of the robot, including all possible action and posture data of the robot, and mechanical analysis data of picking up or carrying heavy objects in all postures; third, the data of picking up and carrying different volume and weight objects in all postures and actions are trained by machine learning to obtain all relative geometric angles S between the two limb short sections connected by each active joint of the robot, S={S1A={S1a1, S1a2, S1a3,...}, S2A={S2a1, S2a2, S2a3,...},...}, S1A is all actions of node S1, S2A is all actions of node S2, and so on; all geometric angles 0 of the force ground contacted by the most fundamental force support point or surface (such as the foot of a humanoid robot) are obtained, and the geometric angles H of the force ground and the horizontal plane are obtained.} from S, O, H, respectively, select one geometric angle to combine, the geometric angle selected by S refers to the combination of the angle of each joint connected to the short limb, machine learning trains all the combined geometric angles and the positional relationship between the target object, infers the feasible relative position, and then based on the relative position relationship, judges the classification, volume and weight of the target object based on deep learning, combines with mechanical structure simulation analysis to predict the best geometric angle combination, and finally predicts the change of each joint geometric angle to achieve the optimal final geometric angle combination target. From the current geometric angle of the robot to the target geometric angle, its adjustment process has a continuous number of geometric angle changes, and according to the principle of force point from root to tip, the robot joints are adjusted step by step to realize the geometric angle change of each short limb (for example, the adjustment sequence of humanoid robot is foot, ankle, lower leg, upper leg, upper body, upper arm, lower arm, wrist, upper finger section, middle finger section, and distal finger section); When the optimal final target geometric angle combination is difficult to achieve due to environmental impact, other geometric angle combinations are predicted based on training and mechanical analysis angle combination data, and the optimization principle is always followed, that is, the optimal solution is selected from the executable geometric angle combination, that is, the optimal geometric angle combination; In the process of realizing the optimal geometric angle combination, each short limb forms a number of continuous geometric angle combinations or action trajectory combinations (action flow) in the process of realizing the final geometric angle combination target, that is, the continuous activity process, and the action flow that realizes the final geometric angle combination target has several, each action flow is also based on machine learning (mainly refers to large model LLMs) training, based on the change of environment or scene, continuously adjusts, infers and predicts the best action flow, for example, based on the real-time appearance of obstacles to change the action flow, and in the robot cooperative work, based on the real-time change of the other party's action, the best action flow is predicted based on machine learning.

[0027] The robot scheduling decision simulation intelligent agent platform has a robot scheduling method:

[0028] First, the SIAP associates with the mechanical simulation analysis module (or connects the simulation analysis software through API), and the AI module and LLMs module in the SIAP train the adjustable posture of each robot and the associated mechanical simulation analysis data, that is, the performance data, including: the load capacity, lifting capacity, pushing capacity, and dragging capacity in each posture, and the three-dimensional space coordinate data of each posture; The way of loading refers to loading in one force point (or surface) or in multiple force points (or surfaces) combination, assuming n (n≥1) force points, when n>1, there are combinations, The load capacity refers to the load threshold range Pw (0≤Pw≤P max ), P maxThe value of E is based on the analysis and test results of the SIAP and the mechanical simulation module associated with the SIAP; the lifting capacity refers to the robot lifting an object (detaching the object from the original placement surface) through the force point and being able to keep balance to complete the up, down, left, right, front and rear movements, the weight Ew (0≤Ew≤E max ) of the lifted object is the lifting capacity, the value of E max is based on the analysis and test results of the SIAP and the mechanical simulation module associated with the SIAP, the lifting force is E, and E satisfies (0≤E≤E ax ); the pushing capacity refers to applying a certain vertical lifting force Eth (0≤Eth max ) to the object, and simultaneously applying a horizontal pushing force THw to the object, the maximum weight Wo max of the object that can be pushed, assuming that the friction force of the object is FR, the value of FR is based on the actual application scene and simulation analysis data, for example, a box placed on a cement ground, based on the contact area of the box and the ground, the weight of the box, the type and smoothness of the ground, different lifting forces Eth applied to the box, and then simulating and analyzing the friction force to determine which boxes the robot can push, for another example, when the object is placed on a tire type chassis (such as a two-wheel or four-wheel cart), a certain lifting force Eth needs to be applied to the two-wheel cart to keep the cart balanced, and then a pushing force THw is applied, combined with the type and smoothness of the ground, the slope of the ground, the self-weight of the cart and the weight of the object, the appropriate horizontal pushing force THw and the lifting force Eth are simulated and analyzed, and the four-wheel cart does not need to apply a horizontal pushing force THw, but only needs to apply a lifting force Eth, the type and smoothness of the ground, the slope of the ground, and the information that the object is placed on the cart or the four-wheel cart or directly on the ground are all ground scene data, F et is the resultant force of the horizontal pushing force THw and the lifting force Eth, which is represented by a vector as The magnitude (modulus) of the resultant force is The direction of the resultant force is the angle θ of the resultant force relative to the horizontal direction,

[0029] Further combined with the weight (mass and volume) of the object and the ground friction, and according to Newton's second law to calculate the acceleration; the pulling capacity and the pushing capacity are the same;

[0030] The pose is characterized by the combination of the relative geometric angles between the two limb short sections connected by each active joint, combined with the angles of the robot root force point (such as the foot) with the ground and the ground with the horizontal plane; further training the motion flow from the initial pose to the target pose, i.e. the change of continuous angle combination, a motion flow consists of one or more actions, an action is regarded as the smallest unit (or the most microscopic) angle combination change, training the trajectory path of each motion flow, i.e. the change of three-dimensional space coordinates, and training the feasible motion flow based on the weight and volume of different pre-handling objects;

[0031] The second step is intensive environment perception. The laser radar and image light sensing (camera) system is deployed on the factory roof or vertical pole to collect the ground environment data of the workshop in a bird's eye view, including three-dimensional space point cloud data and image data. The data is processed by SIAP deployed in the central server;

[0032] The third step is that the AI module and LLMs module in SIAP train the handling object type, volume, weight, and material type reflecting the fragility of the target object, object type, such as light handling of eggs, through deep learning; At the same time, train the environment walking route and identifier, text semantics; Further train the environmental obstacles, and the object information that interacts with the robot, such as the carrier information of the pre-handling object (for example, handling from the ground, shelf, another robot hand, etc.), the information of the target storage carrier (for example, placed on the shelf, another hand of the handover robot), the carrier information includes the material type reflecting the fragility of the carrier, object type, such as handling objects from a person's hand to avoid hurting people;

[0033] The fourth step is that SIAP sends direct or indirect scheduling instructions, task targets, and routes to the robot; further, SIAP sends environmental data within a certain radius range around the robot based on the coordinate position of the robot, and the robot can also request to receive the surrounding environmental data of a certain position;

[0034] The fifth step is to configure the environmental perception system on the robot side, i.e. the vision system (image light sensing system) or simultaneously configure the laser radar and vision system. The environmental perception data of the robot is fused with the environmental perception data of SIAP, i.e. the environmental perception data of different angles are fused to comprehensively judge the environmental information, including the target object information; the robot executes the scheduling, and based on the task target combined with the environmental perception information, it needs to find the target object, and keeps communication with SIAP, combined with the target object information characteristics to predict the feasible coordinate relative position, pose, and motion flow, and adjust the position, pose, and motion flow in time based on real-time environmental changes;

[0035] Step 6, SIAP is responsible for macro-scheduling of robot cluster, and further realizes micro-scheduling among robots, that is, robots autonomously cooperate through wireless communication to flexibly change micro-scheduling scheme, but always follow the target task instruction of SIAP; SIAP is responsible for supervision and realizes group cooperation, and uses optimization algorithm to ensure load balancing;

[0036] Step 7, SIAP integrates external systems, combines with ERP system (such as SAP) or MES (Manufacturing Execution System) to synchronize production data, and simulates and rehearses scheduling results. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a schematic diagram of robot scheduling plan, wherein EPPLAN is embodied robot (in the figure, it specifically refers to robot) resource participating in scheduling, arrow represents logical connection relationship, and the dashed box below the arrow is its equipment scheduling quantization method, which covers closely related indicators and algorithms with business, wherein, EqC is robot, which refers to schedulable robot in all production resources related to order production, EqAP is A robot scheduling plan, which refers to scheduling plan for robot resource made for completing a task, and is one of several robot scheduling combinations, for example, A plan needs to select a from n type 1 robots, i.e. select b from x type 2 robots, i.e. select c from y type 3 robots, i.e. and so on, and then the A plan is EAl1o is robot allocation and sequencing optimization, which refers to allocation of which robots to which tasks; Edur is robot occupation time period calculation, which refers to collection of time periods occupied by robots allocated to corresponding tasks; ECos is robot occupation cost calculation, and different robots have different use costs, so accurate calculation of use cost is performed; RegE is robot configuration rule, and different production environments, management systems and resource conditions have corresponding rules, and robot allocation plan must follow the rules; ConsE is constraint, which integrates constraint conditions limited into allocation of robot resource; TeC is action, which refers to schedulable action in all action flows related to production tasks, TeAP is A action flow plan, which refers to scheduling plan for action combination made for completing a task, and is one of several action scheduling combinations, for example, A plan needs to select a from n carrying a actions, i.e. select b from x b actions, i.e. select c from y c actions, i.e. and so on, and then the A plan is TeCos is the simulation mechanics calculation; TeAll0 is the action sequence; TeDur is the action consumption duration; RegT is the action flow rule; ConsT is the action constraint; ProC is the micro action, which is the micro level description of the action, several actions constitute an action flow, ProAP is the A micro action plan; PrAllo is the micro action arrangement, the idea is similar to the action, PrCos is the micro action simulation mechanics calculation, the idea is as the action, PrDur is the micro action duration, the idea is as the action; RegP is the micro action rule, ConsP is the micro action constraint, the idea is as the action;

[0038] HRPLAN is the humanoid robot resource participating in the scheduling plan, the dashed box under the arrow is the humanoid robot resource scheduling quantification method, wherein, RolC is the function category, which is divided according to the characteristics of the post responsibility; StrenCh is the post strength characteristic, which refers to the subdivision direction characteristics of physical (electric) power, computing power and ability in work; SSM is the scheduling mode, including the rotation mechanism, the post per shift work time mechanism, the post number arrangement mechanism, the coordination mechanism between posts, the post linkage and emergency mechanism; LabR is the humanoid robot (labor) resource characteristics, including the health status, the good at, the environment and the post tolerance of the laborer;

[0039] ModW is a mathematical algorithm, the dashed box under the arrow is a guided algorithm model, referred to as a guided model, wherein, PLU is a resource utilization rate guided model, its calculation method or weight tends to the utilization rate of production resources and production process consumption resources, and is guided by low carbon, low energy consumption, low emission, low material consumption and resource reusability; PLO is a yield priority guided model, which tends to maximize the quantity of products or output per unit of time; PLB is a production yield guided model, which takes the total profit of output or product as the tendency; PLQ is a quality priority model, also known as a minimum failure rate model, which takes improving the yield rate as a priority;

[0040] In this manual, the scheduling quantification method, the shift quantification method, and the guidance algorithm are collectively referred to as the planning and scheduling quantification model, referred to as the quantification model. The quantification model data is input into the intelligent decision-making module, that is, the dotted box containing ATPP, AEP, and HP. The intelligent decision-making module is composed of the LLMs large language model, the deep learning and reinforcement learning model, and the video generation AI model. ATPP, AEP, and HP are the three main functions of the intelligent decision-making module. ATPP is the optimal action and micro-action arrangement; AEP is the optimal embodied robot allocation, HP is the optimal scheduling of humanoid robots, and MicPLAN is the output scheduling plan; InFal is emergency processing, which means that an emergency task is inserted into a schedule that has been scheduled to enter the operating state, that is, the original task resources are occupied, or the original task is damaged or fails, it is temporarily shelved, and other tasks are inserted to occupy the resources. DynR is dynamic adjustment, which means sending real-time status to the intelligent decision-making module, and the intelligent decision-making module reschedules. DETAILED DESCRIPTION

[0041] Step 1: Resource Input

[0042] Resource input data refers to all robot resource data related to production in the manufacturing plant. Resource data needs to be input into SIAP, which maps the resource data to the ODMG module. The logical design structure of the resource input module is as follows:

[0043] Production equipment and process flow are divided into several equipment categories and process categories. Equipment categories and process categories are layered according to the inclusion relationship and sequence relationship, namely the inclusion layer L and the parallel layer S. The inclusion relationship is the nested relationship, and the sequence relationship here refers to the order relationship. Each inclusion layer is given a unique number, namely L i (i=1,2,...,N), assuming that L1 is the first layer, L2 is the second layer, L2 is the sublayer of L1, that is, L1 contains L2, and each sequence layer is given a unique number, namely S j (j=1,2,...,N), S1 is the upper layer of S2, that is, S1 first and then S2; support adding or deleting layers; each L layer and S layer includes one or more nodes Kn, each node is given a unique number, support adding or deleting nodes; support users in the L of device module class i Layer and S j Input device resource information into the nodes in the layer. The device resource information includes device name, device function, device purpose, and device performance. A node can only input one device resource information. When a device E consists of multiple sub-devices, each sub-device is L i+1 E i (-=1, 2, ..., N), then the node KnE where the E device is located is L i Layer, each sub-device belongs to L i+1 layer, in L i+1Each node in the layer inputs L i+1 E i (i = 1, 2, …, N) device resource information; when L i After determining the layer, determine S - layer in each L j layer, each S j layer has one or more devices;

[0044] Similarly, support users to input action information in the nodes of the L i layer and S j layer in the action module class, and only one action information can be input in an action node, when an action group (action combination) T is composed of multiple actions, each action is L i+ 1T i (i = 1, 2, …, N), then the node KnT where the T action combination is located is L i layer, each action belongs to L i+1 layer, and each L i+1 layer has one or more actions; i+1 T i (i = 1, 2, …, N) action information, when L i After determining the layer, determine S i layer in each L j layer, each S j layer has one or more action groups or actions;

[0045] The input device, i.e. the robot, includes the layer, the time sequence layer, which is divided into regular relationship and irregular relationship. The regular relationship is the rule that the large model must follow, and the irregular relationship is the training method that the large model will rely on the self-attention mechanism and the related training data to complete the containing relationship and the time sequence relationship between devices.

[0046] Data processing is divided into layers, i.e. resource input layer, ODMG layer, and Tensor layer. Tensor is a tensor data structure, and ODMG converts the device, process, and progress data input by the resource input layer into a tensor data structure.

[0047] Second step, ODMG knowledge graph maps and structures the input resource data

[0048] ODMG is an abstract structure of ordered and unordered mixed graph, which completes the mapping of resource input to data structure through ODMG. The order is embodied by the determined inclusion relationship and sequence relationship between nodes in the graph, that is, there is a clear inclusion layer relationship and sequence layer relationship between the nodes of the ordered graph, the edge direction reflecting the inclusion layer relationship between the nodes is clear, and the edge direction reflecting the sequence layer relationship between the nodes is also clear. The disorder refers to the temporary inability to determine the inclusion relationship and sequence relationship between device nodes, or the inclusion relationship or sequence relationship between device nodes may exist in multiple forms, that is, the edge direction reflecting the inclusion layer relationship between the nodes of the unordered graph is arbitrary, and the edge direction reflecting the sequence layer relationship between the nodes is also arbitrary.

[0049] The ordered node relationship is the inclusion relationship and sequence relationship defined in advance based on business characteristics, and the unordered node relationship refers to the inclusion relationship and sequence relationship between nodes completed by subsequent machine learning (including conventional machine learning model, large language model, generative diffusion model, etc.) training and reasoning without prior definition. The node with prior defined ordered relationship is called ordered node, and the node without prior defined ordered relationship and requiring machine learning for subsequent training and reasoning to complete the ordered relationship is called unordered node.

[0050] There may be an inclusion or sequence relationship between the unordered node and any ordered node or unordered node, which is determined by machine learning training and reasoning.

[0051] Based on ODMG, the knowledge graph structure of robot resources and action flow is completed, and attributes are included in each node (including node group). The hierarchical structure is realized according to the inclusion layer and sequence layer relationship between action groups, actions, and between action groups and actions. Similarly, robot cluster management is realized by combining one or more robots into a small group, combining multiple small groups into a team, and so on. Each small group is divided into multiple types of work, and each type of work has one or more robots, which are all structured according to ODMG.

[0052] Step 3, TSDB time series data structure collects progress data

[0053] TSDB (Time Series Data Structure) is responsible for describing the work progress data of robots changing with time axis. The actual work progress data of the device (robot) is the work efficiency (unit time output or unit time completion rate) and the work state (running or non-running), which is collected and stored in the time series database in the form of time series data. The time axis runs through the work progress data, and attributes are defined for the work progress data to facilitate association with ODMG.

[0054] Step 4, association and fusion of TSDB and ODMG

[0055] The progress data based on the TSDB and the resource data based on the ODMG are associated according to the ODMG hierarchical structure and the attributes of the resource names in each layer, and the resource names in the same containing layer and sequence layer have unique characteristics, that is, even if the resource names in different layers have the same name, each resource can be uniquely distinguished by the layer prefix mark, and each resource is associated with the TSDB time sequence progress data, as shown in Table 1.

[0056] Table 1: TSDB and ODMG association fusion table

[0057]

[0058] The ODMG data and the TSDB data are respectively tensorized, the TSDB time sequence data is converted into a [B, T, F] matrix or a high-dimensional time sequence feature, and the ODMG knowledge graph is converted into a node and edge feature tensor or a pre-trained embedding vector.

[0059] The data after the fusion and association of the time sequence data structure TSDB and the knowledge graph ODMG is finally sent to a large model for training and reasoning, and the TSDB and ODMG data are both mapped into a tensor format for parallel computing and gradient propagation in a neural network. The specific steps are as follows: first, the time sequence data is converted into a sequence tensor, a sampling window or a fixed time length slice is used to organize continuous efficiency values, sensor readings, etc. into a three-dimensional tensor like [B, T, F], where B is the batch size, T is the time step length, and F is the feature dimension at each step (such as the number of multi-channel sensor values, normalized efficiency values, etc.). Further preprocessing is performed to realize normalization, difference, and frequency domain transformation, and a time sequence feature vector is output. Second, the knowledge graph is converted into a graph embedding tensor, for example, using a graph neural network (GNN), which requires constructing an edge list and relationship type. Third, multi-modal fusion is performed, and the time sequence tensor and the graph tensor are spliced in the feature dimension or cross-modal attention interaction is performed inside the model. The time sequence data is encoded by a Transformer, the graph data is encoded by a GNN, and the TSDB and ODMG representations are fused at the top layer. The graph database is responsible for online and offline graph data services, and the time sequence database is responsible for high-frequency index streaming writing.

[0060] Fourth, scheduling decision and quantitative calculation

[0061] The data obtained by fusing and associating the time series data structure TSDB and the knowledge graph ODMG through the TRANSFORMER self-attention mechanism in the LLM is automatically labeled based on the hierarchical and fusion relationship, the hierarchical relationship and the association relationship of the resource are determined, and the hierarchical relationship and the association relationship are encoded in the Q, K, and V vectors and the weight matrix of the calculation thereof; further, the LLM is trained and learned in more dimensions based on the ODMG resource system and the TSDB time series progress data and combined with the expert knowledge base data, that is, the Q, K, and V vectors and the weight matrix related to the resource and the progress are trained to reflect more dimensional information, which is realized in the following ways: first, the position and direction of the vector in the high-dimensional space, assuming a huge, associated space with hundreds or even thousands of dimensions (such as 512 or 768 dimensions), the Q, K, and V vectors of each Token are a point or an arrow from the origin in this space, after training of a large amount of data, the model will learn to place similar relationships in the space in similar positions; second, the association relationship is a geometric operation between vectors, the core of the self-attention mechanism is the dot product of Q and K, that is, score = Q x K, which can be understood in geometry as measuring the alignment degree or similarity of two vectors, that is, the Q, K, and V vectors are obtained by multiplying the original word embedding (Word Embedding) by the weight matrix (Wq, Wk, Wv), different association relationships are distinguished by adjusting the parameters in the Wq, Wk, and Wv matrices, and through the continuous back propagation and gradient descent process, the best association relationship is gradually learned and adjusted after training a certain amount of knowledge;

[0062] Step 5, robot scheduling, according to Figure 1 , complete the scheduling and action flow scheduling of embodied robots and capricious robots, fuse the mechanical simulation analysis module and the large model module, the simulation preview module, and form an intelligent agent platform based on intelligent allocation of robot clusters for job tasks and intelligent adjustment of robot actions and postures.

Claims

1. The characteristics of the robot scheduling decision simulation agent platform, namely SIAP, are as follows: it mainly includes the large language model LLMs module, resource input module, ODMG module, TSDB module, algorithm tool module ATP, auxiliary enhanced input module AAS, simulation and preview VGM module, robot scheduling MPS module, knowledge base module, perception module, memory module, action module, mechanical simulation and analysis module; the function of the large model, namely the large language model, is realized through the LLMs module of SIAP; by mapping the input production resources into high-level structured data, the resource rule relationship is reflected through the association relationship of the resource nodes of the layer and sequence layer, and each layer is reflected. The number of resource nodes in the system is determined, and each node is given functions, performance (or effects), uses, and attribute information related to the production schedule, so as to realize the method of its LLMs module to understand production resources and rules, and implement this method through SIAP's ODMG module; design a TSDB module to realize real-time progress data and historical progress data that reflect the time series data structure, associate TSDB data with ODMG to realize the fusion of high-level structured resource data and time series progress data, and tensorize them for large model training and reasoning; design a method for the production scheduling system to schedule the production line robotic arms and robots respectively, and implement it through SIAP's robot scheduling MPS module, and integrate the MPS module The API interface is associated with the robotic arm and robot, and the production scheduling plan based on SIAP directly schedules the robotic arm and robot; the APS scheduling system is integrated with the large model LLMs and AI model collection to achieve automated, intelligent advanced planning and scheduling, and accurately control the robot posture and motion flow based on environmental perception; the characteristics of the robot are: first, the API interface is open to the LLMs scheduling instruction system; second, LLMs has obtained the robot's mechanical structure simulation analysis data, including all possible robot movements and posture data, and mechanical analysis data of picking up or carrying heavy objects in all postures; third, machine learning training for picking up and carrying heavy objects in all postures and movements Data for objects of different sizes and weights is obtained. All relative geometric angles S = {S1A = {S1a1, S1a2, S1a3, ...}, S2A = {S2a1, S2a2, S2a3, ...}, ...} between the two limb segments connected by each movable joint of the robot are obtained. S1A represents all the movements of node S1, S2A represents all the movements of node S2, and so on. All geometric angles 0 = {o1, o2, o3, ...} of the force-bearing ground contacted by the most fundamental force-bearing support point or surface (such as the foot of a humanoid robot) are obtained. Then, the geometric angles H = {h1, h2, h3, ...} between the force-bearing ground and the horizontal plane are obtained.}, select a geometric angle from S, O, and H respectively for combination. The geometric angle selected by S refers to the combination of the angles of the limb segments connected by each joint included. Machine learning trains the positional relationship between all the combined geometric angles and the target object, infers the feasible relative position, and then judges the classification, volume, and weight of the target object based on relative position relationship and deep learning. Combined with mechanical structure simulation analysis, the best geometric angle combination is predicted, and finally the geometric angle of each joint is predicted to achieve the optimal final geometric angle combination goal. From the current geometric angle of the robot to the target geometric angle, the adjustment process has several continuous groups of geometric angle changes. According to the principle of force point from the root to the end, the joints of the robot are adjusted step by step to achieve the geometric angle change of each limb segment (for example, the adjustment order of the humanoid robot is foot, ankle, calf, thigh, upper body, upper arm, lower arm, wrist, upper finger segment, middle finger segment). , finger distal segments); when the optimal final target geometric angle combination becomes difficult to achieve due to environmental influences, alternative geometric angle combinations are predicted based on training and mechanical analysis angle combination data, always adhering to the principle of optimization. This means selecting the optimal solution, or the optimal geometric angle combination, from the available geometric angle combinations. In the process of achieving the optimal geometric angle combination, each limb segment forms several continuous geometric angle combinations or motion trajectory combinations, or motion flows, during the continuous movement process to achieve the final geometric angle combination target. There are several motion flows to achieve the final geometric angle combination target. Each motion flow is also trained based on machine learning (primarily referring to large models (LLMs) in this article), continuously adjusted based on changes in the environment or scene, and inferred to predict the optimal motion flow. For example, the motion flow can be changed based on the appearance of obstacles in real time. In collaborative robot operations, the optimal motion flow can be predicted based on the real-time changes in the other party's movements using machine learning.

2. According to claim 1, a robot scheduling decision-making simulation agent platform, wherein the robot scheduling method comprises: Step 1: SIAP is associated with a mechanical simulation analysis module (or connected to simulation analysis software via an API), wherein the AI ​​module and LLMs module in SIAP train each robot's adjustable posture and associated mechanical simulation analysis data, i.e., operational performance data, including: The bearing capacity, lifting capacity, pushing capacity, and towing capacity under each posture in what way, as well as the three-dimensional space coordinate data of each posture; what way of bearing weight means bearing weight in the way of one force application point (or surface) or a combination of multiple force application points (or surfaces). Let there be n (n≥1) force application points. When n>1, there are combinations, The bearing capacity refers to the bearing threshold range Pw (0≤Pw≤P max ) for maintaining body balance and not damaging the mechanical structural parts. The P max value is based on the analysis and test results of the SIAP and the mechanical simulation module associated with SIAP; the lifting capacity refers to the weight Ew (0≤Ew≤E max ) of the object that the robot can lift (detach from the original placement surface of the object) through the force application point and can maintain balance to complete up, down, left, right, front, and back movements. The E max value is based on the analysis and test results of the SIAP and the mechanical simulation module associated with SIAP. The lifting force is E, and E satisfies (0≤E≤E max ); the pushing capacity refers to applying a certain vertical lifting force Eth (0≤Eth<Wo), where Wo is the weight of the object, and at the same time, Eth must satisfy (0≤Eth<E max ), and applying a horizontal thrust THw to the object, and the maximum weight Wo max of the object that can be pushed. Assuming the friction force of the object is FR, the value of FR is based on the actual application scenario and simulation analysis data. For example, for a box placed on a cement floor, based on the contact area between the box and the ground, the weight of the box, the material type and smoothness of the ground, and the different lifting forces Eth applied to the box, the friction force is simulated and analyzed to determine which boxes the robot can push. Another example is when an item is placed on a tire-type chassis (such as a two-wheel or four-wheel trolley). The two-wheel trolley needs to apply a certain lifting force Eth to maintain the balance of the trolley, and then apply a thrust THw. Combining the material type and smoothness of the ground, the ground slope, the self-weight of the trolley and the weight of the object, the appropriate horizontal thrust THw and lifting force Eth are simulated and analyzed. The four-wheel trolley does not need to apply a horizontal thrust THw, but only needs to apply a lifting force Eth. The material type and smoothness of the ground, the ground slope, and the information about whether the item is placed on the trolley or the four-wheel vehicle or directly on the ground all belong to the ground scenario data. F et is the resultant force of the horizontal thrust THw and the lifting force Eth, and is represented by a vector as The magnitude (modulus) of the resultant force is The resultant force The direction of is the angle θ of the resultant force relative to the horizontal direction, Further combined with the object weight (mass and volume) and ground friction, the acceleration is calculated according to Newton's second law; the pulling ability and pushing ability are the same; based on the simulation analysis data, the type of target object (some object types are directly related to material, volume, weight), the target operation scene data, and the robot operation performance data, the robot type corresponding to the target task is predicted, and the optimal posture that the robot should adopt and how to bear the load in the optimal posture are further inferred, including load-bearing capacity, lifting capacity, pushing capacity, and pulling capacity; the posture is characterized by the combination of the relative geometric angles between the two limb segments connected by each active joint, and is combined with the robot's root force point (such as the foot) and The angle of the ground and the angular characteristics between the ground and the horizontal plane are further trained. The action flow from the initial pose to the target pose, that is, the change in continuous angle combinations, is further trained. An action flow consists of one or more actions, and an action is considered the smallest unit (or most microscopic) of angle combination changes. The trajectory path of each action flow, that is, the change in three-dimensional spatial coordinates, is trained. Furthermore, feasible action flows are trained based on the weight and volume of different pre-handled objects. The second step is intensive environmental perception. LiDAR and image light sensing (camera) systems are deployed on the factory ceiling or vertical poles to collect workshop floor environmental data from a bird's-eye view, including three-dimensional spatial point cloud data and image data. This data is processed by the SIAP deployed on the central server. In the third step, the AI ​​module and LLMs module in SIAP are trained through deep learning to determine the type, volume, and weight of objects to be handled, as well as the material type and object type that reflect the fragility of the target object, such as eggs that should be handled gently. At the same time, the environment walking routes and identification symbols and text semantics are trained; further training is carried out on environmental obstacles and information on objects that interact with the robot, such as the carrier information of the pre-carried object (for example, from the ground, shelf, or the hands of another robot), and the target storage carrier information (for example, placed on a shelf or in the hands of another handover robot). The carrier information includes the material type and object type that reflect the fragility of the carrier, such as avoiding injury to people when carrying objects from a person's hands; fourth, SIAP sends direct or indirect scheduling instructions, task objectives, and routes to the robot; further, SIAP sends environmental data within a surrounding radius based on the robot's coordinate position, and the robot can also request to receive environmental data within a certain location; fifth, the robot side is configured with an environmental perception system, namely a visual system (image light sensing system), or a lidar and a visual system at the same time, and the robot's environmental perception data is integrated with the SIAP environmental perception data, that is, the environmental perception data from different angles are integrated to comprehensively judge the environmental information, including the target object information; In the sixth step, the robot executes scheduling and searches for the target object based on the task objectives and environmental perception information, and maintains communication with SIAP. It predicts the feasible coordinate relative position, posture, and motion flow based on the information characteristics of the target object, and adjusts the position, posture, and motion flow in time based on real-time environmental changes; SIAP is responsible for the macro-scheduling of the robot cluster, and further realizes distributed micro-scheduling between robots, that is, the robots work autonomously and collaboratively through wireless communication, flexibly changing the micro-operation plan, but always following the target task instructions of SIAP; SIAP is responsible for supervision, realizing group collaboration, and using optimization algorithms to ensure load balancing; in the seventh step, SIAP integrates external systems, combines ERP systems and MES to synchronize production data, and simulates and rehearses scheduling results.

Images