A multi-machine collaborative scheduling system for human-machine controlled modular cabins

Abstract

This invention provides a multi-machine collaborative scheduling system for a human-machine controlled container, belonging to the field of multi-machine collaborative scheduling and control technology. The system includes: a task semantic understanding and planning module; an environmental feature intelligent classification and surface modeling module connected to the task semantic understanding and planning module; a task semantic enhancement and risk assessment module connected to the environmental feature intelligent classification and surface modeling module; and a multi-machine collaborative decision-making module connected to the task semantic enhancement and risk assessment module. This invention achieves this by closely cooperating through task semantic understanding, environmental modeling, risk assessment, and decision-making functions to form an organic whole. Under the unified scheduling of the system, different types of machines can efficiently and collaboratively complete complex tasks, breaking the limitations of isolated module operation in traditional systems and meeting the stringent requirements for multi-machine collaborative operations in multiple fields and scenarios.

Description

Technical Field

[0001] This invention relates to the field of multi-machine collaborative scheduling and control technology, and in particular to a multi-machine collaborative scheduling system for a human-machine control cabin. Background Technology

[0002] With the rapid development of industrial automation and intelligence, the demand for multi-machine collaborative operations is increasing in human-machine control cabin application scenarios, such as emergency rescue and large-scale engineering construction, where multiple machines of different types (such as drones, ground robots, and special-operation equipment) are often required to work together to complete complex tasks. However, existing multi-machine scheduling systems have many shortcomings:

[0003] On the one hand, task scheduling mainly relies on preset rules or simple priority allocation strategies, making it difficult to handle complex task instructions in natural language form. It lacks a deep understanding of task semantics, leading to unreasonable task allocation and failing to fully leverage the machine's performance advantages. On the other hand, environmental perception and modeling capabilities are limited, often relying on single sensor data or simple map construction methods. This fails to accurately describe the terrain features, obstacle distribution, and semantic attributes of complex environments, making path planning ill-suited to dynamically changing environments, easily resulting in path conflicts and low task execution efficiency. Furthermore, traditional systems have relatively independent modules, lacking effective data interaction and algorithm fusion mechanisms. They cannot adjust strategies in real time based on changes in risk or sudden environmental changes during task execution, making it difficult to meet the efficiency and safety requirements of multi-machine collaborative operations. Summary of the Invention

[0004] This invention provides a multi-machine collaborative scheduling system for human-machine controlled cabins to solve one or more of the problems mentioned above.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A multi-machine collaborative scheduling system for a human-machine controlled mobile cabin includes:

[0007] The task semantic understanding and planning module is used to receive natural language task instructions, parse them to obtain task semantic vectors, task target sets and constraint sets, and construct a cost matrix based on task priority, machine load and estimated task time. The task-machine allocation set is obtained by solving the optimization problem.

[0008] The environmental feature intelligent classification and surface modeling module is connected to the task semantic understanding and planning module. It receives the task-machine assignment set and the constraint set, combines environmental sensor data and original samples, obtains the terrain type set, obstacle probability and ground accessibility score through random forest feature classification, and generates a surface model with semantic labels according to the terrain type by selecting the appropriate fitting method.

[0009] The task semantic enhancement and risk assessment module is connected to the environmental feature intelligent classification and surface modeling module. It receives a surface model with semantic labels, combines the task semantic vector and historical task data, obtains the risk probability and key influencing factor set through random forest risk prediction, and generates task adjustment suggestions through a large language model.

[0010] The multi-machine collaborative decision-making module is connected to the task semantic enhancement and risk assessment module. It receives task adjustment suggestions, risk probabilities and key influencing factor sets. Combining the task-machine allocation set and the surface model with semantic labels, it performs path planning through the improved A* algorithm to obtain an initial path set. It uses the spatiotemporal grid method for conflict detection and uses the game theory payoff function to resolve conflicts to obtain the adjusted path.

[0011] The execution feedback results of the multi-machine collaborative decision-making module are used to update historical task data, thereby optimizing the task semantic understanding and planning module and the task semantic enhancement and risk assessment module.

[0012] In this specification, the task semantic understanding and planning module parses natural language task instructions through a large language model to obtain a task semantic vector, a set of task objectives, and a set of constraints. The set of task objectives includes specific tasks such as material transportation and environmental monitoring, and the set of constraints includes time limits, equipment limitations, and accuracy requirements. The task semantic vector is a vectorized semantic representation of natural language task instructions, which includes task objectives and constraints.

[0013] In this specification, the random forest feature classification in the intelligent classification and surface modeling module of environmental features is composed of multiple decision trees. The training set is generated by sampling the original samples with replacement, and some features are randomly selected to construct the decision tree. The minimum Gini index is used as the basis for node splitting. Finally, the set of terrain types, obstacle probability and ground accessibility score are determined through a voting mechanism.

[0014] In this specification, the set of terrain types obtained by the random forest feature classification is fitted with a local polynomial surface for rocky areas and with a smooth spline surface for grasslands to generate a surface model with semantic labels. This model includes the geometric shape information of the environment as well as semantic attributes such as terrain type, accessibility, and risk level.

[0015] In this specification, the random forest risk prediction in the task semantic enhancement and risk assessment module uses the task semantic vector, features extracted from the semantically labeled surface model, and features extracted from historical task data to form an input feature vector. The risk probability is calculated through the random forest model, and the set of key influencing factors is determined based on the importance of the features.

[0016] In this specification, the task semantic enhancement and risk assessment module concatenates the task semantic information and risk assessment results and inputs them into a large language model to generate task adjustment suggestions. The task adjustment suggestions include changing the task execution device, adjusting the task execution time, and changing the task execution path.

[0017] In this specification, in the multi-machine collaborative decision-making module, the heuristic function of the improved A* algorithm dynamically adjusts the weights according to the risk probability, and plans an initial path set for each machine by combining the distance from the node to the target point and the node's drivability score.

[0018] In this specification, the multi-machine collaborative decision-making module uses a spatiotemporal grid method to divide time and space into grids. Conflict detection is performed by judging whether multiple machines occupy the same spatial grid in the same time slice. If a conflict is detected, the parameters are dynamically adjusted according to the set of key influencing factors using a game theory payoff function. The adjustment strategy of each machine is determined by solving the Nash equilibrium, and the adjusted path is obtained.

[0019] In this specification, the execution feedback results of the multi-machine collaborative decision-making module include task execution results and machine status, which are used to update historical task data, thereby optimizing the large language model parsing and task allocation algorithms in the task semantic understanding and planning module, as well as the random forest risk prediction model and large language model strategy generation in the task semantic enhancement and risk assessment module.

[0020] In this specification, the system also includes a communication module for data transmission between the task semantic understanding and planning module, the environmental feature intelligent classification and surface modeling module, the task semantic enhancement and risk assessment module, and the multi-machine collaborative decision-making module.

[0021] In summary, the present invention has at least the following beneficial effects:

[0022] High-precision task parsing and intelligent allocation: Through deep parsing of natural language task instructions using a large language model, task objectives and constraints are accurately extracted. Combined with an improved task allocation algorithm, which comprehensively considers factors such as task priority and machine load, optimal matching between tasks and machines is achieved, significantly improving the rationality of task allocation and the utilization rate of machine resources. For example, in emergency rescue scenarios, tasks such as search and transportation can be quickly and accurately allocated to the most suitable drones and ground robots, shortening task response time.

[0023] Precise Environmental Modeling and Adaptive Planning: The intelligent environmental feature classification and surface modeling module utilizes random forests and various surface fitting algorithms to perform high-precision modeling of complex environments, generating surface models that include geometric information and semantic attributes. This allows the path planning algorithm to dynamically adjust paths based on environmental risks and accessibility. For example, in mountainous operations, it automatically avoids dangerous areas such as steep slopes and obstacles. Furthermore, it quickly replans paths when the environment changes (e.g., new obstacles appear), improving the safety of multi-machine operations and the success rate of task execution.

[0024] Dynamic Risk Assessment and Intelligent Decision-Making: The task semantic enhancement and risk assessment module integrates task semantics, environmental models, and historical data to accurately predict task execution risks and generates targeted adjustment suggestions using a large language model. The multi-machine collaborative decision-making module dynamically optimizes path planning and conflict resolution strategies based on risk assessment results. For example, under adverse weather conditions, it promptly adjusts the task execution order or replaces execution equipment to ensure the stability and reliability of multi-machine collaborative operations.

[0025] End-to-end data closed-loop optimization: A complete data closed loop is formed between the various modules of the system. The task execution feedback results are used to update historical task data, thereby optimizing core algorithms such as large language models and random forests. As task execution experience accumulates, the system's understanding of tasks, perception of the environment, and ability to respond to risks continuously improve, achieving self-learning and adaptive optimization, and continuously improving the intelligence level and operational efficiency of multi-machine collaborative scheduling.

[0026] High integration and collaborative operation: Task semantic understanding, environment modeling, risk assessment and decision-making functions work closely together to form an organic whole. Under the unified scheduling of the system, different types of machines can efficiently and collaboratively complete complex tasks, breaking the limitations of isolated module operation in traditional systems and meeting the stringent requirements for multi-machine collaborative operation in multiple fields and scenarios. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the architecture of the multi-machine collaborative scheduling system for the human-machine control cabin involved in this invention.

[0029] Figure 2 This is a flowchart illustrating the task semantic understanding and planning module involved in this invention.

[0030] Figure 3This is a flowchart illustrating the environmental feature classification and surface modeling module involved in this invention.

[0031] Figure 4 This is a flowchart illustrating the task semantic enhancement and risk assessment module involved in this invention.

[0032] Figure 5 This is a flowchart illustrating the multi-machine collaborative decision-making module involved in this invention. Detailed Implementation

[0033] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0034] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] This embodiment provides a multi-machine collaborative scheduling system for a human-machine controlled mobile cabin, including:

[0037] The task semantic understanding and planning module is used to receive natural language task instructions, parse them to obtain task semantic vectors, task target sets and constraint sets, and construct a cost matrix based on task priority, machine load and estimated task time. The task-machine allocation set is obtained by solving the optimization problem.

[0038] The environmental feature intelligent classification and surface modeling module is connected to the task semantic understanding and planning module. It receives the task-machine assignment set and the constraint set, combines environmental sensor data and original samples, obtains the terrain type set, obstacle probability and ground accessibility score through random forest feature classification, and generates a surface model with semantic labels according to the terrain type by selecting the appropriate fitting method.

[0039] The task semantic enhancement and risk assessment module is connected to the environmental feature intelligent classification and surface modeling module. It receives a surface model with semantic labels, combines the task semantic vector and historical task data, obtains the risk probability and key influencing factor set through random forest risk prediction, and generates task adjustment suggestions through a large language model.

[0040] The multi-machine collaborative decision-making module is connected to the task semantic enhancement and risk assessment module. It receives task adjustment suggestions, risk probabilities and key influencing factor sets. Combining the task-machine allocation set and the surface model with semantic labels, it performs path planning through the improved A* algorithm to obtain an initial path set. It uses the spatiotemporal grid method for conflict detection and uses the game theory payoff function to resolve conflicts to obtain the adjusted path.

[0041] The execution feedback results of the multi-machine collaborative decision-making module are used to update historical task data, thereby optimizing the task semantic understanding and planning module and the task semantic enhancement and risk assessment module.

[0042] In some embodiments, the task semantic understanding and planning module parses natural language task instructions using a large language model to obtain a task semantic vector, a set of task objectives, and a set of constraints. The set of task objectives includes specific tasks such as material transportation and environmental monitoring, and the set of constraints includes time limits, equipment limitations, and accuracy requirements. The task semantic vector is a vectorized semantic representation of natural language task instructions, containing task objectives and constraints.

[0043] In some embodiments, the random forest feature classification in the environmental feature intelligent classification and surface modeling module is composed of multiple decision trees. A training set is generated by sampling the original samples with replacement, and some features are randomly selected to construct a decision tree. The minimum Gini index is used as the basis for node splitting. Finally, the set of terrain types, obstacle probability and ground accessibility score are determined through a voting mechanism.

[0044] In some embodiments, the set of terrain types obtained by the random forest feature classification is fitted with a local polynomial surface for rocky areas and with a smooth spline surface for grasslands to generate a surface model with semantic labels. This model includes geometric information of the environment as well as semantic attributes such as terrain type, accessibility, and risk level.

[0045] In some embodiments, the random forest risk prediction in the task semantic enhancement and risk assessment module constructs an input feature vector from the task semantic vector, features extracted from the semantically labeled surface model, and features extracted from historical task data. The risk probability is calculated through the random forest model, and a set of key influencing factors is determined based on the importance of the features.

[0046] In some embodiments, the task semantic enhancement and risk assessment module concatenates the task semantic information and risk assessment results and inputs them into a large language model to generate task adjustment suggestions. The task adjustment suggestions include changing the task execution device, adjusting the task execution time, and changing the task execution path.

[0047] In some embodiments, in the multi-machine collaborative decision-making module, the heuristic function of the improved A* algorithm dynamically adjusts the weights according to the risk probability, and plans an initial path set for each machine by combining the distance from the node to the target point and the node's drivability score.

[0048] In some embodiments, the multi-machine collaborative decision-making module uses a spatiotemporal grid method to divide time and space into grids. Conflict detection is performed by determining whether multiple machines occupy the same spatial grid in the same time slice. If a conflict is detected, the parameters are dynamically adjusted based on the set of key influencing factors using a game theory payoff function. The adjustment strategy for each machine is determined by solving the Nash equilibrium, and the adjusted path is obtained.

[0049] In some embodiments, the execution feedback results of the multi-machine collaborative decision-making module include task execution results and machine status, which are used to update historical task data, thereby optimizing the large language model parsing and task allocation algorithms in the task semantic understanding and planning module, as well as the random forest risk prediction model and large language model strategy generation in the task semantic enhancement and risk assessment module.

[0050] In some embodiments, the system further includes a communication module (which can adopt existing communication technologies) for data transmission between the task semantic understanding and planning module, the environmental feature intelligent classification and surface modeling module, the task semantic enhancement and risk assessment module, and the multi-machine collaborative decision-making module.

[0051] The technical concept of this invention is as follows:

[0052] I. System Architecture Overview (Architecture as follows) Figure 1 (As shown)

[0053] The multi-machine collaborative scheduling system of the human-machine control cabin is a highly integrated and intelligent system, consisting of four core modules: task semantic understanding and planning, intelligent environmental feature classification and surface modeling, task semantic enhancement and risk assessment, and multi-machine collaborative decision-making. It employs conventional communication modules for data transmission between modules. In actual operational scenarios, operators issue natural language task commands through the human-machine control cabin. The system parses these commands and, combined with environmental perception data, uses various algorithms to allocate tasks, plan paths, and resolve conflicts among multiple machines, enabling efficient collaborative operation of multiple machines in complex environments. Through data transmission and algorithm integration, the modules form an organic whole, ensuring the efficiency, accuracy, and safety of task execution.

[0054] II. Technical Details of Core Modules

[0055] 2.1 Task Semantic Understanding and Planning Module (process as follows) Figure 2 (As shown)

[0056] 2.1.1 Input

[0057] Natural language task instructions are input into the system in text form, denoted as... .in, The i-th word segmentation unit in the instruction is represented. For example, in the instruction "Within 2 hours, use drone 1 and robot 2 to transport materials from warehouse A to site B", "within", "within 2 hours", and "use" are all word segmentation units; N is the total number of words. The specific value of N after the instruction is segmented depends on the result of the word segmentation algorithm.

[0058] 2.1.2 Large Language Model (LLM) Analysis

[0059] The system uses an LLM based on the Transformer architecture to parse input instructions. Its core self-attention mechanism is implemented through the following formula:

[0060] ;

[0061] It is a query vector matrix, where n+1 represents the number of query vectors, used to find information related to itself in the input data; The key vector dimension affects the granularity and scope of the query.

[0062] Let m be a key vector matrix, where m1 is the number of key-value pairs, which can be seen as an index of the data used for matching with the query vector. The superscript T indicates transpose.

[0063] It is a value vector matrix. The dimension of the value vector actually contains the information that needs to be extracted.

[0064] Processing flow: First, the natural language task instructions are... The data is converted into a vector representation through an embedding layer. The embedding layer then converts each word segmentation unit... Assign a corresponding vector so that it can be processed by the subsequent model.

[0065] The transformed vector is input into the Transformer encoding layer of the LLM. In the encoding layer, the attention weights between vectors at different positions are calculated through a self-attention mechanism to capture the semantic relationships in the instructions. The vector is then further processed by a feedforward neural network to extract key information.

[0066] Finally, output the set of task objectives. Specific tasks such as "material transportation" and "environmental monitoring"; and sets of constraints. This may include time constraints (such as "complete within 2 hours"), equipment constraints (such as "use drone 1 and robot 2"), accuracy requirements, etc.

[0067] Task semantic vector It is the vectorized semantic representation of natural language task instructions in LLM, and its mathematical form is: , The coding layer function of an LLM typically consists of multiple Transformer encoders. Semantic vector dimension, typically 512 or 768, which are common embedding dimensions.

[0068] 2.1.3 Task Allocation Algorithm

[0069] To achieve reasonable task allocation, the system constructs a cost matrix. Where n is the number of tasks, m is the number of machines, and the matrix elements are... The calculation formula is: .

[0070] These are weighting coefficients set according to task requirements and system objectives, and they satisfy... For example, when a task has high timeliness requirements, the time limit can be increased. The value of makes task priority take up a larger proportion in the allocation. Representative task The priority is determined by keywords in the LLM parsing instructions, with a value range of [0,1]. If words such as "urgent" or "important" appear in the instructions, The value will increase accordingly. Indicates machine The current load is calculated by monitoring indicators such as CPU utilization, memory usage, and the number of assigned tasks on the machine, and then normalizing the result. The higher the value, the higher the load. It is a machine Execute the task The estimated time is based on the execution time of similar tasks under similar conditions in historical task data, and is evaluated in combination with the complexity of the current task.

[0071] By solving optimization problems The task allocation scheme is obtained, in which For 0-1 decision variables, when When =1, it indicates a task. Assigned to machine Otherwise, no allocation is made. The final output is the task-machine allocation set. Clearly define the specific tasks that each machine needs to perform.

[0072] 2.2 Intelligent Classification and Surface Modeling Module for Environmental Features (process as follows) Figure 3 (As shown)

[0073] 2.2.1 Input

[0074] Task-Machine Allocation Set: This set clearly defines the task of each machine, guiding the focus of environmental data collection. For example, if a machine... Those assigned to transport supplies will have their environmental data collection in the relevant areas along their transport routes prioritized.

[0075] Set of constraints: The accuracy requirements and time constraints will affect the selection of parameters and data processing strategies for subsequent surface modeling.

[0076] Environmental sensor data: d represents the feature dimension. The data comes from various devices such as LiDAR, vision cameras, temperature and humidity sensors, and barometric pressure sensors, and includes information such as the geometry of the environment, object features, and meteorological conditions.

[0077] Original sample: It consists of historical environmental sensor data, manually labeled terrain categories (such as mountains, plains, and water bodies), and obstacle information (such as the location and size of buildings, trees, and gullies), and is the basic dataset for training the random forest feature classification model.

[0078] 2.2.2 Random Forest Feature Classification

[0079] A random forest consists of N decision trees. Composition, a single decision tree is constructed using the Gini index. Select the splitting attribute. t represents a tree node; K is the total number of categories. In environmental feature classification, categories may include "grassland," "rocks," "roads," "water," etc. It is the proportion of category k samples in node t.

[0080] Model building and training: from raw samples Bootstrap sampling with replacement is performed to generate N training sets, with the number of samples sampled each time being equal to the number of samples in the training set. The same. This sampling method can increase the model's generalization ability, enabling it to adapt to different data distributions. For each training set, randomly select... A decision tree is constructed based on the features. The goal is to minimize the Gini index by continuously splitting the nodes until a stopping condition is met, such as when the number of node samples is less than a preset threshold (e.g., 10 samples) or the Gini index cannot be reduced further.

[0081] Model application: Using environmental sensor data Input the random forest model, and use the formula Determine the classification results, among which This is an indicator function. Final output:

[0082] Terrain type collection It may include "grassland", "rocky area", "sandy area", etc.;

[0083] Obstacle probability This indicates the probability that an obstacle is detected; the higher the value, the greater the probability that an obstacle is present.

[0084] Ground accessibility score This value is used to quantify the suitability of the ground for machine passage; a higher value indicates better accessibility.

[0085] Obstacle probability calculation:

[0086] Random forest outputs prediction probabilities for multiple categories The category probabilities associated with obstacles are obtained by weighted summation. Assume the obstacle category is The corresponding weight is ,but ,in .

[0087] Ground accessibility score calculation:

[0088] Construct a feasibility evaluation function ,in The relevant features output by the random forest (such as terrain type, slope, obstacle probability, etc.) are determined by training with historical data, so that the function output value... Mapped to the [0,100] interval, i.e.

[0089] ;

[0090] in For the normalization function, Mapped to Interval.

[0091] 2.2.3 Surface Fitting Optimization

[0092] Based on the set of terrain types output by the random forest Choose an appropriate fitting method:

[0093] Rocky region: Local polynomial surface fitting is used, the formula is:

[0094] ;

[0095] Where n² is the degree of the polynomial, determined by the set of constraints. The accuracy requirements in the text are dynamically adjusted (e.g.) , It is the initial value of the polynomial degree, a basic value set without considering task constraints; It is a weighting coefficient used to adjust the degree of influence of accuracy requirements on the degree of the polynomial. Its value is set according to the actual situation to balance factors such as accuracy requirements and computational costs. Task constraints Medium precision requirements The normalization result will meet the accuracy requirements. Mapping to a specific range (e.g., [0,1]) to eliminate the influence of different orders of magnitude of precision requirements, making the precision requirements of different tasks comparable when adjusting the polynomial degree; if the terrain precision requirement is high, the value of n 2 is increased to improve the accuracy of the fit. The coefficients are polynomials, which are solved using optimization algorithms such as the least squares method to make the fitted surface as close as possible to the actual terrain.

[0096] Grassland: Fitted using a smooth spline surface, the formula is:

[0097] ;

[0098] As basis functions, Using the coefficients, the least squares method is also used to solve the problem with the goal of minimizing the sum of squared errors between the actual measured data points and the fitted surface, resulting in a smooth surface model suitable for relatively flat and gently changing terrain.

[0099] The final output is a surface model with semantic labels. This model not only includes the geometric shape information of the environment, but also labels the semantic attributes of each region (such as...). And S), such as terrain type, accessibility, risk level, etc., provide a detailed environmental description for subsequent modules.

[0100] Among them, the surface model with semantic labels It is a unified representation that integrates environmental geometric features and semantic attributes, and its mathematical expression is: ; This represents the total number of discrete points in three-dimensional space.

[0101] Geometry section: These are the coordinates of discrete points in three-dimensional space, forming the geometric framework of the surface;

[0102] Semantic tag section: For point The corresponding set of semantic attributes includes:

[0103] Terrain type: (e.g., "rocky area", "grassland");

[0104] Pipeline score: (Provided by random forest classification results);

[0105] Risk level: (Calculated based on slope, obstacle probability, etc.);

[0106] Other attributes: such as environmental parameters like humidity and temperature.

[0107] In some embodiments, the risk level of a point in the surface model calculate: Risk level The higher the value, the higher the risk. These represent the obstacle probability, accessibility score, and slope of point k2, respectively.

[0108] 2.3 Task semantic enhancement and risk assessment module (process as follows) Figure 4 (As shown)

[0109] 2.3.1 Input

[0110] Task semantic vector: The LLM output from the task semantic understanding and planning module is a vectorized semantic representation of natural language task instructions, containing key information such as the task's objectives and constraints.

[0111] Surface model: It comes from the intelligent classification and surface modeling module for environmental features and contains geometric and semantic information about the environment.

[0112] Historical mission data: It stores the execution status of previous tasks, including task instructions, allocation schemes, environmental data during execution, task results, and other information, which are used for model training and reference.

[0113] 2.3.2 Random Forest Risk Prediction

[0114] Constructing input feature vectors ,in Indicates to The feature portion obtained after feature extraction. From historical mission data Relevant features extracted from the data, such as risk profiles in similar environments during historical tasks and task execution success rates, are then used in conjunction with a random forest prediction function. Calculate the probability of risk And based on the importance of each feature in the random forest model, through the formula Determine the set of key influencing factors The algorithm identifies the K1 factors that have the greatest impact on task risk, such as "severe weather" and "complex terrain." FeatureImportance (RF) refers to the feature importance evaluation in the Random Forest (RF) algorithm.

[0115] In some embodiments, risk probability Incorporating into the computational model and : , The sigmoid function maps the output to [0,1]. Weight vector It represents the reciprocal of accessibility and is positively correlated with risk. The superscript T indicates transpose.

[0116] 2.3.3 LLM Strategy Generation

[0117] The task semantic information and risk assessment report are concatenated and then input into the LLM, i.e. Based on the input, LLM leverages its powerful language understanding and generation capabilities to output task adjustment suggestions. Examples include "replacing the task execution device", "adjusting the task execution time", and "changing the task execution path".

[0118] 2.4 Multi-machine collaborative decision-making module (process as follows) Figure 5 (As shown)

[0119] 2.4.1 Path Planning Algorithm

[0120] The input includes a set of task-machine assignments. Surface model Risk probability and set of key influencing factors .

[0121] The improved A* algorithm is used for path planning, with the heuristic function being: .in, This represents the Euclidean distance from node n3 to the target point. Its weighting coefficient; The drivability score for node n3 comes from the intelligent classification and surface modeling module for environmental features. Its weighting coefficient; Based on risk probability Dynamic adjustment, the formula is: , For adjustment coefficients, when At higher levels, increase The value of makes path planning more inclined to select areas with good accessibility and low risk. The final output is the initial set of paths. For each machine, a candidate path is planned from its current location to the target location of the mission.

[0122] 2.4.2 Conflict Detection and Resolution

[0123] Collision detection employs a spatiotemporal grid method, dividing time and space into grids to define a spatiotemporal grid. The system detects path conflicts by determining whether multiple machines occupy the same spatial grid within the same time slice.

[0124] If a conflict is detected, the game theory payoff function is used. To resolve conflicts. The payoff function value of machine i represents its overall payoff (or cost) in conflict resolution; the larger the value, the better the strategy. The task completion time of machine i refers to the time required to execute the current task path, in seconds (s) or other time units. The path risk level of machine i is determined by environmental characteristics (such as obstacle probability, terrain slope, etc.), and the value range is usually [1,5]. The larger the value, the higher the risk. The conflict cost of machine i represents the potential loss due to path conflict (such as collision risk, task delay, etc.), and its value is positively correlated with the probability of conflict and the severity of the consequences.

[0125] Parameters are based on the set of key influencing factors. Dynamic adjustment, such as:

[0126] ;

[0127] ;

[0128] ;

[0129] in, Time weighting coefficient reflects the priority of task timeliness, and its value is positively correlated with the urgency of the task. Risk weighting coefficient reflects the priority of path safety, and its value is positively correlated with the environmental risk level. : Conflict weighting coefficient, which reflects the priority of avoiding machine collisions, and its value is positively correlated with the system security policy. , , They are respectively , , The initial value. To adjust the coefficients, values ​​are assigned based on specific key influencing factors. By solving the Nash equilibrium of the game theory model (this invention does not improve the solution process; refer to existing techniques for the specific solution process), the optimal adjustment strategy for each machine is determined, and the adjusted path is output. This ensures that multiple machines avoid conflicts and work collaboratively during mission execution.

[0130] Mapping rules based on key impact factors , , The value must match The specific factors in the example are as follows:

[0131] like Includes "Mission Urgency Level": Emergency missions (such as rescue): =+0.3 (increase time weight) =-0.2 (Temporarily reduce risk weight to speed up execution). Regular tasks: =0, =0 (using basic weights).

[0132] like Includes "high-risk terrain (such as steep slopes, areas with dense obstacles)": Risk level ≥ 4: =+0.4 (Reduce risk weight to avoid overly conservative path planning). =-0.3 (Increase conflict cost weighting, force obstacle avoidance). Risk level ≤ 2: =-0.1 (enhancing risk aversion) =0.

[0133] like Includes "excessive machine density": dense scenarios (such as multi-machine collaborative transportation): =+0.5 (Reduces the weight of conflict costs, allowing slight path overlap). Sparse scenario: =0.

[0134] Dynamic adjustment range and constraint value interval: usually limited To avoid excessive weight fluctuations that could cause the strategy to fail. Mutual exclusion constraint: When >0 (time priority) and Both cannot be negative simultaneously (to prevent ignoring risks and conflicts). When <0 (strong conflict avoidance) ≤0.2 (balancing efficiency and safety).

[0135] Utilizing historical task data, optimization can be achieved through reinforcement learning (such as Q-learning) or genetic algorithms. , , The parameters are optimized to achieve the best overall performance in terms of multi-machine conflict rate and task completion time.

[0136] III. System Data Closed-Loop Mechanism

[0137] Task Semantic Understanding Module → Environment Modeling Module: Task-Machine Assignment Set Guide the sensor data acquisition range, ensuring the sensor focuses on collecting data from areas relevant to task execution; set of constraints.

[0138] Parameters affecting surface fitting, such as high accuracy requirements, will lead to the use of higher polynomial degrees in surface fitting. Environment Modeling Module → Risk Assessment Module: Surface Models with Semantic Tags and random forest classification results The input feature vector that constitutes the random forest risk prediction model This provides environmental baseline data for risk assessment.

[0139] Risk Assessment Module → Decision Module: Risk Probability Used to adjust weights in path planning algorithms Key Influencing Factors Set Adjusting game theory model parameters; task adjustment suggestions It directly triggers path replanning, enabling the decision-making module to optimize task execution strategies based on risk conditions.

[0140] Decision module → Task semantic module (feedback): Task execution result E (success or failure) and machine status Update historical mission data ; The updates are used to optimize the LLM and Random Forest models, improving their accuracy and adaptability; meanwhile, the execution feedback is also used to adjust the task allocation cost matrix. Parameters, such as adjusting load weights based on the actual performance of the machine, enable continuous optimization and improvement of the system.

[0141] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0142] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

[0143] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0144] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0145] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0146] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0147] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a “unit,” “module,” or “system.” Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0148] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0149] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0150] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.

Claims

1. A multi-machine cooperative scheduling system for a man-machine control shelter, characterized in that, The application comprises: a task semantic understanding and planning module for receiving natural language task instructions, parsing task semantic vectors, task target sets and constraint condition sets, and constructing a cost matrix based on task priority, machine load and task estimated time, and obtaining a task-machine allocation set by solving an optimization problem; an environment feature intelligent classification and surface modeling module connected with the task semantic understanding and planning module, receiving the task-machine allocation set and the constraint condition set, combining environment sensor data and original samples, obtaining a terrain type set, obstacle probability and ground passability score through random forest feature classification, and generating a surface model with semantic labels according to the terrain type; a task semantic enhancement and risk assessment module connected with the environment feature intelligent classification and surface modeling module, receiving the surface model with semantic labels, combining task semantic vectors and historical task data, obtaining a risk probability and a key influence factor set through random forest risk prediction, and generating task adjustment suggestions through a large language model; The multi-machine cooperative decision module is connected with the task semantic enhancement and risk assessment module, receives the task adjustment suggestion, the risk probability and the key influence factor set, combines the task-machine allocation set and the curved surface model with semantic labels, and performs path planning through an improved A An algorithm is used to perform path planning to obtain an initial path set, a space-time grid method is used to perform conflict detection, and a game theory payoff function is used to perform conflict resolution to obtain an adjusted path. wherein the execution feedback result of the multi-machine collaborative decision module is used to update the historical task data, thereby optimizing the task semantic understanding and planning module and the task semantic enhancement and risk assessment module; The improved A The heuristic function of the algorithm dynamically adjusts the weight according to the risk probability, combines the distance from the node to the target point and the passability score of the node, and plans an initial path set for each machine. wherein the improved A The heuristic function of the algorithm is , represents the Euclidean distance from node n3 to the target point, is the weight coefficient thereof; is the passability score of node n3, from the environmental feature intelligent classification and surface modeling module, is the weight coefficient thereof.

2. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, In the task semantic understanding and planning module, a large language model is used to parse natural language task instructions to obtain task semantic vectors, task target sets and constraint condition sets, wherein the task target set includes specific tasks such as material transportation and environment monitoring, the constraint condition set includes time limit, equipment limit and precision requirement, and the task semantic vector is a vectorized semantic representation of the natural language task instruction, containing task target and constraint condition.

3. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, The random forest feature classification in the environment feature intelligent classification and surface modeling module is composed of multiple decision trees, a training set is generated by sampling the original samples with replacement, a decision tree is constructed by randomly selecting part of the features, the Gini index minimization is used as the basis for node splitting, and finally the terrain type set, obstacle probability and ground passability score are determined through a voting mechanism.

4. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 3, characterized in that, The terrain type set obtained by the random forest feature classification adopts local polynomial surface fitting for rock areas and smooth spline surface fitting for grasslands to generate a surface model with semantic labels, which contains geometric shape information of the environment and semantic attributes such as terrain type, passability and risk level.

5. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, In the task semantic enhancement and risk assessment module, the random forest risk prediction constructs an input feature vector with the task semantic vector, features extracted from the surface model with semantic labels and features extracted from historical task data, calculates the risk probability through the random forest model, and determines the key influence factor set according to the feature importance.

6. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, In the task semantic enhancement and risk assessment module, the task semantic information and the risk assessment result are spliced and input into the large language model to generate task adjustment suggestions, which include replacing task execution equipment, adjusting task execution time and changing task execution path.

7. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, In the multi-machine cooperative decision module, time and space are divided into grids by using a space-time grid method, conflict detection is performed by judging whether multiple machines occupy the same space grid in the same time slice, if a conflict is detected, the adjustment strategy of each machine is determined by solving Nash equilibrium according to the dynamic adjustment of parameters based on the key influence factor set by using the payoff function of game theory, and the adjusted path is obtained.

8. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, The execution feedback result of the multi-machine cooperative decision module includes task execution result and machine state, which is used to update historical task data, and further optimize the large language model analysis in the task semantic understanding and planning module and the task allocation algorithm, and the random forest risk prediction model and the large language model strategy generation in the task semantic enhancement and risk assessment module.

9. The multi-machine cooperative scheduling system of the man-machine control shelter according to claim 1, characterized in that, The system further comprises a communication module for realizing data transmission between the task semantic understanding and planning module, the environment feature intelligent classification and curved surface modeling module, the task semantic enhancement and risk assessment module, and the multi-machine cooperative decision module.

Images