Automatic deployment method of robot system

Abstract

The invention provides an automatic deployment method for a robot system, and the method comprises the steps: collecting three-dimensional space data of a target environment through a multi-mode sensor array which comprises a laser radar, a binocular vision camera and an inertial measurement unit; fusing sensor data based on a space-time synchronization algorithm to generate an environment feature matrix; analyzing a user task instruction, extracting a task target and decomposing the task target into time, path priority and safety threshold parameters; constructing a multi-objective optimization model, and generating a candidate path set; adopting an improved Pareto leading edge screening algorithm to select an optimal path sequence; executing virtual verification and adjusting path deviation in the digital twin environment; and the physical execution mechanism is driven to implement deployment operation, and the local path re-planning module is triggered when the deviation is detected to exceed a threshold value. According to the invention, the accuracy, safety and efficiency of path planning can be improved, and the energy consumption and time cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and more specifically, to an automated deployment method for a robot system. Background Art

[0002] The automated deployment of robot systems has wide applications in fields such as industry, service, and healthcare. Existing deployment methods usually rely on single-sensor data or simple path planning algorithms, and it is difficult to handle multi-objective optimization problems in complex dynamic environments. Traditional path planning methods often focus on a single objective, such as the shortest path or the lowest energy consumption, while ignoring safety and dynamic adjustment for real-time environmental changes. In addition, the fusion of multi-modal sensor data in the prior art usually adopts fixed weights and cannot be dynamically adjusted according to environmental changes, resulting in a significant decline in the accuracy and robustness of path planning when visual data is blurred or the environmental dynamics are relatively high.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: existing methods are difficult to achieve multi-objective optimization in complex dynamic environments, the flexibility and adaptability of sensor data fusion are insufficient, and the real-time performance and safety of path planning need to be improved. Summary of the Invention

[0004] The present invention provides an automated deployment method for a robot system, including:

[0005] S100. Collect three-dimensional spatial data of the target deployment environment through a multi-modal sensor array, where the multi-modal sensor array includes a lidar, a binocular vision camera, and an inertial measurement unit;

[0006] S200. Perform dynamic feature extraction on the three-dimensional spatial data, and fuse lidar point cloud data, visual semantic segmentation results, and inertial measurement trajectories based on a spatio-temporal synchronization algorithm to generate an environmental feature matrix;

[0007] S300. Analyze the task instructions input by the user, extract task objectives through a semantic understanding model, and decompose the task objectives into time constraint parameters, path priority parameters, and safety threshold parameters based on a dynamic weight allocation algorithm;

[0008] S400. Construct a multi-objective optimization model, input the environmental feature matrix and the decomposed task parameters into the model, and generate a set of candidate paths that meet the requirements of the shortest time, the lowest energy consumption, and the minimum safety risk;

[0009] S500. Select an optimal path sequence from the set of candidate paths by using an improved Pareto front screening algorithm, where the improvement includes introducing an environmental dynamicity evaluation index and a task priority constraint mechanism;

[0010] S600. Perform virtual verification in the digital twin environment, calculate the path feasibility index through the physical simulation engine, and adjust the path deviation by comparing the actual environmental sensor data;

[0011] S700. Drive the physical execution mechanism to perform the deployment operation. When the detected deviation between the actual path and the planned path exceeds the threshold, trigger the local path replanning module based on the restricted Boltzmann machine.

[0012] As a further improvement of this application, the fusion formula of the spatio-temporal synchronization algorithm in step S200 is:

[0013] F = α·L + β·V + γ·I

[0014] where F is the fused environmental feature matrix; L is the grid map converted from the lidar point cloud; V is the semantic segmentation map extracted from the visual image; I is the pose trajectory constructed by the inertial measurement unit; α, β, and γ are dynamically adjusted fusion weight coefficients, and when the detected blur degree B of the visual image ≥ B threshold then automatically reduce β and increase α, and B threshold is the preset blur threshold.

[0015] As a further improvement of this application, the improved Pareto front screening algorithm in step S500 includes:

[0016] Define the environmental dynamic evaluation index D e ;

[0017] According to the environmental dynamic evaluation index D e dynamically adjust the population diversity parameter. When D e > D threshold then increase the crossover probability based on the logarithmic transformation value of the environmental dynamic.

[0018] As a further improvement of this application, the dynamic weight allocation algorithm in step S300 includes:

[0019] Allocate the initial weight according to the keyword frequency in the task instruction, and adjust the weight through the real-time environmental data feedback. Specifically: when it is detected that the obstacle density increases, adjust the weight w s of the safety threshold parameter to w′ s :

[0020]

[0021] where N obs is the current number of obstacles; N max is the preset maximum obstacle capacity of the environment; w s is the initial safety weight parameter.

[0022] As a further improvement of the present application, the energy function of the local path replanning module in step S700 is defined as:

[0023]

[0024] where v is a visible layer node, representing the current environmental feature vector; h is a hidden layer node, representing candidate path parameters; a i is the bias term of the i-th node in the visible layer; b j is the bias term of the j-th node in the hidden layer; w ij is the connection weight between the visible layer and the hidden layer.

[0025] As a further improvement of the present application, step S600 includes:

[0026] Calculating the deviation value between the virtual path success rate P v and the actual path success rate P r by the dual-domain consistency detection module;

[0027] When ΔP > the preset deviation threshold ε, trigger the re-fusion of the environmental feature matrix and iteratively update the digital twin model.

[0028] As a further improvement of the present application, the objective function of the multi-objective optimization model in step S400 is:

[0029] min f(X) = [f1(X), f2(X), f3(X)]

[0030] where f1(X) is the path length function, calculating the total length of the candidate path set X; is the energy consumption function, where E k is the unit energy consumption of the k-th segment of the path, and t k is the execution time; is the safety risk function, where d i is the Euclidean distance from path point i to the nearest obstacle, and λ i is the obstacle danger coefficient.

[0031] As a further improvement of the present application, the dynamic adjustment rule of the fusion weight coefficient is:

[0032] When the visual image blur degree B ≥ B threshold , it is set that:

[0033]

[0034] At the same time, α is increased proportionally:

[0035] Δα = (1 - β′)(α + γ)

[0036] Among them, B is the measured value of the visual image blurriness; B threshold is the blurriness threshold; k is the weight decay coefficient; β′ is the adjusted visual data weight; Δα is the increment of the lidar weight.

[0037] As a further improvement of the present application, the calculation of the environmental dynamic evaluation index D e further includes:

[0038] Weight the acceleration a of the moving obstacle, and the corrected index is:

[0039]

[0040] Among them, a is the instantaneous acceleration of the obstacle; a max is the preset maximum acceleration threshold; D e is the original environmental dynamic evaluation index.

[0041] As a further improvement of the present application, it further includes an optimization phase after deployment:

[0042] S800. Collect the energy consumption E real and time T real of the actual deployment, and the number of safety incidents N safe ;

[0043] S900. Update the parameters of the multi-objective optimization model through the online learning module.

[0044] According to the above embodiments of the present invention, it has at least the following beneficial effects: Through the dynamic fusion and spatio-temporal synchronization algorithm of multi-modal sensor data, the present invention can improve the accuracy and robustness of environmental perception. Especially in the case of blurred visual data or high environmental dynamics, it can adaptively adjust the sensor weights to ensure the accurate generation of the environmental feature matrix. At the same time, based on the dynamic weight allocation algorithm and the multi-objective optimization model, it can comprehensively consider the path length, energy consumption, and safety risks to generate an optimal path sequence to meet the multi-dimensional requirements of complex deployment tasks.

[0045] In addition, through the improved Pareto front screening algorithm and virtual verification in the digital twin environment, the present invention can further improve the feasibility and adaptability of path planning to ensure that path deviations in the actual deployment process are adjusted in a timely manner. The application of the local path replanning module can quickly respond when a path deviation exceeds the threshold, improving the safety and reliability of the system. Finally, through the online learning mechanism in the optimization phase after deployment, the model parameters can be continuously optimized to keep the system efficient and stable in long-term operation. Brief Description of the Drawings

[0046] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown by way of example and not limitation, wherein:

[0047] Figure 1 It is a schematic flowchart of an automated deployment method for a robot system provided in an embodiment of the present invention. Specific embodiments

[0048] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and then implement the present invention, and not to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to convey the scope of the present invention fully to those skilled in the art.

[0049] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, equipment, method, or computer program product. Therefore, the present invention can be specifically implemented in the following forms: completely hardware, completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0050] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0051] The following reference Figure 1 , Figure 1 is a schematic flowchart of an automated deployment method for a robot system provided in an embodiment of the present invention. As Figure 1 shown, an automated deployment method for a robot system includes:

[0052] S100. Collect three-dimensional spatial data of the target deployment environment through a multi-modal sensor array, where the multi-modal sensor array includes a lidar, a binocular vision camera, and an inertial measurement unit;

[0053] S200. Perform dynamic feature extraction on the three-dimensional spatial data, and generate an environmental feature matrix by fusing lidar point cloud data, visual semantic segmentation results, and inertial measurement trajectories based on a spatio-temporal synchronization algorithm;

[0054] S300. Parse the task instructions input by the user, extract the task objectives through a semantic understanding model, and decompose the task objectives into time constraint parameters, path priority parameters, and safety threshold parameters based on a dynamic weight allocation algorithm;

[0055] S400. Construct a multi-objective optimization model, input the environmental feature matrix and the decomposed task parameters into the model, and generate a set of candidate paths that meet the requirements of the shortest time, the lowest energy consumption, and the minimum safety risk;

[0056] S500. Adopt an improved Pareto front screening algorithm to select the optimal path sequence from the set of candidate paths. The improvement includes introducing an environmental dynamic evaluation index and a task priority constraint mechanism;

[0057] S600. Perform virtual verification in the digital twin environment, calculate the path feasibility index through a physical simulation engine, and adjust the path deviation by comparing the actual environmental sensor data;

[0058] S700. Drive the physical execution mechanism to implement the deployment operation. When the deviation between the actual path and the planned path is detected to exceed the threshold, trigger the local path replanning module based on the restricted Boltzmann machine.

[0059] It should be noted that the present invention collects three-dimensional spatial data of the target deployment environment through a multi-modal sensor array, and the multi-modal sensor array includes a lidar, a binocular vision camera, and an inertial measurement unit. The lidar is used to obtain high-precision distance information, the binocular vision camera is used to capture the visual information of the environment and perform semantic segmentation, and the inertial measurement unit is used to record the pose change of the robot. The combination of these sensors can comprehensively perceive the geometric structure and dynamic changes of the environment, providing reliable data support for subsequent path planning and deployment.

[0060] Specifically, the lidar generates point cloud data by emitting laser beams and receiving reflected signals. The point cloud data can be converted into a grid map, which is used to represent obstacles and feasible regions in the environment. The binocular vision camera captures images through two cameras, generates depth information using stereo vision technology, and combines semantic segmentation algorithms to identify the object categories in the environment. The inertial measurement unit measures the acceleration and angular velocity of the robot through an accelerometer and a gyroscope, and then estimates the pose trajectory of the robot. The data of these sensors are fused under a spatio-temporal synchronization algorithm to generate an environmental feature matrix, which comprehensively combines the geometric information, semantic information, and dynamic change information of the environment.

[0061] Preferably, the spatio-temporal synchronization algorithm can dynamically adjust the fusion weights according to the data quality of the sensors and the environmental dynamics. For example, when the blur degree of the visual image is relatively high, the algorithm will automatically reduce the weight of the visual data and increase the weight of the lidar data to ensure the accuracy of the environmental feature matrix. In addition, the pose trajectory of the inertial measurement unit can be used to correct the time deviation of the sensor data, ensuring the consistency of multi-source data in time and space. Through this dynamic adjustment mechanism, the system can maintain a high-precision environmental perception ability in complex environments.

[0062] In some embodiments, the fusion formula of the spatio-temporal synchronization algorithm in step S200 is as follows:

[0063] F = α·L + β·V + γ·I

[0064] where F is the fused environmental feature matrix; L is the grid map converted from lidar point cloud; V is the semantic segmentation map extracted from visual images; I is the pose trajectory constructed by the inertial measurement unit; α, β, and γ are dynamically adjusted fusion weight coefficients, satisfying α + β + γ = 1, and when the detected visual image blur B ≥ B threshold , β is automatically reduced and α is increased.

[0065] It should be noted that the present invention fuses lidar point cloud data, visual semantic segmentation results, and inertial measurement trajectories through a spatio-temporal synchronization algorithm to generate an environmental feature matrix. The spatio-temporal synchronization algorithm is a technology for processing multi-source sensor data, which can ensure the data consistency of different sensors in time and space. The lidar point cloud data provides the geometric information of the environment, the visual semantic segmentation result provides the semantic information of the environment, and the inertial measurement trajectory provides the pose information of the robot. By dynamically adjusting the fusion weight coefficients, the algorithm can flexibly adjust the contribution degrees of each sensor data according to the quality of the sensor data and the changes in the environment, ensuring the accuracy and robustness of the environmental feature matrix.

[0066] Specifically, the lidar point cloud data generates a point cloud by scanning the environment, and the point cloud data can be converted into a grid map to represent obstacles and feasible regions in the environment. The visual semantic segmentation result is obtained by processing visual images through a deep learning model to identify different object categories in the image, such as pedestrians, vehicles, etc. The inertial measurement trajectory is the robot's motion trajectory deduced from the acceleration and angular velocity data recorded by the inertial measurement unit. The fusion weight coefficients are used to adjust the contribution ratio of each sensor data in the environmental feature matrix, and the sum of the weight coefficients is 1, and they can be dynamically adjusted according to environmental changes. For example, when the visual image blur is high, the weight of the visual data will be automatically reduced, and the weight of the lidar data will increase accordingly.

[0067] Preferably, the spatio-temporal synchronization algorithm can adjust the fusion weight coefficients in real time according to environmental changes. For example, when it is detected that the visual image blur exceeds a preset threshold, the algorithm will automatically reduce the weight of the visual data and increase the weight of the lidar data to ensure the accuracy of the environmental feature matrix. In addition, the inertial measurement trajectory can be used to correct the time deviation of the sensor data to ensure the consistency of multi-source data in time and space. Through this dynamic adjustment mechanism, the system can maintain high-precision environmental perception ability in complex environments and adapt to different environmental changes.

[0068] In some embodiments, the improved Pareto front screening algorithm in step S500 includes:

[0069] (a) Define the environmental dynamics evaluation index

[0070]

[0071] where ΔO is the change in the position of the obstacle within a preset time window; Δt is the length of the time window;

[0072] (b) Dynamically adjust the population diversity parameter according to D e When D e >D threshold the crossover probability is increased from P c to

[0073] P′ c =P c ·(1 + log(D e ))

[0074] where P c is the initial crossover probability; log(D e ) is the natural logarithm transformation value of the environmental dynamics with the natural logarithm as the base.

[0075] It should be noted that the present invention selects the optimal path sequence from the candidate path set through the improved Pareto front screening algorithm. The Pareto front screening algorithm is a multi-objective optimization algorithm used to find a balanced solution among multiple optimization objectives. The improved algorithm introduces an environmental dynamics evaluation index and a task priority constraint mechanism. The environmental dynamics evaluation index is used to measure the change speed of obstacles in the environment, and the task priority constraint mechanism is used to ensure that high-priority tasks are processed first. By introducing these improvements, the algorithm can better adapt to the dynamic environment and generate a path sequence that meets the task requirements.

[0076] Specifically, the environmental dynamics evaluation index measures the environmental dynamics by calculating the change in the position of the obstacle within a preset time window. The length of the time window can be adjusted according to the actual application scenario. A shorter window length can more sensitively reflect the dynamic changes of the environment, while a longer window length can smooth the noise. The task priority constraint mechanism assigns an initial weight based on the keyword frequency in the task instruction and dynamically adjusts the weight according to the real-time environmental data feedback. For example, when the obstacle density is detected to increase, the weight of the safety threshold parameter will automatically increase to ensure the safety of the path.

[0077] Preferably, the improved Pareto front screening algorithm can dynamically adjust the population diversity parameter according to the environmental dynamics evaluation index. For example, when the environmental dynamics is high, the algorithm will automatically increase the crossover probability to improve the diversity of the population, so as to better adapt to the changes in the environment. In addition, the task priority constraint mechanism can dynamically adjust the weights according to the real-time environmental data feedback to ensure that high-priority tasks are processed first. Through this dynamic adjustment mechanism, the algorithm can generate an optimal path sequence that meets the task requirements in a complex dynamic environment and improve the efficiency and adaptability of path planning.

[0078] In some embodiments, the dynamic weight allocation algorithm in step S300 includes:

[0079] Allocate initial weights according to the keyword frequency in the task instruction and adjust the weights through real-time environmental data feedback. Specifically, when it is detected that the obstacle density increases, the weight w of the safety threshold parameter s is adjusted to:

[0080]

[0081] where N obs is the current number of obstacles; N max is the preset maximum obstacle capacity of the environment; w s is the initial safety weight parameter.

[0082] It should be noted that the present invention decomposes the task objective into time constraint parameters, path priority parameters and safety threshold parameters through the dynamic weight allocation algorithm. The dynamic weight allocation algorithm is a method for dynamically adjusting weights according to task instructions and environmental feedback, which can ensure that the task objective is reasonably decomposed under different environmental conditions. The time constraint parameter is used to control the execution time of the task, the path priority parameter is used to determine the importance of path selection, and the safety threshold parameter is used to evaluate the safety of the path. Through dynamic weight allocation, the system can adjust the weights of each parameter according to real-time environmental data to ensure that the task objective is optimally decomposed under different environmental conditions.

[0083] Specifically, the time constraint parameter can be set according to the urgency of the task. For a more urgent task, a shorter time constraint can be set to ensure that the task is completed on time. The path priority parameter can be allocated according to the keyword frequency in the task instruction. The task corresponding to the high-frequency keyword can be set with a higher path priority. The safety threshold parameter is used to evaluate the safety of the path. When it is detected that the obstacle density increases, the weight of the safety threshold parameter will automatically increase to ensure the safety of the path. The initial weights of these parameters can be allocated according to the task instruction and dynamically adjusted through real-time environmental data feedback.

[0084] Preferably, the dynamic weight allocation algorithm can dynamically adjust the weights according to real-time environmental data feedback. For example, when the detected obstacle density increases, the algorithm will automatically increase the weight of the safety threshold parameter to ensure the safety of the path. In addition, the path priority parameter can be dynamically adjusted according to the keyword frequency in the task instruction to ensure that high-priority tasks are processed first. Through this dynamic adjustment mechanism, the system can reasonably decompose the task objectives in a complex dynamic environment and generate an optimal path sequence that meets the task requirements.

[0085] In some embodiments, the energy function of the local path replanning module in step S700 is defined as:

[0086]

[0087] where v is a visible layer node representing the current environmental feature vector; h is a hidden layer node representing the candidate path parameters; a i is the bias term of the i-th node in the visible layer; b j is the bias term of the j-th node in the hidden layer; w ij is the connection weight between the visible layer and the hidden layer.

[0088] It should be noted that in the present invention, when the local path replanning module detects that the deviation between the actual path and the planned path exceeds the threshold, it triggers path replanning. The local path replanning module optimizes the path based on the energy function of the restricted Boltzmann machine, which is a probabilistic generative model for processing high-dimensional data and can effectively handle complex environmental features and path parameters. The energy function is used to evaluate the matching degree between the current environmental features and the candidate path parameters. By minimizing the energy function, the system can generate an optimal path that conforms to the environmental features.

[0089] Specifically, the energy function is composed of visible layer nodes and hidden layer nodes. The visible layer nodes represent the current environmental feature vector, and the hidden layer nodes represent the candidate path parameters. The bias term of the visible layer nodes is used to adjust the weight of the environmental features, the bias term of the hidden layer nodes is used to adjust the weight of the path parameters, and the connection weight between the visible layer and the hidden layer is used to represent the relationship between the environmental features and the path parameters. By adjusting these parameters, the system can generate an optimal path that conforms to the environmental features. When it is detected that the deviation between the actual path and the planned path exceeds the threshold, the system will trigger the local path replanning module, recalculate the energy function, and generate a new path.

[0090] Preferably, the local path replanning module can dynamically adjust the parameters of the energy function according to changes in environmental characteristics. For example, when significant changes in environmental characteristics are detected, the system automatically adjusts the bias terms of the visible layer nodes to reflect the new environmental characteristics. In addition, the bias terms of the hidden layer nodes can be dynamically adjusted according to changes in path parameters to ensure that the generated path conforms to the current environmental conditions. Through this dynamic adjustment mechanism, the system can quickly respond to path deviations in a complex dynamic environment and generate an optimal path that conforms to environmental characteristics.

[0091] In some embodiments, step S600 includes:

[0092] Calculating the success rate P of the virtual path through the dual-domain consistency detection module v And the success rate P of the actual path r Of the deviation value:

[0093] ΔP = |P v P r |

[0094] Wherein, P v Is the success rate of path execution in the digital twin environment; P r Is the success rate feedback by the actual environment sensor; when ΔP > ε, trigger the re-fusion of the environmental feature matrix and iteratively update the digital twin model.

[0095] It should be noted that the present invention calculates the deviation value between the success rate of the virtual path and the success rate of the actual path through the dual-domain consistency detection module, and triggers the re-fusion of the environmental feature matrix when the deviation exceeds the preset threshold. The dual-domain consistency detection module is used to compare the virtual path execution result in the digital twin environment with the sensor feedback result in the actual environment to ensure the accuracy and feasibility of path planning. The success rate of the virtual path refers to the probability of successful path execution simulated in the digital twin environment, and the success rate of the actual path refers to the probability of successful path execution feedback by the sensor in the actual environment. By calculating the deviation value between the two, the system can evaluate the accuracy of path planning and re-fuse the environmental feature matrix when the deviation is large to update the digital twin model.

[0096] Specifically, the success rate of the virtual path can simulate the path execution process in the digital twin environment through a physical simulation engine, and calculate the feasibility and success rate of the path. The success rate of the actual path is evaluated through the sensor feedback data in the actual environment. The deviation value is calculated by comparing the difference between the success rate of the virtual path and the success rate of the actual path. When the deviation value exceeds the preset threshold, the system will trigger the re-fusion of the environmental feature matrix. The re-fused environmental feature matrix will update the digital twin model to ensure the consistency between the virtual environment and the actual environment.

[0097] Preferably, the dual-domain consistency detection module can dynamically adjust the deviation threshold according to environmental changes. For example, in an environment with high dynamics, the system can set a lower deviation threshold to ensure the accuracy of path planning. In addition, the re-fusion of the environmental feature matrix can dynamically adjust the fusion weights according to the quality of sensor data, ensuring that the updated digital twin model can accurately reflect the changes in the actual environment. Through this dynamic adjustment mechanism, the system can maintain the accuracy and feasibility of path planning in a complex dynamic environment and improve the robustness of the system.

[0098] In some embodiments, the objective function of the multi-objective optimization model in step S400 is:

[0099] minf(X)=[f1(X),f2(X),f3(X)]

[0100] where f1(X) is the path length function, which calculates the total length of the candidate path set X; is the energy consumption function, where E k is the unit energy consumption of the k-th segment of the path, and t k is the execution time; is the safety risk function, where d i is the Euclidean distance from path point i to the nearest obstacle, and λ i is the obstacle danger coefficient.

[0101] It should be noted that the present invention generates a candidate path set that satisfies the shortest time, the lowest energy consumption, and the minimum safety risk through a multi-objective optimization model. The multi-objective optimization model is a mathematical model used to optimize multiple objectives simultaneously, which can comprehensively consider factors such as path length, energy consumption, and safety risk in path planning. The path length function is used to calculate the total length of the candidate path, the energy consumption function is used to evaluate the energy consumption during the path execution, and the safety risk function is used to evaluate the safety of the path. By optimizing these objective functions, the system can generate a set of candidate paths that meet multi-dimensional requirements for subsequent screening and selection.

[0102] Specifically, the path length function calculates the total length of each path in the candidate path set and selects a shorter path to reduce the execution time. The energy consumption function evaluates the total energy consumption of the path by calculating the product of the unit energy consumption of each segment of the path during the path execution and the execution time, and selects a path with lower energy consumption to reduce energy consumption. The safety risk function evaluates the safety of the path by calculating the product of the distance from the path point to the nearest obstacle and the obstacle danger coefficient, and selects a path with lower safety risk to ensure the safety of the path. These objective functions are comprehensively optimized through the multi-objective optimization model to generate a set of candidate paths that meet multi-dimensional requirements.

[0103] Preferably, the multi-objective optimization model can dynamically adjust the weights of each objective function according to task requirements. For example, in a task with tight time constraints, the system can increase the weight of the path length function and preferentially select shorter paths. In a task sensitive to energy consumption, the system can increase the weight of the energy consumption function and preferentially select paths with lower energy consumption. In a task with high security requirements, the system can increase the weight of the safety risk function and preferentially select paths with lower safety risks. Through this dynamic adjustment mechanism, the system can generate an optimal path set that meets multi-dimensional requirements according to task needs, and improve the flexibility and adaptability of path planning.

[0104] In some embodiments, the dynamic adjustment rule of the fusion weight coefficient is as follows:

[0105] When the visual image blur degree B≥B threshold , it is set that:

[0106]

[0107] At the same time, α is increased proportionally:

[0108] Δα=(1β′)(α + γ)

[0109] where B is the measured value of the visual image blur degree; B threshold is the blur degree threshold; k is the weight decay coefficient; β′ is the adjusted weight of the visual data; Δα is the increment of the lidar weight.

[0110] It should be noted that the present invention ensures that when the visual image blur degree is high, the weight of the visual data is reduced and the weight of the lidar data is increased by dynamically adjusting the fusion weight coefficient. The fusion weight coefficient is used to adjust the contribution ratio of multi-modal sensor data in the environmental feature matrix. The visual image blur degree refers to the decrease in clarity of the visual image due to factors such as illumination and occlusion. When the visual image blur degree exceeds the preset threshold, the system will automatically reduce the weight of the visual data and increase the weight of the lidar data to ensure the accuracy and robustness of the environmental feature matrix.

[0111] Specifically, the visual image blur degree can be measured by an image processing algorithm, and the blur degree threshold can be set according to the actual application scenario. When the visual image blur degree exceeds the threshold, the weight of the visual data will decay according to an exponential function, and the weight of the lidar data will increase accordingly. The lidar data generates a point cloud by scanning the environment, and the point cloud data can be converted into a grid map to represent obstacles and feasible areas in the environment. By dynamically adjusting the fusion weight coefficient, the system can still maintain the accuracy of the environmental feature matrix when the quality of the visual data deteriorates.

[0112] Preferably, the dynamic adjustment of the fusion weight coefficient can be carried out in real time according to environmental changes. For example, in an environment with poor lighting conditions, the system can set a lower threshold for visual image blurriness to ensure timely adjustment of weights when the quality of visual data deteriorates. In addition, the weight of lidar data can be dynamically adjusted according to the quality of point cloud data, ensuring that its weight is increased when the quality of point cloud data is high. Through this dynamic adjustment mechanism, the system can maintain the accuracy and robustness of the environmental feature matrix in complex environments and adapt to different environmental changes.

[0113] In some embodiments, the calculation of the environmental dynamic evaluation index D e further includes:

[0114] Weight the acceleration a of the moving obstacle, and the corrected index is:

[0115]

[0116] where a is the instantaneous acceleration of the obstacle; a max is the preset maximum acceleration threshold; D e is the original environmental dynamic evaluation index.

[0117] It should be noted that the present invention corrects the environmental dynamic evaluation index by weighting the acceleration of the moving obstacle to more accurately reflect the dynamic changes of the environment. The environmental dynamic evaluation index is used to measure the change speed of obstacles in the environment, and the acceleration of the moving obstacle refers to the change amount of the speed of the obstacle per unit time. By incorporating acceleration into the calculation of the environmental dynamic evaluation index, the system can more sensitively reflect the dynamic changes of obstacles in the environment, thereby improving the adaptability and accuracy of path planning.

[0118] Specifically, the acceleration of the moving obstacle can be measured in real time through sensor data, and the maximum acceleration threshold can be set according to the actual application scenario. The corrected environmental dynamic evaluation index can more accurately reflect the dynamic changes of obstacles in the environment by weighting the original index with the acceleration. The original environmental dynamic evaluation index measures the dynamic nature of the environment by calculating the change amount of the obstacle position within a preset time window, and the corrected index further considers the acceleration of the obstacle, enabling the system to more sensitively respond to rapid changes in the environment.

[0119] Preferably, the corrected environmental dynamic evaluation index can dynamically adjust the weight according to environmental changes. For example, in an environment where the obstacle moves at a relatively high speed, the system can increase the weight of acceleration to improve the sensitivity to dynamic environmental changes. In addition, the maximum acceleration threshold can be dynamically adjusted according to the actual application scenario to ensure that the dynamic changes of obstacles can be accurately reflected in different environments. Through this dynamic adjustment mechanism, the system can more accurately evaluate the environmental dynamicity in a complex dynamic environment and generate an optimal path sequence that conforms to environmental changes.

[0120] In some embodiments, it further includes a post-deployment optimization stage:

[0121] S800. Collect the energy consumption E real 、time T real and the number of safety events N safe ;

[0122] S900. Update the parameters of the multi-objective optimization model through the online learning module, and the update formula is:

[0123]

[0124] where θ old is the current model parameter; θ new is the updated model parameter; ρ is the learning rate; T plan is the expected time of the planned path; is the gradient of the loss function J, and the loss function includes energy consumption and safety weights.

[0125] It should be noted that in the post-deployment optimization stage of the present invention, the energy consumption, time, and number of safety events during actual deployment are collected, and the parameters of the multi-objective optimization model are updated through the online learning module. The post-deployment optimization stage refers to the stage of data collection and analysis of energy consumption, time, and safety events during the execution process after the robot system is actually deployed. The online learning module is a mechanism that can dynamically adjust the model parameters according to actual data. By analyzing the differences between the actual deployment data and the planned data, the parameters of the multi-objective optimization model are updated to improve the accuracy and efficiency of subsequent path planning.

[0126] Specifically, the energy consumption during actual deployment can be used to monitor the energy consumption of the robot system in real time through sensors, the time can be used to record the total time of task execution through a timer, and the number of safety events can be used to detect safety-related events through sensors or a log recording system. The online learning module calculates the gradient of the loss function by comparing the differences between the actual data and the planned data, and dynamically adjusts the model parameters according to the learning rate. The learning rate is used to control the step size of model parameter update. A larger learning rate can accelerate the model convergence speed, and a smaller learning rate can improve the model stability.

[0127] Preferably, the online learning module can dynamically adjust the learning rate according to the quality of the actual deployment data. For example, in an environment with high data noise, the system can set a smaller learning rate to improve the stability of model parameter updates. In addition, the loss function can dynamically adjust the weights according to the task requirements. For example, in a task sensitive to energy consumption, the system can increase the weight of energy consumption to ensure that the model gives priority to energy consumption optimization in subsequent path planning. Through this dynamic adjustment mechanism, the system can continuously optimize the model parameters during long-term operation, improving the accuracy and adaptability of path planning.

[0128] The above embodiments of the present invention have the following beneficial effects: By collecting three-dimensional spatial data of the target deployment environment through a multi-modal sensor array, the present invention can improve the comprehensiveness and accuracy of environmental perception. Especially in complex dynamic environments, it can effectively fuse the data of lidar, vision cameras, and inertial measurement units to generate a high-precision environmental feature matrix. The dynamic feature extraction based on the spatio-temporal synchronization algorithm can ensure the spatio-temporal consistency of multi-source data, avoiding errors caused by asynchronous sensor data. By parsing the task instructions through the semantic understanding model and the dynamic weight allocation algorithm, the task objectives can be decomposed into parameters such as time, path priority, and safety threshold, thus providing accurate inputs for the multi-objective optimization model.

[0129] The multi-objective optimization model can comprehensively consider path length, energy consumption, and safety risks to generate a set of candidate paths that meet multi-dimensional requirements. The improved Pareto front screening algorithm can further optimize the path selection process, enhancing the efficiency and adaptability of path planning. The virtual verification in the digital twin environment can detect potential problems in path planning in advance, and adjust the path deviation through the dual-domain consistency detection module to ensure the feasibility of actual deployment. The local path replanning module can respond quickly when the detected path deviation exceeds the threshold, enhancing the safety and robustness of the system. The online learning mechanism in the post-deployment optimization phase can continuously optimize the model parameters, enabling the system to maintain efficient and stable performance during long-term operation.

[0130] Furthermore, the storage medium of the implementation manner of the present application stores program instructions capable of implementing all the above methods. Among them, the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various implementation manners of the present application. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0131] The above description is only some preferred embodiments of the present invention and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the embodiments of the present invention.

Claims

1. An automated deployment method for a robot system, characterized in that, It includes the following steps: S100. Collect three-dimensional spatial data of the target deployment environment through a multi-modal sensor array, where the multi-modal sensor array includes a lidar, a binocular vision camera, and an inertial measurement unit; S200. Extract dynamic features from the three-dimensional spatial data, and fuse lidar point cloud data, visual semantic segmentation results, and inertial measurement trajectories based on a spatio-temporal synchronization algorithm to generate an environmental feature matrix; S300. Parse the task instructions input by the user, extract task objectives through a semantic understanding model, and decompose the task objectives into time constraint parameters, path priority parameters, and safety threshold parameters based on a dynamic weight allocation algorithm; S400. Construct a multi-objective optimization model, input the environmental feature matrix and the decomposed task parameters into the model, and generate a set of candidate paths that meet the requirements of the shortest time, the lowest energy consumption, and the smallest safety risk; S500. Select an optimal path sequence from the set of candidate paths using an improved Pareto front screening algorithm, where the improvement includes introducing an environmental dynamicity evaluation index and a task priority constraint mechanism; S600. Perform virtual verification in the digital twin environment, calculate the path feasibility index through a physical simulation engine, and adjust the path deviation by comparing the actual environmental sensor data; S700. Drive a physical execution mechanism to implement the deployment operation. When it is detected that the deviation between the actual path and the planned path exceeds the threshold, trigger a local path replanning module based on a restricted Boltzmann machine.

2. The automated deployment method according to claim 1, wherein The fusion formula of the spatio-temporal synchronization algorithm in step S200 is: F = α·L + β·V + γ·I Among them, F is the fused environmental feature matrix; L is the grid map converted from lidar point cloud; V is the semantic segmentation map extracted from visual images; I is the pose trajectory constructed by the inertial measurement unit; α, β, and γ are dynamically adjusted fusion weight coefficients, and when the detected blurriness B of the visual image is ≥ B threshold , β is automatically decreased and α is increased, and B threshold is the preset blurriness threshold.

3. The automated deployment method according to claim 1, wherein The improved Pareto front screening algorithm in step S500 includes: Define the environmental dynamic evaluation index D e ; Evaluate the environmental dynamic index D e Dynamically adjust the population diversity parameter. When D e > D threshold Based on the logarithmic transformation value of the environmental dynamics, increase the crossover probability.

4. The automated deployment method according to claim 1, characterized in that, The dynamic weight allocation algorithm in step S300 includes: Allocate initial weights according to the keyword frequencies in the task instructions, and adjust the weights through real-time environmental data feedback. Specifically, when an increase in obstacle density is detected, the weight w of the safety threshold parameter s is adjusted to w' s : Among them, N obs is the current number of obstacles; N max is the maximum obstacle capacity of the preset environment; w s is the initial safety weight parameter.

5. The automated deployment method according to claim 1, wherein The energy function of the local path replanning module in step S700 is defined as: Among them, v is a visible layer node, representing the current environmental feature vector; h is a hidden layer node, representing the candidate path parameter; a i is the bias term of the i-th node in the visible layer; b j is the bias term of the j-th node in the hidden layer; w ij is the connection weight between the visible layer and the hidden layer.

6. The automated deployment method according to claim 1, wherein Step S600 includes: Calculate the success rate P of the virtual path through the dual-domain consistency detection module v and the actual path success rate P r for the deviation value; When ΔP > the preset deviation threshold ε, trigger the re-fusion of the environmental feature matrix and iteratively update the digital twin model.

7. The automated deployment method according to claim 1, wherein The objective function of the multi-objective optimization model in step S400 is: minf(X) = [f1(X), f2(X), f3(X)] Among them, f1(X) is the path length function, which calculates the total length of the candidate path set X; is the energy consumption function, where E k is the unit energy consumption of the k-th path segment, and t k is the execution time; is the safety risk function, where d i is the Euclidean distance from path point i to the nearest obstacle, and λ i is the obstacle danger coefficient.

8. The automated deployment method according to claim 2, wherein The dynamic adjustment rule of the fusion weight coefficient is: When the visual image blur degree B ≥ B threshold , it is set that: Increase α proportionally at the same time: Δα = (1β′)(α + γ) Among them, B is the measurement value of the visual image blur; B threshold is the blur threshold; k is the weight attenuation coefficient; β′ is the adjusted weight of the visual data; Δα is the increment of the lidar weight.

9. The automated deployment method according to claim 3, wherein The environmental dynamic evaluation index D e The calculation further includes: Weight the acceleration a of the moving obstacle, and the corrected index is: where a is the instantaneous acceleration of the obstacle; a max is the preset maximum acceleration threshold; D e is the evaluation index of the original environmental dynamics.

10. The automated deployment method according to claim 1, wherein It also includes an optimization stage after deployment: S800, collect the energy consumption E of the actual deployment real , time T real and the number of security incidents N safe ; S900. Update the parameters of the multi-objective optimization model through an online learning module.

Images