Robot movement control method and system for intelligent inspection

Abstract

The invention discloses a robot movement control method and system for intelligent inspection, and belongs to the technical field of computers. According to the technical scheme provided by the embodiment of the invention, the problems that a traditional inspection robot is limited in sensing dimension, insufficient in dynamic coupling effect compensation and lack of dynamic strategy adjustment capability in a complex unstructured environment are solved; and the robot can maintain the trajectory tracking precision of the moving base and the operation stability of the tail end of the mechanical arm under severe working conditions such as abrupt slopes and muddy conditions, so that high-adaptability inspection operation is completed.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a robot movement control method and system for intelligent inspection. Background Technology

[0002] In the complex, unstructured environments of heavy industries such as metal mining and beneficiation, intelligent inspection robots need to simultaneously operate their robotic arms to complete delicate tasks such as equipment inspection and pipeline inspection while moving. These environments typically feature complex terrains such as steep slopes and muddy conditions, along with interference factors such as strong vibrations, drastic changes in temperature and humidity, and specific chemical substances, placing extremely high demands on the robots' collaborative operation capabilities.

[0003] Therefore, in industrial inspection environments with complex terrain and harsh working conditions, how to overcome the bottleneck of multi-source information fusion and achieve high-precision, highly adaptive collaborative control of the mobile base and robotic arm has become a key challenge restricting the technological development of the industry. Summary of the Invention

[0004] This application provides a robot mobility control method and system for intelligent inspection, which can achieve high-precision, highly adaptive collaborative control of the mobile base and the robotic arm. The technical solution is as follows: On the one hand, a robot movement control method for intelligent inspection is provided, the method comprising: The robot's multidimensional environmental perception information, on-body dynamic state information, and task control command flow are fused and processed to generate holographic operating state characteristics of the robot. Based on the holographic operating state characteristics and the task control command flow, an adaptive collaborative decision-making mechanism is used to plan the collaborative strategy between the mobile base and the robotic arm, resulting in a global collaborative control strategy and virtual dynamic model reconstruction command. Based on the global collaborative control strategy and the virtual dynamics model reconstruction instructions, the virtual dynamics model parameters of the robot's underlying control loop are configured online to obtain a task-adaptive virtual compliant dynamics model. Based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy, the robot's mobile base and robotic arm are subjected to integrated cooperative motion control to complete the inspection operation.

[0005] On the one hand, a robot mobile control system for intelligent inspection is provided, the system comprising: The fusion processing module is used to fuse the robot's multi-dimensional environmental perception information, on-body dynamic state information and task control command stream to generate holographic operating state features of the robot. The strategy planning module is used to plan the collaborative strategy between the mobile base and the robotic arm based on the holographic operating state characteristics and the task control command flow through an adaptive collaborative decision-making mechanism, so as to obtain the global collaborative control strategy and the virtual dynamic model reconstruction command. An online configuration module is used to configure the virtual dynamics model parameters of the robot's underlying control loop online based on the global collaborative control strategy and the virtual dynamics model reconstruction instructions, so as to obtain a task-adaptive virtual compliant dynamics model. The control module is used to perform integrated coordinated motion control of the robot's mobile base and robotic arm based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy to complete the inspection operation. Attached Figure Description

[0006] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This is a schematic diagram of the implementation environment of a robot movement control method for intelligent inspection provided in an embodiment of this application; Figure 2 This is a flowchart of a robot movement control method for intelligent inspection provided in an embodiment of this application; Figure 3 This is a partial flowchart of a robot movement control method for intelligent inspection provided in an embodiment of this application; Figure 4 This is a partial flowchart of another robot movement control method for intelligent inspection provided in an embodiment of this application; Figure 5 This is a partial flowchart of another robot movement control method for intelligent inspection provided in the embodiments of this application; Figure 6 This is a partial flowchart of another robot movement control method for intelligent inspection provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a robot mobile control system for intelligent inspection provided in an embodiment of this application. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0009] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0010] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain better results.

[0011] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge sub-models to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence.

[0012] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by the respective parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0013] Figure 1 This is a schematic diagram illustrating the implementation environment of a robot movement control method for intelligent inspection provided in this application embodiment. See also... Figure 1 The implementation environment may include a robot controller 110 and a server 140.

[0014] The robot controller 110 is connected to the server 140 via a wireless network or a wired network. The robot controller 110 has an application installed and running that supports robot motion control for intelligent inspection, and is used to control the robot.

[0015] Server 140 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 140 can provide background services for the applications running on robot controller 110.

[0016] Since this application involves a relatively complex calculation process, the computing power of the robot controller 110 may not be able to meet the requirements. In this case, the relevant data will be sent to the server 140, and the server 140 will execute the technical solution provided in the embodiment of this application.

[0017] In traditional intelligent inspection robot systems, the limited perception dimensions manifest as reliance on conventional vision or position sensors, failing to achieve deep fusion of multi-dimensional information such as motor current ripple, mechanical vibration spectrum, and ambient gas concentration. Furthermore, the multi-source perception data is not effectively aligned and weighted, resulting in insufficient early identification of potential equipment faults and environmental risks. Additionally, the separate control architecture of the mobile base and robotic arm lacks effective modeling and compensation for their dynamic coupling effects, leading to decreased positioning accuracy of the robotic arm's end effector during bumpy movement. Moreover, the solutions lack the ability to adjust dynamic collaborative strategies based on holographic perception, failing to optimize overall behavior in real time according to environmental changes, thus limiting the system's collaborative control capabilities in complex terrain and harsh working conditions.

[0018] For example, in inspection tasks on steep slopes at metal mining sites, robots need to move across muddy surfaces and operate their robotic arms to inspect pipelines. The environment is subject to vibration, temperature and humidity changes, and chemical interference. Traditional robot perception systems can only acquire visual images and fail to integrate vibration spectrum and gas concentration information, resulting in the failure to detect minute cracks on the pipeline surface in a timely manner. Simultaneously, the separate control of the mobile base and the robotic arm increases the positioning deviation of the robotic arm's end effector during bumpy conditions, leading to inaccurate contact at the detection points. The collaborative strategy fails to dynamically adjust according to terrain changes, causing unstable trajectory tracking of the mobile base and reduced operational accuracy of the robotic arm, ultimately affecting the completion of the inspection task.

[0019] If the above issues are not addressed, potential equipment malfunctions will not be identified early, increasing safety risks. Insufficient positioning accuracy of the robotic arm leads to the failure of fine inspection tasks, affecting inspection quality. The lack of dynamic adjustment capability in the overall machine coordination strategy makes the system poorly adaptable to harsh environments, potentially causing task interruptions and severely restricting the reliable application of intelligent inspection robots in complex industrial environments.

[0020] To address this, this application proposes a robot movement control method for intelligent inspection, see [link to relevant documentation]. Figure 2 Taking the server as the executing entity as an example, the following steps are included.

[0021] 201. The robot's multi-dimensional environmental perception information, body dynamic state information and task control command flow are fused and processed to generate holographic operating state characteristics of the robot.

[0022] 202. Based on the holographic operation status characteristics and task control command flow, the collaborative strategy planning of the mobile base and the robotic arm is carried out through an adaptive collaborative decision-making mechanism to obtain the global collaborative control strategy and virtual dynamic model reconstruction command.

[0023] 203. Based on the global collaborative control strategy and virtual dynamic model reconstruction instructions, the parameters of the virtual dynamic model are configured online for the robot's underlying control loop to obtain a task-adaptive virtual compliant dynamic model.

[0024] 204. Based on a task-adaptive virtual compliant dynamics model and a global cooperative control strategy, the robot's mobile base and robotic arm are integrated for cooperative motion control to complete inspection tasks.

[0025] This application relates to a robot mobility control method for intelligent inspection, wherein the robot's holographic operational state feature refers to a comprehensive state representation integrating multi-dimensional information from the environment, the robot itself, and the task. Specifically, this feature can be generated by dimensionality reduction processing of multi-source sensing data through principal component analysis, or by feature selection and fusion using support vector machines. For example, a state vector can be formed by statistically analyzing vibration spectrum and ambient gas concentration data collected by a sensor array, which is mainly used to characterize the robot's real-time operational status in complex industrial environments. Furthermore, the adaptive collaborative decision-making mechanism refers to the decision-making process of dynamically generating control strategies based on the operational state. It can use a rule-based expert system for strategy matching, or achieve adaptive parameter adjustment through a fuzzy logic controller. For example, it can output collaborative instructions based on a preset terrain type and task priority mapping table, which is mainly for achieving real-time strategy optimization of the mobile base and the robotic arm. As a preferred implementation method, the task-adaptive virtual compliant dynamics model refers to a control model that dynamically adjusts parameters according to operational requirements. It can use a lookup table method to select a preset parameter set based on the task type, or use a linear interpolation method to correct stiffness and damping parameters online. For example, in pipeline inspection tasks, a pre-stored compliant parameter configuration file can be called to adapt to the operational requirements of different pipe diameters. Its main purpose is to achieve dynamic adaptation of parameters in the underlying control loop.

[0026] Specifically, this method overcomes the problems of limited perception dimensions, insufficient compensation for dynamic coupling effects, and lack of dynamic strategy adjustment capabilities of traditional inspection robots in complex unstructured environments. By constructing holographic state features and adaptively configuring the parameters of the virtual model, the robot can maintain the trajectory tracking accuracy of the moving base and the operational stability of the robotic arm end effector under harsh working conditions such as steep slopes and mud, thereby completing highly adaptive inspection operations.

[0027] In the control of intelligent inspection robots in complex unstructured industrial environments, a mobile control method is implemented to address the problems of limited perception dimensions, insufficient compensation for dynamic coupling effects caused by separate control of the mobile base and robotic arm, and lack of dynamic collaborative strategy adjustment capabilities based on holographic perception. This method first fuses the robot's multi-dimensional environmental perception information, on-body dynamic state information, and task control command flow to generate holographic operational state features. The multi-dimensional environmental perception information encompasses terrain, semantic, and non-visual environmental data; the on-body dynamic state information reflects motor drive and mechanical vibration characteristics; and the task control command flow provides guidance for the operational phase. Through temporal alignment and task context weighting of the multi-modal features, the generated features can dynamically focus on the key dimensions of the current inspection task. Furthermore, based on the holographic operational state features and task control command flow, an adaptive collaborative decision-making mechanism is used to execute the collaborative strategy planning of the mobile base and robotic arm. This mechanism dynamically configures the weight coefficients of the multi-objective optimization function according to the situational deviation and task phase characteristics, generating a global collaborative control strategy and virtual dynamic model reconstruction instructions that balance control accuracy, motion stability, and system energy consumption. Subsequently, the online configuration of virtual dynamics model parameters is applied to the robot's underlying control loop. Based on the global cooperative control strategy and reconfiguration instructions, the dynamic stiffness parameters and variable damping parameters are normalized and range-constrained to obtain a task-adaptive virtual compliant dynamics model. This model can dynamically optimize parameters according to real-time interaction states. Finally, based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy, the integrated cooperative motion control of the mobile base and the robotic arm is executed. By distributing trajectory tracking control commands and impedance control commands to the corresponding control units, deep coupling between the two at the parameter level is achieved.

[0028] For example, in the inspection of steep slope terrain in metal mining and beneficiation environments, the task control command flow identifies the current climbing phase. The system prioritizes strengthening the weights of environmental terrain geometric features and mechanical vibration spectrum features, generating holographic operating state features focused on terrain adaptability. An adaptive collaborative decision-making mechanism automatically increases motion stability optimization weights based on the degree of deviation, planning a collaborative strategy to suppress the impact of bumps. Online configuration of virtual dynamic model parameters reduces dynamic stiffness parameters and adjusts variable damping parameters, forming a compliant control model that ensures the robotic arm's end effector maintains stable positioning when the mobile base moves on steep slopes. Thus, this method, by deeply fusing multi-source information to construct a task-oriented holographic state description, effectively expands the perception dimension and improves the early identification capability of potential equipment failures and environmental risks. Through dynamic configuration optimization of weights and online parameter adjustment, real-time compensation for the dynamic coupling effect between the mobile base and the robotic arm is achieved, significantly improving positioning accuracy in bumpy environments. The adaptive strategy generation and model reconstruction based on holographic perception ensure system stability under drastic temperature and humidity changes and vibration disturbances, thereby achieving high-precision, highly adaptive collaborative inspection operations in complex, unstructured industrial environments.

[0029] In practical applications, some embodiments of this application propose to fuse multidimensional environmental perception information, on-body dynamic state information, and task control command flow of the robot to generate holographic operating state features of the robot. However, in the implementation process, the feature extraction dimension of multi-source information is single, the time series is not synchronized, and the fusion strategy lacks task context adaptability. As a result, the generated holographic features cannot fully represent the dynamic risks and equipment status in complex unstructured environments, thereby weakening the accuracy and environmental adaptability of subsequent collaborative decision-making.

[0030] To address this, this application further proposes fusing multi-dimensional environmental perception information, on-body dynamic state information, and task control command flow of the robot to generate holographic operational state characteristics of the robot. (See [link to relevant documentation]). Figure 3 Taking the server as the executing entity as an example, the following steps are included.

[0031] 301. Multi-source feature extraction is performed on multi-dimensional environmental perception information to obtain environmental topographic geometric features, environmental semantic features, and non-visual environmental features.

[0032] 302. Perform time-frequency domain feature analysis on the dynamic state information of the robot to obtain the dynamic characteristics of the robot's motor drive system and the mechanical vibration spectrum characteristics.

[0033] 303. Perform multimodal feature fusion and time alignment processing on the task control command flow, environmental terrain geometric features, environmental semantic features, non-visual environmental features, dynamic features of motor drive system and mechanical vibration spectrum features to obtain the robot's holographic operating state features.

[0034] Multi-source feature extraction refers to the process of separating different types of features from environmental perception data. This can be achieved using deep learning models or traditional signal processing algorithms. For example, convolutional neural networks can be used to extract geometric features from visual data, natural language processing techniques can be applied to parse semantic information, and sensor fusion algorithms can be used to process non-visual data such as gas concentration and temperature / humidity parameters. The aim is to comprehensively capture multi-dimensional information about the environment and avoid the limitations of a single perception mode. Time-frequency domain feature analysis can be understood as a method of simultaneously analyzing signal features in the time and frequency domains. Specifically, techniques such as short-time Fourier transform, wavelet transform, or Hilbert-Huang transform can be used. The aim is to reveal the transient characteristics and periodic components of the signal, and it is particularly suitable for analyzing dynamic phenomena such as motor current ripple and mechanical vibration. Multimodal feature fusion and time alignment processing refers to the process of synchronizing and integrating feature data from different sources and in different formats. This can be achieved using timestamp-based interpolation alignment methods or dynamic time warping algorithms to process asynchronous data streams. Fusion strategies can include weighted fusion based on attention mechanisms or end-to-end fusion using deep learning models. The goal is to ensure consistency of multi-source information in the time dimension and generate high-fidelity state representations.

[0035] Specifically, the proposed solution employs a layered and progressive processing flow. First, it independently extracts features from environmental perception information and ontological state information, avoiding feature distortion caused by information mixing. Then, it temporally aligns and fuses the task control command flow with the extracted features, ensuring consistency across different sampling rates over time. Furthermore, it dynamically adjusts feature weights using task stage information, ensuring that the generated holographic operational state features closely align with the actual needs of the inspection task. This processing mechanism solves the problems of single-dimensionality, temporal asynchrony, and insufficient task adaptability in multi-source information fusion, providing a high-fidelity, task-oriented state representation foundation for subsequent collaborative decision-making.

[0036] In one possible implementation, the environmental perception unit includes a 3D LiDAR and a multispectral camera mounted on top of the robot for acquiring environmental data. The body state monitoring module is integrated into the robot chassis's vibration sensors and motor current monitoring circuitry. During multi-source feature extraction, LiDAR point cloud data is processed using point cloud processing algorithms to extract terrain geometric features, multispectral images are processed using deep convolutional neural networks to extract environmental semantic features, and gas sensor data is filtered to obtain non-visual environmental features. Time-frequency domain feature analysis uses wavelet transform to decompose the motor current signal, extracting dynamic features, and performs spectral analysis on the vibration signal to obtain mechanical vibration spectral features. In the fusion stage, a long short-term memory network is used to time-align the multimodal feature sequences, and an attention mechanism is applied based on task stage information for weighted fusion, ultimately generating the robot's holographic operating state features.

[0037] The above scheme generates holographic operational status features of the robot, which can comprehensively characterize the dynamic risks and equipment status in complex unstructured environments. This significantly improves the accuracy and environmental adaptability of subsequent collaborative decision-making and solves the problems of single dimension, asynchronous time series, and insufficient task context adaptability in multi-source information fusion.

[0038] In some of the embodiments described above in this application, a method for fusing task control command streams and multimodal features to generate holographic operating state features of a robot is proposed. However, in the implementation process, due to the inconsistency of timestamps from different feature sources and the lack of consideration of task context, the fused features cannot accurately reflect the current task requirements, affecting the accuracy and adaptability of subsequent collaborative control.

[0039] To address this, this application further proposes multimodal feature fusion and time alignment processing of task control command flow, environmental terrain geometry features, environmental semantic features, non-visual environmental features, dynamic features of motor drive system, and mechanical vibration spectrum features to obtain holographic operating state features of the robot, including: By aligning the timestamps and unifying the sampling rates of environmental topographic geometric features, environmental semantic features, non-visual environmental features, dynamic features of motor drive systems, and mechanical vibration spectrum features, a time-synchronized multimodal feature sequence is obtained.

[0040] Based on the task stage information contained in the task control instruction stream, the multimodal feature sequences are weighted and fused according to the task context to obtain a task-oriented fused feature vector.

[0041] The feature dimension of the task-oriented fusion feature vector is reduced and standardized to generate the robot's holographic operating state features.

[0042] Timestamp alignment and sampling rate unification refer to the time base calibration and sampling frequency normalization of multi-source heterogeneous sensor data. This can be achieved using global clock synchronization based on network time protocols or time resampling methods based on spline interpolation. The aim is to eliminate time misalignment caused by sensor transmission delays or differences in hardware sampling rates, ensuring accurate alignment of multimodal feature sequences on a unified time axis. Task context-weighted fusion refers to a fusion mechanism that dynamically adjusts feature weights based on task stages. This can be achieved using weight allocation based on task template matching or lightweight attention network models. The aim is to focus the fusion process on the key feature dimensions of the current task stage and avoid interference from irrelevant environmental noise. Feature dimensionality reduction and standardization refer to the removal of redundant information and unification of dimensions in high-dimensional feature vectors. This can be achieved using feature selection algorithms based on mutual information or maximum variance expansion dimensionality reduction combined with Z-score standardization. The aim is to extract task-related core features and eliminate dimensionality differences, generating a compact and standardized feature representation.

[0043] Specifically, the proposed solution first performs timestamp alignment and sampling rate unification on environmental terrain geometric features, environmental semantic features, non-visual environmental features, motor drive system dynamic features, and mechanical vibration spectrum features to establish a time consistency benchmark for cross-modal data, thereby obtaining a reliable time-synchronized multimodal feature sequence. Based on this, it utilizes the task stage information implicit in the task control command stream to dynamically identify the key feature requirements of the current inspection task and establish weight allocation relationships, performing task-oriented weighted fusion calculations on the multimodal feature sequence. Finally, it filters high-contribution feature dimensions through feature dimension reduction and eliminates the influence of dimensions through standardization, generating structured and task-adaptive robot holographic operating state features. This phased, progressive processing mechanism ensures a reliable conversion from raw multi-source data to precise task features, avoiding the cumulative effects of time distortion and task-irrelevant noise, and providing high-quality input for subsequent collaborative decision-making.

[0044] In one possible implementation of pipeline inspection tasks in a metal mining and beneficiation environment, the environmental perception unit uses a 3D LiDAR to acquire environmental terrain geometry features, a semantic segmentation network to extract environmental semantic features, and a gas sensor array to collect non-visual environmental features. The body state monitoring unit acquires dynamic features of the motor drive system through a Hall current sensor and collects mechanical vibration spectrum features through a MEMS vibration accelerometer. These feature data are sent to a central processing unit, which can be an embedded AI acceleration chip such as the NVIDIA Jetson AGX Xavier. First, timestamp alignment is performed, synchronizing the timestamps of each sensor using the PTP protocol and unifying them to a 100Hz sampling rate. Then, based on the "bend detection" stage information in the task control command stream, a dynamic weight allocation algorithm is used to weight and fuse the feature sequences. Finally, a feature selection module and a standardization unit are applied to generate a holographic operating state feature vector to drive subsequent collaborative decision-making mechanisms.

[0045] Through the above solution, this application solves the problem of feature distortion caused by the asynchronous time of multi-source features and the lack of task context, so that the generated holographic operation status features of the robot can accurately match the current inspection task requirements. It significantly improves the accuracy and adaptability of the coordinated control of the mobile base and the robotic arm in industrial environments with steep slopes, muddy terrain and strong vibration interference, and enables the robotic arm end effector to maintain stable positioning during the robot's bumpy movement.

[0046] In the complex, unstructured environments of heavy industries such as metal mining and beneficiation, intelligent inspection robots need to simultaneously operate their robotic arms to complete delicate tasks such as equipment inspection and pipeline inspection while moving. These environments typically feature complex terrains such as steep slopes and muddy conditions, accompanied by interference factors such as strong vibrations, drastic temperature and humidity changes, and specific chemical substances, placing extremely high demands on the robot's collaborative operation capabilities. In some embodiments described above, a task context-weighted fusion method based on task stage information is proposed to generate a task-oriented fusion feature vector. However, in its implementation, the lack of a dynamic identification mechanism for key task feature requirements leads to the inability to adjust the weight allocation of feature dimensions in real time according to changes in the task stage during multimodal feature fusion. This results in non-critical features being overemphasized while critical features are weakened, ultimately causing the fusion feature vector to become disconnected from the current task requirements, affecting the accuracy of the holographic operational status features and the reliability of subsequent collaborative decision-making.

[0047] In response, this application further proposes a step-by-step approach to obtain a task-oriented fusion feature vector by performing task context-weighted fusion of multimodal feature sequences based on task stage information contained in the task control instruction stream: Identify key features and requirements of the current task stage based on task stage information.

[0048] Based on the key feature requirements of the task, a dynamic weight allocation relationship is established for each feature dimension in the multimodal feature sequence.

[0049] Based on the dynamic weight allocation relationship, a weighted fusion calculation is performed on the multimodal feature sequences to obtain a task-oriented fusion feature vector.

[0050] Specifically, the key feature requirements of a task refer to the differentiated dependencies of multimodal perception features at different stages of an inspection task. This can be achieved using a task semantic parser or a task feature mapping table based on a rule engine. The aim is to accurately locate the core perception dimensions required for the current task stage, avoiding the blindness of feature importance judgment in traditional fixed-weight fusion. The dynamic weight allocation relationship refers to the set of feature dimension weight coefficients generated in real time based on the key feature requirements of the task. This can be implemented using a fuzzy logic controller or an adaptive weight matrix generation algorithm. The goal is to give key features higher priority during the fusion process, overcoming the shortcomings of static weight allocation in responding to dynamic environmental changes. In practical applications, weighted fusion calculation refers to the mathematical operation process of fusing multimodal feature sequences based on the dynamic weight allocation relationship. This can be implemented using a weighted average method or a deep learning fusion network. The goal is to generate highly task-oriented feature vectors, providing accurate input for subsequent holographic operational status feature generation.

[0051] Specifically, this application first identifies the key feature requirements of the current task stage based on the task stage information in the task control instruction stream, clarifying the differentiated dependencies of different task stages on multimodal perception features. For example, in the steep slope movement stage, priority is given to terrain geometric features, while in the equipment detection stage, environmental semantic features are emphasized. Then, based on the identified key feature requirements, a dynamic weight allocation relationship is established for each feature dimension in the multimodal feature sequence, giving higher weight coefficients to key feature dimensions. Finally, the time-synchronized multimodal feature sequence is weighted and fused according to this dynamic weight allocation relationship to generate a task-oriented fused feature vector. This step-by-step processing mechanism, through dynamic weight adjustment driven by task stage information, achieves adaptive optimization of multimodal feature fusion, ensuring that the fusion result closely matches the current task requirements and effectively avoiding feature distortion caused by task changes.

[0052] In inspection tasks within metal mining and beneficiation environments, when a robot enters the equipment inspection phase, the system parses the task control command stream to identify the current task phase as equipment inspection, thereby determining environmental semantic features as the key feature requirement. Based on this, the system establishes a dynamic weight allocation relationship, increasing the weight of environmental semantic features and decreasing the weight of terrain geometric features. Subsequently, based on this weight allocation relationship, a weighted fusion calculation is performed on the multimodal feature sequences to generate a fused feature vector dominated by environmental semantic features, which is used for subsequent holographic operational status feature generation.

[0053] The above technical solution achieves dynamic matching between the multimodal feature fusion process and the changes in task stages, avoiding the problem of feature requirement mismatch when switching task stages, improving the task orientation and environmental adaptability of holographic operating status features, thereby enhancing the robot's ability to identify potential equipment failures and environmental risks in complex industrial environments, and providing reliable support for high-precision collaborative control of the mobile base and robotic arm.

[0054] In practical applications, some of the embodiments described above in this application propose feature dimension reduction and standardization processing to generate robot holographic operating state features. However, in the implementation process, due to the high dimensionality of multi-source perception data in industrial inspection environments, strong noise interference, and dynamic changes in task requirements, there is a lack of intelligent filtering mechanism for feature dimensions. This leads to increased feature redundancy, increased computational burden, and the potential weakening or loss of key task information, which in turn affects the real-time performance and robustness of subsequent collaborative decision-making.

[0055] In this regard, this application further proposes steps including: The importance of the task-oriented fusion feature vectors is evaluated to obtain the contribution ranking of each feature dimension.

[0056] Based on contribution ranking, select the set of target feature dimensions whose contribution reaches a preset threshold.

[0057] The target feature dimension set is standardized and recombined to obtain the robot's holographic operating state features.

[0058] Specifically, feature importance assessment refers to the dynamic quantification of the contribution of each dimension in the task-oriented fusion feature vector to the current inspection task. It can be implemented using decision tree algorithms or random forest models based on task stage requirements. The purpose is to identify key features according to actual operational objectives and avoid the shortcomings of traditional fixed-weight methods that are sensitive to noise or ignore dynamic task priorities in complex environments.

[0059] Among them, selecting the target feature dimension set whose contribution reaches the preset threshold can be understood as adaptively filtering high-value features based on the contribution ranking results. This can be achieved by using dynamic threshold setting based on ranking percentage or hard threshold filtering based on absolute contribution value. The purpose is to significantly reduce data dimensionality while retaining key task information, thus resolving the conflict between limited computing resources and real-time requirements in industrial scenarios.

[0060] In practical applications, standardization and feature recombination specifically eliminate the dimensional differences between different features and optimize the feature structure. For example, Z-score standardization or Min-Max standardization methods can be used. The purpose is to make the input of subsequent decision models consistent and enhance the stability of holographic features under harsh conditions such as drastic changes in temperature and humidity.

[0061] Specifically, the proposed solution assesses the feature importance of task-oriented fused feature vectors, dynamically quantifying the contribution of each dimension to the current task. Further, it selects high-contribution features based on contribution ranking and adaptively prunes redundant dimensions. On this basis, the selected features are standardized and reorganized to optimize feature representation. Because the feature importance assessment dynamically identifies key features in conjunction with task-stage requirements, the assessment results are closely linked to actual operational objectives. Since the set of target feature dimensions with contribution reaching a preset threshold is selected and redundant dimensions are adaptively pruned based on the ranking results, key task information (such as vibration spectrum characteristics in steep terrain) is preserved while significantly reducing data dimensionality. Considering that standardization and feature reorganization eliminate dimensional differences and integrate relevant dimensions, the compact representation capability of holographic features is enhanced. Through the organic connection of the above steps, intelligent screening and optimized reorganization of high-dimensional multi-source sensing data are achieved, resolving the contradiction between information redundancy and the loss of key features.

[0062] In one possible implementation, during inspection operations in a metal mining environment, when the robot is in the pipeline inspection task phase, the task control command flow indicates that the current task phase is equipment inspection. Based on this, the feature importance assessment module analyzes the fused feature vector and identifies environmental semantic features and mechanical vibration spectrum features as having high contributions. Subsequently, the system selects a set of feature dimensions whose contributions exceed a preset threshold, such as retaining equipment fault identifiers from environmental semantic features and resonant frequency components from vibration spectrum features. Finally, these features are Z-score normalized, and the relevant dimensions are reorganized into a compact feature vector, which is then input as the robot's holographic operating state features to the collaborative decision-making mechanism.

[0063] Through the above scheme, this application effectively reduces feature dimension redundancy, lowers the computational burden of the underlying control loop, and ensures the complete preservation of key task information, thereby improving the real-time performance and robustness of subsequent collaborative decision-making. In particular, it maintains the high-precision collaborative control capability of the robot's mobile base and robotic arm in complex unstructured environments.

[0064] In some of the embodiments described above in this application, an adaptive collaborative decision-making mechanism is proposed to plan the collaborative strategy between the mobile base and the robotic arm to achieve global robot control. However, in its implementation, there is a lack of dynamic quantitative characterization of real-time situational changes in complex unstructured environments. This makes it impossible to accurately assess the deviation between the robot's current state and the expected target, and thus it is difficult to dynamically adjust and optimize the priority of the target according to the characteristics of the task stage and environmental interference. As a result, the collaborative strategy planning is prone to response lag or strategy failure under steep slopes, muddy terrain and vibration interference, affecting the positioning accuracy and system stability of the inspection operation.

[0065] In this regard, this application further proposes the following steps, see [link to relevant documentation]. Figure 4 Taking the server as the executing entity as an example, the following steps are included.

[0066] 401. Based on the holographic operating state characteristics, determine the robot's behavioral state space description, and calculate the state deviation between the robot's current state point and the desired state region in the behavioral state space description.

[0067] 402. Based on the situational deviation and task control command flow, configure a multi-objective optimization function with dynamic weight coefficients. Use the multi-objective optimization function to perform parallel optimization of multiple candidate strategy schemes to generate multiple cooperative strategy candidate schemes. Each cooperative strategy candidate scheme includes candidate control strategy and candidate model reconstruction parameters.

[0068] 403. Based on a multi-objective optimization function, the cooperative effectiveness of multiple cooperative strategy candidate schemes is quantitatively evaluated to obtain the evaluation results. Based on the evaluation results, the optimal cooperative strategy candidate scheme is selected from the multiple cooperative strategy candidate schemes, and the global cooperative control strategy and virtual dynamic model reconstruction instructions are extracted from the optimal cooperative strategy candidate scheme.

[0069] Among these, behavioral situation space description refers to abstracting the robot's multidimensional state information into a quantifiable spatial dimension. This can be achieved using dimensional abstraction based on holographic operational state characteristics, such as establishing a metric and semantic label for the abstract behavioral dimension through feature clustering analysis. Situational deviation refers to the quantified deviation between the robot's current state point and the desired state region. This can be achieved by combining geometric distance calculation with semantic difference assessment, such as determining the degree of state deviation through coordinate mapping in spatial topology. The multi-objective optimization function with dynamic weight coefficients refers to a mathematical model that dynamically adjusts the weights of optimization objectives based on task requirements and environmental states. This can be achieved using weight allocation algorithms based on task stage characteristics and situational deviation, such as dynamically generating weight coefficients for each optimization objective through a fuzzy logic system. Multiple candidate strategy schemes refer to combinations of various feasible control strategies generated through parallel optimization. This can be achieved using evolutionary algorithms or random sampling methods, such as generating different combinations of the mobile base and robotic arm trajectories through population evolution operations. Synergy effectiveness quantitative evaluation refers to the multi-dimensional quantitative evaluation of the comprehensive performance of candidate strategy schemes. This can be achieved by weighted combination of multiple evaluation indicators, such as weighting and integrating control accuracy, motion stability, and system energy consumption indicators according to dynamic weights. Selecting the optimal synergy strategy candidate scheme refers to the process of screening Pareto optimal solutions from the candidate scheme set. This can be achieved using non-dominated ranking or multi-criteria decision-making methods, such as ranking and selecting based on the output value of the synergy effectiveness evaluation function.

[0070] Specifically, this application's solution constructs a behavioral situation space description that transforms multi-dimensional information, integrating environmental perception, ontological state, and task commands, into a structured spatial dimension, providing a basic framework for situation assessment. Subsequently, situation deviation is calculated within this space, accurately capturing real-time deviations caused by terrain abrupt changes or vibration disturbances by quantifying the geometric distance and semantic differences between the current state point and the desired state region. A multi-objective optimization function with dynamic weight coefficients is configured based on situation deviation and task control command flow. Weight coefficients for optimization objectives such as control accuracy and motion stability are dynamically allocated according to task stage characteristics and the direction and magnitude of the deviation, allowing the priority of optimization objectives to adaptively adjust with environmental changes. This optimization function is used to solve multiple candidate strategy schemes in parallel, efficiently exploring the combination space of the mobile base's motion trajectory and the robotic arm's joint trajectory, generating a feasible solution set that satisfies dynamic coupling constraints. The collaborative effectiveness of candidate schemes is quantitatively evaluated, and weighted integration is performed using multi-dimensional indicators to comprehensively measure collaborative operation efficiency. Finally, based on the evaluation results, the optimal collaborative strategy candidate scheme is selected, and global collaborative control strategies and virtual dynamic model reconstruction commands are extracted, ensuring that the output control commands simultaneously meet the requirements of high-precision positioning and compliant interaction.

[0071] In one possible implementation, in a metal mining scenario, when the robot faces steep terrain, the behavior state spatial description module determines the spatial configuration, including dimensions such as terrain slope and vibration intensity, based on holographic operational state features. The state deviation calculation module detects a significant deviation of the current state point from the expected area, indicating a positioning risk. The dynamic weight configuration module increases the weight of the control accuracy objective based on task stage characteristics (such as critical detection stages) and the deviation direction. The multi-objective optimization module generates multiple candidate strategy schemes in parallel, including different combinations of adjusting the base speed and robotic arm posture. The collaborative performance evaluation module quantitatively scores each scheme and selects the scheme that achieves the best balance between positioning accuracy and stability. The final extracted global collaborative control strategy guides the robot to traverse the steep slope at a smoother speed while maintaining the stability of the robotic arm's end effector.

[0072] Through the above scheme, this application realizes the dynamic quantitative representation of real-time situational changes in complex unstructured environments. It can accurately assess the degree of deviation between the robot's current state and the expected target, and dynamically adjust and optimize the priority of the target according to the characteristics of the task stage and environmental interference. It solves the problem that the collaborative strategy planning is prone to response lag or strategy failure under steep slopes, muddy terrain and vibration interference, and significantly improves the positioning accuracy and system stability of inspection operations.

[0073] In the complex, unstructured environments of heavy industries such as metal mining and beneficiation, intelligent inspection robots need to simultaneously operate their robotic arms to complete delicate tasks such as equipment inspection and pipeline inspection while moving. These environments typically feature complex terrains such as steep slopes and muddy conditions, accompanied by strong vibrations, drastic temperature and humidity changes, and interference from specific chemical substances, placing extremely high demands on the robot's collaborative operation capabilities. Currently, this field faces the following major technical bottlenecks: First, traditional inspection robots have limited perception systems, relying heavily on conventional vision or position sensors, lacking deep integration of multi-dimensional information such as motor current ripple, mechanical vibration spectrum, and ambient gas concentration, resulting in insufficient early identification of potential equipment failures and environmental risks. Second, the control architecture often adopts a separate control mode for the mobile platform and the robotic arm, lacking effective modeling and compensation for the dynamic coupling effect between the two, making it difficult to maintain the positioning accuracy of the robotic arm's end effector during bumpy movement. Furthermore, related solutions generally lack forward-looking decision-making capabilities based on holographic perception, failing to dynamically adjust the overall collaborative strategy according to environmental changes. In some of the embodiments described above in this application, a behavior situation space description based on holographic operating state characteristics is proposed to calculate the situation deviation. However, in its implementation, the original data of multimodal perception features has high dimensionality, scattered semantics and contains environmental noise. Directly using it for behavior situation description can easily lead to dimensional redundancy and ambiguity of key behavioral elements. It is difficult to dynamically adapt to the needs of different inspection task stages, resulting in inaccurate situation deviation assessment, which in turn affects the accuracy and environmental adaptability of collaborative strategy planning.

[0074] In response, this application further proposes a spatial description of the robot's behavioral state based on holographic operational state characteristics, including: The multimodal perception features in the holographic operational state features are subjected to dimensional abstraction processing to obtain multiple abstract behavioral dimension definitions in the behavioral state space.

[0075] Based on the dynamic state features in the holographic operation state features, the real-time state quantization value of each abstract behavioral dimension is determined.

[0076] Based on the definition of abstract behavioral dimensions and the real-time state quantization values ​​of each abstract behavioral dimension, the behavioral state space description of the robot is determined.

[0077] In practical applications, dimensional abstraction processing refers to transforming raw high-dimensional multimodal perception data into low-dimensional, semantically clear behavioral description dimensions. This can be achieved using generalization techniques such as feature selection algorithms, principal component analysis, or autoencoder networks, aiming to eliminate environmental noise interference and focus on key task elements. The abstract behavioral dimension definition can be understood as a behavioral description unit with clear physical meaning. It can be defined based on task requirements as conceptual dimensions such as terrain adaptability, equipment proximity, or environmental risk, aiming to construct an interpretable behavioral representation framework. Specifically, real-time state quantification refers to the process of converting the robot's dynamic state into numerical indicators. This can be achieved using parallel methods such as motor current signal analysis, vibration spectrum feature extraction, or kinematic parameter calculation, aiming to provide a continuously measurable state representation. The behavioral situation space description can be understood as a structured behavioral state representation model, which can be implemented using higher-level representation methods such as vector space, graph structure, or topological space, aiming to establish a geometric description system of behavioral situations.

[0078] Specifically, the proposed solution transforms multimodal perception features into task-oriented abstract behavioral dimensions through dimensional abstraction. Combined with real-time quantization of dynamic state features, a structured behavioral situation space description is constructed. First, dimensional abstraction filters core behavioral elements based on task semantic information, avoiding noise interference and dimensional redundancy in the original perception data. Second, dynamic state features provide real-time measurable quantification indicators, enabling the behavioral description to dynamically respond to changes in the robot's motion state. Finally, the abstract behavioral dimension definition and real-time state quantification values ​​jointly construct a behavioral situation description with a spatial topological structure, giving behavioral situation assessment both logical structure and real-time dynamic characteristics. This process, through hierarchical abstraction and dynamic mapping mechanisms, achieves a reliable conversion from raw perception data to structured behavioral situations, providing a clear and accurate reference framework for calculating situation deviation.

[0079] In one possible implementation scenario, within a metal mine inspection scenario, the holographic operational status features include environmental gas concentration sensor data, mechanical vibration acceleration signals, and task control command streams. The dimensional abstraction processing module employs a feature selection algorithm to extract core behavioral elements relevant to the current pipeline inspection task from multimodal perception features, forming two abstract behavioral dimension definitions: "pipeline leakage risk" and "terrain passability." The measurement scale for "pipeline leakage risk" is based on semantic labels established using gas concentration feature cluster centers. The dynamic state feature processing unit utilizes the motor drive system current ripple signal to calculate the real-time state quantization value of "pipeline leakage risk" (0.75) and the quantization value of "terrain passability" (0.62). The behavioral situation space construction module maps these quantization values ​​to a predefined two-dimensional vector space, generating a situation description containing the spatial topology and current state coordinates. This description is used for subsequent situation deviation calculations. In this embodiment, the processing unit can specifically be an embedded ARM Cortex-M7 microcontroller, which receives multi-source sensor data via a CAN bus and performs dimensional abstraction processing using a lightweight neural network model.

[0080] The above technical solution addresses the issues of redundant multimodal perception feature dimensions and ambiguity in key behavioral elements, enabling behavioral situation descriptions to dynamically adapt to the needs of different inspection task stages. Specifically, dimensional abstraction eliminates environmental noise interference, ensuring accurate identification of key behavioral elements. The dynamic generation mechanism of real-time state quantification values ​​avoids description lag in bumpy environments. The structured behavioral situation space description provides a clear geometric reference framework for situation deviation calculation. This solution significantly improves the accuracy and environmental adaptability of situation assessment, laying a reliable foundation for the collaborative strategy planning of the mobile base and robotic arm.

[0081] In some of the embodiments described above in this application, a dimensional abstraction process is proposed to be performed on the multimodal perception features in the holographic operation status features to obtain multiple abstract behavioral dimension definitions of the behavioral situation space. However, in this process, the task semantic information is not combined for directional abstraction, which leads to the abstract behavioral dimension definitions being out of touch with the current inspection task requirements and failing to accurately capture the key features of the task (such as the need to focus on terrain stability for pipeline inspection and the need to focus on vibration sensitivity for equipment inspection). This makes the description of the behavioral situation space lack task specificity, thereby affecting the adaptability and decision accuracy of subsequent collaborative strategy planning.

[0082] In response, this application further proposes to perform dimensional abstraction processing on the multimodal perception features in the holographic operational state features, resulting in the definition of multiple abstract behavioral dimensions in the behavioral situation space, including: Based on the task semantic information in the task control instruction flow, the core behavioral elements related to the current inspection task are extracted.

[0083] Dynamic clustering analysis is performed on the multimodal perception features to obtain the feature cluster centers corresponding to each core behavioral element.

[0084] Based on feature cluster centers, we establish measurement scales and semantic labels for abstract behavioral dimensions, forming multiple abstract behavioral dimension definitions in the behavioral situation space.

[0085] Among these steps, extracting core behavioral elements related to the current inspection task based on task semantic information in the task control command flow refers to identifying key behavioral requirements directly related to the inspection task from the task instructions. This can be achieved by parsing keywords in the task instructions using natural language processing techniques or by matching based on a predefined task-behavior mapping table. The aim is to closely link the abstract dimensions with task requirements and avoid interference from irrelevant features. Dynamic clustering analysis of multimodal perception features to obtain feature cluster centers corresponding to each core behavioral element involves dynamically grouping multi-source perception data, such as environmental terrain geometric features and environmental semantic features, according to core behavioral elements. This can be achieved using density-based spatial clustering algorithms or hierarchical clustering methods. The aim is to provide data support for the abstract dimensions and adapt to dynamic environmental changes. Based on the feature cluster centers, establishing measurement scales and semantic labels for the abstract behavioral dimensions to form multiple abstract behavioral dimension definitions in the behavioral situation space involves transforming the clustering results into interpretable and operable engineering parameters. This can be achieved by defining the quantification range and semantic name of each dimension. The aim is to ensure that the description of the behavioral situation space reflects both task semantic requirements and directly guides subsequent collaborative decision-making.

[0086] Specifically, the proposed solution drives the dimensional abstraction process through task semantic information. First, it extracts core behavioral elements from the task control command flow as an abstraction guide, ensuring that the dimensional definition aligns with the inspection task requirements. Then, it performs dynamic clustering analysis on multimodal perception features, grouping the perception data according to core behavioral elements and determining feature cluster centers, thus providing dynamic support for the abstract dimensions from environmental data. Finally, it establishes metrics and semantic labels based on the feature cluster centers, transforming the abstract dimensions into a parameterized description with engineering significance. This task-oriented dimensional abstraction mechanism allows the definition of the behavioral situation space to be dynamically adjusted according to task requirements. It avoids the redundant interference caused by blindly processing multimodal features in traditional abstraction and overcomes the rigidity of fixed dimensional definitions in complex, unstructured environments, thereby providing a task-related basic framework for constructing the behavioral situation space.

[0087] As a specific implementation method, when the robot performs a pipeline inspection task, the task control command flow contains the semantic information of "pipeline inspection," and the system extracts "terrain stability" as the core behavioral element. Dynamic clustering analysis is performed on the multimodal perception features in the holographic operating state characteristics to obtain feature cluster centers related to terrain stability. Based on these cluster centers, a metric scale (such as stiffness parameter range) and semantic label for the "mobility stability" dimension are established, forming an abstract behavioral dimension definition of the behavioral situation space. In this embodiment, the system accurately identifies the special requirements of the pipeline inspection task for terrain stability through task semantic information and dynamically clusters multi-source perception data such as environmental terrain geometric features and mechanical vibration spectrum features, making the abstract dimension closely fit the actual inspection scenario.

[0088] The above technical solutions enable the definition of abstract behavioral dimensions to be closely integrated with the current inspection task requirements, accurately capturing key task characteristics, improving the task-specificity of behavioral situation space description, and enhancing the adaptability and decision-making accuracy of subsequent collaborative strategy planning.

[0089] In some of the embodiments described above in this application, a behavior situation space description of a robot is proposed based on the definition of abstract behavior dimensions and the real-time state quantization values ​​of each abstract behavior dimension. However, in its implementation, if the semantic correlation between the definitions of abstract behavior dimensions is not considered, the generated behavior situation space description may only present a simple superposition of discrete dimensions, which cannot reflect the inherent logical connection between multimodal features such as environmental terrain and mechanical vibration, resulting in distortion of the behavior situation space structure. This leads to deviation in the robot's current state coordinate positioning, affects the accuracy of the situation deviation calculation, and ultimately weakens the adaptability of cooperative strategy planning to complex unstructured environments.

[0090] In response, this application further proposes a spatial topological configuration for generating the behavioral situation space based on the semantic correlation between the definitions of each of the abstract behavioral dimensions.

[0091] The real-time state quantization values ​​of each of the abstract behavioral dimensions are mapped to the spatial topology configuration to determine the robot's current state coordinates in the behavioral situation space.

[0092] Based on the spatial topology and the current state coordinates, a behavioral situation spatial description is generated, which includes spatial structure and real-time state information.

[0093] Specifically, semantic relevance refers to the inherent logical relationship between the definitions of abstract behavioral dimensions. It can be achieved using graph structure modeling or semantic network technology. The purpose is to characterize the dynamic coupling relationship between dimensions such as environmental topographic geometric features and mechanical vibration spectrum features, and to avoid situational fragmentation caused by dimensional isolation.

[0094] Among them, spatial topology configuration can be understood as a structured geometric representation of behavioral state space. It can be constructed using manifold learning or topological data analysis methods. The purpose is to transform discrete dimensions into a high-dimensional space with continuous connectivity, so that the correlation logic between environmental semantic features and mechanical vibration features can be preserved.

[0095] In practical applications, the mapping of state quantization values ​​refers to the transformation of real-time measured feature parameters into coordinate positions in the topological space. This can be achieved using nonlinear embedding or coordinate transformation algorithms. The aim is to integrate time-frequency domain parameters such as dynamic features of the motor drive system, so that state localization can be integrated into the contextual relationships between dimensions.

[0096] Specifically, the proposed solution first generates a spatial topological configuration based on semantic relevance. This configuration, by defining the topological connections between abstract behavioral dimensions, transforms the inherent logic of dimensions such as environmental terrain geometry and mechanical vibration spectrum characteristics into a structured space. Subsequently, the real-time state quantization values ​​of each abstract behavioral dimension are mapped onto this topological configuration. Relying on the semantic constraints of the topological configuration, the discrete state quantization values ​​are integrated into a continuous coordinate system, ensuring that the current state coordinates not only reflect single-dimensional values ​​but also the contextual relationships between dimensions. Finally, a behavioral situation space description is generated based on the spatial topological configuration and the current state coordinates. This description, by solidifying the fusion of topological relationships and dynamic states, provides a structured input for calculating situation deviation, thereby enabling collaborative decision-making mechanisms to accurately identify state deviations in highly disruptive scenarios such as metal mining and beneficiation.

[0097] As a specific implementation, in a metal mining and beneficiation inspection scenario, when a robot travels on steep slopes, the system identifies the semantic correlation between environmental terrain geometric features (such as slope angle) and mechanical vibration spectrum features (such as vibration amplitude at specific frequencies). It then constructs a spatial topology configuration with the terrain and vibration dimensions serving as the robot controllers, where the connection strength between robot controllers reflects the coupling effect between the steep slope and the motor vibration spectrum. The real-time acquired quantized values ​​of the slope angle and vibration amplitude are mapped onto this topology configuration to determine the robot's current state coordinates in the behavioral situation space. Based on the spatial topology configuration and the current state coordinates, a behavioral situation space description is generated. This description includes topological connections and real-time state information, which is used for subsequent collaborative strategy planning.

[0098] The above technical solutions can accurately reflect the dynamic interaction patterns of multi-source information in industrial inspection environments, significantly improve the robot's behavioral situational awareness accuracy under muddy terrain or vibration interference, and enable the collaborative decision-making mechanism to accurately identify state deviations and dynamically adjust the collaborative strategy of the mobile base and the robotic arm, thereby enhancing the environmental adaptability and task reliability of inspection operations.

[0099] In some of the embodiments described above in this application, a multi-objective optimization function with dynamic weight coefficients is proposed for collaborative strategy planning. However, in its implementation, the weights of the optimization objectives are fixed or lack a dynamic adjustment mechanism, and cannot change in real time according to the characteristics of the task stage and the deviation of the situation. This makes it difficult to accurately balance the multi-objective requirements such as control accuracy, motion stability and system energy consumption when robots make collaborative decisions in complex unstructured environments. In particular, in scenarios such as metal mining and beneficiation, the frequent switching of task stages and sudden changes in environmental situation lead to insufficient strategy adaptability.

[0100] In response, this application further proposes a multi-objective optimization function based on situational deviation and mission control command flow, configuring dynamic weight coefficients, including: Based on the task stage characteristics in the task control instruction flow, multiple optimization objectives and their corresponding relative priority relationships are determined.

[0101] Based on the magnitude and direction of the deviation, the dynamic weight coefficients of each optimization objective are determined.

[0102] Based on multiple optimization objectives, relative priority relationships, and dynamic weight coefficients of each optimization objective, an objective function containing multiple weighted optimization objective terms is constructed.

[0103] Among them, task stage characteristics refer to the job progress identification information implicit in the task control command flow, which can be realized by task command semantic parsing or state machine state recognition, with the aim of dynamically associating the optimization objective with the actual evolution stage of the inspection task. The magnitude and direction of the situational deviation refer to the quantified deviation degree and spatial pointing characteristics between the robot's current state point and the desired state region in the behavioral situational space, which can be realized by Euclidean distance calculation or direction vector analysis, with the aim of objectively characterizing the impact of environmental disturbances on the robot's operational state. Dynamic weight coefficients refer to the optimization objective weight parameters that are adjusted in real time according to the situational deviation, which can be realized by nonlinear mapping functions or adaptive lookup table mechanisms, with the aim of establishing a dynamic coupling relationship between the environmental situation and the priority of the optimization objective.

[0104] Specifically, the proposed solution determines optimization objectives and priority relationships through task phase feature analysis, ensuring that optimization objectives such as control accuracy, motion stability, and system energy consumption closely align with the core needs of the current inspection phase. For example, during pipeline inspection, the weight of control accuracy is automatically increased, while during terrain crossing, the weight of motion stability is enhanced. Simultaneously, the weight coefficients of each optimization objective are dynamically adjusted based on the magnitude and direction of the situational deviation. When the situational deviation is large, the weight of motion stability is automatically increased to suppress vibration effects; when the situational deviation is small, the weight of control accuracy is emphasized to optimize end-point positioning. Finally, the dynamically determined optimization objectives, priority relationships, and weight coefficients are integrated to construct a weighted objective function, forming an adaptive decision-making framework driven by both the task phase and the environmental situation. This ensures that the multi-objective optimization process remains dynamically matched to the real-time operating conditions in the unstructured environment.

[0105] In one possible implementation, during pipeline inspection in a metal mining and beneficiation scenario, the task control command flow, after passing through a semantic parsing module, identifies that the system is currently in a high-precision detection phase. The system automatically sets control precision as the primary optimization objective and assigns it a high base weight. When the behavioral situation space detects that the robot's deviation is increased due to muddy terrain, the weight calculation unit increases the motion stability weight coefficient in real time while simultaneously reducing the system energy consumption weight coefficient. Finally, a weighted summation is used to generate an objective function containing three optimization objectives. This objective function is then input into the collaborative strategy planning module to generate a collaborative control strategy between the mobile base and the robotic arm adapted to the current working conditions.

[0106] The above technical solutions enable robots to automatically adjust and optimize target priorities during task phase switching, and dynamically balance the multi-objective requirements of control accuracy, motion stability and system energy consumption when environmental conditions change abruptly. This solves the problem of insufficient strategy adaptability caused by the static optimization function in complex unstructured environments, and significantly improves the robustness of inspection robots in collaborative operations on steep slopes and muddy terrain in scenarios such as metal mining and beneficiation.

[0107] In some of the embodiments described above in this application, a cooperative strategy planning method using a multi-objective optimization function is proposed. However, in its implementation, the optimization solution process suffers from low computational efficiency and insufficient solution quality. Specifically, the generation of candidate strategy schemes depends on a serial processing mechanism, which makes it difficult to respond in real time to dynamically changing environmental conditions and task requirements in complex unstructured environments. This results in the inability to efficiently obtain high-quality cooperative strategy schemes that meet multi-objective optimization, thereby affecting the real-time performance and robustness of the cooperative control of the mobile base and the robotic arm.

[0108] To address this, this application further proposes to generate multiple cooperative policy candidate schemes by performing parallel optimization of multiple candidate schemes using a multi-objective optimization function, including: Generate an initial strategy population containing different combinations of motion trajectories of the mobile base and joint trajectories of the robotic arm.

[0109] By using a multi-objective optimization function, the fitness of each policy scheme in the initial policy population is evaluated in parallel, and the comprehensive fitness value of each policy scheme under the multi-objective optimization function is obtained.

[0110] Based on the comprehensive fitness value, the initial policy population is iteratively updated through an elite retention strategy and population evolution operations, outputting a set of cooperative policy candidate schemes containing multiple non-dominated solutions. Each non-dominated solution corresponds to a cooperative policy candidate scheme that achieves Pareto optimality among multiple optimization objectives.

[0111] The initial strategy population refers to the initial solution set composed of various combinations of mobile platform motion trajectories and robotic arm joint trajectories. This can be achieved through random trajectory generation based on task stage information or reconstruction of historical strategy data. The aim is to construct a solution space covering the multi-dimensional possibilities of interaction between the mobile platform and the robotic arm, avoiding local optima caused by a single initial solution. Parallel fitness evaluation refers to the process of simultaneously calculating the performance of multiple strategy schemes under a multi-objective optimization function. This can be implemented using a multi-threaded parallel computing architecture or a distributed computing robot controller cluster, aiming to overcome the temporal limitations of serial evaluation and significantly shorten the fitness calculation cycle. The elite retention strategy refers to the mechanism of prioritizing the retention of high-fitness individuals during population iteration. This can be achieved through fixed-ratio selection or dynamic fitness threshold selection, aiming to prevent the loss of high-quality strategies during evolution. Population evolution operations refer to the operation of generating new solutions through strategy feature recombination. This can be achieved through crossover operations based on trajectory features and mutation operations based on environmental perturbations, aiming to maintain population diversity and explore new optimization directions. A non-dominated solution is a candidate solution in multi-objective optimization where no other solution is superior to it in all objectives. It can be identified using a fast non-dominated sorting algorithm, with the aim of providing high-quality strategy options that achieve a balance among multiple objectives such as control accuracy, motion stability, and system energy consumption.

[0112] Specifically, the proposed solution first generates a diverse initial policy population, covering multiple trajectory combinations possible for the interaction between the mobile base and the robotic arm, providing a sufficient foundation for optimization. Then, a multi-objective optimization function is used to evaluate the fitness of each policy in the initial population in parallel. Based on dynamically configured weight coefficients according to the situational deviation, the comprehensive performance of each policy on multiple objectives such as control accuracy, motion stability, and system energy consumption is calculated simultaneously. This parallel mechanism significantly shortens the computation cycle. Finally, based on the comprehensive fitness value, an elite retention strategy ensures the continuation of high-fitness policies, while new policies are generated through population evolution. The population is iteratively updated and gradually approaches the Pareto front, outputting a set of cooperative policy candidate schemes containing multiple non-dominated solutions. This process forms a closed-loop optimization system, with each step tightly linked through information flow: the initial population provides input for evaluation, the evaluation results guide the direction of population iteration, the iteration process continuously optimizes the solution quality, and finally outputs a set of candidate schemes that satisfy multi-objective balance, thereby solving the efficiency bottleneck caused by serial processing.

[0113] In one possible implementation, the initial policy population consists of multiple randomly generated combinations of mobile base trajectories and robotic arm joint trajectories. The mobile base trajectories are generated using spline curve fitting, while the robotic arm joint trajectories are planned using polynomial interpolation. Parallel fitness evaluation is implemented using a multi-core processor architecture, distributing policy schemes from the population to multiple computing cores for synchronous execution via a task allocation module. The elite retention strategy selects individuals with higher fitness from the population for preservation. Population evolution operations combine different policy movement patterns through trajectory feature cross-combinations and introduce mutations through parameter perturbations to explore new solution spaces. After multiple rounds of iterative optimization, a set of cooperative policy candidate schemes that do not dominate each other on multiple optimization objectives is output.

[0114] Through the above scheme, this application effectively overcomes the problems of low computational efficiency and insufficient solution quality in the optimization process, realizes the parallel generation and evaluation of candidate strategy schemes, and significantly improves the real-time response capability to dynamically changing environmental situations and task requirements in complex unstructured environments, thereby enabling high real-time performance and strong robustness of the collaborative control of the mobile base and the robotic arm.

[0115] In the complex, unstructured environments of heavy industries such as metal mining and beneficiation, intelligent inspection robots need to simultaneously operate their robotic arms to complete delicate tasks such as equipment inspection and pipeline inspection while moving. These environments typically feature complex terrains such as steep slopes and muddy conditions, accompanied by strong vibrations, drastic temperature and humidity changes, and interference from specific chemical substances, placing extremely high demands on the robot's collaborative operation capabilities. Currently, this field faces the following major technical bottlenecks: First, traditional inspection robots have limited perception systems, relying heavily on conventional vision or position sensors, lacking deep integration of multi-dimensional information such as motor current ripple, mechanical vibration spectrum, and ambient gas concentration, resulting in insufficient early identification of potential equipment failures and environmental risks. Second, the control architecture often adopts a separate control mode for the mobile platform and the robotic arm, lacking effective modeling and compensation for the dynamic coupling effect between the two, making it difficult to maintain the positioning accuracy of the robotic arm's end effector during bumpy movement. Furthermore, related solutions generally lack forward-looking decision-making capabilities based on holographic perception, failing to dynamically adjust the overall collaborative strategy according to environmental changes. In some of the embodiments described above in this application, multiple collaborative strategy candidate schemes are generated based on a multi-objective optimization function. However, in the implementation process, the evaluation of candidate schemes lacks adaptability to dynamic changes in the task, resulting in the inability to accurately quantify the comprehensive performance of each scheme in multiple dimensions such as control accuracy, motion stability, and system energy consumption. Specifically, although the dynamic weight coefficients configured in the above schemes can reflect the impact of task stages and situational deviations, the evaluation process does not effectively combine the weight allocation results with the feature parameters, making the weights of each evaluation index fixed or empirical, which is difficult to adapt to the real-time evolution of task requirements in complex unstructured environments. For example, in scenarios such as metal mining and beneficiation, steep slopes may suddenly increase the demand for motion stability, while the pipeline inspection stage requires an emphasis on control accuracy. If the evaluation weights cannot be dynamically adjusted with the task context, suboptimal strategies may be mistakenly selected, affecting the positioning accuracy of the robotic arm end effector and the overall robustness of the system.

[0116] In this regard, this application further proposes steps for quantitative assessment of synergistic effects, including: Feature parameters are extracted from multiple cooperative strategy candidate schemes, and multiple evaluation indicators are determined based on multi-objective optimization functions and feature parameters. These evaluation indicators include control accuracy, motion stability, and system energy consumption.

[0117] Based on the weight coefficient allocation results and characteristic parameters of the multi-objective optimization function, the weight values ​​of each evaluation index are determined. The evaluation indexes are then weighted and combined based on their weight values ​​to obtain the collaborative effectiveness evaluation function.

[0118] By inputting the feature parameters into the collaborative effectiveness evaluation function, a comprehensive evaluation value for each collaborative strategy candidate scheme is obtained.

[0119] The comprehensive evaluation values ​​of each collaborative strategy candidate are integrated to obtain the evaluation results.

[0120] Among them, characteristic parameters refer to the quantifiable dynamic characteristic data of the candidate cooperative strategy schemes. These can be implemented using parameters such as trajectory tracking error sequences or joint vibration energy spectra, aiming to characterize the performance of the candidate schemes in actual operation. Control accuracy indicators can be understood as a measure of the deviation of the robotic arm's end effector relative to the target position. Specifically, they can be quantified using the statistical characteristics of position errors, aiming to evaluate the accuracy of task execution. Motion stability indicators can be understood as the degree of suppression of posture fluctuations during robot movement. Specifically, they can be characterized using the frequency domain distribution of acceleration signals, aiming to improve operational reliability on bumpy terrain. System energy consumption indicators can be understood as the efficiency of energy consumption during task completion. Specifically, they can be calculated using the power integral of the drive unit, aiming to optimize resource utilization. Weight values ​​refer to the relative importance coefficients of each evaluation indicator in the comprehensive evaluation. They can be dynamically adjusted based on task stage characteristics and situational deviation through a mapping function, aiming to adapt the evaluation system to changes in the task context. The collaborative effectiveness evaluation function can be understood as a mathematical model that integrates multi-dimensional indicators into a single evaluation value. Specifically, it can be constructed using weighted linear combination or nonlinear transformation. Its purpose is to provide objective and comparable evaluation results.

[0121] Specifically, the proposed solution extracts feature parameters from multiple candidate collaborative strategies and determines evaluation indicators such as control accuracy, motion stability, and system energy consumption based on a multi-objective optimization function and these feature parameters, thus achieving a comprehensive characterization of the multi-dimensional performance of the candidate strategies. Subsequently, the weight values ​​of each evaluation indicator are dynamically determined based on the weight coefficient allocation results of the multi-objective optimization function and the feature parameters. This ensures real-time matching of weight allocation with task stage characteristics and situational deviation, avoiding the insufficient adaptability of fixed weights in the face of sudden environmental changes. Building upon this, a task-adaptive collaborative performance evaluation function is constructed by applying the weight values ​​to the weighted combination of evaluation indicators. This function can dynamically adjust the contribution of each indicator according to the current task requirements. Inputting the feature parameters into this evaluation function generates a comprehensive evaluation value for each candidate strategy, thereby transforming heterogeneous performance indicators into a unified quantitative standard. Finally, by integrating the comprehensive evaluation values, the evaluation results are output, providing a reliable basis for selecting the optimal collaborative strategy. This series of steps forms a closed-loop evaluation mechanism, enabling the evaluation process to closely follow changes in the task situation and solving the dynamic adaptability problem of strategy evaluation in complex unstructured environments.

[0122] In one possible implementation, feature parameter extraction is achieved through a real-time data acquisition module in the robot control system, acquiring the trajectory tracking error sequence and joint motor current signals of candidate schemes. In determining the evaluation indicators, the control accuracy indicator uses the statistical characteristics of the end effector position error, the motion stability indicator uses the frequency domain energy distribution characteristics of the chassis accelerometer, and the system energy consumption indicator uses the power integral value of the drive unit. Weight values ​​are determined by querying a preset task stage-weight mapping relationship and dynamically interpolating based on the current situation deviation. The collaborative performance evaluation function is constructed as a weighted linear combination of each indicator. After calculating the comprehensive evaluation value, a non-dominated sorting algorithm is used to integrate the various schemes and output the evaluation result.

[0123] The above technical solution achieves dynamic adaptability in candidate strategy evaluation, enabling the evaluation weights to be adjusted in real time according to the task stage and environmental conditions. This prioritizes motion stability in complex working conditions such as steep slopes and emphasizes control accuracy in the fine detection stage, effectively avoiding the misselection of suboptimal strategies and significantly improving the positioning accuracy of the robotic arm end effector and the overall robustness of the system.

[0124] Specifically, in some of the solutions mentioned above in this application, a virtual compliant dynamic model is proposed to configure the parameters of the robot's underlying control loop online in order to achieve task adaptation. However, in this process, the parameter configuration fails to fully combine the real-time motion state for dynamic adjustment, resulting in the inability to effectively compensate for the dynamic coupling effect between the moving base and the robotic arm under complex working conditions such as bumpy movement, which affects the positioning accuracy of the robotic arm's end effector and the system stability.

[0125] To address this, this application further proposes a method based on a global collaborative control strategy and virtual dynamics model reconstruction instructions to configure the virtual dynamics model parameters of the robot's underlying control loop online, thereby obtaining a task-adaptive virtual compliant dynamics model. (See [link to relevant documentation]). Figure 5 Taking the server as the executing entity as an example, the following steps are included.

[0126] 501. Analyze the virtual dynamics model reconstruction instructions to obtain multiple dynamic stiffness parameters and multiple variable damping parameters for the interaction between the mobile base and the robotic arm.

[0127] 502. Based on the cooperative mode command in the global cooperative control strategy, multiple dynamic stiffness parameters and multiple variable damping parameters are integrated into a unified virtual compliant dynamic model framework.

[0128] 503. Based on the robot's real-time motion state, the parameters in the virtual compliant dynamics model framework are adjusted and optimized online to obtain a task-adaptive virtual compliant dynamics model.

[0129] Among them, the virtual dynamics model reconstruction instruction refers to the instruction signal generated by the upper-level decision-making system for dynamically adjusting the dynamics model. It can be implemented in the form of structured data packets or instruction sets, aiming to indicate the direction of model parameter adjustment according to task requirements. Dynamic stiffness parameters refer to stiffness coefficients that can change in real time during robot movement. They can be calculated based on task semantics using lookup tables or neural networks, aiming to dynamically adjust the compliance characteristics of the robotic arm according to environmental interaction forces. Variable damping parameters refer to damping coefficients that can change in real time during robot movement. They can be implemented using adaptive filtering algorithms or preset dynamic curves, aiming to suppress vibration and improve system stability. Cooperative mode instructions refer to instruction identifiers in the global cooperative control strategy that indicate the current cooperative working mode. They can be represented as state codes or mode labels, aiming to guide parameter integration to adapt to different task scenarios. The unified virtual compliant dynamics model framework refers to the model structure that integrates the dynamic characteristics of the mobile base and the robotic arm. It can be constructed using multibody system dynamics equations or impedance control models, aiming to uniformly handle the dynamic coupling effects between the two. Real-time motion state refers to the robot's current kinematic parameters such as pose, velocity, and acceleration. These can be acquired through sensor fusion technology or state observers, aiming to provide feedback to support dynamic parameter adjustments. Online adjustment and optimization refers to the process of real-time correction of model parameters during system operation. This can be achieved using optimization algorithms or rule engines, aiming to adapt model parameters to environmental changes. Task-adaptive virtual compliant dynamics models are virtual models that can automatically adjust parameters according to task requirements and environmental conditions. This is achieved through parameter adaptation mechanisms, aiming to improve control accuracy and system robustness.

[0130] Specifically, the proposed solution first extracts dynamic stiffness and variable damping parameters by parsing the virtual dynamics model reconstruction instructions. These parameters are directly related to the task instruction flow and environmental perception information, providing task-oriented basic inputs for model construction. Subsequently, utilizing the semantic information of the cooperative mode instructions, the dispersed dynamic stiffness and variable damping parameters are mapped onto a unified virtual compliant dynamics model framework. This achieves a dynamic correlation between the motion trajectory of the moving base and the impedance characteristics of the robotic arm, addressing the problem of missing modeling of dynamic coupling effects in discrete control architectures. Finally, based on real-time motion state feedback, the model parameters are adjusted and optimized online, enabling the virtual model to respond instantly to terrain changes and vibration disturbances. This ensures the robotic arm end effector maintains high-precision positioning capabilities in bumpy environments while improving the overall motion stability of the system.

[0131] In one possible implementation, when the robot performs inspection tasks in steep slope areas of a metal mining environment, the virtual dynamics model reconstruction command is parsed into configuration data containing specific stiffness and damping parameters. The cooperative mode command indicates a "stable movement mode," where the system sets the dynamic stiffness parameters to moderate values ​​to balance the stability of the mobile platform with the manipulator's operational flexibility. Real-time motion status is monitored in real-time by an inertial measurement unit and joint encoders. When ground vibration signals are detected, the system automatically fine-tunes the variable damping parameters to suppress vibration energy transfer, thereby maintaining the positioning accuracy of the manipulator's end effector during pipeline inspection.

[0132] Through the above solution, this application can effectively compensate for the dynamic coupling effect between the mobile base and the robotic arm under complex working conditions such as bumpy movement, significantly improve the positioning accuracy of the robotic arm end effector and the system motion stability, and enable reliable execution of inspection operations in harsh industrial environments.

[0133] In practical applications, some embodiments of this application propose reconstructing virtual dynamics models to obtain parameters for the interaction between the mobile base and the robotic arm. However, in its implementation, there are technical challenges in dynamically generating multi-directional stiffness and damping parameters based on the requirements of the collaborative mode and the real-time interaction status to adapt to the fluctuations in interaction requirements caused by terrain bumps, load changes, and vibration disturbances in complex unstructured environments. Specifically, traditional methods struggle to accurately decompose abstract parameter characteristics into specific motion degrees of freedom and lack a response mechanism for real-time operating conditions. This results in parameter configurations failing to match changes in task stages and environmental disturbances, affecting the compliance and positioning accuracy of collaborative control.

[0134] In response, this application further proposes analytical virtual dynamics model reconstruction instructions to obtain multiple dynamic stiffness parameters and multiple variable damping parameters for the interaction between the mobile base and the robotic arm, including: Extract parameter configuration information from the virtual dynamic model reconstruction instructions, and identify the stiffness parameter characteristics and damping parameter characteristics contained in the parameter configuration information.

[0135] Based on the cooperative mode requirements in the global cooperative control strategy, the stiffness parameter characteristics are mapped to dynamic stiffness parameters in multiple directions, and the damping parameter characteristics are mapped to variable damping parameters in multiple directions.

[0136] Based on the robot's real-time interactive state, the dynamic stiffness parameters and variable damping parameters are normalized and their ranges are constrained to obtain multiple dynamic stiffness parameters and multiple variable damping parameters that meet the current interactive requirements.

[0137] Among them, the virtual dynamics model reconstruction instruction refers to the parameter configuration instruction generated by the upper-level decision-making module. It can be implemented using a data structure combining task stage identifiers and parameter templates, aiming to convey the abstract requirements for model reconstruction. Parameter configuration information can be a set of parameters containing semantic tags, such as stiffness and damping characteristic descriptions encapsulated in JSON format, aiming to structurally express the parameter configuration intent. Cooperative mode requirements refer to the control characteristic requirements associated with the current task stage. They can be dynamically generated based on task semantic tags (such as "climbing mode" or "fine-grained operation mode"), aiming to provide directional guidance for parameter mapping. Normalization processing refers to the standardization operation that eliminates differences in parameter dimensions. It can be implemented using Z-score normalization or Min-Max scaling algorithms, aiming to unify the parameter evaluation benchmark. Parameter range constraints refer to the mechanism that restricts parameter adjustment within the feasible domain. They can be dynamically pruned or smoothed based on preset safety thresholds, aiming to prevent system oscillations caused by parameter exceeding limits.

[0138] Specifically, the solution in this application achieves dynamic parameter adaptation through a closed-loop mechanism of instruction parsing, requirement mapping, and state feedback. First, parameter configuration information is extracted from the reconstructed instructions of the virtual dynamics model, and stiffness and damping parameter characteristics are identified, ensuring complete capture of basic parameter characteristics and avoiding feature omissions. Second, based on the cooperative mode requirements in the global cooperative control strategy, stiffness parameter characteristics are mapped to dynamic stiffness parameters in multiple directions, and damping parameter characteristics are mapped to variable damping parameters in multiple directions. Since the cooperative mode requirements are directly related to the semantics of the task stage, this mapping mechanism dynamically allocates parameter weights for translational and rotational degrees of freedom according to task requirements, enabling the stiffness and damping characteristics to accurately match the differentiated requirements of terrain adaptation of the mobile base and compliant control of the robotic arm's end effector. Finally, based on the robot's real-time interactive state, the dynamic stiffness and variable damping parameters are normalized and their ranges constrained. Through analysis of load dynamic characteristics and motion stability characteristics, the parameter benchmark is dynamically adjusted and the feasible region is constrained, ensuring that the generated parameters always meet the current interactive requirements. Overall, the three steps form a progressive processing chain: the structured parsing of parameter configuration information provides the input basis for mapping, the collaborative mode demand drives multi-directional parameter decomposition to achieve task adaptation, and real-time interactive status feedback completes dynamic parameter calibration, thus constructing a complete generation path from abstract instructions to specific parameters.

[0139] In one possible implementation, the virtual dynamics model reconstruction command is generated by the host computer decision module, and its parameter configuration information is stored in a key-value pair structure containing semantic tags. In pipeline inspection tasks within a metal mining and beneficiation environment, when the global collaborative control strategy identifies the need for a collaborative mode of "precise operation in narrow spaces," the system maps stiffness parameter characteristics to the dynamic stiffness parameters of the robotic arm's end effector in the X, Y, and Z translational directions, as well as the rotational damping parameters around each axis. Simultaneously, the system acquires the interaction status in real time through the inertial measurement unit of the moving base and the joint force sensors of the robotic arm, performs Z-score standardization on the dynamic stiffness parameters, and applies smoothing constraints based on a preset stiffness feasible region (e.g., 0.5-2.0 times the reference value), ultimately outputting a parameter set adapted to the current vibration disturbance conditions.

[0140] Through the above scheme, this application realizes the transformation of virtual dynamic model parameters from static configuration to task adaptation, effectively compensates for the dynamic coupling effect between the mobile base and the robotic arm in complex unstructured environments, enables the robotic arm end effector to maintain high positioning accuracy in bumpy terrain, and ensures the motion stability of the mobile base, thereby solving the problem of decreased cooperative control compliance caused by parameter configuration not matching changes in task stages and environmental disturbances.

[0141] In some embodiments described above in this application, a method is proposed to map stiffness parameter features to dynamic stiffness parameters in multiple directions and damping parameter features to variable damping parameters in multiple directions to achieve interactive parameter configuration between the mobile base and the robotic arm. However, in this process, since the mapping process does not perform parameter decomposition for translational and rotational degrees of freedom and lacks a directional parameter adjustment mechanism based on dynamic weight coefficients, the mobile base cannot accurately match the physical characteristic requirements of different degrees of freedom when vibrating or operating the robotic arm in complex unstructured environments (such as steep slopes and muddy terrain), resulting in a mismatch between stiffness and damping parameter configuration, which in turn affects the positioning accuracy of the robotic arm end effector and the system motion stability.

[0142] To address this, this application further proposes a method based on the cooperative mode requirements in the global cooperative control strategy, mapping the stiffness parameter characteristics to dynamic stiffness parameters in multiple directions, and mapping the damping parameter characteristics to variable damping parameters in multiple directions, including: Analyzing the directional configuration requirements in the cooperative mode requirements, the stiffness parameter characteristics are decomposed into translational degree-of-freedom stiffness characteristics and rotational degree-of-freedom stiffness characteristics, and the damping parameter characteristics are decomposed into translational degree-of-freedom damping characteristics and rotational degree-of-freedom damping characteristics.

[0143] Based on the dynamic weight coefficients in the global collaborative control strategy, the translational degree of freedom stiffness characteristics and rotational degree of freedom stiffness characteristics are mapped to dynamic stiffness parameters in multiple directions, and the translational degree of freedom damping characteristics and rotational degree of freedom damping characteristics are mapped to variable damping parameters in multiple directions.

[0144] The directional configuration requirements refer to the differences in control needs for the mobile base and robotic arm at different degrees of freedom in the collaborative mode. This can be achieved using a task semantic parsing module or an expert rule base, aiming to differentiate the physical constraints on translational and rotational degrees of freedom for different inspection tasks such as steep slope climbing and fine-tuning. The translational degree of freedom stiffness characteristics refer to the stiffness parameter characteristics for x / y / z axis movement. This can be achieved using stiffness estimation methods based on motor current ripple analysis or vibration spectrum feature extraction techniques, aiming to match the anti-slip requirements of the mobile base in muddy terrain. The rotational degree of freedom stiffness characteristics refer to the stiffness parameter characteristics for roll / pitch / yaw axis rotation. This can be achieved using dynamic modeling based on joint encoder data or inverse kinematics algorithms, aiming to ensure the posture stability of the robotic arm during pipeline inspection. The dynamic weight coefficient refers to the real-time adjustment parameters reflecting the deviation between task stage characteristics and the situation. This can be achieved using a fuzzy logic controller or a neural network prediction model, aiming to dynamically adjust the control parameter weights of each degree of freedom according to environmental vibration intensity and task accuracy requirements.

[0145] Specifically, this technical solution achieves refined parameter configuration through a dual mapping mechanism. First, based on the directional configuration requirements of the collaborative mode, stiffness and damping parameters are decomposed into translational and rotational degrees of freedom. This decomposition allows parameter configuration to differentiate the dynamic characteristics of different degrees of freedom. For example, in steep terrain, the anti-slip capability of the translational degree of freedom is enhanced, while in precise operation, the damping of the rotational degree of freedom is reduced to ensure compliance. Second, utilizing the dynamic weighting coefficients in the global collaborative control strategy, the decomposed features are mapped into multi-directional dynamic parameters. The dynamic weighting coefficients are adjusted in real time according to environmental risks and task priorities, enabling the parameter mapping to respond to complex working conditions. For example, under high vibration conditions, the translational stiffness parameter is automatically increased to suppress base sway, while the rotational damping parameter is reduced to ensure the robotic arm's operational compliance. The entire process forms a precise mapping chain from task requirements to physical characteristics, enabling the virtual compliant dynamic model to adaptively match the actual dynamic behavior of the moving base and robotic arm in bumpy environments.

[0146] In one possible implementation, during pipeline inspection tasks in a metal mining environment, when a robot needs to operate its robotic arm to inspect valves on steep terrain, the collaborative mode needs to identify the current task's high stiffness requirement for translational degrees of freedom (to resist base slippage) and low damping requirement for rotational degrees of freedom (to ensure compliant grasping). The system first decomposes the stiffness parameter characteristics into translational and rotational stiffness characteristics, and similarly decomposes the damping parameter characteristics into translational and rotational damping characteristics. Then, based on dynamic weighting coefficients (calculated according to the current vibration intensity and task accuracy requirements), the translational stiffness characteristics are mapped to high stiffness parameters for the x / y / z axes, and the rotational stiffness characteristics are mapped to moderate stiffness parameters for the roll / pitch / yaw axes. Simultaneously, the translational damping characteristics are mapped to high damping parameters to suppress vibration, and the rotational damping characteristics are mapped to low damping parameters to achieve compliant operation. This parameter configuration enables the robot to move stably on steep slopes and to operate the robotic arm precisely.

[0147] Through the above solution, this application achieves precise matching of interaction parameters between the mobile base and the robotic arm in a complex unstructured environment, effectively avoiding the mismatch problem of stiffness and damping parameter configuration, and significantly improving the positioning accuracy of the robotic arm end effector under bumpy conditions and the overall motion stability of the system.

[0148] In some of the embodiments described above in this application, normalization and parameter range constraints are proposed for dynamic stiffness parameters and variable damping parameters based on real-time interactive status to obtain parameters that meet the current interactive requirements. However, in this process, the normalization process may not fully consider the specific impact of load dynamic characteristics and motion stability characteristics, resulting in insufficient parameter adjustment and inability to effectively cope with dynamic changes in complex unstructured environments, thereby affecting the accuracy and stability of the collaborative control between the mobile base and the robotic arm.

[0149] To address this, this application further proposes a method based on the robot's real-time interactive state to normalize and constrain the dynamic stiffness and variable damping parameters, resulting in multiple dynamic stiffness and variable damping parameters that meet the current interaction requirements, including: Dynamic characteristics analysis is performed on the real-time interaction status to extract the load dynamic characteristics and motion stability characteristics during the current interaction process.

[0150] Based on the load dynamic characteristics and motion stability characteristics, the normalization adjustment factors for the dynamic stiffness parameters and variable damping parameters are determined.

[0151] Based on the normalization adjustment factor and the feasible region of the preset parameters, the dynamic stiffness parameters and variable damping parameters are subjected to range constraints and smoothing processing to obtain multiple dynamic stiffness parameters and multiple variable damping parameters that meet the current interaction requirements.

[0152] Among these, load dynamic characteristics refer to the quantitative representation of load changes during robot operation. This can be achieved through motor current harmonic analysis or time-domain feature extraction from a six-dimensional force sensor, aiming to identify transient load fluctuations when the robotic arm grasps an object. Motion stability characteristics can be understood as a comprehensive index of the robot's motion smoothness. This can be achieved through accelerometer spectral energy distribution or motion trajectory jitter amplitude calculation, aiming to characterize the vibration characteristics of the mobile base on bumpy terrain. Normalization adjustment factors refer to the coefficients in the normalization process of dynamically adjusting parameters. This can be achieved through fuzzy rule-based mapping functions or neural network output values, aiming to dynamically adjust the parameter scaling ratio based on load and stability characteristics. The preset parameter feasible region can be understood as the physically feasible boundary range of the parameters. This can be achieved through theoretical calculations based on the robot's dynamics model or statistical intervals of historical working data, aiming to limit the reasonable boundaries of parameter adjustment. Range constraints and smoothing processing refer to the technical means to ensure continuous parameter variation within the feasible region. This can be achieved through sliding window amplitude limiting filtering or spline interpolation smoothing algorithms, aiming to avoid control oscillations caused by abrupt parameter changes.

[0153] Specifically, this application's solution first separates load dynamic characteristics and motion stability characteristics from the real-time interactive state through dynamic characteristic analysis, serving as the core basis for parameter adjustment. Then, based on these two types of characteristics, a normalized adjustment factor is generated, enabling the parameter normalization process to dynamically respond to current operating conditions, such as automatically increasing the normalized benchmark value of the stiffness parameter under high load conditions. Finally, the normalized adjustment factor is combined with a preset feasible region of parameters, and through range constraints and smoothing processing, the parameter adjustment conforms to physical limitations while avoiding abrupt changes, thus forming a continuous and stable parameter output. This technical solution, through a three-stage progressive logic of feature extraction, factor generation, and constraint processing, enables the virtual compliant dynamic model parameters to accurately match the dynamic interaction requirements in complex unstructured environments.

[0154] In one possible implementation, when the robot performs pipeline inspection tasks on steep slopes in a metal mining environment, real-time interactive status data is processed by a dynamic characteristic analysis module to extract the load dynamic characteristics of the robotic arm grasping the pipeline (manifested as motor current harmonic energy concentrated in the 50-100Hz frequency band) and the motion stability characteristics of the moving base on the muddy road surface (manifested as pitch angle acceleration standard deviation exceeding a threshold). Based on these characteristics, a normalization adjustment factor generation module dynamically increases the normalized baseline value of the longitudinal stiffness parameter and reduces the scaling ratio of the rotational damping parameter. Subsequently, under the constraints of the preset parameter feasible domain (stiffness parameter 0.5-2.0 N / m, damping parameter 0.1-0.8 Ns / m), the parameters are smoothly transitioned through sliding window amplitude limiting filtering, ultimately outputting dynamic stiffness parameters and variable damping parameters adapted to the current steep slope operation scenario.

[0155] Through the above scheme, this application achieves precise adaptive adjustment of dynamic stiffness parameters and variable damping parameters, solves the problem of parameter inaccuracy caused by the normalization process not fully considering the dynamic characteristics of the load and the characteristics of motion stability, significantly improves the collaborative control accuracy and motion stability of the mobile base and the robotic arm in complex unstructured environments, and enables the robot to reliably complete high-precision inspection operations in heavy industrial scenarios such as metal mining and beneficiation.

[0156] In some of the embodiments described above in this application, an integrated collaborative motion control based on a task-adaptive virtual compliant dynamics model and a global collaborative control strategy is proposed. However, in the process of its implementation, how to deeply integrate the dynamic parameters of the virtual dynamics model with the collaborative strategy to generate accurate low-level control commands is a challenge. In particular, in complex unstructured environments, traditional discrete control architectures lack a real-time compensation mechanism for the dynamic coupling effect between the moving base and the robotic arm, resulting in a significant decrease in the positioning accuracy of the robotic arm end effector during bumpy movement, which makes it difficult to meet the precision operation requirements in high-vibration and steep-slope terrains in heavy industrial scenarios such as metal mining and beneficiation.

[0157] To address this, this application further proposes a task-adaptive virtual compliant dynamics model and a global cooperative control strategy to achieve integrated cooperative motion control of the robot's mobile base and robotic arm. (See [link to relevant documentation]). Figure 6 Taking the server as the executing entity as an example, the following steps are included.

[0158] 601. Obtain the cooperative control parameters from the task-adaptive virtual compliant dynamic model. The cooperative control parameters include dynamic stiffness parameters and variable damping parameters.

[0159] 602. Based on the global collaborative control strategy, dynamic stiffness parameters, and variable damping parameters, generate trajectory tracking control commands for the moving base and impedance control commands for the robotic arm.

[0160] 603. The trajectory tracking control command and impedance control command are respectively assigned to the mobile base control unit and the robotic arm control unit to realize the integrated coordinated motion control of the mobile base and the robotic arm.

[0161] Among them, the cooperative control parameters refer to the core parameter set characterizing the cooperative motion characteristics of the robot as a whole. These can be implemented using numerical sequences extracted in real-time from the virtual dynamics model or predictive model outputs based on historical working condition data, aiming to provide a dynamic adjustment basis for the underlying control. Specifically, the dynamic stiffness parameter refers to a variable parameter reflecting the system's ability to resist external deformation. It can be implemented using an adaptive lookup table mechanism based on environment perception or a terrain feature mapping function, aiming to dynamically adjust stiffness characteristics according to terrain geometry and vibration spectrum to maintain system stability. The variable damping parameter refers to an adjustable parameter controlling energy dissipation characteristics. It can be implemented using a real-time calculation module based on vibration spectrum analysis or a parameter set predefined in the task phase, aiming to suppress vibration transmission effects in unstructured environments. The trajectory tracking control command refers to the control signal guiding the moving base along the desired path. It can be implemented using optimized outputs generated by model predictive control algorithms or timing command sequences based on feedback correction. The impedance control command refers to the control signal adjusting the robot arm's response characteristics to external interactive forces. It can be implemented using joint torque commands calculated by a virtual spring-damping model or compliance mapping outputs based on task semantics.

[0162] Specifically, the solution in this application outputs collaborative control parameters in real time through a task-adaptive virtual compliant dynamics model, including dynamic stiffness parameters and variable damping parameters. These parameters are dynamically adjusted based on multi-dimensional environmental perception information and the dynamic state information of the robot body, ensuring that the stiffness and damping characteristics match the current working conditions. The global collaborative control strategy provides a task-oriented motion planning benchmark. The two are deeply integrated to generate trajectory tracking control commands for the mobile base and impedance control commands for the robotic arm. Through timing alignment and parameter coupling of the command flow, trajectory tracking can adaptively compensate for terrain disturbances based on dynamic stiffness parameters, while impedance control dynamically adjusts the compliant response of the robotic arm to external vibrations through variable damping parameters. Finally, the generated commands are distributed to the mobile base control unit and the robotic arm control unit, forming a closed-loop collaborative control circuit, effectively suppressing the vibration transmission effect caused by the mobile base's swaying, and enhancing the precise positioning capability of the robotic arm's end effector in highly disturbed environments.

[0163] In one possible implementation, in a metal mining scenario, when the robot is traversing steep, muddy terrain, the task-adaptive virtual compliant dynamics model dynamically adjusts the dynamic stiffness parameters based on the input from vibration sensors and the terrain perception module. The mobile base control unit receives trajectory tracking control commands to smoothly compensate for trajectory deviations caused by terrain undulations. Simultaneously, the robotic arm control unit generates impedance control commands based on variable damping parameters, enabling the robotic arm joints to compliantly adapt to vibration and impact, and maintaining stable contact between the end effector and the pipeline detection target. The entire process ensures strict coordination between the movement of the mobile base and the operation of the robotic arm through the timing synchronization of control commands.

[0164] Through the above solution, this application has achieved high-precision collaborative control of the mobile base and the robotic arm in a complex unstructured industrial environment, and solved the technical problem of the decrease in positioning accuracy of the robotic arm end effector during bumpy movement, thus enabling the reliable execution of fine inspection operations in heavy industrial scenarios such as metal mining and beneficiation.

[0165] In some of the embodiments described above in this application, trajectory tracking control commands for the mobile base and impedance control commands for the robotic arm are proposed to achieve integrated coordinated motion control of the mobile base and the robotic arm. However, in its implementation, there is a lack of a mechanism for accurately parameterizing the desired trajectory characteristics and desired impedance characteristics based on dynamic stiffness parameters and variable damping parameters. This results in the inability of the control commands to dynamically adapt to real-time interactive state changes in complex unstructured environments (such as steep slopes, mud, strong vibrations, and drastic temperature and humidity changes). Consequently, the positioning accuracy of the robotic arm end effector decreases and the motion stability is insufficient, affecting the reliability of the inspection operation.

[0166] In response, this application further proposes a method for generating trajectory tracking control commands for the moving base and impedance control commands for the robotic arm based on a global collaborative control strategy, dynamic stiffness parameters, and variable damping parameters, including: The global collaborative control strategy is analyzed to determine the desired trajectory characteristics of the moving base and the desired impedance characteristics of the robotic arm.

[0167] Based on dynamic stiffness parameters and variable damping parameters, the desired trajectory characteristics and desired impedance characteristics are parametrically transformed to generate trajectory control parameters for the moving base and impedance control parameters for the robotic arm.

[0168] The trajectory tracking control command for the moving base is generated based on the trajectory control parameters, and the impedance control command for the robotic arm is generated based on the impedance control parameters.

[0169] The global collaborative control strategy refers to the set of instructions output by the upper-level decision-making system to guide the coordinated movement of the entire machine. This can be generated using a task planning module or an optimization solver, aiming to ensure logical consistency between the movements of the mobile base and the robotic arm during inspection tasks. Dynamic stiffness parameters are adjustable parameters reflecting the system's resistance to deformation. They can be updated in real-time based on the dynamic characteristics of environmental loads, for example, through online adaptive algorithms, to compensate for vibration disturbances caused by unstructured terrain. Variable damping parameters are variable parameters that adjust the system's energy dissipation characteristics. They can be dynamically configured based on motion stability characteristics, for example, using sliding mode control strategies, to suppress high-frequency jitter in bumpy environments. Desired trajectory characteristics refer to the path geometry and kinematic constraints that the mobile base must follow. These can be represented as time-series functions of position, velocity, and acceleration, aiming to define a safe passage path for the base in complex terrain. Desired impedance characteristics refer to the force-displacement interaction behavior characteristics that the robotic arm must exhibit. These can be modeled as a dynamic relationship between stiffness, damping, and mass, aiming to ensure the contact stability of the end effector during equipment inspection. Parametric transformation refers to the mathematical process of mapping high-level desired characteristics to low-level executable parameters. This can be achieved using nonlinear transformation functions or lookup tables, aiming to establish a precise correlation between task requirements and physical control. Trajectory control parameters are the specific instruction parameters that drive the moving base to perform trajectory tracking. They can include target pose, velocity commands, and tolerance thresholds, aiming to synchronize the base's movement with the robotic arm's operations in time. Impedance control parameters are the core parameter set that adjusts the robotic arm's compliance. They can encompass stiffness matrix elements and damping coefficients, aiming to maintain the positioning accuracy of the end effector in vibrating environments.

[0170] Specifically, the solution in this application first analyzes the global collaborative control strategy, decomposing the inspection task objective into the desired trajectory characteristics of the mobile base and the desired impedance characteristics of the robotic arm, thus forming a logical closed loop between high-level decision-making and low-level execution. Subsequently, the desired characteristics are parametrically transformed based on dynamic stiffness parameters and variable damping parameters. The dynamic stiffness parameters are used to compensate for deformation disturbances caused by changes in terrain load in real time, while the variable damping parameters dynamically suppress unstable components in the vibration spectrum, ensuring that the generated trajectory control parameters and impedance control parameters accurately match the current environmental state. Finally, trajectory tracking control commands for the mobile base and impedance control commands for the robotic arm are generated based on the transformed parameters, enabling the smoothness of the base's motion trajectory and the compliance of the robotic arm's end-effector impedance characteristics to be synergistically optimized in a bumpy environment, thereby avoiding the accumulation of positioning deviations caused by dynamic coupling effects under a discrete control architecture.

[0171] In one possible implementation, the control core employs an embedded microcontroller to receive the global collaborative control strategy. A parsing module extracts the desired trajectory characteristics of the mobile base (such as path curvature and velocity profile) and the desired impedance characteristics of the robotic arm (such as the target stiffness matrix). Subsequently, the microcontroller calls upon dynamic stiffness parameters and variable damping parameters to perform parametric transformations on the desired characteristics—for example, in a steep slope scenario, based on real-time vibration spectrum characteristics, the vertical variable damping parameter is increased while the horizontal dynamic stiffness parameter is decreased, generating trajectory control parameters (including adaptive speed limits) and impedance control parameters (including direction-dependent compliance coefficients) adapted to muddy terrain. Finally, the microcontroller converts the trajectory control parameters into PWM speed control commands for the mobile base's drive motor, and simultaneously maps the impedance control parameters into torque compensation commands for the robotic arm's servo system, achieving synchronized and coordinated movement of the mobile base and robotic arm in a vibration environment.

[0172] Through the above solution, this application can achieve dynamic adaptation of control commands in complex unstructured environments, effectively improving the positioning accuracy and motion stability of the robotic arm end effector during bumpy movement, thereby ensuring the reliable execution of inspection operations in heavy industrial scenarios such as metal mining and beneficiation.

[0173] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0174] Figure 7 This is a schematic diagram of the structure of a robot mobile control system for intelligent inspection provided in an embodiment of this application. See also... Figure 7 The system includes: The fusion processing module 701 is used to fuse the robot’s multi-dimensional environmental perception information, body dynamic state information and task control command stream to generate holographic operating state features of the robot. The strategy planning module 702 is used to perform collaborative strategy planning between the mobile base and the robotic arm based on the holographic operating state characteristics and the task control command flow through an adaptive collaborative decision-making mechanism, so as to obtain a global collaborative control strategy and a virtual dynamic model reconstruction command. The online configuration module 703 is used to configure the virtual dynamic model parameters of the robot's underlying control loop online based on the global collaborative control strategy and the virtual dynamic model reconstruction instruction, so as to obtain a task-adaptive virtual compliant dynamic model. The control module 704 is used to perform integrated coordinated motion control of the robot's mobile base and robotic arm based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy to complete the inspection operation.

[0175] It should be noted that the robot mobile control system for intelligent inspection provided in the above embodiments is only illustrated by the division of the above functional modules when controlling the robot. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the robot mobile control system for intelligent inspection provided in the above embodiments and the robot mobile control method embodiments for intelligent inspection belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0176] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A robot movement control method for intelligent inspection, characterized in that, The method includes: The robot's multidimensional environmental perception information, on-body dynamic state information, and task control command flow are fused and processed to generate holographic operating state characteristics of the robot. Based on the holographic operating state characteristics and the task control command flow, an adaptive collaborative decision-making mechanism is used to plan the collaborative strategy between the mobile base and the robotic arm, resulting in a global collaborative control strategy and virtual dynamic model reconstruction command. Based on the global collaborative control strategy and the virtual dynamics model reconstruction instructions, the virtual dynamics model parameters of the robot's underlying control loop are configured online to obtain a task-adaptive virtual compliant dynamics model. Based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy, the robot's mobile base and robotic arm are subjected to integrated cooperative motion control to complete the inspection operation.

2. The method according to claim 1, characterized in that, The process of fusing the robot's multi-dimensional environmental perception information, on-body dynamic state information, and task control command flow to generate holographic operational state features of the robot includes: Multi-source feature extraction is performed on the multi-dimensional environmental perception information to obtain environmental terrain geometric features, environmental semantic features, and non-visual environmental features. Time-frequency domain feature analysis is performed on the dynamic state information of the robot to obtain the dynamic characteristics of the motor drive system and the mechanical vibration spectrum characteristics of the robot. The task control command stream, the environmental terrain geometry features, the environmental semantic features, the non-visual environmental features, the dynamic features of the motor drive system, and the mechanical vibration spectrum features are subjected to multimodal feature fusion and time alignment processing to obtain the robot's holographic operating state features.

3. The method according to claim 1, characterized in that, Based on the holographic operating state characteristics and the task control command flow, an adaptive collaborative decision-making mechanism is used to plan the collaborative strategy between the mobile base and the robotic arm, resulting in a global collaborative control strategy and virtual dynamics model reconstruction instructions, including: Based on the holographic operating state characteristics, the robot's behavioral state space description is determined, and the deviation between the robot's current state point and the desired state region is calculated in the behavioral state space description. Based on the situational deviation and the task control command flow, a multi-objective optimization function with dynamic weight coefficients is configured. The multi-objective optimization function is used to optimize and solve multiple candidate strategy schemes in parallel to generate multiple cooperative strategy candidate schemes. Each cooperative strategy candidate scheme includes a candidate control strategy and candidate model reconstruction parameters. Based on the multi-objective optimization function, the cooperative effectiveness of the multiple cooperative strategy candidate schemes is quantitatively evaluated to obtain the evaluation results; based on the evaluation results, the optimal cooperative strategy candidate scheme is selected from the multiple cooperative strategy candidate schemes, and the global cooperative control strategy and virtual dynamic model reconstruction instructions are extracted from the optimal cooperative strategy candidate scheme.

4. The method according to claim 3, characterized in that, The determination of the robot's behavioral state space description based on the holographic operating state features includes: The multimodal perception features in the holographic operating state features are subjected to dimensional abstraction processing to obtain multiple abstract behavioral dimension definitions in the behavioral situation space; Based on the dynamic state features in the holographic operating state features, the real-time state quantization value of each of the abstract behavioral dimensions is determined. Based on the definition of the abstract behavioral dimensions and the real-time state quantization values ​​of each of the abstract behavioral dimensions, the behavioral state space description of the robot is determined.

5. The method according to claim 3, characterized in that, The multi-objective optimization function, which configures dynamic weight coefficients based on the situational deviation and the task control command flow, includes: Based on the task stage characteristics in the task control instruction stream, multiple optimization objectives and their corresponding relative priority relationships are determined. Based on the magnitude and direction of the aforementioned deviation, the dynamic weight coefficients of each optimization objective are determined; Based on the multiple optimization objectives, their relative priority relationships, and the dynamic weight coefficients of each optimization objective, an objective function containing multiple weighted optimization objective terms is constructed.

6. The method according to claim 3, characterized in that, The process of parallel optimization of multiple candidate policy schemes using the multi-objective optimization function to generate multiple cooperative policy candidate schemes includes: Generate an initial strategy population containing different combinations of motion trajectories of the mobile base and joint trajectories of the robotic arm; Using the multi-objective optimization function, the fitness of each strategy scheme in the initial strategy population is evaluated in parallel to obtain the comprehensive fitness value of each strategy scheme under the multi-objective optimization function; Based on the comprehensive fitness value, the initial strategy population is iteratively updated through an elite retention strategy and population evolution operations, outputting a set of cooperative strategy candidate schemes containing multiple non-dominated solutions. Each non-dominated solution corresponds to a cooperative strategy candidate scheme that achieves Pareto optimality among multiple optimization objectives.

7. The method according to claim 1, characterized in that, The process of configuring the virtual dynamics model parameters online for the robot's underlying control loop based on the global collaborative control strategy and the virtual dynamics model reconstruction instructions to obtain a task-adaptive virtual compliant dynamics model includes: The virtual dynamics model reconstruction instructions are analyzed to obtain multiple dynamic stiffness parameters and multiple variable damping parameters for the interaction between the mobile base and the robotic arm; Based on the cooperative mode command in the global cooperative control strategy, the multiple dynamic stiffness parameters and the multiple variable damping parameters are integrated into a unified virtual compliant dynamic model framework. Based on the robot's real-time motion state, the parameters in the virtual compliant dynamics model framework are adjusted and optimized online to obtain the task-adaptive virtual compliant dynamics model.

8. The method according to claim 7, characterized in that, The analysis of the virtual dynamics model reconstruction instructions yields multiple dynamic stiffness parameters and multiple variable damping parameters for the interaction between the mobile base and the robotic arm, including: Extract parameter configuration information from the virtual dynamics model reconstruction instructions, and identify the stiffness parameter features and damping parameter features contained in the parameter configuration information; Based on the cooperative mode requirements in the global cooperative control strategy, the stiffness parameter characteristics are mapped to dynamic stiffness parameters in multiple directions, and the damping parameter characteristics are mapped to variable damping parameters in multiple directions. Based on the robot's real-time interaction status, the dynamic stiffness parameters and the variable damping parameters are normalized and their ranges are constrained to obtain the plurality of dynamic stiffness parameters and the plurality of variable damping parameters that meet the current interaction requirements.

9. The method according to claim 1, characterized in that, The task-adaptive virtual compliant dynamics model and the global cooperative control strategy enable integrated cooperative motion control of the robot's mobile base and robotic arm, including: The cooperative control parameters are obtained from the task-adaptive virtual compliant dynamics model, and the cooperative control parameters include dynamic stiffness parameters and variable damping parameters. Based on the global collaborative control strategy, the dynamic stiffness parameters, and the variable damping parameters, trajectory tracking control commands for the moving base and impedance control commands for the robotic arm are generated. The trajectory tracking control command and the impedance control command are respectively assigned to the mobile base control unit and the robotic arm control unit to realize the integrated coordinated motion control of the mobile base and the robotic arm.

10. A robot mobile control system for intelligent inspection, characterized in that, The system includes: The fusion processing module is used to fuse the robot's multi-dimensional environmental perception information, on-body dynamic state information and task control command stream to generate holographic operating state features of the robot. The strategy planning module is used to plan the collaborative strategy between the mobile base and the robotic arm based on the holographic operating state characteristics and the task control command flow through an adaptive collaborative decision-making mechanism, so as to obtain the global collaborative control strategy and the virtual dynamic model reconstruction command. An online configuration module is used to configure the virtual dynamics model parameters of the robot's underlying control loop online based on the global collaborative control strategy and the virtual dynamics model reconstruction instructions, so as to obtain a task-adaptive virtual compliant dynamics model. The control module is used to perform integrated coordinated motion control of the robot's mobile base and robotic arm based on the task-adaptive virtual compliant dynamics model and the global cooperative control strategy to complete the inspection operation.

Images