Sensor integrated sensing-oriented multi-arm drilling machine dynamic cooperative control method and system
By introducing spatiotemporal constraint information to optimize the layout of the multi-arm drilling rig and activating the integrated sensing components, and combining the analysis of multi-source information with the collaborative controller, the problem of low collaborative control efficiency of multi-arm drilling rigs is solved, and efficient and accurate dynamic collaborative control is achieved.
Patent Information
- Application Number
- CN202511266899.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-05
- Publication Date
- 2025-11-21
AI Technical Summary
Multi-arm drilling rigs suffer from low collaborative control efficiency during task execution, and the interaction and coordination between robotic arms are difficult, resulting in inaccurate operation, slow execution speed, and low resource utilization.
By introducing spatiotemporal constraint information for layout optimization, activating integrated sensing components to dynamically collect multi-source information, and analyzing the multi-source information through a collaborative controller to form a collaborative control strategy, the layout and control strategy of the robotic arm are optimized.
It improves the operating efficiency of multi-arm drilling rigs, realizes dynamic collaborative control, and ensures that the robotic arm can complete tasks efficiently and accurately in complex environments.
Smart Images
Figure CN120990564A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of collaborative control technology, specifically to a dynamic collaborative control method and system for multi-arm drilling rigs oriented towards sensor integrated sensing. Background Technology
[0002] Multi-arm drilling rigs typically require multiple robotic arms to work collaboratively to improve operational efficiency when performing complex tasks. However, the collaborative operation of multiple robotic arms presents a problem of low collaborative control efficiency. Due to the significant challenges in the interaction and coordination between the robotic arms, traditional control methods often fail to effectively optimize collaborative control strategies in real-time under dynamic environments. This results in multi-arm drilling rigs facing issues such as inaccurate operation, slow execution speed, and low resource utilization during task execution. Summary of the Invention
[0003] This application provides a dynamic collaborative control method and system for multi-arm drilling rigs based on sensor integrated perception, which is used to address the technical problem of low collaborative control efficiency of multi-arm drilling rigs when performing tasks in the prior art.
[0004] In view of the above problems, this application provides a dynamic collaborative control method and system for multi-arm drilling rigs based on sensor integrated sensing.
[0005] The first aspect of this application provides a dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing, the method comprising: Spatiotemporal constraint information is introduced to optimize the layout of the multi-arm drilling rig, forming an optimal layout strategy. Under the optimal layout strategy, the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig is activated, and the multi-source target information is dynamically collected through the integrated sensing component. The multi-source target information is analyzed by the cooperative controller to obtain a cooperative control strategy. The multi-arm drilling rig is dynamically and cooperatively controlled according to the cooperative control strategy.
[0006] In a possible implementation, spatiotemporal constraint information is introduced to optimize the layout of the multi-arm drilling rig, forming an optimal layout strategy. This includes: extracting spatial constraints from the spatiotemporal constraint information; using the effective length of the guide rail and the working radius of the drilling rig in the spatial constraints as the optimization basis, and the non-overlapping of adjacent robotic arms of the multi-arm drilling rig as the optimization objective, to obtain the maximum number of robotic arms in a single row of guide rails; extracting time constraints from the spatiotemporal constraint information; using the predetermined cutting time and predetermined support time in the time constraints as optimization conditions, to optimize the maximum number of robotic arms, obtaining the optimal number of robotic arms; and forming the optimal layout strategy based on the optimal number of robotic arms.
[0007] In a possible implementation, the maximum number of robotic arms is optimized using the predetermined cutting time and predetermined support time in the time constraint as optimization conditions to obtain the optimal number of robotic arms. This includes: calculating the time difference between the predetermined cutting time and the predetermined support time; matching the correction factor corresponding to the maximum number of robotic arms, and correcting the time difference with the correction factor as a weight to obtain a target time difference; and using the minimum target time difference as the optimization objective to perform optimization analysis on the maximum number of robotic arms to obtain the optimal number of robotic arms.
[0008] In a possible implementation, the cooperative controller analyzes the multi-source information of the target to obtain a cooperative control strategy, including: acquiring a set of target images through the hand-eye vision device in the integrated sensing component; constructing a drilling rig operation space model based on the target image set and establishing a three-dimensional spatial coordinate system of the drilling rig operation space model; sequentially matching the coordinates of the robotic arm and the target drilling point in the three-dimensional spatial coordinate system, respectively denoted as the starting coordinates and the target coordinates; and forming the cooperative control strategy based on the control trajectory from the starting coordinates to the target coordinates through the path planning layer in the cooperative controller.
[0009] In a possible implementation, constructing a drilling rig operation space model based on the target image set includes: extracting a first image group from the target image set and obtaining first point cloud data of the first image group; using a target point cloud dataset constructed based on the first point cloud data as input information for a spatial registration model to obtain an output result, wherein the spatial registration model is a registration fusion model constructed based on the principle of random sampling consistency; and constructing the drilling rig operation space model based on the model parameter information in the output result.
[0010] In a possible implementation, constructing the drilling rig operation space model based on the model parameter information in the output results includes: constructing a spatial point cloud model based on the model parameter information; obtaining basic equipment information of the hand-eye vision device, and performing texture mapping processing on the spatial point cloud model with the basic equipment information as a constraint to obtain a texture mapping result; and using the texture mapping result as the drilling rig operation space model.
[0011] In a possible implementation, after forming the cooperative control strategy based on the control trajectory from the originating coordinates to the target coordinates through the path planning layer in the cooperative controller, the method further includes: dynamically monitoring and obtaining target multi-source perception information through sensor devices in the integrated sensing component; performing fusion preprocessing on the target multi-source perception information to obtain a target perception signal; performing smoothing dead-zone inverse function processing on the target perception signal through the dead-zone pre-compensation layer in the cooperative controller to obtain a first output signal; performing control error analysis on the target perception signal through the adaptive sliding mode control layer in the cooperative controller to obtain a second output signal; and coordinating the first output signal and the second output signal to obtain a control signal and form the cooperative control strategy.
[0012] In a possible implementation, the sensor device includes at least a displacement sensor, a pressure sensor, and a force sensor, wherein the displacement sensor is used to monitor the drill arm position in real time, the pressure sensor is used to monitor the hydraulic oil pressure and flow rate in real time, and the force sensor is used to monitor the friction force in real time.
[0013] In a possible implementation, the adaptive sliding mode control layer in the cooperative controller performs control error analysis on the target sensing signal to obtain a second output signal, including: the adaptive sliding mode control layer determines the control error based on the load displacement and the desired position; the control error is differentiated to establish a switching function; a boundary layer thickness saturation function is introduced to limit the change of the switching function to obtain the second output signal.
[0014] A second aspect of this application provides a dynamic collaborative control system for multi-arm drilling rigs oriented towards sensor integrated sensing, the system comprising: The layout optimization module is used to optimize the layout of the multi-arm drilling rig by introducing spatiotemporal constraint information to form an optimal layout strategy; the information acquisition module is used to activate the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig under the optimal layout strategy, and dynamically collect target multi-source information through the integrated sensing component; the analysis module is used to analyze the target multi-source information through the cooperative controller to obtain a cooperative control strategy; and the control module is used to perform dynamic cooperative control execution of the multi-arm drilling rig according to the cooperative control strategy.
[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application introduces spatiotemporal constraint information to optimize the layout of a multi-arm drilling rig, forming an optimal layout strategy. Under this optimal layout strategy, an integrated sensing component deployed on the robotic arm of the multi-arm drilling rig is activated, and target multi-source information is dynamically collected through the integrated sensing component. The target multi-source information is analyzed by a cooperative controller to obtain a cooperative control strategy. The multi-arm drilling rig is then dynamically and cooperatively controlled according to the cooperative control strategy. This invention solves the technical problem of low cooperative control efficiency of multi-arm drilling rigs in the prior art. By introducing spatiotemporal constraint information for layout optimization, activating integrated sensing components to dynamically collect multi-source information, and analyzing the multi-source information through a cooperative controller to form a cooperative control strategy, the technical effect of improving the operating efficiency of multi-arm drilling rigs and achieving dynamic cooperative control execution is achieved. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a schematic flowchart of a multi-arm drilling rig dynamic collaborative control method based on sensor integrated sensing provided in an embodiment of this application. Figure 2 A schematic diagram of the structure of a multi-arm drilling rig dynamic collaborative control system for sensor-integrated perception provided in an embodiment of this application.
[0018] Figure labeling: Layout optimization module 11, information acquisition module 12, analysis module 13, control module 14. Detailed Implementation
[0019] This application provides a dynamic collaborative control method and system for multi-arm drilling rigs based on sensor integrated perception. It addresses the technical problem of low collaborative control efficiency of multi-arm drilling rigs in the prior art when performing tasks. By introducing spatiotemporal constraint information for layout optimization, activating integrated perception components to dynamically collect multi-source information, and analyzing the multi-source information through a collaborative controller to form a collaborative control strategy, the technical effect of improving the operating efficiency of multi-arm drilling rigs and realizing dynamic collaborative control execution is achieved.
[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0021] It should be noted that any variation of the terms "comprising" and "having" is intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.
[0022] Example 1, as Figure 1 As shown, this application provides a dynamic collaborative control method for multi-arm drilling rigs based on sensor integrated sensing, the method comprising: Step S100: Introduce spatiotemporal constraint information to optimize the layout of the multi-arm drilling rig and form the optimal layout strategy.
[0023] In this embodiment, spatiotemporal constraint information is introduced, including spatial and temporal constraints, which are pre-defined. During layout optimization, the spatial constraints are first extracted from the spatiotemporal constraint information. Using the effective length of the guide rail and the working radius of the drilling rig as the optimization basis, adjacent robotic arms are ensured not to overlap, thereby determining the maximum number of robotic arms per row of guide rails. Then, the temporal constraints are extracted from the spatiotemporal constraint information. By optimizing the predetermined cutting and support times, the maximum number of robotic arms is adjusted to obtain the optimal number of robotic arms. Finally, based on the optimal number of robotic arms, an optimal layout strategy is formed.
[0024] Furthermore, the method provided in the application embodiments, which introduces spatiotemporal constraint information to optimize the layout of the multi-arm drilling rig and form an optimal layout strategy, also includes: Extract the spatial constraints from the spatiotemporal constraint information; using the effective length of the guide rail and the working radius of the drilling rig in the spatial constraints as the optimization basis, and the non-overlapping of adjacent robotic arms of the multi-arm drilling rig as the optimization objective, obtain the maximum number of robotic arms in a single row of guide rails; extract the temporal constraints from the spatiotemporal constraint information; using the predetermined cutting time and predetermined support time in the temporal constraints as optimization conditions, optimize the maximum number of robotic arms to obtain the optimal number of robotic arms; form the optimal layout strategy based on the optimal number of robotic arms.
[0025] In this embodiment, spatial constraints are first extracted from the spatiotemporal constraint information. Spatial constraints include the effective length of the guide rail and the working radius of the drilling rig. The effective length of the guide rail refers to the effective length of the guide rail used to support the movement of the robotic arm, while the working radius of the drilling rig refers to the maximum working range that the robotic arm can reach.
[0026] Next, the optimization is based on the effective length of the guide rail and the working radius of the drilling rig within spatial constraints, with the objective of ensuring that adjacent robotic arms of the multi-arm drilling rig do not overlap. In this process, integer programming is used to determine the maximum number of robotic arms that can be arranged in space by setting constraints (such as the effective length of the guide rail and the working radius). For example, assuming the guide rail is 10 meters long, the working radius of each robotic arm is 2 meters, and each robotic arm must maintain a distance of at least 0.5 meters from each other, integer programming calculates the maximum number of robotic arms that can be accommodated on the guide rail. Through this process, the maximum number of robotic arms for a single row of guide rails is obtained.
[0027] Then, the time constraints are extracted from the spatiotemporal constraint information. The time constraints include the predetermined cutting time and the predetermined support time. The predetermined cutting time refers to the time required for the drilling rig to cut materials or perform other operations, while the predetermined support time refers to the time required for the drilling rig to support the working area.
[0028] Subsequently, the maximum number of robotic arms was optimized using the predetermined cutting and support times within the time constraints as optimization conditions. In this process, the time difference between the predetermined cutting and support times was first calculated to reflect the initial imbalance in operation time. Then, a corresponding correction factor was matched according to different maximum robotic arm numbers, and this correction factor was used as a weight to correct the time difference, resulting in a target time difference that comprehensively considers task intensity and resource allocation. Finally, minimizing the target time difference was used as the optimization objective to complete the optimization analysis of the number of robotic arms, obtaining the optimal number of robotic arms that meets the time efficiency requirements.
[0029] Finally, based on the optimal number of robotic arms and spatial constraints such as the effective length of the guide rail and the working radius of the drilling rig, the optimal position of each robotic arm on the guide rail is calculated through geometric modeling and spatial optimization algorithms (such as heuristic algorithms) to ensure that there is no interference between the robotic arms. Specifically, firstly, the working radius and minimum spacing requirement of each robotic arm are determined, and on this basis, the arrangement of the robotic arms is optimized using algorithms to ensure that they are reasonably distributed within the limited guide rail space. Next, the relative positions between the robotic arms are fine-tuned to optimize their spacing, so that the working area is maximized while avoiding overlap and interference. Then, the working trajectory of each robotic arm is optimized through path planning to ensure that the robotic arms can effectively cover all working areas without wasting time or space. Finally, simulation verification is performed to simulate the layout effect during the operation, ensuring that the working range of each robotic arm does not conflict with other robotic arms, and verifying whether the overall operation efficiency reaches the optimal level, ultimately generating the optimal layout strategy.
[0030] Furthermore, in the method provided in the application embodiment, the maximum number of robotic arms is optimized by using the predetermined cutting time and predetermined support time in the time constraint as optimization conditions to obtain the optimal number of robotic arms, and the method further includes: The time difference between the predetermined cutting time and the predetermined support time is calculated; the correction factor corresponding to the maximum number of robotic arms is matched, and the time difference is corrected with the correction factor as the weight to obtain the target time difference; with the minimum target time difference as the optimization objective, the maximum number of robotic arms is optimized to obtain the optimal number of robotic arms.
[0031] In this embodiment, the time difference between the predetermined cutting time and the predetermined support time is first calculated. The time difference is calculated using simple arithmetic, that is, by directly subtracting the predetermined support time from the predetermined cutting time to obtain a time difference value. For example, if the predetermined cutting time is 30 minutes and the predetermined support time is 20 minutes, then the time difference is 10 minutes.
[0032] Next, based on the maximum number of robotic arms, the corresponding correction factor is retrieved from a pre-defined correction factor matching table. Then, the time difference is corrected using the correction factor as a weight; that is, the correction factor and the time difference are multiplied together to obtain the target time difference.
[0033] Finally, with the goal of minimizing the target time difference, an optimization analysis is performed on the maximum number of robotic arms. In this step, nonlinear optimization algorithms (such as genetic algorithms and particle swarm optimization) are used to adjust the number of robotic arms to minimize the target time difference. Specifically, first, an initial population of robotic arms is generated, and a corresponding correction factor is assigned to each robotic arm using a matching table, and the target time difference is calculated. Then, the optimization algorithm is used to select from different numbers of robotic arms, gradually reducing the target time difference until the optimal number of robotic arms is reached. Ultimately, through this optimization process, the optimal number of robotic arms that minimizes the operation time is found.
[0034] Step S200: Under the optimal layout strategy, activate the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig, and dynamically collect target multi-source information through the integrated sensing component.
[0035] In this embodiment, under the optimal layout strategy, the integrated sensing components deployed on the robotic arm of the multi-arm drilling rig are first activated. These components include vision sensors, force sensors, and displacement sensors. Through these sensing components, the robotic arm collects various types of information from the working environment in real time. The vision sensors capture image data of the working area, providing visual information about the work progress and environmental changes; the force sensors measure the forces applied by the robotic arm during operation, helping to monitor and adjust the operation of the robotic arm. The displacement sensors track the position changes of the robotic arm in real time, ensuring operational accuracy. These sensing components work together to dynamically collect multi-source information about the target, including various types of data such as position, force, and images.
[0036] Step S300: Analyze the target multi-source information through the cooperative controller to obtain a cooperative control strategy.
[0037] In this embodiment, when analyzing multi-source target information using a collaborative controller, a set of target images is first acquired using hand-eye vision devices within the integrated sensing component. This image data is used to establish a visual model of the drilling rig's workspace. Next, based on the acquired target image set, a drilling rig workspace model is constructed, and a three-dimensional spatial coordinate system is established for it. This coordinate system describes the positional relationships between various objects and the robotic arm within the workspace. In the three-dimensional spatial coordinate system, the starting position (originating coordinates) of the robotic arm is sequentially matched with the position of the target drilling point (target coordinates) to ensure that the robotic arm can accurately move to the designated position. Then, through the path planning layer in the collaborative controller, a collaborative control strategy is generated based on the control trajectory from the originating coordinates to the target coordinates.
[0038] Furthermore, in the method provided in the application embodiments, the method further includes analyzing the target multi-source information through a cooperative controller to obtain a cooperative control strategy, and also includes: A target image set is acquired by the hand-eye vision device in the integrated sensing component; a drilling rig operation space model is constructed based on the target image set, and a three-dimensional spatial coordinate system of the drilling rig operation space model is established; the coordinates of the robotic arm and the target drilling point are matched sequentially in the three-dimensional spatial coordinate system and recorded as the starting coordinates and the target coordinates, respectively; the collaborative control strategy is formed by the path planning layer in the collaborative controller based on the control trajectory from the starting coordinates to the target coordinates.
[0039] In this embodiment, the target image set, i.e., image data of the drilling rig's operating area, is first acquired through the hand-eye vision device (a camera or webcam mounted on the robotic arm) in the integrated sensing component. This image data includes information such as objects in the operating environment, the target drilling point, and surrounding obstacles.
[0040] Next, a drilling rig operation space model is constructed based on the target image set. In this process, the first image group is extracted from the target image set, and the first point cloud data of this image group is obtained, i.e., spatial point cloud data generated through image processing technology. Then, the target point cloud dataset constructed based on the first point cloud data is used as input, and a spatial registration model is applied. This model, based on the Random Sample Consensus (RANSAC) principle, performs registration and fusion of the point cloud data to obtain an optimized output result. Finally, based on the model parameter information in the output result, the drilling rig operation space model is constructed. After constructing the drilling rig operation space model, a three-dimensional spatial coordinate system is defined for this model. The three-dimensional spatial coordinate system is the basic framework of the operation space, used to describe the relative positions of all objects and the robotic arm. The origin of this coordinate system is the reference position of the drilling rig, and the X-axis, Y-axis, and Z-axis in the coordinate system represent the forward, lateral, and vertical directions of the operation area, respectively.
[0041] Next, in the three-dimensional coordinate system, the coordinates of the robotic arm and the target drilling point are matched sequentially, and denoted as the starting coordinates and the target coordinates, respectively. The starting coordinates are the current position of the robotic arm, while the target coordinates are the target drilling point position that the robotic arm needs to reach.
[0042] Then, through the path planning layer in the cooperative controller, the optimal control trajectory is calculated based on the distance, direction, and environmental constraints between the starting and target coordinates. This process uses path planning algorithms (such as A* algorithm and Dijkstra's algorithm) to generate the best path for the robotic arm. Path planning not only considers the spatial path of the robotic arm but also ensures that each point on the path meets operational requirements, such as avoiding obstacles, reducing movement time, and ensuring that the robotic arm accurately reaches the target drilling point. Finally, based on the calculated control trajectory, a cooperative control strategy is generated. This strategy guides the robotic arm to move precisely from the starting coordinates to the target coordinates, ensuring efficiency and accuracy during the operation.
[0043] Furthermore, in the method provided in the application embodiments, constructing a drilling rig operation space model based on the target image set further includes: Extract the first image group from the target image set and obtain the first point cloud data of the first image group; use the target point cloud dataset constructed based on the first point cloud data as the input information of the spatial registration model to obtain the output result, wherein the spatial registration model is a registration fusion model constructed based on the principle of random sampling consistency; construct the drilling rig operation space model according to the model parameter information in the output result.
[0044] In this embodiment, a first image group is randomly selected from the target image set, and image processing techniques (such as stereo vision or depth sensors) are used to acquire the first point cloud data of this image group. Point cloud data is three-dimensional spatial data acquired by sensors such as stereo cameras or LiDAR, representing the position and shape of each object surface within the work area. Specifically, the coordinate data of each object surface point is generated by acquiring three-dimensional coordinates in space through depth information in the images or by a depth sensor, thus creating a target point cloud dataset.
[0045] Next, the target point cloud dataset generated based on the first point cloud data is passed as input to the spatial registration model. The purpose of the spatial registration model is to align point cloud data from different sources and perspectives so that they share a unified three-dimensional coordinate system. This registration process uses a registration fusion model built using the Random Sample Consensus (RANSAC) principle. RANSAC is a robust algorithm that iteratively calculates the model by randomly selecting a subset of data points from the point cloud data, and filters out point cloud data that conforms to the model, removing noise and outliers. In this way, optimal matching between different point cloud datasets is ensured, resulting in accurate registration results. By applying the registration fusion model based on the Random Sample Consensus principle, point cloud data from different perspectives or sensors are efficiently aligned, eliminating the influence of perspective differences and noise, thereby obtaining optimized output results. This output result contains registration information of point cloud data collected from multiple perspectives or sensors, and model parameter information obtained through registration, including rotation matrices, translation vectors, and scaling factors.
[0046] Finally, a drilling rig operation space model is constructed based on the model parameter information in the output results. Specifically, firstly, a spatial point cloud model is constructed based on the model parameter information, representing the three-dimensional spatial structure within the operation area. Next, basic equipment information of the hand-eye vision devices is obtained, such as the camera's focal length, field of view, and resolution. Using this information as constraints, texture mapping processing is performed on the spatial point cloud model to add visual details and obtain texture mapping results. Finally, the texture mapping results are applied to the spatial point cloud model to generate the drilling rig operation space model.
[0047] Furthermore, in the method provided in the application embodiments, constructing the drilling rig operation space model based on the model parameter information in the output results further includes: A spatial point cloud model is constructed based on the model parameter information; basic equipment information of the hand-eye vision device is obtained, and the spatial point cloud model is subjected to texture mapping processing with the basic equipment information as a constraint to obtain texture mapping results; the texture mapping results are used as the drilling rig operation space model.
[0048] In this embodiment, a spatial point cloud model is first constructed based on model parameter information. This step uses point cloud stitching and registration techniques to register point cloud data generated from different viewpoints or sensors (such as LiDAR, depth cameras, etc.), merging them into a unified 3D spatial model. Registration uses the Random Sample Consistency (RANSAC) principle to optimize the alignment of the point cloud data. RANSAC removes noise and errors from the data, thereby accurately aligning point cloud data from different sources. Finally, a spatial point cloud model is constructed based on the registered data.
[0049] Next, basic device information for the hand-eye vision device is acquired, including parameters such as focal length, field of view, and image resolution. This information forms the basis for subsequent texture mapping, ensuring accurate alignment between image data and spatial point cloud data. The hand-eye vision device captures environmental images in real time during operation, obtaining visual information about the work area. Focal length determines the zoom level of the field of view, the field of view controls the image's shooting range, and image resolution affects the fineness of texture details. In this step, device information is used to calibrate the mapping position of objects in the image within the spatial point cloud model, ensuring a match between image data and spatial data.
[0050] Then, the basic device information of the hand-eye vision device is used to perform texture mapping on the spatial point cloud model. Texture mapping is the process of attaching an image captured by the vision device to the surface of the point cloud model. In this process, the pixel coordinates in the image are first mapped to the three-dimensional coordinates of the point cloud model to ensure that each image point can be accurately projected onto the surface of the point cloud model. This step is accomplished by the camera projection matrix, which converts the two-dimensional coordinates in the image into positions in the three-dimensional coordinate system, ensuring that the texture can be accurately mapped to the corresponding area of the model. Subsequently, a texture mapping algorithm (such as UV coordinate mapping) is used to accurately map the color, lighting, and detail information in the image onto the spatial point cloud model, forming the texture mapping result.
[0051] Finally, the texture mapping results were used as the drilling rig's working space model.
[0052] Furthermore, in the method provided in the application embodiments, after forming the cooperative control strategy based on the control trajectory from the originating coordinates to the target coordinates through the path planning layer in the cooperative controller, it further includes: The target multi-source sensing information is dynamically monitored by the sensor devices in the integrated sensing component; the target multi-source sensing information is fused and preprocessed to obtain a target sensing signal; the target sensing signal is smoothed by the dead-zone pre-compensation layer in the cooperative controller to obtain a first output signal; the target sensing signal is analyzed by the adaptive sliding mode control layer in the cooperative controller to obtain a second output signal; the first output signal and the second output signal are combined to obtain a control signal and form the cooperative control strategy.
[0053] Furthermore, the method provided in the application embodiments also includes: The sensor device includes at least a displacement sensor, a pressure sensor, and a force sensor, wherein the displacement sensor is used to monitor the drill arm position in real time, the pressure sensor is used to monitor the hydraulic oil pressure and flow rate in real time, and the force sensor is used to monitor the friction force in real time.
[0054] In this embodiment, target multi-source sensing information is first obtained through dynamic monitoring using sensor devices in the integrated sensing component. The sensor devices include displacement sensors, pressure sensors, and force sensors, which are used to monitor the drill arm's position, hydraulic system oil pressure and flow rate, and frictional forces generated during operation, respectively. For example, the displacement sensor captures the drill arm's displacement data in real time to ensure accurate positioning of the robotic arm; the pressure sensor monitors the hydraulic system's operating status to prevent excessively high or low oil pressure; and the force sensor detects the frictional forces generated during operation to adjust the drilling rig's operating status and avoid excessive friction or unstable operation. This information is collected in real time by the sensor devices to generate target multi-source sensing information.
[0055] Next, the multi-source sensing information of the target is fused and preprocessed to obtain the target sensing signal. To improve the accuracy of the multi-source sensing data, Kalman filtering is used for fusion preprocessing. Kalman filtering is a recursive algorithm used to combine data from multiple sensors and reduce noise. In this process, the weight of each sensor data is dynamically adjusted through Kalman filtering to optimize the fused sensing signal. Specifically, the Kalman filter combines the measurements provided by different sensors into an optimal estimate, and through a prediction and update mechanism, it gradually adjusts the system's estimate of the sensing signal to minimize the error of each sensor data. This process provides a smoother and more reliable target sensing signal input.
[0056] Next, the target sensing signal is processed using a smoothing dead-zone inverse function through a dead-zone pre-compensation layer in the cooperative controller to obtain the first output signal. In a control system, the dead zone refers to the region where the system output does not respond immediately to changes in the input signal. Smoothing the dead-zone inverse function processing adjusts the signal appropriately within the dead zone region, ensuring that the output signal no longer exhibits discontinuous or delayed responses. Specifically, an inverse function method is used to adjust changes in the input signal, allowing the control system to maintain a smooth response even within the dead zone, thus obtaining the first output signal, which represents the control signal after dead-zone compensation.
[0057] Next, control error analysis is performed on the target sensing signal using the adaptive sliding mode control layer in the collaborative controller. Specifically, the control error is first determined based on the difference between the load displacement and the desired position. Then, the control error is differentiated to establish a switching function. To ensure the stability of the control signal, a boundary layer thickness saturation function is introduced to limit the range of variation of the switching function, thereby avoiding severe oscillations in the system. Through this process, the second output signal is finally obtained.
[0058] Finally, the first and second output signals are processed collaboratively to obtain the final control signal, forming a collaborative control strategy. The two signals are then fused using a weighted average method to generate a final control signal. This control signal integrates dead-zone compensation and smoothing processing results, as well as control error correction, ensuring smooth response and precise control during drilling operations. The final collaborative control strategy includes specific parameters of the control signal, such as the speed and acceleration of the robotic arm movement, and position adjustments, guiding the drilling rig to accurately complete its tasks.
[0059] Furthermore, in the method provided in the application embodiment, the second output signal is obtained by performing control error analysis on the target sensing signal through the adaptive sliding mode control layer in the cooperative controller, and further includes: The adaptive sliding mode control layer determines the control error based on the load displacement and the desired position; the control error is differentiated to establish a switching function; a boundary layer thickness saturation function is introduced to limit the change of the switching function, and the second output signal is obtained.
[0060] In this embodiment, the adaptive sliding mode control layer first determines the control error based on the difference between the load displacement and the desired position. The load displacement refers to the offset between the actual position of the robotic arm and the target drilling point (desired position). For example, assuming the position of the target drilling point is (10, 5, 3), and due to the influence of external load or friction, the current position of the robotic arm is (9.8, 5.1, 3.2), then the control error is (0.2, -0.1, -0.2).
[0061] Next, the control error is differentiated to establish the switching function. The switching function is a core concept in sliding mode control, used to describe the response behavior when a control error exists. By differentiating the control error, the rate of change of the error is calculated, i.e., the rate at which the error changes over time. In this way, the response speed is dynamically adjusted. When the error is large, the switching function provides a stronger control input to quickly reduce the error; when the error is small, the control strength is reduced to avoid overcorrection. Based on the rate of change of the control error, the switching function uses either a sign function or a linear function to dynamically adjust the control signal according to the magnitude of the error, ensuring stability while achieving rapid convergence.
[0062] Then, a boundary layer thickness saturation function is introduced to limit the variation of the switching function. The role of the boundary layer thickness saturation function is to smooth the switching function when the control error approaches zero, preventing the system from generating excessive control responses. By limiting the variation of the switching function as the error approaches zero, the boundary layer thickness saturation function ensures that the control signal remains stable under small error conditions. This method avoids overreaction of the control system under small errors, thereby maintaining system stability and accuracy. Through this smoothing process, the amplitude of the switching function is limited, reducing system instability.
[0063] After the above steps, the second output signal is obtained. This second output signal is the final control signal obtained by differentiating the control error, generating the switching function, and limiting the boundary layer thickness saturation function. This signal represents the control output after adaptive sliding mode control correction, ensuring that the robotic arm can accurately and stably transition from the current state to the target state.
[0064] Step S400: Perform dynamic collaborative control on the multi-arm drilling rig according to the collaborative control strategy.
[0065] In this embodiment, the multi-arm drilling rig is dynamically controlled according to the aforementioned collaborative control strategy. Specifically, firstly, based on the pre-set control trajectory and parameters, precise control commands are generated for each robotic arm to ensure that each robotic arm can move from its starting position to its target position along the optimal path. The motion trajectory of each robotic arm is refined into specific control parameters, including speed, acceleration, and motion path. These control parameters ensure the accuracy and smoothness of the robotic arm's movement and prevent collisions between different robotic arms.
[0066] During execution, the control signals are dynamically adjusted in real time based on the current state of the robotic arm and changes in the working environment. These changes may include load fluctuations, changes in hydraulic system pressure, or disturbances in the external environment. The control signals are continuously optimized based on real-time sensing data to ensure that the robotic arm operates smoothly along the predetermined trajectory, while also responding to any unexpected changes. For example, when the robotic arm approaches the target, its speed and acceleration are adjusted to avoid excessively fast or slow movements that could affect operational accuracy.
[0067] In summary, the embodiments of this application have at least the following technical effects: This application introduces spatiotemporal constraint information to optimize the layout of a multi-arm drilling rig, forming an optimal layout strategy. Under this optimal layout strategy, an integrated sensing component deployed on the robotic arm of the multi-arm drilling rig is activated, and target multi-source information is dynamically collected through the integrated sensing component. The target multi-source information is analyzed by a cooperative controller to obtain a cooperative control strategy. The multi-arm drilling rig is then dynamically and cooperatively controlled according to the cooperative control strategy. This invention solves the technical problem of low cooperative control efficiency of multi-arm drilling rigs in the prior art. By introducing spatiotemporal constraint information for layout optimization, activating integrated sensing components to dynamically collect multi-source information, and analyzing the multi-source information through a cooperative controller to form a cooperative control strategy, the technical effect of improving the operating efficiency of multi-arm drilling rigs and achieving dynamic cooperative control execution is achieved.
[0068] Example 2, based on the same inventive concept as the multi-arm drilling rig dynamic collaborative control method for sensor integrated perception in the previous examples, such as... Figure 2 As shown, this application provides a dynamic collaborative control system for multi-arm drilling rigs with integrated sensor perception. The system and method embodiments in this application are based on the same inventive concept. The system includes: The layout optimization module 11 is used to introduce spatiotemporal constraint information to optimize the layout of the multi-arm drilling rig and form an optimal layout strategy; the information acquisition module 12 is used to activate the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig under the optimal layout strategy, and dynamically collect target multi-source information through the integrated sensing component; the analysis module 13 is used to analyze the target multi-source information through the cooperative controller to obtain a cooperative control strategy; the control module 14 is used to perform dynamic cooperative control execution on the multi-arm drilling rig according to the cooperative control strategy.
[0069] Furthermore, the system is also used to implement the following functions: Extract the spatial constraints from the spatiotemporal constraint information; using the effective length of the guide rail and the working radius of the drilling rig in the spatial constraints as the optimization basis, and the non-overlapping of adjacent robotic arms of the multi-arm drilling rig as the optimization objective, obtain the maximum number of robotic arms in a single row of guide rails; extract the temporal constraints from the spatiotemporal constraint information; using the predetermined cutting time and predetermined support time in the temporal constraints as optimization conditions, optimize the maximum number of robotic arms to obtain the optimal number of robotic arms; form the optimal layout strategy based on the optimal number of robotic arms.
[0070] Furthermore, the system is also used to implement the following functions: The time difference between the predetermined cutting time and the predetermined support time is calculated; the correction factor corresponding to the maximum number of robotic arms is matched, and the time difference is corrected with the correction factor as the weight to obtain the target time difference; with the minimum target time difference as the optimization objective, the maximum number of robotic arms is optimized to obtain the optimal number of robotic arms.
[0071] Furthermore, the system is also used to implement the following functions: A target image set is acquired by the hand-eye vision device in the integrated sensing component; a drilling rig operation space model is constructed based on the target image set, and a three-dimensional spatial coordinate system of the drilling rig operation space model is established; the coordinates of the robotic arm and the target drilling point are matched sequentially in the three-dimensional spatial coordinate system and recorded as the starting coordinates and the target coordinates, respectively; the collaborative control strategy is formed by the path planning layer in the collaborative controller based on the control trajectory from the starting coordinates to the target coordinates.
[0072] Furthermore, the system is also used to implement the following functions: Extract the first image group from the target image set and obtain the first point cloud data of the first image group; use the target point cloud dataset constructed based on the first point cloud data as the input information of the spatial registration model to obtain the output result, wherein the spatial registration model is a registration fusion model constructed based on the principle of random sampling consistency; construct the drilling rig operation space model according to the model parameter information in the output result.
[0073] Furthermore, the system is also used to implement the following functions: A spatial point cloud model is constructed based on the model parameter information; basic equipment information of the hand-eye vision device is obtained, and the spatial point cloud model is subjected to texture mapping processing with the basic equipment information as a constraint to obtain texture mapping results; the texture mapping results are used as the drilling rig operation space model.
[0074] Furthermore, the system is also used to implement the following functions: The target multi-source sensing information is dynamically monitored by the sensor devices in the integrated sensing component; the target multi-source sensing information is fused and preprocessed to obtain a target sensing signal; the target sensing signal is smoothed by the dead-zone pre-compensation layer in the cooperative controller to obtain a first output signal; the target sensing signal is analyzed by the adaptive sliding mode control layer in the cooperative controller to obtain a second output signal; the first output signal and the second output signal are combined to obtain a control signal and form the cooperative control strategy.
[0075] Furthermore, the system is also used to implement the following functions: The sensor device includes at least a displacement sensor, a pressure sensor, and a force sensor, wherein the displacement sensor is used to monitor the drill arm position in real time, the pressure sensor is used to monitor the hydraulic oil pressure and flow rate in real time, and the force sensor is used to monitor the friction force in real time.
[0076] Furthermore, the system is also used to implement the following functions: The adaptive sliding mode control layer determines the control error based on the load displacement and the desired position; the control error is differentiated to establish a switching function; a boundary layer thickness saturation function is introduced to limit the change of the switching function, and the second output signal is obtained.
[0077] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.
[0078] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
[0079] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.
Claims
1. A dynamic collaborative control method for multi-arm drilling rigs based on sensor integrated sensing, characterized in that, include: Introducing spatiotemporal constraint information to optimize the layout of multi-arm drilling rigs and form the optimal layout strategy; Under the optimal layout strategy, the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig is activated, and multi-source information of the target is dynamically collected through the integrated sensing component. The cooperative controller analyzes the multi-source information of the target to obtain a cooperative control strategy. The multi-arm drilling rig is dynamically and collaboratively controlled according to the aforementioned collaborative control strategy.
2. The dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing as described in claim 1, characterized in that, Introducing spatiotemporal constraint information to optimize the layout of multi-arm drilling rigs, resulting in an optimal layout strategy, including: Extract the spatial constraints from the spatiotemporal constraint information; Using the effective length of the guide rail and the working radius of the drilling rig in the spatial constraints as the optimization basis, and taking the non-overlapping of adjacent robotic arms of the multi-arm drilling rig as the optimization objective, the maximum number of robotic arms in a single row of guide rails is obtained. Extract the time constraint from the spatiotemporal constraint information; Using the predetermined cutting time and predetermined support time in the time constraints as optimization conditions, the maximum number of robotic arms is optimized to obtain the optimal number of robotic arms; The optimal layout strategy is formed based on the optimal number of robotic arms.
3. The dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing as described in claim 2, characterized in that, Using the predetermined cutting time and predetermined support time in the time constraints as optimization conditions, the maximum number of robotic arms is optimized to obtain the optimal number of robotic arms, including: The time difference between the predetermined cutting time and the predetermined support time is calculated. Match the correction factor corresponding to the maximum number of robotic arms, and correct the time difference with the correction factor as the weight to obtain the target time difference; Using the minimum target time difference as the optimization objective, the maximum number of robotic arms is analyzed to obtain the optimal number of robotic arms.
4. The dynamic collaborative control method for multi-arm drilling rigs based on sensor integrated sensing as described in claim 1, characterized in that, By analyzing the multi-source information of the target through a cooperative controller, a cooperative control strategy is obtained, including: The target image set is acquired through the hand-eye vision device in the integrated sensing component; A drilling rig operation space model is constructed based on the target image set, and a three-dimensional spatial coordinate system of the drilling rig operation space model is established. In the three-dimensional spatial coordinate system, the coordinates of the robotic arm and the target drilling point are matched sequentially and denoted as the starting coordinates and the target coordinates, respectively. The collaborative control strategy is formed by the path planning layer in the collaborative controller based on the control trajectory from the starting coordinates to the target coordinates.
5. The dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing as described in claim 4, characterized in that, Constructing a drilling rig operation space model based on the target image set includes: Extract the first image group from the target image set and obtain the first point cloud data of the first image group; The target point cloud dataset constructed based on the first point cloud data is used as the input information of the spatial registration model to obtain the output result. The spatial registration model is a registration fusion model constructed based on the principle of random sampling consistency. The drilling rig operation space model is constructed based on the model parameter information in the output results.
6. The dynamic collaborative control method for multi-arm drilling rigs based on sensor integrated sensing as described in claim 5, characterized in that, The drilling rig operating space model is constructed based on the model parameter information in the output results, including: Construct a spatial point cloud model based on the model parameter information; The basic device information of the hand-eye vision device is obtained, and the spatial point cloud model is subjected to texture mapping processing with the basic device information as a constraint to obtain the texture mapping result. The texture mapping result is used as the drilling rig operating space model.
7. The dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing as described in claim 4, characterized in that, After forming the cooperative control strategy based on the control trajectory from the originating coordinates to the target coordinates through the path planning layer in the cooperative controller, the method further includes: Multi-source sensing information of the target is obtained through dynamic monitoring by the sensor devices in the integrated sensing component; The multi-source sensing information of the target is fused and preprocessed to obtain the target sensing signal; The target perception signal is processed by a smoothing dead-zone inverse function through the dead-zone pre-compensation layer in the cooperative controller to obtain the first output signal. The second output signal is obtained by performing control error analysis on the target sensing signal through the adaptive sliding mode control layer in the cooperative controller. By combining the first output signal and the second output signal, a control signal is obtained, and the coordinated control strategy is formed.
8. The dynamic collaborative control method for multi-arm drilling rigs oriented towards sensor integrated sensing as described in claim 7, characterized in that, The sensor device includes at least a displacement sensor, a pressure sensor, and a force sensor, wherein the displacement sensor is used to monitor the drill arm position in real time, the pressure sensor is used to monitor the hydraulic oil pressure and flow rate in real time, and the force sensor is used to monitor the friction force in real time.
9. The dynamic collaborative control method for multi-arm drilling rigs based on sensor integrated sensing as described in claim 7, characterized in that, The adaptive sliding mode control layer in the collaborative controller performs control error analysis on the target sensing signal to obtain a second output signal, including: The adaptive sliding mode control layer determines the control error based on the load displacement and the desired position; Differentiate the control error and establish the switching function; By introducing a boundary layer thickness saturation function to limit the change of the switching function, the second output signal is obtained.
10. A dynamic collaborative control system for multi-arm drilling rigs oriented towards sensor-integrated sensing, characterized in that, The system is used to execute the sensor-integrated sensing-oriented dynamic collaborative control method for multi-arm drilling rigs as described in any one of claims 1-9, and the system includes: The layout optimization module is used to incorporate spatiotemporal constraint information to optimize the layout of the multi-arm drilling rig and form the optimal layout strategy. The information acquisition module is used to activate the integrated sensing component deployed on the robotic arm of the multi-arm drilling rig under the optimal layout strategy, and dynamically collect target multi-source information through the integrated sensing component. The analysis module is used to analyze the multi-source information of the target through the cooperative controller to obtain a cooperative control strategy; The control module is used to perform dynamic collaborative control execution on the multi-arm drilling rig according to the collaborative control strategy.