Mobile robot control method, apparatus, and mobile robot

Abstract

The application relates to a mobile robot control method, device and mobile robot. The method comprises the following steps: receiving a wiring task for the mobile robot, including a target cabinet, a wiring path from the current position of the mobile robot to the target cabinet and a wire holder; controlling the mobile robot to move along the wiring path, and calculating sensor confidence according to sensor data collected by a plurality of sensors arranged on the mobile robot in real time; dynamically adjusting the weight coefficient of each sensor data involved in the process of globally positioning the mobile robot and locally positioning at least one wire holder based on the sensor confidence; when the mobile robot approaches the target wire holder according to the global positioning, controlling the mobile robot to slow down and stop moving, and starting the local positioning to determine the position of the target wire holder; according to the position of the target wire holder, controlling the mobile robot to press the cable into the target wire holder and fix it, so as to realize accurate wiring control.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a mobile robot control method, apparatus, and mobile robot. Background Technology

[0002] In high-precision operation scenarios such as railway signal room wiring and factory logistics transportation, track-mounted mobile robots have become the preferred solution due to their predetermined paths, high repeatability, and operational safety. However, to complete tasks such as precision wiring, these mobile robots still rely on high-precision perception and positioning systems for end-effector control. However, the perception and positioning technologies of these mobile robots (such as visual positioning and laser positioning) are often affected by factors such as changes in lighting, equipment obstruction, and track vibration in complex industrial environments, leading to positioning drift or data failure, making it difficult to meet the stringent requirements for absolute positional accuracy and stability in wiring operations. Especially in long-distance, multi-branch track environments, how to achieve continuous, stable, and high-precision pose perception along the track for mobile robots, and simultaneously control them to accurately complete wiring actions, remains a prominent technological challenge. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a mobile robot control method, apparatus, and mobile robot.

[0004] In a first aspect, embodiments of this disclosure provide a mobile robot control method, including: Receive a cabling task for a mobile robot, wherein the cabling task includes a target cabinet, a cabling path from the current position of the mobile robot to the target cabinet, and at least one cable tie. Control the mobile robot to move along the wiring path and calculate the sensor confidence level based on the sensor data collected in real time by multiple sensors configured on the mobile robot; The weighting coefficients of each sensor data involved in the process of global localization of a mobile robot and local localization of at least one fixed device are dynamically adjusted based on sensor confidence. When global positioning determines that the distance between the mobile robot and the target wire anchor is less than a preset distance, the mobile robot is controlled to decelerate and stop moving, while local positioning is initiated to determine the position of the target wire anchor. Based on the location of the target cable holder, control the mobile robot to press the cable into the target cable holder and fix it.

[0005] Secondly, embodiments of this disclosure provide a mobile robot control device, including: A receiving unit is used to receive a wiring task for a mobile robot, wherein the wiring task includes a target cabinet, a wiring path from the current position of the mobile robot to the target cabinet, and at least one cable tie. The first control unit is used to control the mobile robot to move along the wiring path and to calculate the sensor confidence level based on the sensor data collected in real time by multiple sensors configured on the mobile robot. An adjustment unit is used to dynamically adjust the weighting coefficients of each sensor data involved in the process of global localization of the mobile robot and local localization of at least one fixed device based on sensor confidence. The second control unit is used to control the mobile robot to decelerate and stop moving when the distance between the mobile robot and the target wire fixer is determined to be less than a preset distance based on global positioning, and at the same time to start local positioning to determine the position of the target wire fixer. The drive unit is used to control the mobile robot to press the cable into the target cable holder and fix it, based on the position of the target cable holder.

[0006] Thirdly, this disclosure provides a mobile robot configured to perform the mobile robot control method as described in the first aspect; wherein, a lidar is fixed to the top of the mobile robot via a bracket, a depth camera is fixed to the bottom of the mobile robot, an inertial measurement unit is installed at the center of the mobile robot and connected to the main control board of the mobile robot, and the drive motor of the mobile robot is also equipped with a wheel encoder.

[0007] Fourthly, embodiments of this disclosure provide an electronic device, including: Memory; Processor; and Computer programs; The computer program is stored in memory and configured to be executed by a processor to implement the first aspect of the method described above.

[0008] Fifthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0009] The mobile robot control method disclosed herein includes: receiving a wiring task for the mobile robot, the wiring task including a target cabinet, a wiring path from the current position of the mobile robot to the target cabinet, and at least one cable holder; controlling the mobile robot to move along the wiring path, and calculating sensor confidence based on sensor data collected in real time by multiple sensors configured on the mobile robot; dynamically adjusting the weight coefficients of each sensor data involved in the global positioning of the mobile robot and the local positioning of at least one cable holder based on the sensor confidence; when the distance between the mobile robot and the target cable holder is determined to be less than a preset distance based on the global positioning, controlling the mobile robot to decelerate and stop moving, and simultaneously initiating local positioning to determine the position of the target cable holder; and controlling the mobile robot to press the cable into the target cable holder and fix it according to the position of the target cable holder, so as to achieve precise wiring control. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the mobile robot control method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of the system structure provided in the embodiments of this disclosure; Figure 3 A schematic diagram of the hardware platform provided in the embodiments of this disclosure; Figure 4 This is a schematic diagram of the sensor layout provided in an embodiment of the present disclosure; Figure 5 A flowchart illustrating another mobile robot control method provided in this embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of the mobile robot control device provided in the embodiments of this disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0013] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0014] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0015] Specifically, in certain scenarios, such as wiring operations in railway signal rooms and factory logistics transportation, track-based mobile robot solutions offer unique advantages over trackless autonomous mobile robots. First, the fixed track provides the mobile robot with a pre-defined, globally optimal path to the target, avoiding the enormous computational burden of real-time global path planning in complex environments and eliminating the risk of system paralysis due to path planning failures or deadlocks. Second, the track physically constrains and guides the mobile robot's movement, enabling it to align with the target and providing a mechanical basis for high-precision, repetitive wiring operations, while significantly reducing the risk of the mobile robot falling or colliding with other equipment. Therefore, track-based mobile robot solutions effectively meet the high requirements of path determinism, operational accuracy, and operational safety.

[0016] However, high-precision cabling tasks place high demands on the perception and localization technologies of track-based mobile robots, and these technologies face severe challenges in track-based movement scenarios. Taking railway signal room cabling operations as an example, the limitations are mainly reflected in: (1) Weak texture and lack of geometric features in the environment: The smooth walls and cabinet surfaces of the signal room usually lack significant visual texture or unique geometric features. This makes the visual SLAM (simultaneous localization and mapping) method, which relies on natural feature extraction and matching, prone to failure, difficult to track and map stably, and not convenient for mobile robots to perceive and locate.

[0017] (2) A large number of highly repetitive structural features exist: The neatly arranged cabinets, cable trays, cables, and cable management devices that serve as key cabling targets in the computer room exhibit high similarity and repetition in visual appearance and 3D point cloud morphology. The environment makes it easy for LiDAR-based SLAM algorithms to generate ambiguity when associating data, mistakenly matching similar features at different locations, thus leading to positioning drift; at the same time, it also makes it difficult for traditional computer vision-based feature recognition methods to accurately distinguish and locate specific cable management devices.

[0018] (3) There is a dynamic narrow space: There may be temporary moving staff or equipment in the computer room, and the mobile robot's running channel is relatively narrow, which puts forward higher requirements for the real-time performance, robustness and obstacle avoidance ability of the positioning system.

[0019] Based on the requirements of signal room layout and wiring operations, the sensing and positioning system should include stable global positioning of the track-mounted mobile robot in the environment to determine the robot's pose for navigation and control, and precise local positioning of the main repetitive feature cable holders within the signal room to coordinate with mechanical structures for accurate and neat cable arrangement within the holders. Existing technical solutions struggle to adequately address these two positioning challenges.

[0020] To address the aforementioned technical problems, this disclosure provides a mobile robot control method that enables high-precision positioning, navigation, and wiring tasks for the mobile robot in complex indoor environments, and reliably assists in completing automated wiring tasks. Detailed explanations are provided through one or more of the following embodiments.

[0021] The mobile robot control method provided in this disclosure is applicable to mobile robot control scenarios. This method can be executed by a mobile robot control device, which can be implemented in software and / or hardware and can be integrated into an electronic device. The electronic device can include, but is not limited to, mobile terminals such as smartphones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers (Tablet PCs), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., as well as fixed terminals such as digital televisions, desktop computers, smart home devices, etc.

[0022] Figure 1 This is a flowchart illustrating the mobile robot control method provided in the embodiments of this disclosure, specifically including as follows: Figure 1 The following steps are shown: In one embodiment, this application applies to the perception and localization of a track-mounted intelligent mobile robot (hereinafter referred to as the mobile robot) in a specific structured environment (such as a railway signal room). The mobile robot is controlled by a perception and localization system (hereinafter referred to as the system). The perception and localization system includes a hardware platform and software algorithms. The hardware platform constitutes the physical foundation of the system and includes a track module, a motion module, an operation module, and a sensor module. The track module is laid out according to the cable tray layout of the signal room, providing the mobile robot with a movement path and cable holder reference. The motion module is a mobile robot platform that can run on the track and is equipped with auxiliary mechanisms for passing through track junctions and turning. The operation module is installed on the motion module and is used to perform precise cable laying and fixing operations. The sensor module integrates a lidar, depth camera, and inertial measurement unit for real-time acquisition of environmental information and the mobile robot's own state data. The software algorithm runs on the mobile robot's host computer and, based on multi-sensor information fusion technology, realizes global localization of the mobile robot, environmental map construction, local localization of the cable holder, and autonomous path planning.

[0023] Understandably, to achieve system functionality, it is necessary to complete the system hardware setup and configuration, as well as the software environment deployment and algorithm implementation. The system hardware setup and configuration refers to installing a track system and a mobile robot platform equipped with integrated sensors, based on the server room layout. Specifically, based on the BIM model or actual measurement data of the signal room, the track layout along the x-axis (i.e., the main track, located on the side of the cabinet, usually wider) and y-axis (i.e., the branch tracks, located between cabinets, usually narrower) is planned to ensure that the tracks cover all cabinet areas requiring cabling. High-strength aluminum alloy can be used for the tracks, which are fixed to the ground or the cable tray on top of the cabinet using brackets. The cable holders on the tracks should be kept basically straight and spaced a certain distance apart. A mobile robot platform integrating sensors such as LiDAR, depth cameras, and IMUs is constructed. For example, the LiDAR (e.g., RPLIDAR A2) is fixed to the top center of the mobile robot platform using a bracket, ensuring an unobstructed field of view. The depth camera (e.g., Intel RealSense D435i) is fixed to the bottom of the platform, and its pitch angle is adjusted so that its optical axis is as perpendicular as possible to the cable holder surface. The IMU (e.g., MPU-9250) is mounted near the robot's center of gravity and securely connected to the main control board. All sensors are connected to a host computer (e.g., NVIDIA Jetson AGX Xavier) via USB or Ethernet interfaces. An electrical and control system is then built. For example, the robot's drive motors are equipped with high-precision encoders (e.g., wheel encoders) to provide odometer information and motion data. The main control board receives sensor data, runs navigation algorithms, and sends motion control commands to the motor drivers. The system can be powered by an onboard battery.

[0024] Understandably, software environment deployment and algorithm implementation refer to the implementation of global localization algorithms and local feature recognition algorithms for mobile robots, assisting them in completing various operational tasks. Through multi-level, multi-modal feature fusion strategies, the inherent structure and features of the data center environment are utilized to transform these into reliable localization information sources. This will be explained in detail through the following examples.

[0025] S101. Receive a cabling task for a mobile robot, wherein the cabling task includes a target cabinet, a cabling path from the current position of the mobile robot to the target cabinet, and at least one cable tie.

[0026] Understandably, after the track is deployed, the mobile robot is launched and controlled remotely or manually to traverse the entire work area along the track, performing LiDAR scanning of the environment. The LiDAR data is transmitted to the main control board via a USB data interface. The main control board processes the data using the Gmapping SLAM algorithm to obtain map information of the environment and generate a raster map, such as a raster map of a computer room. Simultaneously, the map building process is observed in real-time using a visualization tool (such as RViz). Once the mobile robot has explored all accessible areas and the map details are stable, the constructed raster map is saved.

[0027] Understandably, after the mobile robot is powered on and initialized, it loads a grid map and starts global localization and navigation. Subsequently, the operator issues operation tasks (such as wiring tasks) through the human-machine interface of the host computer, specifying the target cabinet and the wiring path. The wiring path refers to the wiring path from the current position of the mobile robot to the target cabinet. The wiring task also includes information such as the inflection points that may be encountered during the movement to the target cabinet and at least one cable tie.

[0028] S102. Control the mobile robot to move along the wiring path and calculate the sensor confidence level based on the sensor data collected in real time by multiple sensors configured on the mobile robot.

[0029] Understandably, based on the above S101, after receiving the operation task, global path planning is performed, and the motors are controlled to drive the mobile robot to start moving along the track. At the same time, sensor data collected in real time by multiple sensors deployed on the mobile robot is received, and the sensor confidence level of each sensor is calculated based on the multiple sensor data.

[0030] Understandably, designing a dynamically weighted multi-sensor fusion framework can also be understood as a dynamically weighted multi-sensor information fusion unit. This unit assigns a real-time confidence weight to key sensors and feature sources, which participates in all subsequent fusion calculations, such as adjusting the observation noise matrix in Kalman filtering and updating weights in particle filtering, to improve the system's robustness in changing environments. The dynamically weighted multi-sensor information fusion unit includes a confidence assessment module and a weight fusion module. The confidence assessment module is used to evaluate at least one of the following confidence levels in real time: visual confidence based on the image's average brightness, contrast, sharpness, and target detection box confidence; LiDAR confidence based on the effective number of points in the point cloud, distribution uniformity, and matching score with the map; IMU confidence based on the data's variance, drift, and deviation from the motion model; and orbital feature confidence based on the jump significance and consistency of short-range laser ranging data.

[0031] The system includes multiple sensors such as LiDAR, depth camera, and inertial measurement unit. Sensor confidence levels include visual confidence, LiDAR confidence, and inertial measurement unit confidence. Sensor data includes depth images, laser scan data, and measurement data.

[0032] Optionally, sensor confidence levels are calculated based on real-time sensor data collected from multiple sensors configured on the mobile robot, including: The first parameter is obtained by calculating the average brightness, contrast, and / or sharpness of the depth image acquired by the depth camera. The visual confidence is evaluated based on the detection confidence of the fixed line output by the pre-trained target detection model and the first parameter. The LiDAR confidence is evaluated based on the quality assessment results of the laser scanning data acquired by the LiDAR, the second parameter obtained by analyzing the effective number of points in a single frame of the laser scanning data and the uniformity of the point cloud distribution, and the matching degree between the scan area represented by the laser scanning data and the pre-constructed grid map. The inertial measurement unit confidence is evaluated based on the characteristic assessment results of the measurement data acquired by the inertial measurement unit, the third parameter obtained by calculating the variance and drift of the measurement data, and the deviation results predicted based on the measurement data.

[0033] Understandably, the visual confidence calculation process for a depth camera is as follows: At least one frame of depth image is acquired in real-time by a depth computer; the visual confidence is comprehensively evaluated based on the quality assessment results of the depth image, a first parameter obtained by calculating the average brightness, contrast, and sharpness of the depth image (e.g., through Laplacian variance), and the confidence of the object detection boxes of the linter output by a pre-trained object detection model (e.g., the YOLOv5 model). Understandably, visual confidence may decrease in situations of insufficient lighting, image blurring, or low linter recognition confidence.

[0034] Understandably, the LiDAR confidence score calculation process is as follows: The LiDAR acquires laser scanning data, which can be understood as point cloud data. The confidence score is assessed based on the quality evaluation results of the point cloud data, a second parameter obtained by analyzing the effective number of points in a single frame of the point cloud, the uniformity of the point cloud distribution, and the matching score (such as the score in the likelihood field) between the current laser scan and the grid map. Understandably, the LiDAR confidence score may decrease when there is a lot of dust in the air, glass reflection, or severe motion distortion.

[0035] Understandably, the calculation process for the IMU's inertial measurement unit confidence level (IMU confidence level) is as follows: the IMU confidence level is assessed based on the internal characteristic evaluation results of the IMU data, the third parameter calculated through the variance and drift of the IMU data, and the bias of the motion model prediction.

[0036] Understandably, sensor confidence also includes track feature confidence. The track feature confidence calculation process is as follows: the track feature confidence is assessed based on the significance and consistency of jumps in short-range laser ranging data, where the short-range laser ranging data is measured by a short-range laser ranging module deployed on the mobile robot.

[0037] Optionally, before performing global localization of the mobile robot and / or local localization of the target wire anchor, the method further includes: If the visual confidence level is less than a first set threshold, in global localization, the weight of motion data or depth image is reduced to a first weight threshold, and the weight of laser scanning data and measurement data is increased to a second weight threshold; in local localization, the weight of the target detection model regarding the wire-fixed object recognition result is reduced to a third weight threshold; and / or, if the LiDAR confidence level continues to decrease to a second set threshold, in both global and local localization, the weight of motion data and measurement data is increased to a fourth weight threshold; and / or, if it is determined that the mobile robot has entered the vicinity of the target wire-fixed object, in local localization, the fusion priority of depth image is increased, and an outlier detection mechanism is triggered, wherein the outlier detection mechanism is used to detect information anomalies in multiple consecutive depth images and report errors; and / or, if the track feature confidence level is greater than a third set threshold, the weight of local localization is increased, and the global localization of the mobile robot is corrected, wherein the track feature confidence level is assessed based on the significance and consistency of jumps in short-range laser ranging data.

[0038] Understandably, the system employs a fuzzy logic controller at the decision-making level to dynamically adjust the data fusion strategy. The following outlines the judgment conditions of the confidence assessment module and the corresponding strategies adopted by the fuzzy logic controller in a typical scenario. Specifically, after calculating the sensor confidence, the system compares the sensor confidence with a set threshold. If ambient lighting deteriorates (i.e., the visual confidence consistently falls below the first set threshold), in global localization, the weight of visual odometry data or visual SLAM contributions is reduced, meaning the weight of depth image or odometry data in data fusion is lowered to the first weight threshold. In local localization, the weight of YOLOv5 (object detection model) recognition results is significantly reduced, and even vision-based wiring guidance is paused, awaiting condition recovery or relying on pre-stored coordinates. Simultaneously, in global localization, the fusion weight of LiDAR and IMU is correspondingly increased to ensure the backbone localization function is unaffected; that is, the weight of visual localization is reduced in both global and local localization, while the weight of LiDAR localization is appropriately increased. The first threshold setting refers to the upper limit of the allowable positioning error, which can be determined according to the wiring accuracy requirements. For example, the value range is set between 0.5 and 1.0 to ensure that the robot's end effector can accurately insert or clamp the cable and avoid wiring failure due to positioning errors. The second threshold setting can be determined by statistically analyzing the critical confidence level when laser positioning fails (error > 2mm), and taking 85% of this value as the safety threshold. For example, the value range is set between 0.5 and 0.7, so that when the laser radar confidence level continuously drops to this value, the action of increasing the weight of motion / measurement data is triggered. The third threshold setting can be determined by measuring the stepped transitions of the track rail or track edge, calculating the signal-to-noise ratio of the transition amplitude and the consistency between multiple frames. When the track feature confidence level is higher than this value, the track outline is considered clear and reliable, and the local positioning weight can be increased and the global positioning can be corrected. For example, the value range is set between 0.8 and 1.0. When visual input quality is poor, over-reliance on motion data can lead to accumulated errors. Too low a weight (e.g., <0.1) can ignore constraints that might be lost visually. A weight value that minimizes the root mean square error (RMSE) of localization can be selected; for example, the third weight threshold could be 0.2. When visual confidence is insufficient, laser scanning data and measurement data can be increased to a target weight value. For example, the second weight threshold can range from 0.6 to 0.8 and can be determined through repeated testing in visual failure scenarios. In local localization, when visual confidence is insufficient, the target detection model (e.g., the result of a wire fastener recognition) is reduced to a lower weight value. This can be achieved through offline testing in low-light, partially occluded scenarios, statistically analyzing the precision of wire fastener detection, and determining the third weight threshold based on the precision; for example, a value ranging from 0.05 to 0.15.

[0039] Understandably, if there are sparse feature regions (i.e., the confidence of the lidar continues to drop to the second set threshold), the weights of odometry and IMU short-term predictions are appropriately increased in the AMCL algorithm, and the range of the localization particle swarm in the AMCL algorithm is increased to tolerate greater uncertainty and wait for re-convergence after entering a feature-rich region.

[0040] Understandably, if a fine-grained wiring operation is performed (i.e., the mobile robot enters the vicinity of the target wire fixer), the visual-depth fusion result will be used first in local localization even if the visual confidence is not optimal, because this is the only way to complete the wiring task. However, this will trigger an outlier detection mechanism. If the depth information is abnormal for multiple consecutive frames, an error will be reported instead of an erroneous action will be performed.

[0041] Understandably, if a track junction is clearly detected, that is, when the mobile robot passes through the track junction (i.e., the track feature confidence reaches its peak), then the weight of this local feature, the junction, is temporarily increased to the highest level, dominating the mobile robot's behavior decisions (such as deceleration, stopping, turning, etc.), and forcing a resampling of the particles in the AMCL algorithm, using this high-confidence landmark to significantly correct the global pose.

[0042] Understandably, the specific logic for triggering the steering mechanism is as follows: the precise distance from the mobile robot to the track section is calculated based on short-range laser ranging data, and used as the input of the deceleration control system to ensure accurate stopping at the section position; at the same time, the high-confidence feature of the track intersection is used to resample the global localization particle set to correct the pose.

[0043] S103. The weighting coefficients of each sensor data involved in the process of global localization of the mobile robot and local localization of at least one fixed device are dynamically adjusted based on sensor confidence.

[0044] Understandably, based on the above S102, the confidence levels of each sensor are quantified and normalized into weight coefficients to achieve dynamic weighted fusion. Specifically, the sensor confidence levels are quantified into weight coefficients through the weight fusion module, and the extended Kalman filter algorithm is used for data fusion. The observation noise matrix and process noise matrix of the extended Kalman filter are dynamically adjusted according to the weight coefficients. The weight coefficients are calculated using formula (1).

[0045] Formula (1) In the formula, For sensors The weighting coefficients, For sensors Real-time confidence levels (i.e., visual confidence, lidar confidence, and IMU confidence). For sensors Weighting adjustment factor For the index of all sensors participating in the fusion, For sensors The real-time confidence level. Based on formula (1), the weight calculation of visual confidence level, lidar confidence level and IMU confidence level is shown in formulas (2) to (4).

[0046] Formula (2) Wherein, formula (2) is the visual weight coefficient. The calculation formula is as follows. In the formula, For visual confidence, This is the visual weight adjustment factor. The formula uses the Softmax function to ensure that the sum of all sensor weights is 1, assuming reliable visual information (such as depth images). (High), its weight It increases automatically.

[0047] Formula (3) Wherein, formula (3) is the weighting coefficient of the lidar. The calculation formula is as follows. In the formula, For LiDAR confidence level, This is the weight adjustment factor for lidar.

[0048] Formula (4) Wherein, formula (4) is the IMU weighting coefficient. The calculation formula is as follows. In the formula, For IMU confidence, This is the IMU weight adjustment factor.

[0049] Understandably, the weighting coefficients calculated above will be directly applied to subsequent fusion algorithms.

[0050] S104. When the distance between the mobile robot and the target wire anchor is determined to be less than the preset distance based on global positioning, the mobile robot is controlled to decelerate and stop moving, and local positioning is initiated to determine the position of the target wire anchor.

[0051] Understandably, based on the above S103, when the mobile robot is globally located, it is determined that the mobile robot is approaching the target wire fastener, and / or, when the bottom sensor deployed on the bottom of the mobile robot detects that it is approaching the target wire fastener, the mobile robot is controlled to decelerate and accurately locate and stop moving. At the same time, the wire fastener visual recognition and positioning module is activated, through which the wire fastener is identified and the three-dimensional coordinates of the wire fastener are calculated.

[0052] Understandably, during global positioning and / or bottom sensor detection, when it is determined that the mobile robot is approaching a track fork, the steering mechanism in the mobile robot is triggered by the short-range laser ranging module.

[0053] Understandably, the global localization and navigation layer employs the Adaptive Monte Carlo Localization (AMCL) algorithm as its core, integrating motion prediction data from the odometry calculated by the mobile robot's wheel encoder and environmental observation data from LiDAR. Through particle filtering, it estimates the mobile robot's precise pose in the map coordinate system in real time. Finally, by combining a global path planner and a local path planner, autonomous navigation and dynamic obstacle avoidance from the starting point to the target point are achieved.

[0054] Optionally, global localization of the mobile robot includes: The system acquires motion data of the mobile robot provided by the wheel encoder configured on the mobile robot. This motion data includes linear velocity, angular velocity, and the robot's rotation angle within a set sampling time interval. A pre-trained motion model calculates the target pose of the chassis mounted on the mobile robot in the map coordinate system based on the set sampling time interval and the motion data. A pre-trained observation model calculates the target coordinates of the laser measurement points emitted by the lidar configured on the mobile robot in the map coordinate system based on the target pose and laser scanning data. Real-time global localization of the mobile robot is then performed based on the target pose and target coordinates.

[0055] Understandably, the design is based on global localization fusion of LiDAR, IMU, and wheel encoder. The aforementioned weighting coefficients affect the dependence on each sensor during the fusion process, thus impacting the global localization result of the mobile robot. Specifically, during the online operation phase of the mobile robot, a pre-saved grid map is loaded. Motion data, LiDAR data, and IMU data are fused based on the AMCL algorithm to achieve real-time localization of the mobile robot within the grid map, i.e., global localization. The specific implementation is as follows: During initialization, a particle filter disperses a large number of particles throughout the grid map or a designated area to represent the possible poses of the mobile robot. As the mobile robot moves, the AMCL algorithm updates the particle positions according to the motion model and updates the weight of each particle based on the matching degree between the LiDAR scan data and the grid map. Then, adaptive resampling is performed. Finally, the particle swarm converges to the vicinity of the mobile robot's actual pose, achieving accurate localization. Subsequently, the coordinate information is sent to the control node of the motion module in the mobile robot via a ROS topic or service. The control node plans the path according to the task requirements, controls the movement of the mobile robot, and operates the wiring mechanism to complete cable fixing.

[0056] Understandably, the motion data of the mobile robot provided by the wheel encoder is obtained, including linear velocity, angular velocity, and the rotation angle of the mobile robot within a set sampling time interval. Subsequently, the target pose of the chassis mounted on the mobile robot in the map coordinate system is calculated by a pre-trained motion model based on the set sampling time interval and the motion data. The motion model of the mobile robot can be expressed as formula (5).

[0057] Formula (5) In the formula, for The pose representation of the robot chassis in the map coordinate system at any time (i.e., the target pose). To set the sampling time interval, Let be the linear velocity of the mobile robot, decomposed along the chassis coordinate system. Let ω be the angular velocity of the mobile robot. This is used to set the turning angle of the mobile robot within the sampling time interval.

[0058] Understandably, a pre-trained observation model calculates the target coordinates in the map coordinate system based on the target pose and laser scanning data, using the laser measurement point emitted by the lidar configured on the mobile robot.

[0059] Optionally, based on the target pose and laser scanning data, a pre-trained observation model calculates the target coordinates in the map coordinate system of the laser measurement point emitted by the lidar configured on the mobile robot, including: The lidar information is calculated based on the laser scanning data. This lidar information includes at least one of the following: the coordinates of the lidar mounted on the chassis in the chassis coordinate system; the distance from the observation feature point determined by the observation model to the lidar's rotation center; the angle between the horizontal axis of the local coordinate system with the rotation center as the origin; the angle between the line connecting the center point of the chassis and the rotation center and the horizontal axis of the chassis coordinate system; the measurement noise of the laser emitted by the lidar in the horizontal and vertical directions in the lidar coordinate system; and the target coordinates in the map coordinate system calculated by the pre-trained observation model based on the target pose and lidar information, using the laser measurement point emitted by the lidar configured on the mobile robot.

[0060] Understandably, the laser observation model for a mobile robot can be expressed as formula (6).

[0061] Formula (6) In the formula, The coordinates of the laser measurement point in the map coordinate system. for The pose representation of the robot chassis in the map coordinate system. The coordinates of the LiDAR (Light Detection and Ranging) sensor mounted on the chassis in the mobile robot's chassis coordinate system. To observe the distance from the feature point to the LiDAR rotation center, Let x be the angle between the x-coordinates of the local coordinate system with the center of rotation as the origin. Number the selected measurement points within one laser scanning cycle. Draw a line connecting the center point of the mobile robot chassis to its rotation center and define the chassis coordinate system. x The included angle of the axis, For the laser in the local lidar coordinate system x, y Measurement noise in the axial direction.

[0062] Optionally, the method further includes, before using a pre-trained motion model to analyze motion data based on a set sampling time interval: The system estimates the attitude change of the mobile robot within a set time period based on the measurement data collected by the inertial measurement unit configured on the mobile robot, and corrects the motion data and laser scan data according to the attitude change; and / or, matches the laser scan data with a pre-built grid map to determine the absolute pose observation data, and corrects the measurement data and motion data according to the absolute pose observation data.

[0063] Understandably, in global localization, a fusion framework centered on extended Kalman filtering is constructed based on the established motion and observation models to address the cumulative drift problem that pure laser SLAM may experience due to feature similarity during long, repetitive track runs. Specifically, the incremental motion information (motion data, also known as odom topic) provided by the wheel encoder is used as the basis for motion model prediction; the three-axis acceleration and angular velocity (measurement data, also known as imu topic) provided by the IMU are used to accurately estimate the short-term attitude changes of the mobile robot, and the IMU data is also used to supplement and correct the odometry data, especially when the mobile robot starts, brakes, or experiences slight slippage, it can effectively compensate for odometry errors and correct motion distortions in the laser scan data caused by the movement of the mobile robot; the geometric scan data of the environment provided by the lidar (laser scan data, also known as scan topic) is used as the input to the observation model. By matching real-time laser scan data with a pre-built grid map, absolute pose observation is provided to correct the cumulative errors of odometry and IMU. The fusion of this LiDAR, IMU, and wheel encoder ensures smooth global localization, high-frequency updates, and long-term stability for mobile robots operating for extended periods in channels lacking unique features.

[0064] Understandably, in the implementation of extended Kalman filtering, the observation noise matrix and process noise matrix are no longer fixed values, but are dynamically scaled based on the LiDAR confidence level and IMU confidence level. Specifically, when using the motion model of a wheeled odometer and IMU to predict the position of a mobile robot at the next moment, the prediction results are easily biased due to factors such as wheel slippage, uneven ground, model simplification, and motor control errors. In this case, the process noise matrix can be used. The covariance represents the random error not described by the motion model. Its basic value can be calibrated offline based on the error characteristics of the mobile robot's motion model, or online based on the real-time confidence level of the IMU. and the instantaneous angular velocity of the mobile robot Dynamic scaling is performed, and its calculation formula is shown in formula (7).

[0065] Formula (7) In the formula, and This is an adjustable positive coefficient. This adjustment strategy causes the filter to reduce the confidence level of the motion model's predictions when IMU data is unreliable or when the mobile robot is making high-speed turns. When the IMU confidence level is low (potentially due to large vibrations or severe drift), An increase indicates less confidence in the motion model's predictions; when the mobile robot's angular velocity... When the slippage is large (e.g., during a sharp turn), the slippage error of a wheel odometer usually increases, therefore the slippage error should be increased accordingly. .

[0066] Meanwhile, when sensors (such as lidar) measure the environment, noise exists in the measured values, which is observed through the noise matrix. This represents the covariance of the measurement noise. For example, when the confidence level of the lidar is low, increasing the noise matrix of the lidar observation accordingly means that the particle filter will reduce its dependence on the lidar observation results in this cycle and rely more on the observation results of the IMU and odometer. The process of dynamically adjusting the lidar observation noise matrix according to its weights is shown in Equation (8).

[0067] Formula (8) In the formula, The preset reference noise matrix is ​​used when the lidar confidence level is high ( When the effective noise matrix is ​​large, Decreasing the particle size will make the particle filter more confident in the lidar's observation data; conversely, increasing its size will decrease its confidence.

[0068] Understandably, the local feature recognition and precise localization layer, targeting the wiring anchors, employs the YOLOv5 deep learning-based target detection algorithm. The model is trained using a pre-created and augmented dataset of anchors (e.g., rotation, cropping, brightness adjustment, noise addition), enabling robust recognition of anchors under different viewpoints and lighting conditions. After anchor recognition, the 3D coordinates of the anchor's center point in the camera coordinate system are calculated using depth information from a depth camera. These coordinates are then transformed into the robot's base coordinate system, guiding the operation module for precise control. For track bifurcation sections requiring precise identification, a laser rangefinder longitudinally scans the track surface. When a jump in the ranging data within consecutive frames, abnormal track surface height, or a significant decrease in echo quality is detected, the section or missing track judgment logic is triggered, assisting the motion module in lowering the bridge arm or retrievable track to navigate the track bifurcation.

[0069] Optionally, local positioning of at least one cable tie is performed to determine the location of the target cable tie, including: The pre-trained target detection model identifies the linchpin in the depth image and outputs the identification result. The pixel coordinates of the linchpin's center point in the identification result are combined with the depth image, and the first three-dimensional coordinates of the linchpin's center point in the camera coordinate system are calculated through back-projection using the camera's intrinsic parameters. During the calculation of the first three-dimensional coordinates, a target pixel region is selected based on the linchpin's center point, and invalid depth values ​​in the target pixel region are removed. Based on the first three-dimensional coordinates and the visual weights in the weighting coefficients, the second three-dimensional coordinates of the linchpin's center point in the camera coordinate system are calculated. The second three-dimensional coordinates of the linchpin's center point are then transformed from the camera coordinate system to the mobile robot's base coordinate system to determine the position of the linchpin.

[0070] Understandably, the specific steps for calculating the 3D coordinates of the linter include: obtaining the center pixel coordinates of the linter bounding box, performing back projection by combining the intrinsic parameters of the depth camera and the depth map, removing outliers by median filtering of the neighborhood depth values, and performing a weighted average of the observation results of multiple frames to obtain the linter's 3D coordinates, where the weights are positively correlated with the visual confidence of each frame.

[0071] Specifically, a local localization system integrating visual and depth information is designed. Deep fusion is then performed within this local localization to provide millimeter-level 3D spatial coordinates (the position of the target wire holder) for the wiring mechanism of the mobile robot, guiding precise robot operation. Specifically, a YOLOv5-based target detection model is deployed to process RGB images (depth images) from a depth camera in real time, quickly locating all wire holders in the image and outputting recognition results including the 2D bounding boxes of all wire holders. The pixel coordinates (u,v) of the target wire holder's center point in the recognition result are combined with the aligned depth map. Through back-projection using the depth camera's intrinsic parameters, the 3D coordinates (Xc,Yc,Zc) of the target wire holder's center point in the camera coordinate system are calculated and denoted as the first 3D coordinates. The target wire holder's center point can be the center point of the bounding box. In calculating the first three-dimensional coordinates, to overcome the potential depth value jumps or invalid points that may occur under specific materials or lighting conditions, a neighborhood median filtering algorithm is adopted. That is, an N×N pixel region is selected around the center point of the detection box as the target pixel region, and invalid values ​​in the target pixel region are removed. The median of all valid depth values ​​is then taken as the final depth value Z, thereby significantly improving the robustness and accuracy of depth measurement. Subsequently, based on the first three-dimensional coordinates and the visual weights in the weighting coefficients, the second three-dimensional coordinates of the target linchpin center point in the camera coordinate system are calculated. The specific calculation method is shown in formula (9).

[0072] Formula (9) In the formula, The second three-dimensional coordinate is for the past. Frame observation results The weighted average is obtained. It is the first Visual weights calculated from frame images Proportional to the value calculated in that frame This ensures that clear, reliable frames have a greater impact on the final result, while the impact of poor-quality frames is automatically suppressed. This guarantees that high-confidence frames contribute more to the final result. The first three-dimensional coordinate is the... The three-dimensional coordinates calculated from a frame image can also be understood as a result of visual observation.

[0073] Understandably, the second three-dimensional coordinates of the target wire clamp center point are transformed from the camera coordinate system (camera_link) to the mobile robot's base coordinate system (base_link) through the ROS TF system, and the third three-dimensional coordinates of the target wire clamp center point (i.e., the position of the target wire clamp) are determined so that the wiring mechanical structure can be directly guided to perform the wire clamping operation.

[0074] Understandably, the multi-sensor information fusion layer uses the TF coordinate transformation system in ROS to uniformly manage the transformation relationships between coordinate systems such as grid maps, odometry, mobile robot chassis, LiDAR, and depth cameras, ensuring that data from different sensors can be fused under a unified reference system. It also uses algorithms such as extended Kalman filtering to fuse feature data from different sensors to generate more accurate and comprehensive environmental perception information.

[0075] S105. Based on the position of the target cable holder, control the mobile robot to press the cable into the target cable holder and fix it.

[0076] Understandably, based on the above S104, according to the positioning information (the position of the target cable holder) obtained from local positioning, the control operation module executes the operation task to drive the mobile robot to press the cable into the target cable holder and fix it. After the single-point wiring task is completed, the mobile robot continues to the next target point and repeats the above operation until the entire wiring task is completed. Throughout the process, the host computer collects and displays the mobile robot's position, status, sensor data, and recognition results in real time, supports manual monitoring and intervention, and can be manually remotely controlled and task adjusted via a handle, enabling the mobile robot to complete temporary special tasks.

[0077] Optionally, the method also includes: Acquire ranging data collected by the short-range laser ranging module configured on the bottom of the mobile robot; if a sudden change in ranging data is detected, control the mobile robot to stop in front of the intersection or at the center of the intersection, and control the mobile robot to perform a turning operation.

[0078] Understandably, a reliable passage mechanism for key nodes is designed. Addressing the unique challenge of turning at intersections for track-based mobile robots, track intersections are introduced as high-level semantic features that can be accurately perceived and utilized into the fused global localization. This solves the problem of reliable perception and autonomous passage for the mobile robot at this critical node and provides auxiliary correction points for global localization. One embodiment is steering control based on short-range laser ranging. Specifically, a short-range laser ranging module installed on the bottom of the mobile robot performs a high-frequency longitudinal profile scan of the track plane directly below. When the mobile robot travels along a straight track, the ranging value remains within a stable range. When approaching a track intersection, a section or missing rail may appear. The short-range laser ranging module immediately detects a drastic change in the ranging data (e.g., a sudden change from a fixed track distance to a larger ground distance). This distance change signal, as a high-confidence, low-latency binary switching signal, is directly sent to the decision and control layer of the mobile robot as a key input parameter of the deceleration control system. This ensures that the robot stops precisely at the section location, reducing its speed to zero, avoiding running off the section or experiencing operational deviations. Subsequently, a series of actions are triggered: the precise positioning and stopping procedure is initiated, and feedback from the wheel encoder is used to ensure that the mobile robot stops accurately in front of the intersection or at the center position; the steering mechanism (such as lowering the bridge arm or retractable track) is controlled to perform steering operations; and enhanced pose update suggestions are provided for AMCL global localization, for example, by using the known structural feature of the intersection to perform a significant convergence optimization on the particle set and correct for possible small drifts.

[0079] In another embodiment, a lightweight track junction image classifier is trained using RGB images captured by a bottom depth camera. As the mobile robot moves, this classifier continuously classifies and identifies images. Once it recognizes a unique visual pattern at a junction and outputs a result with high confidence, this semantic visual feature is used as a trigger signal. Subsequent triggered operations are the same as in the previous embodiment and will not be described in detail here.

[0080] This disclosure provides a mobile robot control method. Through the aforementioned fusion mechanism, the system achieves global positioning via "laser-IMU-odometer," local positioning via "vision-depth" for precise end-effector manipulation, and comprehensive, robust positioning and navigation capabilities for key actions via "laser / vision-intersection features," adapting to the complex indoor environment of railway signal rooms. Specifically, through multi-sensor fusion and a hierarchical control architecture, the mobile robot achieves high-precision positioning, navigation, and wiring tasks in complex indoor environments, reliably assisting in automated wiring tasks. This overcomes the problems of low efficiency, poor accuracy consistency, and high safety risks associated with manual wiring in related technologies. It also addresses the technical bottlenecks faced by mobile robots in confined spaces like signal rooms—complex structures with repetitive features (such as weak wall textures, numerous similar cabinets and cable holders), and dynamic obstacles—including issues like easy global positioning drift, unreliable local feature recognition, poor environmental adaptability, and difficulty in completing high-precision wiring tasks. The method achieves the following beneficial effects: By combining the "laser-IMU-odometer fusion to ensure global trajectory" and "vision-depth fusion to achieve precise end-point operation", the limitations of single sensor technology in data center environments with weak texture and highly repetitive features are effectively overcome. Millimeter-level precise positioning from the vehicle body to the wiring target is achieved, meeting the requirements of wiring operations, significantly improving wiring accuracy, and achieving a balance between high precision and high robustness.

[0081] The system utilizes the Gmapping algorithm to construct an environmental map and employs the AMCL algorithm for adaptive positioning, enabling it to adapt to dynamic changes in the computer room environment. By introducing track intersections as identifiable semantic features, it provides crucial landmark information in repetitive environments. The deep learning-based target detection algorithm effectively addresses interference such as changes in lighting and partial occlusion, reliably identifies cable fixture features, and exhibits a high degree of intelligence, thereby enhancing the system's intelligence and environmental adaptability.

[0082] With track guidance and automated control, mobile robots can autonomously complete path planning and wiring operations, eliminating the reliance on human skill. This not only significantly improves wiring efficiency but also ensures the consistency and aesthetics of the wiring process, reduces the risk of human error, and significantly improves the efficiency and quality of wiring operations.

[0083] By introducing an online assessment of sensor confidence and a dynamic weighted fusion mechanism, the adaptive optimization of the sensing system is achieved. Instead of statically combining sensor data, the system adjusts more reliable sensing information under different environmental conditions, which greatly improves the system's environmental adaptability and overall robustness in complex and unstructured industrial scenarios.

[0084] The system is developed based on the ROS framework, with a clear modular design. The hardware interfaces and software algorithm modules are relatively independent, and the coupling between the functional modules is low, which facilitates functional expansion and maintenance. It can adapt to the cabling needs of signal room with different sizes and layouts, and has good versatility and industrial application potential.

[0085] Based on the above embodiments, Figure 2 The schematic diagram of the system structure provided in this embodiment illustrates that the core of the perception and positioning system lies in constructing a multi-layered, multi-sensor fusion hardware and software collaborative architecture, mainly comprising two parts: a hardware platform and software algorithms. The hardware platform constitutes the physical foundation of the system, including a track module, a motion module, a cabling module, and a sensor system. The track module is laid out according to the layout of the server room cable trays, providing the mobile robot with a movement path and cable holder reference. The motion module is a mobile robot platform that can run on the track and is equipped with auxiliary mechanisms for passing through track junctions and turning. The operation module is installed on the motion module and is used to perform task operations. The sensor system integrates a lidar, a depth camera, an inertial measurement unit, and a short-range laser ranging module for real-time acquisition of environmental information and the mobile robot's own state data. The software algorithm runs on the host computer of the mobile robot. Based on multi-sensor information fusion technology, it realizes global localization, environmental map construction, local feature recognition and autonomous path planning of the mobile robot. It also introduces a dynamic weighted multi-sensor fusion mechanism, which can evaluate the confidence of sensors in real time and dynamically adjust the fusion weights according to the ambient light, feature richness and other conditions. It effectively solves the problems of positioning drift and unreliable recognition caused by weak texture and repetitive features in the signal room environment. At the same time, it uses track intersections as semantic features to assist in positioning and steering control, realizing high-precision and high-robust autonomous wiring operation of the mobile robot in complex signal room environment. Understandably, the system is divided into a functional layer, an algorithm layer, and a physical layer. The functional layer enables functions such as path planning and positioning information feedback. The algorithm layer is implemented based on software algorithms, involving the fusion of modal information such as laser positioning, visual positioning, and odometry to achieve global and local positioning. The physical layer is implemented based on a hardware platform, including LiDAR, IMU, depth cameras, and wheel encoders. Specific implementation details for each layer are provided in the above embodiments and will not be repeated here.

[0086] Based on the above embodiments, Figure 3 This is a schematic diagram of a hardware platform provided in an embodiment of the present disclosure. The hardware platform includes a track module, a motion module, and a wiring module, wherein... The track module is designed based on the prior layout of the data center obtained through BIM technology. It is parallel to the cable trays and includes a main x-axis track laid parallel to the rack arrangement direction and a branch y-axis track perpendicular to it, forming a gridded running path. The track also serves the dual function of guiding the movement of mobile robots and acting as a cable laying channel.

[0087] The motion module, or track-mounted mobile robot body, is the execution carrier of the system. Its mechanical structure is designed with gantry arms and a retractable track mechanism to ensure that the mobile robot can smoothly and reliably turn and switch tracks at track intersections, avoiding jamming or tipping over.

[0088] The operation module is integrated on the motion module and includes a wire feeding mechanism, a wire pressing mechanism and other wiring devices that can be precisely controlled. It is responsible for accurately pressing the cable into the designated wire holder and fixing it based on the global positioning and local feature recognition results.

[0089] Understandable Figure 3 The video also shows a mobile robot wiring scenario, including the cabinet, the tracks and wiring laid around the cabinet.

[0090] Based on the above embodiments, Figure 4 This is a schematic diagram of the sensor layout provided in an embodiment of this disclosure. A LiDAR and IMU are installed on the top of the mobile robot. The LiDAR is used for 360° environmental scanning to acquire high-precision point cloud data and is the main sensor for global positioning and map building. The IMU is installed at the center of gravity or in the center of the mobile robot and integrated into the mobile robot's control core. It provides its own motion state information such as acceleration and angular velocity for motion compensation and improving short-term positioning accuracy. A depth camera and multiple laser ranging modules are installed on the bottom of the mobile robot. For example, the depth camera is installed in the center of the bottom of the mobile robot, facing the direction of the track anchor, to acquire visual images and depth information, enabling the identification and 3D positioning of the track anchor. For example, a laser ranging module is installed at each of the four corners of the base to accurately identify track forks for turning and track switching.

[0091] Based on the above embodiments, Figure 5 A flowchart illustrating another mobile robot control method provided in this embodiment of the disclosure is shown, specifically including the following steps: 1) System power-on initialization; 2) Load map and start global positioning; 3) Receive wiring task and plan global path; 4) Determine if a fork is detected; 5) If a fork is detected, trigger the steering mechanism to smoothly pass through the fork; 6) If no fork is detected, determine if the target cable holder is approaching; 7) If the target cable holder is not approaching, continue with step 4); 8) If the target cable holder is approaching, start depth camera visual positioning to identify the cable holder; 9) Control the wiring module to complete the wiring operation; 10) Determine if the wiring task is completed; 11) If the wiring task is completed, the task ends; 12) If the wiring task is not completed, continue with step 3).

[0092] Understandably, the implementation details of 1) to 12) above are as described in the above embodiments and will not be repeated here.

[0093] Based on the above embodiments, Figure 6 This is a schematic diagram of the structure of a mobile robot control device provided in an embodiment of this disclosure. The mobile robot control device provided in this embodiment can execute the processing flow provided in the mobile robot control method embodiment, such as... Figure 6 As shown, the mobile robot control device includes: The receiving unit 601 is used to receive a wiring task for a mobile robot, wherein the wiring task includes a target cabinet, a wiring path from the current position of the mobile robot to the target cabinet, and at least one cable tie. The first control unit 602 is used to control the mobile robot to move along the wiring path and to calculate the sensor confidence level based on the sensor data collected in real time by multiple sensors configured on the mobile robot. The adjustment unit 603 is used to dynamically adjust the weight coefficients of each sensor data involved in the process of global localization of the mobile robot and local localization of at least one wire fixer based on the sensor confidence level. The second control unit 604 is used to control the mobile robot to decelerate and stop moving when the distance between the mobile robot and the target wire fixer is determined to be less than a preset distance according to global positioning, and at the same time to start local positioning to determine the position of the target wire fixer. The drive unit 605 is used to control the mobile robot to press the cable into the target cable holder and fix it according to the position of the target cable holder.

[0094] The system includes multiple sensors such as lidar, depth cameras, and inertial measurement units.

[0095] Optionally, the first control unit 602 is used for: Calculate the average brightness, contrast, and / or sharpness of the depth image acquired by the depth camera to obtain the first parameter, and evaluate the visual confidence based on the detection confidence of the fixed line output by the pre-trained target detection model and the first parameter. The confidence level of the lidar is assessed based on the quality evaluation results of the laser scanning data collected by lidar, the second parameter obtained by analyzing the effective number of points in a single frame of the laser scanning data and the uniformity of the point cloud distribution, and the matching degree between the scanning area represented by the laser scanning data and the pre-built grid map. The confidence level of the inertial measurement unit is assessed based on the characteristic evaluation results of the measurement data collected by the inertial measurement unit, the third parameter obtained by calculating the variance and drift of the measurement data, and the deviation results predicted based on the measurement data. The sensor confidence levels include visual confidence, lidar confidence, and inertial measurement unit confidence, while the sensor data includes depth images, laser scan data, and measurement data.

[0096] Optionally, the second control unit 604 is used for: The motion data of the mobile robot is obtained from the wheel encoder configured on the mobile robot; wherein, the motion data includes linear velocity, angular velocity and the rotation angle of the mobile robot within a set sampling time interval; The pre-trained motion model calculates the target pose of the chassis mounted on the mobile robot in the map coordinate system based on the set sampling time interval and motion data. Based on the target pose and laser scanning data, the pre-trained observation model calculates the target coordinates in the map coordinate system using the laser measurement point emitted by the lidar configured on the mobile robot. The mobile robot is globally located in real time based on the target pose and target coordinates.

[0097] Optionally, the second control unit 604 is used for: The lidar information is calculated based on the laser scanning data. The lidar information includes at least one of the following: the coordinates of the lidar mounted on the chassis in the chassis coordinate system; the distance from the observation feature point determined by the observation model to the rotation center of the lidar; the angle between the horizontal axis of the local coordinate system with the rotation center as the origin; the angle between the line connecting the center point of the chassis and the rotation center and the horizontal axis of the chassis coordinate system; and the measurement noise of the laser emitted by the lidar in the horizontal and vertical directions in the lidar coordinate system. Based on the target pose and lidar information, the pre-trained observation model calculates the target coordinates in the map coordinate system using the laser measurement point emitted by the lidar configured on the mobile robot.

[0098] Optionally, the mobile robot control device is used for: The robot's attitude change over a set time period is estimated based on measurement data collected by its inertial measurement unit; and the motion data and laser scan data are corrected based on the attitude change; and / or, The laser scan data is matched with a pre-built grid map to determine the absolute pose observation data, and the measurement data and motion data are corrected based on the absolute pose observation data.

[0099] Optionally, the second control unit 604 is used for: Identify fixed objects in depth images using a pre-trained object detection model and output the identification results. The pixel coordinates of the center point of the target wire fixation device in the recognition result are combined with the depth image, and the first three-dimensional coordinates of the center point of the target wire fixation device in the camera coordinate system are calculated by back projection through the camera intrinsic parameters of the depth camera. During the calculation of the first three-dimensional coordinates, the target pixel area is selected based on the center point of the target fixer, and invalid depth values ​​in the target pixel area are removed. Based on the first three-dimensional coordinates and the visual weights in the weighting coefficients, the second three-dimensional coordinates of the target linchpin center point in the camera coordinate system are calculated. The second three-dimensional coordinates of the center point of the target wire anchor are transformed from the camera coordinate system to the base coordinate system of the mobile robot to determine the position of the target wire anchor.

[0100] Optionally, the mobile robot control device is used for: Acquire ranging data collected by the short-range laser ranging module configured on the bottom of the mobile robot; If a sudden change in ranging data is detected, control the mobile robot to stop before the intersection or at the center of the intersection, and control the mobile robot to perform a turning operation.

[0101] Optionally, the mobile robot control device is used for: If the visual confidence level is less than a first preset threshold, in global localization, the weight of motion data or depth image is reduced to a first weight threshold, and the weight of laser scan data and measurement data is increased to a second weight threshold; in local localization, the weight of the target detection model regarding the wire fastener recognition result is reduced to a third weight threshold; and / or, If the LiDAR confidence level continues to decline to the second preset threshold, in both global and local localization, the weights of motion data and measurement data are increased to the fourth weight threshold; and / or, If it is determined that the mobile robot has entered the vicinity of the target wire anchor, then during local localization, the fusion priority of the depth images is increased, and an outlier detection mechanism is triggered. This outlier detection mechanism detects information anomalies in multiple consecutive depth images and reports errors; and / or, If the confidence level of the orbit feature is greater than the third set threshold, the weight of local positioning is increased and the global positioning of the mobile robot is corrected. The confidence level of the orbit feature is evaluated based on the significance and consistency of the jumps in the short-range laser ranging data.

[0102] Figure 6 The mobile robot control device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0103] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. See below for details. Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device 700 in the embodiments of this disclosure. The electronic device 700 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0104] like Figure 7 As shown, the electronic device 700 may include a processing unit 701 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703 to implement the mobile robot control method as described in the embodiments of this disclosure. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0105] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0106] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the mobile robot control method as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0107] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0108] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0109] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0110] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0111] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0113] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0114] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or gateway that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or gateway. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or gateway that includes said element.

[0117] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A mobile robot control method, characterized in that, include: Receive a cabling task for a mobile robot, wherein the cabling task includes a target cabinet, a cabling path from the current position of the mobile robot to the target cabinet, and at least one cable tie. The mobile robot is controlled to move along the wiring path, and the sensor confidence level is calculated based on the sensor data collected in real time by multiple sensors configured on the mobile robot. The weighting coefficients of each sensor data involved in the process of global localization of the mobile robot and local localization of at least one fixed device are dynamically adjusted based on the sensor confidence. When the distance between the mobile robot and the target wire anchor is determined to be less than a preset distance based on the global positioning, the mobile robot is controlled to decelerate and stop moving, and at the same time, the local positioning is activated to determine the position of the target wire anchor. Based on the position of the target cable holder, the mobile robot is controlled to press the cable into the target cable holder and fix it in place.

2. The method according to claim 1, characterized in that, The multiple sensors include a lidar, a depth camera, and an inertial measurement unit. The calculation of sensor confidence based on real-time sensor data collected by the multiple sensors configured on the mobile robot includes: Calculate the average brightness, contrast, and / or sharpness of the depth image acquired by the depth camera to obtain the first parameter, and evaluate the visual confidence based on the detection confidence of the fixed wire output by the pre-trained target detection model and the first parameter. The confidence level of the lidar is assessed based on the quality evaluation results of the laser scanning data collected by the lidar, the second parameter obtained by analyzing the effective number of points in a single frame of the laser scanning data and the uniformity of the point cloud distribution, and the matching degree between the scanning area represented by the laser scanning data and the pre-built grid map. The confidence level of the inertial measurement unit is assessed based on the characteristic evaluation results of the measurement data collected by the inertial measurement unit, the third parameter obtained by calculating the variance and drift of the measurement data, and the deviation results predicted based on the measurement data. The sensor confidence level includes the visual confidence level, the lidar confidence level, and the inertial measurement unit confidence level; the sensor data includes the depth image, the laser scan data, and the measurement data.

3. The method according to claim 1, characterized in that, Global localization of the mobile robot includes: The motion data of the mobile robot provided by the wheel encoder configured on the mobile robot is obtained; wherein, the motion data includes linear velocity, angular velocity and the rotation angle of the mobile robot within a set sampling time interval; The target pose of the chassis mounted on the mobile robot in the map coordinate system is calculated by the pre-trained motion model based on the set sampling time interval and the motion data. The pre-trained observation model calculates the target coordinates in the map coordinate system based on the target pose and laser scanning data, using the laser measurement point emitted by the lidar configured on the mobile robot. The mobile robot is globally located in real time based on the target pose and the target coordinates.

4. The method according to claim 3, characterized in that, The pre-trained observation model calculates the target coordinates in the map coordinate system based on the target pose and laser scanning data, using the laser measurement point emitted by the lidar configured on the mobile robot. This includes: The lidar information is calculated based on the laser scanning data, wherein the lidar information includes at least one of the following: the coordinates of the lidar mounted on the chassis in the chassis coordinate system; the distance from the observation feature point determined by the observation model to the rotation center of the lidar; the angle between the horizontal axis of the local coordinate system with the rotation center as the origin; the angle between the line connecting the center point of the chassis and the rotation center and the horizontal axis of the chassis coordinate system; and the measurement noise of the laser emitted by the lidar in the horizontal and vertical directions in the lidar coordinate system. The pre-trained observation model calculates the target coordinates in the map coordinate system based on the target pose and the lidar information, using the laser measurement point emitted by the lidar configured on the mobile robot.

5. The method according to claim 3, characterized in that, Before the pre-trained motion model is used to perform the motion data based on the set sampling time interval, the method further includes: The inertial measurement unit (IMU) of the mobile robot is used to estimate the robot's attitude change over a set time period, and the motion data and laser scan data are corrected based on the attitude change; and / or, The laser scanning data is matched with a pre-built grid map to determine the absolute pose observation data, and the measurement data and motion data are corrected based on the absolute pose observation data.

6. The method according to claim 2, characterized in that, The step of initiating the local positioning to determine the location of the target wire fastener includes: Identify fixed objects in depth images using a pre-trained object detection model and output the identification results. The pixel coordinates of the center point of the target wire fastener in the recognition result are combined with the depth image, and the first three-dimensional coordinates of the center point of the target wire fastener in the camera coordinate system are calculated by back projection through the camera intrinsic parameters of the depth camera. During the calculation of the first three-dimensional coordinates, the target pixel region is selected based on the center point of the target fixer, and invalid depth values ​​in the target pixel region are removed. Based on the first three-dimensional coordinates and the visual weight in the weighting coefficients, the second three-dimensional coordinates of the center point of the target linchpin in the camera coordinate system are calculated. The second three-dimensional coordinates of the center point of the target wire anchor are transformed from the camera coordinate system to the base coordinate system of the mobile robot to determine the position of the target wire anchor.

7. The method according to claim 1, characterized in that, The method further includes: Acquire ranging data collected by the short-range laser ranging module configured on the bottom of the mobile robot; If a sudden change in the ranging data is detected, the mobile robot is controlled to stop at the intersection or at the center of the intersection, and the mobile robot is controlled to perform a turning operation.

8. The method according to claim 2, characterized in that, Before performing global localization of the mobile robot and / or local localization of the target wire anchor, the method further includes: If the visual confidence level is less than a first preset threshold, in global localization, the weight of motion data or depth image is reduced to a first weight threshold, and the weight of laser scan data and measurement data is increased to a second weight threshold, wherein the motion data is provided by a wheel encoder configured on the mobile robot; in local localization, the weight of the target detection model regarding the wire fastener recognition result is reduced to a third weight threshold; and / or, If the confidence level of the lidar continues to decrease to a second preset threshold, in both global and local positioning, the weights of the motion data and the measurement data are increased to a fourth weight threshold; and / or, If it is determined that the mobile robot has entered the vicinity of the target wire fixer, then during local localization, the fusion priority of the depth images is increased, and an outlier detection mechanism is triggered. This outlier detection mechanism is used to detect information anomalies in multiple consecutive depth images and report errors; and / or, If the confidence level of the orbit feature is greater than a third set threshold, the weight of local positioning is increased and the global positioning of the mobile robot is corrected. The confidence level of the orbit feature is determined based on the significance and consistency of the jumps in short-range laser ranging data.

9. A mobile robot control device, characterized in that, include: A receiving unit is configured to receive a wiring task for a mobile robot, wherein the wiring task includes a target cabinet, a wiring path from the current position of the mobile robot to the target cabinet, and at least one cable tie. The first control unit is used to control the mobile robot to move along the wiring path and to calculate the sensor confidence level based on the sensor data collected in real time by multiple sensors configured on the mobile robot. An adjustment unit is used to dynamically adjust the weighting coefficients of each sensor data involved in the process of global localization of the mobile robot and local localization of the at least one wire fixer based on the sensor confidence level. The second control unit is used to control the mobile robot to decelerate and stop moving when the distance between the mobile robot and the target wire fixer is determined to be less than a preset distance according to the global positioning, and at the same time to start the local positioning to determine the position of the target wire fixer. The drive unit is used to control the mobile robot to press the cable into the target cable holder and fix it according to the position of the target cable holder.

10. A mobile robot, characterized in that, The mobile robot is configured to perform the mobile robot control method as described in any one of claims 1 to 8; wherein, a lidar is fixed to the top of the mobile robot via a bracket, a depth camera is fixed to the bottom of the mobile robot, an inertial measurement unit is installed at the center of the mobile robot and connected to the main control board of the mobile robot, and the drive motor of the mobile robot is also equipped with a wheel encoder.

Images