A method for automatically getting on and off a bus by a robot dog

Abstract

The application provides a kind of machine dog automatic getting on and off control method, it is related to automatic control technical field, obtains the visual image sequence of patrol car entrance and the inertial measurement data of machine dog;Motion compensation is carried out to feature point in visual image sequence by inertial measurement data, and feature point set of patrol car entrance is obtained by screening;The posture data of patrol car is collected, the spatial aggregation is carried out to feature point set, and position correction is carried out in combination with posture data, and the positioning information of patrol car entrance is obtained, the relative pose of machine dog entering ramp is determined according to positioning information;The contact pressure of ramp and the foot end contact state of machine dog are obtained, the joint change feature extraction is carried out to contact pressure of ramp and foot end contact state, and bearing state parameter is obtained;Further, based on relative pose and bearing state parameter, machine dog is controlled to get on and off, and the application can realize progressive fusion determination of visual positioning and bearing state.

Description

Technical Field

[0001] This application relates to the field of automatic control technology, and more specifically, to a method for controlling the automatic boarding and alighting of a robot dog. Background Technology

[0002] Automatic control technology enables robots to complete predetermined tasks without human intervention through sensor perception, controller decision-making, and actuator operation. Automatic vehicle entry and exit control for robot dogs is a crucial element in achieving multi-robot collaborative operations. In scenarios such as patrol and inspection, robot dogs need to frequently and autonomously enter and exit vehicles, and their control precision directly affects the safety and reliability of task execution.

[0003] Robotic dog loading and unloading control typically employs either visual positioning or ultra-wideband positioning to guide the robot back to its designated position. Visual positioning calculates the robot's relative pose by recognizing markers at the rear of the vehicle, while ultra-wideband positioning calculates its position through signal measurements between the vehicle-mounted base station and the robot's tag. However, single-mode visual positioning is susceptible to interference from changes in lighting, viewing angle obstruction, and marker wear, resulting in insufficient positioning stability. Furthermore, relying solely on pose deviation control for homing means that the robot cannot dynamically verify support stability and promptly correct motion parameters after driving onto a slope. Additionally, the lack of a progressive decision-making architecture from visual positioning to load confirmation means that positioning accuracy during actual driving is not effectively guaranteed after positioning deviation correction. Therefore, how to achieve a progressive fusion decision-making system integrating visual positioning and load status to improve the reliability of pose control and slope driving stability during robot loading and unloading is a challenge facing the industry. Summary of the Invention

[0004] This application provides a method for controlling the automatic boarding and alighting of a robot dog, which can realize the progressive fusion judgment of visual positioning and carrying status to improve the reliability of posture control and stability of ramp driving during the boarding and alighting process of the robot dog.

[0005] In a first aspect, this application provides a method for controlling the automatic loading and unloading of a robot dog, the method comprising the following steps: Acquire visual image sequences at the entrance of the patrol vehicle and inertial measurement data of the robot dog; Motion compensation is performed on feature points in the visual image sequence using the inertial measurement data to filter and obtain a set of feature points at the entrance of the patrol vehicle; The attitude data of the patrol vehicle is collected, and the vehicle motion is corrected based on the attitude data to obtain the positioning information of the patrol vehicle entrance. The relative pose of the robot dog entering the ramp is determined based on the positioning information. The ramp contact pressure and the foot contact state of the robot dog are obtained. The joint change features of the ramp contact pressure and the foot contact state are extracted to obtain the bearing state parameters. The robot dog is controlled to get on and off the vehicle based on the relative pose and the carrying state parameters.

[0006] In this embodiment, the robot dog is placed inside the patrol vehicle. The robot dog includes a visual acquisition device, an inertial measurement device, a foot pressure detection device, an electromagnetic fixing claw, an elastic buffer pad, and a controller. The patrol vehicle is a three-in-one patrol vehicle, and the ramp of the patrol vehicle is equipped with pressure sensors.

[0007] In this embodiment, acquiring the visual image sequence at the patrol vehicle entrance and the inertial measurement data of the robot dog specifically includes: A series of visual image frames at the entrance of the patrol vehicle are acquired by a visual acquisition device, and inter-frame feature extraction is performed on the series of visual image frames to obtain a visual image sequence at the entrance of the patrol vehicle. Inertial measurement data of the robot dog is collected during its movement using an inertial measurement device, and the time-series changes of the inertial measurement data are extracted to obtain the robot dog's inertial measurement data.

[0008] In this embodiment, the feature point set for the patrol vehicle entrance is obtained by performing motion compensation on the feature points in the visual image sequence using the inertial measurement data, specifically including: Feature points are extracted from the visual image sequence to obtain visual feature points; Based on the inertial measurement data, the visual feature points are corrected for motion displacement to obtain compensated feature points; The motion consistency between adjacent images is screened for the compensated feature points, and compensated feature points with abnormal motion changes are removed to obtain the feature point set of the patrol vehicle entrance.

[0009] In this embodiment, collecting the attitude data of the patrol vehicle specifically includes: The attitude detection device collects the vehicle attitude information during the patrol vehicle's movement, and the continuous change of the vehicle attitude information is extracted to obtain the amount of vehicle attitude change. Based on the change in vehicle attitude, the attitude change of the vehicle attitude information at adjacent acquisition times is corrected to obtain the attitude data of the patrol vehicle.

[0010] In this embodiment, the process of correcting the vehicle motion based on the posture data to obtain the location information of the patrol vehicle entrance specifically includes: The feature point set is aggregated according to its spatial distribution, and stable feature points that continuously appear in adjacent visual image sequences are extracted to obtain the entry stable feature point set; Based on the attitude data, vehicle motion compensation is performed on the stable feature point set at the entrance to obtain the change in the position of the entrance feature. Based on the change in the location of the entrance feature points, the set of stable entrance feature points is spatially consistent to obtain the entrance positioning feature set. The coordinates of the entrance positioning feature set are corrected using the attitude data, and the positioning is corrected based on the position changes in continuous visual image frames to obtain the positioning information of the patrol vehicle entrance.

[0011] In this embodiment, determining the relative pose of the robot dog entering the ramp based on the positioning information specifically includes: Based on the positioning information, establish the patrol vehicle entrance coordinate reference, obtain the current position of the robot dog, and obtain the robot dog's position coordinates; The robot dog's position coordinates are converted to the patrol vehicle's entrance coordinate reference to obtain the robot dog's position deviation relative to the entrance. The robot dog's current direction of travel is determined based on the inertial measurement data, and the directional deviation between the direction of travel and the entrance direction is calculated. The relative pose of the robot dog as it enters the ramp is determined by the positional deviation and the directional deviation.

[0012] In this embodiment, obtaining the ramp contact pressure and the robot dog's foot contact state specifically includes: The ramp pressure data during the robot dog's entry into the ramp is collected by a pressure sensor, and the ramp pressure data is continuously changed and extracted to obtain the ramp contact pressure. The pressure change data of the robot dog's feet is collected by a foot pressure detection device, and the support state is identified by the pressure change data to obtain the contact state of the robot dog's feet.

[0013] In this embodiment, the joint change feature extraction of the ramp contact pressure and the foot contact state is performed to obtain the bearing state parameters, specifically including: The pressure change is extracted from the ramp contact pressure to obtain ramp pressure time series data; The foot support change is extracted from the foot contact state to obtain foot support time series data; The slope pressure time series data and the foot support time series data are synchronously correlated according to the acquisition time, and the time series correspondence between pressure change and support change is extracted to obtain pressure-support correlation features. Based on the pressure support correlation characteristics, the load-bearing state range of the robot dog on the ramp is determined, and the load-bearing state parameters are obtained.

[0014] In this embodiment, controlling the robot dog to get on and off the vehicle based on the relative pose and the carrying state parameters specifically includes: Based on the position and orientation deviations in the relative pose, determine whether the robot dog meets the pose conditions for entering the ramp, and obtain the pose determination result. When the pose determination result is that the pose condition is met, the support stability of the robot dog on the slope is determined according to the load state interval in the load state parameter, and the support determination result is obtained. When the support determination result is that the support is stable, speed control commands and steering control commands are generated according to the relative pose to control the robot dog to drive along the ramp and complete the getting on and off the vehicle.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The technical solution provided in this application enables a progressive fusion determination of visual positioning and load-bearing status. Firstly, by acquiring visual image sequences at the patrol vehicle entrance and inertial measurement data of the robot dog, and using the inertial measurement data to compensate for motion at visual feature points, visual feature shifts caused by robot dog movement and vehicle vibration can be suppressed. Secondly, by collecting patrol vehicle attitude data and performing spatial aggregation and position correction on the feature point set, the entrance positioning information is transformed to the vehicle coordinate system, and the relative pose of the robot dog entering the ramp is determined. This enables precise mapping of visual positioning results from the pixel plane to physical space, thereby solving the problem that single visual positioning is easily affected by changes in viewing angle and vehicle movement. Then… By acquiring the slope contact pressure and the robot dog's foot contact state and extracting their joint change features, a load-bearing state parameter characterizing the robot dog's support stability on the slope can be obtained. This allows for the establishment of a physical interaction layer that directly verifies the robot dog's slope driving state, which helps to compensate for the lack of physical feedback in pure pose control. Finally, by using the relative pose and load-bearing state parameters as the control basis, a two-level progressive judgment architecture of pose alignment first and load confirmation later can be realized. This enables dynamic verification based on the load-bearing state during the actual entry into the slope after the positioning deviation is corrected, thereby improving the reliability of pose control and slope driving stability during the robot dog's getting on and off the vehicle.

[0016] In summary, the technical solution adopted in this application can realize the progressive fusion judgment of visual positioning and load status, so as to improve the reliability of pose control and slope driving stability of the robot dog during the process of getting on and off the vehicle. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a robot dog automatic loading and unloading control method provided in this application; Figure 2 This is a schematic diagram of the robot dog getting on and off the vehicle in this application; Figure 3 This is an exemplary flowchart of the feature point set for determining the entrance of a patrol vehicle, provided in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Example 1: To better understand the technical solution of this application, the above technical solution will be described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 As shown in the figure, this is a flowchart of a robot dog automatic loading and unloading control method according to this embodiment of the present application. The control method includes the following steps: In step S1, a visual image sequence at the entrance of the patrol vehicle and inertial measurement data of the robot dog are acquired.

[0021] It should be noted that, in this embodiment, the robot dog is placed inside the patrol vehicle. The robot dog includes a visual acquisition device, an inertial measurement device, a foot pressure detection device, an electromagnetic fixing claw, an elastic buffer pad, and a controller. The patrol vehicle is a three-in-one patrol vehicle, and the ramp of the patrol vehicle is equipped with pressure sensors. The three-in-one patrol vehicle refers to a mobile unmanned platform that integrates carrying, transportation, communication relay, and energy supply functions, used to carry the robot dog and provide transportation, communication, and charging services. Preferably, [further details can be found in the original text]. Figure 2As shown in the figure, this is a schematic diagram of the robot dog getting on and off a vehicle according to this application. The ramp of the patrol vehicle is located at the rear of the vehicle, connecting the vehicle's floor to the ground to form a physical passage for the robot dog to get on and off. A visual acquisition device is installed on the robot dog's head or front of its torso to acquire visual image sequences of the patrol vehicle's entrance area and the ramp area. An inertial measurement unit is installed inside the robot dog's torso to acquire three-axis acceleration and three-axis angular velocity data during the robot dog's movement; this data can be used to calculate the robot dog's attitude changes and displacement. A foot pressure detection device is installed on the bottom of each foot of the robot dog to detect... The system measures the contact pressure between each foot and the ground or ramp; electromagnetic locking claws are installed in the storage compartment of the patrol vehicle to lock the robot dog after it has fully entered the compartment; elastic buffer pads are laid on the bottom and side walls of the storage compartment to absorb vibration energy during vehicle movement; the controller is electrically connected to the vision acquisition equipment, inertial measurement equipment, foot pressure detection equipment, electromagnetic locking claws, and pressure sensors, and can receive data collected by each device and output control commands; the pressure sensor is embedded in the ramp surface, and generates a corresponding contact pressure distribution signal when the robot dog's feet act on the ramp surface.

[0022] In this embodiment, the visual image sequence at the entrance of the patrol vehicle and the inertial measurement data of the robot dog can be obtained in the following manner: A series of visual image frames at the entrance of the patrol vehicle are acquired by a visual acquisition device, and inter-frame feature extraction is performed on the series of visual image frames to obtain a visual image sequence at the entrance of the patrol vehicle. Inertial measurement data of the robot dog is collected during its movement using an inertial measurement device, and the time-series changes of the inertial measurement data are extracted to obtain the robot dog's inertial measurement data.

[0023] It should be noted that the visual image sequence in this application refers to image data containing the outline of the patrol vehicle entrance structure and the location of the ramp entrance. Inertial measurement data refers to the data sequence of linear acceleration and angular velocity changes over time during the robot dog's spatial motion.

[0024] In practice, firstly, continuous visual image frames at the entrance of the patrol vehicle can be acquired using a visual acquisition device. The acquired multiple frames are arranged in chronological order of acquisition time to obtain continuous visual image frames. The Shi-Tomasi corner detection algorithm is used to extract corner points from adjacent frames in the continuous visual image frames, resulting in a set of corner point coordinates for adjacent frames. The set of corner point coordinates for adjacent frames is then input into the calcOpticalFlowPyrLK function in the OpenCV library. After calculation, the function outputs the pixel offset direction and offset amount of the corner points between adjacent frames. The pixel offset direction is then compared with the expected motion direction for consistency. Corner points with consistent pixel offset direction and offset amounts less than the maximum possible pixel displacement between adjacent frames are retained. The image formed by the retained corner points is used as the visual image sequence at the entrance of the patrol vehicle. The expected motion direction refers to the overall motion direction of the robot dog as it travels along the ramp, which can be calculated based on inertial measurement data. Then, the robot dog's triaxial acceleration and triaxial angular velocity data are collected using an inertial measurement unit (IMU). The collected triaxial acceleration and angular velocity data are arranged according to the sampling time to obtain inertial measurement time-series data. The inertial measurement time-series data is then processed using a sliding window method. The average value of the triaxial acceleration data within the window is obtained as the window's average acceleration value, and the average value of the triaxial angular velocity data within the window is obtained as the window's average angular velocity value. The window's average acceleration and average angular velocity values ​​are used as the motion state features corresponding to that window. By arranging the motion state features corresponding to all windows in chronological order, the robot dog's inertial measurement data can be obtained.

[0025] In step S2, motion compensation is performed on the feature points in the visual image sequence using the inertial measurement data to obtain the feature point set of the patrol vehicle entrance.

[0026] Preferred, Reference Figure 3 As shown, this figure is an exemplary flowchart for determining the feature point set of the patrol vehicle entrance according to the present application. In this embodiment, the feature point set of the patrol vehicle entrance is obtained by performing motion compensation on the feature points in the visual image sequence using the inertial measurement data, which can be achieved by the following steps: In step S21, feature points are extracted from the visual image sequence to obtain visual feature points; In step S22, the visual feature points are corrected for motion displacement based on the inertial measurement data to obtain compensated feature points; In step S23, the motion consistency between adjacent images is screened for the compensation feature points, and compensation feature points with abnormal motion changes are removed to obtain the feature point set of the patrol vehicle entrance.

[0027] It should be noted that, in this application, visual feature points refer to pixel positions in a visual image sequence where grayscale changes drastically, used to characterize the spatial position of the patrol vehicle entrance structure outline in the image. Motion displacement correction refers to the process of correcting the image coordinates of visual feature points using the robot dog's motion state reflected by inertial measurement data, used to eliminate the feature point position offset caused by the robot dog's motion; compensation feature points refer to visual feature points whose coordinate positions are corrected after motion displacement correction; the feature point set of the patrol vehicle entrance refers to the set of feature points that are selected and retained, which can stably characterize the patrol vehicle entrance structure outline.

[0028] In specific implementation, firstly, the Shi-Tomasi corner detection algorithm is used to extract corners from the visual image sequence, obtaining the pixel coordinates of each corner in the visual image sequence, and all extracted corners are used as visual feature points. Then, the robot dog's attitude change is determined based on the three-axis angular velocity data in the inertial measurement data. Based on the attitude change, the three-axis acceleration data in the inertial measurement data is transformed from the robot dog coordinate system to the world coordinate system, eliminating the gravitational acceleration component in the world coordinate system, and obtaining the robot dog's linear acceleration in the world coordinate system. This linear acceleration is integrated based on the time interval between adjacent visual image acquisition moments to obtain the robot dog's motion displacement estimate. This motion displacement estimate is transformed to the pixel displacement in the image pixel coordinate system according to the projection model of the visual acquisition device. The pixel displacement is subtracted from the pixel coordinates of the visual feature points to obtain the compensated feature points. Finally, the motion direction of the compensated feature points in adjacent visual image sequences is extracted, the distribution of all compensated feature points' motion directions is statistically analyzed, and compensated feature points whose motion directions deviate from the overall distribution's main direction are removed. The remaining compensated feature points are used as the feature point set for the patrol vehicle's entrance.

[0029] It should be noted that the motion direction refers to the pixel displacement direction of the compensation feature point between adjacent visual image sequences; the overall distribution main direction refers to the direction corresponding to the peak interval after the motion directions of all compensation feature points are statistically analyzed by histogram; motion consistency screening refers to the judgment based on the degree of deviation between the motion direction of the compensation feature point and the overall distribution main direction, used to remove compensation feature points whose motion direction deviates from the overall trend.

[0030] In step S3, the attitude data of the patrol vehicle is collected, and the vehicle motion is corrected based on the attitude data to obtain the positioning information of the patrol vehicle entrance. The relative pose of the robot dog entering the ramp is determined based on the positioning information.

[0031] In this embodiment, the attitude data of the patrol vehicle can be collected in the following manner: The attitude detection device collects the vehicle attitude information during the patrol vehicle's movement, and the continuous change of the vehicle attitude information is extracted to obtain the amount of vehicle attitude change. Based on the change in vehicle attitude, the attitude change of the vehicle attitude information at adjacent acquisition times is corrected to obtain the attitude data of the patrol vehicle.

[0032] It should be noted that the vehicle attitude information in this application refers to the pitch and yaw angle data of the patrol vehicle during its operation; the pitch angle of the patrol vehicle refers to the angle of rotation of the vehicle body around the lateral axis. The yaw angle of the vehicle body refers to the angle of rotation of the vehicle body around the vertical axis; the change in vehicle attitude refers to the difference in vehicle attitude information between adjacent data collection times; the attitude data of the patrol vehicle refers to the data sequence of the changes in the pitch and yaw angles of the patrol vehicle over time during its operation.

[0033] In practice, firstly, the pitch and yaw angles of the patrol vehicle are continuously collected by an attitude detection device during its movement. The collected pitch and yaw angles are then arranged according to the collection time to obtain the vehicle's attitude information. Next, the pitch angle at each collection time is subtracted from the pitch angle at the previous collection time to obtain the pitch angle change at that time. Similarly, the yaw angle at each collection time is subtracted from the yaw angle at the previous collection time to obtain the yaw angle change at that time. Finally, the pitch and yaw angle changes at each time are arranged according to the collection time to obtain the vehicle's attitude change. Finally, the vehicle attitude information is corrected based on the changes in vehicle attitude. For each acquisition moment, if the pitch angle change at that moment is greater than the pitch angle change threshold and the trend of this pitch angle change is inconsistent with the pitch angle changes at the previous and subsequent acquisition moments, then the pitch angle at that moment is replaced with the pitch angle at the previous acquisition moment. If the yaw angle change at that moment is greater than the yaw angle change threshold and the trend of this yaw angle change is inconsistent with the yaw angle changes at the previous and subsequent acquisition moments... If the trends are inconsistent, the yaw angle at that moment is replaced with the yaw angle at the previous acquisition moment; the corrected pitch and yaw angle sequences are used as the attitude data of the patrol vehicle; optionally, the pitch angle change threshold and the yaw angle change threshold can both be set to 10 degrees. When the pitch angle change between adjacent acquisition moments exceeds 10 degrees or the yaw angle change exceeds 10 degrees and the change is inconsistent with the trend of the change before and after the acquisition moment, the data is determined to be an abnormal jump and is replaced; when the attitude angle change exceeds the threshold but the change trend is consistent with the trend of the change before and after the acquisition moment, the data is retained.

[0034] It should be noted that continuous change extraction refers to the process of performing differential calculations on the attitude angles at adjacent acquisition times in the vehicle attitude information to obtain the change in attitude angles; attitude change correction refers to the process of using the change in vehicle attitude to remove abnormal attitude angle data. If the change in attitude angle at a certain time is greater than the change threshold and the change trend is inconsistent with the change trend of the previous and subsequent times, the corresponding attitude angle data is judged as abnormal data and replaced. If the change trend is consistent, it is retained.

[0035] In this embodiment, the vehicle motion correction is performed on the feature point set based on the attitude data to obtain the positioning information of the patrol vehicle entrance, which can be specifically achieved in the following manner: The feature point set is aggregated according to its spatial distribution, and stable feature points that continuously appear in adjacent visual image sequences are extracted to obtain the entry stable feature point set; Based on the attitude data, vehicle motion compensation is performed on the stable feature point set at the entrance to obtain the change in the position of the entrance feature. Based on the change in the location of the entrance feature points, the set of stable entrance feature points is spatially consistent to obtain the entrance positioning feature set. The coordinates of the entrance positioning feature set are corrected using the attitude data, and the positioning is corrected based on the position changes in continuous visual image frames to obtain the positioning information of the patrol vehicle entrance.

[0036] It should be noted that the aggregation based on spatial distribution in this application refers to the process of grouping feature points that are spatially adjacent into the same region. The stable feature point set at the entrance refers to the set of feature points that appear continuously in a sequence of adjacent visual images, selected from the feature point set, used to characterize the fixed-position portion of the patrol vehicle entrance structure outline. Vehicle motion compensation refers to using attitude data to eliminate the influence of patrol vehicle motion on the spatial position of the stable feature point set at the entrance. The change in entrance feature position refers to the positional offset of the stable feature point set at the entrance in the spatial coordinate system after vehicle motion compensation. Spatial consistency filtering refers to judging consistency based on the positional change of each feature point in the stable feature point set at the entrance, used to remove feature points whose positional change deviates from the overall distribution. The entrance positioning feature set refers to the set of feature points retained after spatial consistency filtering. The positioning information of the patrol vehicle entrance refers to the spatial positional distribution of the entrance positioning feature set in the vehicle coordinate system, used to describe the position and orientation of the patrol vehicle entrance in the vehicle coordinate system.

[0037] In specific implementation, firstly, the pixel coordinates of each feature point in the feature point set are clustered according to the Euclidean distance between the pixel coordinates. Feature points whose Euclidean distance is less than the clustering distance threshold are grouped into the same cluster. Feature points that appear in multiple consecutive frames of visual image sequences are extracted from each cluster, and the extracted feature points are combined into a stable feature point set for the entrance. Optionally, the clustering distance threshold can be set between 20 and 50 pixels according to the pixel distance between adjacent entrance structure feature points in the visual image sequence. Adjacent feature points in the entrance structure contour are within this pixel distance range. If the distance is less than this range, adjacent feature points of the same structure will be mistakenly divided into different clusters. If the distance is greater than this range, feature points of different structures will be merged into the same cluster. Secondly, an entrance reference plane is established based on the geometric constraints of the structural plane where the patrol vehicle entrance is located. The pixel coordinates of each feature point in the stable feature point set at the entrance are mapped to the entrance reference plane to obtain the position parameters of each feature point on the entrance reference plane. The position parameters of each feature point are then subjected to a reverse rotation based on the pitch and yaw angles in the attitude data to obtain the position parameters after vehicle motion compensation. The difference between the position parameters before and after compensation is calculated to obtain the position offset of each feature point. All position offsets are arranged according to the feature points to obtain the entrance feature position change. Then, the position offset direction of each feature point in the entrance feature position change is statistically analyzed. Feature points whose position offset direction is inconsistent with the overall offset direction are removed, and the remaining feature points form the entrance positioning feature set. Finally, a coordinate system transformation is applied to the position parameters of each feature point in the entrance positioning feature set based on the pitch and yaw angles in the attitude data. The transformed position parameters are used as corrected position parameters. The corrected position parameters of the same feature point in multiple consecutive visual images are averaged to obtain the positioning position parameters of that feature point. The entrance center position is determined based on the spatial distribution of the positioning position parameters of all feature points in the entrance positioning feature set. The feature points in the entrance positioning feature set are fitted with straight line directions. The entrance orientation is determined based on the fitted entrance structure direction. The entrance center position and the entrance orientation are used together as the positioning information of the patrol vehicle entrance.

[0038] In this embodiment, the relative pose of the robot dog entering the ramp can be determined based on the positioning information in the following manner: Based on the positioning information, establish the patrol vehicle entrance coordinate reference, obtain the current position of the robot dog, and obtain the robot dog's position coordinates; The robot dog's position coordinates are converted to the patrol vehicle's entrance coordinate reference to obtain the robot dog's position deviation relative to the entrance. The robot dog's current direction of travel is determined based on the inertial measurement data, and the directional deviation between the direction of travel and the entrance direction is calculated. The relative pose of the robot dog as it enters the ramp is determined by the positional deviation and the directional deviation.

[0039] It should be noted that the patrol vehicle entrance coordinate reference in this application refers to a spatial coordinate system established with the center point of the patrol vehicle entrance as the origin and the direction of the entrance orientation as the coordinate axis. The robot dog's position coordinates refer to the robot dog's three-dimensional position data in space. Position deviation refers to the spatial offset of the robot dog's position coordinates relative to the origin of the patrol vehicle entrance coordinate reference. Orientation deviation refers to the angular difference between the robot dog's travel direction and the patrol vehicle entrance orientation. Relative pose refers to a data set containing the robot dog's spatial position offset and angular difference relative to the patrol vehicle entrance.

[0040] In specific implementation, firstly, the spatial coordinates of the patrol vehicle's entrance center point and the entrance orientation vector are extracted from the positioning information. A spatial coordinate system is established with the entrance center point as the origin and the entrance orientation direction as the coordinate axis. This spatial coordinate system serves as the patrol vehicle's entrance coordinate reference. Based on the robot dog's position in the visual image sequence and the inertial measurement data, the robot dog's current three-dimensional position coordinates in space are determined, and these three-dimensional position coordinates are used as the robot dog's position coordinates. Secondly, the robot dog's position coordinates are transformed from the original spatial coordinate system to the patrol vehicle's entrance coordinate reference, obtaining the robot dog's coordinate values ​​under the patrol vehicle's entrance coordinate reference. The lateral and longitudinal offsets of these coordinate values ​​relative to the origin are used as the robot dog's position deviation relative to the entrance. Then, the robot dog's current travel direction vector is determined based on the inertial measurement data. The robot dog's current travel direction vector and the entrance orientation direction vector are projected onto a horizontal plane, and the angle between the two direction vectors on the horizontal plane is calculated. This angle is used as the direction deviation between the travel direction and the entrance direction. Finally, the position deviation and direction deviation are combined into a two-dimensional array, and the resulting two-dimensional array is used as the robot dog's relative pose when entering the ramp.

[0041] In step S4, the ramp contact pressure and the foot contact state of the robot dog are obtained, and the joint change feature extraction of the ramp contact pressure and the foot contact state is performed to obtain the bearing state parameters.

[0042] In this embodiment, the ramp contact pressure and the robot dog's foot contact state can be obtained in the following ways: The ramp pressure data during the robot dog's entry into the ramp is collected by a pressure sensor, and the ramp pressure data is continuously changed and extracted to obtain the ramp contact pressure. The pressure change data of the robot dog's feet is collected by a foot pressure detection device, and the support state is identified by the pressure change data to obtain the contact state of the robot dog's feet.

[0043] It should be noted that the ramp pressure data in this application refers to the pressure value sequence obtained after analog-to-digital conversion of the original voltage signal collected by the pressure sensor; ramp contact pressure refers to the pressure change sequence obtained after continuously extracting the ramp pressure data; pressure change data refers to the data set of foot pressure values ​​collected by the foot pressure detection device as a function of time; support state recognition refers to the process of determining whether the foot is on the ground or off the ground based on the foot pressure value; foot contact state refers to the set of foot on the ground and off-ground states of the robot dog; contact threshold refers to the pressure value boundary used to distinguish between foot on the ground and off the ground. When the foot pressure value is greater than the contact threshold, it is determined to be on the ground; when the foot pressure value is less than or equal to the contact threshold, it is determined to be off the ground.

[0044] In practice, firstly, pressure sensors embedded in the ramp surface collect pressure signals when the robot dog's paws act on the ramp surface. The pressure sensors convert the pressure signals into voltage signals, which are then converted into digital pressure values ​​by an analog-to-digital converter. The digital pressure values ​​at each sampling time are arranged into a time series, which is then used as the ramp pressure data. Next, the ramp pressure data is processed according to the sampling time order, and the pressure value at each sampling time is subtracted from the pressure value at the previous sampling time to obtain the pressure change at that time. The pressure changes at each time are then arranged according to the sampling time, and the resulting sequence of pressure changes is used as the ramp contact pressure. Then, pressure values ​​of each foot are collected by foot pressure detection devices installed on the bottom of each foot of the robot dog. The pressure values ​​of each foot are arranged according to the sampling time to obtain a time series of pressure values ​​of each foot. The time series of pressure values ​​of all feet are combined into pressure change data. The pressure values ​​of each foot are compared with a contact threshold. The foot with a pressure value greater than the contact threshold is determined to be in the ground state, the foot with a pressure value equal to the contact threshold is determined to be in the air-off state, and the foot with a pressure value less than the contact threshold is determined to be in the air-off state. Optionally, the contact threshold can be set to a positive value close to zero. The reason for setting this value is that when the foot is in the ground, the pressure sensor will output a pressure value significantly greater than zero, while when the foot is off the ground, due to sensor noise and residual signals, the output value is usually not an absolute zero value. Setting this threshold can effectively distinguish between the two states of being in the ground and being off the ground. The ground state and air-off state of each foot are arranged according to the foot number and the sampling time. The set of foot states obtained after arrangement is taken as the foot contact state of the robot dog.

[0045] In this embodiment, the bearing capacity parameters are obtained by jointly extracting the variation features of the ramp contact pressure and the foot contact state, which can be done in the following way: The pressure change is extracted from the ramp contact pressure to obtain ramp pressure time series data; The foot support change is extracted from the foot contact state to obtain foot support time series data; The slope pressure time series data and the foot support time series data are synchronously correlated according to the acquisition time, and the time series correspondence between pressure change and support change is extracted to obtain pressure-support correlation features. Based on the pressure support correlation characteristics, the load-bearing state range of the robot dog on the ramp is determined, and the load-bearing state parameters are obtained.

[0046] In practice, firstly, the pressure changes at each moment in the ramp contact pressure are arranged in chronological order of sampling time, and the resulting sequence of pressure changes is used as the ramp pressure time series data. Secondly, the ground contact state and off-ground state of each foot are binarized and encoded, with the ground contact state encoded as a value 1 and the off-ground state encoded as a value 0. The encoded foot state values ​​are then arranged into a two-dimensional matrix according to the sampling time and foot number, and this two-dimensional matrix is ​​used as the foot support time series data. Then, the ramp pressure time series data and foot support time series data are aligned along the time axis according to the sampling time. Each sampling time is traversed, and the value of the pressure change and each foot code value at that time are read. Sampling times where the pressure change is greater than the pressure change threshold are marked as pressure change events. Sampling times where any foot code value switches between a value of 1 and a value of 0 are marked as support state switching events. The number of times the pressure change events and support state switching events correspond on the time axis is counted to obtain the number of associated events. The total number of support state switching events in the foot contact state is counted. When the total number of support state switching events is greater than zero, the ratio of the number of associated events to the total number of support state switching events is calculated, and this ratio is used as the pressure support association feature. When the total number of support state switching events is equal to zero, the pressure support association feature is marked as zero. Finally, when the ratio is greater than the first proportional threshold, the load-bearing state parameter is marked as the first load-bearing state value; when the ratio is less than or equal to the first proportional threshold and greater than the second proportional threshold, the load-bearing state parameter is marked as the second load-bearing state value; when the ratio is less than or equal to the second proportional threshold, the load-bearing state parameter is marked as the third load-bearing state value. Optionally, the pressure change threshold can be set according to the range of the ramp pressure sensor and the pressure amplitude of the robot dog's foot acting on the ramp surface. Both the first and second proportional thresholds can be set based on empirical data, for example, the first proportional threshold can be 0.8 and the second proportional threshold can be 0.4.

[0047] It should be noted that the ramp pressure time series data in this application refers to the data sequence formed by arranging the pressure changes at each moment in the ramp contact pressure in chronological order; the foot support time series data refers to the data sequence formed by arranging the foot contact state and the foot off the ground state at each moment in chronological order; the pressure change event refers to the sampling moment when the pressure change is greater than the pressure change threshold; the support state switching event refers to the sampling moment when any foot code value switches between a value of 1 and a value of 0; the pressure support correlation feature refers to the degree of correlation between the pressure change event and the support state switching event on the time axis; the load-bearing state interval refers to the load-bearing stability level of the robot dog on the ramp according to the degree of correlation; and the load-bearing state parameter refers to the load-bearing state interval in which the robot dog is currently located.

[0048] In step S5, the robot dog is controlled to get on and off the vehicle based on the relative pose and the carrying state parameters.

[0049] In this embodiment, controlling the robot dog to get on and off the vehicle based on the relative pose and the carrying state parameters can be done in the following way: Based on the position and orientation deviations in the relative pose, determine whether the robot dog meets the pose conditions for entering the ramp, and obtain the pose determination result. When the pose determination result is that the pose condition is met, the support stability of the robot dog on the slope is determined according to the load state interval in the load state parameter, and the support determination result is obtained. When the support determination result is that the support is stable, speed control commands and steering control commands are generated according to the relative pose to control the robot dog to drive along the ramp and complete the getting on and off the vehicle.

[0050] It should be noted that the pose conditions in this application refer to the range of positional and directional deviations that the robot dog needs to meet to enter the ramp; positional deviation refers to the lateral and longitudinal offset of the robot dog relative to the centerline of the ramp entrance; directional deviation refers to the angle between the robot dog's direction of travel and the direction of the ramp entrance centerline; support stability refers to the degree of synchronization between the robot dog's foot support switching and the ramp pressure change in time; speed control command refers to the command value that controls the magnitude of the robot dog's travel speed; steering control command refers to the command value that controls the robot dog's direction of travel deflection, including the deflection direction and deflection angle.

[0051] In specific implementation, firstly, the position deviation and orientation deviation in the relative pose are read. The position deviation value is compared with the position deviation limit, and the orientation deviation value is compared with the orientation deviation limit. When the position deviation value is less than or equal to the position deviation limit and the orientation deviation value is less than or equal to the orientation deviation limit, the pose determination result is marked as satisfied. When the position deviation value is greater than the position deviation limit or the orientation deviation value is greater than the orientation deviation limit, the pose determination result is marked as unsatisfactory. The position deviation limit refers to the maximum lateral offset allowed for the robot dog to enter the ramp, which is half the difference between the ramp width and the robot dog's body width. The orientation deviation limit refers to the maximum heading angle deviation allowed for the robot dog to enter the ramp. The orientation deviation limit is determined based on the geometric relationship between the ramp length, ramp width, and robot dog's body width. The tangent of the orientation deviation limit is equal to half the difference between the ramp width and the robot dog's body width divided by the ramp length. Then, when the pose determination result is marked as satisfied, the bearing state parameter is read. When the bearing state parameter is the first bearing state value, the support determination result is marked as stable; when the bearing state parameter is the second or third bearing state value, the support determination result is marked as unstable.

[0052] Finally, when the support judgment result is marked as stable, the controller determines the speed adjustment coefficient based on the ratio of the absolute value of the position deviation to the position deviation limit. The larger the absolute value of the position deviation, the smaller the speed adjustment coefficient. The maximum speed value is multiplied by the speed adjustment coefficient to obtain the target speed value, which is then filled into the data field of the speed control command. The maximum speed value refers to the maximum speed that the robot dog is allowed to reach when driving normally on a slope. This maximum speed value can be set within the safe operating speed range of the robot dog on a slope based on actual experience. The controller calculates the steering ratio coefficient based on the ratio of the absolute value of the direction deviation to the direction deviation limit. The direction deviation limit is multiplied by the steering ratio coefficient to obtain the target deflection angle. The deflection direction is determined based on the positive or negative sign of the direction deviation value, and the target deflection angle and deflection direction are filled into the data field of the steering control command. The controller sends the speed control command and steering control command to the foot drive unit through the communication interface. The foot drive unit then... The controller reads the target speed value from the speed control command and adjusts the swing frequency of each foot according to the target speed value. It also reads the target deflection angle and direction from the steering control command and adjusts the swing phase difference of each foot according to the target deflection angle and direction, driving the robot dog along the ramp and into the patrol vehicle's storage compartment. After the robot dog is fully inside the storage compartment, the controller sends a locking command to the electromagnetic locking claws. Upon receiving the locking command, the electromagnetic locking claws close, securing the robot dog inside the storage compartment. An elastic buffer pad absorbs the impact energy between the robot dog's feet and the bottom of the storage compartment during the robot dog's entry, completing the getting on and off actions. When the robot dog performs the dismounting action, the controller sends an unlocking command to the electromagnetic locking claws. Upon receiving the unlocking command, the electromagnetic locking claws open, releasing the robot dog from the lock. After unlocking, a reverse speed control command and a steering control command are generated based on the relative posture and the load-bearing state parameters, causing the robot dog to move from the storage compartment along the ramp to the ground.

[0053] It should be noted that the controller is an embedded control board inside the robot dog, and the foot drive unit is the actuator that drives the movement of each foot of the robot dog. The speed control command is a communication data packet containing a target speed value field, and the steering control command is a communication data packet containing a target deflection angle field and a deflection direction field. The communication interface is the data transmission channel between the controller and the foot drive unit. The controller sends the target speed value, target deflection angle, and deflection direction to the foot drive unit in the form of communication data packets through the communication interface. The foot drive unit parses the target speed value, target deflection angle, and deflection direction from the communication data packets and then drives the foot movement. Optionally, the target deflection angle is equal to the ratio of the absolute value of the direction deviation to the direction deviation limit multiplied by the direction deviation limit.

[0054] In summary, the technical solution adopted in this application can realize the progressive fusion judgment of visual positioning and load status, so as to improve the reliability of pose control and slope driving stability of the robot dog during the process of getting on and off the vehicle.

[0055] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0056] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0057] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for controlling the automatic boarding and alighting of a robot dog, characterized in that, The control method includes the following steps: Acquire visual image sequences at the entrance of the patrol vehicle and inertial measurement data of the robot dog; Motion compensation is performed on feature points in the visual image sequence using the inertial measurement data to filter and obtain a set of feature points at the entrance of the patrol vehicle; The attitude data of the patrol vehicle is collected, and the vehicle motion is corrected based on the attitude data to obtain the positioning information of the patrol vehicle entrance. The relative pose of the robot dog entering the ramp is determined based on the positioning information. The ramp contact pressure and the foot contact state of the robot dog are obtained. The joint change features of the ramp contact pressure and the foot contact state are extracted to obtain the bearing state parameters. The robot dog is controlled to get on and off the vehicle based on the relative pose and the carrying state parameters.

2. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, The robot dog is placed inside the patrol vehicle. The robot dog includes a visual acquisition device, an inertial measurement device, a foot pressure detection device, an electromagnetic fixing claw, an elastic buffer pad, and a controller. The patrol vehicle is a three-in-one patrol vehicle, and the ramp of the patrol vehicle is equipped with pressure sensors.

3. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, Acquiring the visual image sequence at the patrol vehicle entrance and the inertial measurement data of the robot dog specifically includes: A series of visual image frames at the entrance of the patrol vehicle are acquired by a visual acquisition device, and inter-frame feature extraction is performed on the series of visual image frames to obtain a visual image sequence at the entrance of the patrol vehicle. Inertial measurement data of the robot dog is collected during its movement using an inertial measurement device, and the time-series changes of the inertial measurement data are extracted to obtain the robot dog's inertial measurement data.

4. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, Motion compensation is performed on feature points in the visual image sequence using the inertial measurement data to obtain the feature point set for the patrol vehicle entrance, specifically including: Feature points are extracted from the visual image sequence to obtain visual feature points; Based on the inertial measurement data, the visual feature points are corrected for motion displacement to obtain compensated feature points; The motion consistency between adjacent images is screened for the compensated feature points, and compensated feature points with abnormal motion changes are removed to obtain the feature point set of the patrol vehicle entrance.

5. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, The specific data collected on the attitude of the patrol vehicle includes: The attitude detection device collects the vehicle attitude information during the patrol vehicle's movement, and the continuous change of the vehicle attitude information is extracted to obtain the amount of vehicle attitude change. Based on the change in vehicle attitude, the attitude change of the vehicle attitude information at adjacent acquisition times is corrected to obtain the attitude data of the patrol vehicle.

6. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, Based on the attitude data, vehicle motion correction is performed on the feature point set to obtain the positioning information of the patrol vehicle entrance, specifically including: The feature point set is aggregated according to its spatial distribution, and stable feature points that continuously appear in adjacent visual image sequences are extracted to obtain the entry stable feature point set; Based on the attitude data, vehicle motion compensation is performed on the stable feature point set at the entrance to obtain the change in the position of the entrance feature. Based on the change in the location of the entrance feature points, the set of stable entrance feature points is spatially consistent to obtain the entrance positioning feature set. The coordinates of the entrance positioning feature set are corrected using the attitude data, and the positioning is corrected based on the position changes in continuous visual image frames to obtain the positioning information of the patrol vehicle entrance.

7. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, Determining the relative pose of the robot dog as it enters the ramp based on the positioning information specifically includes: Based on the positioning information, establish the patrol vehicle entrance coordinate reference, obtain the current position of the robot dog, and obtain the robot dog's position coordinates; The robot dog's position coordinates are converted to the patrol vehicle's entrance coordinate reference to obtain the robot dog's position deviation relative to the entrance. The robot dog's current direction of travel is determined based on the inertial measurement data, and the directional deviation between the direction of travel and the entrance direction is calculated. The relative pose of the robot dog as it enters the ramp is determined by the positional deviation and the directional deviation.

8. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, The acquisition of ramp contact pressure and robot dog foot contact status specifically includes: The ramp pressure data during the robot dog's entry into the ramp is collected by a pressure sensor, and the ramp pressure data is continuously changed and extracted to obtain the ramp contact pressure. The pressure change data of the robot dog's feet is collected by a foot pressure detection device, and the support state is identified by the pressure change data to obtain the contact state of the robot dog's feet.

9. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, The joint variation feature extraction of the ramp contact pressure and the foot contact state yields the following bearing state parameters: The pressure change is extracted from the ramp contact pressure to obtain ramp pressure time series data; The foot support change is extracted from the foot contact state to obtain foot support time series data; The slope pressure time series data and the foot support time series data are synchronously correlated according to the acquisition time, and the time series correspondence between pressure change and support change is extracted to obtain pressure-support correlation features. Based on the pressure support correlation characteristics, the load-bearing state range of the robot dog on the ramp is determined, and the load-bearing state parameters are obtained.

10. The automatic loading and unloading control method for a robot dog as described in claim 1, characterized in that, Controlling the robot dog to get on and off the vehicle based on the relative pose and the carrying state parameters specifically includes: Based on the position and orientation deviations in the relative pose, determine whether the robot dog meets the pose conditions for entering the ramp, and obtain the pose determination result. When the pose determination result is that the pose condition is met, the support stability of the robot dog on the slope is determined according to the load state interval in the load state parameter, and the support determination result is obtained. When the support determination result is that the support is stable, speed control commands and steering control commands are generated according to the relative pose to control the robot dog to drive along the ramp and complete the getting on and off the vehicle.

Images