Omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method for rail crossroad

Abstract

The invention relates to an omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method for a track fork, and relates to the field of machine vision, and the method comprises the steps: constructing a prior three-dimensional map containing semantic information through multi-sensor fusion; precise real-time positioning is realized based on feature matching and an ICP algorithm, and a global optimal path considering distance, coverage and energy consumption is planned by using the improved A; an omnidirectional obstacle avoidance strategy is started through obstacle recognition and risk assessment in the advancing process, and the moving mode is dynamically adjusted; and when arriving at a detection point, adaptively adjusting an observation pose, fusing visual data, infrared data and three-dimensional data, and realizing automatic identification and diagnosis of the apparent defect of the turnout by means of a deep learning model. The system and the method have the advantages that full-process autonomous inspection of the track crossroad is realized by means of multi-sensor fusion and an intelligent algorithm, the system and the method have the capabilities of accurate positioning, flexible obstacle avoidance and comprehensive and efficient defect detection, the system and the method are adaptive to complex scenes, and manual dependence is greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, in particular to an omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method for railway turnout. BACKGROUND

[0002] The turnout is a key throat position in the railway and rail transit network, with complex structure and frequent action, prone to failure, directly affecting the train safety and transportation efficiency. At present, the monitoring of turnout mainly relies on: fixed sensor monitoring: such as close inspection device, indicating rod sensor, etc., which can only monitor the final position state of turnout, cannot monitor the dynamic information such as conversion process, component mechanical wear, foreign matter invasion, and has limited coverage. Artificial regular inspection: the inspection personnel need to enter the site during the train operation gap, and check through visual and simple tool. This way has great personal safety risk, low efficiency, is greatly affected by subjective factors, and it is difficult to effectively check the hidden parts such as turnout bottom and slide bed. Existing track robot inspection: some track robots cannot leave the track flexibly, and are prone to collision or deadlock in the complex multi-track area of turnout. Traditional wheeled / track-type ground robots have insufficient mobility when crossing steel rails, roadbeds and other unstructured terrains, and it is difficult to flexibly adjust the observation posture in narrow space.

[0003] The core technical problems faced by the prior art include: mobility and environmental adaptability contradiction: the turnout environment is complex, including steel rails, roadbeds, flat roads and other terrains, requiring the robot to have high obstacle crossing ability and terrain adaptability. Safe obstacle avoidance and precise positioning requirement: the turnout space is narrow, with many obstacles, and is close to the operating line, requiring the robot to have high autonomy and precise obstacle avoidance ability to prevent itself from colliding or invading the limit. Lack of comprehensive monitoring and intelligence: there is a lack of a method that can intelligently monitor the turnout from multiple angles and close distances, and automatically identify state abnormalities. SUMMARY

[0004] The omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method for railway turnout provided by the present application solves the above technical problems.

[0005] The technical solution of the present application to solve the above technical problems is as follows: An omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method for railway turnout, comprising: The environment data is collected by a plurality of sensors mounted on the omnidirectional mobile robot dog, the data is calibrated and registered by a fusion algorithm, a three-dimensional point cloud map of the turnout area is constructed, key elements are labeled by feature extraction and semantic segmentation algorithm, and a priori map is formed. The machine dog collects point cloud data of the surrounding environment in real time when entering the turnout, matches features with key elements in the prior map, optimizes the matching results through the iterative closest point algorithm, and obtains real-time positioning data; Based on the prior map, the key detection points of the turnout are obtained, and an improved A* algorithm is used to obtain a globally optimal inspection path that covers all key detection points with the shortest path length, the highest detection coverage, and the lowest mobile energy consumption as the optimization goal. When the machine dog moves along the globally optimal inspection path, it scans the front and surrounding environment in real time, distinguishes dynamic obstacles from static obstacles through an obstacle recognition algorithm, performs collision risk assessment, and generates a risk level. When a collision risk is detected, an omnidirectional mobile obstacle avoidance strategy is started, which utilizes the omnidirectional movement capability of the machine dog to generate a local obstacle avoidance path through lateral movement, diagonal movement, or in-place turning, and dynamically adjusts the movement parameters through real-time feedback of the self-position and obstacle state. After the machine dog moves to a preset detection point in the global path, it adjusts the pose based on the target component type of the detection point, adjusts the body pose and head orientation, adopts a dedicated observation pose for different target components, and obtains the best observation angle. Based on the best viewing angle, visual images, temperature distribution data, and three-dimensional profile data of the turnout surface are collected, and based on a pre-trained deep learning model, feature fusion and classification recognition are performed to automatically identify the turnout surface defects.

[0006] Preferably, the environment data is collected by a multi-sensor combination mounted on an omnidirectional mobile robot dog, data calibration and registration are performed using a fusion algorithm, a three-dimensional point cloud map of the turnout area is constructed, key elements are labeled through feature extraction and semantic segmentation algorithms, and a prior map is formed, which specifically includes: The multi-sensor combination includes a laser radar, a high-definition camera, and an inertial measurement unit; Based on the raw data collected by the multi-sensor, time stamp matching is used to achieve spatio-temporal synchronization of the data, and a multi-sensor fusion method is used to complete data registration, and a high-precision three-dimensional point cloud map of the turnout area is constructed. In the high-precision three-dimensional point cloud map, key elements are labeled through feature extraction and semantic segmentation algorithms, the key elements include rail edges, turnout points, switch points, and fixed obstacles, and a prior map containing semantic information is formed.

[0007] Preferably, the machine dog collects point cloud data of the surrounding environment in real time when entering the turnout, matches features with key elements in the prior map, optimizes the matching results through the iterative closest point algorithm, and obtains real-time positioning data specifically includes: After the machine dog enters the turnout, the laser SLAM system is started, and the prior map is loaded. The laser radar continuously scans the surrounding environment to obtain real-time three-dimensional point cloud data of the crossing, and the real-time point cloud data collected by the laser radar is compared with key elements in the prior map through similarity matching to obtain the current position of the robot dog. The feature matching result is optimized by an iterative closest point algorithm to reduce the positional deviation of the real-time point cloud and the key elements in the prior map, and the real-time accurate positioning coordinates of the robot dog are obtained.

[0008] Preferably, the key detection points of the crossing based on the prior map are obtained by using an improved A* algorithm, with the shortest path length, the highest detection coverage, and the lowest mobile energy consumption as the optimization target, and the global optimal inspection path covering all key detection points is obtained, which specifically includes: The three-dimensional coordinates of the key detection points of the crossing based on the prior map are extracted to form a detection point list, with 100% detection coverage as the primary target, and the shortest total path length and the lowest mobile energy consumption of the robot dog as auxiliary optimization targets, and the target priority is determined by weight allocation; The prior map is converted into a grid map, the fixed obstacles marked in the prior map are set as impassable areas, and the improved A* algorithm is used to calculate the optimal path between nodes with each key detection point as a necessary node, to generate a global path draft covering all detection points; The path draft is simulated and deduced to check whether there is path overlap, detection point omission or proximity to obstacles, and if the path is found to be unreasonable, the path is re-planned until the global optimal inspection path is obtained.

[0009] Preferably, the robot dog scans the front and surrounding environment in real time when moving along the global optimal inspection path, and distinguishes between dynamic obstacles and static obstacles through an obstacle recognition algorithm to perform collision risk assessment to generate a risk level, which specifically includes: When the robot dog moves along the global optimal inspection path, the detection module composed of the onboard depth camera and the laser radar scans the front and both sides of the environment, captures the position, distance and contour data of the long-distance obstacles, and collects the texture details of the close-range environment; The data of the two types of sensors are calibrated in space and time to eliminate data redundancy, and the environmental texture and position change characteristics are combined, if the target position does not change in the last 3 frames of data, it is determined as static, if there is displacement, it is marked as dynamic, and the moving direction and speed of the dynamic obstacle are recorded; Based on the relative distance between the obstacle and the robot dog, the size of the obstacle and the motion trajectory of the dynamic target, the collision time and probability are calculated, and the risk level is divided into low, medium and high levels according to the risk degree, and the risk level signal is generated.

[0010] Preferably, when the risk of collision is detected, the omnidirectional mobile obstacle avoidance strategy is started, which utilizes the omnidirectional movement ability of the robot dog to generate a local obstacle avoidance path through lateral movement, diagonal movement or in-place turning, and dynamically adjusts the movement parameters through real-time feedback of the position of the robot dog and the state of the obstacle. After receiving the risk level signal, the response priority is adjusted according to the risk level, and when the risk is high, the robot dog is temporarily braked, and when the risk is low, the robot dog directly enters the obstacle avoidance preparation; Based on the position and size of the obstacle, lateral movement is used when there is an obstacle on the side, diagonal movement is used when there is an obstacle in front, and in-place turning is used when the space is narrow; During obstacle avoidance, the laser radar and depth camera continuously monitor the relative position of the robot dog and the obstacle, and if the position of the obstacle changes, the movement parameters are adjusted in real time to avoid secondary collision, and the coordinate data of the obstacle avoidance path is recorded; After the robot dog completely avoids the obstacle, the current coordinates are confirmed, and the nearest node of the global path is matched, and a connection path from the current position to the node of the global path is generated using a path smoothing transition algorithm.

[0011] Preferably, after the robot dog moves to the preset detection point in the global path, the pose is adjusted based on the target component type of the detection point, and the best observation angle is obtained by adjusting the body pose and head orientation for different target components, specifically including: After the robot dog moves to the preset detection point in the global path, the pose is changed according to the target component type, including when detecting the bottom components such as the sliding bed plate, the front limb joints are controlled to bend and the torso is controlled to sink, and the sensor is stretched into the gap between the turnout bottom with a crawling pose, and when detecting the overall shape of the turnout, the rear leg joints are driven to stretch and the front limbs are supported to lift the body to a preset height to achieve a bird's-eye view; The sensor collects target component images in real time, and based on the image clarity and target component picture proportion, the perpendicularity of the sensor axis and the detection surface is judged, and if there is a deviation, the head joint angle or body pose is adjusted until the target component outline is complete and the details are clear, and after the view angle is calibrated, the current pose is locked.

[0012] Preferably, based on the best view angle, the visual images, temperature distribution data and three-dimensional contour data of the turnout surface are collected, and based on the pre-trained deep learning model, the apparent defects of the turnout are automatically identified through feature fusion and classification, specifically including: After the robot dog locks the best observation pose, the high-definition camera obtains the apparent features of the rail crack, loose or missing bolts and surface rust of the turnout surface, the infrared thermal imager collects the temperature distribution image, and the 3D structured light sensor obtains the three-dimensional contour data of the joint area between the frog and the basic rail; The preprocessed data is input into a pre-trained deep learning model, three types of data are integrated through a feature fusion technology, and defect type, position, size and temperature anomaly data are output.

[0013] The beneficial effects of the present application are: a priori map containing semantic information is constructed through multi-sensor fusion, real-time accurate positioning of the robot dog is ensured by combining SLAM and iterative closest point algorithm, A* algorithm is improved to take coverage, path length and energy consumption as optimization targets, and a globally optimal inspection path is generated, giving consideration to comprehensiveness and economy. Relying on the omnidirectional movement ability, the traffic safety in complex environments is greatly improved through flexible obstacle avoidance modes such as horizontal movement and diagonal movement, combined with dynamic and static obstacle differentiation and risk grading mechanism. For the exclusive observation posture design of different turnout components, multi-dimensional data acquisition such as vision, temperature and three-dimensional profile is carried out, and then feature fusion is carried out through a deep learning model to realize accurate identification and comprehensive monitoring of defects. The whole process runs autonomously from positioning, path planning, obstacle avoidance to detection, which is suitable for complex scenes at turnout crossings, effectively reduces the dependence on manual work, improves the inspection efficiency and defect detection rate, and ensures the safe operation of the turnout. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A schematic diagram of the method of the present application is shown in the figure. Figure 2 A schematic diagram of the formation of the prior map of the present application is shown in the figure. Figure 3 A schematic diagram of the acquisition of real-time positioning data of the present application is shown in the figure. Figure 4 A schematic diagram of the acquisition of the globally optimal inspection path of the present application is shown in the figure. Figure 5 A schematic diagram of the generation of the risk level of the present application is shown in the figure. Figure 6 A schematic diagram of the omnidirectional movement obstacle avoidance of the present application is shown in the figure. Figure 7 A schematic diagram of the acquisition of the best observation angle of the present application is shown in the figure. Figure 8 A schematic diagram of the defect identification of the present application is shown in the figure. Figure 9 The architecture of the electronic device in the present scheme is shown in the figure. Figure 10 The structure of the computer readable storage medium in the present scheme is shown in the figure. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0016] In the description of the present application, the terms "first", "second" are only for descriptive purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0017] In the description of the present application, the term "for example" is used to indicate "as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily interpreted as more preferred or more advantageous than other embodiments. The following description is given in order to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present application can be implemented without using these specific details. In other examples, well-known structures and processes will not be described in detail in order to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope in accordance with the principles and characteristics disclosed in the present application.

[0018] Referring to Figure 1 As shown, the omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method facing the track crossing, comprising: Step one: through the combination of multiple sensors carried on the omnidirectional mobile robot dog, collect environmental data, use fusion algorithm for data calibration and registration, construct three-dimensional point cloud map of the crossing area, label key elements through feature extraction and semantic segmentation algorithm, form prior map; Step two: when the robot dog enters the crossing, real-time acquisition of point cloud data of the surrounding environment, feature matching with key elements in the prior map, optimization of matching results through iterative closest point algorithm, acquisition of real-time positioning data; Step three: based on the prior map, obtain the key detection points of the crossing, use the improved A* algorithm, take the shortest path length, the highest detection coverage and the lowest mobile energy consumption as the optimization target, obtain the global optimal inspection path covering all key detection points; Step four: when the robot dog moves along the global optimal inspection path, real-time scanning of the front and surrounding environment, distinguishing dynamic obstacles and static obstacles through obstacle recognition algorithm, collision risk assessment, and risk level generation; Step five: when the collision risk is detected, start the omnidirectional mobile obstacle avoidance strategy, use the omnidirectional movement ability of the robot dog, generate local obstacle avoidance path through lateral movement, diagonal movement or in-place turning, dynamically adjust the movement parameters through real-time feedback of the position and obstacle state of the robot dog; Step 6: After the robot dog moves to the preset detection point in the global path, it adjusts its pose based on the target component type at the detection point. By adjusting the body posture and head orientation, it adopts a special observation posture for different target components to obtain the best observation angle. Step 7: Collect visual images, temperature distribution data, and 3D contour data of the turnout surface based on the best viewing angle. Based on a pre-trained deep learning model, automatically identify the turnout surface defects through feature fusion and classification.

[0019] See Figure 2 As shown, environmental data is collected through a combination of multiple sensors mounted on an omnidirectional mobile robot dog. A fusion algorithm is used for data calibration and registration to construct a 3D point cloud map of the intersection area. Key elements are labeled using feature extraction and semantic segmentation algorithms to form a priori map, specifically including: The multi-sensor combination includes a lidar, a high-definition camera, and an inertial measurement unit; Based on the raw data collected by the multiple sensors, data spatiotemporal synchronization is achieved through timestamp matching, and data registration is completed using a multi-sensor fusion method to construct a high-precision three-dimensional point cloud map of the junction area. In the high-precision 3D point cloud map, key elements are labeled using feature extraction and semantic segmentation algorithms. These key elements include rail edges, turnout switch rails, frog points, and fixed obstacles, forming a priori map containing semantic information.

[0020] Specifically, based on the attitude data output by the IMU, the point cloud is stitched and optimized every 100 frames, and the iterative nearest point algorithm is used to correct the stitching deviation of adjacent point clouds. Then, the discrete point cloud is transformed into a continuous three-dimensional model through the Poisson surface reconstruction algorithm, with a reconstruction resolution of 0.01 meters, to restore the fine structure of the turnout components, such as the wedge tip of the switch rail, the toe and heel of the frog. The images captured by the high-definition camera are fused with the point cloud model, and the point cloud is given realistic colors through texture mapping technology, so that the map has both three-dimensional structure and visual details. The key element automatic annotation and verification starts the semantic segmentation model. The model is based on the pre-trained U-Net architecture. The training dataset contains 500 sets of turnout scene data. It performs element recognition on the textured 3D point cloud map: the continuous straight line features of the rail edge are extracted by the edge detection algorithm and labeled as rail edge; the turnout switch rail and turnout center are identified and labeled according to the wedge structure of the switch rail and the intersection shape of the frog center; fixed obstacles are labeled by filtering by area and position; after the annotation is completed, the robot dog returns to the initial position, takes a global image by high-definition camera, and the annotation results are manually verified to form the final prior map.

[0021] See Figure 3As shown, when the robot dog enters a fork in the road, it collects point cloud data of the surrounding environment in real time, performs feature matching with key elements in the prior map, and optimizes the matching results through an iterative nearest-point algorithm to obtain real-time positioning data. Specifically, this includes: After the robot dog enters the intersection, it starts the laser SLAM system and loads the prior map at the same time. The LiDAR continuously scans the surrounding environment to obtain real-time 3D point cloud data of the intersection. The real-time point cloud data collected by the LiDAR is compared with the key elements in the prior map through similarity matching to obtain the current position of the robot dog. The feature matching results are optimized by using an iterative nearest-point algorithm to reduce the positional deviation between the real-time point cloud and the key elements of the prior map, thereby obtaining the robot dog's real-time accurate positioning coordinates.

[0022] Specifically, a feature extraction algorithm is used to extract key feature elements from the filtered point cloud. The rail edge is fitted as a continuous straight line feature, the switch point is identified as a cross angle feature, and the bolt group is labeled as a dense point cluster feature. The extracted real-time features are matched with the pre-labeled feature elements with the same name in the prior map, and feature pairs with a matching degree of ≥90% are retained. When the lidar fails to match key elements due to temporary obstruction, the body turning angle is calculated by using the angular velocity data output in real time by the IMU, and the displacement in a short period of time is calculated by combining the acceleration data to maintain the continuity of positioning. The matching real-time point cloud and the prior map key element point cloud are optimized using an iterative nearest point algorithm: the prior map element point cloud is used as the target point set and the real-time point cloud is used as the source point set. The distance error between the two point sets is minimized through iterative calculation, and the matching error is controlled to ≤2 mm. The optimized positioning result is output to the robot dog control system in real time to update the robot's coordinates.

[0023] See Figure 4 As shown, based on the prior map, key detection points at intersections are obtained. An improved A* algorithm is used, with the optimization objectives of shortest path length, highest detection coverage, and lowest movement energy consumption, to obtain the globally optimal inspection path covering all key detection points. Specifically, this includes: Based on the prior map, the three-dimensional coordinates of key detection points at the intersection are extracted to form a list of detection points. The primary goal is to achieve 100% detection coverage, while the shortest total path length and the lowest energy consumption of the robot dog are used as auxiliary optimization goals. The priority of the goals is clarified through weight allocation. The prior map is converted into a grid map, and the fixed obstacles marked in the prior map are set as impassable areas. By improving the A* algorithm, the optimal path between nodes is calculated with each key detection point as a necessary node, and a global path draft covering all detection points is generated. The initial path is simulated and analyzed to check for issues such as path overlap, missing detection points, or proximity to obstacles. If an unreasonable path is found, the path is replanned until the globally optimal inspection path is obtained.

[0024] Specifically, an improved A* algorithm is used for path planning, and multi-objective optimization weights are set for the inspection requirements of intersections: with 100% detection coverage as the basic constraint, the shortest path length and the lowest movement energy consumption are set as the core optimization objectives, with weights of 0.3 and 0.3 respectively; an additional weight is added for prioritizing high-priority detection points, which is achieved by adjusting the grid cost. The grid cost within a 3-meter radius of a high-priority detection point is reduced by 30%, guiding the algorithm to prioritize generating paths through this area; at the same time, the heuristic function is optimized, and combined with the omnidirectional movement characteristics of the robot dog, the cost of diagonal movement is set to 1.4 times that of straight-line movement, which is more in line with the actual movement energy consumption. Starting from the robot dog's entrance, the path passes through all high, medium, and low priority detection points in sequence and finally returns to the entrance. When verifying the feasibility of the path, three key indicators are checked: first, whether all detection points are covered; second, whether the path completely avoids the obstacle-buffer area; and third, whether the turning angle of the path matches the robot dog's omnidirectional movement capability. If a detection point is not covered or is too close to an obstacle, the algorithm weights are readjusted and the path is regenerated.

[0025] See Figure 5 As shown, the robot dog scans the environment in front and around it in real time while moving along the globally optimal inspection path. It distinguishes between dynamic and static obstacles using an obstacle recognition algorithm, performs collision risk assessment, and generates a risk level, specifically including: When the robot dog moves along the globally optimal inspection path, it scans the environment in front and on both sides through a detection module composed of an airborne depth camera and a lidar, capturing the position, distance and contour data of distant obstacles and collecting the texture details of the nearby environment. Data redundancy is eliminated by spatiotemporal calibration of the two types of sensor data. Combined with environmental texture and position change features, if the target position does not change in three consecutive frames of data, it is determined to be static. If there is displacement, it is marked as dynamic. The moving direction and speed of dynamic obstacles are also recorded. Based on the relative distance between the obstacle and the robot dog, the size of the obstacle, and the dynamic target's trajectory, the collision time and probability are calculated, and the risk is classified into three levels: low, medium, and high, and a risk level signal is generated.

[0026] Specifically, during the robot dog's movement, the depth camera and LiDAR collect data in different areas according to the principle of coordinated operation: the LiDAR is responsible for the 5-10 meter long-distance area, focusing on scanning the 1.5-meter range on both sides of the rails and the space above the turnout, recording the three-dimensional coordinates and contours of obstacles; the depth camera focuses on the 0.5-5 meter short-distance area, collecting the surface texture details of obstacles, supplementing the LiDAR's blind spots in recognizing small obstacles at close range; during the data collection process, if the robot dog passes through the turnout area, the LiDAR scanning frequency is automatically increased; For each detected obstacle, its center coordinates are marked in the first, second, and third frames. If the coordinate deviation of the three frames is ≤2 cm, it is determined to be a static obstacle, and its size and fixed position are recorded. If the coordinate deviation of the three frames is >2 cm, it is determined to be a dynamic obstacle, and its moving speed and trajectory are calculated. For obstacles that are difficult to distinguish, the prior map is used. If the obstacle is not marked on the map and its position changes slightly for five consecutive frames, it is determined to be a low-risk dynamic obstacle to avoid misjudgment.

[0027] See Figure 6 As shown, when a collision risk is detected, an omnidirectional obstacle avoidance strategy is activated. Utilizing the robot dog's omnidirectional movement capabilities, it generates a local obstacle avoidance path by moving laterally, diagonally, or turning in place. By providing real-time feedback on its own position and the status of obstacles, it dynamically adjusts movement parameters, specifically including: Upon receiving a risk level signal, the response priority is adjusted according to the risk level. In case of high risk, the vehicle will brake briefly first, and in case of medium or low risk, it will directly enter obstacle avoidance preparation. Based on the location and size of the obstacle, lateral movement is used when there is an obstacle on the side, diagonal movement is used when there is an obstacle in front, and turning on the spot is used when the space is narrow. During obstacle avoidance, the lidar and depth camera continuously monitor the relative position of the robot dog and the obstacle. If the position of the obstacle changes, the movement parameters are adjusted in real time to avoid secondary collisions, and the coordinate data of the obstacle avoidance path is recorded at the same time. After the robot dog has completely bypassed the obstacles, it confirms its current coordinates and matches them with the nearest node in the global path. Then, it uses a path smoothing transition algorithm to generate a connecting path from the current position to the global path node.

[0028] Specifically, based on obstacle information and current position, the control module automatically selects the appropriate omnidirectional movement mode: if the obstacle is directly in front and there is sufficient lateral space, the lateral movement mode is used; if the obstacle occupies lateral space but there is a diagonal passage in front, the diagonal movement mode is used; if the obstacle surrounds the robot dog, the stationary turning mode is initiated; at the same time, a local obstacle avoidance trajectory is generated: based on the principles of no collision, shortest path, and smooth movement, combined with the robot dog's wheel system motion parameters, a continuous trajectory point is planned, with a trajectory curvature radius of not less than 0.3 meters to prevent wheel system jamming; When the lidar detects that the distance between the robot body and a static obstacle exceeds 2 meters or the movement trajectory of a dynamic obstacle is far away from the robot dog, the control module initiates the departure determination program; the positioning system obtains the robot dog's current precise coordinates and compares them with the coordinate range of the obstacle to confirm that the robot body has completely left the obstacle's influence area; if the departure conditions are met for 3 consecutive detections, the obstacle avoidance action is determined to be completed, omnidirectional movement stops, and the path return preparation stage begins.

[0029] See Figure 7 As shown, after the robot dog moves to a preset detection point in the global path, it adjusts its pose based on the target component type at the detection point. By adjusting its body posture and head orientation, it adopts a dedicated observation posture for different target components to obtain the best observation angle. Specifically, this includes: After the robot dog moves to the preset detection point of the global path, it changes to the corresponding posture according to the type of target component. When detecting bottom components such as the slide board, it controls the forelimb joints to bend and the torso to sink, and adopts a crawling posture to insert the sensor into the gap at the bottom of the turnout. When detecting the overall shape of the turnout, it drives the hind leg joints to extend and the forelimbs to support, raising the body to a preset height to achieve a bird's-eye view. The sensor acquires images of the target component in real time. Based on the image clarity and the proportion of the target component in the image, the perpendicularity of the sensor axis to the detection surface is determined. If there is a deviation, the head joint angle or body posture is finely adjusted until the outline of the target component is complete and the details are clear. After the viewing angle is calibrated, the current posture is locked.

[0030] See Figure 8 As shown, visual images, temperature distribution data, and 3D contour data of the turnout surface are collected based on the optimal viewing angle. Based on a pre-trained deep learning model, feature fusion and classification are used to automatically identify surface defects of the turnout. Specifically, this includes: After locking onto the optimal observation posture, the robot dog uses a high-definition camera to acquire the surface features of rail cracks, loose or missing bolts, and surface rust on the turnout surface, uses an infrared thermal imager to collect temperature distribution images, and uses a 3D structured light sensor to acquire three-dimensional contour data of the contact area between the switch rail and the main rail. The preprocessed data is input into a pre-trained deep learning model, and the three types of data are integrated through feature fusion technology to output data on defect type, location, size, and temperature anomalies.

[0031] Specifically, the preprocessed multimodal data is input into a pre-trained deep learning model. The model employs a multi-branch fusion architecture: the image branch uses a CNN to extract crack edges, rust textures, and bolt head morphological features from high-resolution images, outputting the probability of apparent defects; the temperature branch uses an MLP to analyze the temperature distribution of infrared thermograms, marking local temperatures exceeding the normal threshold for similar components as thermal defects; the point cloud branch uses a PointNet network to process 3D point clouds, extracting the three-dimensional contour features of the contact surface between the switch rail and the base rail, and identifying point cloud voids at the gaps. Finally, the model fuses the results from the three branches to determine the defect type, precise location, defect size, and risk level.

[0032] Please see Figure 9 , Figure 9 A schematic diagram of an embodiment of the electronic device provided in this invention, as shown below. Figure 9 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520.

[0033] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided in this invention, as shown below. Figure 10 As shown, this embodiment provides a computer-readable storage medium 600 on which a computer program 611 is stored.

[0034] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0035] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0036] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0037] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0038] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0039] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0040] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for autonomous obstacle avoidance and state monitoring of an omnidirectional mobile robot dog facing a track turnout, characterized in that, The application relates to a method for inspecting a turnout area by using a mobile robot dog. The application comprises the following steps: The robot dog enters the turnout area and collects point cloud data of the surrounding environment in real time, and the key elements in the prior map are matched with the features, the matching results are optimized through the iterative closest point algorithm, and real-time positioning data are obtained. Based on the priori map, the key detection points of the turnout are obtained, and the improved A algorithm is used to obtain the global optimal inspection path covering all key detection points with the shortest path length, the highest detection coverage and the lowest mobile energy consumption. When the robot dog moves along the global optimal inspection path, the front and surrounding environment is scanned in real time, dynamic obstacles and static obstacles are distinguished through an obstacle recognition algorithm, collision risk assessment is carried out, and a risk level is generated. When the collision risk is detected, the omnidirectional mobile obstacle avoidance strategy is started, the omnidirectional movement ability of the robot dog is utilized, a local obstacle avoidance path is generated in the form of lateral movement, diagonal movement or in-place turning, the movement parameters are dynamically adjusted through real-time feedback of the position of the robot dog and the state of the obstacle. After the robot dog moves to the preset detection point in the global path, the target component type based on the detection point is used for pose adjustment, the body posture and the head direction are adjusted, the exclusive observation posture is adopted for different target components, and the best observation angle is obtained. Based on the best observation angle, visual images, temperature distribution data and three-dimensional contour data of the turnout surface are collected, a pre-trained deep learning model is used for feature fusion and classification recognition, and the turnout surface defects are automatically identified.

2. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, The multi-sensor combination comprises a laser radar, a high-definition camera and an inertial measurement unit. Based on the original data collected by the multi-sensor, time stamp matching is adopted to realize the space-time synchronization of the data, a multi-sensor fusion method is adopted to complete data registration, and a high-precision three-dimensional point cloud map of the turnout area is constructed. In the high-precision three-dimensional point cloud map, the key elements are marked through the feature extraction and semantic segmentation algorithm, the key elements comprise a rail edge, a turnout point rail, a switch heart and a fixed obstacle, and a prior map containing semantic information is formed. When the robot dog enters the turnout area, the point cloud data of the surrounding environment is collected in real time, the key elements in the prior map are matched with the features, the matching results are optimized through the iterative closest point algorithm, and real-time positioning data are obtained.

3. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, The robot dog enters the turnout area and starts the laser SLAM system, and the prior map is loaded at the same time. The laser radar continuously scans the surrounding environment to obtain real-time three-dimensional point cloud data of the turnout area, the real-time point cloud data collected by the laser radar is matched with the key elements in the prior map through similarity matching, and the current position of the robot dog is obtained. The feature matching results are optimized through the iterative closest point algorithm, the position deviation of the real-time point cloud and the key elements in the prior map is reduced, and the real-time accurate positioning coordinates of the robot dog are obtained. ​ 4. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, The key detection point of the turnout is obtained based on a priori map, and an improved A Algorithm, with the shortest path length, the highest detection coverage and the lowest mobile energy consumption as the optimization goal, to obtain a global optimal inspection path covering all key detection points, specifically including: The three-dimensional coordinates of key detection points of the turnout are extracted based on the prior map to form a detection point list, with 100% coverage as the primary goal, and the shortest path length and the lowest energy consumption of the robot dog as auxiliary optimization goals, and the priority of the goals is determined by weight allocation; The prior map is converted into a grid map, fixed obstacles marked in the prior map are set as impassable areas, and an improved A algorithm is used to calculate the optimal path between nodes with each key detection point as a necessary node, and a global path draft covering all detection points is generated. The initial path is simulated to check whether there are path overlaps, missing detection points or close-to-obstacle problems, and if the path is found to be unreasonable, the path is re-planned until the globally optimal inspection path is obtained.

5. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, When the robot dog moves along the globally optimal inspection path, it scans the front and surrounding environment in real time, distinguishes between dynamic and static obstacles through an obstacle recognition algorithm, evaluates the collision risk, and generates a risk level signal including: When the robot dog moves along the globally optimal inspection path, it scans the front and surrounding environment through a detection module composed of an onboard depth camera and a laser radar, captures the position, distance and contour data of long-distance obstacles, and collects the texture details of the near-distance environment; The data redundancy of the two types of sensors is eliminated through space-time calibration, and the environmental texture and position change characteristics are combined, if the target position does not change in the last 3 frames of data, it is determined to be static, if there is displacement, it is marked as dynamic, and the moving direction and speed of the dynamic obstacle are recorded; Based on the relative distance between the obstacle and the robot dog, the size of the obstacle and the dynamic target motion trajectory, the collision time and probability are calculated, and the risk level is divided into low, medium and high levels according to the risk degree, and the risk level signal is generated.

6. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, When the collision risk is detected, the omnidirectional mobile obstacle avoidance strategy is started, the omnidirectional movement ability of the robot dog is used, and the local obstacle avoidance path is generated through lateral movement, diagonal movement or in-place turning, and the movement parameters are dynamically adjusted through real-time feedback of the position and obstacle state of the robot dog, including: After receiving the risk level signal, the response priority is adjusted according to the risk level, and the robot dog is temporarily braked for a short time when the risk is high, and directly enters the obstacle avoidance preparation when the risk is low; Based on the position and size of the obstacle, lateral obstacles are moved laterally, front obstacles are moved diagonally, and narrow spaces are turned in place; During obstacle avoidance execution, the laser radar and depth camera continuously monitor the relative position of the robot dog and the obstacle, and if the obstacle position changes, the movement parameters are adjusted in real time to avoid secondary collision, and the coordinate data of the obstacle avoidance path is recorded; After the robot dog completely avoids the obstacle, the current coordinates are confirmed, matched with the nearest node of the global path, and the path smoothing transition algorithm is used to generate a connection path from the current position to the global path node.

7. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, When the robot dog moves to the preset detection point in the global path, the pose is adjusted based on the target component type, the body pose and head orientation are adjusted, and the exclusive observation pose is used for different target components to obtain the best observation angle, including: After the robot dog moves to the preset detection point of the global path, the pose is changed according to the target component type, including detecting the bottom components such as the sliding bed plate, bending the front limb joints, sinking the torso, and stretching the sensor into the gap between the turnout bottom, detecting the overall shape of the turnout, driving the rear leg joints to stretch, supporting the front limbs, and lifting the body to a preset height to achieve a bird's-eye view. The sensor collects the image of the target component in real time, judges the perpendicularity of the sensor axis and the detection surface based on the image definition and the target component picture proportion, adjusts the head joint angle or the body posture if there is deviation until the target component contour is complete and the details are clear, locks the current posture after the view angle calibration is completed.

8. The rail turnout facing omnidirectional mobile robot dog autonomous obstacle avoidance and state monitoring method according to claim 1, characterized in that, The visual image, temperature distribution data and three-dimensional contour data of the turnout surface are collected based on the optimal view angle, the pre-trained deep learning model is used for feature fusion and classification recognition, and the turnout apparent defects are automatically identified, specifically including: After the optimal observation posture is locked, the machine dog obtains the rail crack, bolt loosening or missing and surface rust apparent features of the turnout surface through the high-definition camera, collects the temperature distribution image through the infrared thermal imager, and obtains the three-dimensional contour data of the frog and the basic rail fitting area through the 3D structured light sensor; The pre-processed data is input into the pre-trained deep learning model, three types of data are integrated through feature fusion technology, and the defect type, position, size and temperature abnormal data are output.

9. An electronic device, comprising: It comprises: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby realizing the omnidirectional mobile machine dog autonomous obstacle avoidance and state monitoring method for the rail turnout.

10. A non-transitory computer-readable storage medium, comprising: The storage medium stores a computer software program, and the computer software program is executed by the processor to realize the omnidirectional mobile machine dog autonomous obstacle avoidance and state monitoring method for the rail turnout.

Images