Obstacle identification method, device and storage medium of low-speed patrol robot

Abstract

The application discloses a low-speed patrol robot obstacle identification method and device and a storage medium, relates to the technical field of robot environment perception, and relies on the exclusive working condition advantages of a low-speed patrol robot, that is, stable driving, weak attitude drift, fixed forward perception field of view, and low relative motion rate of scene targets, to construct a complete technical link of visual and radar multi-source feature collection, space-time synchronous alignment, cross-modal neural network deep fusion, and near-field low obstacle identification output. The application breaks through the limitations of traditional decision-level shallow fusion, realizes the redundant complement and bidirectional guidance of visual semantic features and radar geometric features at the feature level, and specifically solves the core technical pain points of near-field low obstacles in a low-speed scene, mispositioning of slow-moving small target features, semantic loss, missed detection, and high false detection.

Description

Technical Field

[0001] This invention relates to the field of robot environmental perception technology, and in particular to an obstacle recognition method, device and storage medium for a low-speed patrol robot. Background Technology

[0002] The core prerequisite for autonomous navigation and safe obstacle avoidance of low-speed patrol robots is accurate obstacle perception and identification. In existing engineering applications, obstacle detection is generally accomplished using a single visual sensor, a single LiDAR sensor, a single millimeter-wave radar sensor, or a traditional decision-level multi-sensor fusion solution. Single sensors have inherent perception limitations: visual sensors are highly sensitive to changes in lighting, backlighting, shadows, and low-light environments at night, and have blind spots for transparent low-lying obstacles such as glass and mirrors. They are prone to misjudgment in areas lacking texture, such as white walls and open ground, and have large errors in near-range depth estimation, making it impossible to accurately determine the distance and height of low-lying obstacles. LiDAR has high geometric ranging accuracy and is unaffected by lighting and texture interference, but lacks semantic classification and texture recognition capabilities. It cannot distinguish between low grass and solid walls, or between ground protrusions and real obstacles. For small low-lying obstacles such as cables and thin baffles, it suffers from sparse point cloud sampling, which can easily be filtered out as ground noise. Millimeter-wave radar has strong environmental anti-interference capabilities, but has low spatial resolution, poor contour restoration of small obstacles and slow-moving pedestrians, lacks accurate height dimension information, and has weak reflected signals from non-metallic low-lying obstacles, making it difficult to meet the near-field low-lying obstacle recognition needs of low-speed robots.

[0003] Existing traditional multi-sensor fusion methods mostly employ shallow fusion modes such as simple overlay of raw data, IOU matching of detected target bounding boxes, and fixed-weight decision-making voting fusion. These general fusion solutions are mostly designed for high-speed mobile vehicles and do not take into account the unique advantages of low-speed patrol robots, such as stable motion, small attitude drift, fixed forward field of view, and low relative target motion rate. They do not achieve deep coupling and spatiotemporal correlation matching of heterogeneous features of vision and radar at the neural network feature level, but only stop at the result level of simple stitching, and cannot perform feature preservation and complementary enhancement for low-speed scenes with low-lying obstacles and slow-moving small targets in the near field.

[0004] In summary, the existing obstacle recognition technology for low-speed patrol robots does not take advantage of the robot's stable operation at low speeds. Vision and radar only achieve shallow fusion at the result level, which leads to problems such as feature space misalignment, semantic information loss, and high false negative and false positive rates for low static obstacles and slow-moving small dynamic obstacles in the forward near-field range of low-speed scenes. This restricts the safety of autonomous navigation and the reliability of obstacle avoidance operation of low-speed patrol robots. Summary of the Invention

[0005] This invention provides an obstacle recognition method, device, and storage medium for low-speed patrol robots, which solves the technical problems of existing obstacle recognition methods for low-speed patrol robots that do not rely on the advantages of the robot's stable low-speed operation. The vision and radar only achieve shallow fusion at the result level, which leads to feature spatial misalignment, semantic information loss, and high false detection rates for low-lying static obstacles and slow-moving small dynamic obstacles in the forward near field of low-speed scenes.

[0006] On one hand, the present invention provides an obstacle recognition method for a low-speed patrol robot, the method comprising the following steps: Image data and radar point cloud data within the forward fixed field of view of the low-speed patrol robot are collected, and based on the working characteristics of the low-speed patrol robot with low speed and small attitude disturbance, the image data and radar point cloud data are synchronized in time and aligned in spatial coordinates. The radar point cloud data is subjected to ground point filtering, low obstacle height screening, and local neighborhood geometric feature calculation to obtain radar geometric features; the image data is subjected to image preprocessing, multi-scale semantic feature extraction, and feature dimension alignment to obtain visual semantic features. The radar geometric features and the visual semantic features are input into a cross-modal coupled neural network. Through modal normalization, feature mapping, bidirectional modal gating, and feature interaction fusion, a fused feature vector is obtained. The fused feature vector is decoded to output obstacle category, relative distance to obstacle, lateral offset, ground clearance and circumscribed contour size, and obstacle 3D information is generated based on the output results.

[0007] In one implementation of the present invention, the low-speed patrol robot is an autonomous mobile device with a cruising linear speed not exceeding a preset low-speed threshold, suitable for structured or semi-enclosed low-speed operation scenarios with a forward fixed perception field of view.

[0008] In one implementation of the present invention, the time synchronization and spatial coordinate alignment include: correcting the sampling time difference between radar data and image data using timestamp interpolation, so that the time error between radar data and image data is controlled within a preset synchronization tolerance range; and transforming the radar coordinate system to the camera coordinate system using a rotation matrix and translation vector obtained through offline calibration.

[0009] In one implementation of the present invention, the radar geometric feature extraction includes: fitting a ground plane model using the RANSAC random sampling consensus algorithm and filtering out ground point clouds; and selecting point clouds whose ground height is within a preset height range as low obstacle-related point clouds.

[0010] In one implementation of the present invention, the radar geometric features include point cloud normal vectors, surface curvature, mean radar reflection intensity, variance of radar reflection intensity, local point cloud density, and height distribution variance, and fixed-dimensional radar features are output through a lightweight point cloud coding network.

[0011] In one implementation of the present invention, the visual semantic feature extraction includes: sequentially performing Gaussian spatial domain denoising, contrast-limited adaptive histogram equalization processing, and gamma grayscale correction on the image, and using the MobileNetV3 network combined with the feature pyramid network to extract multi-scale semantic features.

[0012] In one implementation of the present invention, the cross-modal coupled neural network includes a modality normalization layer, a coupling mapping layer, a bidirectional modality gating unit, a cross-modal feature interaction module, and a residual connection module. It guides visual semantic information through radar geometric information and enhances radar feature expression through visual semantic information.

[0013] In one implementation of the present invention, generating obstacle 3D information includes: using a non-maximum suppression method to remove duplicate detection results, and combining continuous frame mean smoothing to correct distance and height prediction errors, and finally outputting obstacle information including obstacle category, 3D center coordinates, external dimensions, ground clearance, and detection confidence.

[0014] The present invention also provides an obstacle recognition device for a low-speed patrol robot, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the aforementioned obstacle recognition method for a low-speed patrol robot.

[0015] The present invention also provides a non-volatile computer storage medium for obstacle recognition of a low-speed patrol robot, which stores computer-executable instructions that are executed by a processor to implement the aforementioned obstacle recognition method for a low-speed patrol robot.

[0016] The present invention provides an obstacle recognition method, device, and storage medium for a low-speed patrol robot, which has the following beneficial effects: 1. This invention leverages the characteristics of low-speed patrol robots, such as small motion disturbances and stable posture, to perform timestamp interpolation synchronization and offline calibration of extrinsic parameters on radar point clouds and image data. This achieves accurate matching of multi-source sensing information in the spatiotemporal dimension, effectively eliminating inherent positional deviations and temporal misalignments between sensors, and fundamentally improving the feature space misalignment problem during the fusion process.

[0017] 2. This invention constructs a cross-modal coupled neural network, using modal normalization, bidirectional modal gating, and Hadamard product feature interaction mechanisms to deeply fuse radar geometric features and visual semantic features at the feature level. This achieves redundant complementarity and bidirectional guidance between radar spatial positioning capabilities and visual semantic discrimination capabilities, significantly improving the recognition accuracy of low obstacles and targets in texture-deficient areas.

[0018] 3. This invention employs a lightweight point cloud encoding network, a MobileNetV3 backbone network, and a simplified fully connected decoding structure. The overall computational logic is simple and clear, enabling the algorithm model to run efficiently in real time on an embedded hardware platform. This meets the engineering deployment requirements of low latency and low power consumption for obstacle perception during the autonomous navigation of low-speed patrol robots.

[0019] 4. This invention adapts parameters and customizes modules for specific low-speed, slow-moving scenarios. All threshold configurations and fusion logic are designed with the premise of stable low-speed operation, achieving a high degree of fit between the technical solution and the operating characteristics of the low-speed patrol robot. This avoids the complex compensation modules required for high-speed movement scenarios and has clear scenario-specific adaptability and technical targeting.

[0020] 5. This invention resolves the fused feature vector into physical parameters such as obstacle category, 3D coordinates, dimensions, and ground clearance through a fixed decoding network, and combines nonmaximum suppression and inter-frame smoothing post-processing to achieve a stable output from abstract features to quantifiable 3D obstacle bounding boxes, providing a reliable environmental perception foundation for autonomous obstacle avoidance and path planning of low-speed patrol robots. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart of an obstacle recognition method for a low-speed patrol robot provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of an obstacle recognition device for a low-speed patrol robot provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] This invention provides an obstacle recognition method, device, and storage medium for a low-speed patrol robot. The technical solution proposed in this invention will be described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a flowchart illustrating an obstacle recognition method for a low-speed patrol robot, provided as an embodiment of the present invention. Figure 1 As shown, the method mainly includes the following steps: S1: Collect image data and radar point cloud data within the forward fixed field of view of the low-speed patrol robot, and based on the working characteristics of the low-speed patrol robot's low-speed operation and small attitude disturbance, perform time synchronization and spatial coordinate alignment on the image data and radar point cloud data.

[0025] In an optional embodiment of the present invention, the low-speed patrol robot is an autonomous mobile device with a cruising linear speed not exceeding a preset low-speed threshold. It is mainly used in structured and semi-enclosed slow-moving scenarios such as industrial parks, community buildings, substation maintenance, underground garages, and indoor computer rooms. Compared with high-speed moving vehicles, the low-speed patrol robot has inherent advantages such as small motion posture disturbances, smooth acceleration and deceleration, no violent turning bumps, a fixed forward perception field of view, and targets in the scene being in a state of low-speed relative motion. Interference factors such as motion distortion, abrupt attitude changes, and large inter-frame displacements are significantly weakened.

[0026] In an optional embodiment of the present invention, step S1, time synchronization and spatial coordinate alignment, specifically includes: correcting the sampling time difference between radar data and image data using timestamp interpolation, so that the time error between radar data and image data is controlled within a preset synchronization tolerance range, eliminating target position deviation caused by timing misalignment. Regarding spatial alignment, the rotation matrix R and translation vector T are obtained offline using a checkerboard calibration board with hand-eye calibration, and the radar coordinate system is transformed to the camera coordinate system. The transformation formula is as follows:

[0027] In the formula: The coordinates of a point in the radar coordinate system are three-dimensional. The coordinates of the three-dimensional points in the camera coordinate system after transformation; It is a 3×3 rotation matrix; It is a 3×1 translation vector.

[0028] The rotation matrix R and translation vector T are obtained offline using a checkerboard calibration method. They can be fixed and reused for a long time under low-speed conditions without the need for online real-time calibration. However, under high-speed conditions, the chassis vibrates violently and the external parameters are prone to drift, making it impossible to fix and reuse this set of parameters.

[0029] Then, the 3D points in the camera coordinate system are projected to the pixel coordinate system, and the projection formula is as follows:

[0030]

[0031] In the formula: , These are pixel coordinates; , The horizontal and vertical focal lengths of the camera; , The coordinates of the camera's principal point; , , These are the three-dimensional coordinate components of the camera coordinate system. Camera intrinsic parameters are obtained in a single offline calibration, eliminating the need for online correction under low-speed conditions.

[0032] S2: Perform ground point filtering, low obstacle height screening, and local neighborhood geometric feature calculation on the radar point cloud data to obtain radar geometric features; perform image preprocessing, multi-scale semantic feature extraction, and feature dimension alignment on the image data to obtain visual semantic features.

[0033] In an optional embodiment of the present invention, radar point cloud data is collected within the forward fixed-angle field of view of the low-speed patrol robot chassis. The field of view is a fan-shaped detection area deflected to the left and right sides by a preset angle based on the robot's forward central axis. The effective detection range of the radar is limited to the near-field range. Based on the stable chassis posture, small terrain undulations, and no significant drift in calibration parameters of the low-speed robot, the radar geometric feature extraction specifically includes the following sub-steps: Sub-step S21: Fit the ground plane model using the RANSAC random sampling consensus algorithm, set a ground height segmentation threshold, filter out invalid point clouds on the ground plane, and retain the obstacle-related point clouds. The RANSAC random sampling consensus algorithm is used to fit the ground plane model, with a fixed ground height segmentation threshold of 0.02m. This threshold is suitable for low-speed flat-ground structured scenarios, allowing a fluctuation range of 0.015m to 0.025m. Invalid point clouds on the ground plane are filtered out, and obstacle-related point clouds are retained.

[0034] Sub-step S22: Select point clouds with ground clearance within a preset low-height range as low-obstacle associated point clouds, and filter out redundant high-altitude points and stray ground point clouds. This invention defines obstacles with ground clearance between 0.02m and 0.15m as low-obstacles in low-speed scenes, with an allowed height range of 0.02m to 0.18m; select and retain obstacle point clouds within this height range, and filter out redundant high-altitude points and stray ground point clouds.

[0035] Sub-step S23: For each valid radar point, construct a k-nearest neighbor set and calculate multi-dimensional geometric features such as point cloud normal vector, surface curvature, mean radar reflection intensity, variance of radar reflection intensity, local point cloud density, and height distribution variance. Specifically, set the single-point neighborhood parameter k=16, and the value of k can be selected from 12 to 20, with 16 being the preferred value; construct a k-nearest neighbor set for each valid radar point and calculate multi-dimensional geometric features such as point cloud normal vector, surface curvature, mean and variance of radar reflection intensity, local point cloud density, and height distribution variance.

[0036] Valid points are defined as radar points that are within a preset effective distance range from the robot, within a preset low height range, and have at least a preset number of points in their neighborhood. Noise points and outliers are defined as discrete radar points whose neighborhood Euclidean distance is greater than a preset judgment threshold, are isolated with no neighboring related points, and are not within the effective distance and height ranges. Specifically, valid points are defined as radar points that are 0.02m to 2m from the robot, have a height of 0.02m to 0.15m, and have at least 5 points in their neighborhood, with the numerical range including the endpoints. Noise points and outliers are defined as discrete radar points whose neighborhood Euclidean distance is greater than 0.1m, are isolated with no neighboring related points, and are not within the effective distance and height ranges; the 0.1m judgment threshold has an allowed fluctuation range of 0.08m to 0.12m.

[0037] Sub-step S24: Use the PointNet-Lite lightweight point cloud encoding network to encode and reduce the dimensionality of the multi-dimensional geometric features, outputting 512-dimensional radar geometric features. .

[0038] In an optional embodiment of the present invention, visual semantic feature extraction specifically includes the following sub-steps: Sub-step S25: Acquire images from the chassis monocular camera mounted with the radar sharing the forward field of view. Taking advantage of the low-speed robot's lack of high-speed motion blur and fixed perception field of view, the images are sequentially subjected to Gaussian spatial denoising, CLAHE-limited contrast adaptive histogram equalization, and gamma grayscale correction to enhance the expression of the edge contours and texture details of low-lying obstacles.

[0039] Sub-step S26: Using the MobileNetV3-Large neural network as the backbone network, and combining it with the FPN feature pyramid network to extract multi-scale semantic features, taking into account both shallow and low-level edge texture features and deep and high-level semantic category features.

[0040] Sub-step S27: Compress the image semantic features to 512 dimensions, the same as the radar geometric features, using a 1×1 convolution kernel and adaptive global pooling operation, and output the visual semantic features. .

[0041] In an optional embodiment of the present invention, the indicators for evaluating image quality are defined as follows: overexposed pixels are defined as pixels with a value of not less than 250 in an 8-bit grayscale image; the threshold is adjustable in the range of 245 to 250; the image contrast is calculated using the global grayscale range method; and the average gradient magnitude is solved by the mean of the horizontal and vertical gradients of the image.

[0042] S3: Input the radar geometric features and the visual semantic features into a cross-modal coupled neural network, and obtain a fused feature vector through modal normalization, feature mapping, bidirectional modal gating and feature interaction fusion.

[0043] In an optional embodiment of the present invention, a lightweight cross-modal coupled neural network is constructed, specifically designed for the recognition of low-speed near-field low obstacles and slow-moving small targets. It leverages the advantage of continuous, non-instantaneous changes in the features of low-speed targets to achieve bidirectional feature guidance and deep coupling. (Definition symbols) The Hadamard product of a matrix or vector represents the element-wise multiplication of corresponding elements in the same dimension. The network hierarchy is as follows: the ModalNorm layer uses layer normalization; the CoupleLinear layer is a single-layer fully connected linear mapping layer with ReLU activation function; the MLP is a two-layer fully connected network with a preset number of neurons in the intermediate layers, ReLU activation function, and Sigmoid activation function in the output layer.

[0044] Step S3 specifically includes the following sub-steps: Sub-step S31: The network input is 512-dimensional radar features. and 512-dimensional visual features .

[0045] Sub-step S32: Set the ModalNorm modal normalization layer to eliminate feature domain shift caused by slight illumination fluctuations and point cloud sparsity disturbances in low-speed scenes.

[0046] Sub-step S33: Map radar features to geometric guidance features and visual features to semantic enhancement features through the CoupleLinear coupling mapping layer, establishing a fusion logic of radar-dominated geometric localization and vision-dominated semantic classification.

[0047] Sub-step S34: Construct a bidirectional modal gating unit and generate modal adaptive gating weights, as shown in the following formula:

[0048]

[0049] In the formula, For: LiDAR mode gating weights, For: Image modality gating weights, It is a two-layer fully connected network with 128 neurons in the middle layer, ReLU activation function, and Sigmoid activation function in the output layer.

[0050] Sub-step S35: Perform cross-modal feature interaction fusion, using Hadamard product to achieve fine-grained feature coupling, as shown in the following formula:

[0051] For: the coupled feature vector after cross-modal interactive fusion. Sub-step S36: Add a residual connection branch to preserve radar geometric positioning stability and output fused features, as shown in the following formula:

[0052] The final fused feature vector after residual connection and batch normalization is: For example: batch normalization layers, used to accelerate network convergence and suppress feature distribution shift. It consists of a 1×1 convolutional layer, used to perform channel transformation and dimensionality reduction on the fused features.

[0053] S4: Decode the fused feature vector to output obstacle category, relative distance between obstacles, lateral offset, ground clearance and circumscribed contour size, and generate obstacle 3D information based on the output results.

[0054] In an optional embodiment of the present invention, after calculation by the cross-modal coupled neural network, the output fused feature vector only completes the fusion of visual semantics and radar geometric information, without physical information such as spatial location, size, and category. Therefore, a fixed decoding process is required to convert it into three-dimensional real obstacle information. Step S4 specifically includes the following sub-steps: Sub-step S41: Construct a fixed-decoding fully connected network. Set up a three-layer serial fully connected decoding network with fused feature vectors as input. The network parameters are fixed to adapt to low-speed near-field recognition scenarios. The first layer is a fully connected layer with 512-dimensional input and 256-dimensional output, using ReLU as the activation function; the second layer is also a fully connected layer with 256-dimensional input and 128-dimensional output, using ReLU as the activation function; the third layer is a fully connected layer with 128-dimensional input, splitting the output into multiple sets of predicted values.

[0055] Finally, five sets of quantifiable predictable parameters were deciphered from the feature vectors:

[0056] These represent: obstacle category code cls, obstacle distance relative to the robot's positive center d, obstacle horizontal offset xoff, obstacle height above the ground h, and obstacle's overall circumscribed contour dimension wl.

[0057] Sub-step S42: Calculate the spatial position by combining calibration parameters. This relies on the camera's intrinsic parameters, which remain constant under low-speed conditions. The process involves converting pixels to actual physical distances: using the predicted positive distance d and lateral offset xoff, combined with a 120° field-of-view constraint, the 3D coordinates of the obstacle in the camera coordinate system are calculated; then, the extrinsic parameter matrices R and T, obtained from the previous offline calibration, are called to execute the coordinate transformation formula.

[0058] Transform the camera coordinate system coordinates to the robot's body world coordinate system.

[0059] Sub-step S43: Limit the selection of low-altitude obstacles. Strictly follow the preset low-speed low-altitude obstacle judgment rules to select targets with predicted ground height h, retaining only targets with predicted heights in the range of 0.02m to 0.15m, and eliminating high-altitude targets and ground-level clutter targets to accurately locate low-altitude obstacles that are prone to collision in low-speed scenes.

[0060] Sub-step S44: Obstacle 3D bounding box construction. Using the target center point in the robot world coordinate system as a reference, and combining the predicted contour size wl and ground clearance h, a standard 3D cuboid bounding box is generated: the 3D coordinates of the bounding box center are determined. Assign values ​​to the three-dimensional physical dimensions of the bounding box (length, width, and height), and match the category code cls to complete the obstacle type labeling.

[0061] Sub-step S45: Post-processing and filtering of the final output. The Non-Maximum Suppression (NMS) method is used, with an IOU threshold of 0.5 and a confidence threshold of 0.6 to eliminate duplicate detection results for the same obstacle. Considering the continuity of low-speed robot motion, short-time inter-frame mean smoothing is employed to slightly correct distance and height prediction biases. The final standardized output provides complete 3D obstacle information that can be directly used for obstacle avoidance, including: obstacle category + 3D center coordinates in the world coordinate system + 3D external dimensions + ground clearance + detection confidence.

[0062] The above describes an obstacle recognition method for a low-speed patrol robot according to an embodiment of the present invention. Based on the same inventive concept, this invention also provides an obstacle recognition device for a low-speed patrol robot. Figure 2 This is a schematic diagram of an obstacle recognition device for a low-speed patrol robot provided in an embodiment of the present invention, as shown below. Figure 2As shown, the device mainly includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor; wherein the memory 202 stores instructions that can be executed by the at least one processor 201, and the instructions are executed by the at least one processor 201 to enable the at least one processor 201 to complete the aforementioned obstacle recognition method for a low-speed patrol robot.

[0063] In addition, embodiments of the present invention also provide a non-volatile computer storage medium for obstacle recognition of a low-speed patrol robot, which stores computer-executable instructions, which are executed by a processor to implement the aforementioned obstacle recognition method for a low-speed patrol robot.

[0064] This invention leverages the unique advantages of low-speed patrol robots, such as stable movement, minimal attitude drift, fixed forward perception field of view, and low relative motion rate of scene targets. It constructs a complete technical chain encompassing visual and radar multi-source feature acquisition, spatiotemporal synchronization alignment, deep fusion of cross-modal neural networks, and near-field low obstacle recognition output. Breaking through the limitations of traditional decision-level shallow fusion, it achieves redundant complementarity and bidirectional guidance between visual semantic features and radar geometric features at the feature level. This invention specifically addresses the core technical pain points of near-field low obstacles, feature misalignment of slow-moving small targets, semantic loss, and high rates of missed and false detections in low-speed scenarios.

[0065] This invention optimizes and improves upon the practical problems of feature misalignment and recognition deviation in the near-field identification of low-lying obstacles and slow-moving small targets by low-speed patrol robots. Leveraging the inherent advantages of the stable operation and posture of low-speed devices, it possesses the following technical advantages and practical benefits: Firstly, this invention fully utilizes the low-disturbance operation characteristics of low-speed robots to achieve precise spatiotemporal alignment between radar sensing data and visual image data, effectively eliminating spatial positional deviations between multi-source sensing information, and realizing a high degree of spatial uniformity of information collected by different sensors, thereby improving the positional matching disorder problem that occurs during the fusion of multi-source information from the source. Secondly, this invention adopts a feature-level deep fusion method to replace the traditional simple result fusion mode. It can fully integrate the spatial geometric advantages of radar perception and the semantic recognition advantages of visual perception, making up for the perception shortcomings of the two types of single sensors. This effectively improves the device's ability to recognize and adapt to low-lying small obstacles, transparent obstacles, and targets in areas with missing textures, reducing the occurrence of recognition omissions and incorrect judgments. Third, the overall technical architecture of this invention is simplified based on the characteristics of low-speed operation scenarios, eliminating redundant functional modules such as complex motion compensation and real-time external parameter correction required for high-speed motion scenarios. The overall operation logic is simple and clear, and the overall solution is highly lightweight. It can be well adapted to the operating conditions of the embedded hardware platform of the low-speed patrol robot and meet the basic operational needs of real-time perception during the daily autonomous patrol of the equipment. Fourth, the overall design concept, parameter setting rules and information fusion logic of this invention are all designed around the specific working conditions of low-speed and slow-moving scenarios. The overall solution's operating logic is highly compatible with the operating state of low-speed equipment. It cannot be directly adapted to the complex and ever-changing operating conditions of high-speed mobile devices, and has strong scenario-specific adaptability, making the technology application more targeted. Fifth, through a standardized feature decoding and coordinate transformation process, this invention can successfully transform abstract fusion features into real and effective three-dimensional obstacle perception information in the environment. This provides a stable, reliable, and comprehensive environmental perception foundation for the autonomous obstacle avoidance, path planning, and other subsequent autonomous operation functions of low-speed patrol robots. The overall solution is highly practical and can well meet the actual inspection needs of various low-speed structured operation scenarios such as parks, indoor spaces, and underground garages.

[0066] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0067] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0068] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0069] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0070] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0071] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0072] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. An obstacle recognition method for a low-speed patrol robot, characterized in that, The method includes the following steps: Image data and radar point cloud data within the forward fixed field of view of the low-speed patrol robot are collected, and based on the working characteristics of the low-speed patrol robot with low speed and small attitude disturbance, the image data and radar point cloud data are synchronized in time and aligned in spatial coordinates. The radar point cloud data is subjected to ground point filtering, low obstacle height screening, and local neighborhood geometric feature calculation to obtain radar geometric features; the image data is subjected to image preprocessing, multi-scale semantic feature extraction, and feature dimension alignment to obtain visual semantic features. The radar geometric features and the visual semantic features are input into a cross-modal coupled neural network. Through modal normalization, feature mapping, bidirectional modal gating, and feature interaction fusion, a fused feature vector is obtained. The fused feature vector is decoded to output obstacle category, relative distance to obstacle, lateral offset, ground clearance and circumscribed contour size, and obstacle 3D information is generated based on the output results.

2. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The low-speed patrol robot is an autonomous mobile device whose cruising linear speed does not exceed a preset low-speed threshold, and is suitable for structured or semi-enclosed low-speed operation scenarios with a forward fixed perception field of view.

3. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The time synchronization and spatial coordinate alignment include: using timestamp interpolation to correct the sampling time difference between radar data and image data, so that the time error between radar data and image data is controlled within a preset synchronization tolerance range; and using the rotation matrix and translation vector obtained by offline calibration to transform the radar coordinate system to the camera coordinate system.

4. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The radar geometric feature extraction includes: fitting a ground plane model using the RANSAC random sampling consensus algorithm and filtering out ground point clouds; and selecting point clouds with ground heights within a preset height range as low obstacle-related point clouds.

5. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The radar geometric features include point cloud normal vectors, surface curvature, mean radar reflection intensity, variance of radar reflection intensity, local point cloud density, and height distribution variance, and output fixed-dimensional radar features through a lightweight point cloud coding network.

6. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The visual semantic feature extraction includes: sequentially performing Gaussian spatial domain denoising, contrast-limited adaptive histogram equalization, and gamma grayscale correction on the image, and using the MobileNetV3 network combined with the feature pyramid network to extract multi-scale semantic features.

7. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The cross-modal coupling neural network includes a modality normalization layer, a coupling mapping layer, a bidirectional modality gating unit, a cross-modal feature interaction module, and a residual connection module. It guides visual semantic information through radar geometric information and enhances radar feature expression through visual semantic information.

8. The obstacle recognition method for a low-speed patrol robot according to claim 1, characterized in that, The generated obstacle 3D information includes: using non-maximum suppression to remove duplicate detection results, and combining continuous frame mean smoothing to correct distance and height prediction errors, and finally outputting obstacle information including obstacle category, 3D center coordinates, external dimensions, ground clearance, and detection confidence.

9. An obstacle recognition device for a low-speed patrol robot, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform an obstacle recognition method for a low-speed patrol robot as described in any one of claims 1-8.

10. A non-volatile computer storage medium for obstacle recognition in a low-speed patrol robot, storing computer-executable instructions, characterized in that, The computer-executable instructions are executed by the processor to implement an obstacle recognition method for a low-speed patrol robot as described in any one of claims 1-8.

Images