A path planning method and system for guide robots that integrates lidar and camera semantics

Abstract

This invention belongs to the technical field of autonomous mobile robots, and relates to a path planning method and system for a guide robot that integrates lidar and camera semantics. The method includes the following steps: performing visual semantic scanning to identify suspicious areas in the environmental image and generating markers; generating exploration command data packets based on the markers; controlling the lidar to conduct directional exploration of the target space according to the commands, acquiring point cloud data; calling a physical feature analysis program associated with the visual rule type to parse the point cloud and obtain analysis results; generating a physical state report based on the results; performing bidirectional information verification and collaborative decision-making based on the exploration command data packets and the report to form local risk markers; and finally, integrating the markers into the global path planning, dynamically optimizing and outputting the final navigation path. This invention solves the problems of high load and energy consumption caused by global uniform scanning of lidar in existing technologies, and insufficient exploration accuracy in local areas.

Description

Technical Field

[0001] This invention belongs to the technical field of autonomous mobile robots, and relates to a path planning method and system for guide robots that integrates lidar and camera semantics. Background Technology

[0002] In the development of guide robots and related service robots, accurately identifying local risks in the environment and achieving safe and smooth path planning is a significant challenge. These risks typically include temporary liquids on the road surface, foreign objects covering tactile paving, and irregularly shaped objects suspended at low altitudes. The characteristics of these risk areas, such as high reflectivity, subtle height variations, or irregular geometric shapes, present difficulties in identification and differentiation using a single sensor modality. For example, puddles may appear as highly reflective patches, while dry, light-colored paving stones may also exhibit a similar appearance; a cardboard box covering tactile paving may have a similar outline to a hard brick of the same height in a two-dimensional image. This ambiguity introduces uncertainty into the robot's autonomous and safe passage.

[0003] Currently, common solutions mainly fall into two categories: one relies on a single high-precision sensor for depth analysis, such as using a high-resolution 3D LiDAR to perform a uniform and dense scan of the area in front, and detecting potential obstacles through point cloud analysis; the other uses simple parallel processing with multiple sensors, where the vision system and LiDAR work independently, respectively identifying suspicious areas and performing a full-area scan, and then the detection results are superimposed at the application layer. These methods improve environmental perception capabilities to some extent.

[0004] However, the above methods still have limitations in practical applications. Relying on lidar for global uniform high-density scanning leads to a large data processing load and a significant increase in power consumption. Furthermore, due to the even distribution of scanning resources, the detection accuracy for specific small areas may be insufficient to distinguish certain risks. On the other hand, the simple parallel approach of vision and lidar, lacking task-level collaboration and cross-verification between sensors, may lead to resource waste or decision conflicts. For example, suspicious areas marked by the vision system may be difficult to verify effectively because the lidar has not performed targeted analysis, or the specific attributes of certain low-lying coverings may be difficult to confirm due to insufficient scanning density, thus affecting the smoothness and safety of navigation. Summary of the Invention

[0005] In a first aspect, the present invention provides a path planning method for a guide robot that integrates lidar and camera semantics, employing the following technical solution:

[0006] A path planning method for a guide robot that integrates lidar and camera semantics includes the following steps:

[0007] S1. Perform visual semantic scanning to identify suspicious areas in environmental image data and generate corresponding suspicious area markers;

[0008] S2. Based on the suspicious area marker, generate and transmit a detection instruction data packet for triggering collaborative detection, wherein the detection instruction data packet includes at least the visual rule type that triggers the suspicious area marker;

[0009] S3. According to the exploration command data packet, the lidar performs directional exploration of the target's physical space and acquires point cloud data of the target's physical space;

[0010] S4. Call the physical feature analysis program associated with the visual rule type pre-recorded in the exploration instruction data packet to parse the point cloud data and obtain the physical feature analysis results.

[0011] S5. Based on the physical feature analysis results, generate a physical status report, which includes physical status conclusions;

[0012] S6. Based on the exploration command data packet and the physical status report, perform collaborative decision-making to verify two-way information in order to form local risk markers;

[0013] S7. Perform global path planning based on local risk markings and output the final navigation path.

[0014] In a further embodiment of the present invention, step S1 includes the following steps:

[0015] The forward-facing camera of the guide robot is used to collect environmental image data, and a semantic segmentation model is used to perform preliminary analysis of the environmental image data.

[0016] Enable the built-in hybrid risk feature filter, which includes at least one set of preset visual rules for filtering suspicious areas from environmental image data;

[0017] Image regions that meet any visual rule and are filtered by the hybrid risk feature filter are marked as suspicious areas, and suspicious area labels containing the image coordinates of the suspicious area, the visual rule type that triggered the label, and the visual confidence level are generated.

[0018] A further aspect of the present invention involves generating and transmitting a probe command data packet for triggering cooperative probing, comprising the following steps:

[0019] Extract the image coordinate information of suspicious area markers;

[0020] Record the types of visual rules that trigger suspicious area markings and assess the current visual confidence level;

[0021] Image coordinate information, visual rule type, and confidence level are encapsulated into a structured exploration instruction data packet;

[0022] The probe command data packet is transmitted to the collaborative control unit.

[0023] A further aspect of the present invention involves acquiring point cloud data of a target physical space, including the following steps:

[0024] Parse the exploration command data packet and convert the image coordinate information it contains into the target physical space in the lidar coordinate system;

[0025] The scanning mode of the lidar is dynamically adjusted, switching from the conventional environmental scanning mode to the directional exploration mode;

[0026] In directional exploration mode, the laser beam is concentrated and projected onto the target physical space to obtain point cloud data of the target physical space.

[0027] A further aspect of the present invention, obtaining physical feature analysis results, includes the following steps:

[0028] Based on the visual rule type recorded in the exploration command data packet, the corresponding physical feature analysis program is matched and called from the preset analysis program library;

[0029] Input the point cloud data into the invoked physical feature analysis program;

[0030] Run the physical feature analysis program to obtain qualitative or quantitative physical feature analysis results that describe the physical characteristics of the target physical space.

[0031] A further aspect of the present invention generates and feeds back a physical state report, including the following steps:

[0032] Map the results of physical feature analysis to explicit physical state conclusions;

[0033] The confidence level is calculated based on the deviation between the physical feature analysis results and the preset threshold, as well as the point cloud data quality.

[0034] The physical state conclusions and analysis confidence levels are encapsulated to generate a structured physical state report, which is then fed back to the collaborative control unit.

[0035] A further aspect of the present invention generates local risk markers, comprising the following steps:

[0036] Receive probe command data packets and physical status reports;

[0037] Based on the preset decision rule table, the visual rule types in the exploration command data packet and the physical state conclusions in the physical state report are cross-validated.

[0038] Local risk markers are generated based on the cross-validation results. These markers include the spatial location information of the risk area, the risk level, and recommended travel strategies.

[0039] A further aspect of the present invention outputs the final navigation path, including the following steps:

[0040] Invoke the global path planner and overlay local risk markers as a dynamic constraint layer onto the global map;

[0041] Based on the risk level in the local risk markers, adjust the travel cost of the corresponding area in the path planning algorithm.

[0042] In a further embodiment of the present invention, the risk marker includes at least the following fields: a spatial location field for recording the coordinates of the polygon vertices of the risk area in the global map coordinate system; a risk level field for recording the encoded risk severity; and a recommended passage strategy field for recording the specific action instructions associated with the risk level.

[0043] Secondly, this invention provides a path planning system for a guide robot that integrates lidar and camera semantics, employing the following technical solution:

[0044] A path planning system for a guide robot that integrates lidar and camera semantics includes the following modules:

[0045] The suspicious area identification and labeling module is used to perform visual semantic scanning to identify suspicious areas in environmental image data and generate corresponding suspicious area labels;

[0046] The collaborative exploration instruction data packet transmission module generates and transmits exploration instruction data packets for triggering collaborative exploration based on suspicious area markers. The exploration instruction data packets include at least the visual rule type that triggers the suspicious area markers.

[0047] The lidar detection module is used to perform directional detection of the target's physical space according to the detection command data packet, and to acquire point cloud data of the target's physical space;

[0048] The point cloud physical feature parsing module is used to call the physical feature analysis program associated with the visual rule type pre-recorded in the exploration command data packet to parse the point cloud data and obtain the physical feature analysis results.

[0049] The physical status report feedback module generates a physical status report based on the physical feature analysis results, and the physical status report includes physical status conclusions.

[0050] The risk collaborative decision-making module performs collaborative decision-making based on two-way information verification of the exploration command data packet and the physical status report to form local risk markers;

[0051] The navigation path dynamic optimization module performs global path planning based on local risk markers and outputs the final navigation path.

[0052] In summary, the present invention has the following beneficial technical effects:

[0053] 1. A task-driven collaborative mechanism based on exploration command data packets enables on-demand, targeted linkage between the vision system and LiDAR. The exploration command data packets generated by the vision system not only indicate the location of suspicious areas but also specify the triggered visual rule types, allowing the LiDAR to switch to a targeted exploration mode for specific spaces and feature types. This design helps reduce the data processing load and energy consumption generated by continuous global high-density scanning, concentrating high-precision scanning resources on key local areas, thereby improving the quality of feature analysis of critical risk areas while maintaining overall system efficiency.

[0054] 2. By designing a hybrid risk feature filter and its associated physical feature analysis program, a targeted mapping from visual suspicious points to radar-specific analysis was achieved. The pre-set rules within the visual filter correspond to specific risk patterns in guide scenarios. When an investigation command data packet is triggered, the corresponding physical feature analysis program can be invoked based on the rule type, such as determining liquid properties based on echo intensity or analyzing covering features based on geometric shape. This mapping helps improve the targeting and analysis efficiency of point cloud data processing.

[0055] 3. A two-way cross-validation mechanism was established between visual preliminary judgment and lidar physical survey conclusions through collaborative decision-making steps. Based on a preset decision rule table, the exploration command data packet and physical status report are compared. This allows for the determination of a false alarm when visual cues indicate suspicion but lidar feedback shows no abnormalities, or the generation of a corresponding risk marker when both support a risk assessment. This validation mechanism helps improve the overall reliability of risk assessment and reduces unnecessary robot behavior caused by false alarms from a single sensor.

[0056] 4. By integrating real-time generated local risk markers as a dynamic constraint layer into the global path planning process, dynamic responses to temporary and localized risks are achieved. The risk markers include spatial positioning, risk level, and recommended travel strategies. The path planner can adjust the travel costs of corresponding areas accordingly, such as setting extremely high costs for high-risk areas to avoid them, or imposing additional costs for medium-risk areas to suggest deceleration. This approach enables the path planning system to optimize path selection based on real-time environmental information while ensuring navigation safety, thus balancing travel efficiency and stability. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A flowchart illustrating an embodiment of this application is disclosed.

[0059] Figure 2 Structural schematic diagrams of embodiments of this application are disclosed. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The following is in conjunction with the appendix Figure 1 - Figure 2 A preferred description of the present invention is provided below.

[0062] See attached document Figure 1 This invention proposes a path planning method for guide robots that integrates lidar and camera semantics, comprising the following steps:

[0063] S1. Perform visual semantic scanning to identify suspicious areas in environmental image data and generate corresponding suspicious area markers;

[0064] S2. Based on the suspicious area marker, generate and transmit a detection instruction data packet for triggering collaborative detection, wherein the detection instruction data packet includes at least the visual rule type that triggers the suspicious area marker;

[0065] S3. According to the exploration command data packet, the lidar performs directional exploration of the target's physical space and acquires point cloud data of the target's physical space;

[0066] S4. Call the physical feature analysis program associated with the visual rule type pre-recorded in the exploration instruction data packet to parse the point cloud data and obtain the physical feature analysis results.

[0067] S5. Based on the physical feature analysis results, generate a physical status report, which includes physical status conclusions;

[0068] S6. Based on the exploration command data packet and the physical status report, perform collaborative decision-making to verify two-way information in order to form local risk markers;

[0069] S7. Perform global path planning based on local risk markings and output the final navigation path.

[0070] In one embodiment of the present invention, step S1 includes the following steps:

[0071] The system uses the forward-facing camera of the guide robot to collect environmental image data and performs preliminary analysis of the environmental image data using a semantic segmentation model. It then enables a built-in hybrid risk feature filter, which includes at least one set of preset visual rules to filter suspicious areas from the environmental image data. The image areas that meet any visual rule selected by the hybrid risk feature filter are marked as suspicious areas, and a suspicious area label containing the image coordinates of the suspicious area, the type of visual rule that triggered the label, and the visual confidence level is generated.

[0072] Specifically, after the guide robot is activated, its main control unit first activates the camera module installed in front of the robot and facing the direction of travel. This camera continuously acquires raw digital images of the environment in front of the robot, forming a continuous stream of environmental image data. Each frame of environmental image data acquired is transmitted to the robot's embedded processor in real time. The processor pre-loads and runs a trained semantic segmentation model, which is a trained deep neural network capable of pixel-level classification of input images. This model can be trained on a dataset including hundreds of thousands of urban scene images to perform preliminary environmental element parsing. Furthermore, the semantic segmentation model adopts a fully convolutional neural network architecture such as DeepLabV3+ or U-Net. Its training dataset includes the Cityscapes open-source dataset and at least 50,000 guide scene images collected independently, containing specific labels such as tactile paving, sidewalks, puddles, and obstacles. During training, a weighted combination of cross-entropy loss function and Dice coefficient loss function is used to optimize the segmentation accuracy of small-area samples, such as tactile paving stripes. The model performs pixel-by-pixel analysis on the input environmental image data, initially classifies different regions in the image into common environmental element categories such as roads, pedestrians, and vehicles, and generates analysis results with preliminary category labels.

[0073] While completing the initial semantic parsing, the system's collaborative control unit simultaneously activates the built-in hybrid risk feature filter. This filter is a software module based on a predefined set of logical rules. Its goal is to filter out image regions that are difficult for a single visual sensor to clearly and with high confidence due to the interplay of factors such as complex ambient lighting conditions, special surface materials of objects, or minor geometric deformations. The so-called visual rule type refers to the image processing logical unit preset in the hybrid risk feature filter, targeting specific environmental risk physical, optical, or geometric features, including but not limited to high-brightness patch rules, tactile paving anomaly rules, and low-profile rules. These rules are set based on the principles of light reflection, national standard dimensions for tactile paving, and ergonomic obstacle height standards.

[0074] The hybrid risk feature filter loads at least one set of preset visual rules and scans the environmental image data frame by frame after preliminary parsing by the semantic segmentation model. The visual rules are a series of predefined image feature matching conditions within the hybrid risk feature filter. For each candidate region in the image, the filter sequentially applies at least three core visual rules for judgment.

[0075] The first type is the high-brightness patch rule. This rule identifies continuous patchy areas in an image located at non-artificial or natural light sources, whose brightness or reflectivity is significantly higher than the surrounding road texture background. These areas may correspond to water accumulation or oil stains. Specifically, this rule calculates the average brightness value of candidate regions, such as a sliding window. And compared with the pre-calculated average brightness value of the current frame road surface background area. Compare. When the condition is met. When this rule is triggered, This refers to the brightness contrast threshold. The value range is typically 1.2 to 2.0, and its setting is based on statistical analysis of a large amount of laboratory and field test image data containing water accumulation, oil stains, and normal road surfaces. The statistical analysis aims to find a threshold range that can effectively distinguish abnormally bright areas from normal reflective areas, in order to balance the detection rate and false alarm rate. Specifically, by collecting samples of wet and dry bright tiles under different lighting conditions, such as sunny days, cloudy days, and artificial lighting, ROC curves (Receiving Controller Characteristic curves) are plotted, and the threshold closest to (0,1) is selected as the threshold value. The baseline value. For example, it can be set to... A value of 1.5 means that when the average brightness of a candidate region exceeds the background brightness by 50%, it is considered to meet the characteristics of a bright patch. Furthermore, this rule can be further refined by combining region connectivity and shape regularity, such as ensuring the area is within a certain range and the aspect ratio is close to 1, to improve accuracy.

[0076] The second type is the tactile paving anomaly rule. This rule first locates the tactile paving area based on image recognition results, and then analyzes whether there are unexpected interruptions in the continuity of the unique raised stripe pattern of the tactile paving, or whether it is partially or completely covered by images of other objects of different colors or textures. For example, a geometric model of the tactile paving stripes can be established using image processing techniques such as Hough transform or template matching. Assuming the physical width of a standard tactile paving stripe is W, such as 5 cm, and the spacing is D, such as 10 cm, in the image, the physical dimensions are mapped to pixel dimensions using camera calibration parameters to obtain the corresponding pixel width. and pixel pitch The rule detects the continuity of stripes within the tactile paving area: if a continuous interruption in pixel length is detected... If the pixel area of ​​the covered region is not specified, it is considered an interruption exception; or the pixel area of ​​the covered region is not specified. If it is, then it is determined to be a coverage anomaly, where, This is the estimated length of a single tactile paving brick in the image. and This is the proportional threshold. Proportional threshold The value can range from 0.3 to 0.7. The value range can be from 0.3 to 0.8, and its model parameters are set based on public accessibility facility design specifications (such as stripe size tolerance) and image analysis data of actual occlusion scenarios (such as paper or small package coverage). For example, it can be set to... It is 0.5. The value is 0.5, meaning that the rule is triggered when the length of the stripe break exceeds half the width of a single stripe, or when the area covered exceeds half the area of ​​a single tactile paving brick.

[0077] The third type is the low-profile rule. This rule scans the area below a predetermined height from the ground in the image and identifies whether there are irregularly shaped, non-ground-extending, hanging or protruding contour edges. In practice, this requires combining camera intrinsic parameters and pose information to map the pixel coordinates in the image to a 3D space in the robot's coordinate system. A height threshold is set. For example, 0.5 meters corresponds to the maximum permissible height above ground level in actual space that may pose a collision risk to the guide robot or user. All elements in the rule-based scan image whose corresponding spatial height is determined to be below a certain level through monocular depth estimation or prior knowledge are considered to be below this level. The regions are then processed using edge detection algorithms (such as the Canny operator) to extract contours. Next, the distribution of the contour's normal direction is analyzed: the gradient direction at each point on the contour is calculated, and the angle between the gradient and the ground normal is statistically analyzed; this angle is typically perpendicular. If continuous contour segments exist, the angle between the average normal direction and the ground normal is calculated. satisfy ,For example If the angle is 30° and the length of the line segment exceeds a certain pixel threshold, it is determined to be a low-profile protrusion or hanging contour. Height threshold The angle threshold is set based on the knee height of common adults in ergonomics, the body height of the guide robot, and its maneuverability data. The value can range from 20° to 45°, and it is set based on statistical data of the approximate angle between common hanging objects, such as tree branches, cables, signs and the ground.

[0078] When a region in an image is determined to meet any one or more of the aforementioned visual rules, the hybrid risk feature filter classifies it as a suspicious region. Subsequently, a rectangular bounding box that exactly surrounds the suspicious region is generated in the image frame. The pixel coordinates of the bounding box are recorded, along with the specific visual rule type that triggered the label. A visual confidence level is calculated based on the semantic segmentation model and the rule matching accuracy. The visual confidence level is a value ranging from 0 to 100, representing the reliability of the current visual recognition result. For example, below 60 indicates low confidence, 60 to 80 indicates medium confidence, and above 80 indicates high confidence. The calculation can be based on multi-factor fusion. For example, for the bright patch rule, the confidence level is:

[0079]

[0080] in The brightness and contrast exceed the threshold. The score after normalization of the degree, for example, , Scaling factor The sensitivity of the score to excessive brightness is used to control the score. For example, the value can range from 0.2 to 1.0; in this embodiment, it is taken as... It is 0.5. It is the standard deviation of brightness within the candidate region. , It is a weighting coefficient and Weighting coefficients , This can be achieved by training a dataset using machine learning methods such as logistic regression or support vector machines, based on the annotated real risk areas and false positive areas in historical data, to maximize discriminative power. For example, one could take... It is 0.7. The confidence level is 0.3. The confidence level is ultimately mapped to an integer range of 0-100.

[0081] Ultimately, image coordinate information, visual rule type, and visual confidence level are bound together to form a complete label for the suspicious area. It should be noted that the suspicious area label refers to the image area that has been determined by the hybrid risk feature filter to meet the preset risk features and needs further investigation. It is structured data with clear image boundary coordinates, trigger rule type, and visual confidence level.

[0082] For example, in a test dataset of 5000 images of urban sidewalk scenes, when the environmental image data shows a waterlogged area on the road after rain, the semantic segmentation model might initially classify it as a road. The hybrid risk feature filter, applying a high-brightness patch rule, identifies that the waterlogged area forms continuous patches with brightness values ​​exceeding 160% of the average brightness of a dry road surface at non-light source locations. ,satisfy The conditions meet the rule characteristics. Therefore, the waterlogged area is marked as a suspicious area, and its image bounding box coordinates are recorded as the top left corner (200, 300) and the bottom right corner (280, 380). The visual rule type is high-brightness patch, and the visual confidence level is calculated to be 75 based on the consistency of patch brightness and boundary ambiguity. In this example, the image coordinates, rule type, and confidence level data together constitute the complete label for this suspicious waterlogged area.

[0083] In one embodiment of the present invention, step S2 includes the following steps:

[0084] Extract image coordinate information of suspicious area markers; record the visual rule type that triggers the suspicious area markers and evaluate the current visual confidence level; encapsulate the image coordinate information, visual rule type, and confidence level into a structured exploration instruction data packet; and transmit the exploration instruction data packet to the collaborative control unit.

[0085] Specifically, after identifying and marking suspicious areas, image coordinate information is extracted from the suspicious area marking data structure. This image coordinate information directly originates from the pixel coordinate values ​​of the rectangular bounding boxes generated during the marking process. Next, the specific visual rule type that triggered the suspicious area, as well as the associated calculated visual confidence level, are simultaneously read from the same marking data.

[0086] It should be noted that image coordinate information refers to the boundary data extracted from the suspicious area marker and used to locate the area in the two-dimensional image plane. It includes at least the pixel coordinate values ​​of the upper left and lower right corners of the rectangular bounding box, such as the upper left corner (X1, Y1) and the lower right corner (X2, Y2). This data comes directly from the output of the hybrid risk feature filter.

[0087] The visual rule type is a string identifier that indicates which specific rule in the mixed risk feature filter triggered the marking of the current suspicious area. It is a predefined enumeration value, such as a bright patch, tactile paving anomaly, or low profile.

[0088] Subsequently, a structured data container, namely the exploration command data packet, is created. This data packet has a fixed field format, encapsulating this information into three key fields: the first field stores image coordinate information, the second field stores the visual rule type, and the third field stores the visual confidence level. The exploration command data packet does not contain any final judgment or conclusion on the physical properties of the suspicious area; its sole purpose is to carry the questioning signal from the vision system and serve as an explicit trigger command to initiate subsequent LiDAR-assisted exploration.

[0089] After encapsulation, the exploration command data packet is sent from the vision processing module to the central processing module, i.e. the collaborative control unit, which is responsible for coordinating the work of multiple sensors, through the high-speed data bus inside the robot to complete the transmission of the command.

[0090] It should be noted that the exploration instruction data packet is a structured data object used to transmit instructions between modules within the system. It includes a data structure with three mandatory fields: image coordinate information, visual rule type, and visual confidence level. This structure design ensures the integrity and parsability of the instruction information. The collaborative control unit is the core software module of the guide robot's main control system, responsible for receiving instructions or data from different sensors, performing task scheduling, information fusion, and collaborative decision-making.

[0091] For example, from the suspicious area markers targeting waterlogged areas, the image bounding box coordinates (200, 300) and (280, 380) are extracted as image coordinate information. Simultaneously, the visual rule type triggering the marker is read as "highlighted patch," and the associated visual confidence level is 75. Subsequently, these three pieces of information are encapsulated into a single investigation command data packet, where the image coordinate information field values ​​are (200, 300, 280, 380), the visual rule type field value is "highlighted patch," and the visual confidence level field value is 75. This data packet is sent to the collaborative control unit via the internal bus.

[0092] In one embodiment of the present invention, step S3 includes the following steps:

[0093] The system parses the exploration command data packet and converts the image coordinate information it contains into the target physical space in the lidar coordinate system; it dynamically adjusts the lidar scanning mode, switching from the normal environment scanning mode to the directional exploration mode; in the directional exploration mode, it focuses the laser beam onto the target physical space to obtain the point cloud data of the target physical space.

[0094] When the collaborative control unit receives the detection command data packet from step S2, it initiates the lidar directional detection process. Specifically, the collaborative control unit parses the data packet and extracts the image coordinate information field, i.e., the pixel coordinates of the bounding box of the suspicious area in the camera image plane.

[0095] Subsequently, coordinate transformation parameters, pre-determined through joint calibration, are invoked. These parameters establish a mapping relationship between the camera image coordinate system and the LiDAR 3D spatial coordinate system. Using this mapping relationship, the image coordinate information is converted into a 3D spatial region in the LiDAR coordinate system. This region is the target physical space that needs to be explored. The target physical space refers to the 3D spatial range to be explored, defined in the LiDAR coordinate system and obtained from the image coordinate transformation. It is a cubic region bounded by minimum and maximum coordinate values. During the transformation, considering the uncertainty of visual positioning, the calculated target physical space is usually appropriately expanded. For example, assuming the image bounding box transformation yields the center coordinates... and initial size The expanded target space can be set as follows (assuming the ground is z=0):

[0096]

[0097]

[0098]

[0099] in, , , This is an expansion factor, typically ranging from 1.1 to 1.5, used to ensure complete coverage of suspicious areas and compensate for calibration and depth estimation errors. For example, a value of [value missing] can be taken as [value missing]. .

[0100] The collaborative control unit sends a mode switching command to the lidar's control system, dynamically switching the lidar from its continuously operating default conventional environment scanning mode to a directional detection mode. The conventional environment scanning mode is the lidar's default operating mode for large-area environmental perception and mapping. In this mode, the scanning mechanism rotates periodically at a constant speed, and the laser beam uniformly covers a fan-shaped area in front, resulting in a relatively low point cloud density. The directional detection mode is a temporary operating mode activated by the lidar based on a command, performing high-density scanning on a specific small area. In this mode, the scanning mechanism adjusts its trajectory or pauses wide-area scanning, focusing the laser emitter's direction and repeatedly covering the target physical space to achieve a surge in local point cloud density. In directional detection mode, the key control parameters are the scanning duration and the target point cloud density. Scanning duration... Typically, the timeframe is between 50 and 200 milliseconds, and its setting needs to strike a balance between acquiring sufficient data and ensuring system real-time performance. For example, it can be set to... The timeframe is 100 milliseconds. The point cloud density target is the point cloud density that the lidar control system attempts to achieve within the target's physical space during this duration, typically set to 10 to 50 times that of the conventional mode. For example, if the point cloud density at 1 meter in the conventional mode is approximately 100 points / square meter, the target density in the directional mode can be set to 3000 points / square meter. This target value is achieved by adjusting underlying parameters such as the laser emission frequency and scanning angular velocity; the specific parameters depend on the lidar model and performance.

[0101] In directional detection mode, the lidar control system reconfigures the motion parameters of its scanning mechanism and the laser emission timing, focusing the laser beam onto the target physical space within a preset, extremely short time, such as 100 milliseconds, and repeatedly scanning the calculated target physical space. This focused projection significantly increases the number of laser pulses illuminating the target physical space within the same unit of time and space, thereby acquiring point cloud data within that target physical space. Compared to the point cloud uniformly distributed across a large scene in conventional environmental scanning mode, the point cloud data acquired in this mode has higher point cloud resolution and data sampling rate within the target physical space, thus revealing more subtle geometric or reflection features.

[0102] It should be noted that the lidar coordinate system is a three-dimensional rectangular coordinate system centered on the lidar sensor, used to describe the precise position of a point in space. It has an origin, three orthogonal axes, and a unit of length, determined through sensor factory calibration and system joint calibration. The preset extremely short time refers to the preset duration for completing one directional survey; this time is set to balance the need for real-time surveying with the requirement to acquire sufficient data. Point cloud resolution refers to the ability of point cloud data to represent spatial details. In this embodiment, it specifically refers to the average spatial distance between adjacent points within the target physical space; the smaller the distance, the higher the resolution. The data sampling rate in this invention refers to the number of effective laser pulses emitted and received by the lidar into the target physical space per unit time; the larger the number, the higher the sampling rate.

[0103] It should be noted that the specific implementation of the directional detection mode can be adapted to different types of LiDAR hardware: For solid-state LiDAR or MEMS LiDAR that supports dynamic ROI adjustment, the directional detection mode refers to physically adjusting the emission frequency or scanning range to concentrate the laser pulse on the target physical space; for mechanical LiDAR with constant rotation speed, the directional detection mode refers to the logical directional mode, that is, at the software level, data processing of non-target sectors is shielded to free up computing power, and time-domain accumulation or super-resolution algorithms are performed on the point cloud data of the target sector to obtain an equivalent high-density point cloud at the data level.

[0104] For example, continuing from the previous example, the cooperative control unit parses the exploration command data packet and extracts the image coordinate information (200, 300, 280, 380). Using a preset coordinate transformation matrix, it calculates the target physical space range corresponding to the image area in the lidar coordinate system, for example, 0.5 meters to 0.8 meters in the X-axis direction, -0.1 meters to 0.1 meters in the Y-axis direction, and 0 meters to 0.05 meters in the Z-axis direction (height). After applying expansion coefficients (1.2, 1.2, 1.3), the final target physical space range is adjusted to: X: 0.44-0.86, Y: -0.22-0.22, Z: 0-0.065. Subsequently, the lidar is instructed to switch to directional exploration mode. In the next 100 milliseconds, the lidar controls the laser beam to focus on scanning the target physical space and acquire point cloud data of the area. Suppose that, statistically, the point cloud density in the target space reaches 10 points per cubic centimeter in this mode, while in the conventional environmental scanning mode, the point cloud density in the same area is only 0.5 points per cubic centimeter. This illustrates the process from parsing instructions, transforming coordinates, adjusting the scanning mode to acquiring local point cloud data, demonstrating the improvement in point cloud resolution and data sampling rate brought about by the directional exploration mode.

[0105] In one embodiment of the present invention, step S4 includes the following steps:

[0106] Based on the visual rule type recorded in the exploration command data packet, the corresponding physical feature analysis program is matched and called from the preset analysis program library; the point cloud data is input into the called physical feature analysis program; the physical feature analysis program is run to obtain the qualitative or quantitative physical feature analysis results describing the physical features of the target physical space.

[0107] Specifically, after storing the point cloud data acquired for the target physical space, the point cloud data is parsed. First, the collaborative control unit reads the visual rule type field recorded in the exploration command data packet. Based on the string value of this field, it accesses its internally pre-built analysis program library. This program library is an index table stored in non-volatile memory, which establishes a mapping relationship between different visual rule type strings and the corresponding physical feature analysis program execution code or program file path.

[0108] Based on the input visual rule type string, a matching query is performed in the index table, and the associated specified physical feature analysis program is invoked. This program is a pre-built software module in the analysis program library used to extract specific physical attribute information from point cloud data. It receives point cloud data as input, executes specific algorithm logic, and outputs analysis results representing the physical state. Specifically, the point cloud data is loaded into the program's input buffer, and the following analysis logic is executed:

[0109] If the visual rule type is a bright patch, then the laser echo intensity analysis program is invoked. This program traverses all points in the point cloud data that belong to the target physical space, reads the echo intensity value attribute attached to each data point, and calculates the average echo intensity value of all points in the region. Simultaneously, the program defines a geometrically adjacent annular reference region around the target physical space and calculates the average echo intensity of the point cloud within this reference region. Calculate the absolute difference between the target region's average value and the reference region's average value: In addition, the standard deviation of the point cloud echo intensity values ​​within the target area is calculated. To assess its distribution uniformity, the absolute differences are compared. The difference in intensity from the preset threshold and compare standard deviations With the preset uniformity threshold The program comprehensively judges whether the area exhibits the high reflectivity and uniform reflection characteristics unique to liquid surfaces.

[0110] Among them, the intensity difference threshold The value range is, for example, 15 to 30 (relative intensity units), which is based on statistical analysis of more than 100 sets of measured lidar echo intensity data of dry asphalt pavement, paving stones, and pavement with water and oil stains. The aim is to find a value that can effectively distinguish between liquids and dry, highly reflective pavement. For example, it can be set... The uniformity threshold is 20. The value range is, for example, 3 to 8 (relative intensity units), which is set based on the characteristic that liquid surfaces generally reflect light uniformly while dry, rough surfaces show greater fluctuations in reflection intensity. For example, it can be set to... The value is 5. When both conditions are met... and The analysis program outputs qualitative results of high reflectivity and high uniformity characteristics, and may also include quantitative results such as: .

[0111] If the visual rule type is tactile paving anomaly or low profile, the point cloud geometry analysis program is invoked. This analysis program also operates based on point cloud data. For tactile paving anomaly rules, the program focuses on analyzing the suspected edge region of the covering, selecting sampling points near the edge according to a grid, and for each sampling point, calculating the height difference between the two adjacent points on either side along the direction perpendicular to the edge. Horizontal distance from this side The ratio of the two values ​​yields the local height abrupt change gradient value: And calculate their average value. Simultaneously, the program analyzes the point cloud of the area where the covering is located, detecting whether there is a continuous region whose point cloud height is significantly lower than the reference height of the surrounding tactile paving surface and whose point cloud density is lower than the background density, thereby determining whether there are point cloud void features. The key to determining the properties of the covering lies in the average gradient. With preset soft and hard edge gradient thresholds Comparison of soft and hard edge gradient thresholds. The value range is, for example, from 0.2 to 0.4, and it is set by analyzing point cloud data of the edges of 50 different soft and hard coverings, such as cardboard boxes, cloth, wood boards, and bricks. The edges of soft coverings are usually more blurred, with gentle gradient changes; the edges of hard coverings are sharp, with drastic gradient changes. For example, it can be set... .when At this time, it tends to be judged as a soft cover edge feature. If obvious point cloud holes are detected below at the same time, it may indicate that there is a risk of collapse below.

[0112] For low-profile rules, the program uses Principal Component Analysis (PCA) to estimate the 3D spatial normal directions of the neighboring point clouds for each point in the entire suspicious region. Subsequently, the program statistically analyzes all normal direction vectors to determine whether they cluster in space towards one or more specific directions, rather than being randomly distributed or all vertically upwards. Specifically, the pitch angles of the normal vectors can be statistically analyzed. If a significant number of normal vectors have pitch angles... satisfy ,in A normal tilt threshold, such as 20°, indicates the presence of a protrusion or overhang with a specific spatial orientation. Normal tilt threshold The value can range from 15° to 30°, used to filter out changes in the normal caused by minor undulations in the ground. For example, setting... The angle is 25°. The program can output analysis results such as: "Detected a downward convergence of normal directions, with an average pitch angle of 35°."

[0113] After the analysis procedure is completed, the above qualitative or quantitative analysis results will be output.

[0114] It should be noted that the laser echo intensity analysis program is used to analyze the light signal reflection intensity information recorded in point cloud data to infer the reflection characteristics of the target surface. This program is designed based on the physical principle of lidar ranging and the differences in reflectivity of different materials. The echo intensity value is the intensity of the laser pulse signal recorded by the lidar receiver after reflection from the target surface. It is usually standardized into a relative value, which is a scalar value stored along with the coordinates of each three-dimensional spatial point.

[0115] Point cloud geometry analysis programs are used to analyze the three-dimensional spatial distribution and geometric relationships of points in point cloud data to infer the shape, outline, and structural features of target objects. Local height abrupt change gradients are dimensionless numerical values ​​used to quantify the degree of height variation in the vertical direction between adjacent regions in a point cloud; a larger value indicates a sharper edge.

[0116] The distribution of normal directions in three-dimensional space refers to the calculation of the normal vectors of the plane fitted to the local surface of the point cloud and the analysis of the statistical characteristics of the pointing of these vectors in three-dimensional space to infer the surface orientation.

[0117] Point cloud voids refer to the phenomenon in point cloud data where the number of points in a continuous three-dimensional spatial region that should contain surface points is abnormally low or completely missing. This usually indicates that there is a depression or a hollow object below that location.

[0118] For example, the point cloud data from the aforementioned example is received, and the visual rule type field in the probe instruction data packet is identified as a bright patch. Based on this, the collaborative control unit matches and invokes a laser echo intensity analysis program from the program library. This program reads the target spatial point cloud and, assuming, calculates the average echo intensity value. Given a value of 85, calculate the average intensity value of the surrounding reference area. The value is 60. Substituting this into the formula, we get... The value is 25. This is greater than the preset threshold of 20, and the calculated intensity standard deviation within the target area is 3, which is lower than the uniformity threshold of 5. The program outputs the analysis result as: high reflectivity and high uniformity characteristics were detected. This result demonstrates that the laser echo intensity analysis program can be correctly invoked based on the rule type and effectively identify the surface reflection characteristics of suspected liquids using point cloud data.

[0119] In one embodiment of the present invention, step S5 includes the following steps:

[0120] The physical feature analysis results are mapped to explicit physical state conclusions; the analysis confidence level is calculated based on the deviation between the physical feature analysis results and preset thresholds and the point cloud data quality; the physical state conclusions and analysis confidence levels are encapsulated to generate a structured physical state report and fed back to the collaborative control unit.

[0121] Specifically, the system receives the analysis results of point cloud data from the physical feature analysis program. The built-in conclusion mapping logic module then converts these results into physical state conclusions that directly describe the physical nature of the target, based on the type of program being called and its specific output. For example, if the result from the laser echo intensity analysis program simultaneously satisfies… and Then the mapping logic will transform it into a physical state conclusion: liquid plane characteristics. If the result from the point cloud geometry analysis program is... If point cloud holes are detected, the cloud may transform into soft cover with a risk of collapse underneath. This transformation process is completed based on a preset condition judgment table.

[0122] Meanwhile, based on the deviation of the calculated indicators from the threshold in the analysis program and the quality of the point cloud data, the analytical confidence level of this physical exploration was assessed. Analyze confidence levels The calculation can take into account the following factors:

[0123] Feature saliency: For example, in determining liquids;

[0124]

[0125]

[0126] in , This is a scaling factor, such as 0.5;

[0127] Point cloud data quality The score can be determined by combining point cloud density and noise level, and normalized to [0,1].

[0128] final: ,in , , The weights are denoted as , and their sum is 1. These weights can be obtained through regression analysis of historical point cloud data labeled with true states and confidence levels. For example, for liquid judgment, can be taken as . It is 0.4. It is 0.4. It is 0.2. If It is 25. It is 20. It is 3. It is 5. If it is 0.9, then It is 0.5. The value is 0.8, and the calculated value is... The confidence level is also mapped to an integer range of 0-100.

[0129] Subsequently, a structured physical status report is created, which includes at least two core fields: physical status conclusion and analysis confidence level. Finally, the physical status report is sent back to the collaborative control unit that initiated this exploration mission via the internal data bus.

[0130] For example, the analysis results output by the laser echo intensity analysis program are: high reflectivity and high uniformity characteristics were detected. It is 25. (3). The conclusion mapping logic module converts the conclusion into a physical state conclusion: liquid plane characteristics, based on a preset mapping relationship. Simultaneously, based on the aforementioned confidence level calculation method, the analysis confidence level is calculated to be 70%. Subsequently, a physical state report is encapsulated, with the physical state conclusion field valued at "liquid plane characteristics" and the analysis confidence level field valued at "70," and this report is sent back to the collaborative control unit.

[0131] In one embodiment of the present invention, step S6 includes the following steps:

[0132] Receive exploration command data packets and physical status reports; cross-validate the visual rule types in the exploration command data packets and the physical status conclusions in the physical status reports according to the preset decision rule table; generate local risk markers based on the cross-validation results, which include the spatial positioning information of the risk area, the risk level, and the recommended passage strategy.

[0133] Specifically, after both the exploration command data packet and the physical status report have been sent to the collaborative control unit, the collaborative control unit first synchronously receives and parses these two data packets: extracting the visual rule type and visual confidence level from the exploration command data packet. Extract physical state conclusions and analyze confidence levels from the physical state report. .

[0134] The two sets of information, one from vision and the other from LiDAR, were then cross-validated. Cross-validation involves comparing and logically correlating the exploration command data packets from the vision sensor with key information from the physical state report from the LiDAR sensor. The core logic is to compare the visual rule types with the physical state conclusions derived from radar reconnaissance, and based on a pre-defined decision rule table, to determine the final risk assessment of the suspicious area. The pre-defined decision rule table defines the output results corresponding to different information combinations, and it uses an "IF-THEN" rule format. In addition to conclusion matching, confidence thresholds and fusion rules can also be introduced. For example:

[0135] Rule 1: IF (Visual Rule Type = "Highlight Patch") AND (Physical State Conclusion = "Liquid Surface Feature") AND ( AND ( THEN Risk Level = "High Risk", Recommended Strategy = "No Entry / Detour Required"

[0136] Rule 2: IF (Visual Rule Type = "Tactile Path Anomaly") AND (Physical State Conclusion = "Soft Coverage and No Risk of Subsidence") AND ( AND ( THEN Risk Level = "Medium Risk", Recommended Strategy = "Passable but slow down (recommended speed ≤ 0.3 m / s)".

[0137] Rule 3: IF (Visual rule type = "Tactile paving abnormal") AND (Physical state conclusion = "Soft coverage and risk of collapse below") THEN Risk level = "High risk", Recommended strategy = "No passage".

[0138] Rule 4: IF (Physical state conclusion = "flat road surface with no abnormal features") THEN Risk level = "Safe", Recommended strategy = "Normal passage".

[0139] in, and These are the minimum confidence thresholds for visual and radar analysis, respectively, used to filter out low-confidence detections. Confidence Threshold and The value can range from 50 to 70, depending on the system's tolerance for false alarms and missed alarms. For example, it can be set to... It is 60. The value is 60. For medium- and high-risk areas, it can also be adjusted according to... and The specific values ​​are used to fine-tune the recommended strategy parameters, such as the suggested speed value for deceleration.

[0140] Based on the cross-validation results, high-confidence local risk marker data objects are generated. These local risk markers are structured data, with fields including at least: vertex coordinates of the risk area's spatial location polygon obtained through coordinate transformation, a risk level code determined according to the decision rule table, and a associated recommended passage strategy text description. The risk level code can use enumerated values, such as: 0 – safe, 1 – low risk (passable, minor impact), 2 – medium risk (cautious passage / deceleration required), 3 – high risk (no passage allowed). The recommended passage strategy can be associated with specific motion control parameters; for example, for deceleration, the maximum permissible passage speed and maximum angular velocity can be specified. These parameters are obtained through experimental calibration based on robot dynamics, user comfort, and risk level.

[0141] For example, the collaborative control unit simultaneously receives a probe instruction data packet: (Rule type: bright patch, visual confidence) :75), and the physical state report: (Conclusion: liquid plane characteristics, analysis confidence level) :70). Perform cross-validation and query the decision rule table.

[0142] because The rule matches the conclusion, triggering rule 1. The output is qualitatively classified as high risk, generating a "no passage" marker. Based on the image coordinates and coordinate transformation parameters of the suspicious area, the spatial positioning polygon coordinates corresponding to it in the global map are calculated. Finally, a local risk marker is generated, such as: {Spatial positioning: Polygon([x1,y1],...,[xn,yn]), Risk level: 3 (high risk), Recommended strategy: {action: "detour", parameter: null}}.

[0143] In one embodiment of the present invention, step S7 includes the following steps:

[0144] The global path planner is invoked, and local risk markers are overlaid as dynamic constraint layers onto the global map. Based on the risk level in the local risk markers, the travel cost of the corresponding area in the path planning algorithm is adjusted. The global path planner calculates and outputs the final navigation path based on the adjusted map.

[0145] Specifically, after generating local risk markers, the dynamic optimization process for the final navigation path is initiated. First, the main control module of the robot navigation system calls the global path planner software, one of its core components. The global path planner is an algorithm module within the robot navigation software that calculates the optimal or feasible path from the starting point to the target point based on known environmental map information, the robot's kinematic model, and a cost function. This path planner runs continuously, its input being a pre-built global map stored in the robot's memory. The global map is a digital model describing the static structure of the environment, obtained beforehand through mapping or loading. It includes a two-dimensional or three-dimensional data structure containing information such as traversable areas and the locations of fixed obstacles. This map describes the static layout and traversability attributes of the robot's operating environment in the form of a grid or topology map. The path planner's regular function is to calculate a path with the lowest cumulative cost from the starting point to the destination on the map based on the robot's current pose and the target point coordinates. Upon receiving the real-time generated local risk marker data packet, a dynamic constraint layer is created. This is a data structure aligned with the global map space but independent of it, specifically designed to carry temporary, high-priority obstacle or risk information.

[0146] The spatial positioning polygon vertex coordinates from the data unpacked in the local risk marker are transformed and drawn onto the dynamic constraint layer to form one or more closed regions with labels; at the same time, the risk level and recommended passage strategy information are bound to the region as attributes.

[0147] Subsequently, this dynamic constraint layer is overlaid as a high-priority layer on top of the current global map data for use by the path planner. When performing path search and cost calculation, the path planner considers both the static global map's cost layer and the dynamic constraint layer simultaneously. Path costs are typically represented in a grid map or graph structure. For areas identified in the dynamic constraint layer, the path planner adjusts the travel cost of each grid cell (or edge) within that area based on the risk level. .

[0148] If the risk level is high: then set the passage cost of all grids in the area to a maximum value. ,For example Alternatively, obstacle cost values ​​defined internally by the planner can be used. This causes the total cost of any path crossing the area to tend towards infinity, thus effectively eliminating it.

[0149] If the risk level is medium risk: then multiply the passage cost of all grids in the area by a cost coefficient greater than 1. ,Right now Cost coefficient The value typically ranges from 2.0 to 10.0, and the specific value can be determined experimentally based on factors such as risk level segmentation, robot performance, and user comfort requirements. For example, for a medium-risk area requiring deceleration to 0.3 meters per second, a value can be set... At the same time, the planner can record the recommended speed for the area. It is used for subsequent motion control.

[0150] If the risk level is low or safe, the cost is usually not adjusted, or multiplied by a coefficient close to 1 to indicate a slight impact. This cost adjustment strategy causes the path planner to prioritize low-cost paths during the search, thus automatically bypassing high-risk areas or only crossing medium-to-high-risk areas when necessary, but because its cost has been increased, unless the bypass cost is even higher.

[0151] The path planner, based on a map cost model updated with a dynamic constraint layer, re-executes the path search algorithm to calculate a new final navigation path that balances safety and smoothness. This path is output as a series of consecutive map coordinates. Finally, the final navigation path is sent to the robot's underlying motion control unit, driving the robot to perform movement actions according to the path sequence and adjusting based on grid attributes traversed, such as whether a recommended speed is associated. The corresponding speed control commands are generated. The final navigation path is a continuous spatial trajectory that is calculated and optimized by the global path planner and can be directly executed by the robot. It is an ordered sequence of coordinate points that defines the robot's movement route from its current position to the target position.

[0152] It should be noted that the dynamic constraint layer is a real-time updated data layer used to store information about temporarily appearing obstacles or risk areas. This layer is used for registration with the global map space and is independently written to and cleared from the static map. Path cost is the value assigned to each cell or edge on the map in the path planning algorithm, used to quantify the difficulty of traversing that point. The planner searches with the goal of minimizing the total path cost.

[0153] For example, a local risk marker is received, spatially located as polygon P, with a risk level of "no passage". The global path planner is calculating a regular path from point A to point B based on the global map. Upon receiving the marker, the area corresponding to polygon P is drawn on the dynamic constraint layer and marked as "no passage". In subsequent calculations, the planner sets the cost of any path segment crossing polygon P to 10. 9 The original planned path through P became extremely costly, so the planner researched and output a new final navigation path whose coordinate sequence bypassed the polygon region P, increasing the total path length from 15 meters to 16.5 meters.

[0154] If a risk level marker for region Q is received indicating that it is passable but requires deceleration, and its comfort cost coefficient is set to 3, the planner calculates that an alternative path without passing through Q is 17 meters long, while a path passing through Q is 15 meters long. The original cost of the path passing through Q is 15, which increases to 45 after multiplying by the coefficient 3, higher than the cost of the alternative path of 17. Therefore, the planner selects the 17-meter detour path as the final navigation path. If no suitable detour path is available, the planner may output a 15-meter path that passes through Q, while simultaneously generating an instruction to decelerate to 0.3 meters per second before entering region Q.

[0155] See appendix Figure 2 This invention also proposes a path planning system for a guide robot that integrates lidar and camera semantics, comprising the following modules:

[0156] The system comprises the following modules: a suspicious area identification and marking module, which performs visual semantic scanning to identify suspicious areas in environmental image data and generates corresponding suspicious area markings; a collaborative exploration command data packet transmission module, which generates and transmits exploration command data packets to trigger collaborative exploration based on the suspicious area markings, wherein the exploration command data packets include at least the visual rule type that triggers the suspicious area markings; a lidar exploration module, which performs directional exploration of the target physical space based on the exploration command data packets to acquire point cloud data of the target physical space; a point cloud physical feature parsing module, which calls a physical feature analysis program associated with the visual rule type pre-recorded in the exploration command data packets to parse the point cloud data and obtain physical feature analysis results; a physical status report feedback module, which generates a physical status report based on the physical feature analysis results, wherein the physical status report includes physical status conclusions; a risk collaborative decision-making module, which performs collaborative decision-making based on the exploration command data packets and the physical status report to form local risk markings; and a navigation path dynamic optimization module, which performs global path planning based on the local risk markings and outputs the final navigation path.

[0157] It should be noted that the formulas described above, through the principle of dimensional consistency and mathematical standardization methods (such as normalization, dimensionless parameter conversion, or unit system unification), can translate physical quantities with different properties into unitless standard values ​​or superimposed parameters of the same dimension. This eliminates the interference of different dimensions on the computational logic, allowing the formulas to retain the original data distribution characteristics while possessing mathematical rationality and adaptability to objective laws. The descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the invention.

[0158] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A path planning method for a guide robot that integrates lidar and camera semantics, characterized in that, Includes the following steps: S1. Perform visual semantic scanning to identify suspicious areas in environmental image data and generate corresponding suspicious area markers; S2. Based on the suspicious area marker, generate and transmit an exploration instruction data packet for triggering collaborative exploration, wherein the exploration instruction data packet includes at least a visual rule type for triggering the suspicious area marker; the visual rule type is an image processing logic unit for the physical optical or geometric features of environmental risks, including at least one of the following: bright patch rule, tactile paving anomaly rule, or low profile rule; S3. According to the exploration command data packet, the lidar performs directional exploration of the target physical space and acquires point cloud data of the target physical space; S4. Call the physical feature analysis program associated with the visual rule type pre-recorded in the exploration instruction data packet to parse the point cloud data and obtain the physical feature analysis results. S5. Based on the physical feature analysis results, generate a physical state report, which includes: mapping the physical feature analysis results to explicit physical state conclusions; calculating the analysis confidence level based on the deviation of the physical feature analysis results from the preset threshold and the point cloud data quality; and encapsulating the physical state conclusions and the analysis confidence level to generate a structured physical state report. S6. Based on the exploration command data packet and the physical status report, perform collaborative decision-making to verify two-way information in order to form local risk markers; S7. Perform global path planning based on local risk markings and output the final navigation path.

2. The path planning method for a guide robot integrating lidar and camera semantics according to claim 1, characterized in that, Step S1 includes the following steps: The forward-facing camera of the guide robot is used to collect environmental image data, and a semantic segmentation model is used to perform preliminary analysis of the environmental image data. Enable the built-in hybrid risk feature filter, which includes at least one set of preset visual rules for filtering suspicious areas from environmental image data; Image regions that meet any visual rule and are filtered by the hybrid risk feature filter are marked as suspicious areas, and suspicious area labels containing the image coordinates of the suspicious area, the visual rule type that triggered the label, and the visual confidence level are generated.

3. The path planning method for a guide robot integrating lidar and camera semantics according to claim 2, characterized in that, Generating and transmitting a probe command data packet to trigger a cooperative probe includes the following steps: Extract the image coordinate information of suspicious area markers; Record the types of visual rules that trigger suspicious area markings and assess the current visual confidence level; Image coordinate information, visual rule type, and confidence level are encapsulated into a structured exploration instruction data packet; The probe command data packet is transmitted to the collaborative control unit.

4. The path planning method for a guide robot integrating lidar and camera semantics according to claim 1, characterized in that, Acquiring point cloud data of the target physical space includes the following steps: Parse the exploration command data packet and convert the image coordinate information it contains into the target physical space in the lidar coordinate system; The scanning mode of the lidar is dynamically adjusted, switching from the conventional environmental scanning mode to the directional exploration mode; In directional exploration mode, the laser beam is concentrated and projected onto the target physical space to obtain point cloud data of the target physical space.

5. The path planning method for a guide robot integrating lidar and camera semantics according to claim 1, characterized in that, Obtaining physical characteristic analysis results includes the following steps: Based on the visual rule type recorded in the exploration command data packet, the corresponding physical feature analysis program is matched and called from the preset analysis program library; Input the point cloud data into the invoked physical feature analysis program; Run the physical feature analysis program to obtain qualitative or quantitative physical feature analysis results that describe the physical characteristics of the target physical space.

6. The path planning method for a guide robot integrating lidar and camera semantics according to claim 1, characterized in that, Generating local risk markers includes the following steps: Receive probe command data packets and physical status reports; Based on the preset decision rule table, the visual rule types in the exploration command data packet and the physical state conclusions in the physical state report are cross-validated. Local risk markers are generated based on the cross-validation results. These markers include the spatial location information of the risk area, the risk level, and recommended travel strategies.

7. The path planning method for a guide robot integrating lidar and camera semantics according to claim 1, characterized in that, Output the final navigation path, including the following steps: Invoke the global path planner and overlay local risk markers as a dynamic constraint layer onto the global map; Adjust the travel cost of the corresponding area in the path planning algorithm based on the risk level in the local risk markers; The global path planner calculates and outputs the final navigation path based on the adjusted map.

8. The path planning method for a guide robot integrating lidar and camera semantics according to claim 7, characterized in that, Risk markers must include at least the following fields: a spatial location field for recording the coordinates of the polygon vertices of the risk area in the global map coordinate system; a risk level field for recording the encoded risk severity; and a recommended travel strategy field for recording the specific action instructions associated with the risk level.

9. A path planning system for a blind robot that integrates lidar and camera semantics, characterized in that, Includes the following modules: The suspicious area identification and labeling module is used to perform visual semantic scanning to identify suspicious areas in environmental image data and generate corresponding suspicious area labels; The collaborative exploration instruction data packet transmission module generates and transmits exploration instruction data packets for triggering collaborative exploration based on suspicious area markers. The exploration instruction data packets include at least a visual rule type that triggers the suspicious area markers. The visual rule type is an image processing logic unit targeting the physical optical or geometric features of environmental risks, including at least one of the following: high-brightness patch rules, tactile paving anomaly rules, or low-profile rules. The lidar detection module is used to perform directional detection of the target's physical space according to the detection command data packet, and to acquire point cloud data of the target's physical space; The point cloud physical feature parsing module is used to call the physical feature analysis program associated with the visual rule type pre-recorded in the exploration command data packet to parse the point cloud data and obtain the physical feature analysis results. The physical state report feedback module generates a physical state report based on the physical feature analysis results. This includes: mapping the physical feature analysis results to explicit physical state conclusions; calculating the analysis confidence level based on the deviation of the physical feature analysis results from preset thresholds and the point cloud data quality; and encapsulating the physical state conclusions and analysis confidence level to generate a structured physical state report. The risk collaborative decision-making module performs collaborative decision-making based on two-way information verification of the exploration command data packet and the physical status report to form local risk markers; The navigation path dynamic optimization module performs global path planning based on local risk markers and outputs the final navigation path.

Images