Intelligent patrol robot sensing method based on monocular vision and laser radar

Abstract

The invention discloses an intelligent patrol robot sensing method based on monocular vision and a laser radar, and the method comprises the following steps: S1, collecting images and point cloud data, completing the synchronous registration, and constructing a structure program library; s2, extracting an image contour feature map and a point cloud contour model; s3, performing joint coding to generate a fusion representation vector; s4, calculating an image size and a spatial distance, and generating a size-distance joint representation; s5, inputting a Bayesian program learning model to generate a three-dimensional structure program; s6, matching the structure program library to generate a candidate recognition result; s7, generating a bionic evolution result based on candidate result reasoning; and S8, outputting a bionic evolution result and using the bionic evolution result to control the system. According to the invention, the image and the point cloud information are fused, and the target structure identification capability is enhanced.

Description

Technical Field

[0001] This invention relates to the field of robot perception technology, and in particular to a perception method for intelligent patrol robots based on monocular vision and lidar. Background Technology

[0002] Currently, intelligent patrol robots are widely used in security patrols, public area monitoring, and hazardous environment surveys. The accuracy and stability of their perception capabilities directly affect the effectiveness of patrol missions. Existing robot perception systems typically rely on a single visual sensor or LiDAR for environmental information acquisition. Monocular vision can acquire two-dimensional image information of objects and perform target detection and recognition through image processing algorithms, but it suffers from insufficient accuracy in depth estimation and spatial size inference, especially in scenarios with changing lighting, occlusion, and insufficient texture features, which can easily lead to recognition errors. LiDAR can provide high-precision three-dimensional point cloud data, providing relatively reliable depth information for spatial structure modeling, but it lacks image texture and color information. In complex scenes, relying solely on point cloud morphology for recognition may lead to difficulty in distinguishing different objects with similar structures. Furthermore, existing technologies for fusing monocular vision and LiDAR often employ simple data stitching or feature-level stitching, lacking in-depth modeling of the spatial correspondence between different modal information, resulting in limited fusion effects and difficulty in fully leveraging the advantages of both sensors.

[0003] In information processing, traditional methods often rely on rule-based geometric computation or deep learning-based end-to-end recognition networks. These methods frequently suffer from unstable recognition results when faced with multi-scale, deformation, and complex background interference. Particularly in modeling the relationship between size and distance, existing methods often neglect the structured relationship between the image projection size and the actual spatial size, relying solely on single-scale features for matching. This leads to a significant decrease in recognition accuracy at different viewing distances. Furthermore, existing recognition models are mostly static, lacking a modeling mechanism for the dynamic evolution of object contours as viewing angle and distance change, thus failing to achieve continuous perception and reasoning of object shapes similar to human vision.

[0004] In structural procedural modeling, few existing methods abstract the contours, dimensions, and spatial relationships of objects into structural programs and store them as reusable program libraries, lacking rapid matching and reasoning capabilities based on structured representations. Even those techniques that employ 3D modeling and template matching are mostly static matching based on fixed models, unable to adapt to dynamic contour changes in different scenarios, and exhibiting low matching efficiency and long recognition delays. Therefore, existing technologies struggle to simultaneously meet the demands for high accuracy, multi-scenario adaptability, and rapid response.

[0005] Therefore, how to provide a perception method for intelligent patrol robots based on monocular vision and lidar is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a perception method for intelligent patrol robots based on monocular vision and lidar. This invention synchronizes and registers the two-dimensional image information from monocular vision with the three-dimensional point cloud information from lidar. It combines image contour extraction, point cloud contour modeling, and multimodal feature fusion to generate a fused representation vector containing shape features and spatial depth features. Based on the joint representation of size and distance, a three-dimensional structure program is generated by introducing a Bayesian program learning model with graph structure and geometric configuration syntax. The three-dimensional structure program is matched with a structure program library, and combined with dynamic contour reasoning to generate biomimetic evolution results, thereby achieving accurate target recognition and spatial structure reconstruction. This method has the advantages of high fusion accuracy, strong dynamic adaptability, and fast recognition efficiency.

[0007] A perception method for an intelligent patrol robot based on monocular vision and lidar according to an embodiment of the present invention includes the following steps:

[0008] S1. Acquire monocular image data and lidar point cloud data, perform time synchronization and coordinate registration, generate aligned images and point cloud data, and build a structured program library.

[0009] S2. Extract contours and analyze textures from image data to generate image contour feature maps. Filter and complete point cloud data to generate point cloud contour models.

[0010] S3. Jointly encode the image contour feature map and the point cloud contour model to generate a fusion representation vector, which includes shape features and spatial depth features.

[0011] S4. Calculate the image projection size and spatial distance based on the fused representation vector, and generate a joint size-distance representation;

[0012] S5. The size-distance joint representation input is introduced into the Bayesian program learning model of graph structure and geometric configuration syntax to generate a three-dimensional structure program, which is used to represent the spatial contour structure of the target.

[0013] S6. Match the 3D structure program with the structure programs in the structure program library to generate candidate recognition results;

[0014] S7. Construct a program state sequence based on the candidate recognition results, input it into the Bayesian program learning model to perform dynamic contour reasoning, and generate biomimetic evolution results;

[0015] S8. Output the biomimetic evolution result as the final recognition result and input it into the robot control system for path planning and behavior scheduling.

[0016] Optionally, the structure program library includes a three-dimensional structure program obtained by performing program generation operations on the size-distance joint representation of historical target objects through a Bayesian program learning model. The three-dimensional structure program is represented by graph structure and geometric configuration syntax and is used to represent the spatial contour structure of objects.

[0017] Optionally, S2 specifically includes:

[0018] S21. Perform grayscale conversion and high-pass filtering on the image data to enhance the contrast of boundaries and textures in the image and generate an image gradient map.

[0019] S22. Calculate the edge intensity and direction information of each pixel based on the image gradient map, use the edge detection algorithm to extract the boundary pixel contour, and construct the initial image contour map;

[0020] S23. In the process of image edge detection, local gray-level distribution pattern analysis is performed on the neighborhood of each pixel to enhance the response of textured areas in boundary detection and generate image contour feature maps.

[0021] S24. Divide the point cloud data into a three-dimensional spatial grid and perform voxel filtering to retain the representative points of each grid cell, remove isolated noise points, and form a set of filtered points.

[0022] S25. Perform density completion processing on the filtered point set, infer the point positions of the missing regions based on the spatial distribution relationship of the known points, and insert them to fill the missing regions, generating a complete point set;

[0023] S26. Based on the changes in the spatial position and normal vector direction of each point in the completion point set, extract the boundary change point set, construct the point cloud contour structure through the boundary connection algorithm, and output the point cloud contour model.

[0024] Optionally, S3 specifically includes:

[0025] S31. Extract the image coordinate information and edge direction information of the boundary pixels from the image contour feature map;

[0026] S32. Extract the spatial coordinate information and normal vector direction information of the boundary points from the point cloud contour model;

[0027] S33. Based on the registration relationship between image coordinates and spatial coordinates, spatially correspond the boundary pixels with the boundary points to establish a one-to-one pairing relationship between image features and point cloud features.

[0028] S34. Concatenate and encode each pair of image features and point cloud features according to the field order to form a fusion feature unit. The fusion feature unit includes image coordinates, edge direction, spatial coordinates and normal vector direction.

[0029] S35. Combine the fused feature units in sequence to generate a fused representation vector.

[0030] Optionally, S4 specifically includes:

[0031] S41. Extract the image coordinate information of all boundary pixels from the fusion representation vector, determine the minimum and maximum values ​​of the horizontal and vertical axes in the image coordinates, and calculate the projection width and projection height in the image based on the coordinate difference. The image projection width is the difference between the maximum and minimum values ​​of the horizontal coordinates of the boundary pixels, and the image projection height is the difference between the maximum and minimum values ​​of the vertical coordinates.

[0032] S42. Extract the three-dimensional spatial coordinate information of all boundary points from the fused representation vector, calculate the minimum and maximum values ​​of the three axes in the spatial coordinates, and calculate the actual width and height of the object in space based on the difference between the horizontal and vertical coordinates. The spatial width is the difference between the maximum and minimum values ​​of the horizontal coordinates of the boundary points, and the spatial height is the difference between the maximum and minimum values ​​of the vertical coordinates.

[0033] S43. Establish an index correspondence between the image projection width and image projection height and the corresponding spatial width and spatial height, and record the matching information of each pair of image projection dimensions and spatial dimensions;

[0034] S44. Take the image projection width, image projection height, spatial width, and spatial height as elements of the size-distance joint representation, organize them into a set of structured data in a fixed field order, and generate the size-distance joint representation.

[0035] Optionally, S5 specifically includes:

[0036] S51. Extract the image projection width, image projection height, spatial width, and spatial height from the size-distance joint representation into four numerical fields, and perform normalization processing on each field to generate a standardized size feature group.

[0037] S52. Treat each field in the standardized size feature group as a graph node. Each graph node contains three types of attributes: field label, data type, and normalized value.

[0038] S53. Establish edge connection relationships according to the physical association order of the size field in the structural expression. The edge connection between graph nodes indicates that there is a geometric dependency relationship between the fields. The edge connection relationship is a directed edge, and the direction is determined by the structural combination rules.

[0039] S54. Construct a graph structure, which consists of a set of graph nodes and a set of edge connections, to represent the configuration structure between dimensions;

[0040] S55. Define the geometric configuration syntax. The geometric configuration syntax includes node type classification rules, edge connection combination methods, and structural topology restrictions. The classification rules divide graph nodes into length components, height components, and scale components. The edge connection combination methods specify the node connection order. The structural topology restrictions limit the combination form to a linear chain structure or a branching hierarchical structure.

[0041] S56. Input the graph structure and geometric configuration syntax into the Bayesian program learning model. The Bayesian program learning model derives and generates paths from the program space by analyzing the node values, connection methods and grammatical constraints in the graph structure, and determines the most likely program path based on the maximum a posteriori probability principle.

[0042] S57. Construct a three-dimensional structure program based on the generation path. The three-dimensional structure program consists of a node sequence and a connection order. Each node corresponds to a structural unit with specific size attributes, and each connection relationship represents the way structural units are combined in space.

[0043] Optionally, S6 specifically includes:

[0044] S61. Perform a graph structure unfolding operation on the 3D structure program to extract the graph node set and the edge connection set. The graph node set contains local shape units representing the spatial contour. Each graph node contains a size field, spatial coordinates and geometric configuration identifier. The edge connection set contains the connection order and combination method between graph nodes.

[0045] S62. Construct a structural feature description sequence from the set of graph nodes and the set of edge connections. Each element in the structural feature description sequence represents the size field, coordinate position and geometric configuration identifier of the graph node, as well as the starting node number, target node number and connection method number of the edge connection.

[0046] S63. Read the collection of structure programs from the structure program library, perform graph structure unfolding operation on each structure program, and generate a structure feature comparison sequence in the same format as the 3D structure program.

[0047] S64. Perform graph node-level and edge connection-level matching processing on the structural feature description sequence of the 3D structure program and each structural feature comparison sequence in the structure program library. Graph node-level matching is based on the size field, spatial coordinates and geometric configuration identifier, and edge connection-level matching is based on the connection order and connection method number to generate a structural similarity score.

[0048] S65. Sort the structural programs in the structural program library according to the structural similarity score, and select structural programs with a structural similarity score greater than or equal to a preset threshold to form a candidate structural program set.

[0049] S66. Output the identification information and graph structure representation of each structural program in the candidate structural program set as the candidate recognition result.

[0050] Optionally, S7 specifically includes:

[0051] S71. Based on the three-dimensional structure program corresponding to the candidate recognition results, extract the set of nodes and connection relationships contained in the graph structure, and construct the program state sequence according to the topological order of the graph nodes. The program state sequence contains the spatial position, contour structure and size information of each node.

[0052] S72. Input the program state sequence into the Bayesian program learning model, and on the basis of the program state sequence, combine the spatial scale change in the size-distance joint representation to construct a dynamic contour evolution input structure.

[0053] S73. In the Bayesian program learning model, based on graph structure and geometric configuration syntax, the contour deformation process of the program state sequence under continuous spatial scale is inferred, and the state evolution path representing the continuous change of the contour is generated.

[0054] S74. Update the spatial position and structural parameters of each graph node in the program state sequence according to the state evolution path to obtain the biomimetic evolution result;

[0055] S75. The biomimetic evolution results are used as the final recognition results and input into the robot control system for path planning and behavior scheduling.

[0056] The beneficial effects of this invention are:

[0057] This invention integrates multi-source perception data from monocular vision and LiDAR, utilizing time synchronization and coordinate registration techniques to jointly encode the contour features of image data and the spatial structure information of point cloud data. This forms a fused representation vector that combines shape and spatial depth features. Based on the size-distance joint representation, a Bayesian program learning model incorporating graph structure and geometric configuration syntax is constructed to generate 3D structure programs. This design effectively overcomes the limitations of single sensors in terms of accuracy, spatial information acquisition, and contour reconstruction, enabling patrol robots to acquire more complete and accurate target structure information in complex environments.

[0058] Building upon this foundation, this invention matches the generated 3D structure program with a structure program library to filter candidate recognition results, and combines dynamic contour reasoning technology to generate biomimetic evolution results, achieving continuous tracking and dynamic recognition of target state changes. This method improves the stability and adaptability of structure reconstruction while ensuring recognition accuracy, enabling patrol robots to adjust their decisions in real time according to environmental changes during path planning and behavior scheduling, thereby significantly improving task execution efficiency and intelligence in complex scenarios. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Fig. 1 This is an overall flowchart of a perception method for an intelligent patrol robot based on monocular vision and lidar proposed in this invention.

[0061] Fig. 2 This is a schematic diagram of the image and point cloud data fusion structure of an intelligent patrol robot perception method based on monocular vision and lidar proposed in this invention.

[0062] Fig. 3 This is a flowchart illustrating the dynamic contour reasoning process of a perception method for an intelligent patrol robot based on monocular vision and lidar proposed in this invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0064] refer to Figs. 1-3 A perception method for intelligent patrol robots based on monocular vision and lidar includes the following steps:

[0065] S1. Acquire monocular image data and lidar point cloud data, perform time synchronization and coordinate registration, generate aligned images and point cloud data, and build a structured program library.

[0066] S2. Extract contours and analyze textures from image data to generate image contour feature maps. Filter and complete point cloud data to generate point cloud contour models.

[0067] S3. Jointly encode the image contour feature map and the point cloud contour model to generate a fusion representation vector, which includes shape features and spatial depth features.

[0068] S4. Calculate the image projection size and spatial distance based on the fused representation vector, and generate a joint size-distance representation;

[0069] S5. The size-distance joint representation input is introduced into the Bayesian program learning model of graph structure and geometric configuration syntax to generate a three-dimensional structure program, which is used to represent the spatial contour structure of the target.

[0070] S6. Match the 3D structure program with the structure programs in the structure program library to generate candidate recognition results;

[0071] S7. Construct a program state sequence based on the candidate recognition results, input it into the Bayesian program learning model to perform dynamic contour reasoning, and generate biomimetic evolution results;

[0072] S8. Output the biomimetic evolution result as the final recognition result and input it into the robot control system for path planning and behavior scheduling.

[0073] This invention constructs an aligned image-point cloud dataset by acquiring monocular images and LiDAR point cloud data, combined with time synchronization and coordinate registration. Based on this, a structured program library is introduced to provide a program reference foundation for subsequent perception and recognition processes. This approach not only balances the detailed representation of images with the spatial accuracy of point clouds, but also provides a structural paradigm for the morphological modeling and perception of target objects through an integrated structured library mechanism. This helps enhance the robot's ability to fully perceive objects in complex scenes, building a unified chain from perception to structural understanding.

[0074] In this embodiment, the structure program library includes a three-dimensional structure program obtained by performing program generation operations on the size-distance joint representation of historical target objects through a Bayesian program learning model. The three-dimensional structure program is represented by graph structure and geometric configuration syntax and is used to represent the spatial contour structure of objects.

[0075] This invention constructs an image contour feature map by performing grayscale enhancement, edge extraction, and texture saliency analysis on image data. Simultaneously, it performs voxel filtering and density completion operations on point cloud data to generate a point cloud contour model, achieving joint modeling of boundary structures on two-dimensional images and three-dimensional point clouds. This design fully utilizes the complementary characteristics of image texture details and point cloud geometry, laying a high-quality foundation for subsequent feature fusion and effectively improving the clarity and completeness of the target object contour representation.

[0076] In this embodiment, S2 specifically includes:

[0077] S21. Perform grayscale conversion and high-pass filtering on the image data to enhance the contrast of boundaries and textures in the image and generate an image gradient map.

[0078] S22. Calculate the edge intensity and direction information of each pixel based on the image gradient map, use the edge detection algorithm to extract the boundary pixel contour, and construct the initial image contour map;

[0079] S23. In the process of image edge detection, local gray-level distribution pattern analysis is performed on the neighborhood of each pixel to enhance the response of textured areas in boundary detection and generate image contour feature maps.

[0080] S24. Divide the point cloud data into a three-dimensional spatial grid and perform voxel filtering to retain the representative points of each grid cell, remove isolated noise points, and form a set of filtered points.

[0081] S25. Perform density completion processing on the filtered point set, infer the point positions of the missing regions based on the spatial distribution relationship of the known points, and insert them to fill the missing regions, generating a complete point set;

[0082] S26. Based on the changes in the spatial position and normal vector direction of each point in the completion point set, extract the boundary change point set, construct the point cloud contour structure through the boundary connection algorithm, and output the point cloud contour model.

[0083] This invention establishes a one-to-one pairing relationship between image boundary pixels and point cloud boundary points by combining image contour feature maps and point cloud contour models. It constructs fused feature units containing image coordinates, edge directions, spatial positions, and normal vector directions, and sequentially combines them to form a fused representation vector. This mechanism achieves a deep binding of visual and spatial information, enabling each fused feature to possess multidimensional expressive capabilities, providing a multimodal foundation for subsequent scale calculations and configuration modeling of the target structure.

[0084] In this embodiment, S3 specifically includes:

[0085] S31. Extract the image coordinate information and edge direction information of the boundary pixels from the image contour feature map;

[0086] S32. Extract the spatial coordinate information and normal vector direction information of the boundary points from the point cloud contour model;

[0087] S33. Based on the registration relationship between image coordinates and spatial coordinates, spatially correspond the boundary pixels with the boundary points to establish a one-to-one pairing relationship between image features and point cloud features.

[0088] S34. Concatenate and encode each pair of image features and point cloud features according to the field order to form a fusion feature unit. The fusion feature unit includes image coordinates, edge direction, spatial coordinates and normal vector direction.

[0089] S35. Combine the fused feature units in sequence to generate a fused representation vector.

[0090] This invention extracts boundary feature coordinates by fusing representation vectors, calculates the image projection size and 3D spatial size separately, and constructs an index mapping between the two, ultimately generating a size-distance joint representation that includes both projected width and height and actual width and height. This structure unifies the expression of 2D image scale and 3D spatial scale, preserving observational information at image resolution while introducing the geometric scale of real space, which helps the model accurately infer spatial scale changes based on image variations.

[0091] In this embodiment, S4 specifically includes:

[0092] S41. Extract the image coordinate information of all boundary pixels from the fusion representation vector, determine the minimum and maximum values ​​of the horizontal and vertical axes in the image coordinates, and calculate the projection width and projection height in the image based on the coordinate difference. The image projection width is the difference between the maximum and minimum values ​​of the horizontal coordinates of the boundary pixels, and the image projection height is the difference between the maximum and minimum values ​​of the vertical coordinates.

[0093] S42. Extract the three-dimensional spatial coordinate information of all boundary points from the fused representation vector, calculate the minimum and maximum values ​​of the three axes in the spatial coordinates, and calculate the actual width and height of the object in space based on the difference between the horizontal and vertical coordinates. The spatial width is the difference between the maximum and minimum values ​​of the horizontal coordinates of the boundary points, and the spatial height is the difference between the maximum and minimum values ​​of the vertical coordinates.

[0094] S43. Establish an index correspondence between the image projection width and image projection height and the corresponding spatial width and spatial height, and record the matching information of each pair of image projection dimensions and spatial dimensions;

[0095] S44. Take the image projection width, image projection height, spatial width, and spatial height as elements of the size-distance joint representation, organize them into a set of structured data in a fixed field order, and generate the size-distance joint representation.

[0096] This invention normalizes the image projection width and height with their corresponding spatial width and height to construct a graph structure representation of connected nodes and edges. It then introduces a geometric configuration grammar to form a set of constraint rules that can be used for structural modeling. Finally, a Bayesian program learns a model to generate a 3D structure program. This program comprehensively expresses the dimensional relationships and spatial combination methods of object structures, providing grammatical support and a logical foundation for object structure recognition and biomimetic reasoning.

[0097] In this embodiment, S5 specifically includes:

[0098] S51. Extract the image projection width, image projection height, spatial width, and spatial height from the size-distance joint representation into four numerical fields, and perform normalization processing on each field to generate a standardized size feature group.

[0099] S52. Treat each field in the standardized size feature group as a graph node. Each graph node contains three types of attributes: field label, data type, and normalized value.

[0100] S53. Establish edge connection relationships according to the physical association order of the size field in the structural expression. The edge connection between graph nodes indicates that there is a geometric dependency relationship between the fields. The edge connection relationship is a directed edge, and the direction is determined by the structural combination rules.

[0101] S54. Construct a graph structure, which consists of a set of graph nodes and a set of edge connections, to represent the configuration structure between dimensions;

[0102] S55. Define the geometric configuration syntax. The geometric configuration syntax includes node type classification rules, edge connection combination methods, and structural topology restrictions. The classification rules divide graph nodes into length components, height components, and scale components. The edge connection combination methods specify the node connection order. The structural topology restrictions limit the combination form to a linear chain structure or a branching hierarchical structure.

[0103] S56. Input the graph structure and geometric configuration syntax into the Bayesian program learning model. The Bayesian program learning model derives and generates paths from the program space by analyzing the node values, connection methods and grammatical constraints in the graph structure, and determines the most likely program path based on the maximum a posteriori probability principle.

[0104] S57. Construct a three-dimensional structure program based on the generation path. The three-dimensional structure program consists of a node sequence and a connection order. Each node corresponds to a structural unit with specific size attributes, and each connection relationship represents the way structural units are combined in space.

[0105] This invention compares the 3D structure program to be identified with structure programs in a structure program library at the graph structure level, performing field matching at the graph node and edge connection levels, and filtering candidate structure programs based on structural similarity scores, outputting candidate identification results. This process uses a structured approach to express and compare the spatial morphology of the target, improving the consistency and traceability of the identification process, while also ensuring that the identification results have a clear program source and verifiability.

[0106] In this embodiment, S6 specifically includes:

[0107] S61. Perform a graph structure unfolding operation on the 3D structure program to extract the graph node set and the edge connection set. The graph node set contains local shape units representing the spatial contour. Each graph node contains a size field, spatial coordinates and geometric configuration identifier. The edge connection set contains the connection order and combination method between graph nodes.

[0108] S62. Construct a structural feature description sequence from the set of graph nodes and the set of edge connections. Each element in the structural feature description sequence represents the size field, coordinate position and geometric configuration identifier of the graph node, as well as the starting node number, target node number and connection method number of the edge connection.

[0109] S63. Read the collection of structure programs from the structure program library, perform graph structure unfolding operation on each structure program, and generate a structure feature comparison sequence in the same format as the 3D structure program.

[0110] S64. Perform graph node-level and edge connection-level matching processing on the structural feature description sequence of the 3D structure program and each structural feature comparison sequence in the structure program library. Graph node-level matching is based on the size field, spatial coordinates and geometric configuration identifier, and edge connection-level matching is based on the connection order and connection method number to generate a structural similarity score.

[0111] S65. Sort the structural programs in the structural program library according to the structural similarity score, and select structural programs with a structural similarity score greater than or equal to a preset threshold to form a candidate structural program set.

[0112] S66. Output the identification information and graph structure representation of each structural program in the candidate structural program set as the candidate recognition result.

[0113] This invention generates a program state sequence based on candidate recognition results, combines it with a size-distance joint representation input to a Bayesian program learning model, infers the contour evolution path at continuous scales, dynamically updates the spatial position and structural parameters of graph nodes, and generates biomimetic evolution results. This mechanism achieves predictive representation of structural changes of target objects in real-world scenes, balancing the model's generative and continuous evolution capabilities, and provides a reasoning basis for robots to adapt their morphology in dynamic environments.

[0114] In this embodiment, S7 specifically includes:

[0115] S71. Based on the three-dimensional structure program corresponding to the candidate recognition results, extract the set of nodes and connection relationships contained in the graph structure, and construct the program state sequence according to the topological order of the graph nodes. The program state sequence contains the spatial position, contour structure and size information of each node.

[0116] S72. Input the program state sequence into the Bayesian program learning model, and on the basis of the program state sequence, combine the spatial scale change in the size-distance joint representation to construct a dynamic contour evolution input structure.

[0117] S73. In the Bayesian program learning model, based on graph structure and geometric configuration syntax, the contour deformation process of the program state sequence under continuous spatial scale is inferred, and the state evolution path representing the continuous change of the contour is generated.

[0118] S74. Update the spatial position and structural parameters of each graph node in the program state sequence according to the state evolution path to obtain the biomimetic evolution result;

[0119] S75. The biomimetic evolution results are used as the final recognition results and input into the robot control system for path planning and behavior scheduling.

[0120] This invention uses biomimetic evolution results as the final recognition result and inputs them into the robot control system for path planning and behavior scheduling, realizing closed-loop processing from multimodal perception to intelligent control. This mechanism establishes an execution link between structural recognition and control behavior, enabling the structural understanding results to directly drive the robot's behavior system to respond to the environment and execute tasks, achieving efficient implementation and practical application of structural perception in control strategies.

[0121] Example 1:

[0122] To verify the feasibility of this invention in practice, it was applied to an intelligent patrol robot platform equipped with a monocular camera and LiDAR, deployed in a typical park environment, for the perception and recognition of human figures, rodents, and carts. The experiment focused on examining the system's accuracy in target structure recognition and its environmental adaptability during actual patrol tasks.

[0123] This invention acquires monocular images and LiDAR point cloud data within each perception cycle of the robot. Alignment data is generated through time synchronization and coordinate registration to ensure a one-to-one spatial correspondence between the image and point cloud information. Image data undergoes gradient extraction and edge detection to generate an image contour feature map, while point cloud data undergoes voxel filtering and boundary extraction to generate a point cloud contour model. The system jointly encodes the image contour and point cloud contour into a fused representation vector, further calculating the image projection size and spatial distance to construct a size-distance joint representation. This representation is input into a Bayesian program learning model based on graph structure and geometric configuration grammar to generate a 3D structure program. This program is then matched and recognized against a structure program library, outputting a biomimetic evolution result as the final recognition result.

[0124] To verify the superior recognition capabilities of this invention, a comparative experiment was conducted with the traditional YOLOv5 image recognition algorithm. The test selected five target categories: adults, children, bicycles, tricycles, and strollers. Recognition tasks were performed in three typical park scenarios: sunny day, cloudy day, and nighttime. Each target category was tested five times in each scenario. Five metrics were statistically analyzed: recognition accuracy, false recognition rate, size error, distance error, and system response time.

[0125] Traditional YOLOv5 methods primarily rely on two-dimensional features of images for bounding box recognition, making it difficult to obtain true size information and often resulting in recognition errors when targets are of similar size or when the viewing angle changes. In contrast, this invention constructs a representation of the real physical structure by fusing monocular vision and laser point clouds, which improves the ability to model size and spatial relationships and maintains stable recognition performance even under low light and occlusion conditions.

[0126] Table 1 shows a comparison of key performance indicators for some representative targets in typical scenarios during the experiment:

[0127] Target category Scenario Evaluation index Invention method YOLOv5 method Adult Sunny day Recognition accuracy 97.6% 90.3% Adult Sunny day Mean distance error 0.10 m 0.34 m Bicycle Night Size recognition error 0.12 m 0.39 m Child Overcast day Misrecognition rate 2.8% 13.5% Stroller Night Matching structure consistency score 91.4% Not applicable Tricycle Sunny day Recognition stability score ( / 10) 9.2 6.5 All categories Average System response time 0.39 s 0.26 s

[0128] The experimental results show that the method of this invention achieves higher recognition accuracy and lower false recognition rate in all test scenarios, and maintains good stability even when the target has structural complexity or is occluded. Although the response time is slightly longer than that of the YOLOv5 method, its overall recognition performance is better than traditional image methods, making it suitable for deployment in patrol robot systems that require high-reliability perception capabilities.

[0129] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A monocular vision and lidar-based intelligent patrol robot perception method, characterized in that, The method comprises the following steps: S1, collecting monocular image data and laser radar point cloud data, performing time synchronization and coordinate registration to generate aligned image and point cloud data, and constructing a structure library; S2, performing contour extraction and texture analysis on the image data to generate an image contour feature map, and performing filtering and completion on the point cloud data to generate a point cloud contour model; S3, jointly encoding the image contour feature map and the point cloud contour model to generate a fusion representation vector, the fusion representation vector containing shape features and spatial depth features; S4, calculating image projection size and spatial distance based on the fusion representation vector to generate a size-distance joint representation; S5, inputting the size-distance joint representation into a Bayesian program learning model of graph structure and geometric configuration syntax to generate a three-dimensional structure program, the three-dimensional structure program being used to represent the spatial contour structure of the target; S6, matching the three-dimensional structure program with the structure programs in the structure library to generate a candidate recognition result; S7, constructing a program state sequence based on the candidate recognition result, inputting the Bayesian program learning model to perform dynamic contour reasoning, and generating a bionic evolution result; S8, outputting the bionic evolution result as the final recognition result and inputting the result into a robot control system for path planning and behavior scheduling. 2.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The structure library contains three-dimensional structure programs obtained by performing program generation operations on size-distance joint representations of historical target objects through a Bayesian program learning model, the three-dimensional structure programs being represented in graph structure and geometric configuration syntax and being used to represent the spatial contour structure of the objects. 3.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S2 specifically comprises: S21, performing gray scale conversion and high pass filtering on the image data to enhance the contrast of boundaries and textures in the image and generate an image gradient map; S22, calculating the edge intensity and direction information of each pixel based on the image gradient map, extracting the boundary pixel contour using an edge detection algorithm, and constructing an initial image contour map; S23, in the image edge detection process, performing local gray scale distribution pattern analysis on each pixel neighborhood to enhance the response degree of the texture significant area in the boundary detection, and generating an image contour feature map; S24, dividing the point cloud data into three-dimensional space grids and performing voxel filtering to retain the representative points of each grid unit and remove isolated noise points, forming a filtered point set; S25, performing density completion processing on the filtered point set, inferring the point positions of the missing area according to the spatial distribution relationship of the known points, and inserting to fill the missing area, to generate a completed point set; S26, extracting a boundary change point set based on the spatial position and normal vector direction change of each point in the completed point set, constructing a point cloud contour structure through a boundary connection algorithm, and outputting a point cloud contour model. 4.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S3 specifically comprises: S31, extracting the image coordinate information and edge direction information of the boundary pixels from the image contour feature map; S32, extracting the spatial coordinate information and normal vector direction information of the boundary points from the point cloud contour model; S33, according to the registration relationship between the image coordinates and the spatial coordinates, corresponding the boundary pixels and the boundary points in the spatial position, and establishing a one-to-one pairing relationship between the image features and the point cloud features; S34, each pair of image features and point cloud features are spliced and coded according to field order to form a fusion feature unit, and the fusion feature unit contains image coordinates, edge direction, spatial coordinates and normal vector direction; S35, the fusion feature units are combined in order to generate a fusion representation vector. 5.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S4 specifically comprises: S41, image coordinate information of all boundary pixels is extracted from the fusion representation vector, the minimum and maximum values of the horizontal and vertical axes in the image coordinates are determined, and the projection width and height in the image are calculated according to the coordinate difference value, the image projection width is the difference between the maximum and minimum values of the horizontal coordinates of the boundary pixels, and the image projection height is the difference between the maximum and minimum values of the vertical coordinates; S42, three-dimensional spatial coordinate information of all boundary points is extracted from the fusion representation vector, the minimum and maximum values of the three axes in the spatial coordinates are counted respectively, and the actual width and height of the object in space are calculated according to the coordinate difference value of the horizontal and vertical directions, the spatial width is the difference between the maximum and minimum values of the horizontal coordinates of the boundary points, and the spatial height is the difference between the maximum and minimum values of the vertical coordinates; S43, the image projection width, image projection height, corresponding spatial width and spatial height are indexed and corresponded, and the matching information of each pair of image projection size and spatial size is recorded; S44, the image projection width, image projection height, spatial width and spatial height are taken as elements of size-distance joint representation, and are organized into a group of structure data in a fixed field order to generate size-distance joint representation. 6.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S5 specifically comprises: S51, the image projection width, image projection height, spatial width and spatial height in the size-distance joint representation are extracted as four numerical fields respectively, and normalization processing is performed on each field to generate a standardized size feature group; S52, each field in the standardized size feature group is taken as a graph node, and each graph node contains three types of attributes of field label, data type and normalized value; S53, an edge connection relationship is established according to the physical association order of size fields in structure expression, the edge connection between graph nodes represents that there is a geometric dependency relationship between fields, and the edge connection relationship is a directed edge, and the direction is determined by structure combination rules; S54, a graph structure is constructed, which is composed of a set of graph nodes and a set of edge connections, and is used to represent the configuration structure between sizes; S55, a geometric configuration syntax is set, which includes node type classification rules, edge connection combination mode and structure topology restriction, the classification rules divide the graph nodes into length members, height members and proportion members, the edge connection combination mode specifies the node connection order, and the structure topology restriction limits the combination form to linear chain structure or branch hierarchical structure; S56, the graph structure and the geometric configuration syntax are input into a Bayesian program learning model, the Bayesian program learning model generates a path from the program space by analyzing the node values, connection mode and syntax constraint conditions in the graph structure, and determines the most possible program path according to the maximum a posteriori probability principle; S57, constructing a three-dimensional structure program according to the generated path, the three-dimensional structure program being composed of a node sequence and a connection sequence, each node corresponding to a structure unit with specific size attributes, and each connection relationship representing a combination manner of the structure units in space. 7.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S6 specifically includes: S61, performing a graph structure unfolding operation on the three-dimensional structure program to extract a graph node set and an edge connection set, wherein the graph node set contains local shape units representing spatial contours, each graph node contains a size field, a spatial coordinate, and a geometric configuration identifier, and the edge connection set contains a connection sequence and a combination manner between graph nodes; S62, constructing the graph node set and the edge connection set into a structure feature description sequence, each element in the structure feature description sequence representing a size field, a coordinate position, and a geometric configuration identifier of a graph node, and a start node number, a target node number, and a connection manner number of an edge connection; S63, reading a structure program set from a structure program library, and performing a graph structure unfolding operation on each structure program to generate a structure feature comparison sequence in the same format as the three-dimensional structure program; S64, performing graph node level and edge connection level matching processing on the structure feature description sequence of the three-dimensional structure program and each structure feature comparison sequence in the structure program library, the graph node level matching being based on the size field, the spatial coordinate, and the geometric configuration identifier, and the edge connection level matching being based on the connection sequence and the connection manner number, to generate a structure similarity score; S65, sorting the structure programs in the structure program library according to the structure similarity score, selecting structure programs with a structure similarity score greater than or equal to a preset threshold, and constructing a candidate structure program set; S66, outputting the identification information and the graph structure of each structure program in the candidate structure program set as a candidate recognition result. 8.The monocular vision and laser radar based intelligent patrol robot perception method of claim 1, wherein, The S7 specifically includes: S71, extracting a node set and a connection relationship contained in the graph structure according to the three-dimensional structure program corresponding to the candidate recognition result, and constructing a program state sequence according to the topological sequence of the graph nodes, the program state sequence containing spatial positions, contour structures, and size information of each node; S72, inputting the program state sequence into a Bayesian program learning model, constructing a dynamic contour evolution input structure based on the program state sequence and the spatial scale change in the size-distance joint representation; S73, in the Bayesian program learning model, inferring a contour deformation process of the program state sequence at a continuous spatial scale based on the graph structure and the geometric configuration grammar, to generate a state evolution path representing the continuous change of the contour; S74, updating the spatial positions and structure parameters of the graph nodes in the program state sequence according to the state evolution path to obtain a bionic evolution result; S75, inputting the bionic evolution result as the final recognition result into a robot control system for path planning and behavior scheduling.

Images