Bionic human eye target recognition system and method based on monocular vision and laser radar

Abstract

The application discloses a binocular vision and laser radar-based bionic human eye target recognition system and method, which comprises the following steps: S1, collecting and registering synchronous monocular images and laser radar point clouds; S2, extracting image contours and point cloud attributes to generate feature maps and attribute sets; S3, jointly encoding to generate size-distance representations; S4, constructing an improved Ginzburg-Landau phase field energy model based on the representations and attribute sets, obtaining potential function parameters and anisotropy tensors; S5, introducing SE(2) sub-Riemann geodesic direction continuity constraints into a phase field gradient term to adjust the strength; S6, aligning and optimizing evolution of adjacent frame phase fields in time by using optical flow; S7, extracting zero isograms to generate contours and output parameters; S8, inputting the parameters and size-distance representations into a three-dimensional structure program generation module for matching and screening; S9, performing consistency checking and sorting to output a recognition result. The application fuses visual and point cloud features, and improves the recognition accuracy and stability in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of computer vision and lidar fusion perception technology, and in particular to a bionic human eye target recognition system and method based on monocular vision and lidar. Background Technology

[0002] Among existing target recognition technologies, monocular vision has been widely used in fields such as autonomous driving, robot perception, and security monitoring due to its advantages of low equipment cost, simple structure, and ease of deployment. However, monocular vision relies on two-dimensional image information and cannot directly obtain the true spatial depth of the target. It is also susceptible to the effects of lighting, occlusion, and texture loss, leading to a significant decrease in recognition accuracy in complex environments. To address the problem of missing depth information, some studies have introduced lidar, which provides three-dimensional spatial coordinates and reflection intensity information through point cloud data, thereby improving the spatial localization capability of the target. However, traditional image and point cloud fusion methods often rely on simple feature stitching or projection matching, failing to fully utilize the complementary advantages between image contours and point cloud geometric characteristics, resulting in shortcomings in accurate target boundary localization and morphological detail depiction.

[0003] In the application of phase-field models, existing technologies often use the Ginzburg–Landau phase-field energy model for continuous representation and evolution calculation of target boundaries. However, the potential function parameters and diffusion tensors of traditional models are mostly fixed values ​​or set based on single-modal features, lacking an adaptive adjustment mechanism for multimodal data. Such fixed parameter settings are difficult to cope with significant changes in target size, spatial distance, point cloud sparsity, and texture quality in different scenes, easily leading to overly smoothed boundaries or loss of details. At the same time, when processing sequential frames, existing phase-field models usually only perform contour evolution based on spatial information, lacking temporal continuity constraints, which easily leads to problems such as target contour jitter and inconsistency in dynamic scenes.

[0004] In terms of target contour extraction and 3D structure generation, traditional methods mostly extract boundaries directly from segmentation results or generate 3D structures using point cloud surface reconstruction algorithms. They lack refined structural descriptions and matching mechanisms based on contour normals, curvature, and physical scale, making high-precision grammatical retrieval and filtering in structural databases difficult. Furthermore, existing 3D structure generation processes often lack a unified size-distance joint representation when handling targets with different viewpoints and scales, resulting in insufficient robustness to pose and scale changes. In summary, existing fusion recognition technologies based on monocular vision and LiDAR have limitations in feature fusion accuracy, phase field model adaptability, temporal consistency constraints, and 3D structured matching. There is an urgent need to propose a biomimetic human eye target recognition method capable of achieving high-precision, multi-scene adaptability in joint modeling of image contours and point cloud attributes, phase field evolution optimization, and 3D structure procedural generation.

[0005] Therefore, how to provide a bionic human eye target recognition system and method based on monocular vision and lidar is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a bionic human eye target recognition system and method based on monocular vision and lidar. This invention fully utilizes the complementary advantages of monocular image contour features and lidar point cloud attributes to construct a joint size-distance representation. It combines an improved Ginzburg–Landau phase field energy model, a direction continuity constraint based on SE(2) sub-Riemann geodesics, and an optical flow time alignment mechanism to achieve smooth and consistent evolution and accurate extraction of the target contour. At the same time, a three-dimensional structure program generation module is introduced to uniformly encode the normal, curvature, physical scale, and size-distance information to complete high-precision structured matching and recognition. It has the advantages of high recognition accuracy, adaptability to multiple scenarios, and strong robustness of structure matching.

[0007] The bionic human eye target recognition method based on monocular vision and lidar according to embodiments of the present invention includes the following steps: S1. Acquire monocular image frames with synchronized timestamps and lidar point cloud data, and complete the registration of the image and point cloud coordinate systems. S2. Extract image contour features and point cloud normals, curvature, reflectivity and local density to generate image contour feature map and point cloud attribute set; S3. Jointly encode the image contour feature map and the point cloud attribute set, and calculate the size-distance joint representation; S4. Based on the size-distance joint representation and the point cloud attribute set, generate the improved Ginzburg–Landau phase field energy model and obtain the potential function parameters and anisotropic tensor. S5. Introduce the direction continuity constraint based on SE(2) sub-Riemann geodesic in the phase field gradient term, and adjust the phase field intensity by combining the potential function parameter and the anisotropic tensor to ensure smooth and consistent profile. S6. Use optical flow to time-align the phase fields of adjacent frames, optimize the phase field evolution based on potential function parameters and anisotropic tensors, and obtain the phase field of the current frame. S7. Extract zero contour lines from the current frame phase field to generate the target contour, and output the contour normal, curvature and physical scale parameters; S8. Input the contour normal, curvature, and physical scale parameters along with the size-distance joint representation into the 3D structure program generation module for matching and filtering to obtain candidate recognition results; S9. Perform consistency verification and sorting on the candidate recognition results, and output the final recognition result.

[0008] Optionally, the improved Ginzburg–Landau phase field energy model: This includes generating learnable double-well potential function parameters based on size-distance joint representation, image texture quality, point cloud curvature, and local density; This includes an anisotropic tensor that generates a diffusion tensor based on the size-distance joint representation and the point cloud attribute set; A directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced into the phase field gradient term to ensure that the contour growth direction is consistent; By incorporating contour information from adjacent frames, a temporal consistency term is introduced to ensure a smooth transition of the target in the time series.

[0009] Optionally, the process of extracting image contour features and point cloud normals, curvature, reflectivity, and local density, and generating an image contour feature map and a set of point cloud attributes, includes: Edge detection is performed on monocular image frames to obtain the gradient magnitude and gradient direction of each pixel. Redundant edge responses are removed by non-maximum suppression, and an ordered sequence of contour points is formed by tracking the pixel connection relationship. The sequence of contour points is fitted into a set of two-dimensional continuous curves to construct an image contour feature map. A fixed-radius neighborhood search is performed on the lidar point cloud data. A three-dimensional covariance matrix is ​​constructed in each neighborhood. The eigenvector corresponding to the minimum eigenvalue is obtained by using the eigenvalue decomposition method as the unit normal vector. Calculating curvature using the smallest eigenvalue and the sum of eigenvalues ,in It is the smallest eigenvalue. These are the eigenvalues ​​along the x, y, and z directions, respectively; Reflectance is extracted from point cloud intensity information, and the point cloud is divided into fixed voxel units. The number of points in each voxel is counted and divided by the voxel volume to obtain the local density. The normal, curvature, reflectivity, and local density are combined in the order of the point cloud index to generate a set of point cloud attributes.

[0010] Optionally, the calculation process for the joint encoding of the image contour feature map and the point cloud attribute set, and the joint representation of size-distance, includes: The two-dimensional coordinates of each pixel in the image contour feature map are mapped to the three-dimensional coordinate position in the point cloud coordinate system. The point cloud normal, curvature, reflectivity and local density are associated at the mapped position to form spatial attribute matching pairs, which provide a spatial consistency basis for feature fusion. A multi-channel coding tensor is constructed using spatial attribute matching pairs. The first two dimensions of the multi-channel coding tensor correspond to the row and column positions of the image, and the third dimension includes the normal, curvature, reflectivity and local density channels, ensuring that each pixel has feature descriptions of both the image and the point cloud. A spatial scale channel is added to the multi-channel coding tensor. The spatial scale channel is calculated from the width and height of the bounding box of the contour to which the pixel belongs, providing two-dimensional size information of the target on the image plane. At the same time, a depth distance channel is added. The depth distance channel is calculated from the spatial distance from the corresponding three-dimensional point to the origin of the point cloud coordinate system, providing depth information of the target in three-dimensional space. The spatial scale channel and depth distance channel are normalized and then spliced ​​with the normal, curvature, reflectivity and local density channels in the channel dimension to generate a composite feature set. The composite feature set contains two-dimensional size, three-dimensional depth and point cloud attribute information. Perform a linear mapping operation along the channel dimension on the composite feature set to obtain a fixed-length size-distance joint representation vector.

[0011] Optionally, the generation process of the improved Ginzburg–Landau phase field energy model includes: Based on the size-distance joint representation, texture gradient information in the image contour feature map, point cloud normal, curvature, reflectivity and local density, an input feature vector is constructed. The input is a learnable double-well potential function parameter generation unit. The double-well potential function consists of two stable potential wells and one unstable potential well. The depth and position of the stable potential wells are calculated by weighting the aspect ratio of the target two-dimensional contour, spatial distance, point cloud normal consistency, curvature distribution and local density. The position of the unstable potential wells is calculated by the contour gradient magnitude and point cloud sparsity. The potential function shape coefficient is determined by the normalized difference between the size information and the depth information. An anisotropic diffusion tensor is generated based on the size-distance joint representation and the point cloud attribute set. Principal component analysis is performed on the normal distribution of the point cloud to obtain the principal direction unit vector n1 and the orthogonal secondary direction unit vector n2. The principal direction diffusion coefficient D1 is calculated from the normal uniformity, and the secondary direction diffusion coefficient D2 is calculated from the curvature gradient intensity. The diffusion tensor is represented as: ; Where n1 is the direction of the principal component of the point cloud normal, n2 is the direction of curvature change orthogonal to n1, D1 is the diffusion coefficient of the principal direction, and D2 is the diffusion coefficient of the secondary direction. In the gradient term of the phase field energy model, a directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced. The SE(2) space consists of translational components x, y and rotational components θ. The sub-Riemann metric restricts the propagation path to remain smooth and continuous in the x, y and θ dimensions. The directional continuity constraint is achieved by calculating the shortest path length of the geodesics of adjacent positions in the θ dimension and introducing a penalty factor in the energy term. The learnable double-well potential function parameters and anisotropic tensor inputs are combined with the phase field energy term and gradient term according to the model structure to form an improved Ginzburg–Landau phase field energy model, which is then passed to the phase field evolution stage.

[0012] Optionally, the process of using optical flow to time-align the phase fields of adjacent frames includes: The current monocular image frame and the previous monocular image frame are input into the optical flow calculation unit. The optical flow calculation unit is used to calculate the displacement vector field of each pixel on the image plane based on the dense optical flow algorithm. The displacement vector field contains a horizontal component u and a vertical component v, which are used to represent the displacement of the pixel between two time points. The phase field distribution of the previous frame is mapped at the pixel level using the displacement vector field, so that the phase field of the previous frame is aligned with the phase field of the current frame in spatial coordinates, thus obtaining the temporally aligned phase field data of adjacent frames. The phase field data of adjacent frames aligned with time and the phase field data of the current frame are input into the phase field optimization unit. The phase field optimization unit is used to adjust the phase field intensity by combining the potential function parameters and the anisotropic tensor, and optimize the phase field evolution process based on the gradient direction constraint to generate the optimized phase field distribution of the current frame.

[0013] Optionally, the process of extracting zero contour lines from the current frame phase field to generate the target contour includes: Locate the set of points in the current frame phase field distribution where the phase field value is equal to zero, perform connectivity analysis according to the gradient direction of adjacent pixels, and form a continuous zero contour path. The zero contour path is smoothed to eliminate isolated noise points, and curvature constraints are used to maintain the geometric continuity of the contour, resulting in a structurally complete target contour curve. Based on the corresponding points of the target contour curve in three-dimensional space, the contour normal is calculated in conjunction with the registered point cloud coordinate system. The contour normal is a unit vector perpendicular to the local tangent of the contour. After the contour normal is calculated, the local curvature is fitted according to the discrete point distribution of the contour in three-dimensional space. The local curvature is obtained by measuring the rate of change of the normal in the neighborhood of the contour. By combining the spatial scale information of the point cloud with the pixel spacing conversion ratio of the contour in the image coordinate system, the physical scale parameter is calculated. The physical scale parameter is used to characterize the actual size of the target in real space.

[0014] Optionally, the process of inputting the contour normal, curvature, and physical scale parameters along with the size-distance joint representation into the 3D structure generation module and performing matching and filtering includes: The contour normal, curvature, and physical scale parameters are combined with the size-distance joint representation to generate a structural description vector. The structural description vector contains a combination of image contour features and point cloud spatial features in a unified numerical space. The 3D structure program generation module is used to generate 3D structure program instances that conform to the structure program syntax based on the structure description vector. The 3D structure program generation module includes a structure syntax parsing unit, a geometric relation construction unit, and a spatial dimension encoding unit. The structure syntax parsing unit parses the semantic components in the structure description vector into structured syntax nodes. The geometric relation construction unit converts curvature and normal information into geometric topological relationships between nodes. The spatial dimension encoding unit encodes physical scale parameters into spatial metrics associated with nodes. The generated 3D structure program instances are indexed and retrieved in the structure program library to select a set of candidate structure programs whose structure syntax, geometric relations and spatial dimensions all match the 3D structure program instances. For each structural program in the candidate structural program set, the similarity with the 3D structural program instance is calculated based on the matching score function. The similarity score comprehensively considers normal consistency, curvature distribution similarity and physical scale difference. Candidate recognition results are selected in descending order of similarity scores and output to the consistency verification and sorting stage.

[0015] A bionic human eye target recognition system based on monocular vision and lidar according to an embodiment of the present invention includes: The image and point cloud acquisition module is used to acquire monocular image frames with synchronized timestamps and lidar point cloud data, and to perform coordinate system registration. The feature extraction module is used to extract image contour features and point cloud normals, curvature, reflectivity and local density, and generate image contour feature map and point cloud attribute set; The joint encoding module is used to jointly encode the image contour feature map and the point cloud attribute set to generate a dimension-distance joint representation that includes two-dimensional size and three-dimensional depth; The phase-field energy model generation module is used to generate an improved Ginzburg–Landau phase-field energy model based on the size-distance joint representation and the point cloud attribute set, and outputs the potential function parameters and anisotropic tensor. The phase field evolution optimization module is used to introduce the directional continuity constraint based on SE(2) sub-Riemann geodesy, combine the potential function parameters and anisotropic tensors to adjust the phase field intensity, and realize the phase field time alignment and optimization of adjacent frames through the optical flow calculation unit and the phase field optimization unit. The contour extraction module is used to extract zero contour lines in the current frame phase field to form the target contour, and to calculate the contour normal, curvature and physical scale parameters. The 3D structure program generation module is used to input the contour normal, curvature and physical scale parameters and size-distance joint representation into the structure syntax parsing unit, geometric relationship construction unit and spatial size encoding unit to generate 3D structure program instances, and to match and filter candidate structure programs in the structure program library and output candidate recognition results. The consistency verification and sorting module is used to verify and sort the candidate recognition results and output the final recognition result.

[0016] The beneficial effects of this invention are: This invention establishes a size-distance joint representation that integrates two-dimensional image contour features and three-dimensional point cloud spatial attributes by fusing monocular vision and lidar data. This representation can accurately characterize the geometric features of a target under unified encoding of spatial scale and depth information. Based on this, an improved Ginzburg–Landau phase field energy model is constructed, combining learnable double-well potential function parameters with anisotropic tensors and introducing directional continuity constraints based on SE(2) sub-Riemann geodesics. This ensures that the target contour maintains directional consistency and smoothness during evolution. This design effectively solves the problem of unstable contour extraction under complex backgrounds and noise interference, achieving refined capture of target morphology.

[0017] This invention achieves temporal alignment of phase fields between adjacent frames through optical flow calculation and maintains the geometric continuity and stability of the contour in dynamic scenes by combining a phase field optimization mechanism, thus ensuring the temporal consistency of the target. A high-precision target contour is obtained using a zero-contour extraction method, and the normal, curvature, and physical scale parameters are input into a 3D structure program generation module to achieve structured and matchable 3D program instance generation. This method maintains high recognition accuracy and stability under pose changes, multiple background interferences, and different scene conditions, significantly improving the system's robustness in structure matching, spatial modeling, and multi-scene adaptation. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the bionic human eye target recognition method based on monocular vision and lidar proposed in this invention; Figure 2 This is a schematic diagram illustrating the construction of the improved Ginzburg–Landau phase field energy model for the bionic human eye target recognition method based on monocular vision and lidar proposed in this invention. Figure 3 This is a flowchart of the three-dimensional structure generation and matching screening process for the bionic human eye target recognition method based on monocular vision and lidar proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1-3 A biomimetic human eye target recognition method based on monocular vision and lidar includes the following steps: S1. Acquire monocular image frames with synchronized timestamps and lidar point cloud data, and complete the registration of the image and point cloud coordinate systems. S2. Extract image contour features and point cloud normals, curvature, reflectivity and local density to generate image contour feature map and point cloud attribute set; S3. Jointly encode the image contour feature map and the point cloud attribute set, and calculate the size-distance joint representation; S4. Based on the size-distance joint representation and the point cloud attribute set, generate the improved Ginzburg–Landau phase field energy model and obtain the potential function parameters and anisotropic tensor. S5. Introduce the direction continuity constraint based on SE(2) sub-Riemann geodesic in the phase field gradient term, and adjust the phase field intensity by combining the potential function parameter and the anisotropic tensor to ensure smooth and consistent profile. S6. Use optical flow to time-align the phase fields of adjacent frames, optimize the phase field evolution based on potential function parameters and anisotropic tensors, and obtain the phase field of the current frame. S7. Extract zero contour lines from the current frame phase field to generate the target contour, and output the contour normal, curvature and physical scale parameters; S8. Input the contour normal, curvature, and physical scale parameters along with the size-distance joint representation into the 3D structure program generation module for matching and filtering to obtain candidate recognition results; S9. Perform consistency verification and sorting on the candidate recognition results, and output the final recognition result.

[0021] This invention effectively solves the problems of insufficient depth information or missing texture features in target recognition caused by single sensors by fusing monocular visual images with synchronized timestamps and LiDAR point cloud data, and achieving precise alignment of two-dimensional image features with three-dimensional spatial features after coordinate system registration. By combining image contour features with point cloud attributes, the detection accuracy of target boundaries can be improved in complex scenes, especially maintaining stable recognition performance under conditions of illumination changes, partial occlusion, or strong background interference. In addition, the method introduces directional continuity and temporal consistency constraints into the phase field model, making the contour evolution smoother and reducing the risk of target deformation and breakage, thereby significantly improving the stability and robustness of target recognition.

[0022] In this embodiment, the improved Ginzburg–Landau phase field energy model is described as follows: This includes generating learnable double-well potential function parameters based on size-distance joint representation, image texture quality, point cloud curvature, and local density; This includes an anisotropic tensor that generates a diffusion tensor based on the size-distance joint representation and the point cloud attribute set; A directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced into the phase field gradient term to ensure that the contour growth direction is consistent; By incorporating contour information from adjacent frames, a temporal consistency term is introduced to ensure a smooth transition of the target in the time series.

[0023] This invention introduces a learnable double-well potential function and anisotropic diffusion tensor into the Ginzburg–Landau phase-field energy model. This not only adaptively adjusts the potential function shape based on size-distance characteristics but also optimizes the energy distribution by combining image texture quality with point cloud curvature and density attributes, making the phase-field model closer to the actual target structure. Through the directional continuity constraint of the SE(2) sub-Riemann geodesic, the growth direction of the contour is ensured to be consistent in translation and rotation space, effectively suppressing contour jitter and local offset. Combined with a temporal consistency term, the smooth transition of the target in multi-frame sequences is further guaranteed, adapting to dynamic scenes and high-speed moving targets, thus improving the overall continuity and accuracy of recognition.

[0024] In this embodiment, the process of extracting image contour features and point cloud normals, curvature, reflectivity, and local density, and generating an image contour feature map and a set of point cloud attributes, includes: Edge detection is performed on monocular image frames to obtain the gradient magnitude and gradient direction of each pixel. Redundant edge responses are removed by non-maximum suppression, and an ordered sequence of contour points is formed by tracking the pixel connection relationship. The sequence of contour points is fitted into a set of two-dimensional continuous curves to construct an image contour feature map. A fixed-radius neighborhood search is performed on the lidar point cloud data. A three-dimensional covariance matrix is ​​constructed in each neighborhood. The eigenvector corresponding to the minimum eigenvalue is obtained by using the eigenvalue decomposition method as the unit normal vector. Calculating curvature using the smallest eigenvalue and the sum of eigenvalues ,in It is the smallest eigenvalue. These are the eigenvalues ​​along the x, y, and z directions, respectively; Reflectance is extracted from point cloud intensity information, and the point cloud is divided into fixed voxel units. The number of points in each voxel is counted and divided by the voxel volume to obtain the local density. The normal, curvature, reflectivity, and local density are combined in the order of the point cloud index to generate a set of point cloud attributes.

[0025] This invention employs edge detection and non-maximum suppression during image and point cloud feature extraction to ensure the clarity and continuity of image contours, and obtains accurate point cloud normals, curvature, and local density information through three-dimensional covariance matrix analysis. This method enables high-precision combination of contour features and spatial geometric attributes, improving the ability to distinguish fine structures. Simultaneously, by extracting reflectivity and local density, stable attribute descriptions can be maintained in environments with sparse point clouds or complex reflectivity characteristics, thus providing a more reliable data foundation for subsequent feature fusion and size-distance calculation, enhancing the robustness and adaptability of recognition.

[0026] In this embodiment, the calculation process of joint encoding of image contour feature map and point cloud attribute set and joint size-distance representation includes: The two-dimensional coordinates of each pixel in the image contour feature map are mapped to the three-dimensional coordinate position in the point cloud coordinate system. The point cloud normal, curvature, reflectivity and local density are associated at the mapped position to form spatial attribute matching pairs, which provide a spatial consistency basis for feature fusion. A multi-channel coding tensor is constructed using spatial attribute matching pairs. The first two dimensions of the multi-channel coding tensor correspond to the row and column positions of the image, and the third dimension includes the normal, curvature, reflectivity and local density channels, ensuring that each pixel has feature descriptions of both the image and the point cloud. A spatial scale channel is added to the multi-channel coding tensor. The spatial scale channel is calculated from the width and height of the bounding box of the contour to which the pixel belongs, providing two-dimensional size information of the target on the image plane. At the same time, a depth distance channel is added. The depth distance channel is calculated from the spatial distance from the corresponding three-dimensional point to the origin of the point cloud coordinate system, providing depth information of the target in three-dimensional space. The spatial scale channel and depth distance channel are normalized and then spliced ​​with the normal, curvature, reflectivity and local density channels in the channel dimension to generate a composite feature set. The composite feature set contains two-dimensional size, three-dimensional depth and point cloud attribute information. Perform a linear mapping operation along the channel dimension on the composite feature set to obtain a fixed-length size-distance joint representation vector.

[0027] This invention achieves deep fusion of two-dimensional image information and three-dimensional geometric information by mapping image contour feature maps to a point cloud coordinate system and spatially consistentally associating them with point cloud attributes. It utilizes a multi-channel encoded tensor to simultaneously carry normal, curvature, reflectivity, local density, and two-dimensional size and three-dimensional depth information, and generates a size-distance joint representation through normalization and linear mapping, effectively improving the completeness and discriminability of feature representation. This design maintains high matching accuracy under conditions of occlusion, illumination changes, and viewpoint changes, providing robust multi-dimensional feature support for subsequent phase field modeling and recognition.

[0028] In this embodiment, the generation process of the improved Ginzburg–Landau phase field energy model includes: Based on the size-distance joint representation, texture gradient information in the image contour feature map, point cloud normal, curvature, reflectivity and local density, an input feature vector is constructed. The input is a learnable double-well potential function parameter generation unit. The double-well potential function consists of two stable potential wells and one unstable potential well. The depth and position of the stable potential wells are calculated by weighting the aspect ratio of the target two-dimensional contour, spatial distance, point cloud normal consistency, curvature distribution and local density. The position of the unstable potential wells is calculated by the contour gradient magnitude and point cloud sparsity. The potential function shape coefficient is determined by the normalized difference between the size information and the depth information. An anisotropic diffusion tensor is generated based on the size-distance joint representation and the point cloud attribute set. Principal component analysis is performed on the normal distribution of the point cloud to obtain the principal direction unit vector n1 and the orthogonal secondary direction unit vector n2. The principal direction diffusion coefficient D1 is calculated from the normal uniformity, and the secondary direction diffusion coefficient D2 is calculated from the curvature gradient intensity. The diffusion tensor is represented as: ; Where n1 is the direction of the principal component of the point cloud normal, n2 is the direction of curvature change orthogonal to n1, D1 is the diffusion coefficient of the principal direction, and D2 is the diffusion coefficient of the secondary direction. In the gradient term of the phase field energy model, a directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced. The SE(2) space consists of translational components x, y and rotational components θ. The sub-Riemann metric restricts the propagation path to remain smooth and continuous in the x, y and θ dimensions. The directional continuity constraint is achieved by calculating the shortest path length of the geodesics of adjacent positions in the θ dimension and introducing a penalty factor in the energy term. The learnable double-well potential function parameters and anisotropic tensor inputs are combined with the phase field energy term and gradient term according to the model structure to form an improved Ginzburg–Landau phase field energy model, which is then passed to the phase field evolution stage.

[0029] In phase field energy modeling, this invention employs a learnable double-well potential function based on size-distance features and multi-source attributes, combined with geometric information such as normal consistency and curvature distribution, to dynamically adjust the position and depth of stable and unstable potential wells, thereby more accurately depicting the true shape of the target contour. The anisotropic diffusion tensor generated through principal component analysis ensures smooth contour growth in the principal direction and effectively suppresses noise interference in the secondary direction. The introduction of the SE(2) sub-Riemann geodesic constraint ensures the directional continuity of phase field evolution, making it particularly suitable for identifying complex surfaces and non-rigid targets, significantly improving the accuracy and stability of the modeling.

[0030] In this embodiment, the process of time-aligning adjacent frame phase fields using optical flow includes: The current monocular image frame and the previous monocular image frame are input into the optical flow calculation unit. The optical flow calculation unit is used to calculate the displacement vector field of each pixel on the image plane based on the dense optical flow algorithm. The displacement vector field contains a horizontal component u and a vertical component v, which are used to represent the displacement of the pixel between two time points. The phase field distribution of the previous frame is mapped at the pixel level using the displacement vector field, so that the phase field of the previous frame is aligned with the phase field of the current frame in spatial coordinates, thus obtaining the temporally aligned phase field data of adjacent frames. The phase field data of adjacent frames aligned with time and the phase field data of the current frame are input into the phase field optimization unit. The phase field optimization unit is used to adjust the phase field intensity by combining the potential function parameters and the anisotropic tensor, and optimize the phase field evolution process based on the gradient direction constraint to generate the optimized phase field distribution of the current frame.

[0031] This invention achieves pixel-level temporal alignment of the phase fields of adjacent frames through optical flow calculation, enabling the phase field evolution in dynamic scenes to accurately reflect the true motion trajectory of the target. Based on this, phase field optimization is performed by combining potential function parameters and anisotropic tensors, maintaining smooth and consistent contour morphology in the temporal dimension and avoiding contour distortion caused by motion blur or displacement accumulation. This spatiotemporal joint optimization strategy effectively enhances the robustness of the method in high-speed moving target recognition, reduces recognition errors caused by temporal inconsistencies, and improves dynamic tracking and recognition accuracy.

[0032] In this embodiment, the process of extracting zero contour lines from the current frame phase field to generate the target contour includes: Locate the set of points in the current frame phase field distribution where the phase field value is equal to zero, perform connectivity analysis according to the gradient direction of adjacent pixels, and form a continuous zero contour path. The zero contour path is smoothed to eliminate isolated noise points, and curvature constraints are used to maintain the geometric continuity of the contour, resulting in a structurally complete target contour curve. Based on the corresponding points of the target contour curve in three-dimensional space, the contour normal is calculated in conjunction with the registered point cloud coordinate system. The contour normal is a unit vector perpendicular to the local tangent of the contour. After the contour normal is calculated, the local curvature is fitted according to the discrete point distribution of the contour in three-dimensional space. The local curvature is obtained by measuring the rate of change of the normal in the neighborhood of the contour. By combining the spatial scale information of the point cloud with the pixel spacing conversion ratio of the contour in the image coordinate system, the physical scale parameter is calculated. The physical scale parameter is used to characterize the actual size of the target in real space.

[0033] In the zero-contour extraction stage, this invention ensures the geometric continuity and smoothness of the target contour through connectivity analysis and curvature constraints, avoiding breaks or deformation distortions. Simultaneously, by combining the corresponding points of the contour in 3D space with point cloud data, the contour normal, curvature, and physical scale parameters are accurately calculated to achieve a true size estimate of the target. This not only improves the applicability of the contour in geometric modeling but also provides high-precision morphological parameter support for subsequent structure matching and recognition, offering significant advantages, especially in industrial inspection and autonomous navigation requiring precise dimensional measurements.

[0034] In this embodiment, the process of inputting the contour normal, curvature, and physical scale parameters along with the size-distance joint representation into the 3D structure program generation module and performing matching and filtering includes: The contour normal, curvature, and physical scale parameters are combined with the size-distance joint representation to generate a structural description vector. The structural description vector contains a combination of image contour features and point cloud spatial features in a unified numerical space. The 3D structure program generation module is used to generate 3D structure program instances that conform to the structure program syntax based on the structure description vector. The 3D structure program generation module includes a structure syntax parsing unit, a geometric relation construction unit, and a spatial dimension encoding unit. The structure syntax parsing unit parses the semantic components in the structure description vector into structured syntax nodes. The geometric relation construction unit converts curvature and normal information into geometric topological relationships between nodes. The spatial dimension encoding unit encodes physical scale parameters into spatial metrics associated with nodes. The generated 3D structure program instances are indexed and retrieved in the structure program library to select a set of candidate structure programs whose structure syntax, geometric relations and spatial dimensions all match the 3D structure program instances. For each structural program in the candidate structural program set, the similarity with the 3D structural program instance is calculated based on the matching score function. The similarity score comprehensively considers normal consistency, curvature distribution similarity and physical scale difference. Candidate recognition results are selected in descending order of similarity scores and output to the consistency verification and sorting stage.

[0035] This invention fuses contour normals, curvature, and physical scale parameters with size-distance representations to generate a unified structural description vector. This vector is then parsed into structural syntax, geometric topology, and spatial dimension information within the 3D structure generation module. By performing matching searches and similarity scoring in a structure library, candidate structures highly consistent with current observations can be quickly selected, achieving high-precision identification. This method exhibits good generalization and retrieval efficiency in complex structures and multi-target scenarios, effectively shortening identification time and improving the identification success rate.

[0036] A bionic human eye target recognition system based on monocular vision and lidar includes: The image and point cloud acquisition module is used to acquire monocular image frames with synchronized timestamps and lidar point cloud data, and to perform coordinate system registration. The feature extraction module is used to extract image contour features and point cloud normals, curvature, reflectivity and local density, and generate image contour feature map and point cloud attribute set; The joint encoding module is used to jointly encode the image contour feature map and the point cloud attribute set to generate a dimension-distance joint representation that includes two-dimensional size and three-dimensional depth; The phase-field energy model generation module is used to generate an improved Ginzburg–Landau phase-field energy model based on the size-distance joint representation and the point cloud attribute set, and outputs the potential function parameters and anisotropic tensor. The phase field evolution optimization module is used to introduce the directional continuity constraint based on SE(2) sub-Riemann geodesy, combine the potential function parameters and anisotropic tensors to adjust the phase field intensity, and realize the phase field time alignment and optimization of adjacent frames through the optical flow calculation unit and the phase field optimization unit. The contour extraction module is used to extract zero contour lines in the current frame phase field to form the target contour, and to calculate the contour normal, curvature and physical scale parameters. The 3D structure program generation module is used to input the contour normal, curvature and physical scale parameters and size-distance joint representation into the structure syntax parsing unit, geometric relationship construction unit and spatial size encoding unit to generate 3D structure program instances, and to match and filter candidate structure programs in the structure program library and output candidate recognition results. The consistency verification and sorting module is used to verify and sort the candidate recognition results and output the final recognition result.

[0037] The target recognition system constructed in this invention integrates modular functions such as image and point cloud acquisition, feature extraction, joint encoding, phase field modeling, spatiotemporal optimization, contour extraction, and 3D structure matching to form a complete recognition process. Smooth data flow and standardized interfaces between modules enable the system to possess excellent scalability and portability. This system can not only be quickly deployed on different hardware platforms but also flexibly adjust recognition strategies according to task requirements, making it widely applicable in fields such as autonomous driving, robot perception, and industrial inspection, significantly improving the accuracy, stability, and real-time performance of target recognition.

[0038] Example 1: To verify the feasibility of this invention in practice, it was applied to an outdoor road inspection robot platform. The platform is equipped with a monocular camera and a lidar sensor, enabling continuous operation in environments with complex lighting, different road surfaces, and various obstacles. Traditional target recognition methods often suffer from contour breaks and large recognition errors when faced with interference from multiple light sources, the presence of highly reflective objects, and numerous obstructions, leading to deviations in subsequent inspection decisions and path planning. This invention significantly improves these problems through the collaborative perception of monocular vision and lidar, joint encoding of images and point clouds, and an improved Ginzburg–Landau phase-field energy model. The invention has been validated in various typical inspection scenarios.

[0039] During the experiment, the inspection robot first collected monocular image frames with synchronized timestamps and LiDAR point cloud data, and completed the registration of the image and point cloud coordinate systems. After registration, the image processing module extracted image contour features and calculated the normal, curvature, reflectivity, and local density of the point cloud data to generate an image contour feature map and a set of point cloud attributes. Then, these two types of data were jointly encoded to calculate the size-distance joint representation, and an improved Ginzburg–Landau phase field energy model was generated based on this representation. The potential function parameters and anisotropic tensors of this model are derived from the size-distance... The joint analysis of the joint representation and the point cloud attribute set introduces the direction continuity constraint of the SE(2) sub-Riemann geodesic in the phase field gradient term to ensure that the contour growth direction is consistent and smooth. For the processing of continuous frames, the system uses optical flow to time-align the phase field of adjacent frames and generates the phase field of the current frame in combination with the phase field optimization process. The zero contour line is extracted to form the target contour, and the contour normal, curvature and physical scale parameters are output. Then, the system inputs these parameters and the size-distance joint representation into the three-dimensional structure program generation module for target matching and screening, and finally outputs the recognition results after consistency verification and sorting.

[0040] In comparative tests, the method of this invention was compared with traditional monocular vision-based recognition methods and conventional point cloud geometric feature recognition methods. In various scenarios, including reflective road signs, tree occlusion, and complex sidewalk paving, this invention demonstrated superior performance in terms of contour integrity, recognition accuracy, and false detection rate. Test data is shown in Table 1. Table 1. Performance comparison of different recognition methods in typical inspection scenarios.

[0041] As shown in Table 1, the method of the present invention improves contour integrity by 11.6% to 16.6% and recognition accuracy by 9.3% to 12.6% in three complex inspection scenarios, while reducing the false detection rate by 7.4% to 12.5%. This indicates that the present invention can maintain more stable and higher-precision recognition capabilities in environments with high reflection, partial occlusion, and complex textures, ensuring the reliable operation of the inspection robot in changing environments.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A binocular vision and laser radar based bionic human eye target recognition method, characterized in that, Includes the following steps: S1. Acquire monocular image frames with synchronized timestamps and lidar point cloud data, and complete the registration of the image and point cloud coordinate systems. S2. Extract image contour features and point cloud normals, curvature, reflectivity and local density to generate image contour feature map and point cloud attribute set; S3. Jointly encode the image contour feature map and the point cloud attribute set, and calculate the size-distance joint representation; S4. Based on the size-distance joint representation and the point cloud attribute set, generate the improved Ginzburg–Landau phase field energy model and obtain the potential function parameters and anisotropic tensor. S5. Introduce the direction continuity constraint based on SE(2) sub-Riemann geodesic in the phase field gradient term, and adjust the phase field intensity by combining the potential function parameter and the anisotropic tensor to ensure smooth and consistent profile. S6. Use optical flow to time-align the phase fields of adjacent frames, optimize the phase field evolution based on potential function parameters and anisotropic tensors, and obtain the phase field of the current frame. S7. Extract zero contour lines from the current frame phase field to generate the target contour, and output the contour normal, curvature and physical scale parameters; S8. Input the contour normal, curvature, and physical scale parameters along with the size-distance joint representation into the 3D structure program generation module for matching and filtering to obtain candidate recognition results; S9. Perform consistency verification and sorting on the candidate recognition results, and output the final recognition result.

2. The monocular vision and lidar based biomimetic human eye target recognition method of claim 1, wherein, The improved Ginzburg–Landau phase field energy model: This includes generating learnable double-well potential function parameters based on size-distance joint representation, image texture quality, point cloud curvature, and local density; This includes an anisotropic tensor that generates a diffusion tensor based on the size-distance joint representation and the point cloud attribute set; A directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced into the phase field gradient term to ensure that the contour growth direction is consistent; By incorporating contour information from adjacent frames, a temporal consistency term is introduced to ensure a smooth transition of the target in the time series. 3.The monocular vision and lidar based bionic human eye target recognition method of claim 1, wherein, The process of extracting image contour features and point cloud normals, curvature, reflectivity, and local density, and generating image contour feature maps and point cloud attribute sets, includes: Edge detection is performed on monocular image frames to obtain the gradient magnitude and gradient direction of each pixel. Redundant edge responses are removed by non-maximum suppression, and an ordered sequence of contour points is formed by tracking the pixel connection relationship. The sequence of contour points is fitted into a set of two-dimensional continuous curves to construct an image contour feature map. A fixed-radius neighborhood search is performed on the lidar point cloud data. A three-dimensional covariance matrix is ​​constructed in each neighborhood. The eigenvector corresponding to the minimum eigenvalue is obtained by using the eigenvalue decomposition method as the unit normal vector. Calculating curvature using the smallest eigenvalue and the sum of eigenvalues ,in It is the smallest eigenvalue. These are the eigenvalues ​​along the x, y, and z directions, respectively; Reflectance is extracted from point cloud intensity information, and the point cloud is divided into fixed voxel units. The number of points in each voxel is counted and divided by the voxel volume to obtain the local density. The normal, curvature, reflectivity, and local density are combined in the order of the point cloud index to generate a set of point cloud attributes.

4. The bionic human eye target recognition method based on monocular vision and lidar according to claim 1, characterized in that, The calculation process for the joint encoding of image contour feature maps and point cloud attribute sets, and the joint representation of size-distance includes: The two-dimensional coordinates of each pixel in the image contour feature map are mapped to the three-dimensional coordinate position in the point cloud coordinate system. The point cloud normal, curvature, reflectivity and local density are associated at the mapped position to form spatial attribute matching pairs, which provide a spatial consistency basis for feature fusion. A multi-channel coding tensor is constructed using spatial attribute matching pairs. The first two dimensions of the multi-channel coding tensor correspond to the row and column positions of the image, and the third dimension includes the normal, curvature, reflectivity and local density channels, ensuring that each pixel has feature descriptions of both the image and the point cloud. A spatial scale channel is added to the multi-channel coding tensor. The spatial scale channel is calculated from the width and height of the bounding box of the contour to which the pixel belongs, providing two-dimensional size information of the target on the image plane. At the same time, a depth distance channel is added. The depth distance channel is calculated from the spatial distance from the corresponding three-dimensional point to the origin of the point cloud coordinate system, providing depth information of the target in three-dimensional space. The spatial scale channel and depth distance channel are normalized and then spliced ​​with the normal, curvature, reflectivity and local density channels in the channel dimension to generate a composite feature set. The composite feature set contains two-dimensional size, three-dimensional depth and point cloud attribute information. Perform a linear mapping operation along the channel dimension on the composite feature set to obtain a fixed-length size-distance joint representation vector.

5. The bionic human eye target recognition method based on monocular vision and lidar according to claim 1, characterized in that, The generation process of the improved Ginzburg–Landau phase field energy model includes: Based on the size-distance joint representation, texture gradient information in the image contour feature map, point cloud normal, curvature, reflectivity and local density, an input feature vector is constructed. The input is a learnable double-well potential function parameter generation unit. The double-well potential function consists of two stable potential wells and one unstable potential well. The depth and position of the stable potential wells are calculated by weighting the aspect ratio of the target two-dimensional contour, spatial distance, point cloud normal consistency, curvature distribution and local density. The position of the unstable potential wells is calculated by the contour gradient magnitude and point cloud sparsity. The potential function shape coefficient is determined by the normalized difference between the size information and the depth information. An anisotropic diffusion tensor is generated based on the size-distance joint representation and the point cloud attribute set. Principal component analysis is performed on the normal distribution of the point cloud to obtain the principal direction unit vector n1 and the orthogonal secondary direction unit vector n2. The principal direction diffusion coefficient D1 is calculated from the normal uniformity, and the secondary direction diffusion coefficient D2 is calculated from the curvature gradient intensity. The diffusion tensor is represented as: ； Where n1 is the direction of the principal component of the point cloud normal, n2 is the direction of curvature change orthogonal to n1, D1 is the diffusion coefficient of the principal direction, and D2 is the diffusion coefficient of the secondary direction. In the gradient term of the phase field energy model, a directional continuity constraint based on SE(2) sub-Riemann geodesics is introduced. The SE(2) space consists of translational components x, y and rotational components θ. The sub-Riemann metric restricts the propagation path to remain smooth and continuous in the x, y and θ dimensions. The directional continuity constraint is achieved by calculating the shortest path length of the geodesics of adjacent positions in the θ dimension and introducing a penalty factor in the energy term. The learnable double-well potential function parameters and anisotropic tensor inputs are combined with the phase field energy term and gradient term according to the model structure to form an improved Ginzburg–Landau phase field energy model, which is then passed to the phase field evolution stage.

6. The bionic human eye target recognition method based on monocular vision and lidar according to claim 1, characterized in that, The process of using optical flow to time-align the phase fields of adjacent frames includes: The current monocular image frame and the previous monocular image frame are input into the optical flow calculation unit. The optical flow calculation unit is used to calculate the displacement vector field of each pixel on the image plane based on the dense optical flow algorithm. The displacement vector field contains a horizontal component u and a vertical component v, which are used to represent the displacement of the pixel between two time points. The phase field distribution of the previous frame is mapped at the pixel level using the displacement vector field, so that the phase field of the previous frame is aligned with the phase field of the current frame in spatial coordinates, thus obtaining the temporally aligned phase field data of adjacent frames. The phase field data of adjacent frames aligned with time and the phase field data of the current frame are input into the phase field optimization unit. The phase field optimization unit is used to adjust the phase field intensity by combining the potential function parameters and the anisotropic tensor, and optimize the phase field evolution process based on the gradient direction constraint to generate the optimized phase field distribution of the current frame.

7. The bionic human eye target recognition method based on monocular vision and lidar according to claim 1, characterized in that, The process of extracting zero contour lines from the current frame's phase field to generate the target contour includes: Locate the set of points in the current frame phase field distribution where the phase field value is equal to zero, perform connectivity analysis according to the gradient direction of adjacent pixels, and form a continuous zero contour path. The zero contour path is smoothed to eliminate isolated noise points, and curvature constraints are used to maintain the geometric continuity of the contour, resulting in a structurally complete target contour curve. Based on the corresponding points of the target contour curve in three-dimensional space, the contour normal is calculated in conjunction with the registered point cloud coordinate system. The contour normal is a unit vector perpendicular to the local tangent of the contour. After the contour normal is calculated, the local curvature is fitted according to the discrete point distribution of the contour in three-dimensional space. The local curvature is obtained by measuring the rate of change of the normal in the neighborhood of the contour. By combining the spatial scale information of the point cloud with the pixel spacing conversion ratio of the contour in the image coordinate system, the physical scale parameter is calculated. The physical scale parameter is used to characterize the actual size of the target in real space.

8. The bionic human eye target recognition method based on monocular vision and lidar according to claim 1, characterized in that, The process of inputting contour normals, curvature, and physical scale parameters along with the dimension-distance joint representation into the 3D structure generation module for matching and filtering includes: The contour normal, curvature, and physical scale parameters are combined with the size-distance joint representation to generate a structural description vector. The structural description vector contains a combination of image contour features and point cloud spatial features in a unified numerical space. The 3D structure program generation module is used to generate 3D structure program instances that conform to the structure program syntax based on the structure description vector. The 3D structure program generation module includes a structure syntax parsing unit, a geometric relation construction unit, and a spatial dimension encoding unit. The structure syntax parsing unit parses the semantic components in the structure description vector into structured syntax nodes. The geometric relation construction unit converts curvature and normal information into geometric topological relationships between nodes. The spatial dimension encoding unit encodes physical scale parameters into spatial metrics associated with nodes. The generated 3D structure program instances are indexed and retrieved in the structure program library to select a set of candidate structure programs whose structure syntax, geometric relations and spatial dimensions all match the 3D structure program instances. For each structural program in the candidate structural program set, the similarity with the 3D structural program instance is calculated based on the matching score function. The similarity score comprehensively considers normal consistency, curvature distribution similarity and physical scale difference. Candidate recognition results are selected in descending order of similarity scores and output to the consistency verification and sorting stage.

9. A bionic human eye target recognition system based on monocular vision and lidar, comprising performing the bionic human eye target recognition method based on monocular vision and lidar as described in any one of claims 1 to 8, characterized in that, include: The image and point cloud acquisition module is used to acquire monocular image frames with synchronized timestamps and lidar point cloud data, and to perform coordinate system registration. The feature extraction module is used to extract image contour features and point cloud normals, curvature, reflectivity and local density, and generate image contour feature map and point cloud attribute set; The joint encoding module is used to jointly encode the image contour feature map and the point cloud attribute set to generate a dimension-distance joint representation that includes two-dimensional size and three-dimensional depth; The phase-field energy model generation module is used to generate an improved Ginzburg–Landau phase-field energy model based on the size-distance joint representation and the point cloud attribute set, and outputs the potential function parameters and anisotropic tensor. The phase field evolution optimization module is used to introduce the directional continuity constraint based on SE(2) sub-Riemann geodesy, combine the potential function parameters and anisotropic tensors to adjust the phase field intensity, and realize the phase field time alignment and optimization of adjacent frames through the optical flow calculation unit and the phase field optimization unit. The contour extraction module is used to extract zero contour lines in the current frame phase field to form the target contour, and to calculate the contour normal, curvature and physical scale parameters. The 3D structure program generation module is used to input the contour normal, curvature and physical scale parameters and size-distance joint representation into the structure syntax parsing unit, geometric relationship construction unit and spatial size encoding unit to generate 3D structure program instances, and to match and filter candidate structure programs in the structure program library and output candidate recognition results. The consistency verification and sorting module is used to verify and sort the candidate recognition results and output the final recognition result.

Images