Outdoor graph-free navigation positioning path planning method, system, equipment and medium

Abstract

The invention discloses an outdoor graph-free navigation positioning path planning method, system and device and a medium. The method comprises the following steps: calculating an error between two adjacent frames, and calculating pose transformation of a robot; segmenting a photovoltaic module area from the image, and projecting the three-dimensional point cloud data to the photovoltaic module area through a coordinate conversion relation to obtain effective three-dimensional point cloud data; photovoltaic panel plane features, photovoltaic panel upper and lower edge line features and photovoltaic support column-shaped features are extracted from the effective three-dimensional point cloud data; determining a target photovoltaic panel plane and target upper and lower edge line segments according to photovoltaic panel plane features and photovoltaic panel upper and lower edge line features, and determining a target column according to photovoltaic support column-shaped features; and according to the target photovoltaic panel plane, the target upper and lower edge line segments and the target column, correcting the scale drift of the laser speedometer, and adopting the corrected laser speedometer for path planning. According to the invention, the accuracy of outdoor graph-free navigation positioning path planning can be improved.

Description

Technical Field

[0001] This application relates to the field of path planning technology, and in particular to an outdoor mapless navigation and positioning path planning method, system, device and medium. Background Technology

[0002] Existing technologies rely on BeiDou (GPS) or, by integrating inertial measurement units (IMUs) with differential base stations for high-precision positioning. However, real-time kinematic (RTK) positioning is affected by weather, signal blockage, and electromagnetic interference, impacting the accuracy and reliability of robot positioning. Some existing technologies employ a mapping-first, localization-later approach, using laser and vision sensors to create a high-precision map of the photovoltaic power station, then matching and locating within this map, and finally integrating RTK to obtain an accurate and reliable positioning result. However, this mapping-first, localization-later approach has the following drawbacks:

[0003] 1. When the outdoor scene is large, the initial map collection workload is very large, and the early deployment mapping cost is very high; 2. Some scenes have very high similarity, making them difficult to distinguish, and the lack of obvious differentiation features affects the positioning effect; 3. Some scenes are subject to significant changes, causing the map to become invalid and unusable. For example, in photovoltaic power stations in Northwest China, the angle of the power station components changes with sunlight or seasonality, making the established map unusable; 4. In the special scenario of photovoltaic power stations, which is highly structured and periodically repetitive, general odometer calculation methods (such as point cloud ICP or visual feature point matching) are very difficult. In the photovoltaic power station scene, there are similar planes and columns everywhere, which can easily lead to feature matching confusion and misassociation, thus causing odometer drift. Summary of the Invention

[0004] This application aims to propose an outdoor mapless navigation and positioning path planning method, system, device, and medium. This method does not require the establishment of a high-precision map, can more accurately correct odometer drift, and improve the accuracy of outdoor mapless navigation and positioning path planning.

[0005] In a first aspect, embodiments of this application provide an outdoor mapless navigation and positioning path planning method, the method comprising: The lidar and vision sensor are attached to the robot, and the coordinate transformation relationship between the lidar and the vision sensor is calibrated. The system acquires three-dimensional point cloud data of the photovoltaic module using the lidar and images of the photovoltaic module using the vision sensor; it also extracts laser features from the three-dimensional point cloud data and visual features from the images. Based on the coordinate transformation relationship and the laser feature or the visual feature, the features between two adjacent frames are associated to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. Based on the associated feature pairs, the error between two adjacent frames is calculated, and based on the error, the pose transformation of the robot is calculated. The photovoltaic module region is segmented from the image, and the three-dimensional point cloud data is projected onto the photovoltaic module region through the coordinate transformation relationship to obtain effective three-dimensional point cloud data; Extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the columnar features of the photovoltaic support from the effective three-dimensional point cloud data; Based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel, the target photovoltaic panel planar features and the target upper and lower edge line segments are determined; and based on the columnar features of the photovoltaic support, the target column is determined. Based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column, the scale drift of the laser odometry is corrected to obtain a corrected laser odometry. The corrected laser odometry is then used for path planning to obtain the target planned path.

[0006] Compared with the prior art, the first aspect of this application has the following beneficial effects: This method involves binding a LiDAR and a vision sensor to a robot and calibrating the coordinate transformation relationship between them. It acquires 3D point cloud data of a photovoltaic module using the LiDAR and images of the photovoltaic module using the vision sensor. Laser features are extracted from the 3D point cloud data, and visual features are extracted from the images. Based on the coordinate transformation relationship and the laser or visual features, features between adjacent frames are correlated to obtain correlated feature pairs. These correlated feature pairs include those between laser features and those between laser and visual features. Based on the correlated feature pairs, the error between adjacent frames is calculated, and the robot's pose transformation is calculated based on the error. The image is segmented to identify the photovoltaic module region. 3D point cloud data is projected onto the photovoltaic module region using coordinate transformation to obtain effective 3D point cloud data. From this effective 3D point cloud data, the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the columnar features of the photovoltaic support are extracted. Based on the planar and upper and lower edge features of the photovoltaic panel, the target photovoltaic panel planar features and the target upper and lower edge line segments are determined. Similarly, based on the columnar features of the photovoltaic support, the target column is determined. Based on the target photovoltaic panel planar features, the target upper and lower edge line segments, and the target column, the scale drift of the laser odometry is corrected to obtain a corrected laser odometry. This corrected laser odometry is then used for path planning to obtain the target planned path. Thus, by projecting the 3D point cloud data onto the photovoltaic module area through coordinate transformation, valid 3D point cloud data is filtered out, and then the target photovoltaic panel plane, the target upper and lower edge segments, and the target pillar are determined. This two-stage filtering process yields more accurate photovoltaic panel plane, upper and lower edge segments, and pillars. Finally, by comprehensively considering the target photovoltaic panel plane, target upper and lower edge segments, and target pillars, the scale drift of the laser odometry is corrected. This more accurately corrects odometry drift, improves the accuracy of outdoor mapless navigation and positioning path planning, and establishes a high-precision map.

[0007] In some embodiments, determining the target photovoltaic panel plane and the target upper and lower edge line segments based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel includes: Based on the photovoltaic panel planar features, a first photovoltaic panel planar feature that meets a preset angle is selected, wherein the photovoltaic panel planar feature includes a planar normal vector; Based on the characteristics of the upper and lower edge lines of the photovoltaic panel, extract the first upper and lower edge line segments corresponding to the plane of the first photovoltaic panel; Select target upper and lower edge segments and target photovoltaic panel plane that meet the first preset conditions from the first upper and lower edge segments.

[0008] In some embodiments, determining the target column based on the column-like characteristics of the photovoltaic support includes: Based on the column-like characteristics of the photovoltaic support structure, multiple columns that meet the second preset conditions are selected. Perform frame-by-frame matching on the multiple columns to obtain the columns after frame-by-frame matching, and use the columns after frame-by-frame matching as the target columns.

[0009] In some embodiments, the step of correcting the dimensional drift of the laser odometry based on the target photovoltaic panel plane, the upper and lower edge segments of the target, and the target column to obtain a corrected laser odometry includes: Based on the photovoltaic panel plane features corresponding to the target photovoltaic panel plane, a constraint factor for the plane normal vector is constructed, wherein the photovoltaic panel plane features include the photovoltaic plane normal vector; Based on the characteristics of the upper and lower edge lines of the photovoltaic panel corresponding to the target upper and lower edge line segments, a constraint factor for the upper and lower edge lines is constructed, wherein the characteristics of the upper and lower edge lines of the photovoltaic panel are the extracted upper and lower edge line segments of the photovoltaic panel. Based on the column-shaped features of the photovoltaic support corresponding to the target column, a column constraint factor for the photovoltaic support is constructed, wherein the column-shaped features of the photovoltaic support are the central axis of the column. The scale drift of the laser odometer is corrected by using the constraint factors of the plane normal vector, the constraint factors of the upper and lower edge lines, and the column constraint factor of the photovoltaic bracket, thus obtaining the corrected laser odometer.

[0010] In some implementations, the constraint factor for constructing the plane normal vector based on the photovoltaic panel plane features corresponding to the target photovoltaic panel plane includes: Using the photovoltaic plane normal vector corresponding to the target photovoltaic panel plane as the first observation result, the first observation result of the previous frame and the first observation result of the current frame are obtained. The robot's pose at the current moment is determined based on the robot's pose change. The robot pose at the current moment is transformed into the global coordinate system to obtain the robot pose in the global coordinate system; Multiply the first observation result of the previous frame with the robot pose in the global coordinate system to obtain the first prediction result of the current frame; The constraint factor for the plane normal vector is constructed by subtracting the first observation result and the first prediction result of the current frame.

[0011] In some implementations, constructing the column constraint factor of the photovoltaic support based on the column-like characteristics of the photovoltaic support corresponding to the target column includes: Using the central axis of the pillar corresponding to the target pillar as the second observation result, the second observation result of the previous frame and the second observation result of the current frame are obtained. The robot's pose at the current moment is determined based on the robot's pose change. The robot pose at the current moment is transformed into the global coordinate system to obtain the robot pose in the global coordinate system; Multiply the second observation result of the previous frame with the robot pose in the global coordinate system to obtain the second prediction result of the current frame; The column constraint factor of the photovoltaic support is constructed by subtracting the second observation result of the current frame from the second prediction result of the current frame.

[0012] In some embodiments, the step of using the calibrated laser odometry for path planning to obtain the target planned path includes: Using the corrected laser odometry, laser point clouds from multiple consecutive moments are transformed to the same coordinate system to construct local maps of multiple point clouds. Calculate the target photovoltaic plane normal vector in the local map of the multiple point clouds, and extract the target point cloud plane of the photovoltaic module through the target photovoltaic plane normal vector; Based on the target point cloud plane of the photovoltaic module, extract the upper and lower edge line segments in the target point cloud plane, and fit the upper and lower edge line segments in the target point cloud plane to obtain the lower edge trajectory of the photovoltaic module; Outdoor mapless navigation and positioning path planning is performed based on the trajectory of the lower edge of the photovoltaic module.

[0013] Secondly, embodiments of this application also provide an outdoor mapless navigation and positioning path planning system, the system comprising: A transformation relationship calibration unit is used to bind the lidar and vision sensor to the robot and calibrate the coordinate transformation relationship between the lidar and the vision sensor; The first feature extraction unit is used to acquire three-dimensional point cloud data of the photovoltaic module through the lidar, and to acquire images of the photovoltaic module through the vision sensor; to extract the laser features in the three-dimensional point cloud data, and to extract the visual features in the images; The feature pair association unit is used to associate features between two adjacent frames according to the coordinate transformation relationship and the laser feature or the visual feature to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. The pose transformation calculation unit is used to calculate the error between two adjacent frames based on the associated feature pairs, and to calculate the pose transformation of the robot based on the error. The first data filtering unit is used to segment the photovoltaic module area from the image and project the three-dimensional point cloud data onto the photovoltaic module area through the coordinate transformation relationship to obtain effective three-dimensional point cloud data. The second feature extraction unit is used to extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support from the effective three-dimensional point cloud data. The second data filtering unit is used to determine the target photovoltaic panel plane and the target upper and lower edge line segments based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel, and to determine the target column based on the column-shaped features of the photovoltaic support. The path planning unit is used to correct the scale drift of the laser odometer based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column to obtain the corrected laser odometer, and to use the corrected laser odometer for path planning to obtain the target planned path.

[0014] Thirdly, embodiments of this application also provide an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform an outdoor mapless navigation positioning path planning method as described above.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute an outdoor mapless navigation and positioning path planning method as described above.

[0016] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating an embodiment of the outdoor mapless navigation and positioning path planning method provided in this application; Figure 2 This is a schematic diagram of the segmented photovoltaic module area in the best embodiment of the outdoor mapless navigation and positioning path planning method provided in this application; Figure 3This is a schematic diagram of the effective 3D point cloud data projection effect in the best embodiment of the outdoor mapless navigation and positioning path planning method provided in this application; Figure 4 This is a factor graph of laser odometry optimization in the best embodiment of the outdoor mapless navigation and positioning path planning method provided in this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the outdoor mapless navigation and positioning path planning system provided in this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0019] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0020] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0021] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0022] To address the issue of low accuracy in existing outdoor mapless navigation and positioning path planning technologies, this application proposes an outdoor mapless navigation and positioning path planning method, system, device, and medium.

[0023] Reference Figure 1 This application provides a flowchart illustrating an outdoor mapless navigation and positioning path planning method. This method is applied to an electronic device, which may be a server or a mobile terminal, etc. Figure 1 As shown, the outdoor mapless navigation and positioning path planning method may include the following steps: Step S101: Bind the LiDAR and vision sensor to the robot and calibrate the coordinate transformation relationship between the LiDAR and vision sensor; Step S102: Collect three-dimensional point cloud data of photovoltaic modules using LiDAR, and collect images of photovoltaic modules using a visual sensor; extract laser features from the three-dimensional point cloud data, and extract visual features from the images; Step S103: Based on the coordinate transformation relationship and laser features or visual features, associate the features between two adjacent frames to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. Step S104: Calculate the error between two adjacent frames based on the associated feature pairs, and calculate the robot's pose transformation based on the error; Step S105: Segment the photovoltaic module area from the image, and project the three-dimensional point cloud data onto the photovoltaic module area through coordinate transformation to obtain effective three-dimensional point cloud data; Step S106: Extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support from the effective 3D point cloud data; Step S107: Based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel, determine the target photovoltaic panel planar features and the target upper and lower edge line segments; and based on the columnar features of the photovoltaic support, determine the target column. Step S108: Based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column, the scale drift of the laser odometry is corrected to obtain the corrected laser odometry. The corrected laser odometry is then used for path planning to obtain the target planned path.

[0024] In this embodiment, a lidar and a vision sensor are attached to the robot, and the coordinate transformation relationship between the lidar and the vision sensor is calibrated. Three-dimensional point cloud data of the photovoltaic module is acquired using the lidar, and images of the photovoltaic module are acquired using the vision sensor. Laser features are extracted from the three-dimensional point cloud data, and visual features are extracted from the images. Based on the coordinate transformation relationship and the laser or visual features, features between adjacent frames are associated to obtain associated feature pairs. These associated feature pairs include those between laser features and those between laser and visual features. Based on the associated feature pairs, the error between adjacent frames is calculated, and based on the error, the robot's pose transformation is calculated. The photovoltaic module region is segmented from the image, and the 3D point cloud data is projected onto the photovoltaic module region through coordinate transformation to obtain effective 3D point cloud data. From the effective 3D point cloud data, the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the columnar features of the photovoltaic support are extracted. Based on the planar features and upper and lower edge features of the photovoltaic panel, the target photovoltaic panel planar features and the target upper and lower edge line segments are determined. Based on the columnar features of the photovoltaic support, the target columnar features are determined. Based on the target photovoltaic panel planar features, the target upper and lower edge line segments, and the target column, the scale drift of the laser odometry is corrected to obtain the corrected laser odometry. The corrected laser odometry is then used for path planning to obtain the target planned path. Thus, by projecting the 3D point cloud data onto the photovoltaic module area through coordinate transformation, valid 3D point cloud data is filtered out, and then the target photovoltaic panel plane, the target upper and lower edge segments, and the target pillar are determined. This two-stage filtering process yields more accurate photovoltaic panel plane, upper and lower edge segments, and pillars. Finally, by comprehensively considering the target photovoltaic panel plane, target upper and lower edge segments, and target pillars, the scale drift of the laser odometry is corrected. This more accurately corrects odometry drift, improves the accuracy of outdoor mapless navigation and positioning path planning, and establishes a high-precision map.

[0025] The coordinate transformation relationship between the LiDAR and the vision sensor can be calibrated using a checkerboard calibration method, or other techniques known to those skilled in the art. This embodiment does not impose any specific limitations on this.

[0026] The above-mentioned extraction of laser features from 3D point cloud data can be achieved by calculating the curvature or normal vector change of the neighborhood of each point in the point cloud, classifying the point cloud into edge points and corner points, and extracting edge points and corner points. Alternatively, it can be achieved by using existing technologies such as ISS or Harris3D to directly extract stable key points from the 3D point cloud, and using edge points, corner points, and key points as laser features in the extracted 3D point cloud data.

[0027] The aforementioned extraction of visual features from images can be achieved using existing technologies such as ORB (Oriented Fast and Rotated BRIEF), Scale Invariant Feature Transform (SIFT), or Speed-Up Robust Feature Transform (SURF).

[0028] The above-mentioned method associates features between two adjacent frames based on coordinate transformation relationships and laser or visual features to obtain associated feature pairs. This can be achieved by projecting laser features onto the visual sensor coordinate system through coordinate transformation relationships to generate a depth map, and then finding the effective depth value corresponding to the visual features in the depth map. This associates features between two adjacent frames to obtain associated feature pairs between laser features and visual features (i.e., matching pairs of 3D features and 2D features).

[0029] The above-mentioned segmentation of the photovoltaic module region from the image can be achieved by using existing segmentation algorithms such as Segnet or mask R-CNN to identify the region of the photovoltaic module.

[0030] The above-mentioned extraction of the photovoltaic panel planar features, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support from effective three-dimensional point cloud data can be carried out using techniques known to those skilled in the art. This embodiment does not provide a specific description or limitation of these techniques.

[0031] In some implementations, the target photovoltaic panel plane and target upper and lower edge line segments are determined based on the planar features of the photovoltaic panel and the features of the upper and lower edge lines of the photovoltaic panel, including: Based on the planar features of the photovoltaic panel, the first photovoltaic panel plane that meets the preset angle is selected. The planar features of the photovoltaic panel include the planar normal vector. Based on the characteristics of the upper and lower edge lines of the photovoltaic panel, extract the first upper and lower edge line segments corresponding to the plane of the first photovoltaic panel; Select target upper and lower edge segments and target photovoltaic panel plane that meet the first preset conditions from the first upper and lower edge segments.

[0032] In this embodiment, the first photovoltaic panel plane that meets the preset angle is first selected, then the first upper and lower edge line segments corresponding to the first photovoltaic panel plane are extracted, and then the target upper and lower edge line segments and the target photovoltaic panel plane that meet the first preset conditions are selected from the first upper and lower edge line segments. This can more accurately obtain the target photovoltaic panel plane and the target upper and lower edge line segments, laying a good data foundation for the subsequent correction of the scale drift of the laser odometry.

[0033] The above-mentioned selection of a first photovoltaic panel plane that meets the preset angle based on the characteristics of the photovoltaic panel plane can be based on the plane normal vector, selecting a first photovoltaic panel plane with a south-facing angle of 35-55 degrees (i.e., the preset angle). It should be noted that the preset angle in this embodiment can be changed according to the actual situation, and this embodiment does not specifically limit it.

[0034] The aforementioned selection of target upper and lower edge segments and target photovoltaic panel planes that meet the first preset conditions from the first upper and lower edge segments can be achieved by selecting target upper and lower edge segments that are parallel and have a spacing of 4.5 meters between them (i.e., the first preset condition), and then selecting the corresponding target photovoltaic panel planes based on these target upper and lower edge segments. It should be noted that the first preset condition in this embodiment can be changed according to actual circumstances, and this embodiment does not impose specific limitations on it.

[0035] In some implementations, the target column is determined based on the column-like characteristics of the photovoltaic support, including: Based on the column-like characteristics of the photovoltaic support structure, multiple columns that meet the second preset conditions are selected. Perform frame-by-frame matching on multiple pillars to obtain the pillars after frame-by-frame matching, and use the pillars after frame-by-frame matching as the target pillars.

[0036] In this embodiment, by selecting multiple columns that meet the second preset conditions based on the column-like characteristics of the photovoltaic support, the target columns can be obtained more accurately, laying a good data foundation for the subsequent correction of the scale drift of the laser odometer.

[0037] The above-mentioned selection of multiple columns that meet the second preset conditions based on the column-shaped characteristics of the photovoltaic support can be a selection of multiple columns that meet the spacing between the columns (i.e., the second preset conditions) based on the column-shaped characteristics of the photovoltaic support.

[0038] The above-mentioned frame matching of multiple pillars can be achieved by arranging multiple pillars in sequence according to the interval between the pillars.

[0039] In some embodiments, the dimensional drift of the laser odometry is corrected based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column to obtain a corrected laser odometry, including: Based on the photovoltaic panel plane features corresponding to the target photovoltaic panel plane, a constraint factor for the plane normal vector is constructed. The photovoltaic panel plane features include the photovoltaic plane normal vector. Based on the characteristics of the upper and lower edge lines of the photovoltaic panel corresponding to the target upper and lower edge line segments, a constraint factor for the upper and lower edge lines is constructed. The characteristics of the upper and lower edge lines of the photovoltaic panel are the extracted upper and lower edge line segments of the photovoltaic panel. Based on the column-shaped features of the photovoltaic support corresponding to the target column, the column constraint factor of the photovoltaic support is constructed, and the column-shaped features of the photovoltaic support are the central axis of the column; The scale drift of the laser odometer is corrected by minimizing the constraint factors of the plane normal vector, the upper and lower edge lines, and the column constraint factor of the photovoltaic support, thus obtaining the corrected laser odometer.

[0040] In this embodiment, the scale drift of the laser odometer is corrected by minimizing the constraint factors of the plane normal vector, the constraint factors of the upper and lower edge lines, and the column constraint factors of the photovoltaic support. This can more accurately correct the odometer drift and thus improve the accuracy of outdoor mapless navigation and positioning path planning.

[0041] The photovoltaic plane normal vector, the upper and lower edge segments of the photovoltaic panel, and the central axis of the column mentioned above can all be extracted using feature extraction techniques known to those skilled in the art. This embodiment does not specifically limit or describe these features.

[0042] The above-mentioned correction of the scale drift of the laser odometer by minimizing the constraint factors of the plane normal vector, the upper and lower edge lines, and the photovoltaic support column can be achieved by minimizing the constraint factors of the plane normal vector, the upper and lower edge lines, and the photovoltaic support column to correct the scale drift of the laser odometer step by step.

[0043] In some implementations, constraint factors for the plane normal vector are constructed based on the features of the photovoltaic panel plane corresponding to the target photovoltaic panel plane, including: The photovoltaic plane normal vector corresponding to the target photovoltaic panel plane is used as the first observation result, and the first observation result of the previous frame and the first observation result of the current frame are obtained. Determine the robot's pose at the current moment based on the robot's pose change; Transform the robot's current pose to the global coordinate system to obtain the robot's pose in the global coordinate system; Multiply the first observation result of the previous frame with the robot pose in the global coordinate system to obtain the first prediction result of the current frame; The constraint factor for the plane normal vector is constructed by subtracting the first observation result and the first prediction result of the current frame.

[0044] In this embodiment, the scale drift of the laser odometry is corrected by the constraint factor of the constructed plane normal vector, which can lay a good data foundation for improving the accuracy of outdoor mapless navigation and positioning path planning in the later stage.

[0045] The above method of determining the robot pose at the current moment based on the robot's pose transformation can be achieved by calculating the robot pose at the current moment based on the robot pose at the previous moment through the robot's pose transformation.

[0046] The above-mentioned transformation of the robot's current pose to the global coordinate system can be achieved by using a coordinate transformation method known to those skilled in the art.

[0047] In some implementations, based on the column-like characteristics of the photovoltaic support corresponding to the target column, a column constraint factor for the photovoltaic support is constructed, including: The center axis of the pillar corresponding to the target pillar is used as the second observation result, and the second observation result of the previous frame and the second observation result of the current frame are obtained. Determine the robot's pose at the current moment based on the robot's pose change; Transform the robot's current pose to the global coordinate system to obtain the robot's pose in the global coordinate system; Multiply the second observation result of the previous frame with the robot pose in the global coordinate system to obtain the second prediction result of the current frame; The column constraint factor of the photovoltaic support is constructed by subtracting the second observation result and the second prediction result of the current frame.

[0048] In this embodiment, the scale drift of the laser odometer is corrected by the column constraint factor of the constructed photovoltaic support, which can lay a good data foundation for improving the accuracy of outdoor mapless navigation and positioning path planning in the later stage.

[0049] In some implementations, a calibrated laser odometry is used for path planning to obtain the target planned path, including: By using a calibrated laser odometry, laser point clouds from multiple consecutive moments are transformed into the same coordinate system to construct local maps of multiple point clouds. Calculate the target photovoltaic plane normal vector in the local map of multiple point clouds, and extract the target point cloud plane of the photovoltaic module through the target photovoltaic plane normal vector; Based on the target point cloud plane of the photovoltaic module, extract the upper and lower edge line segments in the target point cloud plane, and fit the upper and lower edge line segments in the target point cloud plane to obtain the lower edge trajectory of the photovoltaic module; Outdoor mapless navigation and positioning path planning is performed based on the trajectory of the lower edge of the photovoltaic module.

[0050] In this embodiment, by calculating the target photovoltaic plane normal vector in the local map of multiple point clouds, a more stable target point cloud plane can be obtained, thereby obtaining stable upper and lower edge line segments, which better reduces the error of the upper and lower edge line segments, thus obtaining a more accurate lower edge trajectory of the photovoltaic module, which can improve the accuracy of outdoor mapless navigation and positioning path planning.

[0051] The above-mentioned transformation of laser point clouds at multiple consecutive moments to the same time coordinate system to construct a local map of multiple point clouds can be achieved by using a calibrated laser odometry to transform laser point clouds at multiple consecutive moments to the same time coordinate system. In this case, multiple laser point clouds are superimposed in the same time coordinate system to construct a local map of multiple point clouds.

[0052] The above-mentioned fitting of the upper and lower edge line segments in the target point cloud plane can be performed using the least squares method. Other fitting techniques known to those skilled in the art can also be used for fitting the upper and lower edge line segments in the target point cloud plane. This embodiment does not specifically limit this.

[0053] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below: This embodiment uses both LiDAR and a visual sensor for image-free localization. The specific implementation steps are as follows: I. Determine robot pose transformation by fusing LiDAR and visual sensor data.

[0054] 1. Calibration and synchronization of LiDAR and vision sensors.

[0055] Time synchronization: Hardware synchronization (external trigger) or software synchronization (timestamp interpolation) is typically used to ensure that the timestamps of each frame of data acquired by the vision sensor (such as a camera) and LiDAR are aligned.

[0056] Spatial calibration: Accurately calibrate the transformation extrinsic parameters (i.e., coordinate transformation relationship) between the camera and the LiDAR. The transformation extrinsic parameters include the rotation matrix R and the translation vector t. This is necessary to project a point in one sensor coordinate system onto another sensor coordinate system.

[0057] 2. Extract features that can be used for matching from the data from the two sensors.

[0058] (1) Extract visual features from images.

[0059] Visual features can be extracted from images using feature point detectors and descriptors, such as ORB, SIFT, or SURF, and their corresponding descriptors. ORB is the most commonly used due to its speed and rotation and scale invariance. Neural networks can also be used to extract more robust feature points and descriptors, such as SuperPoint or LF-Net, and their corresponding descriptors.

[0060] Visual feature output: Obtain a set of 2D feature points (i.e., visual features) and the feature vectors described by their corresponding descriptors.

[0061] (2) Extract laser features from point cloud.

[0062] 1) Features extracted based on curvature or normal vectors: Calculate the curvature or normal vector change of the neighborhood of each point in the point cloud, and classify the point cloud into edge points and corner points: large curvature, located at the outline of an object or at a corner (significant geometric features). Planar points: small curvature, located on a flat surface (used to constrain the normal direction).

[0063] 2) Key point extraction: Use algorithms such as ISS or Harris3D to directly extract stable key points from 3D point clouds.

[0064] Laser feature output: Obtain a set of 3D feature points Vectors (including edge points, corner points, and key points).

[0065] 3. Feature association.

[0066] This step involves finding the corresponding features in the previous frame (or local map) for the features of the current frame.

[0067] (1) Laser-assisted visual feature depth association.

[0068] The main idea is: for a 2D visual feature point in an image Using the calibrated transformation extrinsic parameters and the current lidar point cloud, for Assign a relatively accurate 3D depth value. Specific implementation steps: 1) The current laser point cloud By transforming the extrinsic parameters (R and t) and projecting them onto the camera coordinate system, a depth map or a sparse 3D-2D feature mapping is generated.

[0069] 2) For visual feature points The effective depth values ​​within the neighborhood of a feature point are found on the depth map. The 3D coordinates of this feature point can be obtained through nearest neighbor search or interpolation methods. .

[0070] 3) If the depth value of the point is valid, the visual feature point is a feature point with 3D coordinates.

[0071] Advantages of vision and laser fusion: It directly solves the problems of scale blur and depth uncertainty in monocular vision.

[0072] (2) Visual-assisted laser feature matching.

[0073] For 3D laser feature points extracted from 3D point clouds By utilizing its 3D coordinates and transformation extrinsic parameters, it is projected onto the image and associated with visual features to obtain richer descriptive information or verify matching consistency. Specific implementation steps: 1) 3D laser feature points Projecting onto the image plane yields 2D projection points. .

[0074] 2) In Search for visual feature points in the neighborhood If their descriptors are similar enough, a cross-modal matching association is established between laser features and visual features.

[0075] 3) This association can be used for subsequent joint optimization, adding an appearance-based constraint to 3D geometry matching.

[0076] 4. Motion estimation and optimization, calculation and extrapolation of odometer.

[0077] Using the feature correspondence established in step 3 (i.e., associated feature pairs, including 2D-2D, 3D-3D, or more commonly 3D-2D associated feature pairs), an optimization problem is constructed to solve for the relative motion between two frames. (Transformation matrix, i.e., rotation R and translation t between the two frames of data). Construct the residual term: (1) Visual reprojection error (3D-2D).

[0078] For those with 3D coordinates The visual feature points are projected onto the next frame image and matched with the 2D feature points in the next frame image. Calculation error .

[0079] ; in, It is a camera projection model. It is the pose transformation to be determined.

[0080] (2) Laser point cloud matching error (3D-3D).

[0081] 1) Point-to-point error Taking the Iterative Closest Point (ICP) as an example: ; in, yes The corresponding point in the map in the next frame.

[0082] 2) Point-to-surface or line error : Calculate the current frame point The distance to the corresponding local plane or line in the next frame. It offers better numerical stability.

[0083] ; in, It is a plane normal vector.

[0084] 5. Tightly coupled nonlinear optimization.

[0085] The visual reprojection error and laser point cloud matching error are treated as different constraints to construct a unified total cost function. A nonlinear optimization library (such as g2o, Ceres Solver, or GTSAM) is used to minimize this total cost function, while simultaneously solving for the optimal pose transformation. . formula: ; in, , and These are all weighting coefficients, which can be dynamically adjusted based on the sensor confidence level, with a total weight of 1.

[0086] 2. Perform localization without a map based on the determined robot pose transformation.

[0087] In a photovoltaic (PV) power plant scenario, there are clusters of periodically distributed PV modules. Features such as the planar characteristics of the PV panels and the top and bottom edges of the PV panels can be extracted. There are also numerous support structures for mounting the PV modules, with columns of fixed length that periodically vary. The column-like features of these support structures can be extracted. These features extracted from the PV power plant scenario are then used in odometer calculations. The core idea is to improve matching from feature point-based methods to structural constraint-based methods. Specifically, the steps include: 1. Visual-assisted fusion of ambiguous laser data screening.

[0088] Step one has completed the calibration between the visual sensor and the LiDAR. The regions of the photovoltaic modules in the visual image are then identified using detection algorithms or segmentation algorithms such as SegNet and Mask R-CNN. Figure 2 As shown in the figure, the 3D point cloud acquired by the lidar is projected onto the image through extrinsic parameter transformation. The point cloud belonging to the segmented photovoltaic module area is selected as the valid 3D point cloud data to be filtered. The projection effect is as follows. Figure 3 As shown, the striped pattern represents the 3D point cloud of the lidar.

[0089] 2. Extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support.

[0090] 1) Perform planar segmentation on the effective 3D point cloud data (using RANSAC, region growing, or deep learning, etc.). Each photovoltaic panel will form a large point cloud plane (i.e., photovoltaic panel planar features). The parameters output for each point cloud plane include: normal vector. and the center point of the plane .

[0091] 2) For the extracted point cloud plane, calculate its 2D convex hull or perform boundary extraction. Specifically, extract the boundary points of the point cloud plane using a convex hull algorithm (such as Graham scan) or a concave hull algorithm (such as alpha shapes), and perform line segment detection on the boundary points to extract the upper and lower edge line segments of the photovoltaic panel. It should be noted that the extraction of upper and lower edge line segments in this embodiment can employ techniques known to those skilled in the art, which will not be specifically described in this embodiment.

[0092] 3) LiDAR: Using methods based on Euclidean clustering or verticality detection, vertical, columnar point cloud clusters are segmented from the point cloud. Each cluster represents a support column. The central axis of the column (a 3D vertical line, serving as the columnar feature of the photovoltaic support) can be fitted, denoted as... It should be noted that the central axis of the column can be fitted using fitting techniques known to those skilled in the art, but this embodiment does not provide a specific description or limitation of this technique.

[0093] 3. Filtering ambiguous data.

[0094] 1) A photovoltaic power station consists of a long, continuous row of photovoltaic panels. Each panel has two parallel lines, one above the other, with a fixed distance between them (e.g., 4.5 meters). The panels are typically installed at a south-facing angle of 35 to 55 degrees. Specific selection criteria are as follows: Through the normal vector of the photovoltaic panel Photovoltaic panel planes facing south at an angle of 35 to 55 degrees (i.e., the first photovoltaic panel plane) are selected to obtain the selected photovoltaic panel planes; based on the selected photovoltaic panel planes, the upper and lower edge line segments calculated from the corresponding plane contour point cloud are extracted, and then the line segments that meet the conditions (i.e., the target upper and lower edge line segments) and the planes that meet the conditions (i.e., the target photovoltaic panel planes) are selected by using parallel relationships and an interval of 4.5 meters between the upper and lower edge line segments.

[0095] 2) Ambiguous data filtering using periodic data (PV module support columns in point cloud). Multiple support columns can be calculated from each frame of point cloud data. Other similar columnar point clouds may exist in the environment, and the columns and their central axes can also be extracted. Ambiguous data filtering is performed based on the periodic distribution of PV module supports. Specific steps include: For example, the photovoltaic support columns are fixed at 2.5-meter intervals, and multiple columns were identified during the actual testing process; Multiple pillars detected at any time can be discarded due to ambiguity caused by differences in pillar spacing and discrepancies with actual conditions. By using periodic interval distances, pillars in consecutive frames can be better matched to obtain the pillars that match between consecutive frames (i.e., the target pillars).

[0096] 4. Construct the constraint factors for the plane normal vector, the constraint factors for the upper and lower edge lines, and the column constraint factors for the photovoltaic support.

[0097] (1) After obtaining the target photovoltaic panel plane and the target upper and lower edge line segments after screening, the scale drift of the laser odometry is corrected by the constraint factor of the photovoltaic panel and the constraint factor of the upper and lower edge lines respectively: The constraints for constructing the plane normal vector include: 1) Calculate the photovoltaic plane normal vector corresponding to the target photovoltaic panel plane. As an observation result, that is, it is possible to obtain the observation result of the previous frame. and the observation results of the current frame ; 2) Based on the pose transformation obtained above It is possible to calculate the robot's performance. Position at any given moment: (i.e., the transformation from the global coordinate system to the robot coordinate system) (Transformation from robot coordinate system to global coordinate system). It should be noted that the transformation from robot coordinate system to global coordinate system uses techniques known to those skilled in the art, and this embodiment will not describe it in detail.

[0098] 3) Through pose at time and The observation results of the photovoltaic plane normal vector at time t can predict The result of the normal vector at time step is the prediction result. ; 4) Construct the target residual for optimization (i.e., the constraint factor of the plane normal vector): ; Construct target residuals to optimize This ensures that the observed results and the predicted results are as close as possible.

[0099] Constructing constraint factors for upper and lower edge lines: Obtain the straight line equations of the upper and lower edges. The construction idea is the same as that for the constraint factors of the plane normal vector. However, the constraint factors for the upper and lower edge lines are the target upper and lower edge line segments. The target upper and lower edge line segments are used to replace the photovoltaic plane normal vector for calculation. This embodiment will not describe this in detail.

[0100] (2) Based on the matching of the columns in the previous and next frames, perform absolute scale and global correction: The spacing between the support columns is fixed and known. By detecting and tracking multiple columns, the scale drift of the odometer can be directly corrected, which is a very important parameter for monocular vision or pure laser odometers.

[0101] 1) The calculated center axis of the column is used as the observation result, which means that the observation result of the previous frame can be obtained. and the observation results of the current frame ; 2) Based on the pose transformation obtained above It is possible to calculate the robot's performance. Position at any given moment: (i.e., the transformation from the global coordinate system to the robot coordinate system) (Transformation from robot coordinate system to global coordinate system).

[0102] 3) The central axis of the column can be obtained at any given moment. (Observation results in the robot coordinate system) can then yield a set of robot poses and the central axis of the column at multiple moments; 4) Construct the prediction result of the central axis of the column, which is based on the current vehicle pose. Multiplying this by the observation result of the pillar in the previous frame yields: ; 5) Construct the optimized target residual (i.e., the column constraint factor of the photovoltaic support): ; Construct target residuals to optimize This ensures that the observed results and the predicted results are as close as possible.

[0103] The above steps can be used to constrain and optimize the scale drift of the odometer, thereby correcting the scale drift of the laser odometer.

[0104] 5. The calibration process of the improved laser odometer in photovoltaic power station scenarios.

[0105] The above steps have established the planar normal vector constraint factors, the upper and lower edge line constraint factors, and the column constraint factors of the photovoltaic panel. A factor diagram for laser odometer optimization can be constructed based on these multiple factors, as shown in the reference diagram. Figure 4 Among them, the column observation factor, panel observation factor, and edge observation factor correspond to the observation results of the central axis of the column, the photovoltaic plane normal vector, and the upper and lower edge lines, respectively. The column spacing constraint, plane parallelism constraint, and panel width constraint correspond to the column spacing, parallelism, and upper and lower edge line segment interval used for ambiguous data filtering. The specific calibration process of the improved laser odometry is as follows: 1. When the laser odometer is started, The position of the moment as the initial The position (i.e., the origin of the coordinate system), the integral of the inertial measurement unit (IMU) is The coordinates of the top and bottom edges of the photovoltaic panel and the center axis of the support column are identified. , .

[0106] 2. At any time, perform IMU to The integration factor can be obtained by accumulating the integrals at each time step. The laser odometry factor can be obtained by feature matching between consecutive frames of the lidar point cloud (i.e., the content described in step one above). Optimization can be achieved by combining the IMU pre-integration factor and the lidar odometry factor. accurate pose at any moment (that is) to (Coordinate transformation at time), through Can , Converted to Get it at any time , , It can be identified at any time , By constructing optimization factors such as columns, top and bottom edge lines, and photovoltaic panel planes, further optimization can be achieved. Obtain the accurately corrected pose. ,according to renew and Switch to the global coordinate system to prepare for the next prediction update.

[0107] 6. Path planning for robots in photovoltaic power plant scenarios.

[0108] After obtaining the calibrated laser odometry, path planning is then performed: In a photovoltaic power plant scenario, the calibrated laser odometry is used to transform the laser point clouds of multiple consecutive frames into the same coordinate system, resulting in a local map of the point clouds across multiple frames. Specifically: The preceding steps have yielded a calibrated laser odometry, enabling the acquisition of the robot pose transformation matrix between consecutive frames. To obtain a local map of point clouds across multiple frames, a time-sliding window can be used, for example, the current time is... Then take Take 10 consecutive frames as a window, and then transform all poses of the point cloud in the 10 consecutive frames to... In the coordinate system at a given time, the transformed point clouds are then superimposed using the transformation matrix of the corresponding frame to obtain a local map of multiple point clouds. Calculating the local map point clouds is to obtain more point clouds and reduce errors.

[0109] Stable photovoltaic plane normal vectors and cubic polynomials of the upper and lower edge trajectories of photovoltaic modules can be calculated in the local map. xy is the coordinate of the curve under the 2D top-down view of the robot walking, with the origin being the robot itself. a, b, c, and d are the parameters of the curve equation (which can be obtained by fitting the lower edge point cloud). Specifically, the plane can be extracted using RANSAC, and then the lower edge trajectory of the photovoltaic module can be obtained by fitting the upper and lower edges of the obtained point cloud plane using the least squares method.

[0110] In this embodiment, the point cloud is first projected onto the 2D plane where the robot walks before the solution is performed. This curve is basically a straight line, which reduces computational complexity. Extracting the point cloud plane first is to remove points outside the photovoltaic panel plane. Then, the upper and lower edges are extracted based on the plane contour points. Finally, the trajectory curve is solved by fitting the upper and lower edges, which can improve the accuracy of the upper and lower edge trajectories.

[0111] Using the calculated lower edge trajectory of the photovoltaic module, path planning for robot navigation can be performed, enabling autonomous navigation of the robot in photovoltaic scenarios. Specifically: The robot walks along a trajectory parallel to the lower edge of the photovoltaic panel. The robot can also be positioned at a fixed distance (e.g., 1 meter) off the lower edge. By aligning the robot's navigation path with this 1-meter offset lower edge trajectory, the robot's walking trajectory (i.e., the target planned path) can be obtained. Using this information, the robot can achieve mapless navigation, autonomous localization, and target path planning within a photovoltaic power plant by walking along the lower edge trajectory of the module.

[0112] Compared with the prior art, the technical solution of this embodiment has the following advantages: This system achieves precise localization without a map in this scenario and enables trajectory planning, allowing the robot to autonomously and reliably perform localization, trajectory planning, and control in a photovoltaic power station environment. It eliminates the high cost of map creation in this scenario, significantly saving time, and also achieves the fusion of laser and vision technologies in this context.

[0113] Reference Figure 5 This application also provides an outdoor mapless navigation and positioning path planning system, which includes: The transformation relationship calibration unit 501 is used to bind the lidar and vision sensor to the robot and calibrate the coordinate transformation relationship between the lidar and vision sensor. The first feature extraction unit 502 is used to acquire three-dimensional point cloud data of photovoltaic modules through lidar and to acquire images of photovoltaic modules through a visual sensor; to extract lidar features from the three-dimensional point cloud data and to extract visual features from the images. The feature pair association unit 503 is used to associate features between two adjacent frames according to the coordinate transformation relationship and laser features or visual features to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. The pose transformation calculation unit 504 is used to calculate the error between two adjacent frames based on the associated feature pairs, and to calculate the pose transformation of the robot based on the error. The first data filtering unit 505 is used to segment the photovoltaic module area from the image and project the three-dimensional point cloud data onto the photovoltaic module area through coordinate transformation relationship to obtain effective three-dimensional point cloud data. The second feature extraction unit 506 is used to extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support from the effective three-dimensional point cloud data. The second data filtering unit 507 is used to determine the target photovoltaic panel plane and the target upper and lower edge line segments based on the photovoltaic panel plane features and the photovoltaic panel upper and lower edge line features, and to determine the target column based on the photovoltaic support column features; The path planning unit 508 is used to correct the scale drift of the laser odometer based on the plane of the target photovoltaic panel, the upper and lower edge segments of the target, and the target column, to obtain the corrected laser odometer, and to use the corrected laser odometer for path planning to obtain the target planned path.

[0114] It should be noted that since the outdoor mapless navigation and positioning path planning system in this embodiment is based on the same inventive concept as the outdoor mapless navigation and positioning path planning method described above, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0115] Reference Figure 6 This application also provides an electronic device, which includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the outdoor mapless navigation positioning path planning method described above in this disclosure.

[0116] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0117] The electronic devices according to embodiments of this application will now be described in detail.

[0118] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the outdoor mapless navigation positioning path planning method of the embodiments of this disclosure.

[0119] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0120] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described outdoor mapless navigation positioning path planning method.

[0121] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0122] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0123] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0124] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0126] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0127] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0131] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of this application.

[0132] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for outdoor mapless navigation and positioning path planning, characterized in that, The method includes: The lidar and vision sensor are attached to the robot, and the coordinate transformation relationship between the lidar and the vision sensor is calibrated. The system acquires three-dimensional point cloud data of the photovoltaic module using the lidar and images of the photovoltaic module using the vision sensor; it also extracts laser features from the three-dimensional point cloud data and visual features from the images. Based on the coordinate transformation relationship and the laser feature or the visual feature, the features between two adjacent frames are associated to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. Based on the associated feature pairs, the error between two adjacent frames is calculated, and based on the error, the pose transformation of the robot is calculated. The photovoltaic module region is segmented from the image, and the three-dimensional point cloud data is projected onto the photovoltaic module region through the coordinate transformation relationship to obtain effective three-dimensional point cloud data; Extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the columnar features of the photovoltaic support from the effective three-dimensional point cloud data; Based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel, the target photovoltaic panel planar features and the target upper and lower edge line segments are determined; and based on the columnar features of the photovoltaic support, the target column is determined. Based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column, the scale drift of the laser odometry is corrected to obtain a corrected laser odometry. The corrected laser odometry is then used for path planning to obtain the target planned path.

2. The outdoor mapless navigation and positioning path planning method according to claim 1, characterized in that, The step of determining the target photovoltaic panel plane and the target upper and lower edge line segments based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel includes: Based on the photovoltaic panel planar features, a first photovoltaic panel planar feature that meets a preset angle is selected, wherein the photovoltaic panel planar feature includes a planar normal vector; Based on the characteristics of the upper and lower edge lines of the photovoltaic panel, extract the first upper and lower edge line segments corresponding to the plane of the first photovoltaic panel; Select target upper and lower edge segments and target photovoltaic panel plane that meet the first preset conditions from the first upper and lower edge segments.

3. The outdoor mapless navigation and positioning path planning method according to claim 1, characterized in that, The step of determining the target column based on the column-like characteristics of the photovoltaic support includes: Based on the column-like characteristics of the photovoltaic support structure, multiple columns that meet the second preset conditions are selected. Perform frame-by-frame matching on the multiple columns to obtain the columns after frame-by-frame matching, and use the columns after frame-by-frame matching as the target columns.

4. The outdoor mapless navigation and positioning path planning method according to claim 1, characterized in that, The step of correcting the scale drift of the laser odometer based on the target photovoltaic panel plane, the upper and lower edge segments of the target, and the target column to obtain the corrected laser odometer includes: Based on the photovoltaic panel plane features corresponding to the target photovoltaic panel plane, a constraint factor for the plane normal vector is constructed, wherein the photovoltaic panel plane features include the photovoltaic plane normal vector; Based on the characteristics of the upper and lower edge lines of the photovoltaic panel corresponding to the target upper and lower edge line segments, a constraint factor for the upper and lower edge lines is constructed, wherein the characteristics of the upper and lower edge lines of the photovoltaic panel are the extracted upper and lower edge line segments of the photovoltaic panel. Based on the column-shaped features of the photovoltaic support corresponding to the target column, a column constraint factor for the photovoltaic support is constructed, wherein the column-shaped features of the photovoltaic support are the central axis of the column. The scale drift of the laser odometer is corrected by minimizing the constraint factors of the plane normal vector, the upper and lower edge lines, and the column constraint factors of the photovoltaic support, thus obtaining the corrected laser odometer.

5. The outdoor mapless navigation and positioning path planning method according to claim 4, characterized in that, The constraint factor for constructing the plane normal vector based on the photovoltaic panel plane features corresponding to the target photovoltaic panel plane includes: Using the photovoltaic plane normal vector corresponding to the target photovoltaic panel plane as the first observation result, the first observation result of the previous frame and the first observation result of the current frame are obtained. The robot's pose at the current moment is determined based on the robot's pose change. The robot pose at the current moment is transformed into the global coordinate system to obtain the robot pose in the global coordinate system; Multiply the first observation result of the previous frame with the robot pose in the global coordinate system to obtain the first prediction result of the current frame; The constraint factor for the plane normal vector is constructed by subtracting the first observation result and the first prediction result of the current frame.

6. The outdoor mapless navigation and positioning path planning method according to claim 4, characterized in that, The construction of the photovoltaic support column constraint factor based on the column-shaped characteristics of the photovoltaic support corresponding to the target column includes: Using the central axis of the pillar corresponding to the target pillar as the second observation result, the second observation result of the previous frame and the second observation result of the current frame are obtained. The robot's pose at the current moment is determined based on the robot's pose change. The robot pose at the current moment is transformed into the global coordinate system to obtain the robot pose in the global coordinate system; Multiply the second observation result of the previous frame with the robot pose in the global coordinate system to obtain the second prediction result of the current frame; The column constraint factor of the photovoltaic support is constructed by subtracting the second observation result of the current frame from the second prediction result of the current frame.

7. The outdoor mapless navigation and positioning path planning method according to claim 1, characterized in that, The step of using the corrected laser odometry for path planning to obtain the target planned path includes: Using the corrected laser odometry, laser point clouds from multiple consecutive moments are transformed to the same coordinate system to construct local maps of multiple point clouds. Calculate the target photovoltaic plane normal vector in the local map of the multiple point clouds, and extract the target point cloud plane of the photovoltaic module through the target photovoltaic plane normal vector; Based on the target point cloud plane of the photovoltaic module, extract the upper and lower edge line segments in the target point cloud plane, and fit the upper and lower edge line segments in the target point cloud plane to obtain the lower edge trajectory of the photovoltaic module; Outdoor mapless navigation and positioning path planning is performed based on the trajectory of the lower edge of the photovoltaic module.

8. An outdoor mapless navigation and positioning path planning system, characterized in that, The system includes: A transformation relationship calibration unit is used to bind the lidar and vision sensor to the robot and calibrate the coordinate transformation relationship between the lidar and the vision sensor; The first feature extraction unit is used to acquire three-dimensional point cloud data of the photovoltaic module through the lidar, and to acquire images of the photovoltaic module through the vision sensor; to extract the laser features in the three-dimensional point cloud data, and to extract the visual features in the images; The feature pair association unit is used to associate features between two adjacent frames according to the coordinate transformation relationship and the laser feature or the visual feature to obtain associated feature pairs. The associated feature pairs include associated feature pairs between laser features and associated feature pairs between laser features and visual features. The pose transformation calculation unit is used to calculate the error between two adjacent frames based on the associated feature pairs, and to calculate the pose transformation of the robot based on the error. The first data filtering unit is used to segment the photovoltaic module area from the image and project the three-dimensional point cloud data onto the photovoltaic module area through the coordinate transformation relationship to obtain effective three-dimensional point cloud data. The second feature extraction unit is used to extract the planar features of the photovoltaic panel, the upper and lower edge features of the photovoltaic panel, and the column-shaped features of the photovoltaic support from the effective three-dimensional point cloud data. The second data filtering unit is used to determine the target photovoltaic panel plane and the target upper and lower edge line segments based on the planar features of the photovoltaic panel and the upper and lower edge line features of the photovoltaic panel, and to determine the target column based on the column-shaped features of the photovoltaic support. The path planning unit is used to correct the scale drift of the laser odometer based on the target photovoltaic panel plane, the target upper and lower edge segments, and the target column to obtain the corrected laser odometer, and to use the corrected laser odometer for path planning to obtain the target planned path.

9. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the outdoor mapless navigation positioning path planning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the outdoor mapless navigation and positioning path planning method as described in any one of claims 1 to 7.

Images