Photovoltaic panel installation equipment aided navigation method, device, equipment and medium

Abstract

The invention discloses an auxiliary navigation method, device and equipment for photovoltaic panel installation equipment and a medium. Comprising the steps of obtaining point cloud data of a target operation area, performing photovoltaic support candidate grid identification by using a deep learning model based on the point cloud data, and generating an identification result; generating an initial semantic occupation map according to an identification result, and dynamically updating in combination with the updated point cloud data; fitting a navigation base line according to the dynamically updated semantic occupation map, and controlling the photovoltaic panel mounting equipment to move based on the navigation base line. The photovoltaic support candidate grid identification is carried out by using the deep learning model, the defect that a traditional visual sensor is influenced by illumination is overcome, and the robustness of support identification is improved. By dynamically updating the semantic occupation map, the spatial distribution change of the photovoltaic support can be reflected in real time, and the timeliness of the map is ensured. The photovoltaic panel installation equipment is controlled to move based on the navigation base line, it can be ensured that the advancing axis of the equipment is parallel to the trend of the support, the positioning precision is improved, and the operation efficiency of installation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of photovoltaic panel installation, in particular to a photovoltaic panel installation equipment auxiliary navigation method, device, equipment and medium. BACKGROUND

[0002] In the photovoltaic power generation base, most of which are located in vast desertification land, it is suitable to build large-scale photovoltaic arrays, and in construction, automatic photovoltaic installation equipment relying on walking devices is needed to complete component installation. The three-dimensional posture of the equipment needs to be monitored in real time when it is moving, and the moving axis needs to be parallel to the trend of the photovoltaic panel support to ensure the installation positioning accuracy and operation stability. However, there are complex working conditions such as surface undulation and strong wind interference in the working site, and most of them are in the unsheltered and unmanned Gobi, so there is an urgent need for high-precision autonomous navigation technology.

[0003] The existing auxiliary navigation technology often relies on an initial high-precision map, and the heading correction is realized by identifying a preset reference system, similar to the lane line keeping function. The global reference is provided by a satellite navigation and inertial navigation combined system, and the local environment perception uses monocular, binocular vision or millimeter wave radar technology.

[0004] However, in the photovoltaic construction scene, the continuous boundary of the support array has no structured road and no visually recognizable standard reference, which causes the traditional lane line auxiliary model to fail. The satellite navigation signal in the Gobi area is easily disturbed, resulting in positioning deviation, and the millimeter-level precision requires a high-cost ground-based enhancement system, and there is a problem of incomplete coverage. The feature points are overexposed or distorted due to the influence of strong sunlight on the visual sensor, and the millimeter wave radar lacks targeted modeling and cannot distinguish between supports, personnel and vehicles, so the robustness and economy of the existing perception system cannot meet the engineering requirements. SUMMARY

[0005] The present application provides a photovoltaic panel installation equipment auxiliary navigation method, device, equipment and medium, a vehicle and photovoltaic support parallelism detection method based on deep learning, which solves the problems of insufficient vehicle body posture calibration accuracy and environmental interference sensitivity in automatic construction.

[0006] According to one aspect of the present application, a photovoltaic panel installation equipment auxiliary navigation method, device, equipment and medium are provided, and the method comprises:

[0007] Obtaining point cloud data of a target working area, using a deep learning model based on the point cloud data to identify a photovoltaic support candidate grid, and generating an identification result;

[0008] Generating an initial semantic occupancy map according to the identification result, and dynamically updating in combination with updated point cloud data;

[0009] Fitting a navigation baseline according to the dynamically updated semantic occupancy map, and controlling the movement of the photovoltaic panel installation equipment based on the navigation baseline.

[0010] Optionally, the point cloud data of the target operation area is acquired, including: collecting original point cloud of the target operation area by a laser radar; obtaining a preset local map boundary, and retaining the original point cloud within the local map boundary range; determining a point cloud average value based on a preset period, and removing dynamic obstacles through time difference to generate the point cloud data.

[0011] Optionally, the deep learning model is used for photovoltaic support candidate grid identification based on the point cloud data to generate an identification result, including: dividing the point cloud data into voxel grids, performing down-sampling processing on the point cloud data in each voxel grid to generate down-sampled data; generating a feature vector of each voxel grid according to the down-sampled data; performing photovoltaic support candidate grid identification on the feature vector through the deep learning model, adopting a cross-entropy loss function for binary classification, and outputting the identification result, wherein the identification result includes a support installation area and a non-support installation area.

[0012] Optionally, the feature vector of each voxel grid is generated according to the down-sampled data, including: determining the voxel index corresponding to each point cloud data in the down-sampled data, and generating a key-value pair according to the voxel index; based on the key-value pair, the point cloud is distributed to the corresponding voxel grid by using hash mapping, and the statistical quantity of each voxel grid is calculated to generate the feature vector of the voxel grid.

[0013] Optionally, an initial semantic occupancy map is generated according to the identification result, and is dynamically updated in combination with updated point cloud data, including: marking the support installation area in the identification result as an occupancy state, marking the non-support installation area as an idle state, and outputting a binary occupancy map in a matrix form; obtaining updated point cloud data in a movement process of a photovoltaic panel installation device, updating grid occupancy probability based on the updated point cloud data through Bayesian filtering, and dynamically updating the binary occupancy map in combination with Kalman filtering, wherein the Kalman filtering includes a decay factor of dynamic weight adjustment.

[0014] Optionally, a navigation baseline is fitted according to the dynamically updated semantic occupancy map, including: segmenting real-time point cloud data according to the dynamically updated semantic occupancy map, separately performing point cloud clustering on point cloud blocks in the occupancy state, extracting support point cloud and calculating a centroid; projecting the centroid to a plane where the photovoltaic panel installation device is located, and fitting a straight line equation of each centroid by using a least square method to determine the navigation baseline.

[0015] Optionally, the navigation baseline-based control of the movement of the photovoltaic panel installation equipment comprises: acquiring a current equipment pose of the photovoltaic panel installation equipment, determining a vehicle body pose transfer matrix according to the current equipment pose; converting the center of mass into an equipment coordinate system in a horizontal state according to the vehicle body pose transfer matrix, and generating a conversion coordinate; calculating navigation parameters of the photovoltaic panel installation equipment to the navigation baseline according to the conversion coordinate, wherein the navigation parameters comprise a distance, a heading angle and a walking distance; and based on the navigation parameters, adjusting a walking device of the photovoltaic installation equipment in real time to control parallel movement of the photovoltaic panel installation equipment based on the navigation baseline.

[0016] According to another aspect of the present application, there is provided a photovoltaic panel installation equipment assisted navigation device, which comprises:

[0017] An identification result generation module is configured to acquire point cloud data of a target work area, perform photovoltaic support candidate grid identification based on the point cloud data using a deep learning model, and generate an identification result.

[0018] A map generation and update module is configured to generate an initial semantic occupancy map according to the identification result, and dynamically update the initial semantic occupancy map in combination with updated point cloud data.

[0019] An installation equipment navigation module is configured to fit a navigation baseline according to the dynamically updated semantic occupancy map, and control movement of the photovoltaic panel installation equipment based on the navigation baseline.

[0020] According to another aspect of the present application, there is provided an electronic device, which comprises:

[0021] at least one processor;

[0022] and a memory connected in communication with the at least one processor;

[0023] wherein the memory stores a computer program capable of being executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute a photovoltaic panel installation equipment assisted navigation method according to any one of the embodiments of the present application.

[0024] According to another aspect of the present application, there is provided a computer readable storage medium, which stores computer instructions for enabling a processor to execute a photovoltaic panel installation equipment assisted navigation method according to any one of the embodiments of the present application when the processor executes the computer instructions.

[0025] The technical scheme of the embodiment of the present application can accurately distinguish between installed support and non-support areas by using a deep learning model for photovoltaic support candidate grid identification, providing reliable basic identification information for subsequent navigation, effectively overcoming the defects of traditional visual sensors affected by light, and improving the robustness of support identification. By dynamically updating the semantic occupancy map, the spatial distribution changes of the photovoltaic support can be reflected in real time, ensuring the timeliness and accuracy of the map. Based on the navigation baseline, the photovoltaic panel installation equipment is controlled to move, which can ensure that the equipment running axis is parallel to the photovoltaic support direction, significantly improve the installation positioning accuracy and equipment running stability, and improve the work efficiency of photovoltaic support installation.

[0026] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0028] Figure 1 is a flow chart of a photovoltaic panel installation equipment auxiliary navigation method provided by the first embodiment of the present application;

[0029] Figure 2 is a flow chart of another photovoltaic panel installation equipment auxiliary navigation method provided by the second embodiment of the present application;

[0030] Figure 3 is a structural schematic diagram of a photovoltaic panel installation equipment auxiliary navigation device provided by the third embodiment of the present application;

[0031] Figure 4 is a structural schematic diagram of an electronic device for implementing a photovoltaic panel installation equipment auxiliary navigation method of the present application. DETAILED DESCRIPTION

[0032] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0033] It is to be understood that the terminology "first", "second", and the like used in the specification and the claims of the application as well as the preceding drawings is merely intended to distinguish between similar objects and is not intended to convey a specific order or sequence among, or prior to, the described embodiments. It is to be understood that the data thus described with respect to the embodiments of the application can be interchanged, in the appropriate cases, so that the embodiments of the application described herein can be carried out in orders other than those illustrated or described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a list of steps or units is not necessarily limited to those steps or units that are clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or apparatuses.

[0034] Embodiment one

[0035] Figure 1 A flowchart of a photovoltaic panel installation equipment auxiliary navigation method is provided for the first embodiment of the application. The embodiment can be applicable to a photovoltaic panel installation scene. The method can be executed by a photovoltaic panel installation equipment auxiliary navigation device, which can be realized in the form of hardware and / or software. The photovoltaic panel installation equipment auxiliary navigation device can be configured in a controller of the photovoltaic panel installation equipment. As shown in the figure, the method comprises: Figure 1

[0036] S110, acquiring point cloud data of a target work area, using a deep learning model to perform photovoltaic support candidate grid identification based on the point cloud data, and generating an identification result.

[0037] The target work area refers to a construction area of a photovoltaic panel installation. The point cloud data refers to three-dimensional point cloud information in a photovoltaic construction area collected by a laser radar. The deep learning model refers to a VoxelNet model, which is used to identify photovoltaic supports for processed point cloud data. The deep learning model receives a point cloud feature vector processed by voxelization and downsampling, performs binary classification through a cross-entropy loss function, and distinguishes candidate grids where supports are installed and non-support areas. The training data is labeled installation support areas and non-support areas, which can specifically identify photovoltaic supports and overcome the problem of insufficient robustness of traditional sensing methods in complex scenes. The photovoltaic support candidate grid is a grid that may contain photovoltaic supports identified by the VoxelNet model after the point cloud data is divided into voxel grids. The identification result refers to index information of the candidate grid where the photovoltaic support is located output by the VoxelNet model, which clearly indicates which voxel grid may contain photovoltaic supports.

[0038] ​Optionally, the point cloud data of the target working area is acquired, including: collecting original point cloud of the target working area by the laser radar; obtaining a preset local map boundary, and retaining the original point cloud within the local map boundary range; determining a point cloud average value based on a preset period, and removing dynamic obstacles by time difference to generate the point cloud data.

[0039] The original point cloud contains basic information such as coordinates and laser reflection intensity of points. Then, a preset local map boundary is obtained, for example, a range of 10m*10m*5m on the working side of the vehicle can be set, and the controller will only retain the original point cloud within this range, focusing on the working related area and reducing the subsequent calculation amount. Then, based on a preset initialization time, for example, 10s, the point cloud average value is determined, that is, the average value of multiple frames of point cloud within 10s is calculated. At the same time, dynamic obstacles are removed by time difference method, and the following formula (1) is used for calculation:

[0040] P static =P current \(P current ∩P static ) (1)

[0041] P current represents the current frame point cloud, and P static represents the static point cloud, so as to remove dynamic obstacles such as workers and other vehicles, and finally generate point cloud data for subsequent installation of support positioning.

[0042] S120, generating an initial semantic occupancy map according to the recognition result, and dynamically updating in combination with updated point cloud data.

[0043] The initial semantic occupancy map is a binary occupancy map formed by taking the coordinates of the photovoltaic support candidate grid as input, marking the grid where the support is located as an occupancy state, and marking the remaining grids as an idle state, and is in the form of a matrix. The initial semantic occupancy map is a preliminary digital representation of the distribution of photovoltaic supports in the working area, which completes the initialization state setting of the vehicle as a whole and provides an initial reference for subsequent dynamic updating. The updated point cloud data refers to new point cloud data collected by the laser radar in real time during the movement of the vehicle, which is used to dynamically update the semantic occupancy map to reflect the real-time changes of the environment, ensure the consistency of the map and the actual scene, and thus provide reliable environmental information for accurate navigation. The dynamically updated semantic occupancy map is a map obtained by real-time adjustment of the initial semantic occupancy map in combination with the updated point cloud data by methods such as Bayesian filtering and Kalman filtering. In the updating process, dynamic weight adjustment is adopted, a decay factor is added to the historical occupancy data, and the interference of old data is reduced, so as to ensure that the map can adapt to the movement of the vehicle and the changes of the environment, and maintain the real-time and accuracy of positioning.

[0044] Optionally, an initial semantic occupancy map is generated according to the recognition result, and is dynamically updated in combination with updated point cloud data, including: marking the installation support area in the recognition result as an occupancy state, marking the non-installation support area as an idle state, and outputting a binary occupancy map in matrix form; obtaining updated point cloud data in the movement process of the photovoltaic panel installation device, updating grid occupancy probability based on the updated point cloud data, and dynamically updating the binary occupancy map in combination with Kalman filtering, wherein the Kalman filtering includes a decay factor for dynamic weight adjustment.

[0045] Specifically, the controller will mark the grid corresponding to the installation support area as an occupancy state according to the recognition result output by the deep learning model VoxelNet, and mark the grid of the remaining non-installation support area as an idle state, and output a binary occupancy map in matrix form.

[0046] Then, in the movement process of the photovoltaic panel installation device, updated point cloud data collected by the laser radar in real time is obtained. Based on the updated point cloud data, the occupancy probability of the grid is updated by Bayesian filtering, and the following formula (2) is used:

[0047]

[0048] Wherein, P occ (t) represents the occupancy probability of the grid at time t, that is, the probability that the grid belongs to the target area such as photovoltaic support after updating at the current time, P(z t |occ) represents the conditional probability of obtaining the observation value z t at the current time when the grid is in the occupancy state, reflecting the support degree of the observation value to the occupancy state of the grid, P occ (t-1) represents the occupancy probability of the grid at time t-1, which is the probability information at the historical time, P(z t |free) represents the conditional probability of obtaining the observation value z t at the current time when the grid is in the idle state, and 1-P occ (t-1) represents the idle probability of the grid at time t-1. At the same time, the binary occupancy map is dynamically updated in combination with Kalman filtering, specifically, the state estimation of the t-1 frame is used to predict the state and covariance of the t frame, and then the prediction result is corrected by Kalman gain calculation according to the observation value z t of the t frame, and recursive processing is performed, and the output is taken as the input of the next frame. In addition, dynamic weight adjustment is adopted in Kalman covariance prediction, a decay factor is introduced, and the following formula (3) is used for calculation:

[0049]

[0050] Wherein, a represents the current decay factor, a basedenotes the initial decay factor, β is the decay rate coefficient, k denotes the current time point, k t denotes the time point of the last state mutation, by adding a decay factor to the historical occupancy data, the interference of old data is reduced, thereby realizing dynamic updating of the binary occupancy map.

[0051] S130, fitting a navigation baseline according to the dynamically updated semantic occupancy map, and controlling the photovoltaic panel installation equipment to move based on the navigation baseline.

[0052] Wherein, the navigation baseline refers to point cloud clustering of a point cloud block where the installation support is located according to the dynamically updated semantic occupancy map, extracting the support point cloud and calculating its centroid, and then fitting a straight line equation of the centroid coordinates of multiple supports by the least square method. The navigation baseline reflects the trend of the photovoltaic support, which is the baseline for controlling the photovoltaic panel installation equipment to move in parallel. The photovoltaic panel installation equipment refers to an automated equipment used for photovoltaic panel installation in a desert photovoltaic construction scene. The photovoltaic panel installation equipment needs to rely on the navigation baseline to control movement, to ensure that the axis of travel is always parallel to the trend of the photovoltaic panel support, so as to ensure installation positioning accuracy and equipment operation stability.

[0053] Optionally, fitting the navigation baseline according to the dynamically updated semantic occupancy map comprises: segmenting real-time point cloud data according to the dynamically updated semantic occupancy map, separately performing point cloud clustering on point cloud blocks in an occupied state, extracting support point clouds and calculating centroids; projecting the centroids to a plane where the photovoltaic panel installation equipment is located, and fitting a straight line equation of the centroids by the least square method to determine the navigation baseline.

[0054] Specifically, the controller can segment the real-time collected point cloud data according to the dynamically updated semantic occupancy map, and filter out point cloud blocks marked as an occupied state. The filtered blocks correspond to photovoltaic installation supports. Then, the point cloud blocks in the occupied state are separately clustered, thereby extracting point clouds belonging to photovoltaic supports, and calculating the centroid of the support point cloud. The coordinates of the centroid are calculated by the following formula (4):

[0055]

[0056] Wherein, x c denotes the horizontal coordinate of the centroid, y c denotes the vertical coordinate of the centroid, N denotes the number of points in the photovoltaic support point cloud, x i denotes the horizontal coordinate of the i-th point in the point cloud, y i denotes the vertical coordinate of the i-th point in the point cloud.

[0057] Further, the controller projects the calculated center of mass of each support to the plane on which the photovoltaic panel installation device is located, so that the center of mass position is consistent with the spatial reference system of the device. Finally, the least squares method is used to fit the coordinates of the center of mass of the plurality of supports to obtain a straight line equation, and the straight line fitted by the straight line equation is used as the navigation baseline, which reflects the trend of the photovoltaic support and provides a reference for the parallel movement of the photovoltaic panel installation device.

[0058] Optionally, the photovoltaic panel installation device is controlled to move based on the navigation baseline, including: obtaining a current device pose of the photovoltaic panel installation device, determining a vehicle pose transfer matrix according to the current device pose; converting the center of mass to the device coordinate system in the horizontal state according to the vehicle pose transfer matrix to generate a converted coordinate; calculating navigation parameters of the photovoltaic panel installation device to the navigation baseline according to the converted coordinate, wherein the navigation parameters include a distance, a heading angle and a walking distance; and adjusting the walking device of the photovoltaic installation device in real time based on the navigation parameters to control the photovoltaic panel installation device to move in parallel based on the navigation baseline.

[0059] Specifically, the controller can obtain the current device pose of the photovoltaic panel installation device through an inertial measurement unit (IMU), including a pitch angle, a heading angle and a roll angle, and then determine the vehicle pose transfer matrix according to the current device pose. According to the vehicle pose transfer matrix, the previously obtained center of mass coordinates are converted to the device coordinate system in the horizontal state to generate converted coordinates, so that the center of mass position corresponds to the horizontal reference system of the device, facilitating subsequent calculation. Then, the controller calculates the navigation parameters of the photovoltaic panel installation device to the navigation baseline according to the converted coordinates. The distance is the vertical distance from the vehicle to the navigation baseline, which is obtained by the point-to-line calculation formula, the heading angle is calculated according to the projection of the navigation baseline and the current vehicle heading in the vehicle horizontal plane, and the walking distance is the initial value distance of the current cement pile target tracking, which is integrated by the tracking time difference of a single support target, and the walking distance is reset after the device stops. Finally, based on the navigation parameters, the controller can adjust the walking device of the photovoltaic installation device in real time to ensure that the advancing axis of the device is always parallel to the navigation baseline, so as to control the photovoltaic panel installation device to move along a trajectory consistent with the trend of the photovoltaic support, and to ensure the installation positioning accuracy and the device running stability.

[0060] The technical scheme of the embodiment of the present application can accurately distinguish the installed support and the non-support area by using the deep learning model for photovoltaic support candidate grid identification, can provide reliable basic identification information for subsequent navigation, can effectively overcome the defects of the traditional visual sensor affected by light, and can improve the robustness of support identification. By dynamically updating the semantic occupancy map, the spatial distribution change of the photovoltaic support can be reflected in real time, and the timeliness and accuracy of the map can be ensured. Based on the navigation baseline, the movement of the photovoltaic panel installation equipment is controlled, the equipment running axis can be ensured to be parallel to the photovoltaic support trend, the installation positioning accuracy and equipment operation stability can be significantly improved, and the work efficiency of photovoltaic support installation can be improved.

[0061] Embodiment two

[0062] Figure 2 A flowchart of a photovoltaic panel installation equipment auxiliary navigation method provided by the second embodiment of the present application, the embodiment adds the specific process of using a deep learning model to identify photovoltaic support candidate grids based on point cloud data and generating identification results on the basis of the above-mentioned embodiment one. The specific content of steps S210, S250-S260 is substantially the same as that of steps S110-S130 in the first embodiment, and therefore will not be described again in the present embodiment. As shown in the figure, the method comprises the following steps. Figure 2

[0063] S210, acquiring point cloud data of a target work area.

[0064] Optionally, the point cloud data of the target work area is acquired, comprising: collecting original point cloud of the target work area by a laser radar; acquiring a preset local map boundary, and retaining the original point cloud within the local map boundary; determining the average value of the point cloud based on a preset period, and removing dynamic obstacles by time difference to generate the point cloud data.

[0065] S220, dividing the point cloud data into voxel grids, and performing down-sampling processing on the point cloud data in each voxel grid to generate down-sampled data.

[0066] Specifically, the controller can divide the processed point cloud data into voxel grids, and the size of each voxel grid is 0.2m. Then, the point cloud in each voxel grid is down-sampled, the point cloud near the installation height of the laser radar is retained, and the point cloud of other heights is thinned out, so as to reduce the calculation amount and generate the down-sampled data.

[0067] S230, generating a feature vector of each voxel grid according to the down-sampled data.

[0068] ​Specifically, according to the down-sampling data, the feature vector of each voxel grid is generated, specifically, according to the key-value pair, the point cloud is distributed to the corresponding voxel by using hash mapping, and the statistics of the point cloud in each voxel are calculated, such as covariance, height range, etc., and each statistic constitutes a feature vector, which contains the spatial distribution and attribute information of the point cloud.

[0069] Optionally, according to the down-sampling data, the feature vector of each voxel grid is generated, including: determining the voxel index corresponding to each point cloud data in the down-sampling data, and generating a key-value pair according to the voxel index; based on the key-value pair, the point cloud is distributed to the corresponding voxel grid by using hash mapping, and the statistics of each voxel grid are calculated to generate the feature vector of the voxel grid.

[0070] Specifically, for the down-sampled point cloud data, since the voxel grid is divided into a 0.2m grid, the index of the voxel to which each point belongs needs to be calculated, and then the point cloud data is associated with the corresponding voxel index in the form of a key-value pair to generate a key-value pair. Then, the controller can distribute the point cloud data to the corresponding voxel grid based on the generated key-value pair by using hash mapping, and calculate the statistics of the point cloud in each voxel grid, including covariance, height range, point cloud density, height difference, and average reflection intensity, etc. Each statistic constitutes a feature vector of the voxel grid, which is used for subsequent deep learning model recognition.

[0071] S240, identifying the feature vector by a deep learning model to obtain a photovoltaic support candidate grid, adopting a cross-entropy loss function for binary classification, and outputting an identification result, wherein the identification result includes an installed support region and a non-installed support region.

[0072] Specifically, the feature vector is identified by a deep learning model VoxelNet to obtain a photovoltaic support candidate grid. The deep learning model receives the feature vector, adopts a cross-entropy loss function for binary classification, wherein the loss function adopts the following formula (5):

[0073] L = -∑(y i (logp i +(1-y i )log(1-p i )) (5)

[0074] Wherein, L represents a loss value, used to measure the difference between the prediction result of the deep learning model and the true result, y i represents the true class of sample i, if it is an installed support region, it takes value 1, and if it is a non-installed support region, it takes value 0, p i represents the probability that the deep learning model predicts sample i as a positive class, i.e. an installed support region. Finally, the identification result is output, which clearly distinguishes the candidate grid where the support is installed and the grid where the support is not installed, and outputs the index of the candidate grid.

[0075] S250, generating an initial semantic occupancy map according to the recognition result, and dynamically updating in combination with updated point cloud data.

[0076] Optionally, generating an initial semantic occupancy map according to the recognition result, and dynamically updating in combination with updated point cloud data, comprises: marking the installation support area in the recognition result as an occupancy state, marking the non-installation support area as an idle state, and outputting a binary occupancy map in matrix form; obtaining updated point cloud data in the movement process of the photovoltaic panel installation device, updating grid occupancy probability based on the updated point cloud data, and dynamically updating the binary occupancy map in combination with Kalman filtering, wherein the Kalman filtering comprises a decay factor for dynamic weight adjustment.

[0077] S260, fitting a navigation baseline according to the dynamically updated semantic occupancy map, and controlling the movement of the photovoltaic panel installation device based on the navigation baseline.

[0078] Optionally, fitting a navigation baseline according to the dynamically updated semantic occupancy map comprises: segmenting real-time point cloud data according to the dynamically updated semantic occupancy map, separately performing point cloud clustering on point cloud blocks in the occupancy state, extracting support point clouds and calculating the centroid; projecting the centroid to the plane where the photovoltaic panel installation device is located, and using the least squares method to fit a straight line equation of each centroid to determine the navigation baseline.

[0079] Optionally, controlling the movement of the photovoltaic panel installation device based on the navigation baseline comprises: obtaining the current device pose of the photovoltaic panel installation device, determining the vehicle body pose transfer matrix according to the current device pose; converting the centroid to the device coordinate system in the horizontal state according to the vehicle body pose transfer matrix to generate a conversion coordinate; calculating navigation parameters of the photovoltaic panel installation device to the navigation baseline according to the conversion coordinate, wherein the navigation parameters comprise distance, heading angle and walking distance; based on the navigation parameters, adjusting the walking device of the photovoltaic installation device in real time to control the parallel movement of the photovoltaic panel installation device based on the navigation baseline.

[0080] The technical scheme of the embodiment of the application divides the point cloud data into voxel grids, performs downsampling processing on the point cloud data in each voxel grid to generate downsampling data, which can reduce the amount of calculation while retaining key information and improving the efficiency of subsequent processing. By generating feature vectors of each voxel grid, the spatial distribution and attribute features of the point cloud can be extracted to provide effective input for the recognition of the deep learning model. The deep learning model is used to identify the feature vectors of the photovoltaic support candidate grid, a cross-entropy loss function is used for binary classification, and the recognition result is output, which can accurately distinguish between the installation support area and the non-installation support area, and provide reliable basic identification information for subsequent navigation.

[0081] Embodiment three

[0082] Figure 3 A structural schematic diagram of an auxiliary navigation device of a photovoltaic panel installation equipment is provided for embodiment three of the application. As shown, the device comprises: an identification result generation module 310, configured to acquire point cloud data of a target work area, perform photovoltaic support candidate grid identification based on the point cloud data using a deep learning model, and generate an identification result; Figure 3

[0083] a map generation and update module 320, configured to generate an initial semantic occupancy map according to the identification result, and dynamically update in combination with updated point cloud data;

[0084] an installation equipment navigation module 330, configured to fit a navigation baseline according to the dynamically updated semantic occupancy map, and control movement of the photovoltaic panel installation equipment based on the navigation baseline.

[0085] Optionally, the identification result generation module 310 specifically comprises: a point cloud data acquisition unit, configured to: collect original point cloud of the target work area through a laser radar; acquire a preset local map boundary, and retain original point cloud within the local map boundary; determine a point cloud average value based on a preset period, and remove dynamic obstacles through time difference to generate point cloud data.

[0086] Optionally, the identification result generation module 310 specifically comprises: an identification result generation unit, configured to: divide the point cloud data into voxel grids, perform down-sampling processing on point cloud data in each voxel grid to generate down-sampling data; generate a feature vector of each voxel grid according to the down-sampling data; perform photovoltaic support candidate grid identification on the feature vector through a deep learning model, perform two classification using a cross-entropy loss function, and output an identification result, wherein the identification result comprises an installation support area and a non-installation support area.

[0087] Optionally, the identification result generation unit specifically comprises: a feature vector determination sub-unit, configured to: determine a voxel index corresponding to each point cloud data in the down-sampling data, and generate a key-value pair according to the voxel index; based on the key-value pair, assign point cloud to a corresponding voxel grid using hash mapping, and calculate a statistic of each voxel grid to generate a feature vector of the voxel grid.

[0088] Optionally, the map generation and update module 320 is specifically configured to: mark the installation support area in the identification result as an occupancy state, mark the non-installation support area as an idle state, and output a binary occupancy map in a matrix form; acquire updated point cloud data in a movement process of the photovoltaic panel installation equipment, update grid occupancy probability through Bayesian filtering based on the updated point cloud data, and dynamically update the binary occupancy map in combination with Kalman filtering, wherein the Kalman filtering comprises a decay factor of dynamic weight adjustment.

[0089] ​Optionally, the installation equipment navigation module 330 specifically comprises: a navigation baseline fitting unit, configured to: according to the dynamic updated semantic occupancy map, segment the real-time point cloud data, separately perform point cloud clustering on the point cloud blocks of the occupancy state, extract the support point cloud and calculate the centroid; project the centroid to the plane where the photovoltaic panel installation equipment is located, and adopt the least square method to fit the straight line equation of each centroid to determine the navigation baseline.

[0090] Optionally, the installation equipment navigation module 330 specifically comprises: an installation equipment control unit, configured to: acquire the current equipment pose of the photovoltaic panel installation equipment, determine the vehicle body pose transfer matrix according to the current equipment pose; convert the centroid into the equipment coordinate system in the horizontal state according to the vehicle body pose transfer matrix to generate a conversion coordinate; calculate the navigation parameters of the photovoltaic panel installation equipment to the navigation baseline according to the conversion coordinate, wherein the navigation parameters comprise the distance, the heading angle and the walking distance; and based on the navigation parameters, adjust the walking device of the photovoltaic installation equipment in real time to control the photovoltaic panel installation equipment to move in parallel based on the navigation baseline.

[0091] The technical scheme of the embodiment of the application can accurately distinguish between the installation support and the non-support area by using the deep learning model to identify the photovoltaic support candidate grid, can provide reliable basic identification information for subsequent navigation, can effectively overcome the defects of the traditional visual sensor affected by light, and can improve the robustness of support identification. By dynamically updating the semantic occupancy map, the spatial distribution change of the photovoltaic support can be reflected in real time, and the timeliness and accuracy of the map can be ensured. Based on the navigation baseline, the movement of the photovoltaic panel installation equipment can be controlled, the equipment advancing axis can be ensured to be parallel to the photovoltaic support trend, the installation positioning accuracy and the equipment operation stability can be significantly improved, and the work efficiency of the photovoltaic support installation can be improved.

[0092] The photovoltaic panel installation equipment auxiliary navigation device provided by the embodiment of the application can execute the photovoltaic panel installation equipment auxiliary navigation method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0093] Embodiment four

[0094] Figure 4 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the applications described and / or claimed in this document.

[0095] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0096] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a method for assisted navigation of a photovoltaic panel installation device.

[0098] In some embodiments, a photovoltaic panel mounting equipment assisted navigation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the photovoltaic panel mounting equipment assisted navigation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a photovoltaic panel mounting equipment assisted navigation method by any other suitable means (e.g., by means of firmware).

[0099] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0100] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package and partially on a remote machine or entirely on a remote machine or server.

[0101] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0104] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0105] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for auxiliary navigation of photovoltaic panel installation equipment, characterized in that, include: Acquire point cloud data of the target work area, and use a deep learning model to identify candidate grids for photovoltaic support based on the point cloud data, and generate identification results; An initial semantic occupancy map is generated based on the recognition results and dynamically updated using updated point cloud data; A navigation baseline is fitted based on a dynamically updated semantic occupancy map, and the movement of the photovoltaic panel installation equipment is controlled based on the navigation baseline.

2. The method according to claim 1, characterized in that, The acquisition of point cloud data of the target work area includes: The raw point cloud of the target work area is collected using lidar; Obtain a preset local map boundary and retain the original point cloud within the local map boundary range; The point cloud average value is determined based on a preset period, and dynamic obstacles are removed by time difference to generate point cloud data.

3. The method according to claim 1, characterized in that, The process of using a deep learning model to identify candidate grids for photovoltaic supports based on the point cloud data and generating identification results includes: The point cloud data is divided into voxel grids, and the point cloud data in each voxel grid is downsampled to generate downsampled data. Generate feature vectors for each voxel grid based on the downsampled data; The feature vector is used to identify photovoltaic support candidate grids through a deep learning model. The cross-entropy loss function is used for binary classification, and the identification results are output. The identification results include the area where the support is installed and the area where the support is not installed.

4. The method according to claim 3, characterized in that, The step of generating feature vectors for each voxel grid based on the downsampled data includes: Determine the voxel index corresponding to each point cloud data in the downsampled data, and generate key-value pairs based on the voxel index; Based on the key-value pairs, the point cloud is assigned to the corresponding voxel grid using hash mapping, and the statistics of each voxel grid are calculated to generate the feature vector of the voxel grid.

5. The method according to claim 3, characterized in that, The step of generating an initial semantic occupancy map based on the recognition results and dynamically updating it in conjunction with updated point cloud data includes: The areas with mounting brackets in the identification results are marked as occupied, and the areas without mounting brackets are marked as idle, and a binary occupancy map in matrix form is output. The updated point cloud data during the movement of the photovoltaic panel installation equipment is acquired. Based on the updated point cloud data, the grid occupancy probability is updated by Bayesian filtering, and the binary occupancy map is dynamically updated by Kalman filtering. The Kalman filtering includes a decay factor with dynamic weight adjustment.

6. The method according to claim 5, characterized in that, The process of fitting a navigation baseline based on a dynamically updated semantic occupancy map includes: Real-time point cloud data is segmented based on dynamically updated semantic occupancy map. Point cloud blocks in the occupancy state are clustered separately to extract support point clouds and calculate centroids. The centroids are projected onto the plane where the photovoltaic panel installation equipment is located, and the least squares method is used to fit the linear equations of each centroid to determine the navigation baseline.

7. The method according to claim 6, characterized in that, The method of controlling the movement of the photovoltaic panel installation equipment based on the navigation baseline includes: Obtain the current device pose of the photovoltaic panel installation equipment, and determine the vehicle body pose transfer matrix based on the current device pose; Based on the vehicle body pose transfer matrix, the centroid is transformed into the device coordinate system in a horizontal state, generating transformed coordinates; Based on the transformed coordinates, the navigation parameters from the photovoltaic panel installation equipment to the navigation baseline are calculated, wherein the navigation parameters include spacing, heading angle and travel distance; Based on the navigation parameters, the walking device of the photovoltaic installation equipment is adjusted in real time to control the photovoltaic panel installation equipment to move parallel to the navigation baseline.

8. An auxiliary navigation device for photovoltaic panel installation equipment, characterized in that, include: The recognition result generation module is used to acquire point cloud data of the target work area, and use a deep learning model to identify photovoltaic support candidate grids based on the point cloud data to generate recognition results. The map generation and update module is used to generate an initial semantic occupancy map based on the recognition results, and to dynamically update it in combination with updated point cloud data; The installation equipment navigation module is used to fit a navigation baseline based on a dynamically updated semantic occupancy map, and to control the movement of the photovoltaic panel installation equipment based on the navigation baseline.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-7.

Images