Photovoltaic robot-based grass height identification method and system, and robot

Abstract

The invention relates to a photovoltaic robot-based grass height identification method and system and a robot, and the method comprises the steps: carrying out the joint calibration of a camera and a laser radar based on a calibration board, and obtaining a calibration posture relation between the camera and the laser radar; inputting the image in the photovoltaic power station scene into a weed segmentation model to obtain a segmented weed region image; performing coordinate transformation on point cloud data in the photovoltaic power station scene based on the calibration attitude relation, and projecting the point cloud data to the weed area image so as to fuse the point cloud data with the weed area image; and filtering and clustering the point cloud data projected to the weed area image to obtain the grass height of the weed area, so that the grass height identification accuracy and efficiency are improved, and the labor cost and the time consumption are reduced.

Description

Technical Field

[0001] The present application relates to the field of photovoltaic technology, and in particular to a method, system and robot for identifying grass height based on a photovoltaic robot. Background Art

[0002] With the continuous development of the photovoltaic industry, the number of photovoltaic projects has increased rapidly, and the area occupied by photovoltaic power stations has also increased. However, in photovoltaic power stations, weeds that are too tall will block sunlight and reduce the power generation efficiency of photovoltaic panels.

[0003] Traditional manual weeding methods require a lot of time and human resources, are inefficient and costly. Therefore, there is an urgent need for an intelligent solution that can quickly and accurately identify the height of weeds in photovoltaic power plants to improve work efficiency, reduce labor costs, and ensure the power generation efficiency and stable operation of photovoltaic power plants. Summary of the invention

[0004] Based on this, it is necessary to provide a grass height recognition method, system and robot based on a photovoltaic robot to address the above technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying grass height based on a photovoltaic robot, wherein the photovoltaic robot is provided with a camera and a laser radar, which are respectively used to collect images and point cloud data in a photovoltaic power station scene, and the method comprises: Jointly calibrating the camera and the laser radar based on a calibration plate to obtain a calibration posture relationship between the two; Inputting the image in the photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; Based on the calibration pose relationship, coordinate transformation is performed on the point cloud data in the photovoltaic power station scene, and the point cloud data is projected onto the weed area image to be merged with the weed area image; The point cloud data projected onto the weed area image is filtered and clustered to obtain the height of the weeds in the weed area.

[0006] In one embodiment, the filtering and clustering of the point cloud data projected onto the weed area image in sequence comprises: Using a statistical filtering algorithm to filter the point cloud data projected onto the weed area image to remove outlier point cloud data; A straight-through filtering algorithm is used to filter the point cloud data projected onto the weed area image to remove point cloud data outside a preset spatial coordinate range.

[0007] In one embodiment, after filtering the point cloud data projected onto the weed area image, the method includes: Using a clustering algorithm to filter the point cloud data projected onto the weed area image and then clustering them to obtain point cloud data of multiple categories; Based on the spatial position features of the multiple categories of point cloud data, obtaining point cloud data of the weed area in the photovoltaic power station; Based on the point cloud data of the weed area, the height of the grass in the weed area is obtained.

[0008] In one of the embodiments, a density-based clustering algorithm is used to perform clustering processing on the point cloud data projected onto the weed area image to determine the point cloud data of the weed area.

[0009] In one embodiment, the jointly calibrating the camera and the laser radar based on the calibration plate to obtain a calibration pose relationship between the two includes: Using the camera to collect images of the calibration plate at different postures, and using the laser radar to collect point cloud data of the calibration plate at different postures; Extracting first feature points of the calibration plate based on the image of the calibration plate, and extracting second feature points of the calibration plate based on the point cloud data of the calibration plate; Based on the first feature point and the second feature point, a calibration posture relationship between the camera and the laser radar is obtained.

[0010] In one embodiment, the weed segmentation model is obtained by training based on an image dataset containing weeds and non-weeds, including: Annotate a dataset of images containing weeds and non-weeds; The labeled image data set is input into the image detection model for iterative training, and when the training result meets the preset accuracy, a weed segmentation model is obtained.

[0011] In one embodiment, after obtaining the height of grass in the weed area based on the point cloud data projected onto the weed area image, the method includes: determining a maximum height of the weedy area; If the maximum height is greater than the height value set by the operation and maintenance personnel, a message is sent to notify the operation and maintenance personnel to take weeding measures.

[0012] In a second aspect, the embodiment of the present application further provides a grass height recognition system based on a photovoltaic robot, the system comprising: The calibration module is used to jointly calibrate the camera and the lidar based on the calibration plate to obtain the calibration posture relationship between the two; A weed segmentation module is used to input an image in a photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; A projection module, for performing coordinate transformation on the point cloud data in the photovoltaic power station scene based on the calibration posture relationship, and projecting the point cloud data onto the weed area image to be merged with the weed area image; The acquisition module is used to filter and cluster the point cloud data projected onto the weed area image to obtain the height of the grass in the weed area.

[0013] In a third aspect, an embodiment of the present application further provides a photovoltaic robot, comprising a camera, a laser radar, and a processor, wherein the processor is used to execute the method described in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0015] The above-mentioned grass height recognition method, system and robot based on photovoltaic robot, by jointly calibrating the camera and the laser radar based on the calibration plate, obtains the calibration posture relationship between the two, inputs the image in the photovoltaic power station scene into the weed segmentation model, obtains the segmented weed area image, and based on the calibration posture relationship, transforms the coordinates of the point cloud data in the photovoltaic power station scene and projects it onto the weed area image to merge with the weed area image, filters and clusters the point cloud data projected onto the weed area image, obtains the grass height of the weed area, and can quickly and accurately identify the grass height of the weed area, so that the grass height data can be transmitted to the power station operation and maintenance personnel in time, and the operation and maintenance personnel can quickly take removal measures to ensure the power generation efficiency of the photovoltaic power station. Compared with the traditional manual weeding method, this intelligent inspection solution not only improves the accuracy and efficiency of grass height recognition, but also greatly reduces labor costs and time consumption.

[0016] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1is a hardware structure block diagram of a terminal device of a method for identifying grass height based on a photovoltaic robot in an embodiment; Figure 2 is a schematic flow chart of a method for identifying grass height based on a photovoltaic robot in an embodiment; Figure 3 is a weed area image obtained after segmentation in an embodiment; Figure 4 is a result of clustering the point cloud data projected onto the weed area image based on the DBSCAN clustering algorithm in one embodiment; Figure 5 The structure diagram of a grass height recognition system based on a photovoltaic robot in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0020] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1FIG. 1 is a hardware structure block diagram of a terminal of the method for identifying grass height based on a photovoltaic robot in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0021] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the grass height identification method based on the photovoltaic robot in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0022] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0023] The embodiment of the present application provides a method for identifying grass height based on a photovoltaic robot. The photovoltaic robot is provided with a high-resolution camera and a laser radar. The high-resolution camera equipped on the photovoltaic robot can accurately capture images in the photovoltaic power station. The laser radar equipped on the photovoltaic robot can obtain accurate distance and height information of the object, so that the robot can measure the height of the weeds, such as Figure 2 As shown, the method comprises the following steps: S201, jointly calibrating the camera and the laser radar based on a calibration plate to obtain a calibration posture relationship between the two; The joint calibration of camera and lidar is used to align the coordinate systems of the camera and lidar to achieve accurate correspondence between the 3D point cloud and the image. The purpose of calibration is to eliminate the error between the camera and lidar, correspond the image pixels to the point cloud data with height and distance information, and ensure that they can accurately acquire and process environmental information.

[0024] S202, inputting the image in the photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; S203, based on the marked pose relationship, performing coordinate transformation on the point cloud data in the photovoltaic power station scene, and projecting the point cloud data onto the weed area image to be fused with the weed area image; Specifically, this step is to fuse the point cloud data with the pixels of the weed segmentation image, and according to the coordinate system relationship between the camera and the laser radar obtained by the joint calibration in S201, the received laser point cloud data is transformed according to the coordinate system transformation matrix, and the laser point cloud data is projected onto the weed area image, so that each image pixel of the weed area image can correspond to the laser point cloud information, and most of the irrelevant point clouds other than the bushes are filtered out, and then the height and distance of each pixel on the weed area image can be known in combination with the laser point cloud data.

[0025] In the process of fusing the point cloud data with the weed segmentation image pixels, the image after segmenting the grass is used as input, and all pixels outside the segmented area are set to white with a pixel value of (255, 255, 255). Then, if a white pixel is encountered during the projection process, the point cloud data corresponding to the pixel is not received, so as to obtain only the point cloud data of the grass as much as possible and visualize the point cloud data.

[0026] S204, filtering and clustering the point cloud data projected onto the weed area image to obtain the height of the weeds in the weed area.

[0027] Although the pixel area of ​​the grass on the image has been segmented out based on the weed segmentation model, and the height of the grass is known by combining the point cloud data of the lidar, in fact, the grass is not a continuous entity and the accuracy of image segmentation cannot be very accurate. Therefore, some point clouds will pass through the gaps or edge areas in the grass, and the height of objects around the grass is actually obtained. Therefore, when there are other objects around the grass, height recognition is often disturbed. It is also necessary to filter the point cloud data projected onto the image of the weed area to further filter out possible irrelevant discrete point clouds, and then cluster the point cloud data. According to the clustering results, possible irrelevant category point cloud data are further eliminated, so as to accurately find the grass point cloud and obtain the height of the grass in the weed area.

[0028] In the above-mentioned grass height recognition method based on photovoltaic robots, the camera and the laser radar are jointly calibrated based on the calibration plate to obtain the calibration posture relationship between the two, the image in the photovoltaic power station scene is input into the weed segmentation model to obtain the segmented weed area image, and the point cloud data in the photovoltaic power station scene is transformed into coordinates based on the calibration posture relationship, and projected onto the weed area image to be fused with the weed area image, and the point cloud data projected onto the weed area image is filtered and clustered to obtain the grass height of the weed area, so that the grass height of the weed area can be quickly and accurately identified, so that the grass height data can be transmitted to the power station operation and maintenance personnel in time, and the operation and maintenance personnel can quickly take removal measures to ensure the power generation efficiency of the photovoltaic power station. Compared with the traditional manual weeding method, this intelligent inspection solution not only improves the accuracy and efficiency of grass height recognition, but also greatly reduces labor costs and time consumption.

[0029] In one embodiment, the joint calibration of the camera and the laser radar based on the calibration plate in S201 to obtain the calibration pose relationship between the two specifically includes the following steps: S301, using the camera to collect images of the calibration plate at different postures, and using the laser radar to collect point cloud data of the calibration plate at different postures; S302, extracting first feature points of the calibration plate based on the image of the calibration plate, and extracting second feature points of the calibration plate based on point cloud data of the calibration plate; S303: Obtain a calibration posture relationship between the camera and the laser radar based on the first feature point and the second feature point.

[0030] Specifically, the calibration plate is a checkerboard calibration plate, and the camera and laser radar can be jointly calibrated through the cam_lidar_calibration package in the ROS system (Robot Operating System). By simultaneously capturing the camera image and the laser radar point cloud data, the image and point cloud information of the calibration plate at different positions and postures are recorded, and the feature points of the calibration plate, such as corner points or checkerboard intersections, are extracted from the camera image. At the same time, the three-dimensional point cloud corresponding to the feature points of the calibration plate is extracted from the laser radar point cloud data. By comparing the feature points in the camera image and the three-dimensional point cloud in the laser radar point cloud, a correspondence between the two is established. These correspondences can be used to calculate the conversion matrix between the camera and the laser radar, that is, the calibration result.

[0031] In one embodiment, obtaining the weed segmentation model based on the training of the image dataset containing weeds and non-weeds in S202 includes the following: labeling the image dataset containing weeds and non-weeds; inputting the labeled image dataset into the image detection model for iterative training, and obtaining the weed segmentation model when the training result meets the preset accuracy.

[0032] Specifically, this embodiment uses the deep learning algorithm Yolov8 to segment the weed area in the image. It is necessary to prepare an image dataset containing weeds and non-weeds and annotate them. The annotation process includes marking the bounding box of the weeds or the pixel-level segmentation mask in the image. The image dataset is input into the Yolov8 network as a training set for iterative training. The model parameters are optimized through repeated iterations so that it can better adapt to the characteristics of the weed area, learn how to accurately detect and segment the weed area from the image, and thus obtain a weed segmentation model, such as Figure 3 As shown, the weed area image obtained after segmentation based on the weed segmentation model is displayed.

[0033] In one of the embodiments, the filtering processing of the point cloud data projected onto the weed area image in S204 specifically includes the following contents: filtering the point cloud data projected onto the weed area image using a statistical filtering algorithm to remove outlier point cloud data; filtering the point cloud data projected onto the weed area image using a straight-through filtering algorithm to remove point cloud data outside a preset spatial coordinate range.

[0034] Specifically, this embodiment uses the statistical filter in the Open3D point cloud processing library to filter the acquired grass point cloud data, removes the generated irrelevant discrete point clouds, and then combines the actual distance between the photovoltaic arrays in the power station to use a straight-through filtering algorithm to filter out point clouds outside 10 meters in the horizontal x-axis direction (for example, the distance between the photovoltaic arrays is 10 meters) and point clouds above 4 meters in the vertical z-axis direction (weeds are generally not higher than 4 meters). Through the above steps, possible irrelevant discrete point clouds can be further filtered out.

[0035] In one of the embodiments, after filtering the point cloud data projected onto the weed area image in S204, a clustering algorithm is used to filter the point cloud data projected onto the weed area image and then cluster them to obtain point cloud data of multiple categories; based on the spatial position characteristics of the point cloud data of the multiple categories, the point cloud data of the weed area in the photovoltaic power station is obtained; based on the point cloud data of the weed area, the height of the grass in the weed area is obtained.

[0036] Since grasses are often clustered together, although there may be point clouds passing through the gaps in the grasses and hitting entities such as walls behind the grasses, there is often a certain distance between the point clouds of the grasses and the point clouds passing through the inside of the grasses and hitting, for example, subsequent walls. There are also obvious differences in the spatial distribution and quantity of the two points, and both are concentrated in a certain space. Therefore, this embodiment also uses a clustering algorithm to cluster the point cloud data projected onto the image of the weed area to obtain point cloud data of multiple categories; based on the spatial position characteristics of each category of point cloud data, the point cloud data of the weed area in the photovoltaic power station is obtained; based on the point cloud data of the weed area, the height of the grass in the weed area is obtained to improve the accuracy of grass height recognition.

[0037] In one of the embodiments, a density-based clustering algorithm is used to perform clustering processing on the point cloud data projected onto the weed area image to determine the point cloud data of the weed area.

[0038] Density-based clustering algorithms assume that the clustering structure can be determined by the compactness of sample distribution. Usually, density clustering algorithms examine the connectivity between samples from the perspective of sample density, and continuously expand clusters based on connectable samples to obtain the final clustering results.

[0039] DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a representative density-based clustering algorithm. It is based on a set of "neighborhood" parameters to characterize the compactness of sample distribution and can find clusters of any shape. Therefore, the DBSCAN algorithm, which does not require the number of clusters to be specified in advance and can cluster according to spatial distance distribution, is very suitable for distinguishing point clouds clustered in different spatial locations, thereby finding grass point clouds.

[0040] For example, Figure 4 As shown in the figure, due to the difference in height and spatial position between the wall behind the bush and the bush, the DBSCAN clustering algorithm divides the point cloud data into two clusters. Since most of the point clouds except the bush have been filtered out by the weed segmentation model and the image and point cloud fusion algorithm, and the bushes are often clustered in a certain space, the point clouds collected on the bushes will be significantly more than the point clouds passing through the gaps inside them. Therefore, the class with the most point clouds in the cluster after DBSCAN clustering is considered to be the point cloud of the bush, as shown in the figure. Figure 4 As shown, the point cloud in area 100 is the grass part, which accounts for the vast majority of the point cloud, and the point cloud in area 200 is the point cloud hitting the wall, and the point clouds in other areas are considered to be noise points.

[0041] In one of the embodiments, after obtaining the height of grass in the weed area based on the point cloud data projected onto the image of the weed area, the method further includes determining the maximum height of the weed area based on the height of grass in the weed area; if the maximum height is greater than the height value set by the operation and maintenance personnel, a message is sent to notify the operation and maintenance personnel to take weed control measures.

[0042] After the maximum grass height of the weed area is calculated in this embodiment, the point cloud with the highest grass height is marked. If the maximum grass height is detected to be higher than the height value set by the power station operation and maintenance personnel, the operation and maintenance personnel will be notified to take cleaning measures. The weed problem can be responded to quickly and corresponding measures can be taken to maintain the normal operation of the power station and ensure the stable operation and power generation efficiency of the photovoltaic power station. It is suitable for the intelligent operation and maintenance of large photovoltaic power stations, can effectively reduce labor costs and time consumption, and greatly improve the work efficiency of weeding.

[0043] Based on the same technical concept, the embodiment of the present application also provides a grass height recognition system based on a photovoltaic robot, such as Figure 5 As shown, the system comprises: The calibration module 10 is used to jointly calibrate the camera and the laser radar based on the calibration plate to obtain a calibration posture relationship between the two; A weed segmentation module 20 is used to input an image in a photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; A projection module 30 is used to transform the point cloud data in the photovoltaic power station scene into coordinates based on the calibration posture relationship, and project the point cloud data onto the weed area image to be merged with the weed area image; The acquisition module 40 is used to filter and cluster the point cloud data projected onto the weed area image to obtain the height of the grass in the weed area.

[0044] In one embodiment, the acquisition module 40 is further used to: filter the point cloud data projected onto the weed area image using a statistical filtering algorithm to remove outlier point cloud data; filter the point cloud data projected onto the weed area image using a straight-through filtering algorithm to remove point cloud data outside a preset spatial coordinate range.

[0045] In one embodiment, the acquisition module 40 is further used to: use a clustering algorithm to filter the point cloud data projected onto the weed area image and then cluster them to obtain point cloud data of multiple categories; Based on the spatial position features of the multiple categories of point cloud data, obtaining point cloud data of the weed area in the photovoltaic power station; Based on the point cloud data of the weed area, the height of the grass in the weed area is obtained.

[0046] In one embodiment, the obtaining module 40 is further used to: perform clustering processing on the point cloud data projected onto the weed area image using a density-based clustering algorithm to determine the point cloud data of the weed area.

[0047] In one embodiment, the calibration module 10 is also used to: use the camera to collect images of the calibration plate at different postures, and use the laser radar to collect point cloud data of the calibration plate at different postures; based on the image of the calibration plate, extract the first feature point of the calibration plate, and based on the point cloud data of the calibration plate, extract the second feature point of the calibration plate; based on the first feature point and the second feature point, obtain the calibration posture relationship between the camera and the laser radar.

[0048] In one embodiment, the weed segmentation module 20 is further used to: annotate an image dataset containing weeds and non-weeds; input the annotated image dataset into an image detection model for iterative training, and when the training result meets a preset accuracy, obtain a weed segmentation model.

[0049] In one embodiment, the system further includes a notification module, which is used to determine the maximum height of the weed area after obtaining the height of the grass in the weed area; if the maximum height is greater than the height value set by the operation and maintenance personnel, a message is sent to notify the operation and maintenance personnel to take weed control measures.

[0050] Based on the same technical concept, an embodiment of the present application also provides a photovoltaic robot, including a camera, a laser radar, and a processor, wherein the processor is used to execute the steps in any of the above-mentioned embodiments of the grass height recognition method based on the photovoltaic robot.

[0051] Among them, the photovoltaic robot is a crawler photovoltaic robot. The high-resolution camera equipped with the photovoltaic robot can accurately capture images in the photovoltaic power station; the laser radar equipped with the photovoltaic robot can provide accurate distance information, so that the robot can measure the height of weeds. The crawler photovoltaic robot of this embodiment combines camera and laser radar technology. The robot can complete the inspection of the photovoltaic power station and accurately identify the height of objects in a short time. Through intelligent image processing and data analysis algorithms, the photovoltaic robot can quickly and accurately identify weeds that are too high. Once the height of these weeds is identified, the robot will transmit the relevant data to the power station operation and maintenance personnel, so that they can take timely removal measures. This intelligent inspection solution is suitable for the intelligent operation and maintenance of large photovoltaic power stations. It not only improves the accuracy and efficiency of grass height recognition, but also reduces labor costs and time consumption.

[0052] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above-mentioned embodiments of the method for identifying grass height based on a photovoltaic robot are implemented.

[0053] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0054] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying grass height based on a photovoltaic robot, wherein the photovoltaic robot is provided with a camera and a laser radar for collecting images and point cloud data in a photovoltaic power station scene, respectively, characterized in that: The method comprises: Performing joint calibration on the camera and the laser radar based on a calibration plate to obtain a calibration posture relationship between the two; Inputting the image in the photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; Based on the calibration pose relationship, coordinate transformation is performed on the point cloud data in the photovoltaic power station scene, and the coordinates are projected onto the weed area image to be merged with the weed area image; The point cloud data projected onto the weed area image is filtered and clustered to obtain the height of the weeds in the weed area.

2. The method according to claim 1, characterized in that The filtering and clustering processing of the point cloud data projected onto the weed area image in sequence includes: Using a statistical filtering algorithm to filter the point cloud data projected onto the weed area image to remove outlier point cloud data; A straight-through filtering algorithm is used to filter the point cloud data projected onto the weed area image to remove point cloud data outside a preset spatial coordinate range.

3. The method according to claim 2, characterized in that After filtering the point cloud data projected onto the weed area image, the method further comprises: Using a clustering algorithm to filter the point cloud data projected onto the weed area image and then clustering them to obtain point cloud data of multiple categories; Based on the spatial position features of the multiple categories of point cloud data, obtaining point cloud data of the weed area in the photovoltaic power station; Based on the point cloud data of the weed area, the height of the grass in the weed area is obtained.

4. The method according to claim 3, characterized in that A density-based clustering algorithm is used to perform clustering processing on the point cloud data projected onto the weed area image to determine the point cloud data of the weed area.

5. The method according to claim 1, characterized in that The joint calibration of the camera and the laser radar based on the calibration plate to obtain the calibration posture relationship between the two includes: Using the camera to collect images of the calibration plate at different postures, and using the laser radar to collect point cloud data of the calibration plate at different postures; Extracting first feature points of the calibration plate based on the image of the calibration plate, and extracting second feature points of the calibration plate based on the point cloud data of the calibration plate; Based on the first feature point and the second feature point, a calibration posture relationship between the camera and the laser radar is obtained.

6. The method according to claim 1, characterized in that The weed segmentation model obtained by training based on an image dataset containing weeds and non-weeds includes: Annotate a dataset of images containing weeds and non-weeds; The labeled image data set is input into the image detection model for iterative training, and when the training result meets the preset accuracy, a weed segmentation model is obtained.

7. The method according to claim 1, characterized in that After obtaining the height of the grass in the weed area based on the point cloud data projected onto the weed area image, the method includes: determining a maximum height of the weedy area; If the maximum height is greater than the height value set by the operation and maintenance personnel, a message is sent to notify the operation and maintenance personnel to take weeding measures.

8. A grass height recognition system based on a photovoltaic robot, characterized in that: The system comprises: The calibration module is used to jointly calibrate the camera and the lidar based on the calibration plate to obtain the calibration posture relationship between the two; A weed segmentation module is used to input an image in a photovoltaic power station scene into a weed segmentation model to obtain a segmented weed area image; the weed segmentation model is trained based on an image data set containing weeds and non-weeds; A projection module, for performing coordinate transformation on the point cloud data in the photovoltaic power station scene based on the calibration posture relationship, and projecting the point cloud data onto the weed area image to be merged with the weed area image; The acquisition module is used to filter and cluster the point cloud data projected onto the weed area image to obtain the height of the grass in the weed area.

9. A photovoltaic robot, characterized in that: The method comprises a camera, a lidar and a processor, wherein the processor is used to execute the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images