Method for simultaneously detecting passable area and obstacle of plateau photovoltaic power station
By improving the YOLOP algorithm and building a plateau photovoltaic power station data set, the problem of difficulty in detecting passable areas and obstacles in complex scenarios in the prior art is solved, and the accurate detection and identification effect is achieved in the complex environment of plateau photovoltaic power stations.
Patent Information
- Application Number
- CN202311677080.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-05
- Publication Date
- 2025-06-06
AI Technical Summary
The prior art is difficult to accurately detect passable areas and obstacles in complex scenarios of plateau photovoltaic power plants, especially in the presence of background noise, complex shapes and multiple obstacles.
Improve the YOLOP algorithm to build a data set of passable areas and obstacles for plateau photovoltaic power stations. Through the improved YOLOP network, the model parameters can be reduced, and the passable areas and obstacle information can be output in real time.
It realizes more accurate identification of textures and shapes in images in complex environments, can detect multiple types of obstacles, adapt to complex environments, and ensures that autonomous mobile robots operate in complex areas of the terrain of the plateau photovoltaic power station.
Smart Images
Figure CN120107644A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection technology, and more specifically, to a method for simultaneously detecting traversable areas and obstacles in a plateau photovoltaic power station. Background Art
[0002] The proportion of plateau photovoltaic power stations in my country's photovoltaic energy is increasing year by year. In recent years, the rapid development of plateau photovoltaic power stations has put forward new requirements for automated autonomous mobile robot equipment. Autonomous mobile robots can participate in the inspection of photovoltaic power stations. If equipped with corresponding agricultural tools, they can also complete power station maintenance work, such as intelligent weeding. However, plateau photovoltaic power stations are usually located in high-altitude mountainous and hilly areas with complex terrain. Autonomous mobile robots are required to detect and identify the environment through onboard sensors. Therefore, this poses a great challenge to visual recognition technology.
[0003] In the prior art, the detection methods for the traversable areas and obstacles of high-altitude photovoltaic power stations mainly include detection methods based on traditional image processing and detection methods based on convolutional neural networks.
[0004] The detection method based on traditional image processing has the following steps: first, preprocess the image by graying, smoothing, denoising and binarizing; second, use the Canny edge detection algorithm on the preprocessed image to calculate the gradient amplitude and direction to estimate the edge strength and direction at each point; finally, use non-maximum suppression to extract the road edge area, and make logical judgments on passable and impassable areas based on some prior information. Since this type of method is easily affected by background noise, it is suitable for detection in simple scenes where the road surface is smooth and relatively obvious. However, in reality, the scene of plateau photovoltaic power stations is relatively complex, and the shape of the passable area is complex and changeable, which greatly interferes with edge detection. Therefore, this type of detection method faces great challenges in terms of detection accuracy.
[0005] The traversable area detection method based on convolutional neural network is to prepare a road patching data set, train multiple convolutional neural network models, and obtain the trained network model; input the road image to be detected into the target convolutional neural network model to obtain the information in the road image to be detected. This method is only applicable to pictures with high contrast, low noise, relatively simple scenes, and no obstacles such as fallen leaves, water stains, and shadows. However, due to the complexity of the images of the plateau photovoltaic power station area, this type of method is difficult to meet the needs of actual engineering applications.
[0006] At present, there is no good method at home and abroad to simultaneously detect obstacles and the accessible areas of plateau photovoltaic power stations. Summary of the invention
[0007] In order to overcome the shortcomings of the prior art, the present invention provides a method for simultaneously detecting traversable areas and obstacles in a plateau photovoltaic power station. The improved YOLOP algorithm can process image data in real time, so that traversable areas and obstacles can be detected during real-time monitoring. The texture and shape in the image can be more accurately identified, and various types of obstacles can be detected, so that the autonomous mobile robot can adapt to complex environments and complete operations in areas with more complex terrain.
[0008] The technical solution of the present invention is as follows: A method for simultaneously detecting the passable area and obstacles of a highland photovoltaic power station comprises the following steps:
[0009] Step 1: Construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations;
[0010] Step 2: Improve the YOLOP network so that the improved YOLOP network reduces model parameters and can output the semantic segmentation results of the passable area and the obstacle detection results;
[0011] Step 3: Use the PV power station passable area and obstacle dataset to train the improved YOLOP network to obtain a simultaneous detection model for passable areas and obstacles;
[0012] Step 4: Use the passable area and obstacle detection model to detect the plateau photovoltaic power station image to determine whether the road image is passable and whether there are obstacles.
[0013] Furthermore, the specific steps of building the accessible area and obstacle dataset of the plateau photovoltaic power station in Step 1 are as follows:
[0014] Step 1.1: Obtain images of photovoltaic power stations on the plateau, and perform image homogenization and denoising.
[0015] Step 1.2: Label the images after uniform light and denoising, and construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations;
[0016] Step 1.3: Divide the traversable area and obstacle datasets into training set, validation set and test set respectively.
[0017] Furthermore, in Step 1.2, the images after uniform illumination and denoising are annotated. Specifically, the passable areas in each sample image are annotated to generate a pixel-by-pixel classified mask image corresponding to the sample image; the obstacles in each sample image are annotated to generate a txt file with obstacle categories and obstacle position coordinates corresponding to the sample image.
[0018] Furthermore, in Step 1.3, the traversable area and obstacle datasets are divided into training set, validation set, and test set in a ratio of 70%:20%:10%.
[0019] Furthermore, Step 2 is specifically as follows:
[0020] Step 2.1: Replace the YOLOP backbone network CSPDarknet with the MobileOne model block;
[0021] Step 2.2: The MobileOne model block retains three feature maps of different scales and removes the regression part behind the network;
[0022] Step 2.3: Remove the lane segmentation head of YOLOP and reduce the network structure.
[0023] Furthermore, Step 3 is specifically as follows:
[0024] Step 3.1: Set the training parameters, including at least batch size, learning rate, number of detection categories and maximum number of iterations;
[0025] Step 3.2: Input the images of the training set into the improved backbone network to obtain feature maps of three different scales;
[0026] Step 3.3: Input the feature maps of three different scales into the Neck part for upsampling and feature fusion to obtain tensor data of three different scales;
[0027] Step 3.4: For the obstacle detection branch, input the tensor data of three different scales into the prediction part of the detection head, and use the loss function to calculate the branch loss; for the passable area detection branch, input the sampled feature map into the passable area segmentation head, and use the loss function to calculate the branch loss; use the losses of the two branches to perform back propagation and calculate the gradient; update the gradient and model parameters in real time, and use the verification set to verify the network accuracy;
[0028] Step 3.5: Repeat Step 3.1 to Step 3.4. After reaching the maximum number of iterations, the improved YOLOP network completes learning and obtains the weights of the traversable area and target simultaneous detection model.
[0029] Furthermore, the improved YOLOP network includes a feature extraction module, a feature fusion module and a segmentation head module. The feature extraction module slices the input image and extracts feature maps of different scales; the feature fusion module is responsible for fusing global features and local detail features, constructing a coordinate attention module to learn lane line detail features, using the SimSPPF module and the feature pyramid network to expand the model receptive field and fuse features of different scales; the segmentation head module restores the feature map to the input image size and outputs the lane line detection result.
[0030] Furthermore, the convolution layer of the feature extraction module is a 1x1 convolution, and the activation function used is h-swish. The calculation method of the h-swish activation function is as follows:
[0031]
[0032] f(x) = max(0, x)
[0033] Among them, ReLU is an activation function, when x∈[-10.0,-3.0), y=0;
[0034] When x∈(-3.0, 0.0), y<0;
[0035] When x∈(0.0,+∞), y≥0.
[0036] Furthermore, the denoising process in Step 1.1 is specifically as follows: based on Filter-Base, the noise of the processed image is estimated, the noise estimation value is determined, and whether the noise estimation value is greater than the preset noise threshold is judged. If so, the processed image is denoised using the fast non-local uniform value denoising algorithm; if not, the processed image does not need to be denoised; wherein, the noise of the processed image is estimated and the formula for determining the noise estimation value is as follows:
[0037]
[0038]
[0039]
[0040]
[0041] Among them, σ n refers to the noise estimation size, W is the image width, H is the image height, imageI refers to the image pixel, I(x, y) refers to the image pixel point, * is the convolution symbol, and N is the kernel kernel; kernel consists of two filter operators mask L 1 , L 2 composition.
[0042] The present invention according to the above scheme has the following beneficial effects: a method for simultaneous detection of traversable areas and obstacles of a plateau photovoltaic power station, firstly constructing a data set of traversable areas and obstacles of a plateau photovoltaic power station; secondly, improving the YOLOP network, so that the improved YOLOP network can reduce model parameters, reduce network calculation amount, and can simultaneously output traversable area and obstacle information; then using the traversable area and obstacle data set of the plateau photovoltaic power station to train the improved YOLOP network, and obtain a traversable area and obstacle simultaneous detection model; finally, using the plateau photovoltaic power station image as input for forward reasoning operation, outputting traversable pixel area and target detection results; the improved YOLOP algorithm can process image data in real time, thereby detecting traversable areas and obstacles during real-time monitoring, can more accurately identify textures and shapes in images, and can detect various types of obstacles, so that the autonomous mobile robot can adapt to complex environments and complete operations in areas with relatively complex terrain. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1 It is a flow chart of a method for simultaneously detecting passable areas and obstacles of a highland photovoltaic power station in an embodiment of the present invention;
[0045] Figure 2 It is the structural diagram of the existing YOLOP;
[0046] Figure 3 is a structural block diagram of the MobileOne model block in an embodiment of the present invention;
[0047] Figure 4 FIG. 4 is a diagram of an improved YOLOP network structure in an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The following detailed description of the embodiments of the present invention is provided in conjunction with the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the described embodiments.
[0049] In order to better understand the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments:
[0050] See also Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for simultaneously detecting a passable area and obstacles of a highland photovoltaic power station, comprising the following steps:
[0051] Step 1: Construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations;
[0052] Step 2: Improve the YOLOP network so that the improved YOLOP network reduces model parameters and can output the semantic segmentation results of the passable area and the obstacle detection results;
[0053] Step 3: Use the PV power station passable area and obstacle dataset to train the improved YOLOP network to obtain a simultaneous detection model for passable areas and obstacles;
[0054] Step 4: Use the passable area and obstacle detection model to detect the plateau photovoltaic power station image to determine whether the road image is passable and whether there are obstacles.
[0055] Specifically, an embodiment of the present invention provides a method for simultaneously detecting traversable areas and obstacles of a plateau photovoltaic power station. First, a data set of traversable areas and obstacles of a plateau photovoltaic power station is constructed; secondly, a YOLOP network is improved so that the improved YOLOP network can reduce model parameters, reduce network calculation amount, and can simultaneously output traversable area and obstacle information; then, the traversable area and obstacle data set of the plateau photovoltaic power station is used to train the improved YOLOP network to obtain a traversable area and obstacle simultaneous detection model; finally, a forward reasoning operation is performed using an image of the plateau photovoltaic power station as input to output traversable pixel areas and target detection results; the improved YOLOP algorithm can process image data in real time, thereby detecting traversable areas and obstacles during real-time monitoring, can more accurately identify textures and shapes in images, and can detect various types of obstacles, so that the autonomous mobile robot can adapt to complex environments and complete operations in areas with relatively complex terrain.
[0056] Specifically, the steps for constructing the accessible area and obstacle dataset of the plateau photovoltaic power station in Step 1 are as follows:
[0057] Step 1.1: Obtain images of photovoltaic power stations on the plateau, and perform image homogenization and denoising.
[0058] Step 1.2: Label the images after uniform light and denoising, and construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations;
[0059] Step 1.3: Divide the traversable area and obstacle datasets into training set, validation set and test set respectively.
[0060] Specifically, in Step 1.2, the images after uniform illumination and denoising are annotated, specifically: the passable areas in each sample image are annotated to generate a pixel-by-pixel classified mask image corresponding to the sample image; the obstacles in each sample image are annotated to generate a txt file with obstacle categories and obstacle position coordinates corresponding to the sample image.
[0061] Specifically, in Step 1.3, the drivable area and obstacle datasets are divided into training set, validation set and test set in the ratio of 70%:20%:10%.
[0062] See also Figure 3 As shown, specifically, Step 2 is as follows:
[0063] Step 2.1: Replace the YOLOP backbone network CSPDarknet with the MobileOne model block;
[0064] Step 2.2: The MobileOne model block retains three feature maps of different scales and removes the regression part behind the network;
[0065] Step 2.3: Remove the lane segmentation head of YOLOP and reduce the network structure.
[0066] Specifically, Step 3 is as follows:
[0067] Step 3.1: Set the training parameters, including at least batch size, learning rate, number of detection categories and maximum number of iterations;
[0068] Step 3.2: Input the images of the training set into the improved backbone network to obtain feature maps of three different scales;
[0069] Step 3.3: Input the feature maps of three different scales into the Neck part for upsampling and feature fusion to obtain tensor data of three different scales;
[0070] Step 3.4: For the obstacle detection branch, input the tensor data of three different scales into the prediction part of the detection head, and use the loss function to calculate the branch loss; for the passable area detection branch, input the sampled feature map into the passable area segmentation head, and use the loss function to calculate the branch loss; use the losses of the two branches to perform back propagation and calculate the gradient; update the gradient and model parameters in real time, and use the verification set to verify the network accuracy;
[0071] Step 3.5: Repeat Step 3.1 to Step 3.4. After reaching the maximum number of iterations, the improved YOLOP network completes learning and obtains the weights of the traversable area and target simultaneous detection model.
[0072] Specifically, the improved YOLOP network includes a feature extraction module, a feature fusion module and a segmentation head module. The feature extraction module slices the input image and extracts feature maps of different scales; the feature fusion module is responsible for fusing global features and local detail features, constructing a coordinate attention module to learn lane line detail features, using the SimSPPF module and feature pyramid network to expand the model receptive field and fuse features of different scales; the segmentation head module restores the feature map to the input image size and outputs the lane line detection results.
[0073] Specifically, the convolution layer of the feature extraction module is a 1x1 convolution, and the activation function used is h-swish. The calculation method of the h-swish activation function is as follows:
[0074]
[0075] f(x) = max(0, x)
[0076] Among them, ReLU is an activation function, when x∈[-10.0,-3.0), y=0;
[0077] When x∈(-3.0, 0.0), y<0;
[0078] When x∈(0.0,+∞), y≥0.
[0079] Specifically, the denoising process in Step 1.1 is as follows: based on Filter-Base, the noise of the processed image is estimated, the noise estimation value is determined, and whether the noise estimation value is greater than the preset noise threshold is judged. If so, the fast non-local uniform value denoising algorithm is used to denoise the processed image; if not, there is no need to denoise the processed image; wherein, the formula for estimating the noise of the processed image and determining the noise estimation value is as follows:
[0080]
[0081]
[0082]
[0083]
[0084] Among them, σ nrefers to the noise estimation size, W is the image width, H is the image height, imageI refers to the image pixel, I(x, y) refers to the image pixel point, * is the convolution symbol, and N is the kernel kernel; kernel consists of two filter operators mask L 1 , L 2 composition.
[0085] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
[0086] The above is an exemplary description of the present invention in conjunction with the accompanying drawings. It is obvious that the implementation of the present invention is not limited to the above-mentioned method. As long as various improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.
Claims
1. A method for simultaneously detecting accessible areas and obstacles in a highland photovoltaic power station. It is characterized in that The following steps are involved: Step 1: Construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations; Step 2: Improve the YOLOP network so that the improved YOLOP network reduces model parameters and can output the semantic segmentation results of the passable area and the obstacle detection results; Step 3: Use the PV power station passable area and obstacle dataset to train the improved YOLOP network to obtain a simultaneous detection model for passable areas and obstacles; Step 4: Use the passable area and obstacle detection model to detect the plateau photovoltaic power station image to determine whether the road image is passable and whether there are obstacles.
2. A method for simultaneously detecting traversable areas and obstacles in a highland photovoltaic power station as claimed in claim 1, Features: The specific steps of building the accessible area and obstacle dataset of the plateau photovoltaic power station in Step 1 are as follows: Step 1.1: Obtain images of photovoltaic power stations on the plateau, and perform image homogenization and denoising. Step 1.2: Label the images after uniform light and denoising, and construct a dataset of accessible areas and obstacles for high-altitude photovoltaic power stations; Step 1.3: Divide the traversable area and obstacle datasets into training set, validation set and test set respectively.
3. A method for simultaneously detecting traversable areas and obstacles of a highland photovoltaic power station as claimed in claim 2, Features: In Step 1.2, the images after uniform illumination and denoising are annotated. Specifically, the passable areas in each sample image are annotated to generate a pixel-by-pixel classified mask image corresponding to the sample image; the obstacles in each sample image are annotated to generate a txt file with the obstacle category and obstacle position coordinates corresponding to the sample image.
4. A method for simultaneously detecting traversable areas and obstacles of a highland photovoltaic power station as claimed in claim 2, Features: In Step 1.3, the drivable area and obstacle datasets are divided into training set, validation set and test set in the ratio of 70%:20%:10%.
5. A method for simultaneously detecting traversable areas and obstacles of a highland photovoltaic power station as claimed in claim 1, Features: Step 2 is as follows: Step 2.1: Replace the YOLOP backbone network CSPDarknet with the MobileOne model block; Step 2.2: The MobileOne model block retains three feature maps of different scales and removes the regression part behind the network; Step 2.3: Remove the lane segmentation head of YOLOP and reduce the network structure.
6. A method for simultaneously detecting traversable areas and obstacles of a highland photovoltaic power station as claimed in claim 1, Features: Step 3 is as follows: Step 3.1: Set the training parameters, including at least batch size, learning rate, number of detection categories and maximum number of iterations; Step 3.2: Input the images of the training set into the improved backbone network to obtain feature maps of three different scales; Step 3.3: Input the feature maps of three different scales into the Neck part for upsampling and feature fusion to obtain tensor data of three different scales; Step 3.4: For the obstacle detection branch, input the tensor data of three different scales into the prediction part of the detection head, and use the loss function to calculate the branch loss; for the passable area detection branch, input the sampled feature map into the passable area segmentation head, and use the loss function to calculate the branch loss; use the losses of the two branches to perform back propagation and calculate the gradient; update the gradient and model parameters in real time, and use the verification set to verify the network accuracy; Step 3.5: Repeat Step 3.1 to Step 3.
4. After reaching the maximum number of iterations, the improved YOLOP network completes learning and obtains the weights of the traversable area and target simultaneous detection model.
7. A method for simultaneously detecting traversable areas and obstacles in a highland photovoltaic power station as claimed in claim 6, Features: The improved YOLOP network includes a feature extraction module, a feature fusion module and a segmentation head module. The feature extraction module slices the input image and extracts feature maps of different scales. The feature fusion module is responsible for fusing global features and local detail features, building a coordinate attention module to learn lane detail features, using the SimSPPF module and feature pyramid network to expand the model receptive field and fuse features of different scales. The segmentation head module restores the feature map to the input image size and outputs the lane line detection results.
8. A method for simultaneously detecting traversable areas and obstacles in a highland photovoltaic power station as claimed in claim 7, Features: The convolution layer of the feature extraction module is a 1x1 convolution, and the activation function used is h-swish. The calculation method of the h-swish activation function is as follows: f(x) = max(0, x) Among them, ReLU is an activation function, when x∈[-10.0,-3.0), y=0; When x∈(-3.0, 0.0), y<0; When x∈(0.0,+∞), y≥0.
9. A method for simultaneously detecting traversable areas and obstacles in a highland photovoltaic power station as claimed in claim 2, Features: The denoising process in Step 1.1 is as follows: based on Filter-Base, the noise of the processed image is estimated, the noise estimation value is determined, and whether the noise estimation value is greater than the preset noise threshold is judged. If so, the fast non-local uniform value denoising algorithm is used to denoise the processed image. If not, there is no need to denoise the processed image. Among them, the formula for determining the noise estimation value for the processed image is as follows: Among them, σ n refers to the noise estimation size, W is the image width, H is the image height, imageI refers to the image pixel, I(x, y) refers to the image pixel point, * is the convolution symbol, and N is the kernel kernel; kernel consists of two filter operators mask L 1 , L 2 composition.