Unmanned aerial vehicle lens processing method and apparatus, computer device, and storage medium

Abstract

The application relates to a UAV lens processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a to-be-detected image photographed by a UAV lens of a target UAV; the to-be-detected image is used for detecting the attachment of pollutants on the surface of the UAV lens; image features of the to-be-detected image are extracted according to preset feature extraction information, so as to obtain a feature extraction result corresponding to the to-be-detected image; a pollutant detection result of the UAV lens is determined according to the feature extraction result; a target cleaning mode is determined according to a pollution type and a pollution degree; a cleaning instruction corresponding to the target cleaning mode is sent to the target UAV; and the cleaning instruction is used for instructing the target UAV to perform a cleaning operation on the UAV lens according to the target cleaning mode. The method can automatically and effectively clean the UAV camera in time, improves the UAV shooting efficiency, and ensures the UAV shooting quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a UAV lens processing method and device, computer equipment, storage medium and computer program product. BACKGROUND

[0002] With the development of UAV technology, UAVs can be widely used in agriculture, surveying and mapping, environmental monitoring and other fields. When the UAV performs an automatic driving task, various pollutants such as dust, haze, water droplets and the like will adhere to the surface of the lens of the UAV. The adhered pollutants will cause the image clarity to decrease, affect the photographing result, and also affect the control and navigation of the UAV.

[0003] In the traditional technology, the manual cleaning processing method is usually used, which requires the staff to go to the airport site far away to maintain the UAV. When the adhering of pollutants occurs during the execution of the UAV task, it cannot be processed in time, the UAV maintenance efficiency is low, and the task execution effect is poor. SUMMARY

[0004] Therefore, it is necessary to provide a UAV lens processing method, device, computer equipment, storage medium and computer program product capable of improving the UAV lens processing efficiency in view of the above technical problems.

[0005] In a first aspect, the present application provides a UAV lens processing method, comprising:

[0006] obtaining a to-be-detected image photographed by a UAV lens of a target UAV; the to-be-detected image is used to detect the adhering of pollutants on the surface of the UAV lens;

[0007] extracting image features of the to-be-detected image according to preset feature extraction information, to obtain a feature extraction result corresponding to the to-be-detected image; the preset feature extraction information is determined based on the detection of pollutants on the surface of the UAV lens;

[0008] determining a pollutant detection result for the UAV lens according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree;

[0009] determining a target cleaning mode according to the pollution type and the pollution degree, and sending a cleaning instruction corresponding to the target cleaning mode to the target UAV; the cleaning instruction is used to instruct the target UAV to perform a cleaning operation on the UAV lens according to the target cleaning mode.

[0010] In one embodiment, the extracting of the image features of the to-be-detected image according to the preset feature extraction information to obtain the feature extraction result corresponding to the to-be-detected image comprises:

[0011] Obtain the grayscale image corresponding to the image to be detected, and adjust the number of grayscale levels in the grayscale image;

[0012] For each pixel in the adjusted grayscale image, determine the frequency information corresponding to the pixel, and generate a pixel feature matrix based on the frequency information corresponding to each pixel;

[0013] Using the pixel feature matrix and the preset feature extraction information, the feature extraction result corresponding to the image to be detected is obtained.

[0014] In one embodiment, the step of extracting the feature extraction result corresponding to the image to be detected by using the pixel feature matrix and the preset feature extraction information includes:

[0015] The pixel feature matrix is ​​normalized.

[0016] Based on the preset feature extraction information, different types of features are extracted from the processed pixel feature matrix; the preset feature extraction information is used to indicate the type of feature to be extracted.

[0017] Based on the extracted different types of features, the feature vector of the image to be detected is obtained, which is used as the feature extraction result.

[0018] In one embodiment, determining the contaminant detection result for the drone lens based on the feature extraction result includes:

[0019] The feature vector of the image to be detected is input into a pre-trained pollutant classification model;

[0020] Based on the pollutant prediction information output by the pollutant classification model, the pollutant detection results of the UAV lens are obtained;

[0021] The pre-trained pollutant classification model includes a support vector machine, which is used to classify different types of pollutants based on the optimal decision boundary.

[0022] In one embodiment, before the step of extracting image features of the image to be detected according to preset feature extraction information to obtain the feature extraction result corresponding to the image to be detected, the method further includes:

[0023] According to the preprocessing operation information, the image to be detected is preprocessed to obtain the processed image to be detected;

[0024] The processed image to be detected is used to extract image features of the image to be detected according to preset feature extraction information, and a feature extraction result corresponding to the image to be detected is obtained.

[0025] In one of the embodiments, the target unmanned aerial vehicle is provided with a monitoring device, and after the step of sending the cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle, the method further comprises:

[0026] Obtaining lens state information collected by the monitoring device; the lens state information is obtained by monitoring the cleaning state of the unmanned aerial vehicle lens in real time;

[0027] According to the lens state information, a cleaning feedback result for the unmanned aerial vehicle lens is generated; the cleaning feedback result is used to represent the processing condition of the cleaning operation performed on the unmanned aerial vehicle lens.

[0028] In a second aspect, the present application further provides an unmanned aerial vehicle lens processing device, which comprises:

[0029] The image to be detected is obtained by the unmanned aerial vehicle lens of the target unmanned aerial vehicle; the image to be detected is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0030] The image feature extraction module is used to extract image features of the image to be detected according to preset feature extraction information, and a feature extraction result corresponding to the image to be detected is obtained; the preset feature extraction information is determined based on the pollutant detection on the surface of the unmanned aerial vehicle lens;

[0031] The pollutant detection result obtaining module is used to determine a pollutant detection result for the unmanned aerial vehicle lens according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree;

[0032] The cleaning instruction sending module is used to determine a target cleaning mode according to the pollution type and the pollution degree, and send a cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the unmanned aerial vehicle lens according to the target cleaning mode.

[0033] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0034] The image to be detected is obtained by the unmanned aerial vehicle lens of the target unmanned aerial vehicle; the image to be detected is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0035] an image feature extraction module configured to extract image features of the to-be-detected image according to preset feature extraction information, to obtain a feature extraction result corresponding to the to-be-detected image; the preset feature extraction information is determined based on pollution detection on the surface of the unmanned aerial vehicle lens;

[0036] a pollution detection result obtaining module configured to determine a pollution detection result for the unmanned aerial vehicle lens according to the feature extraction result; the pollution detection result includes a pollution type and a pollution degree;

[0037] a cleaning instruction sending module configured to determine a target cleaning mode according to the pollution type and the pollution degree, and send a cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the unmanned aerial vehicle lens according to the target cleaning mode.

[0038] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0039] a to-be-detected image obtaining module configured to obtain a to-be-detected image captured by a unmanned aerial vehicle lens of a target unmanned aerial vehicle; the to-be-detected image is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0040] an image feature extraction module configured to extract image features of the to-be-detected image according to preset feature extraction information, to obtain a feature extraction result corresponding to the to-be-detected image; the preset feature extraction information is determined based on pollution detection on the surface of the unmanned aerial vehicle lens;

[0041] a pollution detection result obtaining module configured to determine a pollution detection result for the unmanned aerial vehicle lens according to the feature extraction result; the pollution detection result includes a pollution type and a pollution degree;

[0042] a cleaning instruction sending module configured to determine a target cleaning mode according to the pollution type and the pollution degree, and send a cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the unmanned aerial vehicle lens according to the target cleaning mode.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and the computer program is executed by a processor to implement the following steps:

[0044] An image to be detected acquisition module is configured to acquire an image to be detected captured by a camera lens of a target unmanned aerial vehicle (UAV), the image to be detected being used to detect an attachment of a contaminant on a surface of the camera lens of the UAV;

[0045] An image feature extraction module is configured to extract image features of the image to be detected according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected, the preset feature extraction information being determined based on a contaminant detection on the surface of the camera lens of the UAV;

[0046] A contaminant detection result obtaining module is configured to determine a contaminant detection result for the camera lens of the UAV according to the feature extraction result, the contaminant detection result including a contaminant type and a contaminant degree;

[0047] A cleaning instruction sending module is configured to determine a target cleaning mode according to the contaminant type and the contaminant degree, and send a cleaning instruction corresponding to the target cleaning mode to the target UAV, the cleaning instruction being used to instruct the target UAV to perform a cleaning operation on the camera lens of the UAV according to the target cleaning mode.

[0048] The above-mentioned method, device, computer device, storage medium and computer program product for processing a camera lens of an unmanned aerial vehicle, by acquiring an image to be detected captured by a camera lens of a target unmanned aerial vehicle (UAV), the image to be detected being used to detect an attachment of a contaminant on a surface of the camera lens of the UAV, then extracting image features of the image to be detected according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected, the preset feature extraction information being determined based on a contaminant detection on the surface of the camera lens of the UAV, determining a contaminant detection result for the camera lens of the UAV according to the feature extraction result, the contaminant detection result including a contaminant type and a contaminant degree, and then determining a target cleaning mode according to the contaminant type and the contaminant degree, and sending a cleaning instruction corresponding to the target cleaning mode to the target UAV, the cleaning instruction being used to instruct the target UAV to perform a cleaning operation on the camera lens of the UAV according to the target cleaning mode, realizes automatic cleaning of the camera of the unmanned aerial vehicle, automatically senses whether the camera lens of the unmanned aerial vehicle is contaminated and performs a corresponding cleaning operation by monitoring the cleanliness of the surface of the camera lens of the unmanned aerial vehicle, can effectively clean the camera of the unmanned aerial vehicle in a timely manner, improves the shooting efficiency of the unmanned aerial vehicle, and ensures the shooting quality of the unmanned aerial vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 FIG. 1 is a flowchart of a method for processing a camera lens of an unmanned aerial vehicle according to an embodiment;

[0050] Figure 2 FIG. 5 is a flowchart of an image feature extraction step according to an embodiment;

[0051] Figure 3This is a flowchart illustrating the drone camera processing method in another embodiment;

[0052] Figure 4 This is a structural block diagram of a drone lens processing device in one embodiment;

[0053] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a method for processing drone camera footage is provided. This embodiment illustrates the method applied to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0056] Step 101: Obtain the image to be detected captured by the drone camera of the target drone;

[0057] As an example, one or more drones can be monitored, such as monitoring the cleanliness of the drone lens surface of each drone in real time, and each drone to be monitored can be used as a target drone.

[0058] The image to be detected can be used to detect the contaminant adhesion on the surface of the drone lens, so as to determine whether the drone lens is contaminated, i.e., the cleanliness of the drone lens surface.

[0059] In practical applications, the drone to be monitored can be used as the target drone. For the target drone, the image data captured by its equipped drone lens can be obtained as the image to be monitored.

[0060] For example, a drone lens equipped on a drone can be used to take pictures in real time, and then the captured image data (i.e. the image to be detected) can be transmitted to the method processing unit. The method processing unit can exist in the drone lens monitoring terminal and transmit data through communication between the drone and the drone lens monitoring terminal. It can also exist in the monitoring device configured on the drone to further perform detection based on the captured image data.

[0061] In step 102, image features of the to-be-detected image are extracted according to preset feature extraction information, to obtain a feature extraction result corresponding to the to-be-detected image.

[0062] The preset feature extraction information can be determined based on pollution detection on the surface of the lens of the unmanned aerial vehicle, for example, in order to detect pollution on the surface of the lens of the unmanned aerial vehicle, the contrast, correlation, energy and entropy of the image can be mainly extracted.

[0063] After obtaining the to-be-detected image, a gray image corresponding to the to-be-detected image can be obtained, and the gray level of the gray image can be adjusted. Then, for each pixel in the adjusted gray image, frequency information corresponding to the pixel can be determined, a pixel feature matrix can be generated according to the frequency information corresponding to each pixel, and then the pixel feature matrix and the preset feature extraction information can be used to extract a feature extraction result corresponding to the to-be-detected image.

[0064] In an optional embodiment, the obtained to-be-detected image can be preprocessed, such as image noise reduction, gray processing and the like, and then the processed to-be-detected image can be subjected to image feature extraction, so as to further improve the accuracy and reliability of feature extraction.

[0065] Specifically, the GLCM (Gray-level Co-occurrence Matrix, gray level co-occurrence matrix) image processing technology can be used to extract features from the preprocessed image (i.e., the processed to-be-detected image), such as the contrast, correlation, energy and entropy of the image, to obtain a feature extraction result corresponding to the to-be-detected image. Thus, by using the GLCM algorithm for pollution detection, high accuracy and reliability can be achieved, and the problems of false positives and false negatives can be effectively avoided.

[0066] In step 103, a pollution detection result for the lens of the unmanned aerial vehicle is determined according to the feature extraction result; the pollution detection result includes a pollution type and a pollution degree.

[0067] In a specific implementation, the feature vector of the to-be-detected image can be input into a pre-trained pollution classification model, and then a pollution detection result of the lens of the unmanned aerial vehicle can be obtained according to pollution prediction information output by the pollution classification model, and the pollution detection result can include a pollution type and a pollution degree.

[0068] In an example, the pre-trained contaminant classification model can include a support vector machine, which can be used to classify different types of contaminants according to an optimal decision boundary, for example, the extracted features (i.e., feature extraction results) can be input into a classifier (i.e., a pre-trained contaminant classification model) for classification, such as a support vector machine. By using a support vector machine (SVM) algorithm for classification, the lens cleanliness (e.g., the cleanliness of the surface of the drone lens) can be classified into two categories: clean and contaminated, according to the feature vector. In the case of determining that the drone lens is contaminated, the type and degree of contamination can be further determined in order to take appropriate cleaning methods for processing.

[0069] Step 104, according to the type and degree of pollution, determine the target cleaning method, send the cleaning instruction corresponding to the target cleaning method to the target drone.

[0070] Among them, the cleaning instruction can be used to instruct the target drone to perform the cleaning operation on the drone lens according to the target cleaning method.

[0071] In actual application, the corresponding relationship between the pollution type, the pollution degree and the cleaning method can be pre-set for different pollution types and pollution degrees, and then the target cleaning method can be determined according to the pollution type and the pollution degree after determining that the drone lens is contaminated, and the cleaning instruction corresponding to the target cleaning method can be sent to the target drone.

[0072] Optionally, in the case of determining that the drone lens is contaminated, the drone lens monitoring terminal can send a cleaning instruction to the main controller of the drone (i.e., the target drone) through the corresponding communication module, to instruct the drone to start its cleaning system to clean the drone lens,

[0073] For example, for different pollution types and pollution degrees, a plurality of cleaning methods can be pre-set, which can include water spraying, air flow, laser ablation and ultrasonic oscillation, so that the optimal cleaning method can be selected for automatic cleaning according to the actual situation.

[0074] Compared with the traditional method, the technical scheme of the embodiment can automatically perceive whether the drone lens is contaminated by monitoring the cleanliness of the surface of the drone lens, and can perform corresponding cleaning operation for different pollution types and pollution degrees, which can ensure the timely and effective cleaning of the drone camera, has high automation degree, does not need manual intervention, reduces the maintenance cost, realizes all-weather automatic cleaning, and improves the efficiency and quality of the drone shooting.

[0075] In the unmanned aerial vehicle lens processing method, the image features of the to-be-detected image are extracted according to the preset feature extraction information, a feature extraction result corresponding to the to-be-detected image is obtained, the pollution detection result of the unmanned aerial vehicle lens is determined according to the feature extraction result, then the target cleaning mode is determined according to the pollution type and the pollution degree, the cleaning instruction corresponding to the target cleaning mode is sent to the target unmanned aerial vehicle, and the unmanned aerial vehicle camera is automatically cleaned. Through monitoring the cleanliness of the surface of the unmanned aerial vehicle lens, it is automatically sensed whether the unmanned aerial vehicle lens is polluted and corresponding cleaning operation is performed, the unmanned aerial vehicle camera can be effectively cleaned in time, the unmanned aerial vehicle shooting efficiency is improved, and the unmanned aerial vehicle shooting quality is ensured.

[0076] In one embodiment, as shown in Figure 2 The image features of the to-be-detected image are extracted according to the preset feature extraction information, a feature extraction result corresponding to the to-be-detected image is obtained, the pollution detection result of the unmanned aerial vehicle lens is determined according to the feature extraction result, then the target cleaning mode is determined according to the pollution type and the pollution degree, the cleaning instruction corresponding to the target cleaning mode is sent to the target unmanned aerial vehicle, and the unmanned aerial vehicle camera is automatically cleaned. Through monitoring the cleanliness of the surface of the unmanned aerial vehicle lens, it is automatically sensed whether the unmanned aerial vehicle lens is polluted and corresponding cleaning operation is performed, the unmanned aerial vehicle camera can be effectively cleaned in time, the unmanned aerial vehicle shooting efficiency is improved, and the unmanned aerial vehicle shooting quality is ensured.

[0077] In step 201, a gray image corresponding to the to-be-detected image is obtained, and the gray level of the gray image is adjusted.

[0078] In step 202, for each pixel in the adjusted gray image, frequency information corresponding to the pixel is determined, and a pixel feature matrix is generated according to the frequency information corresponding to each pixel.

[0079] In step 203, the pixel feature matrix and the preset feature extraction information are used to extract the feature extraction result corresponding to the to-be-detected image.

[0080] In actual application, as an image feature extraction method, the GLCM gray level co-occurrence matrix can be used to describe the spatial relationship between different gray levels in an image. In the process of image processing based on GLCM, the to-be-detected image can be converted into a gray image, and the size and direction of the calculation window can be defined. For example, a fixed-size calculation window can be determined for the gray image, and one or more directions such as horizontal, vertical, and diagonal can be specified. Then, the image gray level of the gray image can be normalized, for example, the gray level of the gray image can be reduced to a specified level (such as 8 levels or 16 levels), and an adjusted gray image is obtained.

[0081] In an example, after the image gray level normalization processing, the frequency (i.e. frequency information) of the pixel pair in the adjusted gray image can be calculated. For example, for each pixel, the gray level combination of the adjacent pixels in the specified direction can be obtained, and the frequency of each combination can be counted. Then, the statistical frequency information can be stored in a matrix, such as a GLCM matrix, i.e. a pixel feature matrix is generated according to the frequency information corresponding to each pixel.

[0082] For example, a grayscale image can be loaded by using the cv2.imread function, and then the number of gray levels can be defined, and then the GLCM matrix can be calculated by using the cv2.calcGLCM function.

[0083] In this embodiment, the grayscale image corresponding to the to-be-detected image is obtained, the number of gray levels of the grayscale image is adjusted, and then for each pixel in the adjusted grayscale image, the frequency information corresponding to the pixel is determined, the pixel feature matrix is generated according to the frequency information corresponding to each pixel, and then the pixel feature matrix and the preset feature extraction information are used to extract the feature extraction result corresponding to the to-be-detected image. The GLCM image processing technology can be used for pollutant detection, which helps to improve the detection accuracy and reliability to avoid misjudgment and missed judgment.

[0084] In one embodiment, the pixel feature matrix and the preset feature extraction information are used to extract the feature extraction result corresponding to the to-be-detected image, which can include the following steps:

[0085] The pixel feature matrix is normalized, different types of features are extracted from the processed pixel feature matrix according to the preset feature extraction information, the preset feature extraction information is used to indicate the type of the feature to be extracted, and the feature vector of the to-be-detected image is obtained according to the extracted different types of features as the feature extraction result.

[0086] In a specific implementation, the GLCM matrix (i.e., the pixel feature matrix) can be normalized, for example, each element in the GLCM matrix can be divided by the sum of all elements to obtain the normalized GLCM matrix, i.e., the processed pixel feature matrix.

[0087] In an example, the texture features can be calculated in combination with the preset feature extraction information and the normalized GLCM matrix, for example, various texture features such as contrast, correlation, energy, and entropy (i.e., different types of features) can be extracted from the normalized GLCM matrix, and then the extracted texture features can be used as the feature vector (i.e., the feature extraction result) of the to-be-detected image, which can be used for image classification and recognition tasks. Thus, based on the GLCM image processing technology, the degree of cleanliness of the unmanned aerial vehicle lens can be actively perceived, and through feature extraction and classification recognition, it can be determined whether there is a pollutant on the lens surface.

[0088] For example, the cv2.compareHist function can be used to calculate the Bhattacharyya distance between the GLCM matrix and the all-zero matrix as the measurement value of the contrast.

[0089] For example, by using Python and OpenCV libraries, the steps of the GLCM algorithm can be implemented as follows:

[0090] import cv2

[0091] import numpy as np

[0092] # Load image

[0093] img = cv2.imread('image.jpg', 0)

[0094] # Define number of gray levels

[0095] levels = 256

[0096] # Calculate GLCM matrix

[0097] glcm = cv2.calcGLCM(img, [1], None, levels, levels)

[0098] # Calculate statistical information

[0099] contrast = cv2.compareHist(xxxx)

[0100] # Display results

[0101] print("Contrast:", contrast)

[0102] In this embodiment, the pixel feature matrix is normalized, and then different types of features are extracted from the processed pixel feature matrix according to the pre-set feature extraction information. Then, the feature vector of the image to be detected is obtained according to the extracted different types of features, as the feature extraction result, which can realize automatic perception of whether the unmanned aerial vehicle lens is contaminated, and improve the accuracy of contaminant detection.

[0103] In one embodiment, the step of determining the contaminant detection result for the unmanned aerial vehicle lens according to the feature extraction result can include the following steps:

[0104] inputting the feature vector of the image to be detected into a pre-trained contaminant classification model; and obtaining the contaminant detection result of the unmanned aerial vehicle lens according to the contaminant prediction information output by the contaminant classification model.

[0105] The pre-trained contaminant classification model can include a support vector machine, which is a binary classification model and can be used to classify different types of contaminants according to an optimal decision boundary.

[0106] In an example, when performing feature classification according to the feature vector (i.e., feature extraction result) of the image to be detected, the feature vector can be input into a classifier (i.e., a pre-trained pollutant classification model) for classification, such as a support vector machine (SVM), an artificial neural network (ANN), a convolutional neural network (CNN), etc., which is not specifically limited in the present embodiment.

[0107] In yet another example, based on a support vector machine (SVM), the following steps can be used for processing:

[0108] 1. Collect data and pre-process: Data can be collected and pre-processed, including but not limited to data cleaning, noise removal, etc.

[0109] 2. Feature extraction: Feature extraction can be performed on the data, which can be used to describe the characteristics of the data by converting the data from its original form to a set of feature vectors.

[0110] 3. Label data: Data can be labeled, such as labeling each data point with the class it belongs to.

[0111] 4. Data division: Data can be divided into training set and test set.

[0112] 5. Train model: The SVM model can be trained using the training set to determine the optimal decision boundary and support vectors. Based on the SVM algorithm, different classes of data can be classified by finding the optimal decision boundary, which is a hyperplane for two-dimensional data, a straight line for two-dimensional data, and a hyperplane for multi-dimensional data. The SVM algorithm can determine the optimal decision boundary and support vectors by solving a convex quadratic programming problem, which can be solved using optimization algorithms.

[0113] 6. Model evaluation: The model performance can be evaluated using the test set to obtain indicators such as accuracy, recall, F1 value, etc.

[0114] 7. Apply model: The trained SVM model (i.e., pre-trained pollutant classification model) can be used for data classification tasks, and subsequent operations can be performed based on the prediction results output by the model.

[0115] In an optional embodiment, a kernel function can be selected during the training of the SVM model to map low-dimensional features to high-dimensional feature space, which can include linear kernel function, polynomial kernel function, and Gaussian kernel function, etc.

[0116] For example, the steps of the support vector machine (SVM) algorithm can be implemented using the scikit-learn library in Python:

[0117] # Import necessary libraries and datasets

[0118] from sklearn import datasets

[0119] from sklearn.model_selection import train_test_split

[0120] from sklearn import svm

[0121] # Load the iris dataset

[0122] iris = datasets.load_iris()

[0123] # Split the dataset into training and testing sets

[0124] X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.4, random_state=0)

[0125] # Create an SVM classifier

[0126] clf = svm.SVC(kernel='linear', C=1).fit(X_train, y_train)

[0127] # Make predictions on the test set

[0128] y_pred = clf.predict(X_test)

[0129] # Output the accuracy of the model

[0130] print("Accuracy:", clf.score(X_test, y_test))

[0131] In this embodiment, by inputting the feature vector of the to-be-detected image into the pre-trained pollutant classification model, and then obtaining the pollutant detection result of the unmanned aerial vehicle lens according to the pollutant prediction information output by the pollutant classification model, the pollution of the unmanned aerial vehicle lens can be automatically perceived and classified, which provides data support for further cleaning operation according to different pollution types and pollution degrees, and improves the unmanned aerial vehicle shooting efficiency.

[0132] In one embodiment, before the step of extracting image features of the to-be-detected image according to the preset feature extraction information to obtain a feature extraction result corresponding to the to-be-detected image, the following steps can also be included:

[0133] According to the pre-processing operation information, the image to be detected is pre-processed to obtain a processed image to be detected; and the processed image to be detected is used to perform the step of extracting the image features of the image to be detected according to the preset feature extraction information, to obtain the feature extraction result corresponding to the image to be detected.

[0134] In actual application, the acquired image to be detected can be pre-processed according to the pre-processing operation information, which can include but is not limited to image noise reduction, gray processing and other operations, and then the image features of the processed image to be detected can be extracted, so as to further improve the accuracy and reliability of feature extraction.

[0135] In the embodiment, the image to be detected is pre-processed according to the pre-processing operation information to obtain a processed image to be detected, and then the processed image to be detected is used to perform the step of extracting the image features of the image to be detected according to the preset feature extraction information, to obtain the feature extraction result corresponding to the image to be detected, which can help improve the accuracy and reliability of feature extraction.

[0136] In one embodiment, the target unmanned aerial vehicle is configured with a monitoring device, and after the step of sending the cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle, the following steps can be included:

[0137] Obtaining lens state information collected by the monitoring device; the lens state information is obtained by real-time monitoring of the cleaning state of the unmanned aerial vehicle lens; generating a cleaning feedback result for the unmanned aerial vehicle lens according to the lens state information; the cleaning feedback result is used to represent the processing condition of the cleaning operation performed on the unmanned aerial vehicle lens.

[0138] In specific implementation, the target unmanned aerial vehicle can be configured with a monitoring device, which can be used to monitor the cleaning state of the unmanned aerial vehicle lens (i.e. lens state information) in real time, and the monitoring result (i.e. cleaning feedback result) can be fed back to the main controller and the operator through the communication module, so as to further maintain and maintain the unmanned aerial vehicle. Thus it can be applied to unmanned aerial vehicle shooting in different environments and conditions, has wide application, and the monitoring device has simple structure, high reliability, easy maintenance and replacement.

[0139] In the embodiment, by obtaining the lens state information collected by the monitoring device, and then generating a cleaning feedback result for the unmanned aerial vehicle lens according to the lens state information, the unmanned aerial vehicle camera can be cleaned in time and the information can be fed back, so as to ensure the quality of unmanned aerial vehicle shooting.

[0140] In one embodiment, as Figure 3As shown, a flowchart of another unmanned aerial vehicle lens processing method is provided. In this embodiment, the method comprises the following steps:

[0141] In step 301, an image to be detected captured by a unmanned aerial vehicle lens of a target unmanned aerial vehicle is acquired, and the image to be detected is subjected to image preprocessing according to preprocessing operation information, to obtain a processed image to be detected. In step 302, a gray-scale image corresponding to the image to be detected is acquired, and the gray-scale level of the gray-scale image is adjusted. For each pixel in the adjusted gray-scale image, frequency information corresponding to the pixel is determined, and a pixel feature matrix is generated according to the frequency information corresponding to each pixel. In step 303, the pixel feature matrix is subjected to normalization processing, and different types of features are extracted from the processed pixel feature matrix according to preset feature extraction information. In step 304, a feature vector of the image to be detected is obtained according to the different types of features extracted, as a feature extraction result. In step 305, the feature vector of the image to be detected is input into a pre-trained pollutant classification model, and a pollutant detection result of the unmanned aerial vehicle lens is obtained according to pollutant prediction information output by the pollutant classification model. In step 306, a target cleaning mode is determined according to the pollution type and the pollution degree, and a cleaning instruction corresponding to the target cleaning mode is sent to the target unmanned aerial vehicle. In step 307, lens state information collected by a monitoring device is acquired, and a cleaning feedback result for the unmanned aerial vehicle lens is generated according to the lens state information. It should be noted that the specific limitations of the above steps can be referred to the specific limitations of the above-mentioned one unmanned aerial vehicle lens processing method, which will not be repeated here.

[0142] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiments of the present application also provide a unmanned aerial vehicle lens processing device for implementing the above-mentioned unmanned aerial vehicle lens processing method. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more unmanned aerial vehicle lens processing device embodiments provided below can be referred to the limitations of the unmanned aerial vehicle lens processing method described above, which will not be repeated here.

[0144] In one embodiment, as shown in Figure 4 A UAV lens processing apparatus is provided, comprising:

[0145] An image to be detected acquisition module 401 is configured to acquire an image to be detected captured by a UAV lens of a target UAV; the image to be detected is used to detect the attachment of pollutants on the surface of the UAV lens;

[0146] An image feature extraction module 402 is configured to extract image features of the image to be detected according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected; the preset feature extraction information is determined based on the detection of pollutants on the surface of the UAV lens;

[0147] A pollutant detection result obtaining module 403 is configured to determine a pollutant detection result for the UAV lens according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree;

[0148] A cleaning instruction sending module 404 is configured to determine a target cleaning mode according to the pollution type and the pollution degree, and send a cleaning instruction corresponding to the target cleaning mode to the target UAV; the cleaning instruction is used to instruct the target UAV to perform a cleaning operation on the UAV lens according to the target cleaning mode.

[0149] In one embodiment, the image feature extraction module 402 comprises:

[0150] A gray level adjustment sub-module is configured to acquire a gray image corresponding to the image to be detected, and adjust the gray level of the gray image;

[0151] A feature matrix generation sub-module is configured to determine frequency information corresponding to each pixel in the adjusted gray image, and generate a pixel feature matrix according to the frequency information corresponding to each pixel;

[0152] A feature extraction result obtaining sub-module is configured to extract the feature extraction result corresponding to the image to be detected by using the pixel feature matrix and the preset feature extraction information.

[0153] In one embodiment, the feature extraction result obtaining sub-module comprises:

[0154] A normalization processing unit is configured to perform normalization processing on the pixel feature matrix;

[0155] A feature extraction unit is configured to extract different types of features from the processed pixel feature matrix according to the preset feature extraction information; the preset feature extraction information is used to indicate the type of feature to be extracted;

[0156] a feature vector obtaining unit, configured to obtain a feature vector of the image to be detected according to the different types of features extracted, as the feature extraction result.

[0157] In an embodiment, the pollution detection result obtaining module 403 comprises:

[0158] a model processing sub-module, configured to input the feature vector of the image to be detected into a pre-trained pollution classification model;

[0159] a pollution classification sub-module, configured to obtain a pollution detection result of the lens of the unmanned aerial vehicle according to pollution prediction information output by the pollution classification model;

[0160] The pre-trained pollution classification model comprises a support vector machine, and the support vector machine is configured to classify different types of pollution according to an optimal decision boundary.

[0161] In an embodiment, the apparatus further comprises:

[0162] a preprocessing module, configured to perform image preprocessing on the image to be detected according to preprocessing operation information, to obtain a processed image to be detected;

[0163] an execution feature extraction module, configured to perform the step of extracting image features of the image to be detected according to the preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected, by using the processed image to be detected.

[0164] In an embodiment, the target unmanned aerial vehicle is configured with a monitoring apparatus, and the apparatus further comprises:

[0165] a monitoring module, configured to acquire lens state information collected by the monitoring apparatus; the lens state information is obtained by monitoring a cleaning state of the lens of the unmanned aerial vehicle in real time;

[0166] a cleaning feedback module, configured to generate a cleaning feedback result for the lens of the unmanned aerial vehicle according to the lens state information; the cleaning feedback result is used to represent a processing condition of performing a cleaning operation on the lens of the unmanned aerial vehicle.

[0167] The above modules in the unmanned aerial vehicle lens processing apparatus can be all or partially implemented by software, hardware, and combinations thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0168] In an embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as follows:Figure 5 The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals, and wireless communication can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a kind of unmanned aerial vehicle lens processing method.

[0169] Those skilled in the art can understand that, Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0170] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the following steps:

[0171] An image to be detected obtained by an unmanned aerial vehicle lens of a target unmanned aerial vehicle is acquired; the image to be detected is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0172] Image features of the image to be detected are extracted according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected; the preset feature extraction information is determined based on the detection of pollutants on the surface of the unmanned aerial vehicle lens;

[0173] According to the feature extraction result, a pollutant detection result for the unmanned aerial vehicle lens is determined; the pollutant detection result includes a pollution type and a pollution degree;

[0174] According to the pollution type and the pollution degree, a target cleaning mode is determined, and a cleaning instruction corresponding to the target cleaning mode is sent to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the unmanned aerial vehicle lens according to the target cleaning mode.

[0175] In one embodiment, the processor, when executing the computer program, also implements the steps of the unmanned aerial vehicle lens processing method in the other embodiments.

[0176] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program, when executed by a processor, implements the following steps:

[0177] An image to be detected is obtained by photographing a target unmanned aerial vehicle lens; the image to be detected is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0178] Image features of the image to be detected are extracted according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected; the preset feature extraction information is determined based on the detection of pollutants on the surface of the unmanned aerial vehicle lens;

[0179] A pollutant detection result for the unmanned aerial vehicle lens is determined according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree;

[0180] A target cleaning mode is determined according to the pollution type and the pollution degree, and a cleaning instruction corresponding to the target cleaning mode is sent to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the unmanned aerial vehicle lens according to the target cleaning mode.

[0181] In one embodiment, the processor, when executing the computer program, also implements the steps of the unmanned aerial vehicle lens processing method in the other embodiments.

[0182] In one embodiment, a computer program product is provided, and the computer program product includes a computer program. The computer program, when executed by a processor, implements the following steps:

[0183] An image to be detected is obtained by photographing a target unmanned aerial vehicle lens; the image to be detected is used to detect the attachment of pollutants on the surface of the unmanned aerial vehicle lens;

[0184] Image features of the image to be detected are extracted according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected; the preset feature extraction information is determined based on the detection of pollutants on the surface of the unmanned aerial vehicle lens;

[0185] A pollutant detection result for the unmanned aerial vehicle lens is determined according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree;

[0186] According to the pollution type and the pollution degree, a target cleaning mode is determined, and a cleaning instruction corresponding to the target cleaning mode is sent to the target UAV; the cleaning instruction is used to instruct the target UAV to perform a cleaning operation on the UAV lens according to the target cleaning mode.

[0187] In one embodiment, the computer program, when executed by the processor, also implements the steps of the UAV lens processing method in the other embodiments described above.

[0188] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0190] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0191] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for processing a drone lens, characterized in that, The method comprises: obtaining a to-be-detected image captured by a drone lens of a target drone; the to-be-detected image is used for detecting the attachment of pollutants on the surface of the drone lens; extracting image features of the to-be-detected image according to preset feature extraction information, to obtain a feature extraction result corresponding to the to-be-detected image; the preset feature extraction information is determined based on pollutant detection on the surface of the drone lens; determining a pollutant detection result for the drone lens according to the feature extraction result; the pollutant detection result comprises a pollution type and a pollution degree; determining a target cleaning mode according to the pollution type and the pollution degree, and sending a cleaning instruction corresponding to the target cleaning mode to the target drone; the cleaning instruction is used to instruct the target drone to perform a cleaning operation on the drone lens according to the target cleaning mode; The method further comprises: obtaining a to-be-detected image corresponding to a gray image, adjusting the gray level of the gray image; for each pixel in the adjusted gray image, determining frequency information corresponding to the pixel, generating a pixel feature matrix according to the frequency information corresponding to each pixel; normalizing the pixel feature matrix; extracting different types of features from the processed pixel feature matrix according to the preset feature extraction information; the preset feature extraction information is used to indicate the type of features to be extracted; obtaining a feature vector of the to-be-detected image as the feature extraction result according to the extracted different types of features; 2. The method of claim 1, wherein, The method further comprises: inputting the feature vector of the to-be-detected image into a pre-trained pollutant classification model; obtaining the pollutant detection result of the drone lens according to pollutant prediction information output by the pollutant classification model; wherein the pre-trained pollutant classification model comprises a support vector machine, and the support vector machine is used to classify different types of pollutants according to an optimal decision boundary. Before the step of extracting image features of the to-be-detected image according to preset feature extraction information to obtain a feature extraction result corresponding to the to-be-detected image, the method further comprises:

3. The method according to claim 1 or 2, characterized in that, performing image preprocessing on the to-be-detected image according to preprocessing operation information to obtain a processed to-be-detected image; using the processed to-be-detected image to perform the step of extracting image features of the to-be-detected image according to preset feature extraction information to obtain a feature extraction result corresponding to the to-be-detected image. The target drone is configured with a monitoring device, and after the step of sending a cleaning instruction corresponding to the target cleaning mode to the target drone, the method further comprises: obtaining lens state information collected by the monitoring device; the lens state information is obtained by real-time monitoring of the cleaning state of the drone lens; According to the lens state information, a cleaning feedback result for the unmanned aerial vehicle lens is generated; the cleaning feedback result is used to represent a processing condition of performing a cleaning operation on the unmanned aerial vehicle lens.

4. The method of claim 1, wherein, The image features include contrast, correlation, energy and entropy of the image to be detected.

5. A drone lens processing apparatus, characterized by, The device comprises: An image to be detected acquisition module is configured to acquire an image to be detected captured by a lens of a target unmanned aerial vehicle, and the image to be detected is used to detect an attachment condition of pollutants on a surface of the lens of the unmanned aerial vehicle. An image feature extraction module is configured to extract image features of the image to be detected according to preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected; the preset feature extraction information is determined based on a pollutant detection on the surface of the lens of the unmanned aerial vehicle. A pollutant detection result obtaining module is configured to determine a pollutant detection result for the lens of the unmanned aerial vehicle according to the feature extraction result; the pollutant detection result includes a pollution type and a pollution degree. A cleaning instruction sending module is configured to determine a target cleaning mode according to the pollution type and the pollution degree, and send a cleaning instruction corresponding to the target cleaning mode to the target unmanned aerial vehicle; the cleaning instruction is used to instruct the target unmanned aerial vehicle to perform a cleaning operation on the lens of the unmanned aerial vehicle according to the target cleaning mode. The image feature extraction module comprises: A gray level adjustment sub-module is configured to acquire a gray image corresponding to the image to be detected, and adjust a gray level of the gray image. A feature matrix generation sub-module is configured to determine frequency information corresponding to each pixel in the adjusted gray image, and generate a pixel feature matrix according to the frequency information corresponding to each pixel. A normalization processing unit is configured to perform normalization processing on the pixel feature matrix. A feature extraction unit is configured to extract different types of features from the processed pixel feature matrix according to the preset feature extraction information; the preset feature extraction information is used to indicate the types of features to be extracted. A feature vector obtaining unit is configured to obtain a feature vector of the image to be detected according to the different types of features extracted, as the feature extraction result. The pollutant detection result obtaining module comprises: A model processing sub-module is configured to input the feature vector of the image to be detected into a pre-trained pollutant classification model. A pollutant classification sub-module is configured to obtain a pollutant detection result of the lens of the unmanned aerial vehicle according to pollutant prediction information output by the pollutant classification model. The pre-trained pollutant classification model comprises a support vector machine, and the support vector machine is used to classify different types of pollutants according to an optimal decision boundary.

6. The apparatus of claim 5, wherein, The device further comprises: A preprocessing module is configured to perform image preprocessing on the image to be detected according to preprocessing operation information, to obtain a processed image to be detected. An execution extraction feature module is configured to perform the step of extracting image features of the image to be detected according to the preset feature extraction information, to obtain a feature extraction result corresponding to the image to be detected, by using the processed image to be detected.

7. The apparatus of claim 5 or 6, wherein, The target unmanned aerial vehicle is configured with a monitoring device, and the device further comprises: a monitoring module, configured to acquire lens state information collected by the monitoring device; the lens state information is obtained by monitoring the cleaning state of the unmanned aerial vehicle lens in real time; a cleaning feedback module, configured to generate a cleaning feedback result for the unmanned aerial vehicle lens according to the lens state information; the cleaning feedback result is used to represent the processing condition of performing cleaning operation on the unmanned aerial vehicle lens.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 4.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 4.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 4. The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 4.

Images