Machine vision unmanned aerial vehicle curtain wall cleaning control system and method

Abstract

The invention provides a machine vision unmanned aerial vehicle curtain wall cleaning control system and method, and belongs to the field of image processing. The system comprises a pollution area identification module, a visual distance processing module, a distance measurement fusion processing module and a pollution level identification module. Wherein the pollution area identification module is used for acquiring a first pollution area, a second pollution area and a pollution similarity coefficient; the visual distance processing module is used for obtaining a first curtain wall distance; the distance measurement fusion processing module is used for collecting a second curtain wall distance and obtaining a curtain wall distance in combination with the first curtain wall distance; the pollution level identification module is used for obtaining the pollution level and carrying out unmanned aerial vehicle cleaning control. The technical problem of insufficient cleaning control precision caused by inaccurate distance measurement during curtain wall cleaning of the unmanned aerial vehicle in the prior art is solved, and the technical effects of improving the curtain wall distance measurement precision and realizing accurate cleaning control are achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a machine vision-based unmanned aerial vehicle curtain wall cleaning control system and method. Background Art

[0002] As urban buildings continue to rise in height, high-rise curtain wall cleaning has become a crucial component of building maintenance. Traditional manual cleaning methods pose safety risks and are expensive, leading drone cleaning technology to become a key development direction for curtain wall cleaning.

[0003] Existing drone-based curtain wall cleaning systems typically use a single distance sensor, such as a laser rangefinder or ultrasonic sensor, to measure distance. However, in practice, this single distance measurement method often suffers from low accuracy due to factors such as the diverse surface materials and reflective properties of curtain walls, as well as changes in ambient lighting. Inaccurate distance measurement between the drone and the curtain wall can lead to improper distance control between the cleaning device and the curtain wall surface, resulting in insufficient cleaning control precision and poor cleaning results. Summary of the Invention

[0004] The present invention aims to solve the technical problem in the prior art of inaccurate distance measurement during UAV curtain wall cleaning, resulting in insufficient cleaning control precision, and provides a machine vision UAV curtain wall cleaning control system and method to solve the problem.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] In a first aspect, the present invention provides a machine vision-based UAV curtain wall cleaning control system, which is applied to a UAV and includes: a pollution area recognition module, which is used to collect a first image and a second image on a target curtain wall through a binocular vision sensor, identify and obtain the corresponding first pollution area and second pollution area, and obtain a pollution similarity coefficient; a visual distance processing module, which is used to process and obtain the first curtain wall distance based on the pixel coordinates of the first pollution area and the second pollution area in the first image and the second image; a ranging fusion processing module, which is used to collect the second curtain wall distance through a ranging sensor, configure visual weights and ranging weights according to the pollution similarity coefficient, and perform weighted calculation on the first curtain wall distance and the second curtain wall distance to obtain the curtain wall distance; a pollution level recognition module, which is used to configure a pollution recognition coefficient according to the curtain wall distance and the pollution similarity coefficient, perform image cropping and pollution level recognition on the first pollution area and the second pollution area, obtain the pollution level, and perform UAV cleaning control.

[0007] In a second aspect, the present invention provides a machine vision-based drone curtain wall cleaning control method, comprising: using a binocular vision sensor to collect a first image and a second image on a target curtain wall, identifying and obtaining corresponding first and second contaminated areas, and obtaining a pollution similarity coefficient; processing and obtaining a first curtain wall distance based on the pixel coordinates of the first and second contaminated areas in the first and second images; using a ranging sensor to collect a second curtain wall distance, configuring a visual weight and a ranging weight based on the pollution similarity coefficient, performing a weighted calculation on the first and second curtain wall distances, and obtaining the curtain wall distance; configuring a pollution identification coefficient based on the curtain wall distance and the pollution similarity coefficient, performing image cropping and pollution level identification on the first and second contaminated areas, obtaining the pollution level, and performing drone cleaning control.

[0008] The beneficial effects of the present invention are:

[0009] The first image and the second image of the target curtain wall are collected by the binocular vision sensor, the corresponding first and second polluted areas are identified and obtained, and the pollution similarity coefficient is obtained, so as to achieve accurate identification and matching of the same polluted area; according to the pixel coordinates of the first and second polluted areas in the first and second images, the first curtain wall distance is obtained by processing, and the image-based distance measurement is realized by using the stereo imaging principle of binocular vision; at the same time, the second curtain wall distance is collected by the ranging sensor, and the visual weight and ranging weight are configured according to the pollution similarity coefficient. The curtain wall distance is obtained by weighted calculation of the first and second curtain wall distances, and the accuracy and reliability of the distance measurement are improved through multi-sensor data fusion; then, the pollution identification coefficient is configured according to the curtain wall distance and the pollution similarity coefficient, the image of the first and second polluted areas is cropped and the pollution level is identified to obtain the pollution level, and the drone cleaning control is carried out according to the pollution level to realize adaptive and precise cleaning based on the pollution degree.

[0010] Through the above technical solution, the curtain wall distance can be accurately measured and the pollution level can be identified, thereby achieving precise cleaning control. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flow chart of a machine vision-based UAV curtain wall cleaning control system provided by the present invention;

[0012] Figure 2 This is a structural schematic diagram of a machine vision-based UAV curtain wall cleaning control method provided by the present invention.

[0013] In the accompanying drawings, the components represented by the reference numerals are as follows:

[0014] Polluted area identification module 11, visual distance processing module 12, ranging fusion processing module 13, pollution level identification module 14. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0017] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0018] Example 1, as Figure 1 As shown, an embodiment of the present invention provides a machine vision UAV curtain wall cleaning control system, which is applied to a UAV, that is, the UAV curtain wall cleaning control system can be deployed as a software system in the control unit of the UAV to control the UAV to perform curtain wall cleaning operations.

[0019] Specifically, the system receives data collected by hardware devices such as binocular vision sensors and ranging sensors carried by the drone, runs the pollution area identification module 11, visual distance processing module 12, ranging fusion processing module 13 and pollution level identification module 14, accurately measures the distance between the drone and the curtain wall and identifies the pollution level, thereby providing support for the drone's flight control system and cleaning actuator, enabling the drone to accurately control the cleaning of the curtain wall and improve the curtain wall cleaning effect.

[0020] The UAV curtain wall cleaning control system includes:

[0021] The contaminated area identification module 11 is used to collect a first image and a second image on the target curtain wall through a binocular vision sensor, identify and obtain the corresponding first contaminated area and second contaminated area, and obtain a contamination similarity coefficient.

[0022] Specifically, the binocular vision sensor includes two left and right cameras, which respectively capture images of the target curtain wall from different perspectives, and the first image and the second image obtained have a certain parallax relationship. Among them, the target curtain wall refers to the specific curtain wall area to be cleaned. The contaminated area identification module 11 preprocesses and extracts features from the collected first and second images, and uses the image recognition algorithm to identify the contaminated area in the image to obtain the first contaminated area and the second contaminated area. Among them, the first contaminated area is the contaminated area identified from the first image, and the second contaminated area is the contaminated area identified from the second image. For example, a threshold segmentation algorithm based on color space conversion, an edge detection algorithm, a morphological processing algorithm or a semantic segmentation algorithm based on deep learning can be used to accurately separate the contaminated area from the normal curtain wall surface by analyzing the grayscale value, color features, texture features and other parameters of the pixels.

[0023] After acquiring the first and second contaminated areas, the contaminated area identification module 11 analyzes the similarity between the first and second contaminated areas in terms of shape, texture, color, and other characteristics, and calculates a contamination similarity coefficient. This coefficient reflects the consistency and reliability of the binocular vision sensor's recognition of the same contaminated area, providing a reference for subsequent distance measurement and pollution level determination. A higher contamination similarity coefficient indicates a more accurate binocular vision sensor's recognition of the same contaminated area and a more reliable distance measurement accuracy.

[0024] Through the contaminated area identification module 11, the curtain wall pollution can be accurately located and reliably identified, effectively avoiding the misjudgment and missed detection problems in traditional methods, laying the foundation for the intelligent cleaning operation of the drone, thereby improving the pertinence and efficiency of the cleaning operation.

[0025] The visual distance processing module 12 is configured to obtain a first curtain wall distance based on the pixel coordinates of the first polluted area and the second polluted area in the first image and the second image.

[0026] Specifically, the visual distance processing module 12 is responsible for determining the distance between the drone and the target curtain wall based on the first and second images captured by the binocular vision sensor and the identified first and second contaminated areas. The first curtain wall distance is obtained by analyzing the pixel coordinates of the first and second contaminated areas within the first and second images.

[0027] First, the visual distance processing module 12 extracts the pixel coordinates of the first contaminated area in the first image and the corresponding pixel coordinates of the second contaminated area in the second image to form a coordinate correspondence. Due to the baseline distance between the left and right cameras of the binocular vision sensor, the same contaminated area will produce pixel coordinate offset in the two images, that is, parallax. The visual distance processing module 12 obtains the pixel offset by calculating the deviation value between the pixel coordinates of the first contaminated area and the second contaminated area. Subsequently, based on the binocular vision ranging principle, combined with the internal parameters (such as focal length, optical center coordinates, etc.) and external parameters (such as baseline distance, rotation matrix, etc.) of the binocular vision sensor, the visual distance processing module 12 converts the pixel offset into an actual spatial distance, thereby obtaining the first curtain wall distance. The first curtain wall distance represents the distance value between the drone and the target curtain wall surface obtained based on the visual measurement method.

[0028] The visual distance processing module 12 can realize non-contact distance measurement, and provide spatial positioning information and pollution level determination reference information for the flight control and cleaning operation of the UAV.

[0029] The ranging fusion processing module 13 is used to collect the second curtain wall distance through the ranging sensor, configure the visual weight and the ranging weight according to the pollution similarity coefficient, and perform weighted calculation on the first curtain wall distance and the second curtain wall distance to obtain the curtain wall distance.

[0030] Specifically, the distance measurement fusion processing module 13 independently collects the second curtain wall distance through the distance measurement sensor, and combines it with the first curtain wall distance obtained by the visual distance processing module 12 to perform fusion processing to obtain a more accurate curtain wall distance.

[0031] First, the ranging fusion processing module 13 uses a ranging sensor such as a laser rangefinder to directly measure the distance between the drone and the target curtain wall surface by emitting a laser beam and receiving the reflected signal to obtain a second curtain wall distance. This second curtain wall distance has the advantages of high measurement accuracy and is not affected by lighting conditions, but may be affected by reflection interference in complex environments. In order to give full play to the advantages of the two ranging methods, the ranging fusion processing module 13 dynamically configures the visual weight and ranging weight based on the pollution similarity coefficient provided by the pollution area identification module 11. When the pollution similarity coefficient is high, it indicates that the reliability of binocular visual ranging is good, and the visual weight will be increased; conversely, when the pollution similarity coefficient is low, the ranging weight will be increased, and more reliance will be placed on the laser ranging results.

[0032] By performing a weighted calculation on the first and second curtain wall distances, the distance measurement fusion processing module 13 obtains a fused curtain wall distance. This fused curtain wall distance combines the advantages of visual ranging and laser ranging, effectively improving ranging accuracy and robustness, and providing a reliable distance reference for subsequent pollution level identification and cleaning control.

[0033] The pollution level identification module 14 is used to configure the pollution identification coefficient according to the curtain wall distance and the pollution similarity coefficient, perform image cropping and pollution level identification on the first pollution area and the second pollution area, obtain the pollution level, and perform drone cleaning control.

[0034] Specifically, the pollution level identification module 14 is responsible for accurately identifying the degree of pollution and generating a corresponding pollution level based on the curtain wall distance and the pollution similarity coefficient, supporting drone cleaning control. By analyzing the combined impact of curtain wall distance and the pollution similarity coefficient on image recognition accuracy, the pollution identification coefficient is dynamically configured to ensure accurate judgment of pollution levels under different operating conditions.

[0035] First, the pollution level identification module 14 analyzes the curtain wall distance provided by the ranging fusion processing module 13. When the curtain wall distance is large, the image resolution captured by the binocular vision sensor is relatively reduced, affecting the clarity of pollution details. Simultaneously, the pollution level identification module 14 considers the pollution similarity coefficient provided by the pollution area identification module 11. A smaller pollution similarity coefficient indicates poor consistency among the polluted areas in the binocular images, further impacting recognition reliability. Based on the acquired curtain wall distance and pollution similarity coefficient, the pollution level identification module 14 determines the corresponding pollution identification coefficient. Based on this pollution identification coefficient, the pollution level identification module 14 accurately crops the first and second polluted areas in the first and second images, extracting the polluted area images containing key pollution information to reduce background interference and improve recognition efficiency. Next, the pollution level identification module 14 analyzes the cropped polluted area images, comprehensively considering characteristic parameters such as the area percentage of the pollution, color depth, and texture complexity, to determine the pollution level. Based on the determined pollution level, the corresponding cleaning control plan can be automatically matched, achieving intelligent and precise control of the drone's cleaning operations. For example, when the pollution level is heavy pollution, deep cleaning is activated, and a composite cleaning method of "detergent + water mist" is adopted, and the spray pressure is set to 0.8-1.2MPa to ensure the effective removal of stubborn stains; when the pollution level is medium to light pollution, conventional cleaning is enabled, and a "pure water mist" cleaning method is adopted, and the spray pressure is controlled at 0.5-0.8MPa, which can both effectively clean and save resources; after the cleaning operation is completed, the water film thickness on the target curtain wall surface is detected by an infrared sensor. When the surface water film thickness is detected to be ≤0.1mm, it automatically switches to the quick drying mode, and the pure airflow mode is used to dry the target curtain wall surface to ensure the cleaning quality.

[0036] Through the pollution level identification module 14, the optimal cleaning strategy can be adaptively selected according to the actual pollution situation, which not only ensures the cleaning effect but also avoids waste of resources, improves the efficiency and quality of the UAV curtain wall cleaning operation, and realizes precise and intelligent cleaning control.

[0037] Furthermore, the contaminated area identification module 11 is further configured to:

[0038] Capturing a first image and a second image on a target curtain wall through a binocular vision sensor;

[0039] Identifying and acquiring polluted pixels in the first image and the second image through image recognition, and obtaining a first polluted pixel distribution and a second polluted pixel distribution when polluted pixels are identified;

[0040] Randomly select and optimize the polluted area within the first polluted pixel distribution and the second polluted pixel distribution to obtain the first polluted area and the second polluted area, as well as the pollution similarity coefficient.

[0041] In one feasible implementation, the contaminated area identification module 11 first uses a binocular vision sensor to synchronously capture a first image and a second image of the target curtain wall, ensuring temporal and spatial consistency between the two images. Subsequently, the contaminated area identification module 11 uses an image recognition algorithm to perform pixel-level analysis on the first and second images. By setting contamination detection thresholds, color feature matching, texture analysis, and other methods, it identifies and marks potentially contaminated pixels pixel by pixel. When contaminated pixels are identified, the contaminated pixel set in the first image is extracted to form a first contaminated pixel distribution, and the contaminated pixel set in the second image is extracted to form a second contaminated pixel distribution.

[0042] After obtaining the first contaminated pixel distribution and the second contaminated pixel distribution, the contaminated area identification module 11 further determines the specific range of the contaminated area. Since the contaminated pixel points may be discretely distributed or there is noise interference, the contaminated area identification module 11 adopts a random selection and optimization strategy for the contaminated area. Specifically, multiple random selections are performed within the first contaminated pixel distribution and the second contaminated pixel distribution to generate candidate contaminated areas of different sizes and positions, and the pollution concentration, connectivity and effectiveness of each candidate area are evaluated through an optimization algorithm to finally determine the optimal first contaminated area and second contaminated area. Subsequently, the contaminated area identification module 11 simultaneously calculates the feature similarity between the first contaminated area and the second contaminated area, including shape matching, pixel distribution similarity and pollution degree consistency, and comprehensively obtains the pollution similarity coefficient to provide an evaluation reference for subsequent distance measurement and level identification.

[0043] Furthermore, the contaminated area identification module 11 is further configured to:

[0044] Using a preset image window, randomly select a first assumed polluted area within the first polluted pixel distribution, and randomly select a second assumed polluted area within the second polluted pixel distribution;

[0045] Calculating pixel similarity between the first assumed contaminated area and the second assumed contaminated area to obtain a first assumed contamination similarity coefficient;

[0046] Continue to randomly select the assumed contaminated area and calculate the pixel similarity until the traversal is completed. Output the first assumed contaminated area and the second assumed contaminated area corresponding to the maximum pixel similarity as the first contaminated area and the second contaminated area, and output the maximum pixel similarity as the contamination similarity coefficient.

[0047] In a preferred embodiment, the contaminated region identification module 11 first sets a preset image window of fixed size to frame a candidate contaminated region within the contaminated pixel distribution. A position within the first contaminated pixel distribution is randomly selected, and the preset image window is placed around that position to frame a first hypothetical contaminated region. Similarly, a corresponding position within the second contaminated pixel distribution is randomly selected to frame a second hypothetical contaminated region.

[0048] Subsequently, the polluted area identification module 11 compares and analyzes the pixels in the first assumed polluted area and the second assumed polluted area one by one, calculates the similarity between the two areas in terms of pixel grayscale value, color distribution, texture characteristics, etc., and quantifies the first assumed pollution similarity coefficient. The larger the value of the first assumed pollution similarity coefficient, the higher the degree of matching between the two assumed polluted areas. In order to find the optimal polluted area matching pair, the polluted area identification module 11 adopts a traversal optimization strategy, and continues to perform multiple random selections within the first polluted pixel distribution and the second polluted pixel distribution, and calculates the corresponding assumed pollution similarity coefficient after each selection. Record and compare the assumed pollution similarity coefficients obtained in all selection processes until the traversal selection of the two polluted pixel distributions is completed.

[0049] The contaminated area identification module 11 then outputs the first and second hypothesized contaminated areas with the greatest pixel similarity as the final first and second contaminated areas, and outputs the corresponding maximum pixel similarity as the contamination similarity coefficient. This optimized selection mechanism yields the most representative and matching contaminated area pairs, providing a high-quality data foundation for subsequent distance measurement and contamination level identification.

[0050] Furthermore, the visual distance processing module 12 is further configured to:

[0051] Obtaining pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image to obtain first pixel coordinates and second pixel coordinates;

[0052] The deviation between the first pixel coordinate and the second pixel coordinate is calculated to obtain a pixel offset, and the first curtain wall distance is calculated based on binocular vision.

[0053] In a preferred embodiment, during the distance measurement process based on binocular vision, the visual distance processing module 12 uses stereoscopic vision geometry principles to calculate the spatial distance between the drone and the curtain wall by analyzing the position differences of the corresponding contaminated areas in the binocular images.

[0054] Specifically, the visual distance processing module 12 first extracts the spatial location information of the first and second contaminated areas determined by the contaminated area identification module 11. The pixel coordinates of the first contaminated area within the first image are obtained and recorded as first pixel coordinates. These coordinates are typically based on the geometric center or feature point of the contaminated area. Similarly, the pixel coordinates of the second contaminated area within the second image are obtained and recorded as second pixel coordinates. These two pixel coordinates respectively reflect the image projection position of the same actual contaminated area from the left and right viewing angles.

[0055] Subsequently, the visual distance processing module 12 calculates the positional deviation between the first pixel coordinate and the second pixel coordinate to obtain a pixel offset. This pixel offset is the parallax in binocular vision, reflecting the parallax effect caused by the baseline distance between the left and right cameras of the binocular vision sensor. It is a basic parameter for stereo distance measurement.

[0056] Afterwards, the visual distance processing module 12 calculates the first curtain wall distance using the binocular visual distance measurement formula based on the parallax principle of binocular visual distance measurement. The binocular visual distance measurement formula is:

[0057]

[0058] Where Z is the first curtain wall distance; B is the baseline distance of the binocular vision sensor, determined through factory calibration or on-site calibration measurement; f is the camera focal length, an intrinsic parameter of the binocular vision sensor, obtained through internal calibration; and d is the calculated parallax (pixel offset). This pixel offset is converted to the actual first curtain wall distance Z using the binocular vision ranging formula. This first curtain wall distance represents the straight-line distance between the drone and the target curtain wall surface, derived using visual measurement methods, and provides important data for subsequent ranging fusion processing.

[0059] Furthermore, the ranging fusion processing module 13 is further configured to:

[0060] The distance to the second curtain wall is collected by a distance measuring sensor, wherein the distance measuring sensor includes a laser rangefinder;

[0061] According to the pollution similarity coefficient, a visual weight correction calculation configuration is performed in combination with a preset visual weight to obtain a visual weight;

[0062] According to the visual weight, configuring calculation to obtain ranging weight;

[0063] The first curtain wall distance and the second curtain wall distance are weightedly calculated according to the visual weight and the distance measurement weight to obtain the curtain wall distance.

[0064] In a preferred embodiment, in the specific process of multi-sensor data fusion, the ranging fusion processing module 13 adopts a dynamic weight allocation strategy to comprehensively utilize the respective advantages of visual ranging and laser ranging to improve the overall accuracy and reliability of distance measurement.

[0065] First, the ranging fusion processing module 13 independently collects the distance to the second curtain wall using a ranging sensor, primarily a laser rangefinder. The laser rangefinder emits laser pulses and receives reflected signals from the target surface, calculating an accurate distance measurement based on the speed of light and time of flight, forming the second curtain wall distance. Laser ranging offers advantages such as high measurement accuracy, fast response speed, and unaffected by lighting conditions. Subsequently, the ranging fusion processing module 13 dynamically adjusts the preset visual weight based on the pollution similarity coefficient provided by the polluted area identification module 11. Specifically, the preset visual weight is first obtained, and then a correction calculation and configuration are performed in conjunction with the pollution similarity coefficient. When the pollution similarity coefficient is high, it indicates good binocular vision consistency in identifying polluted areas and high ranging reliability, so a larger visual weight is assigned. Conversely, when the pollution similarity coefficient is low, it indicates uncertainty in visual ranging, so a smaller visual weight is assigned. For example, the preset visual weight is multiplied by the pollution similarity coefficient to adjust the preset visual weight and calculate the corrected visual weight to obtain the visual weight.

[0066] After obtaining the corrected visual weights, the ranging fusion processing module 13 configures and calculates the corresponding ranging weights based on the weight normalization principle. For example, using a complementary weight distribution method, the ranging weights can be obtained by subtracting the visual weights from 1. Next, based on the calculated visual weights and ranging weights, the ranging fusion processing module 13 performs a weighted fusion calculation on the first curtain wall distance provided by the visual distance processing module 12 and the second curtain wall distance measured by the laser rangefinder to obtain the final curtain wall distance. This fused distance combines the spatial perception capabilities of visual ranging with the high precision of laser ranging, effectively improving the accuracy and robustness of distance measurement.

[0067] Furthermore, the pollution level identification module 14 is further configured to:

[0068] Calculating the ratio of the curtain wall distance to the maximum allowable cleaning curtain wall distance to obtain a distance accuracy coefficient;

[0069] Calculating a configuration pollution identification coefficient based on the distance accuracy coefficient and the pollution similarity coefficient;

[0070] According to the pollution identification coefficient, a pollution level identification network with a corresponding proportion is selected from a pre-constructed pollution level identification network sequence to obtain a called pollution level identification network set;

[0071] Cropping the images of the first polluted area and the second polluted area in the first image and the second image to obtain a first polluted image and a second polluted image, respectively inputting the images into the called pollution level recognition network set, and identifying the sources to obtain a first pollution level set and a second pollution level set;

[0072] Calculating a pollution level according to the first pollution level set and the second pollution level set;

[0073] The cleaning plan of the drone is indexed according to the pollution level, and the drone cleaning control is performed.

[0074] In a preferred embodiment, first, the pollution level identification module 14 performs a distance accuracy assessment on the current working environment. The pollution level identification module 14 obtains the curtain wall distance provided by the ranging fusion processing module 13, and the curtain wall distance represents the actual spatial distance between the current drone and the target curtain wall surface. At the same time, the pollution level identification module 14 calls the preset maximum allowable cleaning curtain wall distance parameter. The maximum allowable cleaning curtain wall distance is a parameter determined based on the safety operation requirements and cleaning effect requirements of the drone cleaning operation. It represents the maximum working distance that the drone needs to maintain with the curtain wall surface when performing the curtain wall cleaning task, so as to ensure that the cleaning equipment can effectively act on the curtain wall surface and ensure the safety of the operation. By calculating the ratio of the curtain wall distance to the maximum allowable cleaning curtain wall distance, the pollution level identification module 14 obtains the distance accuracy coefficient. When the distance accuracy coefficient is close to 1, it indicates that the current curtain wall distance is close to the maximum allowable cleaning curtain wall distance, and the drone is in a relatively far working position. At this time, the resolution of the first image and the second image collected by the binocular vision sensor is relatively low, and the reliability of identifying the pollution details of the first and second pollution areas is reduced; when the distance accuracy coefficient is small, it indicates that the curtain wall distance is relatively close, and the drone is in a better working position, and can obtain higher quality first and second images, which is conducive to accurately identifying the first and second pollution areas, and improving the accuracy of pollution level judgment.

[0075] Then, the pollution level identification module 14 comprehensively considers the distance accuracy coefficient and the pollution similarity coefficient provided by the pollution area identification module 11, and calculates the pollution identification coefficient. Specifically, the pollution identification coefficient = (distance accuracy coefficient + (1-pollution similarity coefficient)) / 2. Through the above calculation, the larger the distance accuracy coefficient, the farther the curtain wall distance, the more difficult it is to identify the first pollution area and the second pollution area, and the larger the pollution identification coefficient; the smaller the pollution similarity coefficient, the worse the consistency between the first pollution area and the second pollution area in the first image and the second image, and the lower the reliability of the binocular vision sensor recognition result, the larger the corresponding (1-pollution similarity coefficient) value, and the larger the pollution identification coefficient. Through this comprehensive evaluation, the pollution identification coefficient can accurately reflect the complexity and uncertainty level of the current pollution level identification task.

[0076] Based on the calculated pollution identification coefficient, the pollution level identification module 14 performs intelligent network selection within a pre-constructed pollution level identification network sequence. This pollution level identification network sequence contains multiple pollution level identification networks trained with different training data sets, each with different identification characteristics. Based on the numerical value of the pollution identification coefficient and a preset selection strategy, the pollution level identification module 14 determines the selection ratio of pollution level identification networks, obtaining multiple pollution level identification networks to form a set of pollution level identification networks. For example, when the pollution identification coefficient is small, indicating good identification conditions, the pollution level identification module 14 selects 60% of the pollution level identification networks; when the pollution identification coefficient is large, indicating a complex identification environment, the pollution level identification module 14 increases the proportion of pollution level identification networks to 80% to cope with the more complex identification environments of the first and second pollution areas. Through this dynamic selection mechanism, the pollution level identification module 14 forms a set of pollution level identification networks optimized for the current task.

[0077] Subsequently, the pollution level identification module 14 performs precise regional cropping on the first and second images. Based on the boundary coordinates of the first and second polluted regions determined by the pollution region identification module 11, the pollution level identification module 14 performs precise cropping within the first image to obtain the first polluted image, and within the second image to obtain the second polluted image. These cropped first and second polluted images not only contain key pollution feature information but also effectively remove interference from irrelevant background areas, thereby improving the processing efficiency and accuracy of subsequent calls to the pollution level identification network set. At the same time, the pollution level identification module 14 can also perform preprocessing operations such as brightness equalization, contrast enhancement, and noise filtering on the first and second polluted images to further optimize the quality of the first and second polluted images.

[0078] Next, the pollution level identification module 14 inputs the pre-processed first pollution image and the second pollution image into each pollution level identification network in the pollution level identification network set for parallel processing. Each pollution level identification network in the pollution level identification network set independently analyzes the pollution features in the first pollution image and the second pollution image based on its training characteristics and algorithm structure, including multi-dimensional feature parameters such as the proportion of pollution area, color depth distribution, texture complexity, and edge clarity. By calling the multi-network collaborative analysis of the pollution level identification network set, the pollution level identification module 14 can evaluate the pollution levels of the first pollution area and the second pollution area from different angles and levels, and obtain more comprehensive and reliable recognition results. Afterwards, the pollution level identification module 14 outputs the first pollution level set and the second pollution level set. The first pollution level set contains the recognition results and confidence scores of multiple pollution level identification networks for the first pollution image, and the second pollution level set contains the recognition results and confidence scores of multiple pollution level identification networks for the second pollution image.

[0079] The pollution level identification module 14 then uses various fusion strategies to comprehensively analyze the first and second pollution level sets. For example, a weighted voting mechanism can be used to assign different weights based on the historical accuracy of each pollution level identification network in the pollution level identification network set. Alternatively, statistical analysis methods can be used to calculate the mean, median, and variance of the identification results for the first and second pollution level sets. A confidence screening mechanism can also be used to prioritize high-confidence pollution level identification results. Through the application of these fusion strategies, the pollution level identification module 14 obtains a unified pollution level that accurately reflects the actual pollution levels in the first and second pollution areas.

[0080] Based on the determined pollution level, the pollution level identification module 14 then matches the most suitable drone cleaning solution to the pollution level indexed in the pre-set drone cleaning solution database. Based on the pollution level, the pollution level identification module 14 selects the appropriate cleaning mode, sets appropriate operating parameters, and generates a detailed sequence of drone cleaning control instructions. These drone cleaning control instructions cover multiple aspects, including drone flight path planning, cleaning equipment start / stop control, spray pressure parameter adjustment, and detergent ratio setting. These instructions ensure that drones can perform precise and efficient curtain wall cleaning operations, enabling drone cleaning control and achieving intelligent pollution control.

[0081] Furthermore, the steps of pre-building the pollution level identification network sequence include:

[0082] According to the data records of curtain wall pollution cleaning, a set of sample pollution images is collected, and the pollution level in each sample pollution image is marked to obtain a set of sample pollution levels;

[0083] Randomly dividing the sample pollution image set and the sample pollution level set according to a preset ratio to obtain first pollution level identification training data, and continuing to divide to obtain Kth pollution level identification training data, where K is a positive integer;

[0084] K pollution level recognition networks are constructed, and K groups of pollution level recognition training data are used respectively to perform supervised training on the K pollution level recognition networks until the accuracy converges, thereby obtaining a pollution level recognition network sequence.

[0085] In a preferred embodiment, when constructing a pollution level recognition network sequence, first, based on the data records of curtain wall pollution cleaning, a set of sample pollution images containing diverse scenes such as different pollution types, pollution degrees, lighting conditions, shooting angles and distances is collected. This sample pollution image set covers various pollution conditions that may be encountered in actual curtain wall cleaning operations, including pollution images of different levels such as light pollution, moderate pollution, and heavy pollution. Subsequently, professional and technical personnel are organized to manually annotate each sample pollution image in the sample pollution image set, and determine the corresponding pollution level label for each sample pollution image based on evaluation criteria such as the proportion of pollution area, pollution color depth, and pollution texture complexity, to form a sample pollution level set. This sample pollution level set forms a one-to-one correspondence with the sample pollution image set, providing standard answers for subsequent supervised learning training.

[0086] Next, the sample pollution image set and the sample pollution level set are randomly divided multiple times according to a preset ratio. Specifically, random sampling and division are performed from the sample pollution image set and the sample pollution level set according to a preset ratio to obtain the first pollution level recognition training data, which contains the sample pollution images sampled in proportion and their corresponding sample pollution level labels. The above random division process is repeated, and each time independent random sampling is performed from the original sample pollution image set and the sample pollution level set according to the same preset ratio to obtain the second pollution level recognition training data, the third pollution level recognition training data, and so on until the Kth pollution level recognition training data, where K is a positive integer. Since each division adopts a random sampling method, each pollution level recognition training data has different data combinations and distribution characteristics. Although the data scale is the same, the sample composition is different, which provides a diverse training data foundation for the subsequent construction of a pollution level recognition network with different recognition characteristics.

[0087] Afterwards, K pollution level recognition networks with the same or different structures are constructed. These pollution level recognition networks may adopt different technical parameters such as deep learning architecture, number of network layers, activation functions or optimization algorithms. K groups of pollution level recognition training data are used respectively to perform independent supervised training on the corresponding K pollution level recognition networks. During the training process, each pollution level recognition network continuously adjusts the network parameters through the back propagation algorithm to learn the mapping relationship from sample pollution images to sample pollution levels. The changes in the recognition accuracy of each pollution level recognition network on the verification data set are continuously monitored. When the accuracy reaches a stable state and no longer increases significantly, it is determined that the training of the pollution level recognition network has converged. After the complete training process, K pollution level recognition networks with different recognition characteristics and generalization capabilities are obtained. These K pollution level recognition networks are arranged according to the training order or performance indicators to form a pollution level recognition network sequence, which provides support for the intelligent network selection and collaborative recognition of the pollution level recognition module 14.

[0088] Example 2, as Figure 2 As shown, based on the same inventive concept of a machine vision drone curtain wall cleaning control system provided in Example 1, an embodiment of the present invention also provides a machine vision drone curtain wall cleaning control method, including:

[0089] Using a binocular vision sensor, a first image and a second image on the target curtain wall are collected to identify and obtain the corresponding first contaminated area and second contaminated area, and obtain a contamination similarity coefficient;

[0090] Processing to obtain a first curtain wall distance according to pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image;

[0091] The distance to the second curtain wall is collected by a distance measuring sensor, and a visual weight and a distance measuring weight are configured according to the pollution similarity coefficient, and a weighted calculation is performed on the first curtain wall distance and the second curtain wall distance to obtain the curtain wall distance;

[0092] According to the curtain wall distance and the pollution similarity coefficient, a pollution identification coefficient is configured, and image cropping and pollution level identification are performed on the first and second pollution areas to obtain the pollution level and perform drone cleaning control.

[0093] Furthermore, a binocular vision sensor is used to capture a first image and a second image on the target curtain wall, identify and obtain the corresponding first contaminated area and second contaminated area, and obtain a contamination similarity coefficient, including:

[0094] Capturing a first image and a second image on a target curtain wall through a binocular vision sensor;

[0095] Identifying and acquiring polluted pixels in the first image and the second image through image recognition, and obtaining a first polluted pixel distribution and a second polluted pixel distribution when polluted pixels are identified;

[0096] Randomly select and optimize the polluted area within the first polluted pixel distribution and the second polluted pixel distribution to obtain the first polluted area and the second polluted area, as well as the pollution similarity coefficient.

[0097] Furthermore, within the first contaminated pixel distribution and the second contaminated pixel distribution, random selection and optimization of contaminated areas are performed to obtain the first contaminated area and the second contaminated area, as well as the contamination similarity coefficient, including:

[0098] Using a preset image window, randomly select a first assumed polluted area within the first polluted pixel distribution, and randomly select a second assumed polluted area within the second polluted pixel distribution;

[0099] Calculating pixel similarity between the first assumed contaminated area and the second assumed contaminated area to obtain a first assumed contamination similarity coefficient;

[0100] Continue to randomly select the assumed contaminated area and calculate the pixel similarity until the traversal is completed. Output the first assumed contaminated area and the second assumed contaminated area corresponding to the maximum pixel similarity as the first contaminated area and the second contaminated area, and output the maximum pixel similarity as the contamination similarity coefficient.

[0101] Furthermore, according to the pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image, obtaining a first curtain wall distance includes:

[0102] Obtaining pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image to obtain first pixel coordinates and second pixel coordinates;

[0103] The deviation between the first pixel coordinate and the second pixel coordinate is calculated to obtain a pixel offset, and the first curtain wall distance is calculated based on binocular vision.

[0104] Furthermore, the second curtain wall distance is collected by a distance measuring sensor, and according to the pollution similarity coefficient, a visual weight and a distance measuring weight are configured, and the first curtain wall distance and the second curtain wall distance are weightedly calculated to obtain the curtain wall distance, including:

[0105] The distance to the second curtain wall is collected by a distance measuring sensor, wherein the distance measuring sensor includes a laser rangefinder;

[0106] According to the pollution similarity coefficient, a visual weight correction calculation configuration is performed in combination with a preset visual weight to obtain a visual weight;

[0107] According to the visual weight, configuring calculation to obtain ranging weight;

[0108] The first curtain wall distance and the second curtain wall distance are weightedly calculated according to the visual weight and the distance measurement weight to obtain the curtain wall distance.

[0109] Furthermore, according to the curtain wall distance and the pollution similarity coefficient, a pollution identification coefficient is configured, image cropping and pollution level identification are performed on the first pollution area and the second pollution area, the pollution level is obtained, and drone cleaning control is performed, including:

[0110] Calculating the ratio of the curtain wall distance to the maximum allowable cleaning curtain wall distance to obtain a distance accuracy coefficient;

[0111] Calculating a configuration pollution identification coefficient based on the distance accuracy coefficient and the pollution similarity coefficient;

[0112] According to the pollution identification coefficient, a pollution level identification network with a corresponding proportion is selected from a pre-constructed pollution level identification network sequence to obtain a called pollution level identification network set;

[0113] Cropping the images of the first polluted area and the second polluted area in the first image and the second image to obtain a first polluted image and a second polluted image, respectively inputting the images into the called pollution level recognition network set, and identifying the sources to obtain a first pollution level set and a second pollution level set;

[0114] Calculating a pollution level according to the first pollution level set and the second pollution level set;

[0115] The cleaning plan of the drone is indexed according to the pollution level, and the drone cleaning control is performed.

[0116] Furthermore, the steps of pre-building the pollution level identification network sequence include:

[0117] According to the data records of curtain wall pollution cleaning, a set of sample pollution images is collected, and the pollution level in each sample pollution image is marked to obtain a set of sample pollution levels;

[0118] Randomly dividing the sample pollution image set and the sample pollution level set according to a preset ratio to obtain first pollution level identification training data, and continuing to divide to obtain Kth pollution level identification training data, where K is a positive integer;

[0119] K pollution level recognition networks are constructed, and K groups of pollution level recognition training data are used respectively to perform supervised training on the K pollution level recognition networks until the accuracy converges, thereby obtaining a pollution level recognition network sequence.

[0120] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0121] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0125] Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts become known.

[0126] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A machine vision UAV curtain wall cleaning control system, characterized in that: The system is applied to a drone, and the system includes: The pollution area identification module is used to collect the first image and the second image on the target curtain wall through a binocular vision sensor, identify and obtain the corresponding first pollution area and the second pollution area, and obtain the pollution similarity coefficient; A visual distance processing module, configured to obtain a first curtain wall distance based on the pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image; a distance measurement fusion processing module, configured to collect the second curtain wall distance through a distance measurement sensor, configure a visual weight and a distance measurement weight according to the pollution similarity coefficient, and perform a weighted calculation on the first curtain wall distance and the second curtain wall distance to obtain the curtain wall distance; The pollution level identification module is used to configure the pollution identification coefficient according to the curtain wall distance and the pollution similarity coefficient, perform image cropping and pollution level identification on the first pollution area and the second pollution area, obtain the pollution level, and perform drone cleaning control.

2. The machine vision UAV curtain wall cleaning control system according to claim 1 is characterized in that: The contaminated area identification module is further used for: Using a binocular vision sensor, a first image and a second image are collected on the target curtain wall; Identifying and acquiring polluted pixels in the first image and the second image through image recognition, and obtaining a first polluted pixel distribution and a second polluted pixel distribution when polluted pixels are identified; Randomly select and optimize the polluted area within the first polluted pixel distribution and the second polluted pixel distribution to obtain the first polluted area and the second polluted area, as well as the pollution similarity coefficient.

3. The machine vision-based UAV curtain wall cleaning control system according to claim 1 is characterized in that: The contaminated area identification module is further used for: Using a preset image window, randomly select a first assumed polluted area within the first polluted pixel distribution, and randomly select a second assumed polluted area within the second polluted pixel distribution; Calculating pixel similarity between the first assumed contaminated area and the second assumed contaminated area to obtain a first assumed contamination similarity coefficient; Continue to randomly select the assumed contaminated area and calculate the pixel similarity until the traversal is completed. Output the first assumed contaminated area and the second assumed contaminated area corresponding to the maximum pixel similarity as the first contaminated area and the second contaminated area, and output the maximum pixel similarity as the contamination similarity coefficient.

4. The machine vision-based UAV curtain wall cleaning control system according to claim 1 is characterized in that: The visual distance processing module is further used for: Obtaining pixel coordinates of the first contaminated area and the second contaminated area in the first image and the second image to obtain first pixel coordinates and second pixel coordinates; The deviation between the first pixel coordinate and the second pixel coordinate is calculated to obtain a pixel offset, and the first curtain wall distance is calculated based on binocular vision.

5. The machine vision-based UAV curtain wall cleaning control system according to claim 1 is characterized in that: The ranging fusion processing module is further used for: The distance to the second curtain wall is collected by a distance measuring sensor, wherein the distance measuring sensor includes a laser rangefinder; According to the pollution similarity coefficient, a visual weight correction calculation configuration is performed in combination with a preset visual weight to obtain a visual weight; According to the visual weight, configuring calculation to obtain ranging weight; The first curtain wall distance and the second curtain wall distance are weightedly calculated according to the visual weight and the distance measurement weight to obtain the curtain wall distance.

6. The machine vision-based UAV curtain wall cleaning control system according to claim 1 is characterized in that: The pollution level identification module is also used for: Calculating the ratio of the curtain wall distance to the maximum allowable cleaning curtain wall distance to obtain a distance accuracy coefficient; Calculating a configuration pollution identification coefficient based on the distance accuracy coefficient and the pollution similarity coefficient; According to the pollution identification coefficient, a pollution level identification network with a corresponding proportion is selected from a pre-constructed pollution level identification network sequence to obtain a called pollution level identification network set; Cropping the images of the first polluted area and the second polluted area in the first image and the second image to obtain a first polluted image and a second polluted image, respectively inputting the images into the called pollution level recognition network set, and identifying the sources to obtain a first pollution level set and a second pollution level set; Calculating a pollution level according to the first pollution level set and the second pollution level set; The cleaning plan of the drone is indexed according to the pollution level, and the drone cleaning control is performed.

7. The machine vision-based UAV curtain wall cleaning control system according to claim 6 is characterized in that: The pre-construction steps of the pollution level identification network sequence include: According to the data records of curtain wall pollution cleaning, a set of sample pollution images is collected, and the pollution level in each sample pollution image is marked to obtain a set of sample pollution levels; Randomly dividing the sample pollution image set and the sample pollution level set according to a preset ratio to obtain first pollution level identification training data, and continuing to divide to obtain Kth pollution level identification training data, where K is a positive integer; K pollution level recognition networks are constructed, and K groups of pollution level recognition training data are used respectively to perform supervised training on the K pollution level recognition networks until the accuracy converges, thereby obtaining a pollution level recognition network sequence.

8. A machine vision-based UAV curtain wall cleaning control method, characterized in that: The method comprises: Using a binocular vision sensor, a first image and a second image on the target curtain wall are collected to identify and obtain the corresponding first contaminated area and second contaminated area, and obtain a contamination similarity coefficient; Processing the first contaminated area and the second contaminated area according to the pixel coordinates of the first and second contaminated areas in the first and second images to obtain a first curtain wall distance; The distance to the second curtain wall is collected by a distance measuring sensor, and a visual weight and a distance measuring weight are configured according to the pollution similarity coefficient, and a weighted calculation is performed on the first curtain wall distance and the second curtain wall distance to obtain the curtain wall distance; According to the curtain wall distance and the pollution similarity coefficient, a pollution identification coefficient is configured, image cropping and pollution level identification are performed on the first and second pollution areas, the pollution level is obtained, and drone cleaning control is performed.

Images