An unmanned aerial vehicle-based ore dressing plant inspection method

Abstract

The application discloses a kind of based on unmanned plane's ore-dressing plant inspection method, it is related to ore-dressing plant inspection technical field, specifically includes the following steps: when the image collected by unmanned plane is affected by oblique light irradiation, the region with shadow interference characteristic in image is identified, and it is evenly divided into several sub-regions;Image disturbance feature information of each sub-region is extracted from the image collected by unmanned plane, and it is analyzed, to determine the shadow gray disturbance degree and disturbance type of each sub-region;According to the disturbance type of each sub-region, the trend judgment strategy of corresponding crack identification path is executed respectively, and dynamic control is carried out.The application solves the problem that crack and shadow are confused and path judgment deviates due to oblique light irradiation when unmanned plane is tilted to shoot ore-dressing equipment, realizes classification identification and dynamic path control based on image disturbance intensity, effectively improves the accuracy of crack identification, the stability of path judgment and the intelligent degree of inspection task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ore dressing plant inspection, and particularly relates to an ore dressing plant inspection method based on a UAV. BACKGROUND

[0002] The ore dressing plant is an important link in mine production, which is responsible for separating the required minerals from the raw ore through a series of processing technologies such as crushing, screening, flotation, etc. In the ore dressing plant, the running state of equipment and facilities directly affects the production efficiency and safety. Therefore, regular inspection of the ore dressing plant is necessary to ensure the normal operation of the equipment and to discover potential faults or safety hazards in a timely manner, which is crucial for ensuring the stability and safety of the production process. The inspection task includes comprehensive monitoring and detection of facilities, equipment, pipelines, etc. The UAV combined with fixed cameras, sensors and AI intelligent analysis technology can realize intelligent inspection of the plant in all directions. Through automatic flight and real-time data collection of the UAV, the temperature of the key equipment can be monitored, the state can be analyzed, and the data can be transmitted to the control platform in real time. The intelligent system of the UAV can automatically identify abnormal conditions and judge the running state of the equipment and potential safety hazards through AI analysis, thereby providing decision support for equipment maintenance and production safety. Through the realization of unmanned, intelligent and automatic inspection, the UAV technology significantly improves the efficiency and accuracy of the inspection, and can cover areas that cannot be reached by traditional manual inspection, ensuring the continuous and safe operation of the ore dressing plant.

[0003] The existing unmanned aerial vehicle-based dressing plant inspection technology realizes intelligent inspection of each link of the dressing plant by combining unmanned aerial vehicle automatic flight, sensors, cameras and AI intelligent analysis system. First, the unmanned aerial vehicle automatically flies through the preset route, covering all areas of the dressing plant, ensuring no omissions. The high-precision sensors and infrared temperature detectors carried on the unmanned aerial vehicle monitor the running state of the equipment in the plant in real time, such as the temperature, humidity, pressure and other key parameters of the pipeline, tank body, motor, ball mill and other equipment. At the same time, the fixed camera and unmanned aerial vehicle camera system obtain real-time image data of the equipment and environment through video monitoring, and identify whether there are abnormalities such as electric leakage, fire, equipment wear and tear through AI image recognition technology. All collected data is transmitted to the ground control platform in real time through the wireless communication system for data analysis and processing. The AI system automatically analyzes the temperature change trend, equipment state and potential faults, issues early warning information in time, and issues inspection instructions through the cluster management platform to control the unmanned aerial vehicle to adjust the flight path and focus on checking the areas with abnormal signs. In addition, all data and reports during the inspection are summarized through the platform to generate a complete inspection report, providing a scientific basis for the maintenance and safety management of the plant equipment. Through this fully automated and unmanned inspection system, the coverage, efficiency and accuracy of the inspection can be greatly improved, ensuring the safe production and continuous and stable operation of the dressing plant equipment.

[0004] The existing technology has the following shortcomings:

[0005] In the dressing plant, when the unmanned aerial vehicle takes pictures at an oblique angle of the equipment close to the window or the reflective metal area, the equipment surface will appear a local strong shadow area similar to the crack gray scale feature due to the influence of natural oblique light or reflection of the plant lighting. Under this specific condition of oblique light irradiation, the shadow and the actual crack direction tend to be consistent, and there is a certain degree of blurred diffusion at the edge of the shadow, thereby interfering with the edge structure judgment in the image, causing visual confusion between the shadow and the crack in the image. Since such shadow areas usually have obvious gray scale mutation characteristics in the image, the existing unmanned aerial vehicle-based dressing plant inspection technology cannot dynamically adjust the direction of the crack recognition path according to the degree of shadow gray scale disturbance under oblique light irradiation, and still uses static image contour features for processing, which is easy to misidentify the shadow as the crack extension path, causing the crack recognition position to deviate or the direction to be wrong. Further, it will not only make the crack path in the inspection result deviate from the true position, mislead the equipment risk level assessment, but also affect the accuracy of subsequent crack development trend modeling analysis, inspection path planning and maintenance scheduling, leading to false detection, missed detection or repeated detection, etc., reducing the overall inspection accuracy and reliability.

[0006] The above information disclosed in the Background section is only for enhancing the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art that is already known to those of ordinary skill in the art. SUMMARY

[0007] The purpose of the present application is to provide a UAV-based ore dressing plant inspection method to solve the problems in the background.

[0008] In order to achieve the above-mentioned purpose, the present application provides the following technical scheme: a UAV-based ore dressing plant inspection method, specifically comprising the following steps:

[0009] In the ore dressing plant, when the UAV collects images at an inclined angle, according to the collected images and their corresponding attitude information, it is determined whether the images collected by the UAV will be affected by oblique light illumination;

[0010] When the images collected by the UAV are affected by oblique light illumination, the regions with shadow interference features in the images are identified and evenly divided into several sub-regions;

[0011] The image disturbance feature information of each sub-region is extracted from the images collected by the UAV, and it is analyzed to determine the shadow gray scale disturbance degree and disturbance type of each sub-region;

[0012] According to the disturbance type of each sub-region, the corresponding crack recognition path direction judgment strategy is executed and dynamically controlled;

[0013] After completing the dynamic control of the crack recognition path of each sub-region, the recognition results are fused and processed, and the complete crack path and its corresponding disturbance level label are output, and according to the disturbance level of each sub-region, an inspection task review list is generated and written into a scheduling list for subsequent retakes and path optimization.

[0014] Preferably, when the images collected by the UAV are affected by oblique light illumination, the regions with shadow interference features in the images are identified, specifically: when the images collected by the UAV are affected by oblique light illumination, the gray value of the collected images is analyzed, the brightness change of all regions of the image is detected, and combined with the distribution of the reflected light source in the image, the regions with shadow interference features in the image are identified, the shadow interference feature region includes boundary blur, brightness change and gray scale change region consistent with the direction of the light source;

[0015] and evenly divided into several sub-regions, specifically: according to the spatial position and size of the shadow interference feature region in the image, the shadow interference region is evenly divided into multiple sub-regions using a gridding algorithm, the gridding algorithm evenly distributes grid points in the shadow region, and dynamically adjusts the division of the sub-regions according to the actual area and shape of the shadow region.

[0016] Preferably, extracting image disturbance feature information of each sub-region from the image collected by the drone and analyzing it to determine the shadow grayscale disturbance degree and disturbance type of each sub-region specifically includes the following steps:

[0017] Extract the image disturbance feature information of each sub-region from the image collected by the UAV and perform preprocessing after extraction;

[0018] Extracting grayscale change feature information and edge blur feature information from the preprocessed image disturbance feature information of each sub-region, and analyzing them after extraction to generate shadow disturbance intensity coefficient and edge blur index of each sub-region respectively;

[0019] Based on the generated shadow disturbance intensity coefficient and edge fuzziness index of each sub-region, a shadow grayscale disturbance evaluation model is constructed, and the shadow grayscale disturbance coefficient of each sub-region is generated by weighted summation;

[0020] Determine a preset threshold interval of shadow grayscale disturbance coefficient, and compare it with the generated shadow grayscale disturbance coefficient of each sub-region after determination, evaluate the degree of shadow grayscale disturbance of each sub-region according to the comparison result, and determine the disturbance type of each sub-region according to the evaluation result.

[0021] Preferably, the logic for obtaining the shadow disturbance intensity coefficient of each sub-region is as follows:

[0022] The grayscale change feature information is extracted from the preprocessed image disturbance feature information of each sub-region, including the grayscale value of each pixel in each sub-region in the image collected by the drone and the grayscale mean of all regions adjacent to each sub-region, and calibrated as and GN i , represents the gray value of the mth pixel in the ith sub-region of the image collected by the UAV, GN i represents the grayscale mean of all regions adjacent to the i-th subregion in the image collected by the UAV, i = 1, 2, 3, ..., k, m = 1, 2, 3, ..., j, k and j are both positive integers;

[0023] Calculate the average grayscale value GPM of all pixels in each sub-region of the image collected by the drone i , according to the formula:

[0024] Calculate the difference between the grayscale value of each pixel in each sub-region of the image collected by the drone and the average grayscale value of all pixels in the sub-region According to the formula:

[0025] determining the optimal values of the first adjustment factor γ and the second adjustment factor δ from the actual image data by a data fitting algorithm, the first adjustment factor γ being used to control the response sensitivity of the intra-subregion gray scale fluctuation to the disturbance coefficient, the second adjustment factor δ being used to adjust the response amplitude of the subregion to the neighborhood gray scale difference;

[0026] calculating the shadow disturbance intensity coefficient of each subregion, and the specific calculation formula is as follows:

[0027]

[0028] In the formula, SDIC i is the shadow disturbance intensity coefficient of the i th subregion.

[0029] Preferably, the acquisition logic of the edge blurring degree index of each subregion is as follows:

[0030] extracting the edge blurring feature information from the image disturbance feature information of each preprocessed subregion, specifically including the edge gradient value of each edge pixel point in each subregion in the image collected by the unmanned aerial vehicle and the gray scale value of each edge pixel point, and being respectively marked as and denotes the edge gradient value of the n th edge pixel point in the i th subregion in the image collected by the unmanned aerial vehicle, denotes the gray scale value of the n th edge pixel point in the i th subregion in the image collected by the unmanned aerial vehicle, i = 1, 2, 3, …, k, n = 1, 2, 3, …, h, k and h are positive integers;

[0031] calculating the average value BGP i of the gray scale values of all edge pixel points in each subregion in the image collected by the unmanned aerial vehicle, and according to the formula:

[0032] calculating the difference value between the gray scale value of each edge pixel point in each subregion in the image collected by the unmanned aerial vehicle and the average value of the gray scale values of all edge pixel points in the subregion according to the formula:

[0033] determining the optimal value of the index adjustment factor α from the actual image data by a data fitting algorithm, the adjustment factor α being used to control the response enhancement degree of the edge gradient to the edge blurring degree index;

[0034] calculating the edge blurring degree index of each subregion, and the specific calculation formula is as follows:

[0035]

[0036] wherein, EBI i is the edge blurring index of the ith sub-region.

[0037] Preferably, based on the generated shadow disturbance intensity coefficient SDIC i and the edge blurring index EBI i , a shadow gray disturbance evaluation model is constructed, and the shadow gray disturbance coefficient of each sub-region is generated by weighted summation, and the specific calculation formula is as follows:

[0038] SGDC i = ω1*SDIC i + ω2*EBI i

[0039] wherein, SGDC i is the shadow gray disturbance coefficient of the ith sub-region, ω1 and ω2 are non-zero weight coefficients of the shadow disturbance intensity coefficient SDIC i and the edge blurring index EBI i of each sub-region, and ω1+ω2=1.

[0040] Preferably, a pre-set shadow gray disturbance coefficient threshold interval [SGDC min , SGDC max ] is determined, and after being determined, it is compared with the generated shadow gray disturbance coefficient SGDC i of each sub-region, the shadow gray disturbance degree of each sub-region is evaluated according to the comparison result, and the disturbance type of each sub-region is determined according to the evaluation result, and the specific comparison and analysis are as follows:

[0041] If SGDC i < SGDC min , the shadow gray disturbance degree of the sub-region is low disturbance degree, and the disturbance type of the sub-region is weak disturbance region;

[0042] If SGDC min ≤ SGDC i ≤ SGDC max , the shadow gray disturbance degree of the sub-region is medium disturbance degree, and the disturbance type of the sub-region is medium disturbance region;

[0043] If SGDC i > SGDC max , the shadow gray disturbance degree of the sub-region is high disturbance degree, and the disturbance type of the sub-region is strong disturbance region.

[0044] Preferably, according to the disturbance type of each sub-region, the walking direction judgment strategy of the corresponding crack recognition path is respectively executed, which specifically includes:

[0045] For the sub-region of the weak disturbance type, the crack recognition path direction judgment strategy executed is specifically: adopting the default reference path recognition algorithm, directly performing linear fitting based on the gray mutation edge in the image, extracting the crack main direction, and extending the crack boundary according to the shortest path principle, without performing redundant filtering, angle correction and auxiliary image fusion processing;

[0046] For the sub-region of the medium disturbance type, the crack recognition path direction judgment strategy executed is specifically: on the basis of executing the default path recognition algorithm, superimposing the edge fusion calculation based on multi-angle image acquisition, performing curvature compensation and node smoothing on the discontinuity breakpoints and direction mutations appearing in the path extraction, and adjusting the path through the historical recognition trajectory;

[0047] For the sub-region of the strong disturbance type, the crack recognition path direction judgment strategy executed is specifically: preferentially calling the disturbance avoidance enhanced recognition mode, performing background enhancement, contrast adjustment and artifact suppression processing on the image, and applying a high-order edge reconstruction algorithm based on model learning in the path judgment process, combining the recognition result of the adjacent non-disturbance region, fitting back and excluding the abnormal recognition path;

[0048] The specific mode of dynamic regulation is: continuously monitoring the disturbance type of each sub-region in the recognition path generation process, switching the corresponding path judgment strategy according to the disturbance type, and introducing a disturbance factor to participate in the path weight calculation in the path generation process, and adjusting the path direction priority in real time; if the recognition confidence of a sub-region is lower than a set threshold, the path segment is marked as a “to-be-reviewed path” and is written into a supplementary shooting scheduling list.

[0049] In the above technical solution, the technical effects and advantages provided by the present application are:

[0050] 1、The present application can accurately identify the local strong shadow region caused by inclined shooting and natural or artificial oblique light source by constructing an image oblique illumination influence judgment mechanism and a shadow interference feature region recognition process, significantly enhancing the system's perception ability of image disturbance sources under complex lighting conditions. Further, by grid division of the shadow region, more fine-grained spatial modeling of image disturbance is realized, providing stable and layered disturbance information support for subsequent path recognition strategies, solving the problem of rough disturbance source positioning and single processing mode in traditional technologies, and improving the analysis basis quality of inspection data from the source.

[0051] 2、The application constructs a complete gray disturbance analysis mathematical model by defining two advanced feature parameters based on image data that can be calculated in real time, "shadow disturbance intensity coefficient" and "edge blurring index", and generates a unified "shadow gray disturbance coefficient" through weighted summation, and finally realizes the accurate division of image disturbance level. This scheme not only integrates the two key interference dimensions of local gray fluctuation and edge sharpness, but also has adjustable and optimized model adaptability, supporting flexible adaptation of image characteristics under different equipment, materials and lighting conditions. By introducing the disturbance coefficient threshold interval for disturbance type division, the system can intelligently perceive the risk level of the image recognition area, significantly improving the context adaptability of crack recognition and the accuracy of classification judgment.

[0052] 3、The application introduces three differentiated crack recognition path strategies based on the disturbance recognition result, corresponding to weak, medium and strong disturbance areas respectively, and combines dynamic strategy switching mechanism and confidence monitoring feedback mechanism to adaptively adjust the algorithm process and judgment logic of the crack recognition path. Especially in the strong disturbance scene, by enhancing the recognition mode, edge reconstruction and path fitting completion, etc., the problems of path drift, fracture or deviation caused by shadow misjudgment are effectively overcome, and finally through disturbance level labeling and path fusion, a complete and reliable crack path atlas is output. The system can also generate a review task list and write it into a scheduling list to form a task closed loop of automatic retake and subsequent optimization, thereby comprehensively improving the accuracy, robustness and full-process closed-loop processing capability of the inspection recognition. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only represent some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0054] Figure 1 The flowchart of the mineral processing plant inspection method based on the unmanned aerial vehicle of the present application. DETAILED DESCRIPTION

[0055] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects of the example implementations to those skilled in the art. Like reference numerals may refer to like elements throughout the description of the figures.

[0056] The present application provides a mineral processing plant inspection method based on an unmanned aerial vehicle as shown in Figure 1 The present application provides a mineral processing plant inspection method based on an unmanned aerial vehicle as shown in

[0057] In the ore dressing plant, when the unmanned aerial vehicle collects images of the equipment at an inclined angle, it is determined whether the images collected by the unmanned aerial vehicle will be affected by oblique light illumination according to the collected images and their corresponding attitude information.

[0058] Determining whether the image is affected by oblique light illumination can be achieved by combining image analysis with attitude data. First, the software extracts the lighting information in the image taken by the unmanned aerial vehicle, that is, by analyzing the distribution of light sources in the image, it detects whether there is strong light illumination or reflection phenomenon. This can be achieved by analyzing the brightness and contrast of the image, for example, by calculating the gray value distribution of the bright area in the image, it is determined whether there is a local strong light or a reflection area. Next, the software will further judge according to the flight attitude information of the unmanned aerial vehicle (such as flight angle, shooting angle and body attitude), combined with the light direction. The sensor data of the unmanned aerial vehicle (such as gyroscope, accelerometer) can provide the angle information of the equipment relative to the ground, and then through the mathematical model, these information is combined with the brightness change of the image to speculate the possibility of the image being affected by oblique light illumination. If the brightness mutation area in the image is consistent with the shooting angle of the unmanned aerial vehicle and the direction of the light source, it can be considered that the image is affected by oblique light illumination.

[0059] By combining image data and attitude information of the unmanned aerial vehicle to determine whether the image is affected by oblique light illumination, the identification accuracy of oblique light interference can be effectively improved. The strong light or reflection area in the image is often very similar to the crack feature and is easy to be misidentified as a crack, which may cause deviation in the subsequent identification path. If only rely on static image features or only use flight data to make judgment, it is easy to miss the complex light changes in the image, resulting in misjudgment. By combining the two, it can more accurately determine which image area is affected by oblique light interference, thereby providing a more reliable basis for subsequent image processing, avoiding unnecessary misidentification, and improving the accuracy of crack identification and the reliability of the inspection result. This method can enhance the intelligence and adaptability of the inspection process, and ensure the inspection effect of the unmanned aerial vehicle under complex light conditions.

[0060] When the image collected by the unmanned aerial vehicle will be affected by oblique light illumination, the region with shadow interference features in the image is identified and evenly divided into several sub-regions.

[0061] In this embodiment, when the image collected by the unmanned aerial vehicle will be affected by oblique light illumination, the region with shadow interference features in the image is identified, specifically: when the image collected by the unmanned aerial vehicle will be affected by oblique light illumination, the brightness change of all regions of the image is detected by analyzing the gray value of the collected image, and the region with shadow interference features in the image is identified by combining the distribution of reflected light sources in the image, the shadow interference feature region includes boundary blur, brightness change and gray change region consistent with the direction of the light source.

[0062] To achieve the detection of shadow interference features in the image, image processing algorithms combined with machine learning models can be used to analyze the brightness changes and the distribution of reflected light sources in the image. The specific implementation steps are as follows: First, the software can detect the brightness change area in the image by analyzing the gray value of the image. By calculating the gray value of each pixel point in the image, and using image difference technology (such as Sobel operator or Laplacian operator) to detect the mutation of gray value, the area with obvious brightness change in the image is highlighted. Then, the software analyzes the distribution of light sources in the image, combined with the shooting angle of the unmanned aerial vehicle and the position of the light source, to identify the reflected light source or local strong light area in the image. This process can be achieved by analyzing the correlation between light source direction and image brightness, and the software uses the attitude information provided by the flight control system to infer the light source in the image, and then determines the brightness change area consistent with the light source direction. For these areas, the software will further extract features such as boundary fuzziness, brightness change amplitude, and gray change consistent with the light source direction, which are significant signs of shadow interference. The purpose of this is to accurately identify the shadow interference area by combining gray change, light source distribution and flight angle. Traditional image processing methods often cannot distinguish the gray features of shadows and cracks, which can easily lead to misidentification. By combining flight attitude and light direction, it can accurately determine which areas are disturbed by oblique light, reducing the confusion between shadow areas and crack paths, and improving the accuracy of image analysis. Finally, through the automatic analysis of the software, more reliable data support can be provided for subsequent path recognition and dynamic control, ensuring the efficiency and accuracy of unmanned aerial vehicle inspection.

[0063] and uniformly divide it into several sub-regions, specifically: according to the spatial position and size of the shadow interference feature area in the image, use a grid algorithm to uniformly divide the shadow interference area into multiple sub-regions, the grid algorithm distributes grid points uniformly in the shadow area, dynamically adjusts the division of sub-regions according to the actual area and shape of the shadow area, to ensure that the size of each sub-region is suitable for subsequent analysis and processing.

[0064] To achieve uniform division of the shadow interference area in the image into multiple sub-regions, a grid division algorithm can be used in combination with image analysis and shape adaptation to dynamically adjust the division strategy. First, the software detects the spatial position and size of the shadow interference area to obtain the size and shape characteristics of the area. Next, a grid division algorithm is used to evenly distribute grid points within the shadow area, which can be adjusted through an automatic adaptation based on the area and shape of the region. Specifically, the size and shape of the grid will be adaptively adjusted according to the actual size and shape of the shadow area. For larger shadow areas, the size of the grid will be larger, while for smaller or irregularly shaped areas, the grid will be refined to ensure that the area of each sub-region is suitable for subsequent analysis and processing. To ensure the accuracy of grid division, the software can use an adaptive algorithm based on the boundary of the region to dynamically adjust the shape and distribution density of the grid. For shadow areas with irregular boundaries, the software will automatically generate irregularly shaped sub-region grids to adapt to different shapes of shadow interference areas. The division of each sub-region is based on the actual position and size of the shadow area, and the size of each sub-region is adjusted to ensure that it is suitable for subsequent image processing tasks such as feature extraction and interference evaluation.

[0065] The purpose of this is to dynamically adjust the size and distribution of the grid to accurately adapt to different sizes and shapes of shadow areas, ensuring the accuracy of subsequent analysis such as shadow interference analysis and crack path identification. A unified grid division strategy can ensure that each sub-region has sufficient analysis data, avoiding missing or uneven processing, improving the detail and accuracy of image analysis, and ultimately improving the overall effectiveness and reliability of unmanned aerial vehicle inspection.

[0066] From the images collected by the unmanned aerial vehicle, the image disturbance feature information of each sub-region is extracted and analyzed to determine the shadow gray scale disturbance degree and disturbance type of each sub-region.

[0067] In this embodiment, the image disturbance feature information of each sub-region is extracted from the images collected by the unmanned aerial vehicle, and analyzed to determine the shadow gray scale disturbance degree and disturbance type of each sub-region, which includes the following steps:

[0068] From the images collected by the unmanned aerial vehicle, the image disturbance feature information of each sub-region is extracted and analyzed to determine the shadow gray scale disturbance degree and disturbance type of each sub-region, which includes the following steps:

[0069] The extraction of image disturbance feature information can be achieved through image processing techniques and feature extraction algorithms. First, the software can divide the image into multiple sub-regions by image segmentation techniques, each sub-region representing a possible shadow disturbance area. Then, gray value analysis is used, such as calculating the mean, standard deviation and gray scale variation amplitude of the pixel gray value in each sub-region. These statistics can effectively reflect the illumination difference and gray scale variation of the sub-region, and thus help identify the existence of shadow area. In addition to gray scale analysis, edge detection algorithms (such as Sobel operator, Canny edge detection) can extract the edge information of the sub-region, helping to identify the boundary between the shadow area and the background in the image. Finally, combined with the local brightness variation (i.e. the difference between each pixel and the regional mean), the features of shadows and illumination changes in the image can be further enhanced, providing more accurate disturbance feature information.

[0070] Image preprocessing is to improve the accuracy of subsequent feature extraction and analysis, remove noise and improve image quality. Common preprocessing steps include noise removal and image enhancement. During image acquisition, noise, blur and uneven illumination may affect the extraction of disturbance features, so first use Gaussian filtering or median filtering to remove random noise in the image, which helps to smooth the image and preserve important edge information. Next, through histogram equalization or contrast enhancement techniques, the contrast of the image is improved, making the gray scale variation of the shadow area more obvious and improving the detail performance of the image. For shadow interference in complex backgrounds, image normalization processing can reduce the differences brought by different lighting conditions, making the disturbance features of the shadow area more prominent and facilitating the subsequent calculation of disturbance intensity and blur index values. The preprocessed image will be clearer and more consistent, providing a more reliable data basis for subsequent feature extraction, analysis and classification.

[0071] From the image disturbance feature information of each preprocessed sub-region, extract the gray scale variation feature information and edge blur feature information, and analyze after extraction to generate the shadow disturbance intensity coefficient and edge blur index of each sub-region respectively;

[0072] To extract the gray scale variation feature information and the edge blur feature information from the image disturbance feature information of each pre-processed sub-region, the following methods can be used: First, the gray scale variation feature information can be realized by local gray scale difference analysis. Specifically, the software can calculate the difference between each pixel in each sub-region and the gray scale mean value of the sub-region to obtain the gray scale variation, that is, the gray scale offset degree of each pixel in the sub-region relative to the mean value. By calculating the difference measure (such as standard deviation, mean difference) of each pixel, the local gray scale variation can be quantified, which can effectively capture the gray scale abrupt change area caused by shadows or reflections in the image. In addition, the edge blur feature information can be extracted by edge detection algorithm (such as Sobel operator, Canny operator, etc.). The edge detection algorithm can calculate the gray scale gradient of each sub-region in the image to identify the edge contour of the shadow area. Then, by measuring the blur degree, such as calculating the gray scale variation rate and blur degree of the edge, the software can quantify the blur degree of the shadow area, reflecting the edge definition of the shadow area. In this way, the quantitative feature information of the gray scale variation and edge blur in the image can be extracted at the same time, providing key data support for subsequent disturbance analysis and recognition.

[0073] Based on the generated shadow disturbance intensity coefficient and edge blur index of each sub-region, a shadow gray scale disturbance evaluation model is constructed, and a shadow gray scale disturbance coefficient of each sub-region is generated by weighted summation;

[0074] A pre-set shadow gray scale disturbance coefficient threshold interval is determined, and after being determined, it is compared with the generated shadow gray scale disturbance coefficient of each sub-region. According to the comparison result, the shadow gray scale disturbance degree of each sub-region is evaluated, and according to the evaluation result, the disturbance type of each sub-region is determined.

[0075] Determining the pre-set threshold range for the shadow grayscale disturbance coefficient can be accomplished through data analysis and statistical modeling. First, the software can perform data cluster analysis based on a large amount of actual image data, using, for example, the K-means clustering algorithm or the Gaussian mixture model (GMM) to analyze different types of disturbance coefficient data. These algorithms can classify the data into different categories based on varying degrees of disturbance characteristics within the image (such as shadow intensity and edge blur) and calculate the distribution of disturbance coefficients for each category. In this way, the software can identify typical values ​​for different disturbance types and calculate corresponding threshold ranges based on these values. Furthermore, statistical analysis methods, such as mean and standard deviation analysis, can be used to determine the normal range of disturbance coefficients. For example, the software can calculate the mean and standard deviation of the shadow grayscale disturbance coefficients in a training dataset and then set a threshold range to the mean plus or minus a certain number of standard deviations, which covers most normal disturbance coefficient values. For images with high-intensity or abnormal disturbances, the software can analyze these data separately and set specific threshold ranges to address abnormal conditions. Through this method, the software can dynamically adjust the threshold range to adapt to different lighting conditions and environmental changes, ensuring accurate assessment in complex environments.

[0076] In this embodiment, the logic for obtaining the shadow disturbance intensity coefficient of each sub-region is as follows:

[0077] The grayscale change feature information is extracted from the preprocessed image disturbance feature information of each sub-region, including the grayscale value of each pixel in each sub-region in the image collected by the drone and the grayscale mean of all regions adjacent to each sub-region, and calibrated as and GN i , represents the gray value of the mth pixel in the ith sub-region of the image collected by the UAV, GN i represents the grayscale mean of all regions adjacent to the i-th subregion in the image collected by the UAV, i = 1, 2, 3, ..., k, m = 1, 2, 3, ..., j, k and j are both positive integers;

[0078] To achieve real-time acquisition of two types of data: the grayscale value of each pixel within each subregion of drone-captured images and the grayscale mean of all adjacent regions, software image processing methods such as image segmentation, sliding windows, and matrix operations can be used. First, after receiving the drone-captured image, the software divides the image into several regular or adaptive subregions using a preset gridding strategy or dynamic region segmentation algorithm. Each subregion is mapped into a two-dimensional grayscale matrix. Subsequently, the system uses image grayscale channel extraction techniques to traverse the grayscale matrix of each subregion and extract its grayscale value pixel by pixel, thus forming a complete data set of "grayscale values ​​of each pixel within the subregion." This data can be used to reflect the internal brightness distribution and grayscale fluctuations of the subregion. After extracting the coordinates of each subregion, the software then expands a certain pixel range based on the region's boundary and uses a sliding window algorithm to obtain all adjacent subregions or adjacent pixel sets. The grayscale values ​​are then averaged to obtain the corresponding "grayscale mean of the adjacent region." This mean value reflects the subregion's baseline relative to the image background or surrounding lighting conditions, helping to determine whether the region is affected by shadows due to sudden grayscale changes. This process can be automatically performed by the image processing engine on a frame-by-frame basis after the image is loaded, enabling real-time calculation and dynamic updates during drone flight. This not only provides fast response times but also adapts to inspection needs using continuous image data.

[0079] Calculate the average grayscale value GPM of all pixels in each sub-region of the image collected by the drone i , according to the formula:

[0080] Calculate the difference between the grayscale value of each pixel in each sub-region of the image collected by the drone and the average grayscale value of all pixels in the sub-region According to the formula:

[0081] The optimal values ​​of the first adjustment factor γ and the second adjustment factor δ are determined from the actual image data through a data fitting algorithm. The first adjustment factor γ is used to control the response sensitivity of the grayscale fluctuation within the sub-region to the disturbance coefficient, and the second adjustment factor δ is used to adjust the response amplitude of the grayscale difference between the sub-region and the neighboring region.

[0082] To determine the optimal values of the first adjustment factor and the second adjustment factor, a data fitting algorithm can be performed by software based on a large amount of actual inspection image data for parameter training and optimization. The specific implementation is as follows: first, a training image set containing a plurality of typical shadow disturbance conditions is constructed, and a true disturbance level label (such as weak disturbance, medium disturbance, and strong disturbance) is manually or semi-automatically assigned to each sub-region marked in the image as a fitting “reference standard”. Subsequently, the software calculates the shadow disturbance intensity coefficient (SDIC) of each sub-region according to the current formula, and continuously adjusts the parameter combination by adjusting the first adjustment factor (controlling the response amplification degree of the internal gray scale fluctuation of the pixel) and the second adjustment factor (controlling the influence weight of the overall gray scale difference between the region and the neighborhood in the disturbance evaluation), so that the difference between the predicted disturbance intensity coefficient and the artificially labeled reference disturbance level is minimized. This process can use the least mean square error (MSE) regression or gradient descent optimization algorithm to complete the fitting, so that the finally selected factor combination has the smallest prediction error under the whole sample. The reason for introducing these two adjustable factors is that there is a nonlinear relationship between the gray scale change characteristics of different regions in the image and the true shadow disturbance influence degree. A fixed calculation formula often cannot adapt to complex and diverse lighting conditions and material reflection characteristics, and the personalized factor combination trained by fitting can enhance the adaptability and discrimination ability of the model, making it more accurate and stable in identifying weak disturbances and strong disturbances in actual applications, thereby improving the universality and reliability of the entire inspection system in multiple scenarios. The entire process can be automatically executed in the image analysis platform without human intervention, thereby meeting the needs of real-time processing and batch modeling.

[0083] The shadow disturbance intensity coefficient of each sub-region is calculated, and the specific calculation formula is as follows:

[0084]

[0085] In the formula, SDIC i is the shadow disturbance intensity coefficient of the i-th sub-region.

[0086] The reason for using this formula to calculate the shadow disturbance intensity coefficient of each sub-region is to comprehensively quantify the dual influence of internal gray scale fluctuation and peripheral region illumination contrast on shadow disturbance intensity, and to enhance the discrimination of disturbance characteristics through exponential and logarithmic operations. Specifically, the numerator part of the formula enhances the influence of the internal gray scale fluctuation of each pixel on the disturbance intensity by the difference between the gray scale value of each pixel and the gray scale mean value of the sub-region in which the pixel is located A power operation is performed to control the weight amplification relationship. The first adjustment factor γ controls the sensitivity of grayscale fluctuations within the subregion to the perturbation coefficient, enabling sensitive capture of local subtle shadow perturbations. The denominator extracts the absolute value of the difference between the grayscale mean of the subregion and the grayscale mean of its adjacent regions, raises it to an exponential power, and then compresses it using a logarithmic function. This enhances the ability of the brightness difference between the subregion and surrounding areas to discriminate against perturbation intensity while avoiding numerical instability caused by extreme values. The second adjustment factor δ controls the degree to which this illumination difference nonlinearly enhances the overall perturbation assessment. The entire calculation logic ensures a normalized assessment by averaging the perturbation value of each pixel (dividing by the number of pixels j), making the perturbation coefficients of regions of different sizes comparable. This provides a reliable, stable, and adjustable assessment basis for the subsequent quantitative classification and dynamic control of the degree of shadow perturbation.

[0087] The shadow disturbance intensity coefficient SDIC of the i-th sub-region i The value of directly reflects the degree to which the sub-region is disturbed by the shadow in the image, and is the core quantitative indicator for evaluating the degree of shadow grayscale disturbance. Specifically, if SDIC i If the value of is large, it means that the grayscale fluctuation of the pixels in the sub-region is more intense, and there is a significant brightness difference with the adjacent regions, indicating that the region is significantly disturbed by shadows and belongs to a strong disturbance region. On the contrary, if SDIC i The smaller the value, the more uniform the grayscale distribution is within the sub-region, and the brightness contrast with the neighboring regions is not strong, indicating that the shadow effect is weak and it belongs to the low disturbance region. Therefore, by setting a set of perturbation coefficient threshold intervals with discrimination, the SDIC i The value range of is used to accurately evaluate the degree of shadow grayscale disturbance of the ith sub-region and classify it as a weak disturbance, medium disturbance or strong disturbance region, which serves as the judgment basis for subsequent crack identification path regulation.

[0088] In this embodiment, the logic for obtaining the edge blur index of each sub-region is as follows:

[0089] The edge blur feature information is extracted from the pre-processed image disturbance feature information of each sub-region, including the edge gradient value and grayscale value of each edge pixel point in each sub-region of the image collected by the drone, and calibrated as and Indicates the edge gradient value of the nth edge pixel in the ith sub-region of the image collected by the drone. represents the grayscale value of the nth edge pixel in the i-th subregion of the image collected by the drone, i = 1, 2, 3, ..., k, n = 1, 2, 3, ..., h, k and h are both positive integers;

[0090] To achieve real-time acquisition of the two types of data, "edge gradient values of each edge pixel point in each sub-region" and "gray values of each edge pixel point", automatic extraction and analysis can be performed in the image data stream after image acquisition by the UAV through the image processing algorithm integrated in the software. Specifically, first, the software divides the image collected by the UAV into sub-regions (such as based on a grid, a sliding window, or a dynamic region segmentation algorithm), and uses an edge detection operator (such as Sobel, Prewitt, or Canny) to perform edge extraction processing on the image data in each sub-region. This process calculates the gradient value of each edge pixel point, i.e., the edge gradient value This value reflects the rate of change of the pixel gray value in the horizontal and vertical directions. The greater the gradient, the more likely it is that the point is at a structural edge or a sudden change in brightness, which has a strong guiding effect on the crack recognition path. At the same time, the software can directly read the original pixel value of these edge pixel points in the gray channel, i.e., the gray value which represents the numerical value of the point in the image brightness space and is a basic parameter for determining the brightness contrast and brightness continuity of the edge. To adapt to the changes in the image during continuous flight of the UAV, the system will input the image frames one by one into the processing engine, and extract these pixel-level feature information in real time in each image frame and cache it, so that the edge gradient value and the gray value can dynamically reflect the local lighting and structural changes in the current inspection scene, thereby supporting subsequent edge blur index calculation and shadow disturbance level recognition. The entire process is highly automated at the software level and does not require human intervention, and can run stably during flight to ensure the continuity and real-time nature of information extraction.

[0091] Calculate the average gray value BGP of all edge pixel points in each sub-region of the image collected by the UAV i According to the formula:

[0092] Calculate the difference between the gray value of each edge pixel point in each sub-region of the image collected by the UAV and the average gray value of all edge pixel points in that sub-region According to the formula:

[0093] Determine the optimal value of the index adjustment factor a from the actual image data through a data fitting algorithm, and the adjustment factor a is used to control the degree of response enhancement of the edge gradient to the edge blur index;

[0094] To achieve the optimal value of the exponential adjustment factor, the software can automatically learn and optimize the value of the factor from a large amount of actual image data through a data fitting algorithm. Specifically, the software will first extract the edge gradient values and brightness change amounts of each sub-region from the images collected by the unmanned aerial vehicle, and calculate the preliminary edge blur index. Then, by comparing the actual edge blur of these images with the artificially calibrated "actual disturbance degree", the software can use data fitting algorithms such as least mean square error (MSE) or gradient descent method to gradually adjust the value of the adjustment factor α to minimize the error between the predicted edge blur and the actual calibration value. This process is achieved through continuous iteration, and each time the factor α is adjusted, the software will automatically adjust the parameters according to the current fitting error to find an optimal factor value that can best match the actual image data.

[0095] The reason for determining the optimal value of the exponential adjustment factor in this way is that the edge blur in the image is a highly dynamic and nonlinear feature, which is not only affected by the edge gradient, but also closely related to factors such as lighting conditions, object surface material, and shooting angle. A fixed adjustment factor may not be able to adapt to changes in different image scenes, so by optimizing this factor through data fitting, it has strong adaptability in various scenes and can more accurately control the influence of edge gradient on the edge blur index, thereby improving the accuracy and stability of the edge blur index calculation. The entire process is automatically performed in the image analysis software, which can achieve real-time learning and dynamic optimization, ensuring that the algorithm can always perform tasks with optimal parameters in a changing environment.

[0096] The edge blur index of each sub-region is calculated, and the specific calculation formula is as follows:

[0097]

[0098] In the formula, EBI i is the edge blur index of the i-th sub-region.

[0099] The reason for using this calculation formula to obtain the edge blur index EBI i of each sub-region is to quantitatively evaluate the coupling relationship between the edge clarity and brightness change characteristics in the shadow area of the image, so as to accurately assess the degree of interference on the image crack recognition path. Specifically, the in the formula is used to extract the edge gradient value of each edge pixel, which reflects the change intensity of the edge of that point, and the exponential adjustment factor α further amplifies the sensitivity to high gradient edges to highlight the dominant role of the edge clear area in the calculation; this gradient value is multiplied by ​It represents the difference between the gray value of the pixel and the mean gray value of the sub-region edge. Taking the logarithm can compress the excessive gray value change response range while maintaining the influence of small changes, so that this item comprehensively reflects the fuzziness of the edge point. This constitutes an inversely proportional adjustment factor. When the brightness change is small (i.e., blurry), this value is large, compressing the molecule and enhancing the recognition of blurred areas. When the brightness change is large (i.e., clear edges), this term approaches 1, preserving the full contribution of the molecule. The final average is calculated in units of h to ensure that the calculation results for regions with different numbers of pixels are comparable. Overall, this formula achieves quantitative and accurate recognition of shadowed and blurred areas through a nonlinear combination of grayscale differences and edge strength, which is a key step in achieving stable crack detection under complex lighting interference.

[0100] Edge blur index EBI of the i-th sub-region i The value of EBI is closely related to the evaluation of the degree of shadow grayscale disturbance in the area, which is specifically reflected in the quantitative reflection of the edge fuzziness of the area, thereby assisting in judging the strength of the shadow interference. i The larger the value, the higher the edge gradient (i.e., clear boundary structure) of the edge pixels in the sub-region is, and the brightness fluctuation is large, indicating that the edge features of the region are unstable and easily affected by oblique light and shadow interference, thus being judged as a strongly disturbed region. On the contrary, when EBI is low, the edge gradient of the edge pixels in the sub-region is high (i.e., clear boundary structure) and the brightness fluctuation is large, indicating that the edge features of the region are unstable and easily affected by oblique light and shadow interference, thus being judged as a strongly disturbed region. i Small values ​​indicate smooth, blurred edges with uniform grayscale distribution, indicating areas with weak texture or continuous brightness. This indicates that this subregion is less affected by shadow interference and can be classified as a weakly disturbed area. Therefore, the edge blur index not only reflects the edge appearance of grayscale disturbances but also plays a key role in assisting in identifying abnormal areas in complex shadow environments. It is one of the core criteria for achieving high-reliability inspection image interpretation.

[0101] In this embodiment, based on the generated shadow disturbance intensity coefficient SDIC of each sub-region i and edge blur index EBI i , construct a shadow grayscale disturbance evaluation model, and generate the shadow grayscale disturbance coefficient of each sub-region by weighted summation. The specific calculation formula is as follows:

[0102] SGDC i =ω1*SDIC i +ω2*EBI i

[0103] Where, SGDC i is the shadow grayscale disturbance coefficient of the i-th sub-region, ω1 and ω2 are the shadow disturbance intensity coefficients SDIC of each sub-region respectively i and edge blur index EBIi The non-zero weight coefficient of , and ω1+ω2=1.

[0104] In order to realize the shadow grayscale disturbance coefficient SGDC of each sub-region i The system will calculate the shadow disturbance intensity coefficient SDIC based on the two core parameters obtained. i Edge blur index EBI i By building a weighted evaluation model, these two indicators are integrated into a unified disturbance evaluation value. In the specific implementation process, the software will call the corresponding SDIC for each sub-area at the same time. i and EBI i value, and introduce two non-zero weight coefficients ω1 and ω2, where ω1 represents SDIC i The relative importance in the comprehensive evaluation, ω2 represents EBI i The sum of the two is always 1 to ensure the normalization and comparability of the results. These two weights can be optimized and set through the training set according to the actual application scenario, or they can be preset as empirical values. For example, in scenes with drastic changes in lighting, the proportion of ω1 can be increased to enhance the response to grayscale disturbances, while in complex backgrounds with significant edge interference, the weight of ω2 can be increased to pay more attention to the influence of blurred edges. Finally, the system follows SGDC. i =ω1*SDIC i +ω2*EBI i The formula completes the fusion calculation, making the SGDC of each sub-region i This becomes a key quantitative indicator that comprehensively reflects the degree to which an object is affected by both shadow grayscale and edge disturbances, providing a unified basis for subsequent disturbance classification and path optimization. The entire process is completed in real time using image analysis software, ensuring continuous output of highly reliable image interference assessment results during inspections.

[0105] In this embodiment, the preset shadow grayscale disturbance coefficient threshold interval [SGDC min , SGDC max ], and after determination, the shadow grayscale disturbance coefficient SGDC of each sub-region is generated i Perform a comparison, evaluate the degree of shadow grayscale disturbance of each sub-region based on the comparison results, and determine the disturbance type of each sub-region based on the evaluation results. The specific comparison analysis is as follows:

[0106] If SGDC i <SGDC min , the shadow grayscale disturbance degree of the sub-region is low disturbance degree, and the disturbance type of the sub-region is weak disturbance region;

[0107] This case means that the region exhibits lower gray level fluctuation and edge blur in the image. In terms of image features, the brightness distribution of such regions is relatively stable, the internal gray value difference is small, and the edge structure is clear without being disturbed by obvious oblique light shadows, reflections or high-contrast boundaries. Therefore, in the crack identification task, this region has higher image stability and readability, and is suitable for directly executing standard image recognition algorithms, with higher accuracy of the recognition result, without additional intervention or post-processing, which can significantly reduce the misidentification rate and subsequent processing cost.

[0108] If SGDC min ≤ SGDC i ≤ SGDC max The shadow gray disturbance degree of the sub-region is a medium disturbance degree, and the disturbance type of the sub-region is a medium disturbance region.

[0109] This case indicates that the image features of the sub-region are disturbed by a medium degree. It is usually manifested as that the gray level change amplitude of the region is not smooth enough, the light and dark transition is not natural enough, and it may be accompanied by a certain degree of edge blur or weak oblique light projection, but it has not reached a degree that seriously affects the identification judgment. In actual inspection tasks, enhanced image recognition strategies or moderate fault tolerance processing need to be introduced for such regions, such as fusing multi-angle image results, dynamic threshold adjustment or soft edge compensation, to improve the recognition stability and accuracy and prevent slight disturbances from causing path deviation or crack breakpoint judgment errors.

[0110] If SGDC i > SGDC max The shadow gray disturbance degree of the sub-region is a high disturbance degree, and the disturbance type of the sub-region is a strong disturbance region.

[0111] This case indicates that the region is disturbed by significant image disturbance, which is usually manifested as that the gray level mutation is severe, the edge structure is chaotic, the shadow overlap is serious, or there is a high reflection region, so that the structure texture in the image is difficult to match the actual device surface features. In this case, traditional crack identification methods are prone to misjudgment, breakage, drift, even missed detection, which seriously affects the accuracy of the inspection result. Therefore, such regions should be identified as high-risk image regions and included in the inspection review list, and priority should be given to scheduling for re-shooting, light reconstruction, path avoidance or algorithm upgrade processing in subsequent scheduling to ensure the stability and intelligence of the overall inspection process.

[0112] According to the disturbance type of each sub-region, a corresponding crack identification path direction judgment strategy is executed and dynamically controlled.

[0113] In this embodiment, according to the disturbance type of each sub-region, a corresponding crack identification path direction judgment strategy is executed, specifically including:

[0114] For the sub-region of the weak disturbance region, the crack recognition path direction judgment strategy is: using the default reference path recognition algorithm, directly based on the gray mutation edge in the image for linear fitting, extracting the main direction of the crack, and extending the crack boundary according to the shortest path principle, without performing redundant filtering, angle correction and auxiliary image fusion processing;

[0115] In the sub-region of the weak disturbance region, because the image gray distribution is uniform, the edge feature is clear and less affected by light, the default reference path recognition algorithm can be directly applied by software to efficiently extract the crack direction without image enhancement or multi-source information compensation. The specific implementation is: first, the high gray change boundary in the image is extracted by using edge detection algorithm (such as Canny or Sobel), the system performs connectivity analysis on these edge pixels, and selects the linear crack candidate region with continuity and extendibility. Then, the least square method or Hough transform is used to linearly fit the edge set to extract the dominant direction line segment of the crack. Then, combined with the path extension logic, the path is automatically extended along the main direction according to the directionality change of the gray gradient in the image until the crack edge gray gradient tends to disappear or the structure is broken. In the whole process, without performing additional filtering, angle correction or cross-frame image fusion operation, the calculation efficiency can be greatly improved. The reason for adopting this simplified process is that the weak disturbance region itself has good image quality and clear structure, and the crack information has high recognition degree in the original image. The basic path extraction strategy can ensure the accuracy and stability of the recognition result, save the calculation resources, improve the real-time response ability of the system, and is especially suitable for the rapid judgment and edge flight judgment task scene in the unmanned aerial vehicle inspection process.

[0116] For the sub-region of the weak disturbance region, the crack recognition path direction judgment strategy is: using the default reference path recognition algorithm, directly based on the gray mutation edge in the image for linear fitting, extracting the main direction of the crack, and extending the crack boundary according to the shortest path principle, without performing redundant filtering, angle correction and auxiliary image fusion processing;

[0117] In the sub-region of the medium disturbance area, due to the existence of a certain degree of transition unevenness or edge blur in the image gray scale, the crack path may appear local fracture, deviation or direction jump, so it is necessary to superimpose multi-angle edge fusion and path repair mechanism on the basis of the default path recognition algorithm to improve the continuity and robustness of the path extraction. The specific implementation manner is: first, the system uses the images of the same region taken by the unmanned aerial vehicle at different angles in the image acquisition, and uses the image registration and geometric correction method to align the images under multiple viewing angles; then, the edge fusion algorithm (such as pixel consistency weighting or edge confidence map superposition) is used to extract the stable overlapping edge features under multiple angles, thereby enhancing the continuity of the broken or blurred boundary in the original image. For the path breakpoints still existing, the software further applies the path fitting algorithm based on the principle of curvature minimization to connect and smooth the direction of the broken section; at the same time, the recognized path in the historical inspection image is called, and the current path is corrected and supplemented through path matching and trajectory deviation analysis, to ensure the coherence of the trend judgment. The reason for adopting this strategy is that the image features of the medium disturbance area are between clear and severe interference, the crack can be detected but is unstable, and if the path deviation is not handled, it is easy to cause structure misjudgment or distortion of the extension trend, so through the multi-angle information fusion and path recall mechanism, the adaptability of the recognition process can be enhanced while ensuring the accuracy, so that the system can still maintain reliable path judgment effect under complex interference.

[0118] For the sub-region of the strong disturbance area, the crack recognition path trend judgment strategy is specifically: preferentially calling the disturbance avoidance enhanced recognition mode, performing background enhancement, contrast adjustment and artifact suppression processing on the image, and applying a high-order edge reconstruction algorithm based on model learning in the path judgment process, in combination with the recognition results of adjacent non-disturbance areas, fitting back and excluding abnormalities to ensure the recognition accuracy of the crack path under strong disturbance environment.

[0119] In the sub-region of strong disturbance type, the image is often affected by serious shadow interference, light spot, metal reflection or blur superposition, etc., resulting in serious distortion of crack edge information in the original image or fusion with the background, and it is difficult for traditional edge detection or gray scale analysis method to stably identify the crack direction. Therefore, the software needs to switch to the disturbance avoidance and enhancement identification mode in priority, and realize the reliable identification of crack direction through multi-step image enhancement and intelligent path back-propagation strategy. The specific implementation mode includes: first, the image is subjected to background enhancement and local contrast stretching, so that the original gray scale distribution is more balanced, and the crack region and the background region form a separable gray scale gradient; at the same time, the artifact suppression algorithm (such as morphological processing and high-frequency component suppression) is introduced to reduce the interference of reflection, highlight or texture noise on the edge structure. Subsequently, the system calls the high-order edge reconstruction model trained based on deep learning or graph neural network to predict the pixel-level edge in the distorted region and reconstruct the potential crack edge direction. When generating the identification path, the software also references the identified path results in the adjacent weak disturbance or medium disturbance region, and performs path fitting and back-propagation by direction extrapolation and boundary mapping method, while identifying the abnormal nodes (such as sharp corners and false edge insertion) in the path and performing rejection processing. The reason for adopting this highly intelligent processing strategy is that the strong disturbance region has exceeded the fault tolerance range of traditional image recognition algorithm in essence, and the extraction of crack information needs to rely on context reasoning, global modeling and interference repair means to complete cooperatively. Only by enhancing the image quality, fusing the model prediction and the information of adjacent regions, can we ensure that there is still a usable crack path judgment result under extreme interference conditions, and ensure the practicability and fault tolerance of the whole inspection system in complex industrial scenes.

[0120] The specific way of dynamic regulation is: continuously monitor the disturbance type of each sub-region during the generation of the identification path, switch the corresponding path judgment strategy according to the disturbance type, and introduce the disturbance factor to participate in the path weight calculation during the generation of the path, and adjust the path direction priority in real time; if the identification confidence of a sub-region is lower than the set threshold, the path segment is marked as "to-be-reviewed path" and written into the supplementary shooting scheduling list synchronously, so as to ensure the dynamic optimization and stable output of the whole process of image identification path.

[0121] In order to realize the whole process dynamic regulation of crack identification path, the system can realize real-time monitoring of the disturbance type of the sub-region in each stage of image processing through the construction of the disturbance perception mechanism and the strategy switching engine in the path generation process, and flexibly adjust the path generation mode and priority according to the preset strategy. The specific implementation method is: after the system completes the disturbance type labeling of each sub-region (i.e. weak disturbance, medium disturbance, strong disturbance), it inputs it as a dynamic parameter into the identification process control module. The module judges the state of each processing region in real time during the path identification execution process, and automatically calls the corresponding crack identification path judgment strategy (such as benchmark identification, fusion correction or enhancement reconstruction). At the same time, the system sets a "disturbance factor" weight parameter for each sub-region, which is calculated based on its SGDC value and EBI value, and is used to participate in path priority scheduling and path continuity optimization, such as connecting the path segment with smaller disturbance factor first in global path splicing, or suppressing the path direction of the region with higher disturbance factor in path conflict judgment. After the path is generated, the system will also calculate the recognition confidence value of each path based on the pixel continuity, direction consistency and edge feature consistency of the crack path. If the value is lower than the preset confidence threshold, the corresponding path segment will be automatically marked as "to-be-rechecked path", and the image number, coordinate position, corresponding region disturbance level and other information will be recorded and written into the supplementary shooting scheduling list as a reference for subsequent flight path optimization and secondary acquisition scheduling. The reason for adopting this dynamic regulation mechanism is that the influence degree of different regions in the inspection image is significantly different, and the fixed path identification process cannot be compatible with all scenes. The dynamic strategy switching and path weight real-time adjustment based on disturbance perception can improve the accuracy of the whole identification and the stability of the path output, and realize information closed loop through the "low confidence supplementary shooting" mechanism to ensure the continuous operation reliability and high fault tolerance of the system under long-time inspection and complex working conditions.

[0122] After completing the dynamic regulation of the crack identification path of each sub-region, the identification results are fused, and the complete crack path and its corresponding disturbance level label are output. According to the disturbance level of each sub-region, an inspection task review list is generated and written into the scheduling list for subsequent supplementary shooting and path optimization.

[0123] After the dynamic regulation of the crack identification path of each sub-region is completed, in order to ensure the integrity of the inspection results and the intelligent scheduling of subsequent tasks, the system needs to perform crack path fusion processing and disturbance level marking mechanism through software, and then generate a review list and a scheduling list, realize the traceability of the identification results and the closed-loop management of task execution. The specific implementation is as follows: the system first splices and fuses the sub-region identification paths processed by their respective strategies according to the spatial position order, automatically repairs the slight deviation or splicing breakage between the crack path segments caused by strategy switching or interference boundary through edge connection matching, path direction consistency judgment and node continuity scoring algorithms, and outputs a complete and structurally coherent crack path. Subsequently, the system marks the disturbance level label (such as weak disturbance, medium disturbance, and strong disturbance) of each sub-region on the path, and establishes the mapping relationship between the path segment and the disturbance level for subsequent analysis. Based on the above label data, the software automatically identifies all path segments classified as strong disturbance regions or path segments with recognition confidence lower than the threshold, and generates an inspection review list according to the timestamp, image number, geographical location information and other metadata. The system records all items in the list to the task queue of the scheduling control module, forming a structured "retake task list" and "path optimization suggestion". The reason for such design is that in the actual inspection process of the unmanned aerial vehicle, affected by environmental light, equipment material, flight attitude and other factors, some areas may still have identification uncertainty even after dynamic strategy processing. Through identification result fusion and disturbance level backtracking, not only a complete inspection path map can be formed, but also potential risk areas can be systematically reviewed, ultimately improving the closed-loop efficiency, task accuracy and system stability of the entire inspection task.

[0124] The above formulas are dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula of the nearest real situation. The preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0125] The above-described embodiments can be implemented in part or in whole through software, hardware, firmware or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When loaded and executed by a computer, the computer instructions or computer programs cause the computer to perform all or part of the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0126] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0127] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0128] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described embodiments are only illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0130] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0131] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for inspecting a mineral processing workshop based on a drone, characterized in that: The specific steps include: In a mineral processing workshop, when a drone captures images of equipment at an oblique angle, it is necessary to determine whether the image captured by the drone is affected by oblique light based on the captured image and its corresponding posture information; When the image collected by the UAV is affected by oblique lighting, the area with shadow interference characteristics in the image is identified and evenly divided into several sub-areas; Extract the image disturbance feature information of each sub-region from the image collected by the UAV, analyze it, and determine the degree and type of shadow grayscale disturbance of each sub-region; The specific steps include: Extract the image disturbance feature information of each sub-region from the image collected by the UAV and perform preprocessing after extraction; Extracting grayscale change feature information and edge blur feature information from the preprocessed image disturbance feature information of each sub-region, and analyzing them after extraction to generate shadow disturbance intensity coefficient and edge blur index of each sub-region respectively; The logic for obtaining the shadow disturbance intensity coefficient of each sub-area is as follows: The grayscale change feature information is extracted from the preprocessed image disturbance feature information of each sub-region, including the grayscale value of each pixel in each sub-region in the image collected by the drone and the grayscale mean of all regions adjacent to each sub-region, and calibrated as and , Indicates the first In the sub-area The gray value of a pixel, Indicates the image captured by the drone and the The grayscale mean of all adjacent regions of a sub-region, , , and are all positive integers; Calculate the average grayscale value of all pixels in each sub-region of the image collected by the drone , according to the formula: ; Calculate the difference between the grayscale value of each pixel in each sub-region of the image collected by the drone and the average grayscale value of all pixels in the sub-region , according to the formula: ; Determine the first adjustment factor from the actual image data through a data fitting algorithm and the second regulatory factor The optimal value of the first adjustment factor The second adjustment factor is used to control the response sensitivity of the grayscale fluctuation within the sub-region to the disturbance coefficient. Used to adjust the response amplitude of the grayscale difference between the sub-region and the neighborhood; Calculate the shadow disturbance intensity coefficient of each sub-area. The specific calculation formula is as follows: Where, For the The shadow disturbance intensity coefficient of each sub-region; The logic for obtaining the edge fuzziness index of each sub-region is as follows: The edge blur feature information is extracted from the pre-processed image disturbance feature information of each sub-region, including the edge gradient value and grayscale value of each edge pixel point in each sub-region of the image collected by the drone, and calibrated as and , Indicates the first In the sub-area The edge gradient value of the edge pixel point, Indicates the first In the sub-area The gray value of the edge pixel, , , and are all positive integers; Calculate the mean grayscale value of all edge pixels in each sub-region of the image collected by the drone , according to the formula: ; Calculate the difference between the grayscale value of each edge pixel in each sub-region of the image collected by the drone and the mean grayscale value of all edge pixels in the sub-region , according to the formula: ; Determine the exponential adjustment factor from actual image data through a data fitting algorithm The optimal value of the adjustment factor Used to control the degree of enhancement of the response of edge gradient to edge blur index; Calculate the edge fuzziness index of each sub-region. The specific calculation formula is as follows: Where, For the The edge blur index of each sub-region; Based on the generated shadow disturbance intensity coefficient and edge fuzziness index of each sub-region, a shadow grayscale disturbance evaluation model is constructed, and the shadow grayscale disturbance coefficient of each sub-region is generated by weighted summation; Determine a preset threshold interval of shadow grayscale disturbance coefficient, and compare it with the generated shadow grayscale disturbance coefficient of each sub-region after determination, evaluate the degree of shadow grayscale disturbance of each sub-region according to the comparison result, and determine the disturbance type of each sub-region according to the evaluation result; According to the disturbance type of each sub-region, the corresponding crack identification path direction judgment strategy is executed and dynamically regulated; After completing the dynamic control of the crack identification path of each sub-area, the identification results are fused and processed to output the complete crack path and its corresponding disturbance level label. Based on the disturbance level of each sub-area, an inspection task review list is generated and written into the scheduling list for subsequent reshooting and path optimization.

2. The method for inspecting a mineral processing workshop based on a drone according to claim 1, characterized in that: When the images collected by the drone are affected by oblique lighting, areas with shadow interference characteristics are identified in the image. Specifically, when the images collected by the drone are affected by oblique lighting, the grayscale value of the collected image is analyzed to detect the brightness changes in all areas of the image. Combined with the distribution of reflected light sources in the image, areas with shadow interference characteristics in the image are identified. Shadow interference characteristic areas include blurred boundaries, brightness changes, and grayscale changes consistent with the direction of the light source. And it is evenly divided into several sub-regions, specifically: according to the spatial position and size of the shadow interference feature area in the image, a gridding algorithm is used to evenly divide the shadow interference feature area into multiple sub-regions, and the gridding algorithm evenly distributes grid points in the shadow area and dynamically adjusts the division of the sub-region according to the actual area and shape of the shadow area.

3. The method for inspecting a mineral processing workshop based on a drone according to claim 2, characterized in that: Based on the shadow disturbance intensity coefficient of each sub-area generated and edge blur index , construct a shadow grayscale disturbance evaluation model, and generate the shadow grayscale disturbance coefficient of each sub-region by weighted summation. The specific calculation formula is as follows: Where, For the The shadow grayscale disturbance coefficient of each sub-region, and are the shadow disturbance intensity coefficients of each sub-area and edge blur index The non-zero weight coefficient of .

4. The method for inspecting a mineral processing workshop based on a drone according to claim 3 is characterized in that: Determine the preset shadow grayscale disturbance coefficient threshold interval , and after determination, the shadow grayscale disturbance coefficients of each sub-region generated Perform a comparison, evaluate the degree of shadow grayscale disturbance of each sub-region based on the comparison results, and determine the disturbance type of each sub-region based on the evaluation results. The specific comparison analysis is as follows: like , the shadow grayscale disturbance degree of the sub-region is low disturbance degree, and the disturbance type of the sub-region is weak disturbance region; like , the shadow grayscale disturbance degree of the sub-region is medium disturbance degree, and the disturbance type of the sub-region is medium disturbance region; like , the shadow grayscale disturbance degree of the sub-region is high disturbance degree, and the disturbance type of the sub-region is a strong disturbance region.

5. The method for inspecting a mineral processing workshop based on a drone according to claim 4, characterized in that: According to the disturbance type of each sub-area, the corresponding crack identification path direction judgment strategy is executed respectively, including: For sub-regions with weak disturbance type, the crack identification path direction judgment strategy is as follows: using the default reference path identification algorithm, linear fitting is performed directly based on the grayscale mutation edges in the image to extract the main direction of the crack, and the crack boundary is extended according to the shortest path principle, without the need for redundant filtering, angle correction and auxiliary image fusion processing; For sub-regions with medium disturbance type, the crack identification path direction judgment strategy is as follows: on the basis of executing the default path identification algorithm, edge fusion calculation based on multi-angle image acquisition is superimposed, curvature compensation and node smoothing are performed for discontinuity breakpoints and directional mutations that appear in the path extraction, and path adjustment is performed based on historical identification trajectory. For sub-regions with strong disturbances, the crack identification path direction determination strategy is as follows: the interference avoidance enhancement recognition mode is prioritized to perform background enhancement, contrast adjustment, and artifact suppression on the image. A high-order edge reconstruction algorithm based on model learning is applied during the path determination process. Combined with the recognition results of adjacent non-disturbance areas, the identification path is fitted, back-propagated, and anomalies are eliminated. The specific method of dynamic control is as follows: during the identification path generation process, the disturbance type of each sub-area is continuously monitored, the corresponding path judgment strategy is switched according to the disturbance type, and the disturbance factor is introduced into the path weight calculation during the path generation process to adjust the path direction priority in real time; if the recognition confidence of a sub-area is lower than the set threshold, the path segment is marked as "path to be reviewed" and simultaneously written into the re-shooting scheduling list.

Images