High wall glass falling risk detection method and system based on unmanned aerial vehicle

Abstract

The invention provides a high-wall glass falling risk detection method and system based on an unmanned aerial vehicle, and relates to the technical field of glass falling risk detection, and the method specifically comprises the steps: S1, carrying a digital camera and an acoustic sensor through the unmanned aerial vehicle, and collecting image information and acoustic signals of to-be-detected glass respectively; s2, using a Canny edge detection algorithm to extract crack edges so as to determine coordinates of a crack starting point and an end point, acquiring the length and the width of the crack, and calculating the density and the concentration coefficient of the crack; acquiring a waveform, a peak value, a valley value, a sound wave amplitude and a spectrogram, and extracting a sound wave frequency; s3, performing data processing on the length, width and concentration coefficient of the crack to generate a crack influence coefficient, and performing data processing on the sound wave amplitude and frequency to generate an acoustic characteristic influence coefficient; and S4, generating a falling risk index, comparing the falling risk index with a preset threshold value, and judging a falling risk level. According to the method, the comprehensiveness and the accuracy of high-wall glass falling risk detection are improved by utilizing multi-dimensional data.

Description

Technical Field

[0001] The present invention relates to the technical field of glass falling risk detection, and in particular to a high-wall glass falling risk detection method and system based on an unmanned aerial vehicle. Background Art

[0002] Architectural high-wall glass is widely used in high-rise buildings because of its beautiful appearance, wide field of vision, fast construction, strong plasticity and other advantages. In the architectural glass industry, my country has experienced rapid development from scratch to market dominance, and has now become the world's largest producer and user. However, affected by factors such as the external environment (such as bad weather) and usage time, glass may be damaged during use, which not only affects the beauty and field of vision of high-wall glass, but more seriously, there are huge safety hazards. Therefore, it is necessary to conduct timely and accurate inspections of architectural high-wall glass. At present, the architectural glass inspection industry mainly uses traditional manual visual inspection methods, which are labor-intensive and time-consuming, and are prone to missed inspections or misdetections due to manual fatigue. In addition, the interior of some high-rise glass is blocked, so that the inspection can only be performed outside the building. At this time, manual visual inspection needs to be combined with personnel working at high altitudes to achieve inspection. Therefore, traditional inspection methods also have certain risks.

[0003] With the rapid development of drone technology, in recent years, there has been a method of using drones to assist in the inspection of building glass curtain walls. This method transmits the pictures taken by the drone to the computer of the inspector in real time. The inspector only needs to view the pictures transmitted back by the drone on the computer and evaluate the status of the building glass curtain wall. This method saves time and labor, and effectively avoids the dangers brought by high-altitude operations. A fully automatic detection method for damaged glass curtain walls of high-rise buildings based on a lightweight deep convolutional neural network, with the publication number CN115565068A, includes constructing a YOLO network, replacing the convolutional layer in the YOLO network with sequentially connected extended convolution, channel-by-channel convolution, point-by-point convolution and pruning layers to obtain a new convolutional layer, and finally using the replaced YOLO network as the first detection network to be constructed; then sparsely constraining the parameters to be learned wi in the pruning layer and pruned the new convolutional layer in the first detection network to obtain a second detection network; finally, sparsely constraining and pruning the scaling factor vector of each BN layer in the second detection network to obtain a third detection network. Therefore, this method can further reduce the number of model parameters while ensuring high accuracy, and realize fully automated detection of glass curtain walls of urban high-rise buildings based on drones.

[0004] However, there are still some shortcomings: As can be seen from the above statement, the above method mainly relies on visual inspection, that is, analyzing the images collected by the camera. Although this detection method can identify obvious cracks and damage, it cannot fully identify some hidden small cracks or structural damage inside the glass by relying solely on visual information. In addition, visual inspection is easily affected by factors such as ambient light, camera resolution and angle, resulting in inaccurate detection results.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a high-wall glass falling risk detection method and system based on an unmanned aerial vehicle to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A high wall glass falling risk detection method based on drone, the specific steps include:

[0009] S1. Use a drone to carry a digital camera and an acoustic sensor to collect image information and acoustic signals of the glass to be inspected;

[0010] S2. Use the Canny edge detection algorithm to extract the edge of the crack from the image information of the glass to be detected. On the edge image, determine the starting point and end point coordinates of the crack by contour detection, obtain the length of the crack, measure the distance between the edges on both sides of the crack, obtain the width of the crack, divide the image of the glass to be detected into several areas, obtain the crack density of each area, obtain the average value and standard deviation of the crack density of all areas according to the crack density of each area, and obtain the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all areas, obtain the waveform of the signal of the glass to be detected by processing the acoustic signal of the glass to be detected, obtain the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtain the sound wave amplitude according to the peak value and valley value of the signal of the glass to be detected, convert the time domain signal into the frequency domain signal by fast Fourier transform, obtain the spectrum diagram of the glass to be detected, and obtain the sound wave frequency according to the spectrum diagram of the glass to be detected;

[0011] S3. Process the length, width and concentration coefficient of the crack of the glass to be tested to generate the crack influence coefficient of the glass to be tested, and process the acoustic wave amplitude and acoustic wave frequency of the glass to be tested to generate the acoustic characteristic influence coefficient of the glass to be tested;

[0012] S4. Process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected to generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and determine the drop risk level of the glass to be detected.

[0013] Furthermore, the Canny edge detection algorithm is used to extract the edge of the crack from the image information of the glass to be detected. On the edge image, the coordinates of the starting point and the end point of the crack are determined by contour detection. The process of obtaining the length of the crack is as follows:

[0014] When the image of the glass to be detected is in color, the color image of the glass to be detected is converted into a grayscale image according to the following formula:

[0015] I gray =0.299·I R +0.587·I G +0.114·I B

[0016] Among them, I gray is the gray value of the pixel in the grayscale image, I R ,I G ,I B They are the intensity values ​​of the red, green and blue channels respectively;

[0017] Use the Canny edge detection algorithm to identify the edges in the image, and use the Sobel operator to find the gradient of the image pixels:

[0018]

[0019] Among them, G x is the horizontal gradient value of the image pixel, G y is the vertical gradient value of the image pixel, * represents the convolution operation, I is a two-dimensional matrix, each element I(x,y) represents the grayscale value of the image at the pixel position (x,y);

[0020] The gradient magnitude and direction of the image pixel are calculated based on the following formula:

[0021]

[0022] Among them, G is the gradient amplitude of the pixel point, and θ is the gradient direction of the pixel point;

[0023] Use high and low thresholds to connect and filter edges, set the highest and lowest thresholds T H and T L ;

[0024] When the gradient amplitude is greater than the maximum threshold, G>T H, then keep the pixel as the edge;

[0025] When the gradient amplitude is between the highest threshold and the lowest threshold, T L <G<T H , then the pixel is retained only when it is connected to a strong edge;

[0026] When the gradient amplitude is less than the minimum threshold, G <T L , then ignore the pixel;

[0027] Use the Euclidean distance formula to calculate the straight-line distance between the crack start and end points, i.e. the length of the crack:

[0028]

[0029] Where L is the length of the crack, (x1, y1) and (x2, y2) are the coordinates of the starting point and end point of the crack, respectively.

[0030] Furthermore, the crack width is obtained according to the following formula:

[0031] Along the length of the crack, measure the distance between the edges of the crack multiple times, and then take the average value as the width of the crack. The calculation formula is as follows:

[0032]

[0033] Where W is the width of the crack, d i is the edge distance measured for the i-th time, N is the number of measurements, i is the index of the measurement number, and the value range of i is 1, 2, ..., N.

[0034] Furthermore, the mean value and standard deviation of the crack density in all regions are obtained, and the crack concentration coefficient is obtained according to the mean value and standard deviation of the crack density in all regions, according to the following formula:

[0035]

[0036] Where C is the crack concentration coefficient, LS η is the number of cracks in the nth region, A η is the area of ​​the nth region, μ D is the average value of crack density in all regions, σ D is the standard deviation of crack density in all regions, D η is the crack density of the ηth region, V is the number of regions, η is the index of the number of regions, and the value range of η is 1, 2, ..., V.

[0037] Further, according to the waveform of the signal of the glass to be detected, the peak value and the valley value of the signal of the glass to be detected are obtained, and according to the peak value and the valley value of the signal of the glass to be detected, the amplitude of the sound wave is obtained, and the time domain signal is converted into a frequency domain signal through fast Fourier transform, and the spectrum diagram of the glass to be detected is obtained. According to the spectrum diagram of the glass to be detected, the process of obtaining the frequency of the sound wave is as follows:

[0038] According to the peak and valley values ​​of the signal of the glass to be tested, the amplitude of the sound wave is obtained according to the following formula:

[0039] FD=FD max -FD min

[0040] Among them, FD is the amplitude of the sound wave, FD max is the peak value of the signal, FD min is the valley value of the signal;

[0041] The process of obtaining the sound wave frequency is as follows:

[0042] Perform fast Fourier transform on the time domain signal to obtain the spectrum of the signal according to the following formula:

[0043]

[0044] Where X[k] is the amplitude and phase of the kth frequency component of the signal in the frequency domain, u[n] is the value of the nth sample point in the time domain, P is the total number of sample points in the signal, n is the index of the sample point in the time domain, and the value range is 0, 1, 2, ..., P-1, and k is the index of the frequency component in the frequency domain. s is the imaginary unit;

[0045] Find the frequency component with the largest amplitude in the spectrum, that is, the peak in the spectrum, and identify the frequency corresponding to the peak, which is the sound wave frequency f of the signal.

[0046] Furthermore, the length, width and crack concentration coefficient of the crack of the glass to be detected are processed to generate the crack influence coefficient of the glass to be detected, according to the following formula:

[0047]

[0048] Among them, XS LW is the crack influence coefficient, L η，j is the length of the jth crack in the nth region, W η，j is the width of the jth crack in the nth region, j is the crack index, and the value range of j is 1, 2, ..., LS η, β is the first adjustment coefficient, which is used to adjust the weight of the crack width in the influence coefficient, β∈[0.1, 5], χ is the second adjustment coefficient, which is used to adjust the offset of the concentration coefficient, χ∈[0, 5], γ is the third adjustment coefficient, which is used to adjust the weight of the concentration coefficient in the influence coefficient, γ∈[0.1, 5].

[0049] Furthermore, the acoustic wave amplitude and acoustic wave frequency of the glass to be tested are processed to generate the acoustic characteristic influence coefficient of the glass to be tested, according to the following formula:

[0050]

[0051] Among them, XS ST is the acoustic characteristic influence coefficient, φ is the fourth adjustment coefficient, which is used to adjust the weight of the sound wave frequency in the influence coefficient, φ∈[0.1, 10], is the fifth adjustment coefficient, which is used to adjust the weight of the sound wave amplitude in the influence coefficient. is the sixth adjustment coefficient, which is used to adjust the weight of the normalized value on the influence coefficient. The acoustic characteristics influence coefficient XS in the formula ST The value range of is [0,1].

[0052] Furthermore, the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be tested are processed to generate the drop risk index of the glass to be tested, according to the following formula:

[0053]

[0054] Among them, XS DL is the drop risk index, τ is an index used to adjust the influence of crack influence coefficient and acoustic characteristic influence coefficient on the drop risk index, τ∈[0,2].

[0055] Furthermore, the process of comparing the drop risk index of the glass to be detected with a preset threshold value to determine the drop risk level of the glass to be detected is as follows:

[0056] When 0≤XS DL If ≤0.3, it is judged as low risk, indicating that the risk of glass falling is low and it can continue to be used;

[0057] When 0.3 <XS DL If ≤0.7, it is judged as medium risk, indicating that the glass has a medium risk of falling;

[0058] When 0.7 <XS DL If ≤1, it is judged as high risk, indicating that the glass has a high risk of falling and needs to be replaced immediately.

[0059] A high-wall glass falling risk detection system based on a drone, the system is used to execute any of the above-mentioned high-wall glass falling risk detection methods based on a drone, comprising:

[0060] A data acquisition module is used to collect image information and acoustic signals of the glass to be inspected by carrying a digital camera and an acoustic sensor on a drone;

[0061] A data processing module, for extracting the edge of a crack from the image information of the glass to be detected by using a Canny edge detection algorithm, determining the starting point and end point coordinates of the crack on the edge image by contour detection, obtaining the length of the crack, obtaining the width of the crack by measuring the distance between the edges on both sides of the crack, dividing the image of the glass to be detected into a number of regions, obtaining the crack density of each region, obtaining the average value and standard deviation of the crack density of all regions according to the crack density of each region, and obtaining the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all regions, obtaining the waveform of the signal of the glass to be detected by processing the acoustic signal of the glass to be detected, obtaining the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtaining the amplitude of the sound wave according to the peak value and valley value of the signal of the glass to be detected, converting the time domain signal into the frequency domain signal by fast Fourier transform, obtaining the spectrum diagram of the glass to be detected, and obtaining the frequency of the sound wave according to the spectrum diagram of the glass to be detected;

[0062] A data analysis module, for processing the length, width and concentration coefficient of the crack of the glass to be detected to generate the crack influence coefficient of the glass to be detected, and for processing the sound wave amplitude and sound wave frequency of the glass to be detected to generate the acoustic characteristic influence coefficient of the glass to be detected;

[0063] The judgment module is used to process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected, generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and judge the drop risk level of the glass to be detected.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The present invention combines digital cameras and acoustic sensors to achieve more comprehensive detection. First, the crack edge information of the glass is extracted through the Canny edge detection algorithm, and the length, width and concentration coefficient of the crack are further calculated, which makes the crack detection more accurate and detailed; secondly, the acoustic signal collected by the drone is used to obtain the spectrum through fast Fourier transform, from which the sound wave amplitude and frequency are extracted, which increases the detection dimension of the internal structure of the glass. Compared with the existing technology that only relies on visual information, the combination of the processing method of acoustic signals can reflect the changes in the internal structure of the glass. By judging the changes in the internal and external structures of the glass, the comprehensiveness and accuracy of the glass falling risk detection are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0067] Figure 2 This is a block diagram of the module composition of the present invention. DETAILED DESCRIPTION

[0068] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0069] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0070] Example:

[0071] See also Figure 1 to Figure 2 , the present invention provides a technical solution:

[0072] A high wall glass falling risk detection method based on drone, such as Figure 1 As shown, the specific steps include:

[0073] S1. Use a drone to carry a digital camera and an acoustic sensor to collect image information and acoustic signals of the glass to be inspected;

[0074] S2. Use the Canny edge detection algorithm to extract the edge of the crack from the image information of the glass to be detected. On the edge image, determine the starting point and end point coordinates of the crack by contour detection to obtain the length of the crack. By measuring the distance between the edges on both sides of the crack, obtain the width of the crack. Divide the image of the glass to be detected into several areas, obtain the crack density of each area, obtain the average value and standard deviation of the crack density of all areas according to the crack density of each area, and obtain the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all areas. By processing the acoustic signal of the glass to be detected, obtain the waveform of the signal of the glass to be detected, obtain the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtain the sound wave amplitude according to the peak value and valley value of the signal of the glass to be detected, convert the time domain signal into the frequency domain signal by fast Fourier transform, obtain the spectrum diagram of the glass to be detected, and obtain the sound wave frequency according to the spectrum diagram of the glass to be detected;

[0075] S3. Process the length, width and concentration coefficient of the crack of the glass to be tested to generate the crack influence coefficient of the glass to be tested, and process the acoustic wave amplitude and acoustic wave frequency of the glass to be tested to generate the acoustic characteristic influence coefficient of the glass to be tested;

[0076] S4. Process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected to generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and determine the drop risk level of the glass to be detected.

[0077] On the basis of the above embodiment, the Canny edge detection algorithm is used to extract the edge of the crack from the image information of the glass to be detected. On the edge image, the coordinates of the starting point and the end point of the crack are determined by contour detection. The process of obtaining the length of the crack is as follows:

[0078] When the image of the glass to be detected is in color, the color image of the glass to be detected is converted into a grayscale image according to the following formula:

[0079] I gray =0.299·I R +0.587·I G +0.114·I B

[0080] Among them, I gray is the gray value of the pixel in the grayscale image, I R ,I G ,I B They are the intensity values ​​of the red, green and blue channels respectively;

[0081] Use the Canny edge detection algorithm to identify the edges in the image, and use the Sobel operator to find the gradient of the image pixels:

[0082]

[0083] Among them, G x is the horizontal gradient value of the image pixel, G y is the vertical gradient value of the image pixel, * represents the convolution operation, I is a two-dimensional matrix, each element I(x, y) represents the grayscale value of the image at the pixel position (x, y).

[0084] The gradient magnitude and direction of the image pixel are calculated based on the following formula:

[0085]

[0086] Among them, G is the gradient amplitude of the pixel point, and θ is the gradient direction of the pixel point.

[0087] Use high and low thresholds to connect and filter edges, set the highest and lowest thresholds T H and T L .

[0088] When the gradient amplitude is greater than the maximum threshold, G>T H , then keep the pixel as the edge;

[0089] When the gradient amplitude is between the highest threshold and the lowest threshold, T L <G<T H , then the pixel is retained only when it is connected to a strong edge;

[0090] When the gradient amplitude is less than the minimum threshold, G <T L , then ignore the pixel.

[0091] The Canny edge detection algorithm is used to extract the edge of the crack from the preprocessed image. On the edge image, the coordinates of the starting and ending points of the crack are determined by contour detection. The straight-line distance between the starting and ending points of the crack, that is, the length of the crack, is calculated using the Euclidean distance formula:

[0092]

[0093] Where L is the length of the crack, (x1, y1) and (x2, y2) are the coordinates of the starting point and end point of the crack respectively;

[0094] Along the length of the crack, measure the distance between the edges of the crack multiple times, and then take the average value as the width of the crack. The calculation formula is as follows:

[0095]

[0096] Where W is the width of the crack, d i is the edge distance measured for the i-th time, N is the number of measurements, i is the index of the measurement number, and the value range of i is 1, 2, ..., N.

[0097] On the basis of the above embodiment, the process of obtaining the average value and standard deviation of the crack density of all regions, and obtaining the crack concentration coefficient according to the average value and standard deviation of the crack density of all regions is as follows:

[0098]

[0099] Where C is the crack concentration coefficient, LS η is the number of cracks in the nth region, A η is the area of ​​the nth region, μ D is the average value of crack density in all regions, σ D is the standard deviation of crack density in all regions, D η is the crack density of the ηth region, V is the number of regions, η is the index of the region, and the value range of η is 1, 2, ..., V.

[0100] On the basis of the above embodiment, the amplitude of the sound wave is obtained according to the peak value and valley value of the signal of the glass to be detected, and the formula is as follows:

[0101] FD=FD max -FD min

[0102] Among them, FD is the amplitude of the sound wave, FD max is the peak value of the signal, FD min is the valley value of the signal;

[0103] Through fast Fourier transform, the time domain signal is converted into a frequency domain signal, and the spectrum of the glass to be tested is obtained. According to the spectrum of the glass to be tested, the process of obtaining the sound wave frequency is as follows:

[0104] Perform fast Fourier transform on the time domain signal to obtain the spectrum of the signal according to the following formula:

[0105]

[0106] Where X[k] is the amplitude and phase of the kth frequency component of the signal in the frequency domain, u[n] is the value of the nth sample point in the time domain, P is the total number of sample points in the signal, n is the index of the sample point in the time domain, and the value range is 0, 1, 2, ..., P-1, and k is the index of the frequency component in the frequency domain. s is the imaginary unit;

[0107] Find the frequency component with the largest amplitude in the spectrum, that is, the peak in the spectrum, and identify the frequency corresponding to the peak, which is the sound wave frequency f of the signal.

[0108] On the basis of the above embodiment, the length, width and crack concentration coefficient of the crack of the glass to be detected are processed to generate the crack influence coefficient of the glass to be detected, according to the following formula:

[0109]

[0110] Among them, XS LW is the crack influence coefficient, L η，j is the length of the jth crack in the nth region, W η，j is the width of the jth crack in the nth region, j is the crack index, and the value range of j is 1, 2, ..., LS η , β is the first adjustment coefficient, which is used to adjust the weight of the crack width in the influence coefficient, β∈[0.1, 5], χ is the second adjustment coefficient, which is used to adjust the offset of the concentration coefficient, χ∈[0, 5], γ is the third adjustment coefficient, which is used to adjust the weight of the concentration coefficient in the influence coefficient, γ∈[0.1, 5].

[0111] The crack influence coefficient XS in the formula LW The value range of is [0,1].

[0112] When XS LW ≈0, indicating that the effect of glass cracks is very small;

[0113] When 0 <XS LW ≤0.5, indicating that the glass has some crack influence, but it is within the acceptable range;

[0114] When 0.5 <XS LW ≤1, indicating that the glass crack has a greater impact.

[0115] When the crack length L η，j When increasing, the normalized value Increase, resulting in crack influence factor XS LW Increase, that is, the length of the crack L η，j The contribution to the impact coefficient increases;

[0116] When the crack width W η，j As the exponential function increases, the value of e βW Rapidly increases, resulting in crack influence factor XS LW Significantly increased, that is, the width of the crack W η，j It has a significant impact on the impact coefficient;

[0117] When the crack concentration coefficient C increases, the logarithmic function log(χ+C) gradually increases. The value of also increases, and the combined effect makes the influence coefficient XS LW Increase.

[0118] Therefore, the length of the crack L η，j , crack width W η，j The crack concentration coefficient C is related to the crack influence coefficient XS LW There is a positive proportional relationship.

[0119] On the basis of the above embodiment, the acoustic wave amplitude and acoustic wave frequency of the glass to be tested are processed to generate the acoustic characteristic influence coefficient of the glass to be tested, according to the following formula:

[0120]

[0121] Among them, XS ST is the acoustic characteristic influence coefficient, φ is the fourth adjustment coefficient, which is used to adjust the weight of the sound wave frequency in the influence coefficient, φ∈[0.1, 10], is the fifth adjustment coefficient, which is used to adjust the weight of the sound wave amplitude in the influence coefficient. is the sixth adjustment coefficient, which is used to adjust the weight of the normalized value on the influence coefficient.

[0122] The acoustic characteristics influence coefficient XS in the formula ST The value range of is [0,1].

[0123] When XS ST ≈0, indicating that the acoustic properties of glass have little effect;

[0124] When 0 <XS ST ≤0.5, indicating that the acoustic properties of the glass have some influence, but are within an acceptable range;

[0125] When 0.5 <XS ST ≤1, indicating that the acoustic properties of glass have a greater impact.

[0126] When the acoustic wave amplitude FD increases, the normalized value Increase, while The value of is also affected by it, which will lead to the influence coefficient XS ST Increase;

[0127] When the sound wave frequency f increases, the normalized value Increase, and the value of sin(φf) is also affected, which will lead to the influence coefficient XS ST Increase;

[0128] When the sound wave amplitude FD or the sound wave frequency f increases, and The value of will also increase, thus affecting the denominator of the formula, making the overall impact coefficient XS ST changes have occurred.

[0129] On the basis of the above embodiment, the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected are processed to generate the drop risk index of the glass to be detected, and the formula is as follows:

[0130]

[0131] Among them, XS DL is the drop risk index, τ is an index used to adjust the influence of crack influence coefficient and acoustic characteristic influence coefficient on the drop risk index, τ∈[0,2].

[0132] The process of comparing the drop risk index of the glass to be detected with the preset threshold value to determine the drop risk level of the glass to be detected is as follows:

[0133] When 0≤XS DL If ≤0.3, it is judged as low risk, indicating that the risk of glass falling is low and it can continue to be used;

[0134] When 0.3 <XS DL If ≤0.7, it is judged as medium risk, indicating that the glass has a medium risk of falling;

[0135] When 0.7 <XS DL If ≤1, it is judged as high risk, indicating that the glass has a high risk of falling and needs to be replaced immediately.

[0136] The β, χ, γ, φ, The specific values ​​of and τ are generally determined by those skilled in the art according to actual conditions. The essence of this formula is a comprehensive analysis by weighted summation. Those skilled in the art collect multiple groups of sample data, and set corresponding adjustment coefficients or indexes for each group of sample data. The set adjustment coefficients or indexes and the collected sample data are substituted into the formula. Through repeated experiments and parameter adjustments, the accuracy of the model output and the rationality of the results are observed, and these adjustment coefficients or indexes are gradually adjusted. The performance and effect of the model under different parameter settings are compared to find the optimal coefficient combination. The calculated adjustment coefficients or indexes are screened and averaged to obtain β, χ, γ, φ, and the value of τ.

[0137] In addition, the size of the adjustment coefficient or index is a specific value obtained by quantifying each parameter. In order to facilitate subsequent comparison, the size of the coefficient depends on the amount of sample data and the technical personnel in this field preliminarily set the corresponding adjustment coefficient or index for each set of sample data. It is not unique as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0138] like Figure 2 To achieve the above purpose, the present invention also provides the following technical solutions:

[0139] A high-wall glass falling risk detection system based on a drone, the system is used to execute any of the above-mentioned high-wall glass falling risk detection methods based on a drone, comprising:

[0140] A data acquisition module is used to collect image information and acoustic signals of the glass to be inspected by carrying a digital camera and an acoustic sensor on a drone;

[0141] A data processing module, for extracting the edge of a crack from the image information of the glass to be detected by using a Canny edge detection algorithm, determining the starting point and end point coordinates of the crack on the edge image by contour detection, obtaining the length of the crack, obtaining the width of the crack by measuring the distance between the edges on both sides of the crack, dividing the image of the glass to be detected into a number of regions, obtaining the crack density of each region, obtaining the average value and standard deviation of the crack density of all regions according to the crack density of each region, and obtaining the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all regions, obtaining the waveform of the signal of the glass to be detected by processing the acoustic signal of the glass to be detected, obtaining the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtaining the amplitude of the sound wave according to the peak value and valley value of the signal of the glass to be detected, converting the time domain signal into the frequency domain signal by fast Fourier transform, obtaining the spectrum diagram of the glass to be detected, and obtaining the frequency of the sound wave according to the spectrum diagram of the glass to be detected;

[0142] A data analysis module, for processing the length, width and concentration coefficient of the crack of the glass to be detected to generate the crack influence coefficient of the glass to be detected, and for processing the sound wave amplitude and sound wave frequency of the glass to be detected to generate the acoustic characteristic influence coefficient of the glass to be detected;

[0143] The judgment module is used to process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected, generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and judge the drop risk level of the glass to be detected.

[0144] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0145] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A high wall glass falling risk detection method based on drone, characterized in that: The specific steps include: S1. Use a drone to carry a digital camera and an acoustic sensor to collect image information and acoustic signals of the glass to be inspected; S2. Use the Canny edge detection algorithm to extract the edge of the crack from the image information of the glass to be detected. On the edge image, determine the starting point and end point coordinates of the crack by contour detection, obtain the length of the crack, measure the distance between the edges on both sides of the crack, obtain the width of the crack, divide the image of the glass to be detected into several areas, obtain the crack density of each area, obtain the average value and standard deviation of the crack density of all areas according to the crack density of each area, and obtain the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all areas, obtain the waveform of the signal of the glass to be detected by processing the acoustic signal of the glass to be detected, obtain the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtain the sound wave amplitude according to the peak value and valley value of the signal of the glass to be detected, convert the time domain signal into the frequency domain signal by fast Fourier transform, obtain the spectrum diagram of the glass to be detected, and obtain the sound wave frequency according to the spectrum diagram of the glass to be detected; S3. Process the length, width and concentration coefficient of the crack of the glass to be tested to generate the crack influence coefficient of the glass to be tested, and process the acoustic wave amplitude and acoustic wave frequency of the glass to be tested to generate the acoustic characteristic influence coefficient of the glass to be tested; S4. Process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected to generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and determine the drop risk level of the glass to be detected.

2. The high wall glass falling risk detection method based on drone according to claim 1 is characterized by: The Canny edge detection algorithm is used to extract the edge of the crack from the image information of the glass to be detected. On the edge image, the coordinates of the starting point and end point of the crack are determined by contour detection. The process of obtaining the length of the crack is as follows: When the image of the glass to be detected is in color, the color image of the glass to be detected is converted into a grayscale image according to the following formula: I gray =0.299·I R +0.587·I G +0.114·I B Among them, I gray is the gray value of the pixel in the grayscale image, I R ,I G ,I B They are the intensity values ​​of the red, green and blue channels respectively; Use the Canny edge detection algorithm to identify the edges in the image, and use the Sobel operator to find the gradient of the image pixels: Among them, G x is the horizontal gradient value of the image pixel, G y is the vertical gradient value of the image pixel, * represents the convolution operation, I is a two-dimensional matrix, each element I(x,y) represents the grayscale value of the image at the pixel position (x,y); The gradient magnitude and direction of the image pixel are calculated based on the following formula: Among them, G is the gradient amplitude of the pixel point, and θ is the gradient direction of the pixel point; Use high and low thresholds to connect and filter edges, set the highest and lowest thresholds T H and T L ; When the gradient amplitude is greater than the maximum threshold, G>T H , then keep the pixel as the edge; When the gradient amplitude is between the highest threshold and the lowest threshold, T L <G<T H , then the pixel is retained only when it is connected to a strong edge; When the gradient amplitude is less than the minimum threshold, G <T L , then ignore the pixel; Use the Euclidean distance formula to calculate the straight-line distance between the crack start and end points, i.e. the length of the crack: Where L is the length of the crack, (x1, y1) and (x2, y2) are the coordinates of the starting point and end point of the crack, respectively.

3. The high wall glass falling risk detection method based on drone according to claim 2 is characterized by: The crack width is obtained according to the following formula: Along the length of the crack, measure the distance between the edges of the crack multiple times, and then take the average value as the width of the crack. The calculation formula is as follows: Where W is the width of the crack, d i is the edge distance measured for the i-th time, N is the number of measurements, i is the index of the measurement number, and the value range of i is 1, 2, ···, N.

4. The high wall glass falling risk detection method based on drone according to claim 3 is characterized by: The mean and standard deviation of the crack density in all regions are obtained, and the crack concentration coefficient is obtained based on the mean and standard deviation of the crack density in all regions, according to the following formula: Where C is the crack concentration coefficient, LS η is the number of cracks in the nth region, A η is the area of ​​the nth region, μ D is the average value of crack density in all regions, σ D is the standard deviation of crack density in all regions, D η is the crack density of the ηth region, V is the number of regions, η is the index of the region, and the value range of η is 1, 2, ..., V.

5. The high wall glass falling risk detection method based on drone according to claim 1 is characterized in that: According to the waveform of the signal of the glass to be detected, the peak value and valley value of the signal of the glass to be detected are obtained, and according to the peak value and valley value of the signal of the glass to be detected, the amplitude of the sound wave is obtained. Through fast Fourier transform, the time domain signal is converted into a frequency domain signal, and the spectrum diagram of the glass to be detected is obtained. According to the spectrum diagram of the glass to be detected, the process of obtaining the sound wave frequency is as follows: According to the peak and valley values ​​of the signal of the glass to be tested, the amplitude of the sound wave is obtained according to the following formula: FD=FD max -FD min Among them, FD is the amplitude of the sound wave, FD max is the peak value of the signal, FD min is the valley value of the signal; The process of obtaining the sound wave frequency is as follows: Perform fast Fourier transform on the time domain signal to obtain the spectrum of the signal according to the following formula: Where X[k] is the amplitude and phase of the kth frequency component of the signal in the frequency domain, u[n] is the value of the nth sample point in the time domain, P is the total number of sample points in the signal, n is the index of the sample point in the time domain, and the value range is 0, 1, 2, ..., P-1, and k is the index of the frequency component in the frequency domain. s is the imaginary unit; Find the frequency component with the largest amplitude in the spectrum, that is, the peak in the spectrum, and identify the frequency corresponding to the peak, which is the sound wave frequency f of the signal.

6. The high wall glass falling risk detection method based on drone according to claim 4 is characterized by: The length, width and crack concentration coefficient of the crack of the glass to be tested are processed to generate the crack influence coefficient of the glass to be tested, based on the following formula: Among them, XS LW is the crack influence coefficient, L η，j is the length of the jth crack in the nth region, W η，j is the width of the jth crack in the nth region, j is the crack index, and the value range of j is 1, 2, ..., LS η , β is the first adjustment coefficient, which is used to adjust the weight of the crack width in the influence coefficient, β∈[0.1, 5], χ is the second adjustment coefficient, which is used to adjust the offset of the concentration coefficient, χ∈[0, 5], γ is the third adjustment coefficient, which is used to adjust the weight of the concentration coefficient in the influence coefficient, γ∈[0.1, 5].

7. The high wall glass falling risk detection method based on drone according to claim 5 is characterized by: The acoustic wave amplitude and acoustic wave frequency of the glass to be tested are processed to generate the acoustic characteristic influence coefficient of the glass to be tested, based on the following formula: Among them, XS ST is the acoustic characteristic influence coefficient, φ is the fourth adjustment coefficient, which is used to adjust the weight of the sound wave frequency in the influence coefficient, φ∈[0.1, 10], is the fifth adjustment coefficient, which is used to adjust the weight of the sound wave amplitude in the influence coefficient. is the sixth adjustment coefficient, which is used to adjust the weight of the normalized value on the influence coefficient. The acoustic characteristics influence coefficient XS in the formula ST The value range of is [0,1].

8. The high wall glass falling risk detection method based on drone according to claim 1 is characterized by: The crack influence coefficient and acoustic characteristic influence coefficient of the glass to be tested are processed to generate the drop risk index of the glass to be tested, based on the following formula: Among them, XS DL is the drop risk index, τ is an index used to adjust the influence of crack influence coefficient and acoustic characteristic influence coefficient on the drop risk index, τ∈[0,2].

9. The high wall glass falling risk detection method based on drone according to claim 8 is characterized by: The process of comparing the drop risk index of the glass to be detected with the preset threshold value to determine the drop risk level of the glass to be detected is as follows: When 0≤XS DL If ≤0.3, it is judged as low risk, indicating that the risk of glass falling is low and it can continue to be used; When 0.3 <XS DL If ≤0.7, it is judged as medium risk, indicating that the glass has a medium risk of falling; When 0.7 <XS DL If ≤1, it is judged as high risk, indicating that the glass has a high risk of falling and needs to be replaced immediately.

10. A high-wall glass falling risk detection system based on a drone, the system being used to execute the high-wall glass falling risk detection method based on a drone as claimed in any one of claims 1 to 9, characterized in that: include: A data acquisition module is used to collect image information and acoustic signals of the glass to be inspected by carrying a digital camera and an acoustic sensor on a drone; A data processing module, for extracting the edge of a crack from the image information of the glass to be detected by using a Canny edge detection algorithm, determining the starting point and end point coordinates of the crack on the edge image by contour detection, obtaining the length of the crack, obtaining the width of the crack by measuring the distance between the edges on both sides of the crack, dividing the image of the glass to be detected into a number of regions, obtaining the crack density of each region, obtaining the average value and standard deviation of the crack density of all regions according to the crack density of each region, and obtaining the concentration coefficient of the crack according to the average value and standard deviation of the crack density of all regions, obtaining the waveform of the signal of the glass to be detected by processing the acoustic signal of the glass to be detected, obtaining the peak value and valley value of the signal of the glass to be detected according to the waveform of the signal of the glass to be detected, obtaining the amplitude of the sound wave according to the peak value and valley value of the signal of the glass to be detected, converting the time domain signal into the frequency domain signal by fast Fourier transform, obtaining the spectrum diagram of the glass to be detected, and obtaining the frequency of the sound wave according to the spectrum diagram of the glass to be detected; A data analysis module, for processing the length, width and concentration coefficient of the crack of the glass to be detected to generate the crack influence coefficient of the glass to be detected, and for processing the sound wave amplitude and sound wave frequency of the glass to be detected to generate the acoustic characteristic influence coefficient of the glass to be detected; The judgment module is used to process the crack influence coefficient and acoustic characteristic influence coefficient of the glass to be detected, generate a drop risk index of the glass to be detected, compare the drop risk index of the glass to be detected with a preset threshold, and judge the drop risk level of the glass to be detected.

Images