A bridge monitoring system based on artificial intelligence

Abstract

The application belongs to the field of bridge monitoring, and discloses a bridge monitoring system based on artificial intelligence, which comprises a UAV and an image recognition device; the UAV is used for photographing the monitoring area on the surface of the bridge to obtain the surface image of the monitoring area on the surface of the bridge; the image recognition device comprises an image screening module and an artificial intelligence recognition module; the image screening module is used for screening the surface image of the monitoring area based on the set rules to obtain a target image; and the artificial intelligence recognition module is used for identifying the target image by using a pre-trained deep learning model and outputting an identification result; the application improves the probability that part of the pixel points in the surface image can correctly screen the surface image with the optimal quality, thereby avoiding screening the surface image based on all the pixel points, effectively improving the screening efficiency of the surface image, and improving the monitoring efficiency of the process of monitoring the bridge.

Description

Technical Field

[0001] This invention relates to the field of bridge monitoring, and more particularly to a bridge monitoring system based on artificial intelligence. Background Technology

[0002] When monitoring visible defects on the surface of bridges, such as cracks, image recognition is typically used. The monitoring process involves first acquiring surface images of the bridge, then filtering these images to obtain those that meet the recognition requirements. These images are then input into a pre-trained deep learning model for identification, and the model outputs the defects present in these images, such as cracks and holes.

[0003] In order to avoid the inability to effectively monitor a certain area of ​​a bridge due to quality issues when acquiring surface images, existing technologies typically take multiple surface images of the same area.

[0004] During the screening process, surface images with better quality are usually retained. Existing technologies typically calculate the quality index of the surface image based on all the pixels in the surface image, and then screen based on the quality index. This results in low screening efficiency and affects the monitoring efficiency of the entire monitoring process. Summary of the Invention

[0005] The purpose of this invention is to disclose an artificial intelligence-based bridge monitoring system that solves the problem of how to improve the efficiency of surface image screening when using image recognition to monitor bridges.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention provides an artificial intelligence-based bridge monitoring system, including a drone and an image recognition device;

[0008] The drone is used to photograph the monitoring area of ​​the bridge surface to obtain surface images of the monitoring area of ​​the bridge.

[0009] The image recognition device includes an image screening module and an artificial intelligence recognition module;

[0010] The image filtering module is used to filter surface images of the monitoring area based on set rules to obtain target images;

[0011] The artificial intelligence recognition module is used to recognize target images using a pre-trained deep learning model and output the recognition results;

[0012] The surface images of the monitoring area are filtered based on the set rules to obtain the target images, including:

[0013] Calculate the screening value for each surface image corresponding to the same monitoring area, including:

[0014] For monitoring area A, let UA represent the set of surface images of monitoring area A;

[0015] The first step is to use IMGA to represent the surface image in monitoring area A, and IMGAU to represent the set of pixels in IMGA;

[0016] The second step is to randomly select a pixel from IMGA as the judgment pixel.

[0017] The third step is to calculate the edge probability coefficients of the pixels.

[0018] Fourth step: If the edge probability coefficient is less than or equal to the set edge probability coefficient threshold, the pixel to be judged will be deleted from IMGAU and proceed to the fifth step; if the edge probability coefficient is greater than the set edge probability coefficient threshold, proceed to the sixth step.

[0019] Fifth step: Determine if the number of pixels in the IMGA is less than [a certain value]. NIMGA is the total number of pixels in IMGA. If it is, proceed to step eight; otherwise, proceed to step two.

[0020] Step 6: Perform boundary recognition based on the judged pixels to obtain the boundary set corresponding to the judged pixels;

[0021] Step 7: Determine if the number of pixels in the IMGA is less than [a certain value]. If yes, proceed to step eight; if no, remove the pixels in the boundary set from IMGA and proceed to step two.

[0022] Step 8: Use UNPIX to represent the union of all boundary sets, and calculate the screening value of IMGA based on UNPIX;

[0023] The surface image with the highest filter value is selected as the target image.

[0024] Preferably, the formula for calculating the marginal probability coefficient is:

[0025]

[0026] edgpro b N represents the edge probability coefficient of pixel b. b This represents a 3×3 area centered at b. b Within, the number of pixels with a grayscale value smaller than b, value b This represents the grayscale value of b, jdg b This represents the boundary judgment value of b, if areab Within the range, if there are two pixels with the same grayscale value as b, then in JDG... b The value is 1, otherwise, jdg b The value is 0; w1, w2 and w3 represent the first weight, the second weight and the third weight, respectively.

[0027] Preferably, boundary recognition is performed based on the judged pixels to obtain the boundary set corresponding to the judged pixels, including:

[0028] Use EDGU c Represents the boundary set corresponding to pixel c, and obtains EDGU. c The process is as follows:

[0029] S1, store pixel c into EDGU c Pixel c is used as a relay pixel and enters S2;

[0030] S2, obtain the set of pixels midU that belong to IMGAU in the 8-neighborhood of the relay pixel;

[0031] S3, calculate the grayscale judgment value between each pixel in midU and the relay pixel respectively;

[0032] S4: Determine whether the minimum grayscale judgment value is greater than the set grayscale judgment value threshold. If yes, end the calculation; otherwise, proceed to S5.

[0033] S5, take the pixel corresponding to the smallest judgment value as the next relay pixel, and store the pixel corresponding to the smallest judgment value into EDGU. c Remove the pixel corresponding to the smallest judgment value from IMGAU and proceed to S2.

[0034] Preferably, the formula for calculating the grayscale judgment value is:

[0035] jdvalue c,d =|value c -value d |

[0036] jdvalue c,d The value represents the grayscale value of pixels c and d. d This represents the grayscale value of pixel d.

[0037] Preferably, the screening value for IMGA is calculated based on UNPIX, including:

[0038] Calculate the IMGA screening value using the following formula:

[0039]

[0040] filval represents the IMGA filter value, NUNPIX represents the total number of pixels in the UNPIX, and dist represents the total number of pixels in the UNPIX. i λ represents the average distance between pixel i and the other pixels in UNPIX, rownum represents the number of rows of pixels in IMGA, and λ represents the filtering weight, which is greater than 0.3 and less than 0.7.

[0041] Preferably, the artificial intelligence recognition module includes a database unit, a training unit, and a recognition unit;

[0042] The database unit is used to store images of the bridge surface used to train the deep learning model, as well as the types of defects present in the images;

[0043] The training unit is used to train the deep learning model based on the images stored in the database unit, and obtain the trained deep learning model.

[0044] The recognition unit is used to input the target image into the trained deep learning model and obtain the recognition result output by the trained deep learning model.

[0045] Preferably, if there is a defect in the target image, the identification result is the type of defect; if there is no defect in the target image, the identification result is normal detection.

[0046] Preferably, the deep learning model is trained based on the images stored in the database unit to obtain a trained deep learning model, including:

[0047] S10, preprocess the images stored in the database unit to obtain a preprocessed image set;

[0048] S20, divide the images in the preprocessed image set into training set, validation set and test set;

[0049] S30: Train the deep learning model using the training set and adjust the parameters of the deep learning model; during the training process, use the validation set to evaluate the performance of the deep learning model after parameter adjustment in order to further adjust the parameters of the deep learning model.

[0050] S40 uses the test set to test the deep learning model obtained in S30 and evaluate the performance metrics of the deep learning model.

[0051] If the performance metrics meet the requirements, it means that the deep learning model has been trained.

[0052] Beneficial effects:

[0053] Compared with existing technologies, this invention increases the probability of correctly selecting the highest quality surface image based on a subset of pixels in the surface image, thereby avoiding the need to select surface images based on all pixels, effectively improving the surface image selection efficiency, and thus improving the monitoring efficiency of the bridge monitoring process. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of an artificial intelligence-based bridge monitoring system according to the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention provides an artificial intelligence-based bridge monitoring system, including a drone and an image recognition device;

[0058] The drone is used to photograph the monitoring area of ​​the bridge surface to obtain surface images of the monitoring area of ​​the bridge.

[0059] The drone takes pictures of each monitoring area separately, and takes multiple surface images of each monitoring area. The obtained surface images are then transmitted to the image recognition device.

[0060] The image recognition device includes an image screening module and an artificial intelligence recognition module;

[0061] The image filtering module is used to filter surface images of the monitoring area based on set rules to obtain target images;

[0062] The artificial intelligence recognition module is used to recognize target images using a pre-trained deep learning model and output the recognition results;

[0063] The surface images of the monitoring area are filtered based on the set rules to obtain the target images, including:

[0064] Calculate the screening value for each surface image corresponding to the same monitoring area, including:

[0065] For monitoring area A, let UA represent the set of surface images of monitoring area A;

[0066] The first step is to use IMGA to represent the surface image in monitoring area A, and IMGAU to represent the set of pixels in IMGA;

[0067] The second step is to randomly select a pixel from IMGA as the judgment pixel.

[0068] The third step is to calculate the edge probability coefficients of the pixels.

[0069] Fourth step: If the edge probability coefficient is less than or equal to the set edge probability coefficient threshold, the pixel to be judged will be deleted from IMGAU and proceed to the fifth step; if the edge probability coefficient is greater than the set edge probability coefficient threshold, proceed to the sixth step.

[0070] Fifth step: Determine if the number of pixels in the IMGA is less than [a certain value]. NIMGA is the total number of pixels in IMGA. If it is, proceed to step eight; otherwise, proceed to step two.

[0071] Step 6: Perform boundary recognition based on the judged pixels to obtain the boundary set corresponding to the judged pixels;

[0072] Step 7: Determine if the number of pixels in the IMGA is less than [a certain value]. If yes, proceed to step eight; if no, remove the pixels in the boundary set from IMGA and proceed to step two.

[0073] Step 8: Use UNPIX to represent the union of all boundary sets, and calculate the screening value of IMGA based on UNPIX;

[0074] The surface image with the highest filter value is selected as the target image.

[0075] The above scheme increases the probability of correctly selecting the highest quality surface image based on a subset of pixels in the surface image, thereby avoiding the need to select surface images based on all pixels. This effectively improves the efficiency of surface image selection and, consequently, the monitoring efficiency of the bridge monitoring process.

[0076] During the acquisition of the set, this invention continuously and randomly judges the pixels in the IMGA, and only judges one-third of the pixels to obtain a screening value sufficient to correctly represent the quality of the IMGA. Because this invention continuously selects pixels with a higher probability of belonging to the edges of areas with defects such as cracks during the judgment process, and then obtains the boundary set based on these pixels, it can quickly obtain the edge pixels for calculating the screening value, without having to calculate the screening value based on all pixels, thus improving the effectiveness of the screening value.

[0077] Preferably, the monitoring area of ​​the bridge surface can be set according to the shooting parameters of the camera carried by the UAV. For example, when shooting the side of the bridge, the image of the side is divided into multiple regions, so that adjacent regions overlap. The larger the shooting range of the UAV, the larger the area of ​​the region. Each of the obtained regions is used as the monitoring area.

[0078] Preferably, the formula for calculating the marginal probability coefficient is:

[0079]

[0080] edgpro b N represents the edge probability coefficient of pixel b. b This represents a 3×3 area centered at b. b Within, the number of pixels with a grayscale value smaller than b, value b This represents the grayscale value of b, jdg b This represents the boundary judgment value of b, if area b Within the range, if there are two pixels with the same grayscale value as b, then in JDG... b The value is 1, otherwise, jdg b The value is 0; w1, w2 and w3 represent the first weight, the second weight and the third weight, respectively.

[0081] Specifically, the greater the difference between the gray value of pixel b and the surrounding pixels, and the smaller the difference between the gray value of pixel b and the pixel with the largest gray value, the higher the probability of meeting the boundary judgment requirements. In this way, the edge probability coefficient of pixel b is larger, which can effectively represent the probability of pixel b belonging to the boundary position from multiple different aspects, which is beneficial to improving the accuracy of the target image obtained by the present invention.

[0082] Preferably, the first weight is 0.2, the second weight is 0.4, and the third weight is 0.4.

[0083] Preferably, the threshold for the edge probability coefficient is 0.7.

[0084] Preferably, boundary recognition is performed based on the judged pixels to obtain the boundary set corresponding to the judged pixels, including:

[0085] Use EDGU c Represents the boundary set corresponding to pixel c, and obtains EDGU. c The process is as follows:

[0086] S1, store pixel c into EDGU c Pixel c is used as a relay pixel and enters S2;

[0087] S2, obtain the set of pixels midU that belong to IMGAU in the 8-neighborhood of the relay pixel;

[0088] S3, calculate the grayscale judgment value between each pixel in midU and the relay pixel respectively;

[0089] S4: Determine whether the minimum grayscale judgment value is greater than the set grayscale judgment value threshold. If yes, end the calculation; otherwise, proceed to S5.

[0090] S5, take the pixel corresponding to the smallest judgment value as the next relay pixel, and store the pixel corresponding to the smallest judgment value into EDGU. c Remove the pixel corresponding to the smallest judgment value from IMGAU and proceed to S2.

[0091] During boundary recognition, this invention only calculates the grayscale judgment value of pixels in midU. This successfully avoids repeated judgments of pixels and improves judgment efficiency. When the minimum grayscale judgment value is less than the judgment value threshold, it indicates that the probability of pixels surrounding the relay pixel also belonging to the boundary region is low. At this point, the acquisition of the boundary set ends. The above process enables the extension of the boundary where pixel c is located, thereby effectively identifying pixels surrounding the boundary.

[0092] Preferably, the formula for calculating the grayscale judgment value is:

[0093] jdvalue c,d =|value c -value d |

[0094] jdvalue c,d The value represents the grayscale value of pixels c and d. d This represents the grayscale value of pixel d.

[0095] Preferably, the grayscale judgment value threshold is 6.

[0096] Preferably, the screening value for IMGA is calculated based on UNPIX, including:

[0097] Calculate the IMGA screening value using the following formula:

[0098]

[0099] filval represents the IMGA filter value, NUNPIX represents the total number of pixels in the UNPIX, and dist represents the total number of pixels in the UNPIX. i λ represents the average distance between pixel i and the other pixels in UNPIX, rownum represents the number of rows of pixels in IMGA, and λ represents the filtering weight, which is greater than 0.3 and less than 0.7.

[0100] In calculating the screening value, this invention utilizes both the total number of pixels in the UNPIX array as a parameter and the distance differences between pixels in the UNPIX array. This ensures that the more pixels belonging to the image's boundary regions in the IMGA array, and the wider their distribution, the higher the screening value. Consequently, images with clearer boundaries and larger boundaries have higher screening values, which is beneficial for effectively selecting high-quality surface images.

[0101] Preferably, the screening weight is 0.55.

[0102] Preferably, the artificial intelligence recognition module includes a database unit, a training unit, and a recognition unit;

[0103] The database unit is used to store images of the bridge surface used to train the deep learning model, as well as the types of defects present in the images;

[0104] The training unit is used to train the deep learning model based on the images stored in the database unit, and obtain the trained deep learning model.

[0105] The recognition unit is used to input the target image into the trained deep learning model and obtain the recognition result output by the trained deep learning model.

[0106] Specifically, images of the bridge surface can be captured using drones, fixed cameras, or handheld devices. These images should include cracks of different types and sizes, as well as crack-free sections of the bridge surface.

[0107] Preferably, if there is a defect in the target image, the identification result is the type of defect; if there is no defect in the target image, the identification result is normal detection.

[0108] Specifically, the types of defects include cracks, holes, and exposed rebar.

[0109] Preferably, the deep learning model is trained based on the images stored in the database unit to obtain a trained deep learning model, including:

[0110] S10: Preprocess the images stored in the database unit to obtain a preprocessed image set.

[0111] Specifically, the preprocessing process is as follows:

[0112] Image enhancement: Image enhancement processing, such as adjusting brightness, contrast, and denoising, to improve image quality and model robustness.

[0113] Data augmentation: Expanding the dataset through operations such as rotation, flipping, and scaling increases the diversity of training samples and prevents model overfitting.

[0114] S20 divides the images in the preprocessed image set into training set, validation set and test set.

[0115] A common partitioning ratio is 60-80% for the training set, 10-20% for the validation set, and 10-20% for the test set. The specific ratio can be adjusted according to the amount of data and actual needs.

[0116] S30: Train the deep learning model using the training set and adjust the parameters of the deep learning model; during the training process, use the validation set to evaluate the performance of the deep learning model after parameter adjustment in order to further adjust the parameters of the deep learning model.

[0117] The model is trained using a training set, with parameters continuously adjusted via forward and backpropagation algorithms. Common optimization algorithms include SGD and Adam. The model's performance is then evaluated using a validation set, and hyperparameters (such as learning rate and batch size) are adjusted to optimize the model. After each training epoch or several training iterations, the model's performance is tested using a validation set. This helps monitor for overfitting (i.e., the model performs well on the training set but poorly on unseen data). This, in turn, helps in selecting the optimal model structure and parameter settings, such as determining the optimal learning rate, number of hidden layers, and regularization parameters.

[0118] S40 uses the test set to test the deep learning model obtained in S30 and evaluates the performance metrics of the deep learning model.

[0119] The data in the test set has never been used during model training, thus providing an objective evaluation of the model's performance on new data. This evaluates the model's effectiveness in real-world applications and determines its final performance metrics, such as accuracy, precision, and recall.

[0120] If the performance metrics meet the requirements, it means that the deep learning model has been trained.

[0121] To determine whether a performance metric meets the requirements, the metric can be compared with its corresponding threshold. For example, the accuracy rate can be greater than the set accuracy threshold.

[0122] If the performance metrics do not meet the requirements, a trained deep learning model can be obtained by replacing the image of the bridge surface used for training and re-training the deep learning model.

[0123] In this invention, the deep learning model can be a convolutional neural network (CNN), commonly including Faster R-CNN, YOLO (You Only Look Once), Mask R-CNN, etc.

[0124] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A bridge monitoring system based on artificial intelligence, characterized in that, Including drones and image recognition devices; The drone is used to photograph the monitoring area of ​​the bridge surface to obtain surface images of the monitoring area of ​​the bridge. The image recognition device includes an image screening module and an artificial intelligence recognition module; The image filtering module is used to filter surface images of the monitoring area based on set rules to obtain target images; The artificial intelligence recognition module is used to recognize target images using a pre-trained deep learning model and output the recognition results; The surface images of the monitoring area are filtered based on the set rules to obtain the target images, including: Calculate the screening value for each surface image corresponding to the same monitoring area, including: For monitoring area A, let UA represent the set of surface images of monitoring area A; The first step is to use IMGA to represent the surface image in monitoring area A, and IMGAU to represent the set of pixels in IMGA; The second step is to randomly select a pixel from IMGA as the judgment pixel. The third step is to calculate the edge probability coefficients of the pixels. Fourth step: If the edge probability coefficient is less than or equal to the set edge probability coefficient threshold, the pixel to be judged will be deleted from IMGAU and proceed to the fifth step; if the edge probability coefficient is greater than the set edge probability coefficient threshold, proceed to the sixth step. Fifth step: Determine if the number of pixels in the IMGA is less than [a certain value]. , The total number of pixels in the IMGA. If yes, proceed to step eight; otherwise, proceed to step two. Step 6: Perform boundary recognition based on the judged pixels to obtain the boundary set corresponding to the judged pixels; Step 7: Determine if the number of pixels in the IMGA is less than [a certain value]. If yes, proceed to step eight; otherwise, remove the pixels in the boundary set from IMGA and proceed to step two. Step 8: Use UNPIX to represent the union of all boundary sets, and calculate the screening value of IMGA based on UNPIX; The surface image with the highest filter value is used as the target image; The formula for calculating the marginal probability coefficient is: This represents the edge probability coefficient of pixel b. Indicates that b is the center. area Within, the number of pixels with a grayscale value smaller than b. This represents the grayscale value of b. This represents the boundary judgment value of b, if Within the range, if there are two pixels with the same gray value as b, then... The value is 1, otherwise, The value is 0; , and These represent the first weight, the second weight, and the third weight, respectively. The IMGA screening value is calculated based on UNPIX, including: Calculate the IMGA screening value using the following formula: This represents the filter value for IMGA. This represents the total number of pixels in UNPIX. This represents the average distance between pixel i and the remaining pixels in the UNPIX, and rownum represents the row number of pixels in the IMGA. Indicates the filtering weight. Greater than 0.3 and less than 0.

7.

2. The bridge monitoring system based on artificial intelligence according to claim 1, characterized in that, Boundary recognition is performed based on the identified pixels to obtain the set of boundaries corresponding to the identified pixels, including: use Represents the boundary set corresponding to pixel c, and obtains... The process is as follows: S1, store pixel c. Pixel c is used as a relay pixel and enters S2; S2, obtain the set of pixels midU that belong to IMGAU in the 8-neighborhood of the relay pixel; S3, calculate the grayscale judgment value between each pixel in midU and the relay pixel respectively; S4: Determine whether the minimum grayscale judgment value is greater than the set grayscale judgment value threshold. If yes, end the calculation; otherwise, proceed to S5. S5, take the pixel corresponding to the smallest judgment value as the next relay pixel, and store the pixel corresponding to the smallest judgment value in... Remove the pixel corresponding to the smallest judgment value from IMGAU and proceed to S2.

3. The bridge monitoring system based on artificial intelligence according to claim 2, characterized in that, The formula for calculating the grayscale judgment value is: This represents the grayscale values ​​of pixels c and d. This represents the grayscale value of pixel d.

4. The bridge monitoring system based on artificial intelligence according to claim 1, characterized in that, The artificial intelligence recognition module includes a database unit, a training unit, and a recognition unit; The database unit is used to store images of the bridge surface used to train the deep learning model, as well as the types of defects present in the images; The training unit is used to train the deep learning model based on the images stored in the database unit, and obtain the trained deep learning model. The recognition unit is used to input the target image into the trained deep learning model and obtain the recognition result output by the trained deep learning model.

5. The bridge monitoring system based on artificial intelligence according to claim 4, characterized in that, If a defect exists in the target image, the identification result is the type of defect; if no defect exists in the target image, the identification result is normal detection.

6. The bridge monitoring system based on artificial intelligence according to claim 4, characterized in that, The deep learning model is trained based on the images stored in the database unit to obtain a trained deep learning model, including: S10, preprocess the images stored in the database unit to obtain a preprocessed image set; S20, divide the images in the preprocessed image set into training set, validation set and test set; S30: Train the deep learning model using the training set and adjust the parameters of the deep learning model; during the training process, use the validation set to evaluate the performance of the deep learning model after parameter adjustment in order to further adjust the parameters of the deep learning model. S40 uses the test set to test the deep learning model obtained in S30 and evaluate the performance metrics of the deep learning model. If the performance metrics meet the requirements, it means that the deep learning model has been trained.

Images