Urban infrastructure maintenance method and system based on image recognition

Abstract

The invention relates to the technical field of city infrastructure maintenance, and discloses a city infrastructure maintenance method and system based on image recognition, and the method comprises the following steps: S1, obtaining a city infrastructure image; s2, carrying out image recognition on the urban infrastructure image, and screening out defect images containing defects; s3, performing feature extraction on the defect image to obtain defect features; s4, determining the risk level of the urban infrastructure based on the defect features; and S5, based on the risk level of the urban infrastructure, arranging maintenance for the urban infrastructure. Through technical innovation, the problems of low efficiency, insufficient accuracy, unreasonable resource allocation and the like in the current urban infrastructure maintenance are effectively solved, and the urban infrastructure maintenance is promoted to develop towards a more intelligent, efficient and sustainable direction.

Description

Technical Field

[0001] The present invention relates to the technical field of urban infrastructure maintenance, and in particular to an urban infrastructure maintenance method and system based on image recognition. Background Art

[0002] With the acceleration of global urbanization, the health of urban infrastructure, as the basic skeleton supporting urban operations, directly affects the normal operation of cities and the quality of life of residents. Urban infrastructure covers a wide range, from bridges, roads, tunnels in the transportation network to underground pipe systems, and the maintenance and management of each link is crucial. However, the traditional manual inspection method is not only time-consuming and labor-intensive, but also inefficient, and difficult to detect potential problems in a timely and accurate manner, often resulting in small problems accumulating into major hidden dangers, increasing maintenance costs, and even causing safety accidents. In recent years, with the rapid development of artificial intelligence technology, especially the advancement of image recognition technology, new solutions have been provided for the intelligent management of urban infrastructure. Image recognition technology can automatically identify specific targets or abnormal situations in a large amount of image data through machine learning models, significantly improving the accuracy and efficiency of detection, not only reducing the workload of manual inspections, but also enabling rapid census of large-scale infrastructure, timely detection of hidden dangers, and providing a scientific basis for preventive maintenance.

[0003] Despite this, the current urban infrastructure management technology based on image recognition still faces a series of challenges and shortcomings. First, the existing technology still has room for improvement in the fine-grainedness of defect identification, especially in quantifying the degree of defects, which often requires more sophisticated image processing technology and algorithm support. Furthermore, the risk assessment system mostly relies on artificially set rules and thresholds, lacks dynamic adaptability and flexibility, and is difficult to fully and objectively reflect the actual status and potential risks of the facilities. In addition, the existing technology is often simple and extensive in the formulation of response strategies after defect identification, such as the generation of maintenance plans, and does not fully consider complex factors such as optimal resource allocation and cost-benefit analysis, which affects the efficient execution of maintenance management.

[0004] For example, the patent application with publication number CN1 17522650A discloses a method and system for realizing digital maintenance of urban lighting facilities, including wirelessly connected lighting monitoring equipment and platform system, wherein the lighting monitoring equipment includes a lighting centralized controller acting on a distribution cabinet, a single lamp controller acting on a lamp pole, and a disconnection alarm acting on a cable, which can control and monitor lighting facilities in real time; the platform system includes: a system configuration unit for generating configuration information; an automatic operation unit for realizing automatic operation of lighting facilities; a maintenance management unit for filtering faults and automatically generating fault repair work orders; a message notification unit for pushing the fault information of lighting facilities and the assignment information of repair work orders; and a database for storing configuration information. The invention can detect faults in a timely and accurate manner, avoid false alarms, missed alarms, and repeated alarms of faults, accurately dispatch work orders according to the priority of the fault type, monitor and evaluate the entire process of fault repair, and realize closed-loop digital management of urban lighting facility maintenance.

[0005] For example, the patent application with publication number CN117350908A discloses a city-level transportation infrastructure maintenance system, including: an intelligent inspection module for obtaining images and data information of transportation infrastructure; a data analysis module for processing and analyzing the collected images and data; an anomaly detection module for identifying abnormal conditions of transportation infrastructure; a maintenance decision module for formulating maintenance plans, optimizing resource layout and scheduling; a maintenance execution module for carrying out actual maintenance work, repairing and reinforcing transportation infrastructure according to maintenance decisions; and a data management module for storing, managing and querying collected data. This invention has the effect of improving maintenance efficiency, reducing maintenance costs, and discovering and solving potential problems in advance.

[0006] The above patents all have the problems raised by this background technology: risk assessment lacks dynamic adaptability and flexibility, and it is difficult to fully and objectively reflect the actual situation and potential risks of the facilities; the generation of maintenance plans is relatively simple and rough, which affects the efficient execution of maintenance management.

[0007] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgement or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the invention

[0008] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide an urban infrastructure maintenance method and system based on image recognition to solve the problems of low efficiency, insufficient accuracy and unreasonable resource allocation faced in the current maintenance of urban infrastructure, and to promote urban management to develop in a more intelligent, efficient and sustainable direction.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] In one aspect, the present invention provides a method for urban infrastructure maintenance based on image recognition, comprising the following steps: S1: acquiring an image of urban infrastructure;

[0011] S2: performing image recognition on the urban infrastructure image to screen out defective images containing defects;

[0012] S3: extracting features from the defect image to obtain defect features;

[0013] S4: Determine the risk level of the urban infrastructure based on the defect characteristics;

[0014] S5: Arrange maintenance of urban infrastructure based on its risk level.

[0015] As a preferred solution of the urban infrastructure maintenance method based on image recognition described in the present invention, wherein: the urban infrastructure image includes a bridge image, a road image, a drainage pipe image, and a tunnel image;

[0016] The defects included in the defect image are specifically as follows:

[0017] The defects contained in the bridge images include bridge cracks, rust, structural deformation, and concrete spalling;

[0018] Defects contained in road images include road cracks, potholes, worn markings, and water seepage on the road surface;

[0019] The defects contained in the drainage pipe image include blockage, corrosion and damage, dislocation and fracture, and vegetation invasion;

[0020] Tunnel images included defects including lining cracks, water seepage, and surface spalling.

[0021] As a preferred solution of the urban infrastructure maintenance method based on image recognition of the present invention, the image recognition method is as follows:

[0022] S21: collecting images of urban infrastructure containing each defect and images of urban infrastructure in normal state, and performing preprocessing;

[0023] S22: Each collected urban infrastructure image is labeled by category;

[0024] S23: Build image recognition model;

[0025] S24: dividing the images with completed category annotation into a training set, a validation set, and a test set, and performing model training of the image recognition model through the training set;

[0026] S25: Perform model verification and tuning of the image recognition model through the validation set; perform model testing, evaluation and adjustment of the image recognition model through the test set;

[0027] S26: Save the trained image recognition model and deploy the application, input the preprocessed urban infrastructure image for image recognition, and filter out defective images containing defects.

[0028] As a preferred solution of the urban infrastructure maintenance method based on image recognition of the present invention, the defect characteristics are as follows:

[0029] The defect characteristics of the bridge cracks are the length and width of the bridge cracks; the defect characteristics of the corrosion are the corrosion area; the defect characteristics of the structural deformation are the distortion angle; the defect characteristics of the concrete spalling are the spalling area and spalling depth;

[0030] The defect characteristics of the road cracks are the length and width of the road cracks; the defect characteristics of the potholes are the diameter and depth of the potholes; the defect characteristics of the marking line wear are the remaining length and brightness ratio; the defect characteristics of the road water seepage are the water seepage area;

[0031] The defect characteristics of the blockage are the percentage of reduction in the cross-sectional area of ​​the pipeline; the defect characteristics of the corrosion damage are the corrosion area; the defect characteristics of the dislocation fracture are the displacement and fracture length; the defect characteristics of the vegetation invasion are the vegetation area;

[0032] The defect types included in the lining cracks are lining crack length and lining crack width; the defect characteristics included in the water leakage are the number of leakage points; and the defect type included in the surface spalling is the spalling area.

[0033] As a preferred solution of the urban infrastructure maintenance method based on image recognition of the present invention, the method for determining the risk level of urban infrastructure is as follows:

[0034] S41: establishing a risk level set and a defect feature set for the defect image;

[0035] S42: normalizing the defect features;

[0036] S43: construct a weight set of each defect feature, denoted as W;

[0037] S44: Construct a relationship matrix between defect characteristics and risk levels as follows:

[0038] Select the membership function, calculate the membership degree of each defect feature for each risk level, and form the membership degree into a relationship matrix R;

[0039] S45: Calculate the risk level of urban infrastructure based on the weight set and the relationship matrix; the method is as follows:

[0040] Construct the risk level comprehensive matrix A, the formula is as follows:

[0041] A = W·R;

[0042] A is an m-dimensional row vector in the following form:

[0043] A={a1,a2,......,a m};

[0044] Among them, a j represents the probability that the risk level of urban infrastructure is the jth level; the value range of j is 1, 2, ······, m;

[0045] Determine the risk level of urban infrastructure as follows:

[0046] Divide the interval (0, 1] into m segments, where the pth segment is Among them, the value range of p is 1, 2, ..., m; a q It represents the probability that the risk level of urban infrastructure is the qth level; a random number b is generated with a value range of (0, 1]; if the value of b falls within the pth interval, the risk level of urban infrastructure is the pth level.

[0047] As a preferred solution of the urban infrastructure maintenance method based on image recognition of the present invention, the maintenance arrangement includes calculating the inspection cycle and selecting the maintenance time; wherein the method for calculating the inspection cycle is as follows:

[0048] Assuming the risk level of any urban infrastructure is level p, the calculation formula for the inspection cycle is as follows:

[0049]

[0050] Wherein, T represents the inspection cycle, T0 represents the regular inspection cycle, and α is the adjustment factor.

[0051] As a preferred solution of the urban infrastructure maintenance method based on image recognition of the present invention, the method of selecting the maintenance time is as follows:

[0052] Set the risk level threshold β. If the risk level of any urban infrastructure is greater than level β, select the maintenance time; determine the maintenance duration, and calculate the target factor corresponding to different maintenance start times. The formula is as follows:

[0053] C = c0 × exp(λ × (t-t0)) + c t1×T s +c t2 ×T s ;

[0054] Among them, C represents the target factor, λ represents the adjustment coefficient; t represents the time point when the maintenance starts, t0 represents the current time point, T s represents the maintenance time; c0 represents the risk cost factor, which represents the potential loss cost caused by the current risk level; c t1 Represents the labor cost factor, which means that it starts at time t and lasts for T s The maintenance labor cost per unit time in the time period; c t2 represents the maintenance cost factor, which means that it starts at time t and lasts for T s The downtime maintenance cost per unit time within the time period;

[0055] An optimization algorithm is used to minimize the target factor, and the start maintenance time point that minimizes the target factor is extracted as the selection result of the maintenance time.

[0056] In a second aspect, the present invention provides an urban infrastructure maintenance system based on image recognition, including an image acquisition module, a preprocessing module, an image recognition module, a feature extraction module, a risk level module, and a maintenance module; wherein:

[0057] The image acquisition module is used to acquire images of urban infrastructure;

[0058] The preprocessing module is used to preprocess the urban infrastructure image;

[0059] The image recognition module is used to perform image recognition on the basic urban image and filter out defective images;

[0060] The feature extraction module is used to extract defect features from the defect image;

[0061] The risk level module determines the risk level of the urban infrastructure based on the defect characteristics;

[0062] The maintenance module arranges maintenance of the urban infrastructure based on the risk level of the urban infrastructure.

[0063] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0064] Through automated image recognition technology, images of infrastructure with defects can be quickly screened out. Compared with traditional manual inspections, it significantly improves the detection speed and recognition accuracy, reduces omissions or misjudgments caused by human factors, and ensures that potential problems are discovered in a timely manner. Detailed feature extraction is performed on defective images screened out by image recognition, and the infrastructure is classified into risk levels based on the extracted defect features, making the risk assessment process more objective and systematic, which is conducive to giving priority to high-risk infrastructure and optimizing resource allocation. According to the risk level, the maintenance plan is formulated using optimization algorithms, which can not only ensure that the infrastructure is maintained in a timely manner, but also reasonably arrange the inspection cycle and maintenance time based on cost-effectiveness, avoid unnecessary maintenance cost expenditures, and improve the economy and efficiency of maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0066] Figure 1 A flow chart of the urban infrastructure maintenance method based on image recognition provided by the present invention;

[0067] Figure 2 A schematic diagram of the structure of the urban infrastructure maintenance system based on image recognition provided by the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0069] Example 1

[0070] This embodiment introduces a method for urban infrastructure maintenance based on image recognition. Figure 1 , the method comprises the following steps:

[0071] S1: Acquire urban infrastructure images;

[0072] The urban infrastructure images include bridge images, road images, drainage pipe images, and tunnel images;

[0073] After acquiring the urban infrastructure images, image preprocessing is performed, including filtering noise reduction, geometric correction, and size standardization. First, median filtering, Gaussian filtering and other technologies are used to reduce random noise in urban infrastructure images, improve image quality, and avoid noise interference in subsequent feature extraction; then the image deformation caused by shooting angle, lens distortion and other factors is corrected to ensure that the lines and shapes in the image are as close to reality as possible, which is convenient for subsequent measurement and analysis. Finally, size standardization is performed to adjust the urban infrastructure images to a uniform size for subsequent image processing and to ensure comparability between images from different sources or at different times.

[0074] S2: performing image recognition on the urban infrastructure image to screen out defective images containing defects;

[0075] The defects included in the defect image are specifically as follows:

[0076] The defects contained in the bridge images include bridge cracks, rust, structural deformation, and concrete spalling. The above bridge defects directly affect the load-bearing capacity and stability of the bridge. Early detection and repair can avoid catastrophic accidents such as bridge collapse and protect the safety of pedestrians and vehicles.

[0077] Defects contained in road images include road cracks, potholes, worn markings, and water seepage in the road surface. Timely detection of the above road defects can reduce traffic accidents, improve driving comfort and safety, reduce vehicle losses caused by poor road conditions, prevent roadbed damage caused by water infiltration, reduce the risk of road collapse, and protect the underlying drainage system from damage.

[0078] The defects contained in the drainage pipe image include blockage, corrosion and damage, dislocation and fracture, and vegetation invasion. By checking the above pipe defects, we can ensure smooth drainage, prevent urban waterlogging, reduce environmental pollution and public health problems caused by sewage overflow, and avoid more serious damage to underground infrastructure.

[0079] Tunnel images contain defects such as lining cracks, water leakage, and surface peeling. As an important traffic channel, the safety of tunnels is directly related to the safety of personnel and the smooth flow of traffic. Timely discovery and repair of these tunnel defects can prevent tunnel structure deterioration, avoid collapse or safety accidents, and ensure the long-term stable operation of the tunnel.

[0080] The image recognition method is as follows:

[0081] S21: collect images of urban infrastructure with each defect and images of urban infrastructure in normal state, and perform preprocessing; balance the number of images of each category as much as possible to avoid classification bias caused by category imbalance;

[0082] S22: Each collected urban infrastructure image is labeled with a category, that is, each image is labeled with a corresponding category label, such as "bridge cracks", "normal bridge", etc.

[0083] S23: Build an image recognition model; use pre-trained image classification models as the basis, such as ResNet, VGG, MobileNet, etc. These models have been pre-trained on large image datasets (such as ImageNet) and can recognize a variety of images, making them suitable as basic models for transfer learning. Fine-tune the basic model. Optional fine-tuning includes unfreezing part of the top-level network, replacing the last layer of the network structure with a deconvolution layer, and adjusting hyperparameters such as the learning rate.

[0084] S24: dividing the images with completed category annotation into a training set, a validation set, and a test set, and performing model training of the image recognition model through the training set;

[0085] S25: Perform model verification and tuning of the image recognition model through the validation set; perform model testing, evaluation, and adjustment of the image recognition model through the test set. Monitor performance indicators (such as accuracy, confusion matrix) on the validation set to avoid overfitting. Adjust model parameters or network architecture based on the validation results until satisfactory performance is achieved. Then evaluate model performance on the test set to ensure that the model has strong generalization capabilities. Analyze the model's error cases to understand in which scenarios the model performs poorly and consider whether further data enhancement or model adjustment is needed.

[0086] S26: Save the trained image recognition model and deploy the application, input the preprocessed urban infrastructure image for image recognition, and filter out defective images containing defects.

[0087] The present embodiment involves a variety of urban infrastructure, including bridges, roads, drainage pipes, and tunnels; for each of the above urban infrastructures, a model is trained separately for image recognition; for example, by executing steps S21 to S25, an image recognition model for defect recognition of bridge images is obtained, wherein the training set, the validation set, and the test set all cover only bridge images; then steps S21 to S25 are re-executed to obtain an image recognition model for defect recognition of road images, wherein the training set, the validation set, and the test set used this time all cover only road images; and so on, a total of 4 image recognition models are trained, which are used to recognize 4 types of urban infrastructure images respectively.

[0088] S3: extracting features from the defect image to obtain defect features;

[0089] The defect characteristics are as follows:

[0090] The defect characteristics of the bridge cracks are the length and width of the bridge cracks; the defect characteristics of the rust are the rust area; the defect characteristics of the structural deformation are the distortion angle. The bridge may be distorted when subjected to asymmetric loads or special environmental conditions. The distortion angle of the bridge deck can be calculated through image analysis to evaluate its impact on structural safety; the defect characteristics of the concrete spalling are the spalling area and spalling depth;

[0091] The defect features of the road cracks are the length and width of the road cracks; the defect features of the potholes are the diameter and depth of the potholes; the defect features of the marking line wear are the remaining length and brightness ratio. The brightness ratio is the ratio of the color brightness of the worn marking line to the new marking line. The color of the worn marking line will become dim, and the contrast with the road surface will be reduced, reducing its visual impact; the defect feature of the road seepage is the seepage area;

[0092] The defect characteristics of the blockage are the percentage of reduction in the cross-sectional area of ​​the pipeline; the defect characteristics of the corrosion damage are the corrosion area; the defect characteristics of the dislocation fracture are the displacement and fracture length; the defect characteristics of the vegetation invasion are the vegetation area;

[0093] The defect types included in the lining cracks are lining crack length and lining crack width; the defect characteristics included in the water leakage are the number of leakage points; and the defect type included in the surface spalling is the spalling area.

[0094] The feature extraction method is as follows:

[0095] The method for extracting the bridge crack length, bridge crack width, pavement crack length, pavement crack width, lining crack length, lining crack width, and fracture length is to identify the crack or fracture edge through an edge detection algorithm (such as Canny edge detection), and then determine the length and width of the crack through straight line fitting.

[0096] The reduction percentage of the rust area, the peeling area, the water seepage area, and the pipe cross-sectional area; the feature extraction method of the corrosion area, the vegetation area, and the peeling area is to first segment the corresponding defects, and then calculate the corresponding defect area according to the number of pixels in the defect area;

[0097] The feature extraction method of the peeling depth, pit diameter, pit depth, remaining length and displacement is to first segment the corresponding defects and then calculate the defect length or depth by stereo vision technology;

[0098] The feature extraction method of the distortion angle is to use image registration technology to compare the images before and after deformation to calculate the distortion angle of the bridge deck;

[0099] The feature extraction method of the brightness ratio is to determine the wear area by using a color model and brightness analysis, and calculate the brightness ratio by comparing the brightness values ​​of the new and old markings.

[0100] S4: Based on the defect characteristics, determine the risk level of the urban infrastructure; the method is as follows:

[0101] S41: establishing a risk level set and a defect feature set for the defect image;

[0102] The risk level set is denoted as F, which is the level set of defect risk of any urban infrastructure, in the following form:

[0103] F={f1,f2,……,f m};

[0104] Among them, f j Indicates that the risk level is level j, and the value range of j is 1, 2, ..., m, where m is a positive integer. The larger the j, the higher the risk level and the more serious the defect.

[0105] The defect feature set is denoted as X, which is a set of defect features of urban infrastructure corresponding to any defect image, and is in the following form:

[0106] X={x1,x2,…,x n};

[0107] Among them, x i is the value of the i-th defect feature, the value range of i is 1, 2, ..., n, and n is the number of defect features of urban infrastructure;

[0108] S42: Normalize the defect features; the method is as follows:

[0109] Set the first threshold x for each state parameter imin , the second threshold x imax ;

[0110] The value of each defect feature is normalized according to the following formula:

[0111]

[0112] Among them, x′ i Defect feature x i Normalized value.

[0113] S43: Construct a weight set for each defect feature; the method is as follows:

[0114] Calculate the constant weight of each defect feature based on the analytic hierarchy process;

[0115] Calculate the weight of each defect feature using the following formula:

[0116]

[0117] Among them, w′ i is the constant weight of the i-th defect feature; w i is the weight of the i-th defect feature; w′ k is the constant weight of the k-th defect feature; x′ k is the normalized value of the k-th defect feature, and the value range of p is 1, 2, ..., n;

[0118] Construct the weight set of each defect feature, denoted as W, in the following form:

[0119] W = {w1, w2, ..., w n}.

[0120] S44: Construct a relationship matrix between defect characteristics and risk levels as follows:

[0121] Select the membership function, calculate the membership of each defect feature for each risk level, and form the membership into a relationship matrix R in the following form:

[0122]

[0123] Among them, r ij It represents the membership of the i-th defect feature to the j-th risk level. The value range of i is 1, 2, ..., n; the value range of j is 1, 2, ..., m.

[0124] S45: Calculate the risk level of urban infrastructure based on the weight set and the relationship matrix; the method is as follows:

[0125] Construct the risk level comprehensive matrix A, the formula is as follows:

[0126] A=W·R.

[0127] A is an m-dimensional row vector in the following form:

[0128] A={a1,a2,......,a m};

[0129] Among them, a j represents the probability that the risk level of urban infrastructure is the jth level; the value range of j is 1, 2, ······, m;

[0130] Determine the risk level of urban infrastructure as follows:

[0131] Divide the interval (0, 1] into m segments, where the pth segment is Among them, the value range of p is 1, 2, ..., m; aq It represents the probability that the risk level of urban infrastructure is level q;

[0132] Generate a random number b with a value range of (0, 1]; if the value of b falls within the pth interval, the risk level of the urban infrastructure is the pth level. The risk level comprehensive matrix gives the probability of the urban infrastructure being at each risk level, but the risk level with the highest probability may not be the actual risk level of the urban infrastructure. By mapping the probability of each risk level to the interval length, generating a random number and observing which risk level the random number falls into, the unpredictability in the real scene can be simulated; at the same time, the greater the probability of the risk level being selected by the random number, the greater the probability, which can provide a higher risk level judgment accuracy and has good robustness.

[0133] S5: Arrange maintenance of urban infrastructure based on its risk level.

[0134] The maintenance arrangement includes calculating the inspection cycle and selecting the maintenance time;

[0135] The method for calculating the inspection cycle is as follows:

[0136] Assuming the risk level of any urban infrastructure is level p, the calculation formula for the inspection cycle is as follows:

[0137]

[0138] Among them, T represents the inspection cycle, T0 represents the regular inspection cycle, which is set by technicians in this field based on industry experience; α is an adjustment factor, which is set by technicians in this field based on actual needs. According to the above formula, the higher the risk level, that is, the larger the value of p, the smaller the inspection cycle T, thereby strengthening the inspection of urban infrastructure with higher risk levels to ensure that potential facility failures and possible dangerous situations can be discovered in a timely manner.

[0139] The method for selecting the maintenance time is as follows:

[0140] Set the risk level threshold β. If the risk level of any urban infrastructure is greater than level β, select the maintenance time, determine the maintenance duration, and calculate the target factors corresponding to different maintenance start times. The formula is as follows:

[0141] C = c0 × exp(λ × (t-t0)) + c t1 ×T s +c t2 ×T s ;

[0142] Among them, C represents the target factor, λ represents the adjustment coefficient, which is set according to actual needs; t represents the time point when maintenance starts, t0 represents the current time point, and T s represents the maintenance time, which is set by technical personnel based on experience; c0 represents the risk cost factor, which represents the potential loss cost caused by the current risk level; c t1 Represents the labor cost factor, which means that it starts at time t and lasts for T s The maintenance labor cost per unit time in the time period; C t2 represents the maintenance cost factor, which means that it starts at time t and lasts for T s The downtime maintenance cost per unit time in the time period; c0, c t1 、c t2 All are set according to expert experience. In this embodiment, it is assumed that the maintenance time of any urban infrastructure is fixed, and the technical personnel in this field estimate and determine the maintenance time of each urban infrastructure based on experience. According to the above formula, as the time point t at which maintenance starts increases, the target factor increases exponentially, which means that for defective urban infrastructure, the later the maintenance starts, the larger the target factor; different time points t at which maintenance starts correspond to different maintenance cost factors and labor cost factors. Therefore, an optimization algorithm, such as a genetic algorithm, is used to minimize the target factor, and the maintenance start time point that minimizes the target factor is extracted as the result of the maintenance time selection. This can find the best balance between timely maintenance and lower maintenance costs, and while timely repairing defective facilities, minimize the inconvenience caused by facility maintenance to the city.

[0143] Example 2

[0144] This embodiment is the second embodiment of the present invention; it is based on the same inventive concept as embodiment 1, Figure 2 This embodiment introduces an urban infrastructure maintenance system based on image recognition, including an image acquisition module, a preprocessing module, an image recognition module, a feature extraction module, a risk level module, and a maintenance module; wherein:

[0145] The image acquisition module is used to obtain images of urban infrastructure. The module uses high-definition cameras and other sensor devices to regularly shoot and monitor urban infrastructure and collect images of urban infrastructure. These devices can be installed on drones, satellites, or directly fixed on the infrastructure.

[0146] The preprocessing module is used to preprocess the urban infrastructure images; clean and preprocess the collected original urban facility images, including removing noise, correcting perspective, size cropping, etc., to facilitate subsequent image recognition and feature extraction.

[0147] The image recognition module is used to perform image recognition on the basic urban image and screen out defective images; the module is configured with an image recognition model, which is based on a pre-trained image classification model, such as ResNet, VGG, MobileNet, etc., and is fine-tuned and trained with a large amount of labeled data to ensure that the model can accurately identify different defect types.

[0148] The feature extraction module is used to extract defect features from the defect image; the module is configured with relevant algorithm models and tools for extracting image features, including but not limited to image segmentation algorithms (such as semantic segmentation based on threshold, clustering, deep learning), edge detection algorithms (Canny, Sobel operators, etc.), feature matching algorithms (S IFT, SURF, ORB, etc.), stereo vision algorithms, etc.

[0149] The risk level module determines the risk level of the urban infrastructure based on the defect characteristics; the module calculates the risk level through fuzzy comprehensive evaluation.

[0150] The maintenance module arranges maintenance for urban infrastructure based on the risk level of urban infrastructure, including calculating the inspection cycle and selecting the maintenance time plan. The photo inspection cycle is set based on the risk level. The higher the risk level, the shorter the inspection cycle. Combined with the budget, human resources, and considering factors such as downtime cost and labor cost, the maintenance time is reasonably selected.

[0151] The specific functional implementation of each of the above modules can be found in the relevant contents of the urban infrastructure maintenance method based on image recognition described in Example 1, and will not be elaborated here.

[0152] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of the present invention, which are all within the protection of the present invention.

Claims

1. A method for urban infrastructure maintenance based on image recognition, Features: The following steps are involved: S1: Acquire urban infrastructure images; S2: performing image recognition on the urban infrastructure image to screen out defective images containing defects; S3: extracting features from the defect image to obtain defect features; S4: Determine the risk level of the urban infrastructure based on the defect characteristics; S5: Arrange maintenance of urban infrastructure based on its risk level.

2. A method for urban infrastructure maintenance based on image recognition as claimed in claim 1, Features: The urban infrastructure images include bridge images, road images, drainage pipe images, and tunnel images; The defects included in the defect image are specifically as follows: The defects contained in the bridge images include bridge cracks, rust, structural deformation, and concrete spalling; Defects contained in road images include road cracks, potholes, worn markings, and water seepage on the road surface; The defects contained in the drainage pipe image include blockage, corrosion and damage, dislocation and fracture, and vegetation invasion; Tunnel images included defects including lining cracks, water seepage, and surface spalling.

3. A method for urban infrastructure maintenance based on image recognition as claimed in claim 2, Features: The image recognition method is as follows: S21: collecting images of urban infrastructure containing each defect and images of urban infrastructure in normal state, and performing preprocessing; S22: Each collected urban infrastructure image is labeled by category; S23: Build image recognition model; S24: dividing the images with completed category annotation into a training set, a validation set, and a test set, and performing model training of the image recognition model through the training set; S25: Perform model verification and tuning of the image recognition model through the validation set; perform model testing, evaluation and adjustment of the image recognition model through the test set; S26: Save the trained image recognition model and deploy the application, input the preprocessed urban infrastructure image for image recognition, and filter out defective images containing defects.

4. A method for urban infrastructure maintenance based on image recognition as claimed in claim 3, Features: The defect characteristics are as follows: The defect characteristics of the bridge cracks are the length and width of the bridge cracks; the defect characteristics of the corrosion are the corrosion area; the defect characteristics of the structural deformation are the distortion angle; the defect characteristics of the concrete spalling are the spalling area and spalling depth; The defect characteristics of the road cracks are the length and width of the road cracks; the defect characteristics of the potholes are the diameter and depth of the potholes; the defect characteristics of the marking line wear are the remaining length and brightness ratio; the defect characteristics of the road water seepage are the water seepage area; The defect characteristics of the blockage are the percentage of reduction in the cross-sectional area of ​​the pipeline; the defect characteristics of the corrosion damage are the corrosion area; the defect characteristics of the dislocation fracture are the displacement and fracture length; the defect characteristics of the vegetation invasion are the vegetation area; The defect types included in the lining cracks are lining crack length and lining crack width; the defect characteristics included in the water leakage are the number of leakage points; and the defect type included in the surface spalling is the spalling area.

5. A method for urban infrastructure maintenance based on image recognition as claimed in claim 4, Features: The method for determining the risk level of urban infrastructure is as follows: S41: establishing a risk level set and a defect feature set for the defect image; S42: normalizing the defect features; S43: construct a weight set of each defect feature, denoted as W; S44: Construct a relationship matrix between defect characteristics and risk levels as follows: Select the membership function, calculate the membership degree of each defect feature for each risk level, and form the membership degree into a relationship matrix R; S45: Calculate the risk level of urban infrastructure based on the weight set and the relationship matrix; the method is as follows: Construct the risk level comprehensive matrix A, the formula is as follows: A=W·R. A is an m-dimensional row vector in the following form: A={a 1 ,a 2 ,……,a m }; Among them, a j represents the probability that the risk level of urban infrastructure is the jth level; the value range of j is 1, 2, ······, m; Determine the risk level of urban infrastructure as follows: Divide the interval (0, 1] into m segments, where the pth segment is The value range of p is 1, 2, ..., m; a q It represents the probability that the risk level of urban infrastructure is the qth level; a random number b is generated with a value range of (0, 1]; if the value of b falls within the pth interval, the risk level of urban infrastructure is the pth level.

6. A method for urban infrastructure maintenance based on image recognition as claimed in claim 5, Features: The maintenance arrangement includes calculating the inspection cycle and selecting the maintenance time; wherein the method for calculating the inspection cycle is as follows: Assuming the risk level of any urban infrastructure is level p, the calculation formula for the inspection cycle is as follows: Among them, T represents the inspection cycle, T 0 represents the regular inspection cycle; α is the adjustment factor.

7. A method for urban infrastructure maintenance based on image recognition as claimed in claim 6, Features: The method for selecting the maintenance time is as follows: Set the risk level threshold β. If the risk level of any urban infrastructure is greater than level β, select the maintenance time; determine the maintenance duration, and calculate the target factor corresponding to different maintenance start times. The formula is as follows: C=c 0 ×exp(λ×(t-t 0 ))+c t1 ×T s +c t2 ×T s ; Among them, C represents the target factor, λ represents the adjustment coefficient, t represents the time point when maintenance starts, and t 0 Indicates the current time point, T s Indicates the maintenance time; c 0 represents the risk cost factor, which indicates the potential loss cost caused by the current risk level; c t1 Represents the labor cost factor, which means that it starts at time t and lasts for T s The maintenance labor cost per unit time in the time period; c t2 represents the maintenance cost factor, which means that it starts at time t and lasts for T s The downtime maintenance cost per unit time within the time period; An optimization algorithm is used to minimize the target factor, and the start maintenance time point that minimizes the target factor is extracted as the selection result of the maintenance time.

8. An urban infrastructure maintenance system based on image recognition, which is used to implement an urban infrastructure maintenance method based on image recognition as claimed in any one of claims 1 to 7. Features: It includes image acquisition module, preprocessing module, image recognition module, feature extraction module, risk level module and maintenance module; among which: The image acquisition module is used to acquire images of urban infrastructure; The preprocessing module is used to preprocess the urban infrastructure image; The image recognition module is used to perform image recognition on the basic urban image and filter out defective images; The feature extraction module is used to extract defect features from the defect image; The risk level module determines the risk level of the urban infrastructure based on the defect characteristics; The maintenance module arranges maintenance of the urban infrastructure based on the risk level of the urban infrastructure.

Images