A subway sky-ground line integrated platform monitoring method and system based on image recognition

Abstract

The present invention relates to the field of image processing and discloses a subway sky-line integrated platform monitoring method and system based on image recognition, which is used to solve the problem that when collecting image data during subway sky-line monitoring, the monitoring area is blocked by high-speed trains. The method comprises the following steps: collecting image data of an inspection area; if there is no inspection area blocked by a running train in the inspection area, generating a three-dimensional real-scene model of the subway line; if there is an inspection area blocked by a running train, completing the image data of the blocked area and generating a three-dimensional real-scene model of the subway line, marking all reconstructed images, calculating a hazard impact factor and a confidence factor, evaluating the actual degree of damage of each inspection area, calculating a maintenance index based on the actual degree of damage, and judging whether to perform subway line maintenance, thereby effectively improving the accuracy of subway line monitoring and subway safety.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically to a method and system for monitoring a subway sky-ground line integrated platform based on image recognition. Background Art

[0002] The metro sky-ground line refers to the comprehensive structure and equipment system within the metro system, including tracks and tunnels or overhead structures. It provides the basic support and power supply for train operations, is a crucial component of metro operational safety and stability, and is a key area for maintenance and monitoring.

[0003] With the acceleration of urbanization, subways, as a core mode of urban public transportation, have become increasingly important. Their operational safety and maintenance efficiency are directly linked to the stable operation of urban transportation systems. Monitoring key components of subway lines is particularly crucial in routine maintenance. These components are subject to dynamic loads from frequent train operations, leading to potential risks such as crack expansion, deformation, and wear. Existing technologies for monitoring subway lines primarily rely on manual inspections, data collection from fixed sensors, and image acquisition using single cameras.

[0004] For example, the neural network-based subway track obstacle detection method and system disclosed in the invention patent announcement with announcement number CN116721093B includes: obtaining multiple track images under different colors of lighting; processing the multiple track images under different colors of lighting based on a graph neural network model to determine multiple track segments to be detected; obtaining surveillance videos of the multiple track segments to be detected and images of the multiple track segments to be detected under different colors of lighting; using a probability determination model based on the surveillance videos of the multiple track segments to be detected and the images of the multiple track segments to be detected under different colors of lighting to determine the probability of obstacle existence in the multiple track segments to be detected; determining the detection method for each of the multiple track segments to be detected based on the obstacle existence probability of the multiple track segments to be detected. This method can improve the efficiency of detecting track obstacles.

[0005] For example, the invention patent with publication number CN118608453A discloses a subway track contact line wear detection method based on image geometric features, which includes: detecting different wear conditions of the subway contact line to solve the problem of difficulty in detecting subway contact line wear in actual scenarios. Compared with the traditional least squares fitting, the RANSAC algorithm fitting has irreplaceable advantages in actual scenarios. It can avoid the interference of abnormal points and outliers caused by uncontrollable factors such as machine jitter during the actual sampling process, making this method more suitable for detecting subway contact lines under various wear conditions. Therefore, this method has accurate positioning, short time consumption, and does not require human participation, which can lay a good foundation for subsequent contact line wear statistics and even subway maintenance.

[0006] However, in the process of implementing the technical solutions of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In actual applications, fixed sensors cannot cover all areas, manual inspections are prone to missing hidden dangers in key areas, and during monitoring, high-speed trains may block the monitoring field of view, causing image acquisition interruption or omissions. Summary of the Invention

[0008] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method and system for monitoring a subway sky-land line integrated platform based on image recognition to solve the problems existing in the above-mentioned background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A method for monitoring a subway sky-land line integrated platform based on image recognition comprises the following steps: Step 1: arranging a camera device on a drone; setting the required inspection area in the subway line, and dynamically collecting image data of each inspection area in real time by the drone during image acquisition, and saving the image data of each inspection area into a real-time image database; Step 2: performing occlusion detection on the image data in the real-time image database, and if it is detected that there is no inspection area occluded by a running train, a three-dimensional real-scene model of the subway line is generated by a real-scene three-dimensional technology; if it is detected that there is an inspection area occluded by a running train, the inspection area occluded by the running train is recorded as an occluded area, and the inspection area not occluded by the running train is recorded as an unoccluded area; using a regional reconstruction algorithm to complete the image data of the occluded area to obtain a reconstructed image, and comparing the reconstructed image with the unoccluded area, the real-scene three-dimensional model is generated. The domain image data is used to generate a 3D real-life model of the subway line through real-life 3D technology; Step 3: If there is an inspection area blocked by a running train, all reconstructed images are marked in the generated 3D real-life model, and the blocked area data is obtained. The hazard impact factor is calculated based on the blocked area data; the image data of the blocked area is obtained, and the confidence factor is calculated based on the blocked image data; Step 4: In the 3D real-life model, the structural characteristics of each inspection area are collected, and the actual degree of damage of each inspection area is evaluated based on the structural characteristics; Step 5: The actual degree of damage of each inspection area is clustered, and a maintenance index is calculated based on the clustering results; Step 6: Whether to perform subway line maintenance is determined based on the maintenance index; Step 7: Regular drone patrols are used to capture the latest subway line image data and continuously update the 3D real-life model.

[0011] Preferably, the steps of detecting whether there is an inspection area blocked by a running train are as follows: for each inspection area, obtain the corresponding image data in the real-time database and record it as the current image; obtain the initial image of each inspection area as the background image; for each inspection area, compare the degree of change of each pixel in the current image and the background image, set a change threshold, compare the degree of pixel change with the change threshold, if the degree of pixel change is greater than or equal to the change threshold, then record the pixel as a changed pixel, if the degree of pixel change is less than the change threshold, then record the pixel as an unchanged pixel; connect the changed pixels to obtain the image change area, and obtain the image change area. The area of the image change area and the area of the background image are compared, and the occlusion degree is calculated by ratio between the area of the image change area and the area of the background image; an occlusion threshold is set, and the occlusion degree is compared with the occlusion threshold. If the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is occlusion by a running train in the inspection area; if the occlusion degree is less than the occlusion threshold, it is determined that there is no occlusion by a running train in the inspection area; the occlusion situation of each inspection area is counted. If there is no occlusion by a running train in all inspection areas, it is determined that there is no inspection area occluded by a running train; if there is occlusion by a running train in an inspection area, it is determined that there is occlusion by a running train.

[0012] Preferably, the steps for obtaining the risk impact factor are: obtaining the historical maintenance times and service life of the shielded area; obtaining the number of trains passing through the shielded area during the detection period, the average speed of each train passing through the shielded area, and the time each train passes through the shielded area; calculating the load of a single train based on the average speed of each train passing through the shielded area and the time each train passes through the shielded area; summing the loads of the trains passing through the shielded area during the detection period to obtain the total operating load, with areas with high train traffic density bearing a greater load; normalizing the historical maintenance times, service life, and total operating load, and evaluating the risk impact factor based on the normalized historical maintenance times, service life, and total operating load. The specific acquisition method is: Where, Expressed as the risk impact factor, Represents the number of historical maintenance times, Indicated as the service life, Expressed as total operating load.

[0013] Preferably, the confidence factor acquisition step comprises: performing Laplace transform on each pixel in the occluded area image using Laplace transform to obtain a Laplace transform value, calculating the Laplace transform mean of the occluded area image, and evaluating the image blur degree based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are: Where, Represents the degree of image blur, Represented as the Laplace transform value of the i-th pixel, Expressed as the Laplace transform mean, is the total number of pixels; the pixel difference between adjacent frames in the occluded area image is calculated to obtain the frame difference; the confidence factor is obtained based on the image blur level, temporal consistency, and occlusion level evaluation.

[0014] Preferably, the steps for obtaining the actual damage degree of each inspection area are as follows: in the three-dimensional real-scene model, for the unobstructed areas in the inspection area, the structural characteristics of the unobstructed areas are obtained, and the actual damage degree of the unobstructed areas is obtained according to the structural characteristics evaluation; for the obstructed areas in the inspection area, the structural characteristics of the obstructed areas are obtained, and the initial damage degree is obtained according to the structural characteristics evaluation, and the initial damage degree of the obstructed areas is obtained in the same manner as the actual damage degree of the unobstructed areas, and the initial damage degree, the hazard impact factor and the confidence factor are multiplied to obtain the actual damage degree of the obstructed areas; the actual damage degree of the unobstructed areas is integrated with the actual damage degree of the obstructed areas to obtain the actual damage degree of each inspection area.

[0015] Preferably, the steps for obtaining the actual degree of damage in the unobstructed area are as follows: obtaining crack data of the detection component in the unobstructed area by image processing technology, the crack data including the total number of cracks, crack width and crack length, and obtaining a crack influence coefficient based on the crack data; obtaining the initial three-dimensional coordinates of each detection point of the detection component in the unobstructed area, obtaining the real-time three-dimensional coordinates of each detection point of the detection component through the three-dimensional real scene model, calculating the offset of each detection point using the Euclidean distance, calculating the average offset of all detection points, and recording it as the component offset influence coefficient; extracting the wear area and the detection component area of the detection component in the unobstructed area by the edge detection algorithm, obtaining the area of the wear area and the area of the detection component area, and calculating the wear degree by ratio of the area of the wear area to the area of the detection component area; normalizing the crack influence coefficient, the component offset influence coefficient and the wear degree, and evaluating the actual degree of damage in the unobstructed area based on the normalized crack influence coefficient, the component offset influence coefficient and the wear degree. The specific acquisition steps are as follows: Where, Indicates the actual degree of damage. Expressed as the crack influence coefficient, Expressed as the component offset influence coefficient, Indicated as the degree of wear, 、 、 It is expressed as the crack influence coefficient, component displacement influence coefficient and wear degree weight coefficient.

[0016] Preferably, the steps for clustering the actual damage degree of each inspection area are as follows: Step 5.1: taking the actual damage degree as the clustering feature, taking the actual damage degree of all inspection areas as the data set, and each actual damage degree in the data set as a data point; Step 5.2: using the silhouette coefficient method to determine the optimal clustering number K of the data set; Step 5.3: randomly selecting K data points in the data set as initial cluster centers, for each data point, calculating its Euclidean distance to each initial cluster center, for each data point, traversing the K initial cluster centers, and assigning it to the cluster cluster corresponding to the nearest initial cluster center; Step 5.4: after traversing all data points, obtaining the initial cluster cluster, for each initial cluster cluster, calculating the mean of the data points in it to obtain a new cluster center; Step 5.5: repeating steps 5.3 and 5.4 until the cluster center no longer changes, and obtaining the final cluster cluster and the final cluster center.

[0017] Preferably, the step of calculating the maintenance index based on the clustering processing result is: calculating the ratio of the number of data points in each final cluster to the total number in the data set to obtain the weight of each final cluster; and performing weighted summation of the weight of each final cluster and the final cluster center to obtain the maintenance index.

[0018] Preferably, the step of judging whether to perform subway line maintenance based on the maintenance index is: setting a maintenance threshold, comparing the maintenance index with the maintenance threshold; if the maintenance index is less than the maintenance threshold, it is judged that the current subway line is in good condition and the subway line maintenance is not performed; if the maintenance index is greater than or equal to the maintenance threshold, it is judged that the current subway line is in poor condition and the subway line maintenance is performed.

[0019] Preferably, a subway sky-land line integrated platform monitoring system based on image recognition comprises: an image acquisition module for collecting image data of each inspection area in the subway line through a drone, and transmitting the image data of each inspection area to an occlusion detection module and a three-dimensional real scene model generation module; an occlusion detection module for detecting, based on the image data of each inspection area, whether there is an inspection area occluded by a running train, and if there is an inspection area occluded by a running train, transmitting the image data to an image reconstruction module; if there is no inspection area occluded by a running train, transmitting the image data to a three-dimensional real scene model generation module; an image reconstruction module for reconstructing the occluded image of the inspection area to obtain a reconstructed image, and transmitting the reconstructed image to the three-dimensional real scene model generation module; a three-dimensional real scene model generation module for generating a three-dimensional real scene model through real-scene three-dimensional technology, and transmitting the three-dimensional real scene model to a maintenance judgment module; a maintenance judgment module for judging whether maintenance is required based on the generated three-dimensional real scene model; a model update module for acquiring subway line image data in real time and continuously updating the three-dimensional real scene model.

[0020] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0021] 1. Define the required inspection areas along the subway line and use drones to dynamically capture image data for each inspection area in real time during image acquisition. This effectively addresses the issues of fixed sensors failing to cover all areas and manual inspections often missing critical areas. Drones, with their high flexibility and wide coverage, can quickly reach complex, dangerous, or hard-to-reach areas, ensuring comprehensive coverage of all checkpoints. Furthermore, compared to manual inspections, drones significantly reduce the limitations of manual operations and the risk of missing critical areas due to fatigue, subjective judgment, or limited field of view.

[0022] 2. Using real-world 3D technology, a 3D realistic model of the subway line is generated. This model faithfully reproduces the spatial information of the subway line and its surrounding environment, providing an intuitive and detailed data foundation for precise positioning and comprehensive analysis. Compared to traditional 2D floor plans, the 3D model more clearly depicts the complex structure of the line, track configuration, and the spatial distribution of key components, facilitating the identification of potential hazards.

[0023] 3. Use a regional reconstruction algorithm to complete the image data of the occluded area to obtain a reconstructed image. This algorithm not only significantly improves the accuracy and continuity of monitoring, but also reduces the omission of hidden dangers caused by missing data, thereby improving the reliability of hidden danger identification and fault detection. At the same time, it can reduce the workload of manual data collection, saving time and costs, and providing technical support for the efficient operation and maintenance and safety monitoring of subway lines.

[0024] 4. Regular drone patrols capture the latest subway line imagery and continuously update the 3D reality model. Drone patrols efficiently and comprehensively cover the entire line, acquiring the latest high-resolution imagery data for timely updating of the 3D reality model, dynamically reflecting the actual status of the line and its surroundings. Compared to traditional static models, dynamically updated 3D models can more accurately identify changes and potential risks in the subway line, such as track deviation, equipment wear, and environmental erosion. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flow chart of a subway sky-land line integrated platform monitoring method based on image recognition provided in an embodiment of the present application.

[0026] Figure 2 This is a structural diagram of a subway sky-land line integrated platform monitoring system based on image recognition provided in an embodiment of the present application.

[0027] Figure 3 Schematic diagram of the load of each train in the embodiment of this application. DETAILED DESCRIPTION

[0028] The technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. In addition, the forms of the various structures described in the following embodiments are merely examples. The subway sky-land line integrated platform monitoring method and system based on image recognition involved in the present invention are not limited to the various structures described in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0029] The present invention provides a method for monitoring a subway sky-ground line integrated platform based on image recognition, such as Figure 1 As shown, the following steps are included:

[0030] Step 1: Install a camera on the drone to collect image data. Set the required inspection areas on the subway line. During the image acquisition period, the drone dynamically collects image data for each inspection area in real time. Each image data is marked with a corresponding timestamp and the corresponding inspection area. The image data of each inspection area is saved in a real-time image database.

[0031] Step 2: Use occlusion detection to check the image data in the real-time image database to see if there is any inspection area blocked by a running train. If no inspection area is blocked by a running train, a 3D real-life model of the subway line is generated using real-life 3D technology.

[0032] If the inspection area is blocked by a running train, the inspection area blocked by the running train is recorded as the blocked area, and the inspection area not blocked by the running train is recorded as the unblocked area;

[0033] A regional reconstruction algorithm is used to complete the image data of the occluded area to obtain a reconstructed image. Based on the reconstructed image and the image data of the unoccluded area, a 3D real-scene model of the subway line is generated using real-scene 3D technology;

[0034] Occlusion detection uses computer vision technology to analyze monitoring images or videos to determine whether a target area is obscured by a moving train or other dynamic objects. This technology is used to identify dynamic changes in obscured areas. In integrated platform monitoring of subway sky and ground lines, this technology effectively detects whether an inspection area is obscured by a passing train, providing foundational support for subsequent monitoring.

[0035] Regional reconstruction algorithms use known data to infer and complete images of occluded or missing areas. Based on the spatial characteristics of the image, time series information, or multi-view data, these algorithms use interpolation methods, texture-based expansion, or deep learning models to generate approximate images of missing areas. In occluded areas, the image is completed by combining the characteristics of surrounding pixels and historical information, resulting in a complete and as realistic a reconstructed image as possible.

[0036] Realistic 3D technology is a modeling technology based on multi-source data. Through high-precision image acquisition and processing, it transforms the real-world physical environment into a highly realistic 3D digital model. This technology utilizes algorithms such as image matching, point cloud data processing, and texture mapping to accurately reproduce terrain, buildings, and infrastructure. It is widely used in urban planning, engineering design, and smart transportation. In subway line modeling, Realistic 3D technology can generate a comprehensive 3D model that includes the surrounding environment, topography, and station facilities, providing powerful support for project visualization and management.

[0037] In this embodiment, it should be specifically explained that the steps for detecting whether there is an inspection area blocked by a running train are:

[0038] For each inspection area, obtain the corresponding image data in the real-time database and record it as the current image;

[0039] Obtain the initial image of each inspection area as the background image. The initial image is the image of the inspection area when it is just built and serves as the benchmark for the unobstructed scene.

[0040] For each inspection area, compare the degree of change of each pixel in the current image and the background image, set a change threshold, and compare the pixel change degree with the change threshold. If the pixel change degree is greater than or equal to the change threshold, the pixel is recorded as a changed pixel. If the pixel change degree is less than the change threshold, the pixel is recorded as an unchanged pixel. The change degree is the absolute value of the color value difference at each pixel position;

[0041] Connect the changed pixels to obtain the image change region, obtain the area of the image change region and the area of the background image, and calculate the ratio of the area of the image change region to the area of the background image to obtain the occlusion degree;

[0042] Set an occlusion threshold and compare the occlusion degree with the occlusion threshold. If the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is a running train blocking the inspection area; if the occlusion degree is less than the occlusion threshold, it is determined that there is no running train blocking the inspection area.

[0043] The occlusion situation of each inspection area is counted. If there is no occlusion by running trains in all inspection areas, it is determined to be an inspection area without occlusion by running trains; if there is occlusion by running trains in one or more inspection areas, it is determined to be an inspection area with occlusion by running trains.

[0044] In this embodiment, it should be specifically explained that the steps of using the regional reconstruction algorithm to complete the image data of the occluded area to obtain the reconstructed image are:

[0045] Extract pixel values in the current image except the image change area as known pixel values, and mark the pixels in the image change area as data that needs to be reconstructed;

[0046] The interpolation method uses the known pixel values at the edge of the image change area to fill the image change area through mathematical interpolation to obtain the reconstructed image.

[0047] Interpolation is an image reconstruction method that uses known pixel values at the edges of changing regions of an image to mathematically infer unknown pixel values within occluded or changing regions. Interpolation analyzes the spatial distribution of known pixel values to estimate reasonable values for unknown pixels. Common methods include bilinear interpolation and bicubic interpolation.

[0048] In this embodiment, it should be specifically explained that the steps of generating a 3D real scene model of a subway line using real scene 3D technology are as follows:

[0049] Use LiDAR to scan subway lines and obtain high-precision 3D point cloud data. Perform noise removal, filtering, and coordinate registration on the point cloud data to ensure data accuracy.

[0050] Combined with the image data in the real-time image database, the images are aligned using image registration technology, and the scanned point cloud data are merged into an initial model using point cloud stitching technology;

[0051] Use structured light technology to accurately scan the details of the initial model and extract the surface shape of the initial model; use optical feature extraction algorithms such as edge detection and texture mapping to supplement the surface texture and detail information of the initial model;

[0052] The point cloud data is converted into a 3D mesh model, and the image data is applied to the model surface using texture mapping to enhance the details and realism of the initial model to obtain a 3D real scene model.

[0053] Image registration is the process of spatially aligning images taken from different viewpoints, sensors, or at different times. The core of this process is to extract image feature points (such as corners, edges, or texture features) and use a matching algorithm to match these feature points across images. This process then calculates a transformation matrix to achieve geometric alignment between images.

[0054] Point cloud stitching technology combines point cloud data from different scanning locations or collected at different times into a complete 3D point cloud model. The core of this technology is to match and register the overlapping areas between different point cloud datasets, calculate the spatial transformation relationship between them, and then seamlessly stitch together multiple point cloud data sets.

[0055] Structured light technology projects a known light stripe or pattern onto an object's surface and uses a camera to capture the deformation of these stripes to determine the object's three-dimensional shape. The deformation of the light stripe on the surface is directly related to the object's geometry. By analyzing these deformations, the depth and shape of the object's surface can be accurately calculated.

[0056] Step 3: If there is an inspection area blocked by a running train, mark all reconstructed images in the generated 3D reality model, obtain the blocked area data, calculate the hazard impact factor based on the blocked area data, obtain the blocked area image data from the real-time image database, and calculate the confidence factor based on the blocked image data;

[0057] In this embodiment, it should be specifically explained that the steps for obtaining the risk impact factor are:

[0058] Obtain the historical maintenance times of the blocked area. The areas with higher maintenance times are more likely to have hidden dangers.

[0059] Get the service life of the obstructed area. The service life is the time from the construction of the obstructed area to the present. Areas with longer service life are more likely to have aging problems.

[0060] Obtain the number of trains passing through the obscured area during the detection period, the average speed of each train passing through the obscured area, and the time each train takes to pass through the obscured area;

[0061] like Figure 3 As shown in Figure 2, the load of a single train is calculated based on the average speed of each train passing through the blocked area and the time each train passes through the blocked area. The specific steps for obtaining the load are as follows:

[0062] ;

[0063] Where, Expressed as train load, Expressed as the average speed, Expressed as passing time;

[0064] Table 1 Train load

[0065]

[0066] As shown in Table 1, in one specific embodiment, by analyzing the impact of a train's average speed and transit time on train load, it is possible to identify differences in load distribution among different trains when passing through an obstructed area. For example, a higher average speed and a shorter transit time may result in a smaller train load, while a lower average speed and a longer transit time may increase the train load. These load changes reflect the operating pressure and power consumption characteristics experienced by the train when passing through an obstructed area. This analysis can be further applied to track operation monitoring. By capturing abnormalities in load distribution, potential risks to track or train operation can be assessed.

[0067] The total operating load is calculated by summing the loads of trains passing through the blocked area during the detection period. Areas with high train traffic density bear greater loads.

[0068] Normalize the historical maintenance times, service life, and total operating load, and evaluate the hazard impact factor based on the normalized historical maintenance times, service life, and total operating load. The specific method of obtaining the hazard impact factor is as follows:

[0069] ;

[0070] Where, Expressed as the risk impact factor, Represents the number of historical maintenance times, Indicated as the service life, Expressed as total operating load.

[0071] In this embodiment, it should be specifically explained that the steps for obtaining the confidence factor are:

[0072] Use Laplace transform to perform Laplace transform on each pixel in the occluded area image to obtain the Laplace transform value, calculate the Laplace transform mean of the occluded area image, and evaluate the blur degree of the image based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are as follows:

[0073] ;

[0074] Where, Represents the degree of image blur, Represented as the Laplace transform value of the i-th pixel, Expressed as the Laplace transform mean, is the total number of pixels;

[0075] Calculate the pixel difference between adjacent frames in the occluded area image to obtain the frame difference, and normalize the frame difference to obtain the temporal consistency. The closer the temporal consistency is to 1, the more consistent the temporal sequence is.

[0076] The confidence factor is obtained based on the image blur, temporal consistency, and occlusion level. The specific steps are as follows:

[0077] ;

[0078] Where, Expressed as confidence factor, Represents the degree of image blur, It is expressed as the degree of occlusion, Indicated as temporal consistency.

[0079] In image processing, the Laplace transform typically refers to convolution of an image using the Laplace operator, with the goal of detecting edges or regions of change within the image. The Laplace operator is a second-order differential operator that can highlight image details, particularly edges or contours, by calculating the grayscale changes in the area surrounding each pixel. In image processing of occluded areas, the Laplace transform helps reveal edge features of the occluded area, providing information about the occluded region and facilitating subsequent image restoration or reconstruction.

[0080] Step 4: In the 3D real-world model, collect the structural characteristics of each inspection area and evaluate the actual damage extent of each inspection area based on the structural characteristics;

[0081] In this embodiment, it should be specifically explained that the steps for obtaining the actual damage extent of each inspection area are as follows:

[0082] In the 3D real-scene model, for the unobstructed areas in the inspection area, the structural characteristics of the unobstructed areas are obtained, and the actual damage degree of the unobstructed areas is evaluated based on the structural characteristics;

[0083] For the occluded areas in the inspection area, the structural characteristics of the occluded areas are obtained. The initial damage degree of the occluded areas is evaluated based on the structural characteristics. The initial damage degree of the occluded areas is obtained in the same way as the actual damage degree of the unobstructed areas. The actual damage degree of the occluded areas is calculated by multiplying the initial damage degree, the risk impact factor, and the confidence factor. The confidence factor represents the reliability of the detection results of the occluded areas. It combines factors such as image quality, time sequence consistency, and occlusion degree to determine the confidence in the damage of the occluded areas.

[0084] The actual damage degree of the unobstructed area is integrated with the actual damage degree of the obstructed area to obtain the actual damage degree of each inspection area.

[0085] In this embodiment, it should be specifically explained that the steps for obtaining the actual damage degree of the unblocked area are:

[0086] The crack data of the detected components in the unobstructed area are obtained through image processing technology. The crack data includes the total number of cracks, crack width, and crack length. The crack influence coefficient is obtained based on the crack data. The specific acquisition method is as follows:

[0087] ;

[0088] Where, Expressed as the crack influence coefficient, is the crack length of the j-th crack, It is expressed as the crack width of the jth crack;

[0089] Obtain the initial 3D coordinates of each detection point of the detection component in the unobstructed area, obtain the real-time 3D coordinates of each detection point of the detection component through the 3D real scene model, calculate the offset of each detection point using Euclidean distance, and calculate the average offset of all detection points, which is recorded as the component offset influence coefficient;

[0090] The wear area and the detection component area of the detection component in the unblocked area are extracted by an edge detection algorithm, the area of the wear area and the area of the detection component area are obtained, and the wear degree is calculated by calculating the ratio of the wear area area to the area of the detection component area;

[0091] The crack influence coefficient, component offset influence coefficient, and wear degree are normalized, and the actual damage degree of the unobstructed area is evaluated based on the normalized crack influence coefficient, component offset influence coefficient, and wear degree. The specific acquisition steps are as follows:

[0092] ;

[0093] Where, Indicates the actual degree of damage. Expressed as the crack influence coefficient, the presence of cracks in a component is a key indicator of damage severity. As cracks expand, the overall strength and stability of the material gradually decrease, potentially leading to further damage or failure. Therefore, an increase in the crack influence coefficient generally indicates that the damage to the structure has worsened, indicating that the part may have or will soon lose its load-bearing capacity, thus affecting the safety and reliability of the entire system. Expressed as the component offset influence coefficient, the degree of offset that occurs in a component during use directly reflects the severity of its damage. When a component deflects or deforms during operation, it is usually due to long-term wear, stress concentration, fatigue and other factors. These changes will cause the component's geometry to deviate, thereby affecting its function and performance. As the offset increases, the load distribution, stability and accuracy of the component will be affected, which may lead to further damage or failure of the component. By monitoring and calculating the component offset influence coefficient, potential damage areas can be identified in a timely manner, and by predicting and analyzing the offset trend, the remaining life of the component can be evaluated, providing a scientific basis for maintenance and replacement decisions, and avoiding safety hazards caused by excessive wear and accidental failure. Expressed as the degree of wear, wear generally refers to the gradual peeling or change of surface materials due to friction, contact and long-term use. This phenomenon will directly affect the performance, shape and function of the material. When the wear exceeds a certain critical value, the load-bearing capacity, stability and durability of the component will be significantly reduced, leading to more serious damage. When the wear reaches a certain level, the damage to the structure may rapidly increase, thereby affecting the function and safety of the entire system. 、 、 Expressed as the crack influence coefficient, component displacement influence coefficient and wear degree weight coefficient, and , 、 、 The specific value is determined by professionals according to the actual situation, for example, 、 、 It can be 0.3, 0.2, or 0.5.

[0094] Step 5: Cluster the actual damage degree of each inspection area and calculate the repair index based on the clustering results;

[0095] In this embodiment, it should be specifically explained that the steps of clustering the actual damage degree of each inspection area are as follows:

[0096] Step 5.1: Use the actual damage extent as the clustering feature, and the actual damage extent of all inspected areas as the dataset, with each actual damage extent in the dataset being a data point;

[0097] Step 5.2: Use the silhouette coefficient method to determine the optimal number of clusters K for the data set;

[0098] The Silhouette Coefficient method measures the quality of clustering results by evaluating the relative relationship between the compactness of each data point within its cluster and its distance to the nearest other cluster. The optimal number of clusters is determined by calculating the average Silhouette Coefficient for different numbers of clusters K and finding the K that gives the maximum value. The core idea of this method is to balance intra-cluster compactness with inter-cluster separation to find the most reasonable cluster division.

[0099] Step 5.3: Randomly select K data points in the data set as the initial cluster centers. For each data point, calculate the Euclidean distance from each initial cluster center. The specific method is to obtain ,in Expressed as the Euclidean distance from the data point to the cluster center, where Represented as data points, Represented as the initial cluster center, for each data point, traverse the K initial cluster centers and assign it to the cluster corresponding to the nearest initial cluster center;

[0100] Euclidean distance is a measure of the straight-line distance between two points and is the most commonly used distance calculation method. In K-means clustering, Euclidean distance is used to assess the similarity between a data point and the cluster center. A smaller distance indicates a data point is closer to the cluster center and therefore more likely to belong to that cluster. This distance is intuitive and easy to calculate, making it suitable for clustering numerical data.

[0101] Step 5.4: After traversing all data points, we get the initial clusters. For each initial cluster, we calculate the mean of the data points in it and get the new cluster center.

[0102] Step 5.5: Repeat steps 5.3 and 5.4 until the cluster center no longer changes, and obtain the final cluster and the final cluster center.

[0103] In this embodiment, it should be specifically explained that the steps for calculating the maintenance index according to the clustering processing results are:

[0104] The weight of each final cluster is calculated by calculating the ratio of the number of data points in each final cluster to the total number of data points in the data set;

[0105] The maintenance index is obtained by adding the weight of each final cluster to the final cluster center.

[0106] Step 6: Determine whether to carry out subway line maintenance based on the maintenance index;

[0107] In this embodiment, it should be specifically explained that the steps for determining whether to perform subway line maintenance based on the maintenance index are as follows:

[0108] A maintenance threshold is set and the maintenance index is compared with the maintenance threshold. If the maintenance index is less than the maintenance threshold, it is judged that the current subway line is in good condition and no subway line maintenance is performed; if the maintenance index is greater than or equal to the maintenance threshold, it is judged that the current subway line is in poor condition and subway line maintenance is performed.

[0109] Step 7: Use drones to regularly patrol the area to capture the latest subway line image data and continuously update the 3D reality model.

[0110] In this embodiment, it should be specifically explained that, Figure 2 As shown, a subway sky-ground line integrated platform monitoring system based on image recognition, the system includes:

[0111] An image acquisition module is used to collect image data of each inspection area in the subway line through a drone, and transmit the image data of each inspection area to the occlusion detection module and the 3D real scene model generation module;

[0112] An occlusion detection module is used to detect, based on the image data of each inspection area, whether there is an inspection area occluded by a running train. If there is an inspection area occluded by a running train, the image data is transmitted to the image reconstruction module; if there is no inspection area occluded by a running train, the image data is transmitted to the 3D real scene model generation module;

[0113] An image reconstruction module is used to reconstruct the occluded image of the inspection area to obtain a reconstructed image, and transmit the reconstructed image to the 3D real scene model generation module;

[0114] A three-dimensional real scene model generation module is used to generate a three-dimensional real scene model using real scene three-dimensional technology and transmit the three-dimensional real scene model to the maintenance judgment module;

[0115] A maintenance judgment module is used to judge whether maintenance is needed based on the generated three-dimensional real scene model;

[0116] The model update module is used to obtain subway line image data in real time and continuously update the three-dimensional real-scene model.

[0117] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0118] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for monitoring a subway sky-ground line integrated platform based on image recognition, characterized in that: The following steps are involved: Step 1: Set up the camera on the drone; Set the required inspection areas on the subway line, and use drones to dynamically collect image data of each inspection area in real time during image acquisition, and save the image data of each inspection area into the real-time image database; Step 2: Perform occlusion detection on the image data in the real-time image database. If no inspection area is blocked by running trains, a 3D real-life model of the subway line is generated using real-life 3D technology. If the inspection area is blocked by a running train, the inspection area blocked by the running train is recorded as the blocked area, and the inspection area not blocked by the running train is recorded as the unblocked area; A regional reconstruction algorithm is used to complete the image data of the occluded area to obtain a reconstructed image. Based on the reconstructed image and the image data of the unoccluded area, a 3D real-scene model of the subway line is generated using real-scene 3D technology; Step 3: If there is an inspection area blocked by a running train, mark all reconstructed images in the generated 3D reality model, obtain the blocked area data, and calculate the hazard impact factor based on the blocked area data; Obtaining image data of the occluded area, and calculating a confidence factor based on the occluded image data; The steps for obtaining the risk impact factor are: Obtain the historical maintenance times and service life of the obstructed area; Obtain the number of trains passing through the obscured area during the detection period, the average speed of each train passing through the obscured area, and the time each train takes to pass through the obscured area; The load of a single train is calculated based on the average speed of each train passing through the obstruction area and the time each train spends passing through the obstruction area; The total operating load is calculated by summing the loads of trains passing through the blocked area during the detection period. Areas with high train traffic density bear greater loads. Normalize the historical maintenance times, service life, and total operating load, and evaluate the hazard impact factor based on the normalized historical maintenance times, service life, and total operating load. The specific method of obtaining the hazard impact factor is as follows: ； Where, Expressed as the risk impact factor, Represents the number of historical maintenance times, Indicated as the service life, Expressed as total operating load; Step 4: In the 3D real-world model, collect the structural characteristics of each inspection area and evaluate the actual damage extent of each inspection area based on the structural characteristics; The steps for obtaining the actual damage extent of each inspection area are as follows: In the 3D real-scene model, for the unobstructed areas in the inspection area, the structural characteristics of the unobstructed areas are obtained, and the actual damage degree of the unobstructed areas is evaluated based on the structural characteristics; For the obstructed areas in the inspection area, the structural characteristics of the obstructed areas are obtained, and the initial damage degree is evaluated based on the structural characteristics. The initial damage degree of the obstructed areas is obtained in the same way as the actual damage degree of the unobstructed areas. The actual damage degree of the obstructed areas is calculated by multiplying the initial damage degree, the hazard impact factor, and the confidence factor. The actual damage degree of each inspection area is obtained by integrating the actual damage degree of the unobstructed area with the actual damage degree of the obstructed area; Step 5: Cluster the actual damage degree of each inspection area and calculate the repair index based on the clustering results; Step 6: Determine whether to carry out subway line maintenance based on the maintenance index; Step 7: Use drones to regularly patrol the area to capture the latest subway line image data and continuously update the 3D reality model.

2. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of detecting whether there is an inspection area blocked by a running train are as follows: For each inspection area, obtain the corresponding image data in the real-time database and record it as the current image; Obtaining an initial image of each inspection area as a background image; For each inspection area, compare the degree of change of each pixel in the current image and the background image, set a change threshold, and compare the pixel change degree with the change threshold. If the pixel change degree is greater than or equal to the change threshold, the pixel is recorded as a changed pixel. If the pixel change degree is less than the change threshold, the pixel is recorded as an unchanged pixel. Connect the changed pixels to obtain the image change region, obtain the area of the image change region and the area of the background image, and calculate the ratio of the area of the image change region to the area of the background image to obtain the occlusion degree; Set an occlusion threshold and compare the occlusion degree with the occlusion threshold. If the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is a running train blocking the inspection area. If the occlusion degree is less than the occlusion threshold, it is determined that there is no running train occlusion in the inspection area; The occlusion status of each inspection area is counted. If there is no running train occlusion in all inspection areas, it is determined that there is no running train occlusion in the inspection area. If there is an inspection area blocked by a running train, it is determined that there is an inspection area blocked by a running train.

3. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The confidence factor acquisition steps are: Use Laplace transform to perform Laplace transform on each pixel in the occluded area image to obtain the Laplace transform value, calculate the Laplace transform mean of the occluded area image, and evaluate the blur degree of the image based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are as follows: ； Where, Represents the degree of image blur, ) represents the Laplace transform value of the i-th pixel, Expressed as the Laplace transform mean, is the total number of pixels; Calculate the pixel difference between adjacent frames in the occluded area image to obtain the frame difference; The confidence factor is obtained based on the image blur, temporal consistency and occlusion level.

4. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps for obtaining the actual damage degree of the unblocked area are: The crack data of the inspection component in the unobstructed area is obtained by image processing technology. The crack data includes the total number of cracks, crack width and crack length. The crack influence coefficient is obtained based on the crack data. Obtain the initial 3D coordinates of each detection point of the detection component in the unobstructed area, obtain the real-time 3D coordinates of each detection point of the detection component through the 3D real scene model, calculate the offset of each detection point using Euclidean distance, and calculate the average offset of all detection points, which is recorded as the component offset influence coefficient; The wear area and the detection component area of the detection component in the unblocked area are extracted by an edge detection algorithm, the area of the wear area and the area of the detection component area are obtained, and the wear degree is calculated by calculating the ratio of the wear area area to the area of the detection component area; The crack influence coefficient, component offset influence coefficient, and wear degree are normalized, and the actual damage degree of the unobstructed area is evaluated based on the normalized crack influence coefficient, component offset influence coefficient, and wear degree. The specific acquisition steps are as follows: ； Where, Indicates the actual degree of damage. Expressed as the crack influence coefficient, Expressed as the component offset influence coefficient, Indicated as the degree of wear, 、 、 It is expressed as the crack influence coefficient, component displacement influence coefficient and wear degree weight coefficient.

5. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of clustering the actual damage degree of each inspection area are as follows: Step 5.1: Use the actual damage extent as the clustering feature, and the actual damage extent of all inspected areas as the dataset, with each actual damage extent in the dataset being a data point; Step 5.2: Use the silhouette coefficient method to determine the optimal number of clusters K for the data set; Step 5.3: Randomly select K data points in the data set as initial cluster centers. For each data point, calculate its Euclidean distance to each initial cluster center. For each data point, traverse the K initial cluster centers and assign it to the cluster corresponding to the initial cluster center closest to it. Step 5.4: After traversing all data points, we get the initial clusters. For each initial cluster, we calculate the mean of the data points in it and get the new cluster center. Step 5.5: Repeat steps 5.3 and 5.4 until the cluster center no longer changes, and obtain the final cluster and the final cluster center.

6. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of calculating the maintenance index according to the clustering processing results are: The weight of each final cluster is calculated by calculating the ratio of the number of data points in each final cluster to the total number of data points in the data set; The maintenance index is obtained by adding the weight of each final cluster to the final cluster center.

7. The method for monitoring a subway sky-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of determining whether to perform subway line maintenance based on the maintenance index are as follows: A maintenance threshold is set and the maintenance index is compared with the maintenance threshold. If the maintenance index is less than the maintenance threshold, the current subway line is judged to be in good condition and no subway line maintenance is performed. If the maintenance index is greater than or equal to the maintenance threshold, the current subway line is judged to be in poor condition and subway line maintenance is performed.

8. A subway ground-line integrated platform monitoring system based on image recognition, used to implement the subway ground-line integrated platform monitoring method based on image recognition according to any one of claims 1 to 7, characterized in that: The system comprises: An image acquisition module is used to collect image data of each inspection area in the subway line through a drone, and transmit the image data of each inspection area to the occlusion detection module and the 3D real scene model generation module; An occlusion detection module is used to detect, based on the image data of each inspection area, whether there is an inspection area occluded by a running train. If there is an inspection area occluded by a running train, the image data is transmitted to the image reconstruction module; if there is no inspection area occluded by a running train, the image data is transmitted to the 3D real scene model generation module; An image reconstruction module is used to reconstruct the occluded image of the inspection area to obtain a reconstructed image, and transmit the reconstructed image to the 3D real scene model generation module; A three-dimensional real scene model generation module is used to generate a three-dimensional real scene model using real scene three-dimensional technology and transmit the three-dimensional real scene model to the maintenance judgment module; A maintenance judgment module is used to judge whether maintenance is needed based on the generated three-dimensional real scene model; The model update module is used to obtain subway line image data in real time and continuously update the three-dimensional real-scene model.

Images