Subway heaven and earth wire integrated platform monitoring method and system based on image recognition

Abstract

The invention relates to the field of image processing, and discloses a subway heaven and earth wire integrated platform monitoring method and system based on image recognition, which are used for solving the problem that the view of a monitoring area is shielded by a train running at a high speed when image data are collected during subway heaven and earth wire monitoring. If the inspection area shielded by the running train does not exist in the inspection area, generating a three-dimensional live-action model of the subway line; if the inspection area shielded by the running train exists, complementing the image data of the shielded area, generating a three-dimensional live-action model of the subway line, marking all reconstructed images, calculating to obtain a dangerous influence factor and a confidence factor, and evaluating each inspection area to obtain an actual damage degree; the maintenance index is calculated according to the actual damage degree, whether subway line maintenance is carried out or not is judged, and the accuracy of subway line monitoring and subway safety are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically to a subway ground-to-ground line integrated platform monitoring method and system based on image recognition. Background Art

[0002] The subway sky-ground line refers to the comprehensive structure and equipment system involving tracks and tunnels or the top of elevated structures in the subway line system. It covers the basic support and power supply guarantee for train operation, is an important part of the safety and stability of subway operations, and is also a key area for maintenance and monitoring.

[0003] With the acceleration of urbanization, subways are one of the core modes of urban public transportation. Their operational safety and maintenance efficiency are directly related to the stable operation of the urban transportation system. In the daily maintenance of subway lines, the monitoring of key components of subway ground lines is particularly important. These components are subject to dynamic loads brought by high-frequency train operation, leading to hidden dangers such as crack expansion, deformation, and wear. In the existing technology, the monitoring of subway ground lines mainly relies on manual inspections, fixed sensor data collection, and image acquisition by a single camera device.

[0004] For example, the neural network-based subway track obstacle detection method and system announced in the invention patent with announcement number: CN116721093B includes: obtaining multiple track pictures under different color lighting; processing the multiple track pictures under different color lighting based on the graph neural network model to determine multiple track segments to be detected; obtaining monitoring videos of multiple track segments to be detected and pictures of multiple track segments to be detected under different color lighting; using a probability determination model based on the monitoring videos of multiple track segments to be detected and the pictures of multiple track segments to be detected under different color lighting to determine the probability of obstacle existence in multiple track segments to be detected; determining the detection method of each track segment to be detected in the multiple track segments to be detected based on the probability of obstacle existence in the multiple track segments to be detected. This method can improve the detection efficiency of track obstacles.

[0005] For example, the invention patent with publication number: CN118608453A discloses a subway track contact line wear detection method based on image geometric features, including: detecting different wear conditions of the subway contact line to solve the problem of difficulty in subway contact line wear detection in actual scenarios. Compared with the traditional least squares fitting, the RANSAC algorithm fitting has irreplaceable advantages in actual scenarios. It can avoid the interference of abnormal points and outliers caused by uncontrollable factors such as machine jitter during the actual sampling process, making this method more suitable for the detection of subway contact lines under various wear conditions. Therefore, this method has accurate positioning, short time consumption, and no human participation, which can lay a good foundation for subsequent contact line wear statistics and even subway maintenance.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems: In actual applications, fixed sensors cannot cover all areas, manual inspections are prone to miss hidden dangers in key areas, and during monitoring, high-speed trains may block the monitoring field of view, causing image acquisition interruptions or omissions. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method and system for monitoring a subway sky-ground line integrated platform based on image recognition to solve the problems existing in the above-mentioned background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions: A subway sky-ground line integrated platform monitoring method based on image recognition comprises the following steps: step 1: arranging a camera device on a drone; setting the required inspection area in the subway line, and dynamically collecting image data of each inspection area in real time by using the drone during image acquisition, and saving the image data of each inspection area into a real-time image database; step 2: performing occlusion detection on the image data in the real-time image database, and if it is detected that there is no inspection area occluded by a running train, a three-dimensional real-scene model of the subway line is generated by using a real-scene three-dimensional technology; if it is detected that there is an inspection area occluded by a running train, the inspection area occluded by the running train is recorded as an occluded area, and the inspection area not occluded by the running train is recorded as an unoccluded area; using a regional reconstruction algorithm to complete the image data of the occluded area to obtain a reconstructed image, and based on the reconstructed image and the unoccluded area, a three-dimensional real-scene model of the subway line is generated. The domain image data is used to generate a three-dimensional real-life model of the subway line through real-life three-dimensional technology; Step 3: If there is an inspection area blocked by a running train, all reconstructed images are marked in the generated three-dimensional real-life model, and the blocked area data is obtained, and the danger impact factor is calculated based on the blocked area data; the image data of the blocked area is obtained, and the confidence factor is calculated based on the blocked image data; Step 4: In the three-dimensional real-life model, the structural characteristics of each inspection area are collected, and the actual degree of damage of each inspection area is obtained based on the structural characteristics evaluation; Step 5: The actual degree of damage of each inspection area is clustered, and the maintenance index is calculated based on the clustering processing results; Step 6: Whether to perform subway line maintenance is determined based on the maintenance index; Step 7: Regularly patrol by drones to capture the latest subway line image data and continuously update the three-dimensional real-life model.

[0009] Preferably, the step of detecting whether there is an inspection area blocked by a running train is as follows: for each inspection area, obtain the corresponding image data in the real-time database and record it as the current image; obtain the initial image of each inspection area as the background image; for each inspection area, compare the degree of change of each pixel in the current image and the background image, set a change threshold, compare the degree of pixel change with the change threshold, if the degree of pixel change is greater than or equal to the change threshold, then record the pixel as a changed pixel, if the degree of pixel change is less than the change threshold, then record the pixel as an unchanged pixel; connect the changed pixels to obtain the image change area, and obtain the image change area. The area of ​​the image change area and the area of ​​the background image are compared, and the occlusion degree is calculated by ratio of the area of ​​the image change area to the area of ​​the background image; an occlusion threshold is set, and the occlusion degree is compared with the occlusion threshold. If the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is occlusion by a running train in the inspection area; if the occlusion degree is less than the occlusion threshold, it is determined that there is no occlusion by a running train in the inspection area; the occlusion situation of each inspection area is counted, and if there is no occlusion by a running train in all inspection areas, it is determined to be an inspection area without occlusion by a running train; if there is occlusion by a running train in an inspection area, it is determined to be an inspection area with occlusion by a running train.

[0010] Preferably, the step of obtaining the risk impact factor is as follows: obtaining the historical maintenance times and service life of the shielded area; obtaining the number of trains passing through the shielded area during the detection period, the average speed of each train passing through the shielded area, and the time each train passes through the shielded area; calculating the load of a single train according to the average speed of each train passing through the shielded area and the time each train passes through the shielded area; summing the loads of the trains passing through the shielded area during the detection period to obtain the total operating load, and the area with high train traffic density bears a greater load; normalizing the historical maintenance times, service life, and total operating load, and evaluating the risk impact factor according to the normalized historical maintenance times, service life, and total operating load, and the specific acquisition method is as follows: ; In the formula, Expressed as the risk impact factor, Represents the number of historical maintenance times, Expressed as the useful life, Expressed as total operating load.

[0011] Preferably, the confidence factor acquisition step is: using Laplace transform to perform Laplace transform on each pixel in the occluded area image to obtain a Laplace transform value, calculating the Laplace transform mean of the occluded area image, and evaluating the image blur degree based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are: ; In the formula, Represents the degree of image blur, Represented as the Laplace transform value of the i-th pixel, Expressed as the Laplace transformed mean, is the total number of pixels; the pixel difference between adjacent frames in the occluded area image is calculated to obtain the frame difference; the confidence factor is obtained according to the image blur degree, temporal consistency and occlusion degree evaluation.

[0012] Preferably, the steps for obtaining the actual degree of damage of each inspection area are as follows: in the three-dimensional real-scene model, for unobstructed areas in the inspection area, the structural characteristics of the unobstructed areas are obtained, and the actual degree of damage of the unobstructed areas is obtained according to the structural characteristics evaluation; for obstructed areas in the inspection area, the structural characteristics of the obstructed areas are obtained, and the initial degree of damage is obtained according to the structural characteristics evaluation, and the initial degree of damage of the obstructed areas is obtained in the same manner as the actual degree of damage of the unobstructed areas, and the initial degree of damage, the hazard impact factor and the confidence factor are multiplied to obtain the actual degree of damage of the obstructed areas; the actual degree of damage of the unobstructed areas is integrated with the actual degree of damage of the obstructed areas to obtain the actual degree of damage of each inspection area.

[0013] Preferably, the step of obtaining the actual degree of damage in the unobstructed area is as follows: obtaining crack data of the detection component in the unobstructed area by image processing technology, the crack data including the total number of cracks, crack width and crack length, and obtaining the crack influence coefficient according to the crack data; obtaining the initial three-dimensional coordinates of each detection point of the detection component in the unobstructed area, obtaining the real-time three-dimensional coordinates of each detection point of the detection component by a three-dimensional real-scene model, calculating the offset of each detection point by using the Euclidean distance, calculating the average offset of all detection points, and recording it as the component offset influence coefficient; extracting the wear area and the detection component area of ​​the detection component in the unobstructed area by an edge detection algorithm, obtaining the area of ​​the wear area and the area of ​​the detection component area, and calculating the wear degree by ratio of the area of ​​the wear area to the area of ​​the detection component area; normalizing the crack influence coefficient, the component offset influence coefficient and the wear degree, and evaluating the actual degree of damage in the unobstructed area according to the normalized crack influence coefficient, the component offset influence coefficient and the wear degree, the specific acquisition steps are as follows: ; In the formula, Indicates the actual degree of damage. Expressed as the crack influence coefficient, Expressed as the component offset influence coefficient, Expressed as the degree of wear, , , It is expressed as the crack influence coefficient, component displacement influence coefficient and wear degree weight coefficient.

[0014] Preferably, the steps of clustering the actual damage degree of each inspection area are as follows: Step 5.1: taking the actual damage degree as the clustering feature, taking the actual damage degree of all inspection areas as the data set, and each actual damage degree in the data set as a data point; Step 5.2: using the silhouette coefficient method to determine the optimal clustering number K of the data set; Step 5.3: randomly selecting K data points in the data set as initial clustering centers, for each data point, calculating its Euclidean distance to each initial clustering center, for each data point, traversing the K initial clustering centers, and assigning it to the clustering cluster corresponding to the nearest initial clustering center; Step 5.4: after traversing all data points, obtaining the initial clustering clusters, for each initial clustering cluster, calculating the mean of the data points therein to obtain a new clustering center; Step 5.5: repeating steps 5.3 and 5.4 until the clustering center no longer changes, and obtaining the final clustering cluster and the final clustering center.

[0015] Preferably, the step of calculating the maintenance index based on the clustering processing result is: calculating the ratio of the number of data points in each final cluster cluster to the total number in the data set to obtain the weight of each final cluster cluster; and performing weighted summation of the weight of each final cluster cluster and the final cluster center to obtain the maintenance index.

[0016] Preferably, the step of judging whether to perform subway line maintenance based on the maintenance index is: setting a maintenance threshold, comparing the maintenance index with the maintenance threshold, if the maintenance index is less than the maintenance threshold, judging that the current subway line is in good condition and no subway line maintenance is performed; if the maintenance index is greater than or equal to the maintenance threshold, judging that the current subway line is in poor condition and no subway line maintenance is performed.

[0017] Preferably, a subway sky-ground line integrated platform monitoring system based on image recognition comprises: an image acquisition module, which is used to collect image data of each inspection area in the subway line through a drone, and transmit the image data of each inspection area to an occlusion detection module and a three-dimensional real scene model generation module; an occlusion detection module, which is used to detect whether there is an inspection area occluded by a running train according to the image data of each inspection area, and if there is an inspection area occluded by a running train, transmit the image data to an image reconstruction module; if there is no inspection area occluded by a running train, transmit the image data to a three-dimensional real scene model generation module; an image reconstruction module, which is used to reconstruct the occluded image of the inspection area to obtain a reconstructed image, and transmit the reconstructed image to the three-dimensional real scene model generation module; a three-dimensional real scene model generation module, which is used to generate a three-dimensional real scene model through real-scene three-dimensional technology, and transmit the three-dimensional real scene model to a maintenance judgment module; a maintenance judgment module, which is used to judge whether maintenance is needed according to the generated three-dimensional real scene model; a model update module, which is used to obtain subway line image data in real time and continuously update the three-dimensional real scene model.

[0018] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Set the required inspection areas in the subway line, and use drones to dynamically collect image data for each inspection area in real time during image acquisition, effectively solving the problem that fixed sensors cannot cover all areas and manual inspections easily miss key areas. Drones are highly flexible and have a wide coverage capability. They can quickly reach complex, dangerous or hard-to-reach areas to ensure comprehensive coverage of all inspection points. At the same time, compared with manual inspections, drones can significantly reduce the limitations of manual operations and reduce the risk of missing key areas due to fatigue, subjective judgment or limited vision.

[0019] 2. Generate a 3D real-life model of the subway line through real-life 3D technology. The 3D real-life model can truly restore the spatial information of the subway line and its surrounding environment, providing an intuitive and detailed data basis for accurate positioning and comprehensive analysis. Compared with traditional 2D plane maps, the 3D model can more clearly present the complex structure of the line, the track shape and the spatial distribution of key components, making it easier to discover potential hazards.

[0020] 3. Use the regional reconstruction algorithm to complete the image data of the blocked area to obtain the reconstructed image. The regional reconstruction algorithm can not only significantly improve the accuracy and continuity of monitoring, but also reduce the omission of hidden dangers caused by data loss, thereby improving the reliability of hidden danger identification and fault detection. At the same time, it can reduce the workload of manual data collection, save time and cost, and provide technical support for the efficient operation and maintenance and safety monitoring of subway lines.

[0021] 4. Through regular drone patrols, the latest subway line image data is captured and the 3D real-life model is continuously updated. Drone patrols can efficiently and comprehensively cover the entire line, obtain the latest high-resolution image data, and use it to timely update the 3D real-life model, thereby dynamically reflecting the actual status of the line and the surrounding environment. Compared with traditional static models, dynamically updated 3D models can more accurately identify changes and potential risks in subway lines, such as track deviation, equipment wear or environmental erosion. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flow chart of a subway sky-land line integrated platform monitoring method based on image recognition provided in an embodiment of the present application.

[0023] Figure 2 A structural diagram of a subway sky-land line integrated platform monitoring system based on image recognition provided in an embodiment of the present application.

[0024] Figure 3 Schematic diagram of the load of each train in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are only examples. The subway sky-ground line integrated platform monitoring method and system based on image recognition involved in the present invention are not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0026] The present invention provides a method for monitoring a subway ground-to-ground line integrated platform based on image recognition, such as Figure 1 As shown, the following steps are included: Step 1: A camera device is installed on the drone to collect image data; the required inspection area in the subway line is set, and during the image collection period, the drone is used to collect image data for each inspection area in real time and dynamically, and each image data is marked with a corresponding timestamp and the corresponding inspection area, and the image data of each inspection area is saved in a real-time image database; Step 2: Use occlusion detection to detect the image data in the real-time image database to see if there is an inspection area blocked by a running train. If there is no inspection area blocked by a running train, generate a 3D real scene model of the subway line using real scene 3D technology; If the inspection area is blocked by a running train, the inspection area blocked by the running train is recorded as the blocked area, and the inspection area not blocked by the running train is recorded as the unblocked area; The image data of the blocked area is supplemented by using a regional reconstruction algorithm to obtain a reconstructed image, and a three-dimensional real scene model of the subway line is generated by using real scene three-dimensional technology based on the reconstructed image and the image data of the unblocked area; Occlusion detection is a technology that uses computer vision technology to analyze monitoring images or videos to determine whether the target area is blocked by running trains or other dynamic objects, and is used to identify dynamic changes in the blocked area. In the monitoring of the subway sky-ground line integrated platform, this technology can effectively detect whether the inspection area is blocked by the passage of trains, providing basic support for subsequent monitoring.

[0027] The regional reconstruction algorithm is a technology that uses known data to infer and complete the image of the occluded or missing area. The algorithm uses interpolation methods, texture-based extensions, or deep learning models based on the spatial characteristics of the image, time series information, or multi-view data to generate an approximate image of the missing area. In the occluded area, the image of the occluded area is completed by combining the surrounding pixel features and historical information to generate a complete and as realistic as possible reconstructed image.

[0028] Realistic 3D technology is a modeling technology based on multi-source data. Through high-precision image acquisition and processing, it converts the real-world physical environment into a highly realistic 3D digital model. This technology uses algorithms such as image matching, point cloud data processing, and texture mapping to accurately restore terrain, buildings, and infrastructure, and is widely used in urban planning, engineering design, and smart transportation. In the modeling of subway lines, realistic 3D technology can generate a comprehensive 3D model that includes the surrounding environment of the line, terrain, and station facilities, providing strong support for engineering visualization and management.

[0029] In this embodiment, it should be specifically explained that the steps of detecting whether there is an inspection area blocked by a running train are: For each inspection area, the corresponding image data in the real-time database is obtained and recorded as the current image; Obtain an initial image of each inspection area as a background image. The initial image is the image of the inspection area just built, which serves as a benchmark for an unobstructed scene. For each inspection area, compare the change degree of each pixel in the current image and the background image, set a change threshold, and compare the pixel change degree with the change threshold. If the pixel change degree is greater than or equal to the change threshold, the pixel is recorded as a changed pixel. If the pixel change degree is less than the change threshold, the pixel is recorded as an unchanged pixel. The change degree is the absolute value of the color value difference at each pixel position. Connect the changed pixels to obtain the image change region, obtain the area of ​​the image change region and the area of ​​the background image, and calculate the ratio of the area of ​​the image change region to the area of ​​the background image to obtain the occlusion degree; Set an occlusion threshold, compare the occlusion degree with the occlusion threshold, if the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is a running train occlusion in the inspection area; if the occlusion degree is less than the occlusion threshold, it is determined that there is no running train occlusion in the inspection area; The occlusion situation of each inspection area is counted. If there is no occlusion by running trains in all inspection areas, it is determined as an inspection area without occlusion by running trains; if there is occlusion by running trains in one or more inspection areas, it is determined as an inspection area with occlusion by running trains.

[0030] In this embodiment, it should be specifically explained that the steps of using the regional reconstruction algorithm to complete the image data of the blocked area to obtain the reconstructed image are: Extract pixel values ​​in the current image except the image change area as known pixel values, and mark the pixels in the image change area as data that need to be reconstructed; The interpolation method uses the known pixel values ​​at the edge of the image change area to fill the image change area through mathematical interpolation to obtain the reconstructed image.

[0031] Interpolation is an image reconstruction method that uses known pixel values ​​at the edge of the image change area to infer unknown pixel values ​​in the occluded area or the change area through mathematical formulas. Interpolation analyzes the spatial distribution of known pixel values ​​to estimate the reasonable values ​​of unknown pixels. Common methods include bilinear interpolation and bicubic interpolation.

[0032] In this embodiment, it should be specifically explained that the steps of generating a 3D real scene model of a subway line by using real scene 3D technology are as follows: Use laser radar to scan the subway line to obtain high-precision 3D point cloud data, remove noise, filter and coordinate registration of the point cloud data to ensure data accuracy; Combined with the image data in the real-time image database, the images are aligned using image registration technology, and the scanned point cloud data are merged into an initial model using point cloud stitching technology; Use structured light technology to accurately scan the details of the initial model and extract the surface shape of the initial model; use optical feature extraction algorithms, such as edge detection and texture mapping, to supplement the surface texture and detail information of the initial model; The point cloud data is converted into a 3D mesh model, and the image data is applied to the model surface using texture mapping to enhance the details and realism of the initial model to obtain a 3D real scene model.

[0033] Image registration technology is the process of spatially aligning images taken from different perspectives, different sensors or different times. The core is to extract feature points (such as corners, edges or texture features) in the image, match these feature points in different images through matching algorithms, and then calculate the transformation matrix to achieve geometric alignment between images.

[0034] Point cloud stitching technology is the process of merging point cloud data collected from different scanning positions or at different times into a complete 3D point cloud model. The core is to calculate the spatial transformation relationship between different point cloud data sets by matching and registering the overlapping areas between them, and then seamlessly stitching multiple point cloud data together.

[0035] Structured light technology is a technology that projects known light stripes or patterns onto the surface of an object and uses a camera to capture the deformation of these stripes to obtain the three-dimensional shape of the object. The deformation of the light stripes on the surface of the object is directly related to the geometric shape of the object. By analyzing these deformations, the depth information and shape of the object surface can be accurately calculated.

[0036] Step 3: If there is an inspection area blocked by a running train, all reconstructed images are marked in the generated 3D real scene model, and the blocked area data is obtained. The danger impact factor is calculated based on the blocked area data. The blocked area image data is obtained from the real-time image database, and the confidence factor is calculated based on the blocked image data. In this embodiment, it should be specifically explained that the steps for obtaining the risk impact factor are: Get the historical maintenance times of the blocked area. The areas with higher maintenance times are more likely to have hidden dangers. Get the service life of the shielded area. The service life is the time from the construction of the shielded area to the present. Areas with longer service life are more likely to have aging problems. Obtain the number of trains passing through the blocked area during the detection period, the average speed of each train passing through the blocked area, and the time each train takes to pass through the blocked area; like Figure 3 As shown, the load of a single train is calculated based on the average speed of each train passing through the obstruction area and the passing time of each train in the obstruction area. The specific acquisition steps are: ; In the formula, Expressed as train load, Expressed as the average speed, Expressed as passing time; Table 1 Train load

[0037] As shown in Table 1, in a specific embodiment, by analyzing the impact of the average speed and passing time of the train on the train load, the differences in load distribution of different trains when passing through the obstruction area can be found. For example, a higher average speed and a shorter passing time may result in a smaller train load value, while a lower average speed and a longer passing time may increase the train load value. These changes in load reflect the operating pressure and power consumption characteristics that the train is subjected to when passing through the obstruction area. This analysis can be further applied to track operation monitoring. By capturing abnormal load distribution, the potential risks of track or train operation can be evaluated.

[0038] The total operating load is calculated by summing the loads of trains passing through the blocked area during the detection period. Areas with high train traffic density bear greater loads. The historical maintenance times, service life and total operating load are normalized, and the hazard impact factor is obtained based on the normalized historical maintenance times, service life and total operating load. The specific acquisition method is as follows: ; In the formula, Expressed as the risk impact factor, Represents the number of historical maintenance times, Expressed as the useful life, Expressed as total operating load.

[0039] In this embodiment, it should be specifically explained that the confidence factor acquisition steps are: Use Laplace transform to perform Laplace transform on each pixel in the occluded area image to obtain the Laplace transform value, calculate the Laplace transform mean of the occluded area image, and evaluate the image blur degree based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are as follows: ; In the formula, Represents the degree of image blur, Represented as the Laplace transform value of the i-th pixel, Expressed as the Laplace transformed mean, is the total number of pixels; Calculate the pixel difference between adjacent frames in the occluded area image to obtain the frame difference, normalize the frame difference to obtain the temporal consistency, and the closer the temporal consistency is to 1, the more consistent the temporal sequence is; The confidence factor is obtained based on the image blur, temporal consistency, and occlusion level. The specific acquisition steps are as follows: ; In the formula, Expressed as the confidence factor, Represents the degree of image blur, It is expressed as the degree of occlusion, Indicated as temporal consistency.

[0040] In image processing, Laplace transform usually refers to the convolution operation of an image using the Laplace operator, with the purpose of detecting edges or changing areas in the image. The Laplace operator is a second-order differential operator that can highlight the details of an image, especially the edges or contours of an image, by calculating the grayscale changes in the area around each pixel. In image processing of occluded areas, the Laplace transform helps to reveal the edge features of the occluded part, thereby providing information about the occluded area, which is helpful for subsequent image restoration or reconstruction.

[0041] Step 4: In the 3D real-life model, collect the structural characteristics of each inspection area, and evaluate the actual damage degree of each inspection area based on the structural characteristics; In this embodiment, it should be specifically explained that the steps for obtaining the actual damage degree of each inspection area are: In the three-dimensional real scene model, for the unobstructed area in the inspection area, the structural characteristics of the unobstructed area are obtained, and the actual damage degree of the unobstructed area is evaluated according to the structural characteristics; For the occluded area in the inspection area, the structural characteristics of the occluded area are obtained, and the initial damage degree is obtained according to the structural characteristics. The initial damage degree of the occluded area is obtained in the same way as the actual damage degree of the unoccluded area. The initial damage degree, the risk impact factor and the confidence factor are multiplied to obtain the actual damage degree of the occluded area. The confidence factor represents the reliability of the detection result of the occluded area, and combines factors such as image quality, time sequence consistency, and occlusion degree to judge the confidence in the damage of the occluded area. The actual damage degree of the unobstructed area is integrated with the actual damage degree of the obstructed area to obtain the actual damage degree of each inspection area.

[0042] In this embodiment, it should be specifically explained that the steps for obtaining the actual damage degree of the unblocked area are: The crack data of the detected parts in the unblocked area are obtained by image processing technology. The crack data includes the total number of cracks, crack width and crack length. The crack influence coefficient is obtained according to the crack data. The specific acquisition method is as follows: ; In the formula, Expressed as the crack influence coefficient, is the crack length of the jth crack, It is represented as the crack width of the jth crack; Obtain the initial 3D coordinates of each detection point of the detection component in the unobstructed area, obtain the real-time 3D coordinates of each detection point of the detection component through the 3D real scene model, calculate the offset of each detection point using the Euclidean distance, calculate the average offset of all detection points, and record it as the component offset influence coefficient; The wear area and the detection component area of ​​the detection component in the unblocked area are extracted by an edge detection algorithm, the area of ​​the wear area and the area of ​​the detection component area are obtained, and the wear degree is calculated by calculating the ratio of the area of ​​the wear area to the area of ​​the detection component area; The crack influence coefficient, component offset influence coefficient and wear degree are normalized, and the actual damage degree of the unblocked area is evaluated based on the normalized crack influence coefficient, component offset influence coefficient and wear degree. The specific acquisition steps are as follows: ; In the formula, Indicates the actual degree of damage. Expressed as the crack influence coefficient, the condition of the crack in a component is a key indicator of the severity of damage. As the crack expands, the overall strength and stability of the material will gradually decrease, which may lead to further damage or failure. Therefore, an increase in the crack influence coefficient usually indicates that the degree of damage to the structure has increased, indicating that the part may have or will soon lose its bearing capacity, thus affecting the safety and reliability of the entire system. Expressed as the component offset influence coefficient, the degree of offset that occurs during the use of a component directly reflects the severity of its damage. When a component is offset or deformed during operation, it is usually caused by long-term wear, stress concentration, fatigue and other factors. These changes will cause the component's geometry to deviate, which in turn affects its function and performance. As the offset increases, the load distribution, stability and accuracy of the component will be affected, which may lead to further damage or failure of the component. By monitoring and calculating the component offset influence coefficient, potential damage areas can be identified in a timely manner, and by predicting and analyzing the offset trend, the remaining life of the component can be evaluated, providing a scientific basis for maintenance and replacement decisions, and avoiding safety hazards caused by excessive wear and accidental failure. Expressed as the degree of wear, wear usually refers to the gradual peeling or change of surface materials due to friction, contact and long-term use. This phenomenon will directly affect the performance, shape and function of the material. When the wear exceeds a certain critical value, the load-bearing capacity, stability and durability of the component will be significantly reduced, leading to more serious damage. When the wear reaches a certain level, the damage to the structure may rapidly increase, thus affecting the function and safety of the overall system. , , It is expressed as the weight coefficient of crack influence coefficient, component displacement influence coefficient and wear degree, and , , , The specific value is determined by professionals according to the actual situation, for example, , , It can be 0.3, 0.2, or 0.5.

[0043] Step 5: Cluster the actual damage degree of each inspection area, and calculate the repair index based on the clustering results; In this embodiment, it should be specifically explained that the steps of clustering the actual damage degree of each inspection area are: Step 5.1: Take the actual damage degree as the clustering feature, take the actual damage degree of all inspection areas as the data set, and each actual damage degree in the data set as a data point; Step 5.2: Use the silhouette coefficient method to determine the optimal number of clusters K for the data set; The silhouette coefficient method measures the quality of clustering results by evaluating the relative relationship between the compactness of each data point in its cluster and its distance to the nearest other cluster. The optimal number of clusters is determined by calculating the average silhouette coefficient under different cluster numbers K and finding the K corresponding to the maximum value. The core idea of ​​this method is to balance the compactness within the cluster and the separation between clusters to find the most reasonable cluster division.

[0044] Step 5.3: Randomly select K data points in the data set as the initial cluster centers. For each data point, calculate its Euclidean distance to each initial cluster center. The specific method is: ,in It is expressed as the Euclidean distance from the data point to the cluster center, where Represented as data points, Represented as the initial cluster center, for each data point, traverse the K initial cluster centers and assign it to the cluster corresponding to the nearest initial cluster center; Euclidean distance is a measure of the straight-line distance between two points and is the most commonly used distance calculation method. In K-means clustering, Euclidean distance is used to evaluate the similarity between a data point and the cluster center. The smaller the distance, the closer the data point is to the cluster center and therefore more suitable for belonging to the cluster. This distance is intuitive and easy to calculate, and is suitable for clustering tasks that process numerical data.

[0045] Step 5.4: After traversing all data points, the initial clusters are obtained. For each initial cluster, the data points in it are calculated to get the new cluster center. Step 5.5: Repeat steps 5.3 and 5.4 until the cluster center no longer changes, and obtain the final cluster and the final cluster center.

[0046] In this embodiment, it should be specifically explained that the steps of calculating the maintenance index according to the clustering processing result are: The weight of each final cluster is calculated by calculating the ratio of the number of data points in each final cluster to the total number of data points in the data set; The weight of each final cluster is added to the final cluster center to obtain the maintenance index.

[0047] Step 6: Determine whether to carry out subway line maintenance based on the maintenance index; In this embodiment, it should be specifically explained that the steps of determining whether to perform subway line maintenance according to the maintenance index are as follows: A maintenance threshold is set, and the maintenance index is compared with the maintenance threshold. If the maintenance index is less than the maintenance threshold, it is judged that the current subway line is in good condition and no subway line maintenance is performed; if the maintenance index is greater than or equal to the maintenance threshold, it is judged that the current subway line is in poor condition and subway line maintenance is performed.

[0048] Step 7: Use drones to regularly patrol the area to capture the latest subway line image data and continuously update the 3D real-life model.

[0049] In this embodiment, it should be specifically explained that Figure 2 As shown, a subway sky-ground line integrated platform monitoring system based on image recognition, the system includes: An image acquisition module is used to collect image data of each inspection area in the subway line through a drone, and transmit the image data of each inspection area to the occlusion detection module and the 3D real scene model generation module; The occlusion detection module is used to detect whether there is an inspection area occluded by a running train based on the image data of each inspection area, and if there is an inspection area occluded by a running train, the image data is transmitted to the image reconstruction module; if there is no inspection area occluded by a running train, the image data is transmitted to the three-dimensional real scene model generation module; An image reconstruction module is used to reconstruct the occluded image of the inspection area to obtain a reconstructed image, and transmit the reconstructed image to the three-dimensional real scene model generation module; A three-dimensional real scene model generation module is used to generate a three-dimensional real scene model through real scene three-dimensional technology, and transmit the three-dimensional real scene model to the maintenance judgment module; A maintenance judgment module is used to judge whether maintenance is needed based on the generated three-dimensional real scene model; The model update module is used to obtain subway line image data in real time and continuously update the three-dimensional real-scene model.

[0050] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0051] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A subway ground-to-ground line integrated platform monitoring method based on image recognition, characterized in that: The following steps are involved: Step 1: Set up the camera on the drone; Set the required inspection areas in the subway line, collect image data of each inspection area dynamically and in real time through drones during image acquisition, and save the image data of each inspection area into the real-time image database; Step 2: Perform occlusion detection on the image data in the real-time image database. If the inspection area is not blocked by the running train, a 3D real scene model of the subway line is generated by real scene 3D technology; If the inspection area is blocked by a running train, the inspection area blocked by the running train is recorded as the blocked area, and the inspection area not blocked by the running train is recorded as the unblocked area; The image data of the blocked area is supplemented by using a regional reconstruction algorithm to obtain a reconstructed image, and a three-dimensional real scene model of the subway line is generated by using real scene three-dimensional technology based on the reconstructed image and the image data of the unblocked area; Step 3: If there is an inspection area blocked by a running train, all reconstructed images are marked in the generated 3D real scene model, and the blocked area data is obtained, and the danger impact factor is calculated based on the blocked area data; Obtain image data of the occluded area, and calculate a confidence factor based on the occluded image data; Step 4: In the 3D real-life model, collect the structural characteristics of each inspection area, and evaluate the actual damage degree of each inspection area based on the structural characteristics; Step 5: Cluster the actual damage degree of each inspection area, and calculate the repair index based on the clustering results; Step 6: Determine whether to carry out subway line maintenance based on the maintenance index; Step 7: Use drones to regularly patrol the area to capture the latest subway line image data and continuously update the 3D real-life model.

2. According to the method of claim 1, the method is characterized by: The steps of detecting whether there is an inspection area blocked by a running train are as follows: For each inspection area, the corresponding image data in the real-time database is obtained and recorded as the current image; Obtaining an initial image of each inspection area as a background image; For each inspection area, compare the change degree of each pixel in the current image and the background image, set a change threshold, compare the pixel change degree with the change threshold, if the pixel change degree is greater than or equal to the change threshold, then the pixel is recorded as a changed pixel, if the pixel change degree is less than the change threshold, then the pixel is recorded as an unchanged pixel; Connect the changed pixels to obtain the image change region, obtain the area of ​​the image change region and the area of ​​the background image, and calculate the ratio of the area of ​​the image change region to the area of ​​the background image to obtain the occlusion degree; Set an occlusion threshold, compare the occlusion degree with the occlusion threshold, and if the occlusion degree is greater than or equal to the occlusion threshold, it is determined that there is a running train occluding the inspection area; If the occlusion degree is less than the occlusion threshold, it is determined that there is no occlusion of the running train in the inspection area; The occlusion situation of each inspection area is counted. If there is no occlusion by running trains in all inspection areas, it is determined that there is no occlusion by running trains in the inspection area. If there is an inspection area blocked by a running train, it is determined that there is an inspection area blocked by a running train.

3. According to the method of image recognition based on subway ground-to-ground line integrated platform monitoring in claim 1, it is characterized in that: The steps for obtaining the risk impact factor are: Obtain the historical maintenance times and service life of the blocked area; Obtain the number of trains passing through the blocked area during the detection period, the average speed of each train passing through the blocked area, and the time each train takes to pass through the blocked area; The load of a single train is calculated based on the average speed of each train passing through the blocked area and the passing time of each train in the blocked area; The total operating load is calculated by summing the loads of trains passing through the blocked area during the detection period. Areas with high train traffic density bear greater loads. The historical maintenance times, service life and total operating load are normalized, and the hazard impact factor is obtained based on the normalized historical maintenance times, service life and total operating load. The specific acquisition method is as follows: ； In the formula, Expressed as the risk impact factor, Represents the number of historical maintenance times, Expressed as the useful life, Expressed as total operating load.

4. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 1, characterized in that: The confidence factor acquisition steps are: Use Laplace transform to perform Laplace transform on each pixel in the occluded area image to obtain the Laplace transform value, calculate the Laplace transform mean of the occluded area image, and evaluate the image blur degree based on the Laplace transform value of each pixel and the Laplace transform mean. The specific acquisition steps are as follows: ； In the formula, Represents the degree of image blur, Represented as the Laplace transform value of the i-th pixel, Expressed as the Laplace transformed mean, is the total number of pixels; Calculate the pixel difference between adjacent frames in the occluded area image to obtain the frame difference; The confidence factor is obtained based on the image blur, temporal consistency, and occlusion level.

5. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps for obtaining the actual damage degree of each inspection area are as follows: In the three-dimensional real scene model, for the unobstructed area in the inspection area, the structural characteristics of the unobstructed area are obtained, and the actual damage degree of the unobstructed area is evaluated according to the structural characteristics; For the blocked area in the inspection area, the structural characteristics of the blocked area are obtained, and the initial damage degree is obtained according to the structural characteristics. The initial damage degree of the blocked area is obtained in the same way as the actual damage degree of the unblocked area. The initial damage degree, the hazard impact factor and the confidence factor are multiplied to obtain the actual damage degree of the blocked area. The actual damage degree of the unobstructed area is integrated with the actual damage degree of the obstructed area to obtain the actual damage degree of each inspection area.

6. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 5, characterized in that: The steps for obtaining the actual damage degree of the unblocked area are: The crack data of the detected component in the unblocked area is obtained by image processing technology, the crack data includes the total number of cracks, crack width and crack length, and the crack influence coefficient is obtained according to the crack data; Obtain the initial 3D coordinates of each detection point of the detection component in the unobstructed area, obtain the real-time 3D coordinates of each detection point of the detection component through the 3D real scene model, calculate the offset of each detection point using the Euclidean distance, calculate the average offset of all detection points, and record it as the component offset influence coefficient; The wear area and the detection component area of ​​the detection component in the unblocked area are extracted by an edge detection algorithm, the area of ​​the wear area and the area of ​​the detection component area are obtained, and the wear degree is calculated by calculating the ratio of the area of ​​the wear area to the area of ​​the detection component area; The crack influence coefficient, component offset influence coefficient and wear degree are normalized, and the actual damage degree of the unblocked area is evaluated based on the normalized crack influence coefficient, component offset influence coefficient and wear degree. The specific acquisition steps are as follows: ； In the formula, Indicates the actual degree of damage. Expressed as the crack influence coefficient, Expressed as the component offset influence coefficient, Expressed as the degree of wear, , , It is expressed as the crack influence coefficient, component displacement influence coefficient and wear degree weight coefficient.

7. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of clustering the actual damage degree of each inspection area are as follows: Step 5.1: Take the actual damage degree as the clustering feature, take the actual damage degree of all inspection areas as the data set, and each actual damage degree in the data set as a data point; Step 5.2: Use the silhouette coefficient method to determine the optimal number of clusters K for the data set; Step 5.3: Randomly select K data points in the data set as the initial cluster centers. For each data point, calculate the Euclidean distance from each initial cluster center. For each data point, traverse the K initial cluster centers and assign it to the cluster corresponding to the nearest initial cluster center. Step 5.4: After traversing all data points, the initial clusters are obtained. For each initial cluster, the data points in it are calculated to get the new cluster center. Step 5.5: Repeat steps 5.3 and 5.4 until the cluster center no longer changes, and obtain the final cluster and the final cluster center.

8. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 1, characterized in that: The step of calculating the maintenance index according to the clustering processing result is: The weight of each final cluster is calculated by calculating the ratio of the number of data points in each final cluster to the total number of data points in the data set; The weight of each final cluster is added to the final cluster center to obtain the maintenance index.

9. The method for monitoring a subway ground-to-ground line integrated platform based on image recognition according to claim 1, characterized in that: The steps of judging whether to carry out subway line maintenance according to the maintenance index are as follows: A maintenance threshold is set, and the maintenance index is compared with the maintenance threshold. If the maintenance index is less than the maintenance threshold, it is judged that the current subway line is in good condition and no subway line maintenance is performed; if the maintenance index is greater than or equal to the maintenance threshold, it is judged that the current subway line is in poor condition and subway line maintenance is performed.

10. A subway ground-to-ground line integrated platform monitoring system based on image recognition, used to implement a subway ground-to-ground line integrated platform monitoring method based on image recognition according to any one of claims 1 to 9, characterized in that: The system comprises: An image acquisition module is used to collect image data of each inspection area in the subway line through a drone, and transmit the image data of each inspection area to the occlusion detection module and the 3D real scene model generation module; The occlusion detection module is used to detect whether there is an inspection area occluded by a running train based on the image data of each inspection area, and if there is an inspection area occluded by a running train, the image data is transmitted to the image reconstruction module; if there is no inspection area occluded by a running train, the image data is transmitted to the three-dimensional real scene model generation module; An image reconstruction module is used to reconstruct the occluded image of the inspection area to obtain a reconstructed image, and transmit the reconstructed image to the three-dimensional real scene model generation module; A three-dimensional real scene model generation module is used to generate a three-dimensional real scene model through real scene three-dimensional technology, and transmit the three-dimensional real scene model to the maintenance judgment module; A maintenance judgment module is used to judge whether maintenance is needed based on the generated three-dimensional real scene model; The model update module is used to obtain subway line image data in real time and continuously update the three-dimensional real-scene model.

Images