A guide sign integrity monitoring method and system based on image recognition

Abstract

The application relates to the technical field of image recognition, and discloses a guide sign integrity monitoring method and system based on image recognition, which comprises the following steps: acquiring an image containing a guide sign; extracting edge contour information of the guide sign in the image; identifying an edge segment with a fracture in the edge contour information as a target edge segment; extracting tangent vectors on both sides of the fracture position of the target edge segment, and judging whether the tangent vectors on both sides of the fracture position meet a collinearity condition; when the tangent vectors meet the collinearity condition, acquiring a spatial distribution center feature of the guide sign in the image; comparing the spatial distribution center feature with a preset position reference, preliminarily determining a fracture attribute of the target edge segment according to a comparison result, and secondarily checking the preliminary determination result. The application effectively avoids false alarms caused by external shielding and safety risks caused by failure to identify real damage, and improves the intelligent level and safety of metro facility operation and maintenance.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method and system for monitoring the integrity of wayfinding signs based on image recognition. Background Technology

[0002] The suspended directional signs widely installed in subway stations are key facilities for guiding passenger flow and conveying information. These signs are fixed to the ceiling via hangers, bolts, and other connectors. They are subject to the effects of train vibrations and environmental changes over a long period of time, which may lead to problems such as loose connections and structural deformation, posing safety hazards.

[0003] Therefore, automated monitoring of the integrity of directional signage is crucial. Existing monitoring methods typically employ image acquisition and analysis technology, but this approach faces a significant technical challenge in the complex environment of subways. Specifically, when discontinuous or broken edges of directional signs appear in images captured by cameras, the system struggles to accurately distinguish whether this phenomenon is caused by actual structural damage (such as gaps formed by signboard displacement due to loose mounting points) or by temporary external obstruction (such as luggage or umbrellas temporarily covering part of the sign). These two situations can appear highly similar in a single frame, leading to misjudgments by the automated system. Misjudging obstruction as damage triggers unnecessary emergency repairs, wasting manpower and resources; conversely, ignoring damage as obstruction may allow safety hazards to develop unchecked, with serious consequences. This ambiguity in judgment significantly reduces the reliability of existing image monitoring methods, making it difficult to meet the dual demands of high safety and high efficiency in subway operations.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application discloses a method and system for monitoring the integrity of directional signs based on image recognition, aiming to solve the technical problem in existing technologies that it is difficult to accurately distinguish whether the edge breakage of directional signs is caused by structural damage or external obstruction.

[0006] In a first aspect, this application discloses an image recognition-based method for monitoring the integrity of wayfinding signs in subway stations. The method includes the following steps: Get an image containing directional signs; Extract the edge contour information of the guide signs in the image; Identify broken edge segments in the edge contour information and use them as target edge segments; Extract the tangent vectors of the target edge segment on both sides of the fracture location, and determine whether the tangent vectors on both sides of the fracture location meet the collinearity condition; When the tangent vectors meet the collinearity condition, the spatial distribution center feature of the guide sign in the image is obtained; The spatial distribution center features are compared with the preset location benchmark. Based on the comparison results, the fracture attributes of the target edge segment are initially determined, and the initial determination results are then verified.

[0007] This technical solution establishes a multi-dimensional analysis framework encompassing geometric morphology and spatial location. Firstly, through tangent collinearity analysis, edge fractures potentially caused by occlusion from straight objects or structural translation are preliminarily identified, excluding fractures caused by irregular deformations such as wrinkles or damage to the marker itself. Then, by comparing the spatial distribution center features with a reference position, it is determined whether the marker as a whole has shifted. This allows for a preliminary, evidence-based judgment of fracture attributes within a single image frame, laying the foundation for subsequent accurate identification and significantly improving the accuracy of distinguishing between structural damage and external occlusion.

[0008] Further, the steps of extracting the tangent vectors on both sides of the fracture location of the target edge segment and determining whether the tangent vectors on both sides of the fracture location meet the collinearity condition include: Extract the tangent vectors of the target edge segment on both sides of the fracture location; Calculate the angle between the tangent vectors on both sides, and calculate the linear distance between the fracture points on both sides of the fracture location of the target edge segment; Determine whether the included angle is less than a preset angle threshold and whether the linear distance is less than a preset distance threshold; If the included angle is less than a preset angle threshold and the linear distance is less than a preset distance threshold, the tangent vectors on both sides of the fracture location are determined to be collinear. Otherwise, the tangent vectors on both sides of the fracture location are determined to be collinear.

[0009] This technical solution transforms the abstract collinearity condition into specific computational indicators. By setting angle and distance thresholds, the judgment process becomes more accurate and robust, effectively filtering out minor deviations caused by image noise or slight curvature of markers. This ensures that only geometrically highly continuous breaks are considered to meet the criteria, thereby improving the reliability of collinearity judgment.

[0010] Furthermore, the steps for extracting edge contour information of the guide signs in the image include: The image is processed for brightness equalization to eliminate the impact of changes in ambient light on image contrast. The image after brightness equalization is geometrically corrected using a pre-calibrated perspective correction matrix, transforming the image into an orthophoto image. Extract edge contour information from orthophotos.

[0011] This technical solution preprocesses the original image before core analysis. Brightness equalization overcomes problems such as uneven lighting and reflections in subway stations, while perspective correction eliminates geometric distortions caused by the tilt of the camera's shooting angle. This ensures that subsequent edge extraction and geometric measurements are performed from a standardized, distortion-free overhead view, greatly improving the accuracy of feature extraction and the reliability of all subsequent calculations, providing a high-quality data foundation for the effectiveness of the entire method.

[0012] Furthermore, when the tangent vectors satisfy the collinearity condition, the steps for obtaining the spatial distribution center features of the guide sign in the image include: When the tangent vectors meet the collinearity condition, the edge segments on both sides of the break position are linearly extended along the vector direction of the tangent vectors to obtain the completed edge contour information. The geometric center of the region enclosed by the completed edge contour information is calculated to obtain the first reconstruction center, and the first reconstruction center is used as the spatial distribution center feature.

[0013] This technical solution proposes a method for intelligently estimating the geometric center of an object under conditions of incomplete information. By linearly extending the broken edges, the outline of the occluded or missing portion is virtually reconstructed, and the geometric center is calculated based on this logically complete shape. Compared to directly using the incomplete outline to calculate the center, this first reconstructed center can more realistically reflect the theoretical position of the guide sign before it was occluded, providing a more stable and accurate benchmark for subsequent positional deviation comparisons, and significantly improving the accuracy of determining whether the sign has undergone overall displacement.

[0014] Furthermore, the step of comparing the spatial distribution center features with a preset location benchmark and preliminarily determining the fracture attributes of the target edge segment based on the comparison results includes: Calculate the positional deviation of the first reconstruction center relative to the preset positional reference; Determine whether the positional deviation is within the preset allowable deviation range. If the positional deviation is within the allowable deviation range, determine that the spatial distribution center features conform to the positional reference. If the spatial distribution center features match the location benchmark, the fracture attribute is initially determined to be external shading; otherwise, the fracture attribute is initially determined to be structural damage.

[0015] This technical solution clarifies the logical rules for preliminary judgment based on positional deviation. If the reconstructed center position is still within the allowable error range, it indicates that the sign itself has not moved, and the edge breakage is likely caused by external object obstruction; conversely, if the center position deviates from the reference, it indicates that the sign has moved as a whole, which is highly likely to be structural damage. This step transforms the complex reasoning process into a clear decision, making the preliminary judgment automated and logically rigorous.

[0016] Furthermore, the steps for obtaining the image containing the directional signs include: Real-time monitoring of train arrival and departure signals within subway stations; When no train entry or exit signal is detected, images are acquired at a preset first image acquisition frequency; When a train entering or leaving the station is detected, the first image acquisition frequency is increased to a preset second image acquisition frequency, and the image acquired at the second image acquisition frequency is stabilized to obtain a clear and stable image.

[0017] This technical solution introduces an image acquisition strategy that is linked to environmental events. Considering that trains entering and leaving stations are the main causes of vibration and structural loosening, the system proactively increases the acquisition frequency during these critical periods, enabling it to capture the process of damage occurrence or development with a higher probability. During stable periods, the frequency is reduced, saving computing and storage resources. This dynamically adjusted acquisition mechanism optimizes the allocation of monitoring resources, improving the targeting and efficiency of monitoring.

[0018] Furthermore, the method also includes: When a train arrival or departure signal is detected, images from multiple consecutive sampling periods before and after the time the train arrival or departure signal is detected are retrieved. For each sample period of the image, a set of corresponding candidate tangent clusters is generated to form the prediction envelope. The predicted envelopes corresponding to multiple consecutive sampling periods are superimposed in the same coordinate system, and the second reconstruction center is determined in the superimposed overlapping region.

[0019] This technical solution utilizes time-series image sequences acquired at high frequencies to generate a predicted envelope. It integrates edge jitter information caused by minute vibrations over a short period of time. The resulting second reconstruction center is based on statistical results of multiple frames over a period of time. Compared to the first reconstruction center of a single frame image, its position is more stable and more resistant to accidental noise and instantaneous jitter in a single frame image, providing more reliable data points for dynamic trajectory analysis.

[0020] Furthermore, the steps for secondary verification of fracture properties include: Obtain the movement trajectory of the second reconstruction center; If the movement trajectory shows unidirectional irreversible slippage, the preliminary judgment result will be uniformly adjusted to structural damage; Otherwise, the preliminary judgment will be upheld.

[0021] This technical solution introduces dynamic analysis over time, a crucial tool for distinguishing between occlusion and damage. External occlusions are typically mobile and random, resulting in chaotic or recoverable changes in the reconstructed center trajectory. In contrast, structural damage, such as sagging markers due to loose bolts, manifests as a continuous, unidirectional, and irreversible slippage of the reconstructed center. By analyzing this macroscopic trend in the trajectory, the two can be effectively differentiated from a dynamic perspective, providing a strong verification of the initial static assessment and significantly reducing the false positive rate.

[0022] Furthermore, the steps following the secondary verification of fracture properties also include: When the movement trajectory exhibits unidirectional irreversible slippage, extract the local surface texture features of the target edge segment at the break location; Calculate the texture similarity between local surface texture features and pre-stored standard surface texture features; When the texture similarity of at least one target edge segment at the fracture location is lower than the preset similarity threshold, the judgment result of the secondary verification is maintained as structural damage; When the texture similarity of all target edge segments at the fracture location is not lower than the similarity threshold, the judgment result of the secondary verification is adjusted to external occlusion with slip features.

[0023] This technical solution adds a refined screening process for special and difficult scenarios. Based on an initial determination of slippage, the material at the fracture point is further examined. If the texture at the fracture point does not match the standard texture of the signboard itself, it indicates that what is being observed may not be the fracture surface of the signboard, but rather a foreign object attached to the signboard and slowly sliding along with it (such as a lightweight object being caught). This texture comparison step can accurately identify this composite situation that combines slippage and occlusion characteristics, pushing the accuracy of monitoring to a new level and avoiding false alarms.

[0024] Secondly, this application also discloses an image recognition-based wayfinding sign integrity monitoring system for performing the steps in any of the foregoing methods, including: The image acquisition module is used to acquire images containing directional signs; The edge extraction module is used to extract the edge contour information of guide signs in the image; The fracture recognition module is used to identify edge segments with fractures in the edge contour information and use them as target edge segments. The comparison module is used to extract the tangent vectors of the target edge segment on both sides of the fracture position and determine whether the tangent vectors on both sides of the fracture position meet the collinearity condition. The feature extraction module is used to obtain the spatial distribution center features of the guide sign in the image when the tangent vectors meet the collinearity condition; The judgment and verification module is used to compare the spatial distribution center features with the preset position benchmark, preliminarily determine the fracture attributes of the target edge segment based on the comparison results, and perform secondary verification on the preliminary judgment results.

[0025] This technical solution provides a physical system architecture that can implement the above monitoring methods. It deconstructs the complex method process into functionally defined modules, providing a clear blueprint for the engineering implementation of the methods and making it highly operable.

[0026] In summary, this application provides a method and system for monitoring the integrity of wayfinding signs based on image recognition. First, it uses collinearity analysis of tangents at the edge breakage point to initially screen the geometric shape of the breakage. Then, it virtually reconstructs the occluded contour and calculates its geometric center, comparing it with a preset benchmark position to determine whether the overall sign has shifted, thus completing a preliminary static assessment. Furthermore, this application introduces dynamic analysis based on time-series images. By tracking the movement trajectory of the reconstructed center over time, it utilizes the essential difference in dynamic characteristics between unidirectional irreversible slippage caused by structural damage and random movement caused by external occlusion to perform a secondary verification of the preliminary assessment results. This comprehensive judgment mechanism, from static geometric analysis to dynamic behavior analysis, can accurately identify the root cause of edge breaks, greatly improving monitoring accuracy, effectively avoiding false alarms caused by external occlusion and safety risks arising from failure to identify actual damage, and enhancing the intelligence and safety of subway facility operation and maintenance. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a method for monitoring the integrity of wayfinding signs based on image recognition, provided in an embodiment of this application.

[0028] Figure 2 This is a schematic diagram of the structure of a wayfinding sign integrity monitoring system based on image recognition, provided in an embodiment of this application.

[0029] Labeling Explanation: 210, Image Acquisition Module; 220, Edge Extraction Module; 230, Fracture Recognition Module; 240, Comparison Module; 250, Feature Extraction Module; 260, Judgment and Verification Module. Detailed Implementation

[0030] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0031] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] Firstly, referring to Figure 1 This application provides a method for monitoring the integrity of wayfinding signs based on image recognition, used to monitor the integrity of wayfinding signs in subway stations. The method includes the following steps: S1. Obtain the image containing the directional signs; S2. Extract the edge contour information of the guide signs in the image; S3. Identify broken edge segments in the edge contour information and use them as target edge segments; S4. Extract the tangent vectors of the target edge segment on both sides of the fracture position, and determine whether the tangent vectors on both sides of the fracture position meet the collinearity condition. S5. When the tangent vectors meet the collinearity condition, obtain the spatial distribution center features of the guide sign in the image; S6. Compare the spatial distribution center features with the preset position benchmark, preliminarily determine the fracture attributes of the target edge segment based on the comparison results, and perform a second verification on the preliminary determination results.

[0033] The target edge segment refers to a discontinuous and clearly interrupted portion that appears on the complete outline of the guide sign extracted by the image processing algorithm. This discontinuity may be caused by physical occlusion, or by defects or displacement of the structure itself.

[0034] Collinearity is a geometric constraint used to assess whether the edge segments on both sides of a break point still macroscopically belong to the same straight line. The basic idea is that if a complete straight-edged object is occluded by another straight-edged object, then although the edges of the objects appear broken visually on both sides of the occluded area, their directions of extension should be consistent.

[0035] The spatial distribution center feature is a geometric parameter used to characterize the overall position of a guide sign on a two-dimensional image plane. Ideally, it can be understood as the geometric center or centroid of the sign's image region.

[0036] The position reference is a set of coordinates that are pre-measured and stored through a calibration program when the guide sign is installed and in good condition. This coordinate corresponds to a stable position of the guide sign in the camera coordinate system.

[0037] Fracture properties are a qualitative judgment of the root cause of edge fracture phenomena. In the context of this application, they are mainly divided into two types: external shading and structural damage.

[0038] First, images containing directional signs are acquired using cameras deployed within the subway station. These cameras can be fixed industrial cameras, such as megapixel network cameras with global shutters, to avoid motion blur caused by high-speed moving objects as trains pass. Image acquisition can be a periodic process, such as taking a still image every 5 seconds and transmitting it to a backend analysis server.

[0039] Next, after receiving the image, the analysis server needs to extract the edge contour information of the guide sign. A basic implementation involves first converting the image to grayscale, then applying a standard edge detection operator, such as the Sobel operator or the Laplacian operator, to initially identify areas in the image with drastic brightness changes; these areas typically correspond to the edges of objects. Subsequently, a series of morphological operations, such as erosion and dilation, are used to eliminate noise and connect discontinuous edge points, ultimately forming one or more closed or open contour curves. At this step, the system obtains a set of coordinates for a large number of pixels, which depicts the edges of the guide sign and its surrounding environment.

[0040] Next, it is necessary to identify broken edge segments from these edge contour information. In practice, this can be achieved by traversing the pixel sequence that constitutes the main contour of the guide marker. During this traversal, the Euclidean distance between two adjacent pixels is calculated. On a continuous edge, this distance is typically 1 pixel or √2 pixels. If the system detects that the distance between two consecutively recorded pixels is significantly greater than a preset small threshold, such as greater than 5 pixels, then a break can be considered to exist between these two points. These two points are the break points, and the continuous edge segments they occupy are defined as the target edge segments.

[0041] After identifying the target edge segment, it is necessary to extract the tangent vectors on both sides of the break point. This can be done by taking a small segment of consecutive pixels (e.g., a preset number of consecutive pixels) near each break point, for example, taking 10 pixels inward from the break point. Then, linear fitting is performed on the coordinates of these 10 points, for example, using the least squares method to fit a straight line. The direction vector of this fitted line can be used as the tangent vector at the break point. After obtaining the tangent vectors on both sides, it is necessary to determine whether they meet the collinearity condition. This can be done by checking whether the directions of the two vectors are basically parallel and whether the perpendicular distance between the lines they lie on is small enough. If the difference in the direction angles of the two vectors is very small and the two fitted lines almost overlap, then it can be considered that the collinearity condition is met. The significance of this step is that it can initially filter out edge breaks caused by bending, curling, or irregular damage to the sign itself, because in this case, the tangent directions on both sides of the break point usually have a clear inflection point, which does not meet the collinearity condition. Only those breaks that appear to be cut by a straight object will be sent to the next step of analysis.

[0042] Once it's determined that the tangent vectors meet the collinearity condition, it's necessary to obtain the spatial distribution center feature of the guide sign in the image. This can be achieved by directly calculating the geometric center of the region enclosed by all visible outline pixels belonging to the guide sign in the current image. Specifically, the average of the x and y coordinates of all these pixels is used to obtain a center point coordinate. This center point coordinate is then used as the spatial distribution center feature.

[0043] Finally, the calculated spatial distribution center feature is compared with a preset position benchmark. The position benchmark is a standard image taken during system initialization, showing the directional sign intact and without any obstructions. Its spatial distribution center is calculated using the same method, and its coordinates are stored. The comparison process involves calculating the pixel distance between the center point calculated in the current frame and this benchmark center point. Based on this distance, a preliminary determination of the breakage attribute is made. For example, an empirical deviation threshold, such as 15 pixels, can be set. If the calculated distance is less than 15 pixels, it can be preliminarily determined that the breakage is likely caused by external object occlusion. Conversely, if the distance is greater than 15 pixels, the system preliminarily determines that the overall position of the sign has shifted, which is highly likely due to structural problems such as loose hanging points; therefore, the breakage attribute is determined to be structural damage. Subsequently, the system will perform a secondary verification of this preliminary determination result to further improve accuracy.

[0044] Through the above series of steps, compared with simply observing images, the method of this application establishes a logical reasoning chain based on geometric analysis, which can make a reasonable preliminary judgment on the attributes of edge breakage based on a single frame image, significantly improving the reliability of automated monitoring.

[0045] Furthermore, to make the collinearity condition determination more accurate, the steps of extracting the tangent vectors on both sides of the break point of the target edge segment and determining whether the tangent vectors on both sides of the break point meet the collinearity condition can specifically include: Extract the tangent vectors of the target edge segment on both sides of the fracture location; Calculate the angle between the tangent vectors on both sides, and calculate the linear distance between the fracture points on both sides of the fracture location of the target edge segment; Determine whether the included angle is less than a preset angle threshold and whether the linear distance is less than a preset distance threshold; If the included angle is less than a preset angle threshold and the linear distance is less than a preset distance threshold, the tangent vectors on both sides of the fracture location are determined to be collinear. Otherwise, the tangent vectors on both sides of the fracture location are determined to be collinear.

[0046] After extracting the tangent vectors on both sides, the angle between them can be calculated using the dot product formula. Specifically, when extracting the tangent vectors on both sides of the break point of the target edge segment, the vector directions of the tangent vectors at the two break points are generally set to be directions that are closer to each other. For example, if the two break points are A and B, and the corresponding two tangent vectors are V1 and V2, then the vector direction of V1 points towards the side of break point B, and the vector direction of V2 points towards the side of break point A. The cosine of their angle is equal to the dot product of -V1 and V2 (or V1 and -V2) divided by the product of the magnitudes of the two vectors. The angle can then be obtained using the inverse cosine function. This angle intuitively reflects the consistency of the direction of the edges on both sides of the break. At the same time, the Euclidean distance between the coordinates of the two break points is directly calculated to obtain the linear distance, which reflects the size of the break area. Subsequently, these two calculated values ​​are compared with preset thresholds. For example, the angle threshold can be set to 5 degrees, and the linear distance threshold can be set to 30% of the diagonal length of the sign. The angle threshold is primarily used to ensure directional consistency, excluding corners or bends in the signage itself. The distance threshold, on the other hand, excludes cases where two unrelated edge segments, though oriented in the same direction, are far apart and are incorrectly associated. It also limits the scope of the analysis, focusing only on small, localized fractures. Only when the edges on both sides of the fracture point are not only almost oriented in the same direction, but the gap itself is not too large, is it considered a fracture possibly caused by occlusion or translation. This dual-condition quantitative judgment makes the collinearity determination process more standardized and reliable, avoiding fuzzy qualitative assessments.

[0047] Furthermore, to improve the accuracy of all subsequent analyses from the source, the step of extracting edge contour information for guide signs in the image may include: The image is processed for brightness equalization to eliminate the impact of changes in ambient light on image contrast. The image after brightness equalization is geometrically corrected using a pre-calibrated perspective correction matrix, transforming the image into an orthophoto image. Extract edge contour information from orthophotos.

[0048] This process involves preprocessing the original image. The lighting environment inside a subway station is very complex, with direct light from overhead lights, reflected light from the floor tiles, and dynamic light from surrounding advertising light boxes. These can create bright spots or shadows on the surface of directional signs, leading to localized overexposure or underexposure of the image and affecting the accuracy of edge detection. To address this, brightness equalization processing can be performed on the image first. A specific implementation method is to use adaptive histogram equalization techniques, such as the CLAHE algorithm. This algorithm does not use a single transform function on the entire image, but rather divides the image into several small regions and performs histogram equalization on each region separately. This effectively improves the local contrast of the image in different lighting areas, making the edges of the signs clearer and more discernible under various lighting conditions.

[0049] Furthermore, since cameras are typically mounted at an angle, the images of directional signs will exhibit perspective distortion, meaning that a rectangular sign will appear as a trapezoid in the image. This distortion severely interferes with subsequent geometric measurements. Therefore, geometric correction is required after brightness equalization. This necessitates a one-time calibration process. During system deployment, maintenance personnel need to identify the pixel coordinates of the four corner points of the directional sign in the image within the camera's field of view. Simultaneously, based on the actual dimensions of the sign, such as 1.5 meters long and 0.6 meters wide, and the correspondence between these four pairs of coordinates (or their proportional relationships) and the image coordinates, a 3x3 perspective correction matrix can be calculated. In each subsequent monitoring session, this fixed perspective correction matrix can be multiplied by the brightness-equalized image. Through a remapping operation, the trapezoidal sign image can be stretched back to a standard rectangle. This corrected image is the orthophoto, which provides a perspective view of the sign from directly above. Edge extraction performed on such a distortion-free, uniformly lit, ideal image greatly ensures the accuracy and consistency of the results.

[0050] Furthermore, to more accurately estimate the true position of the directional sign when it is occluded, the step of obtaining the spatial distribution center features of the directional sign in the image, when the tangent vectors meet the collinearity condition, may include: When the tangent vectors meet the collinearity condition, the edge segments on both sides of the break position are linearly extended along the vector direction of the tangent vectors to obtain the completed edge contour information. The geometric center of the region enclosed by the completed edge contour information is calculated to obtain the first reconstruction center, and the first reconstruction center is used as the spatial distribution center feature.

[0051] Specifically, once it's confirmed that the edges on both sides of the fracture meet the collinearity condition, it's assumed that these two edges were originally connected. Then, following the directions of the previously calculated tangent vectors on both sides, two virtual straight line segments are drawn from the two fracture points into the gap, until these two line segments intersect. These two virtual straight line segments, together with the original visible edge contour, constitute a logically complete and closed contour, i.e., the completed edge contour information. Then, the geometric center of the area enclosed by this completed contour is calculated, i.e., the first reconstruction center. Compared to directly calculating the center of the incomplete contour, this first reconstruction center, because it considers the expected shape of the obscured part, can more accurately reflect the theoretical center position of the directional sign when it is not obscured. For example, if the upper half of a rectangular sign is completely obscured, the center of the incomplete part will be located at the center of the lower half, while the first reconstruction center calculated after virtually completing and reconstructing the entire rectangle will accurately return to the geometric center of the entire rectangle. This method makes the extraction of spatial distribution center features more robust to occlusion.

[0052] Furthermore, to make the preliminary judgment more logical and reliable, the step of comparing the spatial distribution center features with a preset location benchmark and preliminarily determining the fracture attributes of the target edge segment based on the comparison results may include: Calculate the positional deviation of the first reconstruction center relative to the preset positional reference; Determine whether the positional deviation is within the preset allowable deviation range. If the positional deviation is within the allowable deviation range, determine that the spatial distribution center features conform to the positional reference. If the spatial distribution center features match the location benchmark, the fracture attribute is initially determined to be external shading; otherwise, the fracture attribute is initially determined to be structural damage.

[0053] This step refines and clarifies the aforementioned judgment process. After calculating the first reconstruction center, the Euclidean distance between this center point and the pre-stored position reference point is calculated; this distance is the position deviation. Then, this position deviation is compared with a preset allowable deviation range. This allowable deviation range is not a zero value, but a small interval with a certain tolerance, such as a circle with a radius of 10 pixels. The purpose of setting this range is to tolerate some normal, unstructured minor disturbances, such as slight, high-frequency vibrations caused by a train passing by, or unavoidable calculation errors of a few pixels during image processing. If the calculated position deviation falls within this allowable range, that is, the first reconstruction center and the reference point are almost in the same position, the system determines that the spatial distribution center feature conforms to the position reference. Based on this, it is inferred that since the actual center position of the sign has not moved, the break at the edge can only be caused by external object occlusion. Therefore, the break attribute is initially determined to be external occlusion. Conversely, if the position deviation exceeds this allowable range, for example, reaching 30 pixels, the spatial distribution center feature is determined to not conform to the position reference. This indicates that even after eliminating the impact of occlusion on the center point calculation through virtual completion, the center position of the sign still showed a significant shift. The only reasonable explanation is that the sign itself underwent physical displacement, i.e., a structural problem. Therefore, the fracture is initially determined to be structural damage. This judgment logic based on the deviation range provides a clear and quantitative decision-making basis for the automated system.

[0054] Based on the preliminary judgment method based on single-frame images, in order to cope with the complexity of dynamic changes in the subway environment, especially to more reliably distinguish between image jitter caused by instantaneous vibration and continuous displacement caused by structural loosening, this application further proposes a series of optimized technical solutions.

[0055] To obtain richer information at critical moments, the steps of acquiring images containing directional signs can specifically include: Real-time monitoring of train arrival and departure signals within subway stations; When no train entry or exit signal is detected, images are acquired at a preset first image acquisition frequency; When a train entering or leaving the station is detected, the first image acquisition frequency is increased to a preset second image acquisition frequency, and the image acquired at the second image acquisition frequency is stabilized to obtain a clear and stable image.

[0056] Specifically, trains entering and leaving stations are the most significant external factors causing vibrations in directional signs, and also the times when structural damage is most likely to occur or worsen. Therefore, this period requires focused monitoring. This can be achieved by connecting to the subway's Automatic Train Control (ATS) or signaling system via a network interface to obtain real-time train occupancy information for specific sections. When a train entering or leaving the platform or adjacent tunnel where the target directional sign is located is received, it is considered a high-risk monitoring period.

[0057] During stable periods, when no train arrival or departure signals are detected, the camera can use a lower initial image acquisition frequency, such as one frame per minute. This is sufficient for routine static inspection needs, while effectively saving network bandwidth and backend server storage and computing resources.

[0058] Once a train's arrival or departure signal is detected, the camera immediately increases its acquisition frequency to a higher secondary image acquisition frequency, such as 10 frames per second. This high-frequency acquisition can capture continuous dynamic changes during vibration. Since vibrations caused by the train can also cause camera shake, the high-frequency acquired image sequence may exhibit inter-frame jitter. Therefore, these images need to be stabilized. One feasible technique is to use Electronic Image Stabilization (EIS) algorithms. This algorithm analyzes the displacement of feature points (such as fixed screws or corners in the background) between consecutive frames, calculates the global motion (translation, rotation, scaling) of each frame relative to a reference frame, and then performs inverse compensation on the image through affine or perspective transformation, thereby generating a visually stable and clear image sequence.

[0059] Based on stable image sequences acquired at high frequencies, more refined and reliable secondary verification can be performed. Therefore, the method may further include: When a train arrival or departure signal is detected, images from multiple consecutive sampling periods before and after the time the train arrival or departure signal is detected are retrieved. For each sample period of the image, a set of corresponding candidate tangent clusters is generated to form the prediction envelope. The predicted envelopes corresponding to multiple consecutive sampling periods are superimposed in the same coordinate system, and the second reconstruction center is determined in the superimposed overlapping region.

[0060] This process aims to calculate a more stable center point than the first reconstruction center of a single frame by analyzing multiple frames of images over a short period of time. Specifically, when a train enters or leaves the station, the system retrieves a video stream before and after the event, for example, from 1 second before the event to 3 seconds after the event, totaling 40 frames (at a capture frequency of 10fps).

[0061] For each of the 40 image frames, the aforementioned edge extraction and break detection are performed. For each identified target edge segment, when calculating the tangent at its break point, considering that minor vibrations may cause slight oscillations in the tangent direction, instead of calculating only one optimal tangent, a cluster of candidate tangents is generated. This can be achieved by generating several candidate tangents with a preset small angular deviation (e.g., ±0.5 degrees, ±1 degree) from the optimal tangent vector after calculation. This set of tangents constitutes the cluster of candidate tangents. Extending each tangent in this cluster yields a narrow, elongated band, i.e., the prediction envelope. This envelope represents the set of all possible positions and directions of the occluded edge under the vibration state of the current frame, reflecting the measurement uncertainty.

[0062] Next, the predicted envelopes generated from each of the 40 frames are superimposed in the same perspective-corrected coordinate system. Because the movement of external obstructions (such as passengers) is relatively rapid and random, their positions and shapes vary significantly across different frames, resulting in a more discrete distribution of the corresponding predicted envelopes after superposition. However, the structure of the directional signs themselves is fixed, and even under vibration, the range of their edge swing is relatively limited. Therefore, the predicted envelopes representing the actual sign edges will highly overlap within a very small area after superposition. By analyzing the pixel density of this superimposed image, the region with the highest overlap density is found, and the geometric center of this region is calculated. This center is defined as the second reconstruction center. This second reconstruction center is based on the statistical average of information from multiple frames over a period of time. It effectively filters out high-frequency vibrations and accidental noise from single-frame images, and its position is more stable and reliable than the first reconstruction center calculated from any single-frame image.

[0063] After obtaining the second reconstruction center, the steps for secondary verification of fracture properties can specifically include: Obtain the movement trajectory of the second reconstruction center; If the movement trajectory shows unidirectional irreversible slippage, the preliminary judgment result will be uniformly adjusted to structural damage; Otherwise, the preliminary judgment will be upheld.

[0064] Specifically, a series of second reconstruction center points can be calculated continuously within multiple time windows (e.g., calculating one second reconstruction center every 2 seconds). Connecting these center points in chronological order constitutes the movement trajectory of the second reconstruction center. This trajectory reveals the macroscopic movement trend of the guide sign over a period of time. At this point, pattern recognition of the trajectory's shape is required.

[0065] If the trajectory points jump randomly and disorderly within a small range, or exhibit the characteristic of briefly deviating and then returning to their original position, this closely matches the behavior pattern of random movement or brief passage of external obstructions. In this case, the system considers there to be no evidence of permanent structural displacement and therefore maintains the initial assessment of external obstruction.

[0066] However, if the trajectory point exhibits a continuous movement in a certain direction (e.g., continuously downwards or to the left) without any rebound, this constitutes unidirectional irreversible slippage. This movement pattern is a typical characteristic of structural damage; for example, a suspension nut gradually loosens due to vibration, causing a sign to slowly and continuously sink or tilt. This irreversible slippage cannot be simulated by temporary external occlusion. Once this trajectory pattern is identified, it will have extremely high confidence to overturn any preliminary judgment based on static images. Regardless of whether the previous preliminary judgment was external occlusion or uncertain, it will be uniformly adjusted to structural damage, and the highest level of alarm will be triggered immediately.

[0067] Finally, to handle an extremely rare but possible special case, the steps following the secondary verification of fracture properties also include: When the movement trajectory exhibits unidirectional irreversible slippage, extract the local surface texture features of the target edge segment at the break location; Calculate the texture similarity between local surface texture features and pre-stored standard surface texture features; When the texture similarity of at least one target edge segment at the fracture location is lower than the preset similarity threshold, the judgment result of the secondary verification is maintained as structural damage; When the texture similarity of all target edge segments at the fracture location is not lower than the similarity threshold, the judgment result of the secondary verification is adjusted to external occlusion with slip features.

[0068] This step addresses a scenario where a foreign object (such as a lightweight plastic bag or strip of cloth blown by the wind and hanging on the edge of a sign) is partially fixed to the sign and slowly slides down the edge due to gravity or continuous airflow. In this case, the object's trajectory will also exhibit unidirectional irreversible sliding, which may be misjudged as structural damage.

[0069] To distinguish this situation, upon detecting irreversible slippage, it locates the edge break in the image and extracts a local image patch of the visible object surface at the break. Then, it calculates the surface texture features of this image patch. Texture features can be represented using Local Binary Pattern (LBP), Gray-Level Co-occurrence Matrix (GLCM), or feature vectors extracted by a pre-trained deep learning network. Simultaneously, the system pre-stores the standard surface texture features of the guide sign under normal conditions (e.g., the texture of a brushed metal panel, or the color and uniformity features of a painted surface).

[0070] The similarity between the currently extracted local texture features and pre-stored standard texture features is calculated (e.g., comparing the chi-square distance of the LBP histogram or calculating the cosine similarity of the feature vectors). If the similarity is very low (below a preset similarity threshold, such as 0.4), it indicates that the surface of the object exposed at the fracture point is made of a completely different material from the sign itself, confirming that the fracture was caused by structural displacement of the sign itself (e.g., separation of the boom from the sign body). In this case, the judgment of structural damage is maintained.

[0071] Conversely, if the similarity is very high (not lower than the similarity threshold), it means that the surface texture of the object at the break point is highly consistent with the texture of the sign itself. This is illogical in the case of structural damage (because damage exposes the internal structure or creates new fracture surfaces, changing the texture). The only reasonable explanation is that the break is actually part of an obstruction, and this obstruction happens to be very similar in material or color to the sign's surface, and is slowly sliding. Therefore, the judgment will be corrected to an external obstruction with sliding characteristics. This avoids false alarms for this extremely rare case and improves the accuracy of monitoring.

[0072] Secondly, referring to Figure 2 This application also provides an image recognition-based wayfinding sign integrity monitoring system for performing the steps in any of the foregoing methods, the system comprising: Image acquisition module 210 is used to acquire an image containing directional signs; Edge extraction module 220 is used to extract the edge contour information of guide signs in the image; The fracture recognition module 230 is used to identify edge segments with fractures in the edge contour information as target edge segments; The comparison module 240 is used to extract the tangent vectors on both sides of the fracture position of the target edge segment and determine whether the tangent vectors on both sides of the fracture position meet the collinearity condition. Feature extraction module 250 is used to obtain the spatial distribution center features of the guide sign in the image when the tangent vectors meet the collinearity condition; The judgment and verification module 260 is used to compare the spatial distribution center features with the preset position benchmark, preliminarily determine the fracture attributes of the target edge segment based on the comparison results, and perform secondary verification on the preliminary judgment results.

[0073] The image acquisition module 210, in terms of hardware, can correspond to one or more network cameras deployed in the subway station, along with their corresponding drivers and data receiving interfaces. This module is responsible for capturing images at a specified frequency according to control instructions and outputting them in a digital format (such as JPEG or raw data stream).

[0074] The edge extraction module 220 can be a software functional unit running on a backend server. It receives raw image data from the image acquisition module 210 and executes a series of image preprocessing algorithms, such as grayscale conversion, brightness equalization, perspective correction, and the core edge detection algorithm, ultimately outputting a set of pixel coordinates that can describe the outline of the guide sign.

[0075] The fracture recognition module 230 receives the contour data output by the edge extraction module 220, locates the discontinuities on the contour by traversing and calculating the distance between points, and outputs relevant information of the target edge segment, such as the coordinates of the fracture point and the edge segment data to which it belongs.

[0076] The comparison module 240 receives the output of the fracture identification module 230, calculates the tangent vectors on both sides of the fracture point, and determines whether the collinearity condition is met based on quantitative indicators such as the included angle and distance.

[0077] The feature extraction module 250 is responsible for calculating spatial features after the comparison module 240 confirms that the collinearity condition is met. It can calculate the geometric center of the incomplete contour according to different strategies, or calculate the first reconstructed center after performing virtual completion, or even calculate a more stable second reconstructed center based on the temporal image sequence.

[0078] The judgment and verification module 260 receives the center feature calculated by the feature extraction module 250 and compares it with the pre-stored position benchmark to complete the preliminary static judgment. Furthermore, it is also responsible for analyzing the movement trajectory of the center feature, performing dynamic behavior analysis, completing secondary verification, and possibly combining texture analysis for final refined identification. Finally, the module outputs the final judgment conclusion on the fracture attributes, such as external occlusion or structural damage, and can trigger corresponding alarms or log entries accordingly.

[0079] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for monitoring the integrity of wayfinding signs based on image recognition, used to monitor the integrity of wayfinding signs in subway stations, characterized in that, The steps of this method include: Obtain an image containing the guide sign; Extract the edge contour information of the guide sign in the image; The edge segments with breaks are identified in the edge contour information and designated as target edge segments. Extract the tangent vectors of the target edge segment on both sides of the fracture location, and determine whether the tangent vectors on both sides of the fracture location meet the collinearity condition; When the tangent vectors meet the collinearity condition, the spatial distribution center features of the guide markers in the image are obtained; The spatial distribution center features are compared with a preset position benchmark. Based on the comparison results, the fracture attributes of the target edge segment are initially determined, and the initial determination results are then verified a second time.

2. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 1, characterized in that, The step of extracting the tangent vectors of the target edge segment on both sides of the fracture location and determining whether the tangent vectors on both sides of the fracture location meet the collinearity condition includes: Extract the tangent vectors of the target edge segment on both sides of the fracture location; Calculate the angle between the tangent vectors on both sides, and calculate the linear distance between the break points on both sides of the break location of the target edge segment; Determine whether the included angle is less than a preset angle threshold and whether the linear distance is less than a preset distance threshold; If the included angle is less than a preset angle threshold and the linear distance is less than a preset distance threshold, it is determined that the tangent vectors on both sides of the fracture location meet the collinearity condition. Otherwise, the tangent vectors on both sides of the fracture location are determined to be collinear.

3. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 1, characterized in that, The step of extracting the edge contour information of the guide sign in the image includes: The image is subjected to brightness equalization processing to eliminate the impact of ambient light changes on image contrast; The image after brightness equalization is geometrically corrected using a pre-calibrated perspective correction matrix, transforming the image into an orthophoto image. The edge contour information is extracted from the orthophoto image.

4. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 1, characterized in that, The step of obtaining the spatial distribution center feature of the guide sign in the image when the tangent vector meets the collinearity condition includes: When the tangent vector meets the collinearity condition, the edge segments on both sides of the break position are linearly extended along the vector direction of the tangent vector to obtain the completed edge contour information. The geometric center of the region enclosed by the completed edge contour information is calculated to obtain the first reconstruction center, and the first reconstruction center is used as the spatial distribution center feature.

5. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 4, characterized in that, The step of comparing the spatial distribution center features with a preset positional benchmark and preliminarily determining the fracture attribute of the target edge segment based on the comparison result includes: Calculate the positional deviation of the first reconstruction center relative to the preset positional reference; Determine whether the positional deviation is within a preset allowable deviation range. When the positional deviation is within the allowable deviation range, determine that the spatial distribution center feature conforms to the positional reference. If the spatial distribution center feature matches the location benchmark, the fracture attribute is preliminarily determined to be external shading; otherwise, the fracture attribute is preliminarily determined to be structural damage.

6. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 1, characterized in that, The step of acquiring the image containing the guide marker includes: Real-time monitoring of train arrival and departure signals within the aforementioned subway station; When no train entry or exit signal is detected, the image is acquired at a preset first image acquisition frequency; When the train's entry or exit signal is detected, the first image acquisition frequency is increased to a preset second image acquisition frequency, and the image acquired at the second image acquisition frequency is subjected to image stabilization processing to obtain a clear and stable image.

7. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 6, characterized in that, The method also includes: When the train's entry or exit signal is detected, the images from multiple consecutive sampling periods before and after the time when the train's entry or exit signal is detected are retrieved. For the target edge segment in the image retrieved for each sampling period, a set of corresponding candidate tangent clusters is generated to form a prediction envelope. The predicted envelopes corresponding to multiple consecutive sampling periods are superimposed in the same coordinate system, and a second reconstruction center is determined within the superimposed overlapping region.

8. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 7, characterized in that, The step of performing secondary verification of the fracture properties includes: Obtain the movement trajectory of the second reconstruction center; If the movement trajectory exhibits unidirectional irreversible slippage, the preliminary judgment result will be uniformly adjusted to structural damage; Otherwise, the preliminary determination remains unchanged.

9. The method for monitoring the integrity of wayfinding signs based on image recognition according to claim 8, characterized in that, The steps following the secondary verification of the fracture properties also include: When the movement trajectory exhibits unidirectional irreversible slippage, the local surface texture features of the target edge segment at the fracture location are extracted; Calculate the texture similarity between the local surface texture features and the pre-stored standard surface texture features; When the texture similarity of at least one target edge segment at the fracture location is lower than the preset similarity threshold, the judgment result of the secondary verification is maintained as structural damage; When the texture similarity of all target edge segments at the fracture location is not lower than the similarity threshold, the judgment result of the secondary verification is adjusted to external occlusion with slip features.

10. A wayfinding sign integrity monitoring system based on image recognition, used to perform the steps of the method according to any one of claims 1 to 9, characterized in that, include: The image acquisition module is used to acquire images containing directional signs; The edge extraction module is used to extract the edge contour information of the guide sign in the image; A fracture identification module is used to identify edge segments with fractures in the edge contour information as target edge segments. The comparison module is used to extract the tangent vectors on both sides of the fracture position of the target edge segment and determine whether the tangent vectors on both sides of the fracture position meet the collinearity condition. The feature extraction module is used to obtain the spatial distribution center features of the guide sign in the image when the tangent vector meets the collinearity condition; The determination and verification module is used to compare the spatial distribution center features with a preset position benchmark, preliminarily determine the fracture attribute of the target edge segment based on the comparison result, and perform a secondary verification on the preliminary determination result.

Images