Subway display screen defect detection method based on image processing

Abstract

This invention discloses a defect detection method for subway display screens based on image processing. The method involves acquiring images of the display screen, performing perspective correction and display area localization, and constructing a unified pixel coordinate system. Within this coordinate system, a detection unit grid is divided, and multi-dimensional features such as brightness, color, and texture are extracted to form a feature vector set. A statistical benchmark is established based on normal samples, and the deviation degree of each detection unit is calculated to generate a deviation distribution map. The deviation results are then filtered and spatial consistency constraints are applied to obtain candidate anomaly locations. Multi-scale spatial statistical methods are combined to analyze the aggregation characteristics of candidate locations, forming multi-scale response information. Based on the deviation degree and spatial aggregation characteristics, a joint judgment is made to generate defect areas and size levels. Finally, the image data, defect areas, and structured results are associated, encapsulated, and output to achieve the localization and classification of display screen defects.

Description

Technical Field

[0001] This invention relates to the field of image processing and intelligent detection technology, and in particular to a defect detection method for subway display screens based on image processing. Background Technology

[0002] With the informatization and intelligentization of urban rail transit systems, subway onboard displays, as important information dissemination carriers, have made automatic detection of their operational status a crucial link in ensuring operational safety and passenger service quality. Currently, defect detection of onboard displays mostly adopts methods based on image acquisition and simple image analysis, achieving anomaly identification through edge detection, threshold segmentation, or single feature judgment, and in some scenarios, combining statistical models or deep learning methods for auxiliary detection.

[0003] Existing technologies still have significant shortcomings in complex operating environments. On the one hand, traditional methods often rely on single brightness or color features for discrimination, lacking the ability to comprehensively model texture and multi-dimensional features, making it difficult to accurately distinguish between display anomalies and normal fluctuations, resulting in prominent false positives and false negatives. On the other hand, methods based on global statistics or simple thresholds ignore the spatial clustering characteristics of defects, failing to effectively identify local structural anomalies, especially under noise interference or local brightness unevenness, leading to poor stability of detection results. Furthermore, existing methods typically lack multi-scale spatial analysis mechanisms, making it difficult to effectively distinguish defects of different sizes, and also difficult to achieve precise location and hierarchical representation of defect areas, failing to meet the application requirements of automotive display systems for high-precision, structured detection results.

[0004] Therefore, how to provide a defect detection method for subway displays based on image processing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention proposes a defect detection method for subway display screens based on image processing and spatial statistical analysis. By constructing multi-dimensional feature vectors and combining statistical deviation analysis and multi-scale spatial aggregation modeling, the method can detect and classify abnormal areas of the display screen.

[0006] The image processing-based defect detection method for subway displays according to embodiments of the present invention includes the following steps: S1. Acquire the display screen image, complete perspective correction and display area positioning, and output the aligned display image and corresponding pixel coordinate system; S2. Construct a detection unit grid based on the pixel coordinate system to generate a set of multivariable feature vectors; S3. Based on normal samples, establish a mean vector and covariance matrix, calculate the Hotelling T-squared deviation for the multivariate feature vector set, and generate a deviation map corresponding to the detection unit. S4. Perform threshold filtering and non-maximum suppression on the deviation map to generate a candidate point set, and bind the detection unit coordinates, deviation amount and detection unit neighborhood index to each candidate point; S5. Using the candidate point set as spatial point input, calculate the Ripley K function under the preset multi-scale distance set to obtain the clustering intensity sequence at each scale, and generate a multi-scale clustering response vector for each candidate point. S6. Calculate the anomaly comprehensive score based on the deviation and multi-scale clustered response vector, determine the defect seed point, perform connectivity aggregation on the defect seed point based on the detection unit neighborhood index to generate a defect region mask, and determine the defect size level according to the dominant scale in the clustering intensity sequence of the defect region at each scale. S7. The aligned display image, deviation map, defect area mask, defect size level and structured defect results are associated and encapsulated to form a record entry, written to the vehicle data recorder and sent to the monitoring terminal simultaneously.

[0007] Optionally, S1 includes: S11. Trigger the vehicle-mounted image acquisition device to acquire the original image on the display screen, write the original image to the frame buffer, and read the acquisition timestamp and camera calibration parameters to generate an acquisition frame containing the original image and calibration parameters. S12. Denoise and contrast normalize the acquired frames to generate a preprocessed image; extract edge maps from the preprocessed image and perform edge connection and break repair processing on the edge maps to generate a binary edge map. S13. Perform line detection in the edge binary map to obtain a set of candidate lines for the border, and calculate the set of intersection points based on the set of candidate lines for the border; filter the set of corner points of the display area in the set of intersection points using quadrilateral geometric consistency constraints, and sort the set of corner points according to the geometric orientation of the display area to form a corner point sequence; S14. Construct a set of vertex coordinates of the target rectangle based on the corner point sequence, establish the correspondence between the corner point sequence and the set of vertex coordinates of the target rectangle, and solve for the perspective transformation matrix; S15. Perform reverse mapping on the coordinates of each pixel within the target rectangle, and use the perspective transformation matrix to map the target pixel coordinates to the original image coordinates; perform interpolation and resampling on the non-integer coordinates obtained by mapping to form an aligned display image filled pixel by pixel; perform validity judgment and pixel filling control on the mapped coordinates that exceed the effective range of the original image. S16. Establish a pixel coordinate system with the top left corner pixel of the aligned display image as the origin, and define the horizontal direction as the first coordinate axis and the vertical direction as the second coordinate axis; assign a two-dimensional coordinate index to each pixel in the aligned display image, and generate a mapping table between the two-dimensional coordinate index and the linear address index; S17. Bind and encapsulate the aligned display image with the pixel coordinate system data structure, and output an alignment result data packet containing the aligned display image, pixel coordinate index mapping table and perspective transformation matrix. Optionally, S2 includes: S21. Receive the aligned display image and pixel coordinate system, extract the effective pixel set of the display area based on the pixel coordinate system, generate the detection unit grid according to the preset grid rules, and write the detection unit number, boundary coordinates and center coordinates for each detection unit. S22. For each detection unit, traverse the set of valid pixels according to the boundary coordinates to form a pixel sequence of the detection unit, and simultaneously form a relative coordinate sequence of pixels within the detection unit. S23. Calculate the brightness statistics for the pixel sequence of the detection unit. The brightness statistics include the average brightness and the brightness dispersion, which satisfy the following: ; in, To detect grayscale values ​​in the pixel sequence of the detection unit, The number of pixels in the detection unit; S24. Calculate the chromaticity statistics for the pixel sequence of the detection unit and generate the chromaticity feature sub-vector; S25. Calculate texture statistics for the pixel sequence of the detection unit and generate texture feature sub-vectors; S26. The luminance statistics, chromaticity feature vector and texture feature vector are concatenated in a fixed dimension order to form a multivariate feature vector, and the multivariate feature vector is bound to the detection unit number, boundary coordinates and center coordinates to form a detection unit feature record. S27. Collect all feature records of detection units in order of detection unit number to form a multivariable feature vector set, and simultaneously generate an index mapping table of the multivariable feature vector set to the pixel coordinate system.

[0008] Optionally, S3 includes: S31. Obtain the aligned display image of the display screen in a defect-free state, generate the corresponding multivariate feature vector set, and write the multivariate feature vector of each normal sample into the normal sample feature library according to the detection unit number. S32. Extract all multivariate feature vectors under the same detection unit number from the normal sample feature library, calculate the mean vector and covariance matrix corresponding to the detection unit number, and write the mean vector and covariance matrix into the statistical benchmark table; the statistical benchmark table is indexed according to the detection unit number; S33. Divide the multivariate feature vector into N feature groups according to luminance statistics, chroma feature sub-vectors, and texture feature sub-vectors; calculate the group mean and group covariance for each feature group to obtain a set of group covariance matrices; calculate the inter-group correlation matrix for the group mean-centered sequence of different feature groups to obtain the inter-group correlation matrix; construct a joint covariance matrix based on the set of group covariance matrices and the inter-group correlation matrix, and write the joint covariance matrix into the statistical benchmark table; S34. Calculate the Hotelling T-squared deviation: Receive the set of multivariate feature vectors corresponding to the image to be detected. Read the mean vector and joint covariance matrix from the statistical benchmark table one by one according to the detection unit number. Calculate the Hotelling T-squared deviation for each detection unit. The Hotelling T-squared deviation satisfies the following: ; in, For the first Multivariate feature vectors of each detection unit For the statistical benchmark table and the first The mean vector corresponding to each detection unit number For the statistical benchmark table and the first The inverse of the joint covariance matrix corresponding to each detection unit number; [This is followed by a list of all detection units.] A deviation sequence is formed according to the detection unit number; S35. Based on the established detection unit grid and the center coordinates of the detection unit, each of the deviation sequences... Write the grid positions corresponding to the detection units to generate an offset map that corresponds one-to-one with the detection units, and establish an index relationship between the offset map and the pixel coordinate system; S36. Using the deviation map as input, perform structural analysis to obtain the deviation structure map. The structural analysis includes: S361, Threshold Segmentation: Apply a statistical threshold to the deviation map to generate a binary outlier map. In the binary outlier map, the values ​​of the outlier grid are marked as outliers, and the values ​​of the non-outlier grids are marked as non-outliers. S362, Neighborhood Counting Mapping: Based on the detection unit grid, a fixed neighborhood template is established. For each grid in the binary anomaly map, the number of abnormal grids within the coverage area of ​​its neighborhood template is counted to generate a neighborhood anomaly counting map. S363, Structural Screening: The neighborhood anomaly count map and the binary anomaly map are jointly screened. Grids that meet the condition that "the current grid is anomaly and the neighborhood anomaly count reaches the preset cluster judgment condition" are marked as in-cluster anomaly grids, and the remaining anomaly grids are marked as isolated anomaly grids, forming a point-cluster separation map. S364. Connectivity Extraction: On the point cluster separation graph, perform connectivity marking on abnormal meshes within the cluster to obtain a set of candidate structural domains; for each candidate structural domain, calculate the domain boundary, the number of domain meshes, the domain circumscribed rectangle and the main direction, and generate a structural domain attribute table. S365. Associate and encapsulate the point cluster separation graph, candidate structural domain set, and structural domain attribute table with the deviation graph, and output the deviation structural graph data package. S37. Link and output the deviation graph, deviation sequence, statistical benchmark table, and deviation structure graph data package. Optionally, S4 includes: S41. Receive the deviation map, the detection unit grid, the detection unit center coordinates and the detection unit adjacency table, wherein the detection unit adjacency table records the set of first-order adjacent detection unit numbers corresponding to each detection unit. S42. Read the Hotelling T-squared deviation history sequence of each detection unit in a defect-free state from the normal sample feature library. For each detection unit, calculate the upper limit threshold of the history sequence. The upper limit threshold is determined by sorting the history sequence from smallest to largest and taking the deviation value at a preset ranking position. Compare the deviation of the detection unit in the deviation map to be detected with the upper limit threshold of the corresponding detection unit. Detection units with deviations not less than the upper limit threshold are marked as initial abnormal units, forming an initial abnormal unit set. S43. Perform non-maximum suppression on the initial abnormal unit set: For each initial abnormal unit, read its first-order adjacent detection unit number set and obtain the deviation of adjacent detection units. If the deviation of the current initial abnormal unit is less than the deviation of any adjacent detection unit, delete the initial abnormal unit. Repeat the above comparison for the retained initial abnormal units until there are no initial abnormal units that can be deleted, and obtain the local extreme value abnormal unit set. S44. Perform neighborhood consistency screening on the set of local extreme value anomalies: For each local extreme value anomaly, read its first-order neighbor detection unit number set and second-order neighbor detection unit number set, and count the number of first-order neighbor detection units belonging to the initial anomaly unit set and the number of second-order neighbor detection units belonging to the initial anomaly unit set respectively; when the number of anomalies in the first-order neighbor detection units and the number of anomalies in the second-order neighbor detection units simultaneously meet the preset quantity condition, the local extreme value anomaly unit is retained; otherwise, the local extreme value anomaly unit is deleted. S45. Generate candidate points for the retained local extreme value anomaly units: use the center coordinates of the detection unit of the local extreme value anomaly unit as the coordinates of the candidate point, use the deviation of the detection unit as the deviation of the candidate point, and use the detection unit number as the candidate point identifier to form a candidate point record; S46. Bind a detection unit neighborhood index to each candidate point record: Taking the detection unit number corresponding to the candidate point identifier as the center, read the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers, and write the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers into the candidate point record as the detection unit neighborhood index. S47. Collect all candidate point records to form a candidate point set, and establish a mapping table from candidate point identifiers to detection unit grid positions, so that each candidate point in the candidate point set can be traced back to the grid position and detection unit boundary coordinates in the deviation map. Optionally, S5 includes: S51. Receive the candidate point set and the center coordinates of the detection unit bound to it, extract the two-dimensional coordinates of all candidate points to form a spatial point set; receive the boundary of the display area corresponding to the aligned display image, read the boundary coordinates of the four boundaries of the display area in the pixel coordinate system, and form a spatial statistical observation window; S52. Construct a distance scale sequence: Read the multi-scale distance set from the distance scale configuration table and sort it in ascending order to obtain the distance scale sequence; establish a squared distance threshold for each distance scale in the distance scale sequence and write it into the scale index table; S53, Single-pairing-multi-scale cumulative point-pair counting construction process: S531. Establish a grid index for the spatial point set: Based on the detection unit grid, write each candidate point into the point list of the corresponding grid unit according to its detection unit number, and generate an index structure from grid unit to point list. S532. Determine the grid search range corresponding to the maximum distance scale: Using the maximum distance scale as the upper limit, calculate the grid expansion range it covers in the detection unit grid coordinate system, and write the expansion range into the grid search template; S533. Perform a neighborhood grid scan for each candidate point: with the grid cell where the candidate point is located as the center, enumerate the adjacent grid cells according to the grid search template, read the point list of the adjacent grid cells, and form point pairs with the current candidate point one by one. S534. Calculate the squared distance for each point pair: Calculate the horizontal and vertical differences of the two-dimensional coordinates of the candidate points at both ends of the point pair to obtain the squared distance; discard the point pair when the squared distance is greater than the squared threshold of the maximum distance scale; write the squared distance into the point pair distance record table when the squared distance is not greater than the squared threshold of the maximum distance scale. S535. Sort the point-to-point distance record table by distance from smallest to largest and write it into the distance count histogram table in sequence; perform prefix accumulation on the distance count histogram table according to the distance scale sequence to obtain the cumulative count of point pairs corresponding to each distance scale. After one point pair construction and one sorting, obtain the cumulative count of point pairs for all distance scales. S54, Explicit Overlap Ratio—Point-to-Point Weighted Boundary Correction Construction Process: S541. Calculate the boundary distance for each candidate point: Read the two-dimensional coordinates of the candidate point and the four boundary coordinates of the observation window, calculate the shortest distance from the candidate point to each boundary, and form a record of the candidate point boundary distances. S542. Construct an overlap ratio for each candidate point and each distance scale: When the shortest distance from a candidate point to all four boundaries is not less than the distance scale, the overlap ratio of the candidate point at the distance scale is recorded as 1; when there is at least one boundary that makes the shortest distance less than the distance scale, construct an overlap ratio between the candidate point's circular neighborhood and the observation window. The overlap ratio is obtained through the following steps: with the candidate point as the center and the distance scale as the radius, generate a set of circular sampling points on the circumference according to the angle step table; determine whether each circular sampling point falls within the observation window; use the ratio of the number of circular sampling points falling within the observation window to the total number of circular sampling points as the overlap ratio of the candidate point at the distance scale. S543. Perform point-to-point weight generation for overlap ratio: For each candidate point and each distance scale, take the reciprocal of its overlap ratio as the boundary correction weight and write it into the "candidate point number - distance scale - weight" ternary table. S55. Calculate the Ripley K-function value sequence: Using the observation window area and the number of candidate points as normalization parameters, based on the cumulative count of point pairs at each distance scale, and combined with boundary correction weights, calculate the Ripley K-function value for each distance scale. The Ripley K-function satisfies... ; in, For the observation window area, The number of candidate points. For the first With the Distance between candidate points As an indication of conditions, For the ternary table Candidate points at the distance scale The lower boundary correction weights; corresponding to all distance scales Arranged according to distance scale sequence to form cluster intensity sequence; S56. Generate multi-scale clustered response vectors for candidate points: For each candidate point, count the number of neighboring candidate points within the distance scale sequence one by one, and apply boundary correction weights to the candidate point during the count. Concatenate the obtained multi-scale neighborhood statistics according to the distance scale sequence to form the multi-scale clustered response vector of the candidate point. Bind the multi-scale clustered response vector with the detection unit coordinates, deviation, and detection unit neighborhood index in the candidate point record to form a candidate point clustered attribute record. S57. Collect all candidate point aggregation attribute records to form a candidate point multi-scale aggregation response set, and output the aggregation intensity sequence and the candidate point multi-scale aggregation response set.

[0009] Optionally, S6 includes: S61. Receive the candidate point set and its bound detection unit number, detection unit center coordinates, deviation amount and first-order and second-order detection unit neighborhood index, and receive the candidate point multi-scale aggregation response vector and distance scale sequence, and establish a candidate point index table. S62. Construct a "Detection Unit Number - Statistical Threshold" table based on the historical sequence of deviation in normal samples, and construct a "Distance Scale - Spatial Threshold" table based on the historical sequence of aggregation response at each distance scale in normal samples; S63. For each candidate point, compare the aggregation response with the spatial threshold on a scale-by-scale according to the distance scale to generate a list of passing scales. Select the scale with the largest ratio of aggregation response to spatial threshold from the list of passing scales and lock it as the main scale. Calculate the anomaly comprehensive score. When the deviation of the candidate point is not less than the corresponding statistical threshold and the list of passing scales is not empty, the candidate point is determined as a defect seed point and the main scale index is recorded. S64. Using the defect seed point as input, perform connectivity aggregation based on the neighborhood index of the detection unit. During the aggregation process, only neighborhood units with the same main scale index and whose anomaly comprehensive score is not greater than the score corresponding to the current extended unit are merged into the same defect region. Output the set of detection unit numbers of the defect region. S65. Generate a defect region mask based on the boundary coordinates of the detection unit, and determine the defect size level by the position of the main scale index of the defect region in the distance scale sequence. Output the list of defect seed points, the defect region mask, and the defect size level.

[0010] Optionally, S7 includes: S71. Receive the aligned display image, deviation map, defect area mask, defect size level and structured defect results, and generate a record entry containing the record entry identifier and acquisition timestamp. S72. Traverse the region identifiers in the defect region mask, calculate the coordinates of the region's outer rectangle and the region's center coordinates based on the corresponding detection unit number set, and construct a defect result list. S73. Establish an index association between the region identifiers in the defect results list and the corresponding positions in the defect region mask, deviation map, and aligned display image to form an association index table; S74. Serialize and encapsulate the aligned display image, deviation map, defect area mask, defect size level, defect result list and associated index table according to the preset data segment directory, and generate a record entry data body containing record entry identifier, acquisition timestamp and pixel coordinate system identifier. S75. Write the record item data body into the vehicle data recorder and establish the corresponding index record. At the same time, extract the defect result list to generate a monitoring message, and send it to the monitoring terminal synchronously through the vehicle communication link and record the sending status.

[0011] The beneficial effects of this invention are: This invention calculates the Hotelling T-squared deviation and performs joint statistical discrimination on multi-dimensional features such as brightness, color and texture in the detection unit. Compared with single feature or single threshold methods, it can more accurately characterize the degree of abnormality of the detection unit relative to normal samples, and improve the sensitivity and accuracy of preliminary screening of display defects.

[0012] This invention uses the Ripley K function to quantify the spatial clustering characteristics of candidate outliers at different distance scales. Compared with the method of judging based solely on isolated outliers, this method can more effectively distinguish random noise from real defect areas, improve the reliability of defect identification, and enhance the ability to distinguish defects of different sizes. Attached Figure Description

[0013] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the image processing-based defect detection method for subway displays proposed in this invention; Figure 2 This is a flowchart of the deviation map generation process based on Hotelling T-squared deviation proposed in this invention. Detailed Implementation

[0014] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0015] refer to Figures 1-2 A defect detection method for subway display screens based on image processing includes the following steps: S1. Acquire the display screen image, complete perspective correction and display area positioning, and output the aligned display image and corresponding pixel coordinate system; S2. Construct a detection unit grid based on the pixel coordinate system, extract the brightness statistics, chromaticity statistics and texture statistics of each detection unit, and generate a set of multivariate feature vectors. S3. Based on normal samples, establish a mean vector and covariance matrix, calculate the Hotelling T-squared deviation for the multivariate feature vector set, and generate a deviation map corresponding to the detection unit. S4. Perform threshold filtering and non-maximum suppression on the deviation map to generate a candidate point set, and bind the detection unit coordinates, deviation amount and detection unit neighborhood index to each candidate point; S5. Using the candidate point set as spatial point input, calculate the Ripley K function under the preset multi-scale distance set to obtain the clustering intensity sequence at each scale, and generate a multi-scale clustering response vector for each candidate point. S6. Calculate the anomaly comprehensive score based on the deviation amount and multi-scale clustered response vector. When the deviation amount exceeds the statistical threshold and the clustering intensity exceeds the spatial threshold at at least one distance scale, the corresponding candidate point is determined as the defect seed point. Based on the neighborhood index of the detection unit, perform connectivity aggregation on the defect seed point to generate a defect region mask, and determine the defect size level according to the dominant scale in the clustering intensity sequence of the defect region at each scale. S7. The aligned display image, deviation map, defect area mask, defect size level and structured defect results are associated and encapsulated to form a record entry, written to the vehicle data recorder and sent to the monitoring terminal simultaneously.

[0016] In this embodiment, S1 includes: S11. Trigger the vehicle-mounted image acquisition device to acquire the original image on the display screen, write the original image to the frame buffer, and read the acquisition timestamp and camera calibration parameters to generate an acquisition frame containing the original image and calibration parameters. S12. Denoise and contrast normalize the acquired frames to generate a preprocessed image; extract edge maps from the preprocessed image and perform edge connection and break repair processing on the edge maps to generate a binary edge map. S13. Perform line detection in the edge binary map to obtain a set of candidate lines for the border, and calculate the set of intersection points based on the set of candidate lines for the border; filter the set of corner points of the display area in the set of intersection points using quadrilateral geometric consistency constraints, and sort the set of corner points according to the geometric orientation of the display area to form a corner point sequence; S14. Construct a set of vertex coordinates of the target rectangle based on the corner point sequence, establish the correspondence between the corner point sequence and the set of vertex coordinates of the target rectangle, and solve for the perspective transformation matrix; S15. Perform reverse mapping on the coordinates of each pixel within the target rectangle, and use the perspective transformation matrix to map the target pixel coordinates to the original image coordinates; perform interpolation and resampling on the non-integer coordinates obtained by mapping to form an aligned display image filled pixel by pixel; perform validity judgment and pixel filling control on the mapped coordinates that exceed the effective range of the original image. S16. Establish a pixel coordinate system with the top left corner pixel of the aligned display image as the origin, and define the horizontal direction as the first coordinate axis and the vertical direction as the second coordinate axis; assign a two-dimensional coordinate index to each pixel in the aligned display image, and generate a mapping table between the two-dimensional coordinate index and the linear address index; S17. Bind and encapsulate the aligned display image with the pixel coordinate system data structure, and output an alignment result data packet containing the aligned display image, pixel coordinate index mapping table and perspective transformation matrix.

[0017] In this embodiment, S2 includes: S21. Receive the aligned display image and pixel coordinate system, extract the effective pixel set of the display area based on the pixel coordinate system, generate the detection unit grid according to the preset grid rules, and write the detection unit number, boundary coordinates and center coordinates for each detection unit. S22. For each detection unit, traverse the set of valid pixels according to the boundary coordinates to form a pixel sequence of the detection unit, and simultaneously form a relative coordinate sequence of pixels within the detection unit. S23. Calculate the luminance statistics for the pixel sequence of the detection unit. The luminance statistics include average luminance and luminance dispersion, and the average luminance and luminance dispersion satisfy the following conditions: ; in, To detect grayscale values ​​in the pixel sequence of the detection unit, The number of pixels in the detection unit; S24. Calculate chromaticity statistics for the pixel sequence of the detection unit. Specifically, perform unified color space mapping on the color components of the pixels in the detection unit, extract the hue component and saturation component, and calculate the hue mean, hue dispersion, saturation mean and saturation dispersion respectively to generate chromaticity feature sub-vectors. S25. Calculate texture statistics for the pixel sequence of the detection unit. Specifically, perform directional gradient calculation on the gray value within the detection unit, and calculate the average gradient magnitude, gradient magnitude dispersion, and gradient direction consistency index to generate texture feature sub-vectors. S26. The luminance statistics, chromaticity feature vector and texture feature vector are concatenated in a fixed dimension order to form a multivariate feature vector, and the multivariate feature vector is bound to the detection unit number, boundary coordinates and center coordinates to form a detection unit feature record. S27. Collect all feature records of detection units in order of detection unit number to form a multivariable feature vector set, and simultaneously generate an index mapping table of the multivariable feature vector set to the pixel coordinate system.

[0018] In this embodiment, S3 includes: S31. Establish a normal sample feature library: Obtain the aligned display image of the display screen in a defect-free state, generate the corresponding multivariate feature vector set, and write the multivariate feature vector of each normal sample into the normal sample feature library according to the detection unit number, so that each record in the normal sample feature library contains at least the detection unit number and the corresponding multivariate feature vector. S32. Construct statistical benchmark parameters: Extract all multivariate feature vectors under the same detection unit number from the normal sample feature library, calculate the mean vector and covariance matrix corresponding to the detection unit number, and write the mean vector and covariance matrix into the statistical benchmark table; establish an index in the statistical benchmark table according to the detection unit number so that the corresponding mean vector and covariance matrix can be directly read according to the detection unit number in the detection stage; S33, Improved Step 1 – Covariance Construction Process for Group Correlation Preservation: Divide the multivariate feature vectors into multiple feature groups according to luminance statistics, chrominance feature sub-vectors, and texture feature sub-vectors; calculate the group mean and group covariance for each feature group to obtain a set of group covariance matrices; then calculate the inter-group correlation matrix for the group mean-centered sequence of different feature groups to obtain the inter-group correlation matrix; construct a joint covariance matrix based on the set of group covariance matrices and the inter-group correlation matrix, so that the joint covariance matrix contains both intra-group correlation structure and inter-group correlation structure, and write the joint covariance matrix into the statistical benchmark table; S34. Calculate the Hotelling T-squared deviation: Receive the set of multivariate feature vectors corresponding to the image to be detected. Read the mean vector and joint covariance matrix from the statistical benchmark table one by one according to the detection unit number. Calculate the Hotelling T-squared deviation for each detection unit. The Hotelling T-squared deviation satisfies the following: ; in, For the first Multivariate feature vectors of each detection unit For the statistical benchmark table and the first The mean vector corresponding to each detection unit number For the statistical benchmark table and the first The inverse of the joint covariance matrix corresponding to each detection unit number; [This is followed by a list of all detection units.] A deviation sequence is formed according to the detection unit number; S35. Generate Deviation Map: Based on the established detection unit grid and the center coordinates of the detection units, generate the deviation map for each unit in the deviation sequence. Write the grid position corresponding to the detection unit, generate an offset map that corresponds one-to-one with the detection unit, and establish an index relationship between the offset map and the pixel coordinate system so that each grid position in the offset map can be traced back to the boundary coordinates and center coordinates of the corresponding detection unit; S36. Improved Step Two – Construction Process of Deviation Map Structure Analysis: Using the deviation map as input, perform structure analysis to obtain the deviation structure map. The structure analysis includes: S361, Threshold Segmentation: Apply a statistical threshold to the deviation map to generate a binary outlier map. In the binary outlier map, the values ​​of the outlier grid are marked as outliers, and the values ​​of the non-outlier grids are marked as non-outliers. S362, Neighborhood Counting Mapping: Based on the detection unit grid, a fixed neighborhood template is established. For each grid in the binary anomaly map, the number of abnormal grids within the coverage area of ​​its neighborhood template is counted to generate a neighborhood anomaly counting map. S363, Structural Screening: The neighborhood anomaly count map and the binary anomaly map are jointly screened. Grids that meet the condition that "the current grid is anomaly and the neighborhood anomaly count reaches the preset cluster judgment condition" are marked as in-cluster anomaly grids, and the remaining anomaly grids are marked as isolated anomaly grids, forming a point-cluster separation map. S364. Connectivity Extraction: On the point cluster separation graph, perform connectivity marking on abnormal meshes within the cluster to obtain a set of candidate structural domains; for each candidate structural domain, calculate the domain boundary, the number of domain meshes, the domain circumscribed rectangle and the main direction, and generate a structural domain attribute table. S365. Output Deviation Structure Graph: Associate and encapsulate the point cluster separation graph, candidate structural domain set, and structural domain attribute table with the deviation graph, and output the deviation structure graph data package so that subsequent steps can simultaneously call the numerical deviation information of the deviation graph and the spatial structure information of the structural domain. S37. Link and output the deviation graph, deviation sequence, statistical benchmark table, and deviation structure graph data package.

[0019] In this embodiment, S4 includes: S41. Receive the deviation map, the detection unit grid, the detection unit center coordinates and the detection unit adjacency table, wherein the detection unit adjacency table records the set of first-order adjacent detection unit numbers corresponding to each detection unit. S42. Read the Hotelling T-squared deviation history sequence of each detection unit in a defect-free state from the normal sample feature library. For each detection unit, calculate the upper limit threshold of the history sequence. The upper limit threshold is determined by sorting the history sequence from smallest to largest and taking the deviation value at a preset ranking position. Compare the deviation of the detection unit in the deviation map to be detected with the upper limit threshold of the corresponding detection unit. Detection units with deviations not less than the upper limit threshold are marked as initial abnormal units, forming an initial abnormal unit set. S43. Perform non-maximum suppression on the initial abnormal unit set: For each initial abnormal unit, read its first-order adjacent detection unit number set and obtain the deviation of adjacent detection units. If the deviation of the current initial abnormal unit is less than the deviation of any adjacent detection unit, delete the initial abnormal unit. Repeat the above comparison for the retained initial abnormal units until there are no more initial abnormal units that can be deleted, and obtain the local extreme value abnormal unit set. S44. Perform neighborhood consistency screening on the set of local extreme value anomalies: For each local extreme value anomaly, read its first-order neighbor detection unit number set and second-order neighbor detection unit number set, and count the number of first-order neighbor detection units that belong to the initial anomaly unit set and the number of second-order neighbor detection units that belong to the initial anomaly unit set respectively; when the number of anomalies in the first-order neighbor detection units and the number of anomalies in the second-order neighbor detection units simultaneously meet the preset quantity conditions, retain the local extreme value anomaly unit; otherwise, delete the local extreme value anomaly unit. S45. Generate candidate points for the retained local extreme value anomaly units: Use the center coordinates of the detection unit of the local extreme value anomaly unit as the coordinates of the candidate point, use the deviation of the detection unit as the deviation of the candidate point, and use the detection unit number as the candidate point identifier to form a candidate point record. S46. Bind a detection unit neighborhood index to each candidate point record: Taking the detection unit number corresponding to the candidate point identifier as the center, read the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers, and write the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers into the candidate point record as the detection unit neighborhood index. S47. Collect all candidate point records to form a candidate point set, and establish a mapping table from candidate point identifiers to detection unit grid positions, so that each candidate point in the candidate point set can be traced back to the grid position and detection unit boundary coordinates in the deviation map.

[0020] In this embodiment, S5 includes: S51. Receive the candidate point set and the center coordinates of the detection unit bound to it, extract the two-dimensional coordinates of all candidate points to form a spatial point set; receive the boundary of the display area corresponding to the aligned display image, read the boundary coordinates of the four boundaries of the display area in the pixel coordinate system, and form an observation window for spatial statistics. S52. Construct a distance scale sequence: Read the multi-scale distance set from the distance scale configuration table and sort it in ascending order to obtain a distance scale sequence; establish a squared distance threshold for each distance scale in the distance scale sequence and write it into the scale index table so that subsequent distance comparisons can be made by directly comparing the squared distance with the squared threshold. S53, Improved Step 1 – Single Pairing – Multi-Scale Cumulative Point Pair Counting Construction Process: S531. Establish a grid index for the spatial point set: Based on the detection unit grid, write each candidate point into the point list of the corresponding grid unit according to its detection unit number, and generate an index structure from grid unit to point list. S532. Determine the grid search range corresponding to the maximum distance scale: Using the maximum distance scale as the upper limit, calculate the grid expansion range it covers in the detection unit grid coordinate system, and write the expansion range into the grid search template; S533. Perform a neighborhood grid scan for each candidate point: Using the grid cell where the candidate point is located as the center, enumerate the adjacent grid cells according to the grid search template, read the point list of the adjacent grid cells, and form point pairs with the current candidate point one by one. S534. Calculate the squared distance for each point pair: Calculate the horizontal and vertical differences of the two-dimensional coordinates of the candidate points at both ends of the point pair to obtain the squared distance; discard the point pair when the squared distance is greater than the squared threshold of the maximum distance scale; write the squared distance into the point pair distance record table when the squared distance is not greater than the squared threshold of the maximum distance scale. S535. Sort the point-to-point distance record table by distance from smallest to largest and write it into the distance count histogram table in sequence; perform prefix accumulation on the distance count histogram table according to the distance scale sequence to obtain the cumulative count of point pairs corresponding to each distance scale, so as to obtain the cumulative count of point pairs for all distance scales after one point pair construction and one sorting. S54, Improved Step Two – Explicit Overlap Ratio – Point-to-Pair Weighted Boundary Correction Construction Process: S541. Calculate the boundary distance for each candidate point: Read the two-dimensional coordinates of the candidate point and the four boundary coordinates of the observation window, calculate the shortest distance from the candidate point to each boundary, and form a record of the candidate point boundary distances. S542. Construct the overlap ratio for each candidate point and each distance scale: When the shortest distance from the candidate point to all four boundaries is not less than the distance scale, the overlap ratio of the candidate point at that distance scale is recorded as 1; when there is at least one boundary that makes the shortest distance less than the distance scale, construct the overlap ratio between the circular neighborhood and the observation window for the candidate point. The overlap ratio is obtained through the following steps: with the candidate point as the center and the distance scale as the radius, generate a set of circular sampling points on the circumference according to the angle step table; determine whether each circular sampling point falls within the observation window; use the ratio of the number of circular sampling points falling within the observation window to the total number of circular sampling points as the overlap ratio of the candidate point at that distance scale. S543. Perform point-pair weight generation on overlap ratio: For each candidate point and each distance scale, take the reciprocal of its overlap ratio as the boundary correction weight and write it into the "candidate point number - distance scale - weight" ternary table for subsequent point-pair counting and weighting. S55. Calculate the Ripley K-function value sequence: Using the observation window area and the number of candidate points as normalization parameters, based on the cumulative count of point pairs at each distance scale, and combined with boundary correction weights, calculate the Ripley K-function value for each distance scale. The Ripley K-function satisfies: ; in, For the observation window area, The number of candidate points. For the first With the Distance between candidate points As an indication of conditions, For the ternary table Candidate points at the distance scale The lower boundary correction weights; corresponding to all distance scales Arranged according to distance scale sequence to form cluster intensity sequence; S56. Generate multi-scale clustered response vectors for candidate points: For each candidate point, count the number of neighboring candidate points within the distance scale sequence one by one, and apply boundary correction weights to the candidate point during the count. Concatenate the obtained multi-scale neighborhood statistics according to the distance scale sequence to form the multi-scale clustered response vector of the candidate point. Bind the multi-scale clustered response vector with the detection unit coordinates, deviation, and detection unit neighborhood index in the candidate point record to form a candidate point clustered attribute record. S57. Collect all candidate point aggregation attribute records to form a candidate point multi-scale aggregation response set, and output the aggregation intensity sequence and the candidate point multi-scale aggregation response set.

[0021] In this embodiment, S6 includes: S61. Receive a set of candidate points, and read the detection unit number, detection unit center coordinates, deviation, and first-order and second-order detection unit neighborhood indices bound to each candidate point; receive the candidate point's multi-scale clustered response vector and distance scale sequence, wherein the multi-scale clustered response vector is arranged according to the distance scale sequence, and the vector's first-order... The clustered response of the dimension is determined by the distance scale. The number of candidate points in the neighborhood of a candidate point is counted and boundary correction weights are applied. A candidate point index table is established according to the candidate point number so that the candidate point index table can index the deviation, the clustered response vector and the neighborhood index. S62. Construct a gating threshold table: For each detection unit number, read the historical sequence of deviation corresponding to that number from the normal sample feature library and sort it by value. Take the deviation at the preset ranking position as the statistical threshold for that number to form a "Detection Unit Number - Statistical Threshold" table; For each distance scale in the distance scale sequence, sort the historical sequence of the aggregation response of the corresponding candidate point of the normal sample by value and take the aggregation response at the preset ranking position as the spatial threshold for that scale to form a "Distance Scale - Spatial Threshold" table. S63. Improved Step 1 – Defect Seed Generation Process for Master Scale Locking – Dual-Gated Scoring: For each candidate point, read the statistical threshold from the “Detection Unit Number – Statistical Threshold” table based on the detection unit number; read the spatial threshold for each distance scale from the “Distance Scale – Spatial Threshold” table based on the distance scale sequence; and aggregate the response vector of the candidate points at multiple scales. The dimension response value is denoted as and will be related to the distance scale The corresponding spatial threshold is denoted as Perform comparisons scale-by-scale on the distance scale sequence, when Not less than Time will measure Enter the scale list; calculate the ratio scale by scale in the scale list. Select the scale with the largest ratio as the primary scale and lock its index. And the value of the aggregation response corresponding to the principal scale is determined as ; Calculate the anomaly comprehensive score according to the locked principal scale, and the anomaly comprehensive score satisfies: ; in, The deviation of the candidate point. Assign a statistical threshold to each detection unit. The index of the candidate point in the multi-scale clustered response vector is The dimensional values, The index in the "Distance Scale - Spatial Threshold" table is The threshold; when Not less than Furthermore, if the scale list is not empty, the candidate point is written into the defect seed point list, and the candidate point number and main scale index are recorded. Combined with anomaly score; S64, Improved Step Two – Connectivity Aggregation Process for Consistent Main Scale and No Score Increase: Using the defect seed point list as input, establish an unassigned set according to the candidate point number; take a seed point from the unassigned set as the region starting point, add its detection unit number to the region queue, and record the region's main scale index; dequeue one detection unit number in a loop, read its first-order neighborhood index set and second-order neighborhood index set, and filter neighborhood units in the neighborhood index set that simultaneously meet the following conditions and add them to the region queue: the candidate point corresponding to the neighborhood unit exists in the defect seed point list; the neighborhood unit's main scale index is the same as the region's main scale index; the neighborhood unit's abnormal comprehensive score is not greater than the abnormal comprehensive score of the candidate point corresponding to the currently dequeued unit; remove the candidate points added to the region queue from the unassigned set; when the region queue is empty, output a record of "Region Identifier – Detection Unit Number Set – Region Main Scale Index – Region Score Peak Point Number" for a defect region; repeat the above process until the unassigned set is empty. S65. Generate a defect region mask: For each defect region record, read the boundary coordinates of the detection unit according to the detection unit number set, write the pixel position covered by the boundary coordinates into the region identifier in the mask image, form a defect region mask with the same size as the aligned display image, and write the region identifier and the region main scale index into the region attribute table. S66. Determine the defect size level: Use the position of the region's main scale index in the distance scale sequence as the defect size level, and bind the defect size level with the region identifier and write it into the region attribute table. S67. Output the list of defect seed points, the defect region mask, and the region attribute table.

[0022] In this embodiment, S7 includes: S71. Receive the aligned display image, deviation map, defect area mask, defect size level and structured defect results, read the acquisition timestamp, and generate record entry identifiers; S72. Construct a structured defect result record: Traverse the region identifiers in the defect region mask, read the corresponding detection unit number set for each region identifier, query the detection unit boundary coordinates based on the detection unit number set, generate the region's outer rectangle coordinates and region center coordinates, and write the region identifier, outer rectangle coordinates, region center coordinates, defect size level, and maximum deviation within the region into the defect result list. S73. Establish an association index table: Associate the region identifiers in the defect result list with the pixel position set in the defect region mask, the grid position set in the deviation amount map, and the pixel boundary coordinates in the aligned display image, respectively, to generate an association index table of "region identifier - mask index - deviation amount index - image index"; S74. Generate record entry header: Write record entry identifier, acquisition timestamp, pixel coordinate system identifier and data segment directory. The data segment directory shall at least contain the starting offset and length of the aligned display image data segment, the deviation map data segment, the defect area mask data segment, the defect size level data segment, the defect result list data segment and the associated index table data segment. S75. Generate record entry data body: Serialize and write the aligned display image, deviation map, defect area mask, defect size level, defect result list and associated index table according to the data segment directory order, and write a verification field at the end of the record entry. S76, Write to vehicle data recorder: Call the vehicle data recorder's write interface to write the record entry header and record entry data body to the sequential storage area, and write the index record of "record entry identifier - storage address - data length - verification field" in the index area; S77. Synchronously send to the monitoring terminal: Extract monitoring message fields from the defect result list and associated index table, encapsulate the record entry identifier and collection timestamp, send to the monitoring terminal through the vehicle communication link, and write the "record entry identifier - sending status - confirmation information" sending status record locally.

[0023] Example: To verify the feasibility of this invention, the image processing-based subway display screen defect detection method proposed in this invention was applied to the automated operation and maintenance detection scenario of a rail transit vehicle information display system. The onboard display screen is installed inside the carriage and is used to display line information, station names, transfer prompts, arrival reminders, and operational status information. Due to the long-term operation of trains under conditions of vibration, temperature rise, voltage fluctuations, and frequent starts and stops, the display screen is prone to problems such as localized dark spots, uneven brightness, color shift, stripes, localized black patches, abnormally blurred edges, and distortion in localized pixel areas after prolonged use. Existing manual inspection methods rely on visual observation by maintenance personnel, which is not only inefficient but also easily influenced by subjective experience, making it difficult to detect small-area defects that are initially formed or discontinuously distributed in a timely manner. Existing image detection methods based on simple thresholds typically judge anomalies only based on sudden changes in brightness or color. When there is reflection, obstruction, viewing angle shift, or background disturbance within the carriage, normal display fluctuations are easily misjudged as fault areas, and it is also difficult to form a stable identification of continuously distributed true defects. To address the aforementioned issues, this invention employs a subway display screen defect detection method based on image processing, which performs standardized processing, multi-feature statistical analysis, spatial clustering discrimination, and structured output of defect areas on the vehicle display screen image.

[0024] In practical applications, the image acquisition device inside the train carriage is fixedly installed opposite the display screen. The acquisition device includes an industrial camera, a fixed-focus lens, a trigger control module, and an onboard processing unit. The industrial camera periodically acquires raw images from the display screen and sends them to a frame buffer. Upon receiving the images, the onboard processing unit first reads the camera calibration parameters and performs denoising and contrast normalization on the raw images to ensure a consistent grayscale distribution range under different acquisition conditions. Then, edge information is extracted from the preprocessed image, and edges are connected and broken to obtain a binary edge map. Candidate straight lines for the display screen border are detected in the binary edge map, and the four corner points are determined based on their intersection relationships. The perspective transformation matrix is ​​then calculated using the correspondence between the four corner points and the vertices of the target rectangle. Based on this perspective transformation matrix, inverse mapping and interpolation resampling are performed on the pixel coordinates within the target rectangle to obtain an aligned display image. Simultaneously, a two-dimensional pixel coordinate system is established with the upper left corner of the aligned display image as the origin, binding the position of each pixel to a linear address index, forming a unified coordinate basis for subsequent feature extraction and result mapping.

[0025] After obtaining the aligned display image, the display area is divided into detection units according to a preset grid rule. In practice, the aligned display image with a resolution of 1280×320 is divided into a 40×10 detection unit grid, resulting in 400 detection units. Each detection unit corresponds to fixed boundary coordinates and center coordinates. For each detection unit, its internal effective pixels are traversed, and luminance statistics, chrominance statistics, and texture statistics are extracted. Among them, luminance statistics include average luminance and luminance dispersion, which are used to characterize the overall luminance level and stability of the unit; chrominance statistics are obtained by mapping through a unified color space to obtain hue mean, hue dispersion, saturation mean, and saturation dispersion, which are used to reflect whether the displayed color has shifted; texture statistics are obtained by calculating the gradient amplitude mean, gradient amplitude dispersion, and gradient direction consistency index through gray-level gradient calculation, which are used to reflect texture changes such as stripes, local blurring, and edge anomalies. Then, the above features are concatenated in a fixed order to form a multivariate feature vector corresponding one-to-one with the detection unit number, and a multivariate feature vector set is generated. In this way, each local area of ​​the display screen is no longer represented by a single brightness value, but by multi-dimensional statistical features that can simultaneously describe changes in brightness, color, and texture.

[0026] To construct a statistical benchmark under normal conditions, defect-free display images from a continuous and stable operation phase were pre-collected as normal samples. For each normal sample image, the same perspective correction, mesh generation, and multivariate feature extraction processes as the images to be inspected were performed. Then, a normal sample feature library was established according to the detection unit number. In the normal sample feature library, each detection unit corresponds to a set of historical multivariate feature vectors. The mean vector and covariance matrix were calculated for all sample feature vectors under the same detection unit number. To preserve the correlation within and between different feature groups of brightness, chroma, and texture, simple covariance was not used directly. Instead, the group covariance of brightness, chroma, and texture groups was calculated separately, followed by the correlation matrix between the mean-centered sequences of each group, and finally, a joint covariance matrix was constructed. In this way, the statistical benchmark not only preserves the variation patterns within individual features but also retains the correlation structure between different features.

[0027] Once the image to be detected is entered, the mean vector and joint covariance matrix of the statistical benchmark table are read one by one according to the detection unit number. The Hotelling T-squared deviation is calculated for the multivariate feature vector of the current detection unit. This deviation reflects the comprehensive deviation of the current detection unit from the normal state. Subsequently, the deviations of all detection units are backfilled according to the grid position to generate a deviation map. Each grid position in the deviation map corresponds one-to-one with a local area in the display image, thus intuitively reflecting the spatial distribution of anomalies. To further distinguish between isolated noise and structurally significant anomalous regions, structural analysis is also performed on the deviation map. First, a binary anomaly map is generated based on a statistical threshold. Then, based on a fixed neighborhood template, the number of anomalous grids around each anomalous grid is counted to form a neighborhood anomaly count map. For grids that simultaneously meet the criteria of "the current grid is anomalous and the neighborhood anomaly count reaches the preset cluster judgment condition", they are marked as intra-cluster anomalous grids; the remaining anomalous grids are marked as isolated anomalous grids. Then, connected component labeling is performed on intra-cluster anomalous grids, and the candidate structural domain set and its boundaries, grid number, and circumscribed rectangle are output. This process allows for the initial differentiation between continuous abnormal regions and scattered noise during the deviation mapping stage.

[0028] In the candidate point generation stage, the historical sequence of Hotelling T-squared deviation of each detection unit in a defect-free state is read from normal samples, sorted by size, and the value at the preset ranking position is selected as the upper limit threshold corresponding to that detection unit. If the deviation in the image to be detected is not less than the upper limit threshold, the corresponding detection unit is marked as an initial anomalous unit. Subsequently, non-maximum suppression is performed on the initial anomalous units. For each initial anomalous unit, its deviation from the first-order adjacent detection units is compared. If the current deviation is not a local maximum, the unit is deleted. After local extreme value screening, neighborhood consistency screening is performed, that is, the number of anomalous units in its first-order and second-order neighborhoods are counted respectively. Only local extreme value anomalous units that simultaneously meet the preset quantity condition are retained. For the retained local extreme value anomalous units, the detection unit center coordinates are used as candidate point coordinates, and the deviation is used as the candidate point deviation. The detection unit numbers of the first-order and second-order neighborhoods are written into the candidate point record to form a candidate point set. In this way, isolated anomalies caused by random noise are avoided from being directly regarded as real defects.

[0029] In the spatial clustering analysis phase, the two-dimensional coordinates of the candidate point set are used as the spatial point set input, and the boundary of the display area corresponding to the image is used as the observation window. In practice, the preset multi-scale distance sets are 6 pixels, 12 pixels, 18 pixels, 24 pixels, and 30 pixels. First, a grid index structure is established for the candidate points. Then, the neighborhood search range is determined with the maximum distance scale as the upper limit. A neighborhood grid scan is performed on each candidate point to construct point pairs and calculate the squared distance. Point pairs not exceeding the maximum distance threshold are written into the point pair distance record table. The point pair distances are then sorted, and prefix accumulation is performed according to the distance scale to obtain the cumulative count of point pairs at each scale. Considering that candidate points close to the image boundary may lead to missing statistical ranges, the shortest distance to the four boundaries is calculated for each candidate point. If the neighborhood of a candidate point is clipped from the observation window at a certain distance scale, the overlap ratio is calculated using circular sampling, and the reciprocal of the overlap ratio is used as the boundary correction weight. Based on the cumulative count of point pairs and the boundary correction weight, the Ripley K function values ​​at each scale are calculated to form a clustering intensity sequence. Simultaneously, for each candidate point, the number of its neighboring candidate points at each distance scale is counted, and a multi-scale clustered response vector is formed by combining the boundary correction weights. In this way, we can not only know whether a point itself is anomalous, but also whether it is in a spatially continuous anomalous region.

[0030] In the defect region determination stage, a "Detection Unit Number - Statistical Threshold" table and a "Distance Scale - Spatial Threshold" table are constructed respectively. For each candidate point, its deviation is compared with the statistical threshold of the corresponding detection unit number, and its multi-scale aggregation response vector is compared dimension by dimension with the spatial threshold of each distance scale to obtain a list of passing scales. If a candidate point reaches the spatial threshold at at least one distance scale, the scale with the largest ratio of aggregation response to spatial threshold is selected from the list of passing scales as the main scale, and an anomaly comprehensive score is calculated based on the deviation and the main scale aggregation response. When the deviation of a candidate point reaches the statistical threshold and the list of passing scales is not empty, the candidate point is determined as a defect seed point. Then, starting from the defect seed point, connectivity aggregation is performed based on the detection unit neighborhood index. During the aggregation process, only neighborhood units that satisfy the same main scale index and whose anomaly comprehensive score is not greater than the current extended unit score are included in the same defect region. After the aggregation is completed, the region identifier is written into a mask map with the same size as the aligned display image based on the set of detection units in the defect region to form a defect region mask; and the order of the main scale index of the region in the distance scale sequence is written into the region attribute table as the defect size level. Therefore, what is ultimately obtained is no longer discrete outliers, but a complete defect region with boundary range, scale level and location attributes.

[0031] During the results output phase, the aligned display image, deviation map, defect area mask, defect size level, and defect result list are uniformly encapsulated. The defect result list records the area identifier, bounding rectangle coordinates, area center coordinates, maximum deviation within the area, and defect size level. Furthermore, a correlation index table of "area identifier—mask index—deviation index—image index" is established, enabling each defect area to be traced back to its pixel position in the mask, its grid position in the deviation map, and its corresponding boundary range in the original image. The encapsulated record entries are written to the vehicle-mounted data recording device and synchronously transmitted to the monitoring terminal for viewing and subsequent review by maintenance personnel.

[0032] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A defect detection method for subway display screens based on image processing, characterized in that, include: S1. Acquire the display screen image, complete perspective correction and display area positioning, and output the aligned display image and corresponding pixel coordinate system; S2. Construct a detection unit grid based on the pixel coordinate system to generate a set of multivariable feature vectors; S3. Based on normal samples, establish mean vector and covariance matrix, calculate Hotelling T squared deviation for multivariable feature vector set, and generate deviation map corresponding to detection unit; S4. Perform threshold filtering and non-maximum suppression on the deviation map to generate a candidate point set, and bind the detection unit coordinates, deviation amount and detection unit neighborhood index to each candidate point; S5. Using the candidate point set as spatial point input, calculate the Ripley K function under the preset multi-scale distance set to obtain the clustering intensity sequence at each scale, and generate a multi-scale clustering response vector for each candidate point. S6. Calculate the anomaly comprehensive score based on the deviation and multi-scale clustered response vector, determine the defect seed point, perform connectivity aggregation on the defect seed point based on the detection unit neighborhood index to generate a defect region mask, and determine the defect size level according to the dominant scale in the clustering intensity sequence of the defect region at each scale. S7. The aligned display image, deviation map, defect area mask, defect size level and structured defect results are associated and encapsulated to form a record entry, written to the vehicle data recorder and sent to the monitoring terminal simultaneously.

2. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S1 includes: S11. Trigger the vehicle-mounted image acquisition device to acquire the original image on the display screen, write the original image to the frame buffer, and read the acquisition timestamp and camera calibration parameters to generate an acquisition frame containing the original image and calibration parameters. S12. Denoise and contrast normalize the acquired frames to generate a preprocessed image; extract edge maps from the preprocessed image and perform edge connection and break repair processing on the edge maps to generate a binary edge map. S13. Perform line detection in the edge binary map to obtain a set of candidate lines for the border, and calculate the set of intersection points based on the set of candidate lines for the border; filter the set of corner points of the display area in the set of intersection points using quadrilateral geometric consistency constraints, and sort the set of corner points according to the geometric orientation of the display area to form a corner point sequence; S14. Construct a set of vertex coordinates of the target rectangle based on the corner point sequence, establish the correspondence between the corner point sequence and the set of vertex coordinates of the target rectangle, and solve for the perspective transformation matrix; S15. Perform reverse mapping on the coordinates of each pixel within the target rectangle, and use the perspective transformation matrix to map the target pixel coordinates to the original image coordinates; perform interpolation and resampling on the non-integer coordinates obtained by mapping to form an aligned display image filled pixel by pixel; perform validity judgment and pixel filling control on the mapped coordinates that exceed the effective range of the original image. S16. Establish a pixel coordinate system with the top left corner pixel of the aligned display image as the origin, and define the horizontal direction as the first coordinate axis and the vertical direction as the second coordinate axis; assign a two-dimensional coordinate index to each pixel in the aligned display image, and generate a mapping table between the two-dimensional coordinate index and the linear address index; S17. Bind and encapsulate the aligned display image with the pixel coordinate system data structure, and output an alignment result data packet containing the aligned display image, pixel coordinate index mapping table and perspective transformation matrix.

3. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S2 includes: S21. Receive the aligned display image and pixel coordinate system, extract the effective pixel set of the display area based on the pixel coordinate system, generate the detection unit grid according to the preset grid rules, and write the detection unit number, boundary coordinates and center coordinates for each detection unit. S22. For each detection unit, traverse the set of valid pixels according to the boundary coordinates to form a pixel sequence of the detection unit, and simultaneously form a relative coordinate sequence of pixels within the detection unit. S23. Calculate the brightness statistics for the pixel sequence of the detection unit. The brightness statistics include the average brightness and the brightness dispersion, which satisfy the following: ; in, To detect grayscale values ​​in the pixel sequence of the detection unit, The number of pixels in the detection unit; S24. Calculate the chromaticity statistics for the pixel sequence of the detection unit and generate the chromaticity feature sub-vector; S25. Calculate texture statistics for the pixel sequence of the detection unit and generate texture feature sub-vectors; S26. The luminance statistics, chromaticity feature vector and texture feature vector are concatenated in a fixed dimension order to form a multivariate feature vector, and the multivariate feature vector is bound to the detection unit number, boundary coordinates and center coordinates to form a detection unit feature record. S27. Collect all feature records of detection units in order of detection unit number to form a multivariable feature vector set, and simultaneously generate an index mapping table of the multivariable feature vector set to the pixel coordinate system.

4. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S3 includes: S31. Obtain the aligned display image of the display screen in a defect-free state, generate the corresponding multivariate feature vector set, and write the multivariate feature vector of each normal sample into the normal sample feature library according to the detection unit number. S32. Extract all multivariate feature vectors under the same detection unit number from the normal sample feature library, calculate the mean vector and covariance matrix corresponding to the detection unit number, and write the mean vector and covariance matrix into the statistical benchmark table; the statistical benchmark table is indexed according to the detection unit number; S33. Divide the multivariate feature vector into N feature groups according to luminance statistics, chroma feature sub-vectors, and texture feature sub-vectors; calculate the group mean and group covariance for each feature group to obtain a set of group covariance matrices; calculate the inter-group correlation matrix for the group mean-centered sequence of different feature groups to obtain the inter-group correlation matrix; construct a joint covariance matrix based on the set of group covariance matrices and the inter-group correlation matrix, and write the joint covariance matrix into the statistical benchmark table; S34. Calculate the Hotelling T-squared deviation: Receive the set of multivariate feature vectors corresponding to the image to be detected. Read the mean vector and joint covariance matrix from the statistical benchmark table one by one according to the detection unit number. Calculate the Hotelling T-squared deviation for each detection unit. The Hotelling T-squared deviation satisfies the following: ; in, For the first Multivariate feature vectors of each detection unit For the statistical benchmark table and the first The mean vector corresponding to each detection unit number For the statistical benchmark table and the first The inverse of the joint covariance matrix corresponding to each detection unit number; [This is followed by a list of all detection units.] A deviation sequence is formed according to the detection unit number; S35. Based on the established detection unit grid and the center coordinates of the detection unit, each of the deviation sequences... Write the grid positions corresponding to the detection units to generate an offset map that corresponds one-to-one with the detection units, and establish an index relationship between the offset map and the pixel coordinate system; S36. Using the deviation map as input, perform structural analysis to obtain the deviation structure map. The structural analysis includes: S361, Threshold Segmentation: Apply a statistical threshold to the deviation map to generate a binary outlier map. In the binary outlier map, the values ​​of the outlier grid are marked as outliers, and the values ​​of the non-outlier grids are marked as non-outliers. S362, Neighborhood Counting Mapping: Based on the detection unit grid, a fixed neighborhood template is established. For each grid in the binary anomaly map, the number of abnormal grids within the coverage area of ​​its neighborhood template is counted to generate a neighborhood anomaly counting map. S363, Structural Screening: The neighborhood anomaly count map and the binary anomaly map are jointly screened. Grids that meet the condition of "the current grid is anomaly and the neighborhood anomaly count reaches the preset cluster judgment condition" are marked as in-cluster anomaly grids, and the remaining anomaly grids are marked as isolated anomaly grids, forming a point-cluster separation map. S364. Connectivity Extraction: On the point cluster separation graph, perform connectivity marking on abnormal meshes within the cluster to obtain a set of candidate structural domains; for each candidate structural domain, calculate the domain boundary, the number of domain meshes, the domain circumscribed rectangle and the main direction, and generate a structural domain attribute table. S365. Associate and encapsulate the point cluster separation graph, candidate structural domain set, and structural domain attribute table with the deviation graph, and output the deviation structural graph data package. S37. Link and output the deviation graph, deviation sequence, statistical benchmark table, and deviation structure graph data package.

5. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S4 includes: S41. Receive the deviation map, the detection unit grid, the detection unit center coordinates and the detection unit adjacency table, wherein the detection unit adjacency table records the set of first-order adjacent detection unit numbers corresponding to each detection unit. S42. Read the Hotelling T-squared deviation history sequence of each detection unit in a defect-free state from the normal sample feature library. For each detection unit, calculate the upper limit threshold of the history sequence. The upper limit threshold is determined by sorting the history sequence from smallest to largest and taking the deviation value at a preset ranking position. Compare the deviation of the detection unit in the deviation map to be detected with the upper limit threshold of the corresponding detection unit. Detection units with deviations not less than the upper limit threshold are marked as initial abnormal units, forming an initial abnormal unit set. S43. Perform non-maximum suppression on the initial abnormal unit set: For each initial abnormal unit, read its first-order adjacent detection unit number set and obtain the deviation of adjacent detection units. If the deviation of the current initial abnormal unit is less than the deviation of any adjacent detection unit, delete the initial abnormal unit. Repeat the above comparison for the retained initial abnormal units until there are no initial abnormal units that can be deleted, and obtain the local extreme value abnormal unit set. S44. Perform neighborhood consistency screening on the set of local extreme value anomalies: For each local extreme value anomaly, read its first-order neighbor detection unit number set and second-order neighbor detection unit number set, and count the number of first-order neighbor detection units belonging to the initial anomaly unit set and the number of second-order neighbor detection units belonging to the initial anomaly unit set respectively; when the number of anomalies in the first-order neighbor detection units and the number of anomalies in the second-order neighbor detection units simultaneously meet the preset quantity condition, the local extreme value anomaly unit is retained; otherwise, the local extreme value anomaly unit is deleted. S45. Generate candidate points for the retained local extreme value anomaly units: use the center coordinates of the detection unit of the local extreme value anomaly unit as the coordinates of the candidate point, use the deviation of the detection unit as the deviation of the candidate point, and use the detection unit number as the candidate point identifier to form a candidate point record; S46. Bind a detection unit neighborhood index to each candidate point record: Taking the detection unit number corresponding to the candidate point identifier as the center, read the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers, and write the set of first-order adjacent detection unit numbers and the set of second-order adjacent detection unit numbers into the candidate point record as the detection unit neighborhood index. S47. Collect all candidate point records to form a candidate point set, and establish a mapping table from candidate point identifiers to detection unit grid positions, so that each candidate point in the candidate point set can be traced back to the grid position and detection unit boundary coordinates in the deviation map.

6. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S5 includes: S51. Receive the candidate point set and the center coordinates of the detection unit bound to it, extract the two-dimensional coordinates of all candidate points to form a spatial point set; receive the boundary of the display area corresponding to the aligned display image, read the boundary coordinates of the four boundaries of the display area in the pixel coordinate system, and form a spatial statistical observation window; S52. Construct a distance scale sequence: Read the multi-scale distance set from the distance scale configuration table and sort it in ascending order to obtain the distance scale sequence; establish a squared distance threshold for each distance scale in the distance scale sequence and write it into the scale index table; S53, Single-pairing-multi-scale cumulative point-pair counting construction process: S531. Establish a grid index for the spatial point set: Based on the detection unit grid, write each candidate point into the point list of the corresponding grid unit according to its detection unit number, and generate an index structure from grid unit to point list. S532. Determine the grid search range corresponding to the maximum distance scale: Using the maximum distance scale as the upper limit, calculate the grid expansion range it covers in the detection unit grid coordinate system, and write the expansion range into the grid search template; S533. Perform a neighborhood grid scan for each candidate point: with the grid cell where the candidate point is located as the center, enumerate the adjacent grid cells according to the grid search template, read the point list of the adjacent grid cells, and form point pairs with the current candidate point one by one. S534. Calculate the squared distance for each point pair: Calculate the horizontal and vertical differences of the two-dimensional coordinates of the candidate points at both ends of the point pair to obtain the squared distance; discard the point pair when the squared distance is greater than the squared threshold of the maximum distance scale; write the squared distance into the point pair distance record table when the squared distance is not greater than the squared threshold of the maximum distance scale. S535. Sort the point-to-point distance record table by distance from smallest to largest and write it into the distance count histogram table in sequence; perform prefix accumulation on the distance count histogram table according to the distance scale sequence to obtain the cumulative count of point pairs corresponding to each distance scale. After one point pair construction and one sorting, obtain the cumulative count of point pairs for all distance scales. S54, Explicit Overlap Ratio—Point-to-Point Weighted Boundary Correction Construction Process: S541. Calculate the boundary distance for each candidate point: Read the two-dimensional coordinates of the candidate point and the four boundary coordinates of the observation window, calculate the shortest distance from the candidate point to each boundary, and form a record of the candidate point boundary distances. S542. Construct an overlap ratio for each candidate point and each distance scale: When the shortest distance from a candidate point to all four boundaries is not less than the distance scale, the overlap ratio of the candidate point at the distance scale is recorded as 1; when there is at least one boundary that makes the shortest distance less than the distance scale, construct an overlap ratio between the candidate point's circular neighborhood and the observation window. The overlap ratio is obtained through the following steps: with the candidate point as the center and the distance scale as the radius, generate a set of circular sampling points on the circumference according to the angle step table; determine whether each circular sampling point falls within the observation window; use the ratio of the number of circular sampling points falling within the observation window to the total number of circular sampling points as the overlap ratio of the candidate point at the distance scale. S543. Perform point-to-point weight generation for overlap ratio: For each candidate point and each distance scale, take the reciprocal of its overlap ratio as the boundary correction weight and write it into the "candidate point number - distance scale - weight" ternary table; S55. Calculate the Ripley K-function value sequence: Using the observation window area and the number of candidate points as normalization parameters, based on the cumulative count of point pairs at each distance scale, and combined with boundary correction weights, calculate the Ripley K-function value for each distance scale. The Ripley K-function satisfies... ; in, For the observation window area, The number of candidate points. For the first With the Distance between candidate points As an indication of conditions, For the ternary table Candidate points at the distance scale The lower boundary correction weights; corresponding to all distance scales Arranged according to distance scale sequence to form cluster intensity sequence; S56. For each candidate point, count the number of its neighboring candidate points within the distance scale according to the distance scale sequence, and apply boundary correction weights to the candidate points during the count. Concatenate the obtained multi-scale neighborhood statistics according to the distance scale sequence to form the multi-scale clustered response vector of the candidate point. Bind the multi-scale clustered response vector with the detection unit coordinates, deviation and detection unit neighborhood index in the candidate point record to form the candidate point clustered attribute record. S57. Collect all candidate point aggregation attribute records to form a candidate point multi-scale aggregation response set, and output the aggregation intensity sequence and the candidate point multi-scale aggregation response set.

7. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S6 includes: S61. Receive the candidate point set and its bound detection unit number, detection unit center coordinates, deviation amount and first-order and second-order detection unit neighborhood index, and receive the candidate point multi-scale aggregation response vector and distance scale sequence, and establish a candidate point index table. S62. Construct a "Detection Unit Number - Statistical Threshold" table based on the historical sequence of deviation in normal samples, and construct a "Distance Scale - Spatial Threshold" table based on the historical sequence of aggregation response at each distance scale in normal samples; S63. For each candidate point, compare the aggregation response with the spatial threshold on a scale-by-scale according to the distance scale to generate a list of passing scales. Select the scale with the largest ratio of aggregation response to spatial threshold from the list of passing scales and lock it as the main scale. Calculate the anomaly comprehensive score. When the deviation of the candidate point is not less than the corresponding statistical threshold and the list of passing scales is not empty, the candidate point is determined as a defect seed point and the main scale index is recorded. S64. Using the defect seed point as input, perform connectivity aggregation based on the neighborhood index of the detection unit. During the aggregation process, only neighborhood units with the same main scale index and whose anomaly comprehensive score is not greater than the score corresponding to the current extended unit are merged into the same defect region. Output the set of detection unit numbers of the defect region. S65. Generate a defect region mask based on the boundary coordinates of the detection unit, and determine the defect size level by the position of the main scale index of the defect region in the distance scale sequence. Output the list of defect seed points, the defect region mask, and the defect size level.

8. The image processing-based defect detection method for subway displays according to claim 1, characterized in that, S7 includes: S71. Receive the aligned display image, deviation map, defect area mask, defect size level and structured defect results, and generate a record entry containing the record entry identifier and acquisition timestamp. S72. Traverse the region identifiers in the defect region mask, calculate the coordinates of the region's outer rectangle and the region's center coordinates based on the corresponding detection unit number set, and construct a defect result list. S73. Establish an index association between the region identifiers in the defect results list and the corresponding positions in the defect region mask, deviation map, and aligned display image to form an association index table; S74. Serialize and encapsulate the aligned display image, deviation map, defect area mask, defect size level, defect result list and associated index table according to the preset data segment directory, and generate a record entry data body containing record entry identifier, acquisition timestamp and pixel coordinate system identifier. S75. Write the record item data body into the vehicle data recorder and establish the corresponding index record. At the same time, extract the defect result list to generate a monitoring message, and send it to the monitoring terminal synchronously through the vehicle communication link and record the sending status.

Images