Intelligent station entrance and exit automatic inspection method and system based on image processing

Abstract

The invention relates to the technical field of image processing, in particular to an intelligent station entrance and exit automatic inspection method and system based on image processing, and the method comprises the steps: S1, carrying out the inspection processing of a monitoring image shot by a CCTV monitoring system for the entrance and exit of a subway station, and obtaining a first inspection result and a second inspection result; s2, on the basis of the second inspection result and a pre-obtained escalator running state inspection result, whether the guide lamp in the monitoring picture is normally displayed or not is judged according to a preset logic, and a display inspection result of the guide lamp is obtained; s3, sequentially performing screen fault detection and text content detection on the picture of the PIS screen in the CCTV monitoring system to obtain a screen fault detection result and a text content detection result of the PIS screen; and S4, generating an inspection report based on the first inspection result, the second inspection result, the display inspection result of the guide lamp, the screen fault detection result of the PIS screen and the text content detection result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an automatic inspection method and system for smart station entrances and exits based on image processing. Background Art

[0002] With the rapid development of public transportation, subways, as one of the most common modes of transportation for residents, face challenges such as large passenger flow and dense population. One of the daily tasks of subway station staff is to inspect the entrance and exit areas of the station. However, due to the large area of ​​the station, the entrances and exits are scattered and far apart, and daily inspection work is often time-consuming and labor-intensive.

[0003] In actual operation, station staff need to inspect whether there is crowd gathering at the station entrance and exit area, check whether the PIS (passenger information system) screen and guide lights at the station entrance and exit are displaying normally, whether the escalators at the entrance and exit are operating normally, and whether the information displayed on the PIS screen above the escalator is accurate. These tasks are crucial to ensure the safety of station operation and a good experience for passengers. However, the existing inspection method is inefficient and prone to missing problems due to human negligence. There is an urgent need for a more efficient and accurate inspection method to improve the station management level and improve the work efficiency of station staff. Summary of the invention

[0004] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an automatic inspection method and system for smart station entrances and exits based on image processing.

[0005] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0006] In a first aspect, an embodiment of the present invention provides a method for automatic inspection of entrances and exits of a smart station based on image processing, the method comprising:

[0007] S1. Performing patrol inspection on the surveillance images captured by the entrance and exit of the subway station by the CCTV surveillance system installed at the exit of the subway station, and obtaining a first patrol inspection result and a second patrol inspection result of the surveillance images;

[0008] The first inspection result is that there is a gathering of people at the entrance and exit of the subway station in the monitoring picture or there is no gathering of people at the entrance and exit of the subway station in the monitoring picture;

[0009] The second inspection result is that the guide light in the monitoring screen is on or the guide light in the monitoring screen is off;

[0010] S2, based on the second inspection result and the pre-obtained inspection result of the escalator operation status, judging whether the guide light in the monitoring screen is displayed normally according to the pre-set logic, and obtaining the display inspection result of the guide light;

[0011] S3, for the collected images of the PIS screen in the CCTV monitoring system set at the exit of the subway station, screen fault detection and text content detection are performed in sequence to obtain the screen fault detection result and text content detection result of the PIS screen;

[0012] S4. Generate an inspection report based on the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen, and the text content detection result;

[0013] The inspection report includes the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen and the text content detection result.

[0014] Preferably, the S1 specifically includes:

[0015] S11, using a crowd gathering detection algorithm to process a monitoring picture taken by a CCTV monitoring system installed at a subway station exit, and obtaining a first inspection result of the monitoring picture;

[0016] S12. Using a guide light on / off detection algorithm, the CCTV monitoring system installed at the subway station exit processes the monitoring picture taken at the subway station entrance and exit to obtain a second inspection result of the monitoring picture.

[0017] Preferably, the S11 specifically includes:

[0018] S11-1, using a background subtraction algorithm to perform background subtraction processing on the monitoring images taken by the CCTV monitoring system installed at the subway station exit, to obtain the monitoring images after the background subtraction processing;

[0019] S11-2, using a target detection algorithm to identify and mark all personnel in the surveillance image after background subtraction processing, and obtain a personnel location image;

[0020] S11-3, using a density estimation algorithm to convert the personnel location image into a density map;

[0021] S11-4, based on the density map, obtaining a density value at any position in the density map;

[0022] S11-5. Determine a first inspection result of the monitoring image based on a density value at any position in the density map and a preset density threshold.

[0023] Preferably, the S11-4 specifically includes:

[0024] Based on the density map, the density value of any position in the density map is obtained using formula (1);

[0025] The formula (1) is:

[0026]

[0027] D(x, y) represents the density value at the position (x, y) in the density map;

[0028] K(xx i ,yy i ) represents any other i-th position (x i ,y i ) on the (x, y) position;

[0029] δ i is an indicator function, indicating that at position (x i ,y i ) Is there any personnel present? If yes, then i =1, otherwise δ i =0;

[0030] n is the total number of people in the density map;

[0031]

[0032] Where σ is the standard deviation of the Gaussian distribution;

[0033] exp is the exponential function with base e.

[0034] Preferably,

[0035] The escalator running status inspection result includes escalator status information and escalator running direction information;

[0036] The escalator status information is information that the escalator is in a fault state or information that the escalator is in a normal state.

[0037] Preferably, the S3 specifically includes:

[0038] S31, using the PIS screen on / off detection algorithm, to perform screen fault detection on the image of the PIS screen in the CCTV monitoring system installed at the exit of the subway station, and obtain the screen fault detection result of the PIS screen;

[0039] The screen fault detection result of the PIS screen is that the screen of the PIS screen is in a lit state or the screen of the PIS screen is in an off state;

[0040] S32, when the screen fault detection result of the PIS screen is that the screen of the PIS screen is in a light-up state, using an OCR text recognition algorithm to perform text recognition detection on the screen of the PIS screen to obtain a text content recognition result;

[0041] The text content recognition result includes: the text content in the image of the PIS screen recognized by the OCR text recognition algorithm;

[0042] S33, comparing the text content recognition result with the preset text content to obtain a comparison result;

[0043] S34: Determine whether the text content detection result is normal or abnormal based on the comparison result and a preset similarity threshold.

[0044] Preferably, the S33 specifically includes:

[0045] Formula (2) is used to calculate the total similarity between the text content recognition result and the preset text content, and the total similarity is used as the comparison result;

[0046] The formula (2) is:

[0047] O = α × C + β × P + γ × S;

[0048] Among them, O is the total similarity between the text content recognition result and the pre-set text content; C is the character matching degree between the text content recognition result and the pre-set text content; P is the position matching degree between the text content recognition result and the pre-set text content; S is the semantic matching degree between the text content recognition result and the pre-set text content; α, β, γ are pre-set weighting coefficients respectively, and 1=α+β+γ.

[0049] Preferably,

[0050] The character matching degree C between the text content recognition result and the preset text content is calculated by formula (3);

[0051] The formula (3) is:

[0052]

[0053] Wherein, MC is the number of characters that match between the text content recognition result and the pre-set text content; M is the total number of characters in the text content recognition result; TCR is the total number of characters in the pre-set text content;

[0054] The position matching degree P between the text content recognition result and the preset text content is calculated by formula (4);

[0055] The formula (4) is:

[0056]

[0057] m is the mth character in the text content recognition result;

[0058] dm is the Euclidean distance between the position of the mth character in the recognized text and the corresponding character position in the preset text in the text content recognition result; τ is a preset value;

[0059] The semantic matching degree S between the text content recognition result and the preset text content is calculated by formula (5);

[0060] The formula (5) is:

[0061]

[0062] M ∪ TCR is the total number of characters in the union of the total characters in the text content recognition result and the total characters of the preset text content; ω is the preset character importance factor.

[0063] Preferably, the S34 specifically includes:

[0064] If the comparison result is greater than the preset similarity threshold, determining that the text content detection result is normal;

[0065] If the comparison file is less than or equal to the preset similarity threshold, the text content detection result is determined to be abnormal.

[0066] On the other hand, this embodiment also provides a smart station entrance and exit automatic inspection system based on image processing, including:

[0067] at least one processor; and

[0068] At least one memory is communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the automatic inspection method for smart station entrances and exits based on image processing as described in the first aspect.

[0069] The beneficial effects of the present invention are:

[0070] The image processing-based intelligent station entrance automatic inspection method and system of the present invention automatically inspects the station entrance through the CCTV monitoring system, which reduces the workload of manual inspection and improves efficiency, especially in the detection of crowded areas and equipment failures. The method integrates crowd gathering detection, guide light status detection, escalator operation status inspection, PIS screen fault detection and text content detection, ensuring the normal operation of each important link of the station entrance.

[0071] The image processing-based intelligent station entrance automatic inspection system of the present invention can automatically generate inspection reports containing multiple detection results, which is convenient for subsequent management and decision-making, and reduces the possibility of human recording errors. Through real-time detection and automatic early warning, potential safety hazards, such as excessive gathering of people or failure of key equipment, can be discovered in time, which helps to improve the overall safety of the station.

[0072] The image processing-based intelligent station entrance and exit automatic inspection method of the present invention can be seamlessly integrated with existing CCTV systems, PIS systems and other station management systems, which is convenient for expansion and upgrading, and realizes intelligent and automated station management. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is a flow chart of an automatic inspection method for smart station entrances and exits based on image processing of the present invention;

[0074] Figure 2 This is a connection diagram of the intelligent station entrance automatic inspection system for image processing in the third embodiment of the present invention;

[0075] Figure 3 This is a flow chart of the automatic inspection method for smart station entrances and exits based on image processing in Example 3 of the present invention. DETAILED DESCRIPTION

[0076] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0077] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0078] Embodiment 1

[0079] See also Figure 1This embodiment provides a method for automatic inspection of smart station entrances and exits based on image processing, the method comprising:

[0080] S1. Performing patrol inspection on the surveillance images captured by the entrance and exit of the subway station by the CCTV surveillance system installed at the exit of the subway station, and obtaining a first patrol inspection result and a second patrol inspection result of the surveillance images;

[0081] The first inspection result is that there is a gathering of people at the entrance and exit of the subway station in the monitoring picture or there is no gathering of people at the entrance and exit of the subway station in the monitoring picture;

[0082] The second inspection result is that the guide light in the monitoring screen is on or the guide light in the monitoring screen is off;

[0083] The S1 specifically includes:

[0084] S11, using a crowd gathering detection algorithm to process a monitoring picture taken by a CCTV monitoring system installed at a subway station exit, and obtaining a first inspection result of the monitoring picture;

[0085] S12. Using a guide light on / off detection algorithm, the CCTV monitoring system installed at the subway station exit processes the monitoring picture taken at the subway station entrance and exit to obtain a second inspection result of the monitoring picture.

[0086] The S11 specifically includes:

[0087] S11-1, using a background subtraction algorithm to perform background subtraction processing on the monitoring images taken by the CCTV monitoring system installed at the subway station exit, to obtain the monitoring images after the background subtraction processing;

[0088] S11-2, using a target detection algorithm to identify and mark all personnel in the surveillance image after background subtraction processing, and obtain a personnel location image;

[0089] S11-3, using a density estimation algorithm to convert the personnel location image into a density map;

[0090] S11-4, based on the density map, obtaining a density value at any position in the density map;

[0091] S11-5. Determine a first inspection result of the monitoring image based on a density value at any position in the density map and a preset density threshold.

[0092] S2, based on the second inspection result and the pre-obtained inspection result of the escalator operation status, judging whether the guide light in the monitoring screen is displayed normally according to the pre-set logic, and obtaining the display inspection result of the guide light;

[0093] The escalator operation status inspection result includes escalator status information and escalator operation direction information; the escalator status information is escalator fault status information or escalator normal status information.

[0094] For example, at the exit of a subway station, guide lights are usually used to indicate the direction of travel or remind passengers of the status of the escalator (such as "escalator in operation" or "escalator paused"). In order to ensure that the guide lights work properly, it is necessary not only to detect their on and off status, but also to combine the actual operating status of the escalator to determine whether the guide lights are displayed correctly.

[0095] Second inspection result: Assume that through the previous step S12, it is found that the guide light is currently in the on state ("guide light is on"). Escalator operation status inspection result: Assume that in the previous inspection, we learned that the escalator is currently in the running state ("escalator is running").

[0096] The pre-set logic judgment process in this embodiment is: Assume that the pre-set logic is: Logic 1: When the escalator is running, the guide light should be on and display "Escalator Running". Logic 2: When the escalator stops, the guide light should be off or display "Escalator Paused". In this embodiment, the escalator is running, so according to Logic 1, the guide light should be on.

[0097] According to the second inspection result "guide light on" and the escalator inspection result "escalator running", it can be determined whether the status of the guide light is normal. According to logic 1, if the guide light is on and displays "escalator running" when the escalator is running, the guide light display is normal. Assuming that the guide light is indeed on and displays "escalator running" at this time, it can be concluded that the display inspection result of the guide light is normal. However, if the guide light is on, but displays an error message (for example, it displays "escalator paused") or is off, this will be judged as "the guide light display is abnormal." Guide light display inspection result: the guide light is on and displays the correct information when the escalator is running, so the display is normal.

[0098] S3, for the collected images of the PIS screen in the CCTV monitoring system set at the exit of the subway station, screen fault detection and text content detection are performed in sequence to obtain the screen fault detection result and text content detection result of the PIS screen;

[0099] S4. Generate an inspection report based on the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen, and the text content detection result;

[0100] The inspection report includes the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen and the text content detection result.

[0101] For example, suppose a CCTV surveillance camera is installed at the exit of a subway station to monitor the flow of people and the status of the guide lights. An automatic inspection method based on image processing is used to analyze the surveillance footage to determine whether people are gathering and the status of the guide lights.

[0102] The images captured by surveillance cameras usually contain static backgrounds (such as floors and walls) and dynamic foregrounds (such as walking people). In this step, a background subtraction algorithm is used to remove the static background from the surveillance image and only retain the dynamic foreground. For example, the images of the floor and walls are removed from the image to obtain an image containing only people. In the image after background subtraction, the target detection algorithm is used to identify and mark the positions of all people. For example, the positions of 5 people are marked in the image, and each person is circled with a border. Next, the image with the marked positions of people is converted into a density map. The density map is a two-dimensional image, in which the value of each position indicates the density of people in the area. For example, in the density map at a certain exit, if there are many people standing together, the density value of the area will be higher. Extract the density value of any position from the density map. For example, extract the density value at the center of the exit. If the value is high, it means that the people are densely packed at this position. Based on the extracted density value, it is compared with the preset density threshold. If the density value exceeds the threshold, it is judged that there is a gathering of people in the surveillance image; otherwise, it is judged that there is no gathering of people.

[0103] In the step of detecting whether the guide light is on or off in this embodiment, the same monitoring screen is analyzed, but this time the target is the state of the guide light. For example, the color recognition algorithm in the image is used to determine whether the guide light is on (glowing) or off. Assuming that the guide light is at a specific position in the picture, when an obvious brightness value is detected at that position (for example, the RGB value is higher than a certain threshold), the guide light is determined to be on; otherwise it is determined to be off. The first inspection result: Through the calculation and comparison of the density value, we know that there is currently a gathering of people at the exit of the subway station. The second inspection result: The guide light is currently in a bright state. This process demonstrates how to automatically analyze the monitoring screen through an image processing algorithm, obtain the real-time status of the entrances and exits of the subway station, and make corresponding judgments.

[0104] The image processing-based intelligent station entrance automatic inspection method and system in this embodiment, because the station entrance is automatically inspected by CCTV monitoring system, the workload of manual inspection is reduced and the efficiency is improved, especially in the detection of crowded areas and equipment failures. The method integrates crowd gathering detection, guide light status detection, escalator operation status inspection, PIS screen fault detection and text content detection, ensuring the normal operation of each important link of the station entrance.

[0105] In the practical application of this embodiment, the S11-4 specifically includes:

[0106] Based on the density map, the density value of any position in the density map is obtained using formula (1);

[0107] The formula (1) is:

[0108]

[0109] D(x, y) represents the density value at the position (x, y) in the density map;

[0110] K(xx i ,yy i ) represents any other i-th position (x i ,y i ) on the (x, y) position;

[0111] δ i is an indicator function, indicating that at position (x i ,y i ) Is there any personnel present? If yes, then i =1, otherwise δ i =0; In this embodiment, the indicator function makes the calculation of density value only consider those locations where people actually exist, without being disturbed by blank areas. In this way, the formula only calculates where there are actually people, avoiding the generation of false density.

[0112] n is the total number of people in the density map;

[0113]

[0114] Wherein, σ is the standard deviation of the Gaussian distribution; in this embodiment, σ determines the influence range between positions. When v is large, the influence range between positions is expanded, which is suitable for detecting relatively dispersed gatherings of people; when σ is small, the influence range between positions is reduced, which is suitable for detecting closely packed gatherings of people.

[0115] exp is the exponential function with base e.

[0116] In this embodiment, by calculating the density value of any position in the density map, formula (1) can accurately reflect the distribution of people around the position. Formula (1) takes into account the influence of each position and weights it according to the distance between the positions, so that the density value depends not only on the position itself, but also on the distribution of people at the surrounding positions. Since Gaussian distribution is used in formula (1) to smooth and weight the influence between positions, the influence of farther positions is naturally attenuated, which conforms to the physical characteristics of people gathering in real scenes.

[0117] In the practical application of this embodiment, S3 specifically includes:

[0118] S31, using the PIS screen on / off detection algorithm, to perform screen fault detection on the image of the PIS screen in the CCTV monitoring system installed at the exit of the subway station, and obtain the screen fault detection result of the PIS screen;

[0119] The screen fault detection result of the PIS screen is that the screen of the PIS screen is in a lit state or the screen of the PIS screen is in an off state;

[0120] S32, when the screen fault detection result of the PIS screen is that the screen of the PIS screen is in a light-up state, using an OCR text recognition algorithm to perform text recognition detection on the screen of the PIS screen to obtain a text content recognition result;

[0121] The text content recognition result includes: the text content in the image of the PIS screen recognized by the OCR text recognition algorithm;

[0122] S33, comparing the text content recognition result with the preset text content to obtain a comparison result;

[0123] S34: Determine whether the text content detection result is normal or abnormal based on the comparison result and a preset similarity threshold.

[0124] For example, suppose a CCTV monitoring system is installed at the exit of a subway station to monitor the status and display content of the PIS screen (passenger information system screen). The PIS screen usually displays important information, such as train arrival time, line information, or emergency notices. Image processing technology is needed to automatically detect the status of the PIS screen to ensure that it operates normally and displays the correct content. First, use the on-off detection algorithm to analyze whether the PIS screen is on. For example, the algorithm detects the brightness value of the screen area. If the brightness value is higher than a certain threshold, the screen is judged to be "on"; otherwise, it is "off". Example: Assume that in the current screen, the brightness detection result of the PIS screen area is higher than the set threshold, so the screen fault detection result of the PIS screen is "the screen of the PIS screen is on". After determining that the PIS screen is on, the OCR (optical character recognition) algorithm is used to recognize the text content in the PIS screen. The OCR algorithm converts the image text in the PIS screen into readable text data. Example: Assume that the OCR recognizes the text displayed on the PIS screen as: "The next train will arrive in 5 minutes." Compare the text content recognized by OCR with the preset text content to confirm whether the information displayed on the screen is correct. Example: Assume that the preset correct text content is: "The next train will arrive in 5 minutes", the text recognized by OCR is exactly the same as the preset text, so the comparison result is consistent. According to the text comparison result and the preset similarity threshold, determine whether the text content detection result of the PIS screen is normal or abnormal. Example: If the text is exactly the same or the similarity exceeds the set threshold (such as similarity>90%), the text content detection result is normal. If the text recognized by OCR is significantly different from the preset content or the similarity is lower than the threshold (such as similarity<90%), the result is abnormal.

[0125] This process in this embodiment ensures that the PIS screen is not only physically lit up normally, but also displays the correct information. Through on-off detection, OCR recognition and text comparison, the system can automatically monitor the status of the PIS screen, discover and report potential problems in a timely manner, and ensure the accuracy and timeliness of passenger information.

[0126] Specifically, the S33 specifically includes:

[0127] Formula (2) is used to calculate the total similarity between the text content recognition result and the preset text content, and the total similarity is used as the comparison result;

[0128] The formula (2) is:

[0129] O = α × C + β × P + γ × S;

[0130] Among them, O is the total similarity between the text content recognition result and the pre-set text content; C is the character matching degree between the text content recognition result and the pre-set text content; P is the position matching degree between the text content recognition result and the pre-set text content; S is the semantic matching degree between the text content recognition result and the pre-set text content; α, β, γ are pre-set weighting coefficients respectively, and 1=α+β+γ.

[0131] In this embodiment, the character matching, position matching and semantic matching are combined to calculate the total similarity in a weighted manner, which can fully reflect the similarity of the text content. This comprehensive consideration method can capture different aspects of the text content, such as the accuracy of the characters, the relative position of the characters in the text, and the semantic similarity of the text, avoiding errors caused by a single factor.

[0132] The character matching degree C between the text content recognition result and the preset text content is calculated by formula (3);

[0133] The formula (3) is:

[0134]

[0135] Wherein, MC is the number of characters that match between the text content recognition result and the pre-set text content; M is the total number of characters in the text content recognition result; TCR is the total number of characters in the pre-set text content;

[0136] Formula (2) of this embodiment can accurately quantify the character consistency between the recognized text and the preset text, and is applicable to situations where characters in the recognition results are missing, replaced or added, thereby ensuring the accuracy of character matching.

[0137] The position matching degree P between the text content recognition result and the preset text content is calculated by formula (4);

[0138] The formula (4) is:

[0139]

[0140] m is the mth character in the text content recognition result;

[0141] dm is the Euclidean distance between the position of the mth character in the recognized text and the corresponding character position in the preset text in the text content recognition result; τ is a preset value;

[0142] In this embodiment, the Euclidean distance is used to calculate the distance between the character position in the text content recognition result and the corresponding character position in the preset text to obtain the position matching degree. Formula (4) takes into account the spatial position of the character in the text, so that even if the characters are the same, if the position deviation is large, it can be reflected in the matching degree, thereby improving the sensitivity to the change of text layout.

[0143] The semantic matching degree S between the text content recognition result and the preset text content is calculated by formula (5);

[0144] The formula (5) is:

[0145]

[0146] M4∪TCR is the total number of characters in the union of the total characters in the text content recognition result and the total characters of the preset text content; ω is the preset character importance factor.

[0147] Specifically, the S34 specifically includes:

[0148] If the comparison result is greater than the preset similarity threshold, determining that the text content detection result is normal;

[0149] If the comparison file is less than or equal to the preset similarity threshold, the text content detection result is determined to be abnormal.

[0150] In this embodiment, the threshold judgment mechanism provides a clear judgment standard, ensuring that within the set similarity range, it is possible to accurately judge whether the text is displayed normally, thereby reducing the risk of misjudgment and missed judgment.

[0151] Embodiment 2

[0152] This embodiment also provides an intelligent station entrance and exit automatic inspection system based on image processing, including: at least one processor; and at least one memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the intelligent station entrance and exit automatic inspection method based on image processing as described in Example 1.

[0153] Embodiment 3

[0154] See also Figure 2, the present embodiment provides an automatic inspection method for smart station entrances and exits based on image processing, which is executed by an automatic inspection system for smart station entrances and exits based on image processing. The system collects the CCTV (closed-circuit television) monitoring system of comprehensive monitoring to realize the entrance and exit images, and is also connected to the escalator system and the PIS monitoring system respectively; the present embodiment provides an automatic inspection method for smart station entrances and exits based on image processing, and processes the collected images through a video analysis system. Specifically, it includes a personnel gathering detection algorithm, a guide light on and off detection algorithm, and a PIS screen text recognition algorithm, and outputs the video image analysis results. At the same time, the escalator system is used to detect the operating status of the entrance and exit escalators, and combined with the guide light video image analysis results, it is realized whether the light box content is displayed normally, thereby realizing the automatic inspection of the entrance and exit area, and generating the analysis results of the entrance and exit inspection items.

[0155] This embodiment provides a method for automatic inspection of smart station entrances and exits based on image processing, see Figure 3 , specifically including:

[0156] First, according to the CCTV monitoring system at the exit of the subway station, real-time monitoring images are obtained, images of entrances and exits are collected, and a crowd gathering detection algorithm is used to determine whether there is a crowd gathering at the entrance and exit, and the inspection results are recorded.

[0157] For the guide lights in the entrance and exit images, the guide light on and off detection algorithm is used to detect the on and off status of the guide lights, and the on and off inspection status of the guide lights is recorded based on the detection results.

[0158] Check whether there is any alarm information at the escalator fault alarm point in the escalator system. If there is a fault alarm in the escalator, record the escalator fault inspection item; if there is no fault alarm, record the inspection item as normal. Then, according to the running direction of the escalator, combined with the detection result of the light position of the guide light, make a business judgment, output the judgment result of whether the direction display of the guide light is normal, and record the inspection result of the content displayed by the guide light.

[0159] Subsequently, using the PIS (Passenger Information System) screen information collection camera in the CCTV monitoring system, first use the PIS screen on / off detection algorithm to detect the on / off status of the entrance and exit PIS screen. If the PIS screen fails and the screen is off, record the PIS screen fault inspection abnormality item; if the PIS screen is on, use the PIS screen OCR (optical character recognition) text recognition algorithm to detect the text in the PIS screen area. Then, combine the PIS content pre-recorded in the comprehensive monitoring PIS monitoring system to verify the recognition results. According to the verification results, determine whether the text content displayed on the current PIS screen is consistent with the pre-recorded PIS content. If consistent, record the inspection item as normal; if inconsistent, record the PIS screen content inspection item as abnormal. Finally, based on the above-mentioned inspection results, output the inspection report of the entrance and exit area.

[0160] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0161] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0162] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0163] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0164] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for automatic inspection of smart station entrances and exits based on image processing, characterized in that: The method comprises: S1. Performing patrol inspection on the surveillance images captured by the entrance and exit of the subway station by the CCTV surveillance system installed at the exit of the subway station, and obtaining a first patrol inspection result and a second patrol inspection result of the surveillance images; The first inspection result is that there is a gathering of people at the entrance and exit of the subway station in the monitoring picture or there is no gathering of people at the entrance and exit of the subway station in the monitoring picture; The second inspection result is that the guide light in the monitoring screen is on or the guide light in the monitoring screen is off; S2, based on the second inspection result and the pre-obtained inspection result of the escalator operation status, judging whether the guide light in the monitoring screen is displayed normally according to the pre-set logic, and obtaining the display inspection result of the guide light; S3, for the collected images of the PIS screen in the CCTV monitoring system set at the exit of the subway station, screen fault detection and text content detection are performed in sequence to obtain the screen fault detection result and text content detection result of the PIS screen; S4. Generate an inspection report based on the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen, and the text content detection result; The inspection report includes the first inspection result, the second inspection result, the display inspection result of the guide light, the screen fault detection result of the PIS screen and the text content detection result.

2. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 1 is characterized in that: The S1 specifically includes: S11, using a crowd gathering detection algorithm to process a monitoring picture taken by a CCTV monitoring system installed at a subway station exit, and obtaining a first inspection result of the monitoring picture; S12. Using a guide light on / off detection algorithm, the CCTV monitoring system installed at the subway station exit processes the monitoring picture taken at the subway station entrance and exit to obtain a second inspection result of the monitoring picture.

3. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 2 is characterized in that: The S11 specifically includes: S11-1, using a background subtraction algorithm to perform background subtraction processing on the monitoring images taken by the CCTV monitoring system installed at the subway station exit, to obtain the monitoring images after the background subtraction processing; S11-2, using a target detection algorithm to identify and mark all personnel in the surveillance image after background subtraction processing, and obtain a personnel location image; S11-3, using a density estimation algorithm to convert the personnel location image into a density map; S11-4, based on the density map, obtaining a density value at any position in the density map; S11-5. Determine a first inspection result of the monitoring image based on a density value at any position in the density map and a preset density threshold.

4. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 3 is characterized in that: The S11-4 specifically includes: Based on the density map, the density value of any position in the density map is obtained using formula (1); The formula (1) is: D(x, y) represents the density value at the position (x, y) in the density map; K(xx i ,yy i ) represents any other i-th position (x i ,y i ) on the (x, y) position; δ i is an indicator function, indicating that at position (x i ,y i ) Is there any personnel present? If yes, then i =1, otherwise δ i =0; n is the total number of people in the density map; Where σ is the standard deviation of the Gaussian distribution; exp is the exponential function with base e.

5. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 4 is characterized in that: The escalator running status inspection result includes escalator status information and escalator running direction information; The escalator status information is information that the escalator is in a fault state or information that the escalator is in a normal state.

6. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 5 is characterized in that: The S3 specifically includes: S31, using the PIS screen on / off detection algorithm, to perform screen fault detection on the image of the PIS screen in the CCTV monitoring system installed at the exit of the subway station, and obtain the screen fault detection result of the PIS screen; The screen fault detection result of the PIS screen is that the screen of the PIS screen is in a lit state or the screen of the PIS screen is in an off state; S32, when the screen fault detection result of the PIS screen is that the screen of the PIS screen is in a light-up state, using an OCR text recognition algorithm to perform text recognition detection on the screen of the PIS screen to obtain a text content recognition result; The text content recognition result includes: the text content in the image of the PIS screen recognized by the OCR text recognition algorithm; S33, comparing the text content recognition result with the preset text content to obtain a comparison result; S34: Determine whether the text content detection result is normal or abnormal based on the comparison result and a preset similarity threshold.

7. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 2 is characterized in that: The S33 specifically includes: Formula (2) is used to calculate the total similarity between the text content recognition result and the preset text content, and the total similarity is used as the comparison result; The formula (2) is: O = α × C + β × P + γ × S; Among them, O is the total similarity between the text content recognition result and the pre-set text content; C is the character matching degree between the text content recognition result and the pre-set text content; P is the position matching degree between the text content recognition result and the pre-set text content; S is the semantic matching degree between the text content recognition result and the pre-set text content; α, β, γ are pre-set weighting coefficients respectively, and 1=α+β+γ.

8. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 7 is characterized in that: The character matching degree C between the text content recognition result and the preset text content is calculated by formula (3); The formula (3) is: Wherein, MC is the number of characters that match between the text content recognition result and the pre-set text content; M is the total number of characters in the text content recognition result; TCR is the total number of characters in the pre-set text content; The position matching degree P between the text content recognition result and the preset text content is calculated by formula (4); The formula (4) is: m is the mth character in the text content recognition result; d m is the Euclidean distance between the position of the mth character in the recognized text and the position of the corresponding character in the preset text in the text content recognition result; τ is a preset value; The semantic matching degree S between the text content recognition result and the preset text content is calculated by formula (5); The formula (5) is: M∪TCR is the total number of characters in the union of the total characters in the text content recognition result and the total characters of the preset text content; ω is a preset character importance factor.

9. The method for automatic inspection of smart station entrances and exits based on image processing according to claim 8 is characterized in that: The S34 specifically includes: If the comparison result is greater than the preset similarity threshold, determining that the text content detection result is normal; If the comparison file is less than or equal to the preset similarity threshold, the text content detection result is determined to be abnormal.

10. An intelligent station entrance and exit automatic inspection system based on image processing, characterized in that: include: at least one processor; as well as At least one memory is communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the automatic inspection method for smart station entrances and exits based on image processing as described in any one of claims 1-9.

Images