Display screen fault detection method, system and device and storage medium

Abstract

The invention discloses a display screen fault detection method, system and device and a storage medium, and the method comprises the steps: obtaining a monitoring video stream of a display screen, and obtaining a display screen monitoring image set according to the monitoring video stream; performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set; performing feature extraction on the display screen correction image set to obtain a display screen feature image set; detecting the display screen feature image set by using a pre-trained fault detection model to obtain fault types corresponding to the display screen feature images in the display screen feature image set one by one; and when the fault types of all the display screen feature images meet a preset condition, determining that the display screen has a fault, and outputting a target fault type corresponding to the display screen. According to the invention, real-time intelligent fault detection of the display screen can be realized, the accuracy is high, various types of display screen faults can be detected at the same time, full-automatic intelligent detection alarm is realized, manual inspection by operators on duty is not needed, and the operation and maintenance efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of rail transportation and artificial intelligence technology, and in particular to a display screen fault detection method, system, device and storage medium. Background Art

[0002] The passenger information system is an important subsystem in rail transit communications. It is a system that relies on multimedia network technology, takes computer systems as its core, and uses stations and on-board display terminals as media to provide information services to passengers. Therefore, in this system, it is crucial whether the on-board display terminal displays normally.

[0003] The main form of on-board display terminal is the display screen. In the actual project implementation, it was found from the statistical information of the operation and maintenance department that in the long-term use, the display screen often has faults such as black screen, screen distortion and color cast. At present, for these display faults of the display screen, it is often necessary to rely on the relevant on-duty personnel to discover and report them to the system, which is not convenient for management, and usually the staff cannot discover the fault in time, and the efficiency of equipment fault handling is low, which affects the passenger experience.

[0004] Therefore, it is of great practical significance to provide a technology that can accurately detect display screen failures in real time. Summary of the invention

[0005] In view of this, the present invention provides a display screen fault detection method, system, device and storage medium to solve the problem in the prior art that the method of manually discovering display faults is inconvenient to manage and the fault detection and processing efficiency is low.

[0006] The present invention provides a display screen fault detection method for rail transit, the method comprising:

[0007] Obtaining a monitoring video stream of the display screen, and obtaining a display screen monitoring image set based on the monitoring video stream;

[0008] Performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set;

[0009] Performing feature extraction on the display screen correction image set to obtain a display screen feature image set;

[0010] Using a pre-trained fault detection model, the display screen feature image set is detected to obtain a fault type corresponding to each display screen feature image in the display screen feature image set;

[0011] When the fault types of all the display screen characteristic images meet a preset condition, it is determined that the display screen has a fault, and a target fault type corresponding to the display screen is output.

[0012] Optionally, obtaining a display screen monitoring image set according to the monitoring video stream includes:

[0013] Sampling the surveillance video stream according to a preset sampling frequency to obtain a sampled image set;

[0014] The pre-trained target detection model is used to perform target detection on the sampled image set to obtain the display screen monitoring image set.

[0015] Optionally, the target detection model includes any one of a Faster R-CNN model, an R-FCN model and an SSD model.

[0016] Optionally, performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set includes:

[0017] Selecting any monitoring image from the display screen monitoring image set, performing grayscale processing and Gaussian blur filtering processing on the selected monitoring image in sequence, and obtaining a display screen processing image corresponding to the selected monitoring image;

[0018] Perform edge detection on the selected display screen processed image to obtain a corresponding display screen contour image;

[0019] According to the display screen outline image, determining the coordinates of four vertices in the display screen outline image;

[0020] Acquire a target image corresponding to the display screen outline image and target coordinates of the four vertex coordinates in the target image;

[0021] Calculating a perspective transformation matrix according to the four vertex coordinates and the four target coordinates;

[0022] Performing perspective transformation on the selected monitoring image according to the perspective transformation matrix to obtain a display screen correction image corresponding to the selected monitoring image;

[0023] Traversing each monitoring image in the display screen monitoring image set, and performing perspective transformation on each monitoring image in the same way to obtain the display screen correction image corresponding to each monitoring image one by one;

[0024] The display screen correction image set is obtained according to all the display screen correction images.

[0025] Optionally, the extracting features from the display screen correction image set to obtain a display screen feature image set includes:

[0026] Selecting any one display screen correction image in the display screen correction image set, and sequentially upsampling and normalizing the selected display screen correction image to obtain a normalized image corresponding to the selected display screen correction image;

[0027] Segmenting the selected normalized image to obtain a plurality of display screen sub-images corresponding to the selected normalized image;

[0028] Performing feature extraction on each display screen sub-image in the selected normalized image respectively, and obtaining a segmentation feature image corresponding to each display screen sub-image in the selected normalized image;

[0029] Performing feature fusion on all segmented feature images in the selected normalized image to obtain a display screen feature image corresponding to the selected normalized image;

[0030] Traversing each display screen correction image in the display screen correction image set, and obtaining a display screen characteristic image corresponding to each display screen correction image in the same manner;

[0031] The display screen characteristic image set is obtained according to the display screen characteristic images of all display screen correction images.

[0032] Optionally, the use of a pre-trained fault detection model to detect the display screen feature image set to obtain a fault type corresponding to each display screen feature image in the display screen feature image set includes:

[0033] Inputting the display screen feature image set into the fault detection model to obtain the proportion of each display screen feature image in the display screen feature image under each potential fault type;

[0034] Select any one display screen characteristic image, and determine the potential fault type whose proportion is greater than a first preset threshold as the fault type corresponding to the selected display screen characteristic image;

[0035] According to the same method, the fault type corresponding to each display screen characteristic image in the display screen characteristic image set is determined.

[0036] Optionally, when the fault types of all the display screen characteristic images meet a preset condition, determining that the display screen has a fault, and outputting a target fault type corresponding to the display screen include:

[0037] Counting the cumulative amount of all the characteristic images of the display screen under each of the fault types;

[0038] When the accumulated amount is greater than a second preset threshold, it is determined that the display screen has a fault and a fault alarm signal is issued; the fault type corresponding to the accumulated amount being greater than the second preset threshold is determined as the target fault type of the display screen and output.

[0039] In addition, the present invention also provides a display screen fault detection system for rail transit, which is applied to the above-mentioned display screen fault detection method, and includes:

[0040] An image acquisition module is used to acquire a monitoring video stream of a display screen, and obtain a display screen monitoring image set according to the monitoring video stream;

[0041] An image transformation module, used for performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set;

[0042] A feature extraction module, used to extract features from the display screen correction image set to obtain a display screen feature image set;

[0043] A fault classification module is used to detect the display screen feature image set using a pre-trained fault detection model to obtain a fault type corresponding to each display screen feature image in the display screen feature image set;

[0044] The fault determination module is used to determine that the display screen has a fault when the fault types of all the characteristic images of the display screen meet a preset condition, and output a target fault type corresponding to the display screen.

[0045] In addition, the present invention also provides a display screen fault detection device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps in the aforementioned display screen fault detection method when executed.

[0046] In addition, the present invention also provides a computer storage medium, which includes: at least one instruction, and when the instruction is executed by a computer, the method steps in the above-mentioned display screen fault detection method are implemented.

[0047] The beneficial effects of the present invention are as follows: first, a monitoring video stream is used to obtain a display screen monitoring image set, which can automatically locate the position of the display screen in the monitoring video stream, and extract an image set of an area of ​​interest, that is, a display screen monitoring image set containing the display screen; then, through perspective transformation of the image set, it can adapt to the display screen monitoring image set under different viewing angles in the operation and maintenance monitoring video, thereby realizing display fault detection under different viewing angles; then, through feature extraction, the key features in the display screen correction image set can be accurately captured to obtain a display screen feature image set, which is convenient for subsequent use of a fault detection model to automatically identify and multi-classify each display screen feature image in the display screen feature image set on the fault type, and obtain a one-to-one corresponding fault type for each display screen feature image; finally, when the fault types of all display screen feature images meet preset conditions, it is determined that the display screen has a fault and its corresponding target fault type is determined;

[0048] The display screen fault detection method, system, device and storage medium of the present invention can realize real-time intelligent detection of display screen faults with high accuracy, and can simultaneously detect multiple types of display screen faults, and can automatically detect and alarm intelligently, without the need for manual inspection by on-duty personnel, thereby improving operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0050] Figure 1 A flow chart of a display screen fault detection method in Embodiment 1 of the present invention is shown;

[0051] Figure 2 A flow chart of obtaining a display screen monitoring image set in the first embodiment of the present invention is shown;

[0052] Figure 3 A flowchart of perspective transformation of a display screen monitoring image set in the first embodiment of the present invention is shown;

[0053] Figure 4 A flow chart showing feature extraction of a display screen correction image set in the first embodiment of the present invention is shown;

[0054] Figure 5 A flow chart of detecting a display screen feature image set in the first embodiment of the present invention is shown;

[0055] Figure 6 It shows a flow chart of determining that a display screen has failed and outputting a target failure type corresponding to the display screen in the first embodiment of the present invention;

[0056] Figure 7 A schematic structural diagram of a display screen fault detection system in Embodiment 2 of the present invention is shown. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0058] Embodiment 1

[0059] like Figure 1 As shown, a display screen fault detection method is used in rail transportation, and the method includes:

[0060] S1: Obtain a monitoring video stream of a display screen, and obtain a display screen monitoring image set according to the monitoring video stream;

[0061] S2: Performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set;

[0062] S3: extracting features from the display screen correction image set to obtain a display screen feature image set;

[0063] S4: using a pre-trained fault detection model, detecting the display screen feature image set to obtain a fault type corresponding to each display screen feature image in the display screen feature image set;

[0064] S5: When the fault types of all the display screen characteristic images meet a preset condition, it is determined that the display screen has a fault, and a target fault type corresponding to the display screen is output.

[0065] In this embodiment, the monitoring video stream is first used to obtain a display screen monitoring image set, which can automatically locate the position of the display screen in the monitoring video stream and extract an image set of an area of ​​interest, that is, a display screen monitoring image set containing the display screen; then, through the perspective transformation of the image set, it can adapt to the display screen monitoring image set under different viewing angles in the operation and maintenance monitoring video, thereby realizing display fault detection under different viewing angles; then, through feature extraction, the key features of the display screen correction image set can be accurately captured to obtain a display screen feature image set, which is convenient for the subsequent use of a fault detection model to automatically identify and multi-classify each display screen feature image in the display screen feature image set on the fault type, and obtain a one-to-one corresponding fault type for each display screen feature image; finally, when the fault types of all display screen feature images meet the preset conditions, it is determined that the display screen has a fault and its corresponding target fault type is determined.

[0066] The display screen fault detection method of this embodiment can realize real-time intelligent detection of display screen faults with high accuracy, and can detect multiple types of display screen faults at the same time, with fully automatic intelligent detection and alarm, without the need for manual inspection by on-duty personnel, thereby improving operation and maintenance efficiency.

[0067] In this embodiment, the display screen includes but is not limited to LCD display screen, LED display screen and OLED display screen, etc. For the convenience of explanation, the display screen in this embodiment takes LCD display screen as an example, including dynamic map screen and passenger compartment LCD screen used in rail transportation tools. .

[0068] The fault types of the display screen in this embodiment include black screen, blue screen, distorted screen, color cast, video freeze, etc. Since the display screen monitoring video stream is collected in real time, the display screen monitoring image set obtained according to the display screen monitoring video stream is also real-time, so the process of fault detection according to the display screen monitoring image set is real-time, and the target fault type obtained is also real-time. The target fault type detected at a certain moment can be one or more, that is, one or more display screen faults can be detected at the same time at each moment.

[0069] In this embodiment, rail transit includes transportation tools such as buses, subways, trains, and high-speed trains.

[0070] Each step of the display screen fault detection method of this embodiment is described in detail below.

[0071] In the present embodiment S1, the monitoring video stream of the display screen can be acquired by a monitoring device installed in the rail transit vehicle (such as a camera installed directly opposite the display screen).

[0072] Preferably, if Figure 2 As shown, in this embodiment S1, a display screen monitoring image set is obtained according to the monitoring video stream, including:

[0073] S11: sampling the surveillance video stream according to a preset sampling frequency to obtain a sampled image set;

[0074] S12: Using a pre-trained target detection model, perform target detection on the sampled image set to obtain the display screen monitoring image set.

[0075] By regularly sampling the monitoring video stream, real-time sampling images of the display screen can be obtained. Each time a sample is taken, a corresponding sampling image is obtained. The final sampling image set can not only reflect the real characteristics of the display screen, which is convenient for the target fault type detected in the subsequent fault detection to be consistent with the real situation of the display screen, ensuring the accuracy of fault detection; it can also avoid processing and analyzing the entire monitoring video stream, reduce the amount of data, and improve the fault detection efficiency to a certain extent on the basis of ensuring the accuracy of fault detection. Using the pre-trained target detection model, the area of ​​interest can be automatically located to obtain the monitoring image that truly contains the display screen (i.e., the display screen monitoring image set). On the one hand, it avoids the interference of the image of the non-display area on the image of the display area, and on the other hand, it further reduces the amount of data and further improves the fault detection efficiency.

[0076] Specifically, in S11 of this embodiment, the preset sampling frequency can be pre-set and adjusted according to actual conditions, such as regular sampling at a fixed frequency of 5fps, 10fps, etc.

[0077] Specifically, in S12 of this embodiment, the target detection model includes any one of a Faster R-CNN model, an R-FCN model and an SSD model.

[0078] Faster R-CNN is an object detection model based on the regional convolutional neural network (RCNN). It introduces the region generation network (RPN) to achieve end-to-end training, which greatly improves the detection speed and accuracy.

[0079] FCN (Region-based Fully Convolutional Network) is a target detection model based on the fully convolutional network (FCN). It solves the problem of spatial information loss in target detection by introducing a position-sensitive score map.

[0080] SSD (Single Shot Multibox Detector) is a single-stage object detector that achieves fast and accurate object detection by simultaneously completing object classification and bounding box regression in one network.

[0081] Using any of the three target detection models mentioned above to perform target detection on the sampled image set can automatically locate the position of the display screen in the sampled image set more efficiently and accurately, thereby obtaining an accurate display screen monitoring image set.

[0082] The specific target detection process and training process of the above three target detection models in this embodiment all adopt conventional operation methods, and the specific details are not repeated here.

[0083] Preferably, if Figure 3 As shown, this embodiment S2 includes:

[0084] S21: Selecting a monitoring image from the display screen monitoring image set, and sequentially performing grayscale processing and Gaussian blur filtering processing on the selected monitoring image to obtain a display screen processing image corresponding to the selected monitoring image;

[0085] S22: performing edge detection on the selected display screen processed image to obtain a corresponding display screen contour image;

[0086] S23: determining the coordinates of four vertices in the display screen outline image according to the display screen outline image;

[0087] S24: Acquire a target image corresponding to the display screen outline image and target coordinates of the four vertex coordinates in the target image respectively;

[0088] S25: Calculating a perspective transformation matrix according to the four vertex coordinates and the four target coordinates;

[0089] S26: performing perspective transformation on the selected monitoring image according to the perspective transformation matrix to obtain a display screen correction image corresponding to the selected monitoring image;

[0090] S27: traversing each monitoring image in the display screen monitoring image set, and performing perspective transformation on each monitoring image in the same way to obtain the display screen correction image corresponding to each monitoring image one by one;

[0091] S28: Obtain the display screen correction image set according to all the display screen correction images.

[0092] In practical applications, it is difficult for a monitoring device to collect the video status of an LCD display screen from a 90-degree vertical viewing angle. Therefore, perspective distortion exists in the display screen monitoring image set, which needs to be corrected by perspective transformation to eliminate the perspective distortion.

[0093] In the above perspective transformation process, grayscale processing and Gaussian blur filtering are first performed, which can effectively reduce the influence of regional noise and facilitate the accuracy of subsequent perspective transformation; then edge detection is performed to obtain the display screen contour image, which can facilitate the accurate positioning of the four vertex coordinates. The four vertex coordinates are non-collinear points and can effectively serve as the reference points for subsequent perspective transformation; then the target image corresponding to the display screen contour image and the position coordinates of the four vertex coordinates in the target image (i.e., target coordinates) are obtained, which can facilitate the calculation of the perspective transformation matrix of the entire monitoring image based on the four vertex coordinates and the four target coordinates; finally, the calculated perspective transformation matrix is ​​used to perform perspective transformation on the entire monitoring image, and the display screen correction image of the monitoring image after perspective transformation can be obtained, so as to realize distortion correction under different viewing angles and adapt to LCD display screen fault detection under different viewing angles in the operation and maintenance monitoring video. Among them, each monitoring image in the display screen monitoring image set is perspective transformed according to the same method, and the display screen correction image set corresponding to the entire display screen monitoring image set can be obtained.

[0094] Among them, in this embodiment S25, the cv2.getPerspectiveTransform(scr,dst) function in the image processing library such as OpenCV can be used, and the four vertex coordinates in the original image and the corresponding four target coordinates in the target image are taken as input, where scr is the input variable of the four vertex coordinates, dst is the input variable of the four target coordinates, and the four vertex coordinates and the four target coordinates are all input in the form of arrays. The function will calculate the perspective transformation matrix based on the correspondence between these points.

[0095] Preferably, if Figure 4 As shown, this embodiment S3 includes:

[0096] S31: selecting any one display screen correction image in the display screen correction image set, and sequentially upsampling and normalizing the selected display screen correction image to obtain a normalized image corresponding to the selected display screen correction image;

[0097] S32: Segment the selected normalized image to obtain a plurality of display screen sub-images corresponding to the selected normalized image;

[0098] S33: extracting features from each display screen sub-image in the selected normalized image, respectively, to obtain a segmented feature image corresponding to each display screen sub-image in the selected normalized image;

[0099] S34: performing feature fusion on all segmented feature images in the selected normalized image to obtain a display screen feature image corresponding to the selected normalized image;

[0100] S35: traversing each display screen correction image in the display screen correction image set, and obtaining a display screen feature image corresponding to each display screen correction image in the same manner;

[0101] S36: Obtaining the display screen characteristic image set according to the display screen characteristic images of all display screen correction images.

[0102] For any display screen correction image, upsampling is first performed to improve image resolution and image quality, and then normalization is performed to help converge faster in subsequent multi-classification, reduce the amount of calculation, and improve accuracy; then the normalized image after upsampling and normalization is segmented, and feature extraction is performed on each display screen sub-image after segmentation. On the one hand, the processing process can be simplified, and on the other hand, the feature extraction accuracy can be improved through local processing after segmentation, which is more helpful to capture the most critical features; finally, feature fusion is performed on the segmented feature image obtained by feature extraction after segmentation, which can restore the segmented image to a complete image on the basis of ensuring all features, ensuring the overall image quality of the complete image, and can adapt to the automatic intelligent detection of faults when the LCD display screen is partially blocked in the operation and maintenance monitoring video, so that the subsequent multi-classification of fault types is more in line with the actual situation of the display screen, and effectively improves the accuracy of feature extraction of the display screen correction image set.

[0103] Preferably, if Figure 5 As shown, this embodiment S4 includes:

[0104] S41: inputting the display screen feature image set into the fault detection model to obtain the proportion of each display screen feature image in the display screen feature image under each potential fault type;

[0105] S42: selecting any one display screen characteristic image, and determining the potential fault type whose proportion is greater than a first preset threshold as the fault type corresponding to the selected display screen characteristic image;

[0106] S43: According to the same method, the fault type corresponding to each display screen characteristic image in the display screen characteristic image set is determined.

[0107] Through the above process, the fault type corresponding to each display feature image can be detected in real time, and the multi-classification of each display feature image on the fault type can be realized efficiently and accurately, rather than being limited to a single fault type, which is more in line with the actual situation of the display screen in rail transit. Among them, there may be features under multiple potential fault types in any display feature image, which may be classified into multiple potential fault types during the multi-classification process, and the proportion under each potential fault type is output at the same time. At this time, the potential fault type with a proportion greater than the first preset threshold is determined as its corresponding fault type, and the potential fault type with a proportion less than or equal to the first preset threshold is discarded, which is more in line with the actual fault type of the display screen in rail transit.

[0108] It should be understood that the first preset threshold can be pre-set and adjusted according to actual conditions, and is not limited here. For any display feature image, there may be one or more potential fault types whose proportion is greater than the first preset threshold, and the corresponding fault types determined are one or more. For example, the fault types determined by a certain display feature image are screen distortion, color cast, and video freeze.

[0109] Preferably, after S4, the method further includes:

[0110] S6: Pre-build database;

[0111] S7: taking each display screen characteristic image and the fault type corresponding to each display screen characteristic image as historical display screen fault data, and storing the historical display screen fault data in a database;

[0112] S8: Optimize the pre-trained fault detection model using the historical display screen fault data in the database.

[0113] By storing these feature images and their fault types, it is possible to accumulate LCD screen fault image data during actual operation, providing a continuously updated data basis for subsequent training and optimization of fault detection models. This not only realizes the effective use of monitoring system data, but also achieves continuous iterative optimization of fault detection models, thereby improving the accuracy and robustness of the system.

[0114] Preferably, if Figure 6 As shown, this embodiment S5 includes:

[0115] S51: Counting the cumulative amount of all the characteristic images of the display screen under each of the fault types;

[0116] S52: When the accumulated amount is greater than a second preset threshold, it is determined that the display screen has a fault and a fault alarm signal is issued; the fault type corresponding to the accumulated amount being greater than the second preset threshold is determined as the target fault type of the display screen and outputted.

[0117] When the cumulative amount of the same type of faults is greater than the second preset threshold, the display screen will be finally determined to be faulty, and the same type of fault will be determined as the final target fault type and output; this embodiment determines whether the display screen has a fault and the corresponding target fault type by the cumulative amount under each fault type, which can comprehensively consider the fault detection situation of the entire display screen feature image set, better reflect the actual situation of the display screen in rail transit vehicles, and the fault detection accuracy is higher. Among them, when the display screen is determined to be faulty, a fault alarm signal is issued, which can promptly notify relevant personnel and make decisions, which helps to improve the passenger experience.

[0118] It should be understood that, similar to the first preset threshold, the second preset threshold can also be pre-set and adjusted according to actual conditions, and is not limited here. For all display feature images, there may be one or more fault types whose cumulative amount is greater than the second preset threshold, and the determined target fault type corresponds to one or more. For example, if the fault type whose cumulative amount in all display feature images is greater than the second preset threshold is a flower screen, then the display screen fails and the target fault type is a flower screen. When the cumulative amount of the same type of fault type in all display feature images does not exceed the second preset threshold, it can be determined that the display screen has not failed and no fault alarm signal is issued.

[0119] The above-mentioned complete display screen fault detection method of this embodiment has the following advantages:

[0120] 1. Use deep learning technology to determine the fault type. By training a large amount of image data, the model can automatically learn and identify the fault characteristics of the LCD screen, realize automatic identification of the fault type, and provide real-time feedback on the detection results during train operation to improve passengers' travel experience. The fault detection model can simultaneously detect multiple types of LCD display faults, such as black screen, blue screen, flowery screen, color cast, video freeze, and other faults.

[0121] 2. Use the pre-trained target detection model to obtain the display monitoring image set, which can automatically locate the LCD display position in the monitoring video stream and accurately extract the image of the display area.

[0122] 3. Perspective transformation is used to achieve perspective correction of display screen images, which can adapt to LCD display screen fault detection under different viewing angles in operation and maintenance monitoring videos.

[0123] 4. The feature segmentation and splicing strategy is adopted, which can adapt to the automatic intelligent detection of faults when the LCD display screen is partially blocked in the operation and maintenance monitoring video.

[0124] 5. By storing and accumulating historical display screen failure data, continuously updating and optimizing deep learning algorithms, effective utilization of monitoring system video data is achieved.

[0125] 6. The entire detection method is highly scalable and customizable, and the model parameters and structure can be adjusted according to the detection requirements of different LCD display fault types.

[0126] Embodiment 2

[0127] like Figure 7 As shown, a display screen fault detection system is used for rail transit and is applied to the display screen fault detection method of embodiment 1, comprising:

[0128] An image acquisition module is used to acquire a monitoring video stream of a display screen, and obtain a display screen monitoring image set according to the monitoring video stream;

[0129] An image transformation module, used for performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set;

[0130] A feature extraction module, used to extract features from the display screen correction image set to obtain a display screen feature image set;

[0131] A fault classification module is used to detect the display screen feature image set using a pre-trained fault detection model to obtain a fault type corresponding to each display screen feature image in the display screen feature image set;

[0132] The fault determination module is used to determine that the display screen has a fault when the fault types of all the characteristic images of the display screen meet a preset condition, and output a target fault type corresponding to the display screen.

[0133] In this embodiment, firstly, the image acquisition module is used to obtain a display screen monitoring image set by using a monitoring video stream, so that the position of the display screen in the monitoring video stream can be automatically located, and an image set of an area of ​​interest, that is, a display screen monitoring image set containing the display screen, is extracted; then, the image transformation module is used to perform perspective transformation on the image set, so that it can adapt to the display screen monitoring image set under different viewing angles in the operation and maintenance monitoring video, thereby realizing display fault detection under different viewing angles; then, the feature extraction module is used to perform feature extraction, so that the key features in the display screen correction image set can be accurately captured, and a display screen feature image set can be obtained, so that the fault classification module can be used to automatically identify and multi-classify each display screen feature image in the display screen feature image set by using a fault detection model, so as to obtain a fault type corresponding to each display screen feature image one by one; finally, the fault determination module is used to determine that a display screen fault occurs and its corresponding target fault type is determined only when the fault types of all display screen feature images meet preset conditions.

[0134] The display screen fault detection system of this embodiment can realize real-time intelligent detection of display screen faults with high accuracy, and can detect multiple types of display screen faults at the same time, with fully automatic intelligent detection and alarm, without the need for manual inspection by on-duty personnel, thus improving operation and maintenance efficiency.

[0135] The functions of the display screen fault detection system described in this embodiment correspond to the steps of the display screen fault detection method in Example 1. For details not included in this embodiment, please refer to Example 1 and Figures 1 to 6 The detailed description will not be repeated here.

[0136] Embodiment 3

[0137] A display screen fault detection device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps in the display screen fault detection method of embodiment 1 when executed.

[0138] By storing the computer program in the memory and running it on the processor, it can realize real-time intelligent detection of display screen faults with high accuracy, and can detect multiple types of display screen faults at the same time, with fully automatic intelligent detection and alarm, without the need for manual inspection by on-duty personnel, thus improving operation and maintenance efficiency.

[0139] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of a computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0140] The memory can be used to store computer programs and / or models. The processor realizes various functions of the computer device by running or executing the computer programs and / or models stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.). In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0141] It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by a computer program. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer programs may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0144] This embodiment further provides a computer storage medium, which includes: at least one instruction, and when the instruction is executed, the method steps in the display screen fault detection method of the first embodiment are implemented.

[0145] By executing a computer storage medium containing at least one instruction, real-time intelligent detection of display screen faults can be achieved with high accuracy, and multiple types of display screen faults can be detected simultaneously. Fully automatic intelligent detection and alarm are provided, and manual inspection by on-duty personnel is not required, thereby improving operation and maintenance efficiency.

[0146] Similarly, for details not yet included in the third embodiment, please refer to the first embodiment, the second embodiment and the Figures 1 to 7 The detailed description will not be repeated here.

[0147] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A display screen fault detection method for rail transit, characterized in that: The method comprises: Obtaining a monitoring video stream of the display screen, and obtaining a display screen monitoring image set based on the monitoring video stream; Performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set; Performing feature extraction on the display screen correction image set to obtain a display screen feature image set; Using a pre-trained fault detection model, the display screen feature image set is detected to obtain a fault type corresponding to each display screen feature image in the display screen feature image set; When the fault types of all the display screen characteristic images meet a preset condition, it is determined that the display screen has a fault, and a target fault type corresponding to the display screen is output.

2. The method according to claim 1, characterized in that: The step of obtaining a display screen monitoring image set according to the monitoring video stream includes: Sampling the surveillance video stream according to a preset sampling frequency to obtain a sampled image set; The pre-trained target detection model is used to perform target detection on the sampled image set to obtain the display screen monitoring image set.

3. The method according to claim 2, characterized in that The target detection model includes any one of a Faster R-CNN model, an R-FCN model and an SSD model.

4. The method according to claim 1, characterized in that: The step of performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set includes: Selecting any monitoring image from the display screen monitoring image set, performing grayscale processing and Gaussian blur filtering processing on the selected monitoring image in sequence, and obtaining a display screen processing image corresponding to the selected monitoring image; Perform edge detection on the selected display screen processed image to obtain a corresponding display screen contour image; According to the display screen outline image, determining the coordinates of four vertices in the display screen outline image; Acquire a target image corresponding to the display screen outline image and target coordinates of the four vertex coordinates in the target image; Calculating a perspective transformation matrix according to the four vertex coordinates and the four target coordinates; Performing perspective transformation on the selected monitoring image according to the perspective transformation matrix to obtain a display screen correction image corresponding to the selected monitoring image; Traversing each monitoring image in the display screen monitoring image set, and performing perspective transformation on each monitoring image in the same way to obtain the display screen correction image corresponding to each monitoring image one by one; The display screen correction image set is obtained according to all the display screen correction images.

5. The method according to claim 1, characterized in that The step of extracting features from the display screen correction image set to obtain a display screen feature image set includes: Selecting any one display screen correction image in the display screen correction image set, and sequentially upsampling and normalizing the selected display screen correction image to obtain a normalized image corresponding to the selected display screen correction image; Segmenting the selected normalized image to obtain a plurality of display screen sub-images corresponding to the selected normalized image; Performing feature extraction on each display screen sub-image in the selected normalized image respectively, and obtaining a segmentation feature image corresponding to each display screen sub-image in the selected normalized image; Performing feature fusion on all segmented feature images in the selected normalized image to obtain a display screen feature image corresponding to the selected normalized image; Traversing each display screen correction image in the display screen correction image set, and obtaining a display screen characteristic image corresponding to each display screen correction image in the same manner; The display screen characteristic image set is obtained according to the display screen characteristic images of all display screen correction images.

6. The method according to claim 1, characterized in that The method of using the pre-trained fault detection model to detect the display screen feature image set to obtain a fault type corresponding to each display screen feature image in the display screen feature image set includes: Inputting the display screen feature image set into the fault detection model to obtain the proportion of each display screen feature image in the display screen feature image under each potential fault type; Select any one display screen characteristic image, and determine the potential fault type whose proportion is greater than a first preset threshold as the fault type corresponding to the selected display screen characteristic image; According to the same method, the fault type corresponding to each display screen characteristic image in the display screen characteristic image set is determined.

7. The method according to claim 6, characterized in that When the fault types of all the display screen characteristic images meet a preset condition, determining that the display screen has a fault, and outputting a target fault type corresponding to the display screen, includes: Counting the cumulative amount of all the characteristic images of the display screen under each of the fault types; When the accumulated amount is greater than a second preset threshold, it is determined that the display screen has a fault and a fault alarm signal is issued; the fault type corresponding to the accumulated amount being greater than the second preset threshold is determined as the target fault type of the display screen and output.

8. A display screen fault detection system for rail transit, characterized in that: The method for detecting display screen faults as claimed in any one of claims 1 to 7 comprises: An image acquisition module is used to acquire a monitoring video stream of a display screen, and obtain a display screen monitoring image set according to the monitoring video stream; An image transformation module, used for performing perspective transformation on the display screen monitoring image set to obtain a display screen correction image set; A feature extraction module, used to extract features from the display screen correction image set to obtain a display screen feature image set; A fault classification module is used to detect the display screen feature image set using a pre-trained fault detection model to obtain a fault type corresponding to each display screen feature image in the display screen feature image set; The fault determination module is used to determine that the display screen has a fault when the fault types of all the characteristic images of the display screen meet a preset condition, and output a target fault type corresponding to the display screen.

9. A display screen fault detection device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps according to any one of claims 1 to 7 when executed.

10. A computer storage medium, characterized in that: The computer storage medium comprises: at least one instruction, which implements the method steps according to any one of claims 1 to 7 when the instruction is executed by a computer.

Images