A display screen real-time detection method and system based on multi-path video monitoring

Abstract

The application relates to a display screen real-time detection method and system based on multi-path video monitoring, which comprises the following steps: capturing multi-path real-time monitoring video streams through a video stream acquisition module and transmitting the multi-path real-time monitoring video streams to a computer end; acquiring key frames of each path of video streams according to fixed time intervals through a key frame acquisition module to obtain frame images to be detected; acquiring display images of multiple screens in the frame images to be detected through a screen image extraction module; detecting the quality of the display images through a quality detection module to determine whether the display images appear in pure color display, screen flower and other damage conditions; outputting the detection results, monitoring videos and screen display images to an interface display module and a record storage module by using a Qt development platform to realize sound and light alarm, visual display and local storage functions. The application can realize real-time monitoring of multiple display screens in a scene, accurately and efficiently detect the abnormalities of the screens and intuitively display the results, so that the normal use of the display screens is ensured.

Description

Technical Field

[0001] This invention belongs to the field of video detection technology, specifically relating to a real-time detection method and system for displays based on multi-channel video surveillance. Background Technology

[0002] With the development of technology, displays made of different materials and technologies are increasingly widely used in advertising, live performances, sporting events, e-commerce live streaming, and large-scale conferences, becoming important information dissemination tools. However, during use, due to human or non-human factors such as power fluctuations and ambient temperature, displays may exhibit abnormalities such as solid color display or screen distortion. These abnormalities not only affect display quality and user experience but also cause economic losses. Therefore, it is necessary to conduct quality inspections on displays in use to promptly identify abnormalities, thereby ensuring effective information dissemination, improving user experience, and ensuring the safety and stability of venue operations.

[0003] Simultaneous manual monitoring of multiple displays is limited by time and space constraints, prone to oversights, and inefficient. When there are many displays in a venue, more management personnel are needed for monitoring, increasing human resource costs. Therefore, intelligent technology can enable managers to accurately and efficiently monitor all displays in the venue. The detection system can quickly identify problems with the displays, issuing alarms to prompt managers to take appropriate measures, thereby reducing the impact and losses caused by malfunctions. Simultaneously, the detection system can collect data on various anomalies, which managers can analyze and process, adjusting monitoring strategies in a timely manner, improving display maintenance efficiency and cost-effectiveness, and ultimately enhancing management overall.

[0004] In recent years, many documents have studied methods for detecting screen quality. For example, Chinese patent document CN115187534A discloses "A method, device, equipment and medium for detecting robot screens," which proposes a method for detecting the quality of robot display screens. However, this method does not address how to detect multiple screens simultaneously, nor does it provide a clear and intuitive display of the screen content and detection results, thus having certain limitations. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention proposes a real-time detection method and system for display screens based on multi-channel video surveillance, which enables real-time detection of all display screens in a venue and visualizes the detection results for easy management by administrators, thereby ensuring the normal use of the display screens.

[0006] Terminology Explanation:

[0007] Qt is a cross-platform C++ application development framework that provides a rich set of graphical interfaces and function libraries for developing GUI applications. A Qt-based visual system refers to an application or system with a graphical interface developed using the Qt framework. It provides a series of visual controls and layout managers, enabling the easy implementation of various complex interface effects. Furthermore, Qt offers a powerful signal and slot mechanism and multi-threading support, allowing developers to efficiently create feature-rich and fast-responding visual systems. This invention proposes a Qt-based visual system that can detect display anomalies in real time and visualize the detection results. It can also save anomaly logs and other information locally for easy management by administrators.

[0008] The technical solution adopted in this invention is as follows:

[0009] A real-time detection method for a display screen based on multi-channel video surveillance includes:

[0010] Acquire multiple video streams;

[0011] Acquire the frame image to be tested from each video stream;

[0012] Extract the screen image to be detected from the display images of multiple screens in the frame image to be tested;

[0013] Perform anomaly detection on each screen image to be inspected to determine if a fault has occurred;

[0014] The process involves anomaly detection for each screen image to be tested to determine whether a fault has occurred. This includes grayscale processing, filtering, and calculation of the mean and variance. If the threshold is less than the variance, it is determined that no fault has occurred; otherwise, a fault has occurred.

[0015] According to a preferred embodiment of the present invention, the real-time detection method for a display screen based on multi-channel video surveillance further includes:

[0016] The interface display specifically refers to displaying the real-time video stream of the meeting venue captured by the camera, the extracted screen image to be detected, and the anomaly detection results on the Qt interface.

[0017] According to a preferred embodiment of the present invention, the real-time detection method for a display screen based on multi-channel video surveillance further includes:

[0018] Recording and storage include: recording screen malfunctions and saving the captured video streams locally, which are then automatically deleted periodically.

[0019] According to a preferred embodiment of the present invention, anomaly detection is performed on each screen image to be detected to determine whether a fault has occurred, specifically including:

[0020] 1) For each screen image Y to be detected lk Perform grayscale processing to obtain the k-th grayscale screen image Z to be detected. lk , l=1,2,...,S, k=1,2,...,N l N l For M l Total number of displays in the system;

[0021] 2) Use the Prewitt operator on the Z-image of the test image. lk Filtering is performed to obtain the corresponding filtered image Z. lk The calculation formula is shown in equation (I):

[0022]

[0023] In equation (I), * represents the convolution operation, abs() represents the operation of taking the absolute value of all elements in the matrix, and the symbol [] is used to represent a two-dimensional matrix;

[0024] 3) Calculate the Z-axis of the filtered image. lk The mean of ' is calculated using the formula shown in equation (II):

[0025]

[0026] In formula (II), For image Z lk The mean of ', row(Z) lk ') and col(Z lk ') represent the number of rows and columns of the image, respectively, Z lk '(i,j) represents the image Z lk The gray value at the midpoint (i,j);

[0027] 4) Calculate the Z-axis of the filtered image. lk The variance of ' is calculated using the formula shown in equation (III):

[0028]

[0029] In formula (III), For image Z lk The variance of ';

[0030] 5) Set the threshold Th var ,like Then it is determined that the screen represented by the image does not exhibit any abnormalities; if Then it is determined that there is an abnormal situation on the screen.

[0031] Further preferred, the threshold Th varThe value ranges from 50 to 100.

[0032] According to a preferred embodiment of the present invention, acquiring multiple video streams includes:

[0033] 6) Based on the location of each display screen, install surveillance cameras at S fixed positions to collect video signals from that location;

[0034] 7) Connect multiple surveillance cameras to the input of the video matrix via network cables;

[0035] 8) Connect the output of the video matrix to the input of the computer using a video capture card to complete the acquisition of multiple video streams;

[0036] 9) Display multiple video streams;

[0037] 10) Save multiple video streams.

[0038] According to a preferred embodiment of the present invention, acquiring the frame image to be tested from each video stream includes:

[0039] For the l-th video stream, keyframes are captured at fixed time intervals to obtain the image M of the frame to be detected captured in the l-th video stream. l l = 1, 2, ..., S; keyframe interval t gap The calculation formula is shown in equation (IV):

[0040]

[0041] In equation (IV), f viedo The frame rate of the current video stream, expressed in frames per second.

[0042] According to a preferred embodiment of the present invention, extracting the screen image to be detected from the display images of multiple screens in the frame image to be tested includes:

[0043] 11) Based on each display screen in the frame image M to be detected l At a fixed position in the image, the image X will be displayed on the k-th screen. lk From M respectively l Extract from the middle, k = 1, 2, ..., N l N l For M l Total number of displays in the system;

[0044] 12) Use perspective transformation to display image X on the screen. lk Perform correction and obtain the image X displayed on the screen. lk The corrected k-th screen image to be detected, Y lk ,include:

[0045] Let the screen display image Xlk In the frame image M to be detected l The coordinates of the four vertices in the middle, clockwise from the top left corner, are (m1, n1), (m2, n2), (m3, n3), and (m4, n4). The perspective transformation matrix H is shown in equation (V):

[0046]

[0047] In equation (V), a, b, and c represent the scaling, shearing, and translation of the transformed image along the x-axis, respectively; d, e, and f represent the scaling, shearing, and translation of the transformed image along the y-axis, respectively; g represents the distortion of the transformed image along the depth direction; and h represents the perspective transformation of the transformed image along the depth direction.

[0048] Corrected screen image Y to be detected lk The coordinates of the four vertices, from the top left corner clockwise, are (0,0), (d...). x ,0)(d x ,d y ), (0,d y ), where d x With d y The calculation formulas are shown in equations (VI) and (VII):

[0049] d x =max{m1,m2,m3,m4}-min{m1,m2,m3,m4}(VI)

[0050] d y =max{n1,n2,n3,n4}-min{n1,n2,n3,n4}(VII)

[0051] The specific parameters of matrix H are determined as shown in equation (VIII):

[0052]

[0053] Solving the above linear equations determines the specific values ​​of a, b, c, d, e, f, g, and h in matrix H; let (u, v) be X. lk The coordinates of a certain pixel are used to calculate a new coordinate (x, y, w) using equation (IX), as shown in equation (IX):

[0054]

[0055] Normalize its coordinates to ensure the image is scaled correctly, as shown in equation (X):

[0056]

[0057] In equation (X), (x', y') is X after perspective transformation. lk A certain pixel in Y lk Coordinates in;

[0058] For X lk Perform the coordinate transformation above on all pixels to obtain the screen image Y to be detected. lk , l=1,2,...,S, k=1,2,...,N l N l For M l The total number of displays in the system.

[0059] According to a preferred embodiment of the present invention, acquiring multiple video streams and acquiring the frame image to be tested from each video stream includes: using the multi-threading mechanism of the visualization system developed in Qt to create S video stream acquisition sub-threads GetVideo1, GetVideo2, ..., GetVideoS, which are used to realize the real-time acquisition, parameter setting and local saving of the S video streams; and sending these S video streams to the main interface for playback, while simultaneously realizing the acquisition of key frames of the frame image to be tested in each video stream;

[0060] Extracting the screen image to be detected from the display images of multiple screens in the frame image to be tested includes: creating S sub-threads GetScreenImg1, GetScreenImg2, ..., GetScreenImgS to receive the frame image to be detected sent by the video stream acquisition sub-thread, extracting the screen image, correcting it through perspective transformation, obtaining the screen image to be detected, and sending the screen image to be detected to the main interface for real-time display.

[0061] For each screen image to be inspected, anomaly detection is performed to determine whether a fault has occurred. This includes creating S quality inspection sub-threads DetectVideo1, DetectVideo2, ..., DetectVideoS to perform quality inspection on the screen images to be inspected extracted from the S video streams, determine whether the screen has any abnormalities, and send the detection results to the main interface for visualization.

[0062] A further preferred embodiment employs the AbnormalWrite sub-thread for storing abnormal records, which receives abnormal detection results sent by all quality detection sub-threads and simultaneously writes the time, location, device, display screen, and current administrator information of the abnormality into the SQL database.

[0063] The VideoAutoDelete sub-thread for deleting local records is used to view the creation dates of Excel files and video files saved in the local folder, compare them with the current date, and automatically delete the files if they have been saved for more than 30 days.

[0064] A real-time display screen detection system based on multi-channel video surveillance includes: a video stream acquisition module, a key frame acquisition module, a screen image extraction module, a quality detection module, an interface display module, and a recording and storage module;

[0065] The video stream acquisition module is used to acquire multiple video streams.

[0066] The keyframe acquisition module is used to acquire the frame image to be tested from each video stream;

[0067] The screen image extraction module is used to extract the screen image to be detected from the display images of multiple screens in the frame image to be tested;

[0068] The quality inspection module is used to perform anomaly detection on each screen image to be inspected and to determine whether a fault has occurred.

[0069] The interface display module is used to display the real-time monitoring images of the meeting venue captured by the camera, the screen display images, and the anomaly detection results on the Qt interface.

[0070] The recording and storage module is used to record screen malfunctions and save the captured video stream locally, which is then automatically deleted periodically.

[0071] The beneficial effects of this invention are as follows:

[0072] 1. This invention can acquire multiple video streams in real time and perform quality detection on all displays in the video streams to determine whether any abnormalities have occurred.

[0073] 2. This invention can display the detection results intuitively and provide audible and visual alarms to remind managers to handle the situation promptly;

[0074] 3. This invention can record acquired video streams, abnormal display conditions, etc. locally and automatically delete them periodically, thereby improving management efficiency. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the real-time display screen detection system based on multi-channel video surveillance according to the present invention;

[0076] Figure 2 This is a schematic diagram of the display interface for the screen quality detection results of the present invention;

[0077] Figure 3 This is a schematic diagram of the interface for recording exceptions according to the present invention;

[0078] Figure 4 This is a schematic diagram of an example of a frame image to be detected in a certain video stream;

[0079] Figure 5 This is a schematic diagram of Example 2 of the frame images to be detected in a certain video stream;

[0080] Figure 6 This is a schematic diagram of Example 3 of the frame images to be detected in a certain video stream;

[0081] Figure 7 From Figure 4 A schematic diagram of the screen image to be detected extracted from the image.

[0082] Figure 8 From Figure 4 A schematic diagram of the second screen image to be detected extracted from the image;

[0083] Figure 9 From Figure 5 A schematic diagram of the screen image to be detected extracted from the image.

[0084] Figure 10 From Figure 5 A schematic diagram of the second screen image to be detected extracted from the image;

[0085] Figure 11 From Figure 6 A schematic diagram of the screen image to be detected extracted from the image.

[0086] Figure 12 From Figure 6 A schematic diagram of the second screen image to be detected extracted from the image. Detailed Implementation

[0087] The present invention will be further described below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0088] Example 1

[0089] A real-time detection method for a display screen based on multi-channel video surveillance includes:

[0090] Acquire multiple video streams;

[0091] Acquire the frame image to be tested from each video stream;

[0092] Extract the screen image to be detected from the display images of multiple screens in the frame image to be tested;

[0093] Perform anomaly detection on each screen image to be inspected to determine if a fault has occurred;

[0094] The process involves anomaly detection for each screen image to be tested to determine whether a fault has occurred. This includes grayscale processing, filtering, and calculation of the mean and variance. If the threshold is less than the variance, it is determined that no fault has occurred; otherwise, a fault has occurred.

[0095] Example 2

[0096] The real-time detection method for a display screen based on multi-channel video surveillance described in Embodiment 1 differs in that:

[0097] The real-time detection method for displays based on multi-channel video surveillance also includes:

[0098] The interface display specifically refers to displaying the real-time video stream of the meeting venue captured by the camera, the extracted screen image to be detected, and the anomaly detection results on the Qt interface.

[0099] Recording and storage include: recording screen malfunctions and saving the captured video streams locally, which are then automatically deleted periodically.

[0100] Example 3

[0101] The real-time detection method for a display screen based on multi-channel video surveillance described in Embodiment 1 differs in that:

[0102] For each screen image to be inspected, anomaly detection is performed to determine whether a fault has occurred. Specifically, this includes:

[0103] 1) For each screen image Y to be detected lk Perform grayscale processing to obtain the k-th grayscale screen image Z to be detected. lk , l=1,2,...,S, k=1,2,...,N l N l For M l Total number of displays in the system;

[0104] 2) Use the Prewitt operator on the Z-image of the test image. lk Filtering is performed to obtain the corresponding filtered image Z. lk The calculation formula is shown in equation (I):

[0105]

[0106] In equation (I), * represents the convolution operation, abs() represents the operation of taking the absolute value of all elements in the matrix, and the symbol [] is used to represent a two-dimensional matrix;

[0107] 3) Calculate the Z-axis of the filtered image. lk The mean of ' is calculated using the formula shown in equation (II):

[0108]

[0109] In formula (II), For image Z lk The mean of ', row(Z) lk ') and col(Z lk ') represent the number of rows and columns of the image, respectively, Z lk '(i,j) represents the image Z lk The gray value at the midpoint (i,j);

[0110] 4) Calculate the Z-axis of the filtered image. lk The variance of ' is calculated using the formula shown in equation (III):

[0111]

[0112] In formula (III), For image Z lk The variance of ';

[0113] 5) Set the threshold Th var ,like Then it is determined that the screen represented by the image does not exhibit any abnormalities; if This indicates an abnormal situation has occurred on the screen. Threshold Th var The value ranges from 50 to 100. It is mainly used to distinguish between normal and abnormal images and can be adjusted after testing according to the on-site environment.

[0114] Acquiring multiple video streams; including:

[0115] 6) Based on the location of each display screen, install Hikvision surveillance cameras at S fixed positions to collect video signals from that location;

[0116] 7) Connect multiple surveillance cameras to the input end of the video matrix via network cables; a video matrix is ​​an electronic device that can output m video signals to n monitoring devices arbitrarily through array switching.

[0117] 8) Connect the output of the video matrix to the input of the computer using a video capture card to complete the acquisition of multiple video streams;

[0118] 9) Transmit multiple video streams to the interface display module for display;

[0119] 10) Transmit multiple video streams to the recording and storage module for storage.

[0120] Acquire the frame image to be tested from each video stream, including:

[0121] For the l-th video stream, keyframes are captured at fixed time intervals to obtain the image M of the frame to be detected captured in the l-th video stream. l l = 1, 2, ..., S; keyframe interval t gap The calculation formula is shown in equation (IV):

[0122]

[0123] In equation (IV), f viedo The frame rate of the current video stream, expressed in frames per second.

[0124] Extract the screen image to be detected from the display images of multiple screens in the frame image to be tested, including:

[0125] 11) Based on each display screen in the frame image M to be detected l At a fixed position in the image, the image X will be displayed on the k-th screen. lk From M respectively l Extract from the middle, k = 1, 2, ..., N l N l For M l Total number of displays in the system;

[0126] 12) Directly cropped images will have certain distortions, which will affect the detection results. Therefore, the perspective transformation method is used to modify the screen-displayed image X. lk Perform correction and obtain the image X displayed on the screen. lk The corrected k-th screen image to be detected, Y lk ,include:

[0127] Let the screen display image X lk In the frame image M to be detected l The coordinates of the four vertices in the middle, clockwise from the top left corner, are (m1, n1), (m2, n2), (m3, n3), and (m4, n4). The perspective transformation matrix H is shown in equation (V):

[0128]

[0129] In equation (V), a, b, and c represent the scaling, shearing, and translation of the transformed image along the x-axis, respectively; d, e, and f represent the scaling, shearing, and translation of the transformed image along the y-axis, respectively; g represents the distortion of the transformed image along the depth direction; and h represents the perspective transformation of the transformed image along the depth direction.

[0130] Corrected screen image Y to be detected lk The coordinates of the four vertices, from the top left corner clockwise, are (0,0), (d...). x ,0)(d x ,dy ), (0,d y ), where d x With d y The calculation formulas are shown in equations (VI) and (VII):

[0131] d x =max{m1,m2,m3,m4}-min{m1,m2,m3,m4}(VI)

[0132] d y =max{n1,n2,n3,n4}-min{n1,n2,n3,n4}(VII)

[0133] The specific parameters of matrix H are determined as shown in equation (VIII):

[0134]

[0135] Solving the above linear equations determines the specific values ​​of a, b, c, d, e, f, g, and h in matrix H; let (u, v) be X. lk The coordinates of a certain pixel are used to calculate a new coordinate (x, y, w) using equation (IX), as shown in equation (IX):

[0136]

[0137] Due to the original image X lk Since the image is two-dimensional and has no z-axis dimension, it is set to 1. However, the transformed image is three-dimensional, so its coordinates are normalized to ensure the correct scale of the image, as shown in equation (X):

[0138]

[0139] In equation (X), (x', y') is X after perspective transformation. lk A certain pixel in Y lk Coordinates in;

[0140] For X lk Perform the coordinate transformation above on all pixels to obtain the screen image Y to be detected. lk , l=1,2,...,S, k=1,2,...,N l N l For M l The total number of displays in the system.

[0141] The system acquires multiple video streams and extracts test frame images from each stream. This includes using Qt to develop a multi-threaded mechanism within the visualization system, creating S video stream acquisition sub-threads (GetVideo1, GetVideo2, ..., GetVideoS) to achieve real-time acquisition, parameter setting, and local saving of the S video streams. These S video streams are then sent to the main interface for playback, while simultaneously capturing keyframes of the test frame images from each video stream.

[0142] Extracting the screen image to be detected from the display images of multiple screens in the frame image to be tested includes: creating S sub-threads GetScreenImg1, GetScreenImg2, ..., GetScreenImgS to receive the frame image to be detected sent by the video stream acquisition sub-thread, extracting the screen image, correcting it through perspective transformation, obtaining the screen image to be detected, and sending the screen image to be detected to the main interface for real-time display.

[0143] For each screen image to be inspected, anomaly detection is performed to determine whether a fault has occurred. This includes creating S quality inspection sub-threads DetectVideo1, DetectVideo2, ..., DetectVideoS to perform quality inspection on the screen images to be inspected extracted from the S video streams, determine whether the screen has any abnormalities, and send the detection results to the main interface for visualization.

[0144] An abnormal record storage sub-thread, AbnormalWrite, is used to receive abnormal detection results sent by all quality inspection sub-threads, and simultaneously write the time, location, device, display screen, and current administrator of the abnormality into the SQL database.

[0145] The VideoAutoDelete sub-thread for deleting local records is used to view the creation dates of Excel files and video files saved in the local folder, compare them with the current date, and automatically delete the files if they have been saved for more than 30 days.

[0146] The main interface is designed using Qt Creator. It displays multiple video streams and their corresponding screen images in real time, converting the detection results of each screen into text colors for display. If the detection result is normal, the corresponding text on the screen is displayed in green; if the detection result is abnormal, the text is displayed in red, and the system also emits an alarm sound. Multiple "Start" and "Stop" buttons are added to control the detection of each video stream, and a "Sound On / Off" button is added to control the start and stop of the alarm sound. When there are many monitors in the venue, multiple detection interfaces can be added to detect the monitors. Screen quality detection results are shown below. Figure 2 As shown.

[0147] Design an "Exception Log" interface to display data from a SQL database in form format, and add function buttons to allow manipulation of the data in the form; add an "Export Local" button to export the form as an Excel file to your local computer; add a "View Local Files" button to pop up the folder where Excel and video files are stored. The exception log interface is as follows. Figure 3 As shown.

[0148] Record storage includes:

[0149] Use the VideoWriter() function provided by OpenCV to save each video stream in a local folder, named after the shooting location and the current time.

[0150] Use the insert function to write each detected anomaly record into the SQL database;

[0151] In the record storage module, the QXlsx::Document function provided by the QtXlsxWriter library is used to create an Excel file in a local folder, with the current date and time as the file name;

[0152] Use the QXlsx::write function provided by the QtXlsxWriter library to create four list items in an Excel spreadsheet: "Time", "Location", "Faulty Equipment", "Faulty Screen", and "Manager".

[0153] Each detection record in the SQL database is inserted into an Excel spreadsheet and saved using the QXlsx::saveAs function provided by the QtXlsxWriter library.

[0154] Figure 4 This is a schematic diagram of an example of a frame image to be detected in a certain video stream; Figure 5 This is a schematic diagram of Example 2 of the frame images to be detected in a certain video stream; Figure 6 This is a schematic diagram of Example 3 of the frame images to be detected in a certain video stream; Figure 7 From Figure 4 A schematic diagram of the screen image to be detected extracted from the image. Figure 8 From Figure 4 A schematic diagram of the second screen image to be detected extracted from the image; Figure 9 From Figure 5 A schematic diagram of the screen image to be detected extracted from the image. Figure 10 From Figure 5 A schematic diagram of the second screen image to be detected extracted from the image; Figure 11 From Figure 6A schematic diagram of the screen image to be detected extracted from the image. Figure 12 From Figure 6 A schematic diagram of the second screen image to be detected extracted from the image. The calculated variance results are shown in Table 1:

[0155] Table 1

[0156] Figure 7 normal 173.31 Figure 8 abnormal 5.07 Figure 9 normal 337.76 Figure 10 abnormal 8.68 Figure 11 abnormal 18.58 Figure 12 normal 186.61

[0157] Based on this sample, the variance calculated for abnormal images is significantly smaller than that for normal images. If the threshold is set to 50, normal images can be distinguished from abnormal images.

[0158] Example 4

[0159] A real-time display screen detection system based on multi-channel video surveillance, such as Figure 1 As shown, it includes: a video stream acquisition module, a keyframe acquisition module, a screen image extraction module, a quality detection module, an interface display module, and a recording and storage module;

[0160] The video stream acquisition module is used to acquire multiple video streams.

[0161] The keyframe acquisition module is used to acquire the frame image to be tested from each video stream;

[0162] The screen image extraction module is used to extract the screen image to be detected from the display images of multiple screens in the frame image to be tested;

[0163] The quality inspection module is used to perform anomaly detection on each screen image to be inspected and to determine whether a fault has occurred.

[0164] The interface display module is used to display the real-time monitoring images of the meeting venue captured by the camera, the screen display images, and the anomaly detection results on the Qt interface.

[0165] The recording and storage module is used to record screen malfunctions and save the captured video stream locally, which is then automatically deleted periodically.

Claims

1. A real-time detection method for a display screen based on multi-channel video surveillance, characterized in that, include: Acquire multiple video streams; Acquire the frame image to be tested from each video stream; Extract the screen image to be detected from the display images of multiple screens in the frame image to be tested; Perform anomaly detection on each screen image to be inspected to determine if a fault has occurred. The process involves anomaly detection for each screen image to be detected to determine whether a fault has occurred. This includes grayscale processing, filtering, and calculation of the mean and variance. If the threshold is less than the variance, it is determined that no fault has occurred; otherwise, a fault has occurred. For each screen image to be inspected, anomaly detection is performed to determine whether a fault has occurred. Specifically, this includes: 1) For each screen image to be detected Perform grayscale processing to obtain the first... k A grayscale screen image to be detected , , , for Total number of displays in the system; 2) Use the Prewitt operator on the image to be tested. Filtering is performed to obtain the corresponding filtered image. The calculation formula is shown in equation (I): （I） In equation (I), * denotes the convolution operation. abs () represents the operation of taking the absolute value of all elements in a matrix, and the symbol [ ] is used to represent a two-dimensional matrix; 3) Calculate the filtered image The mean value is calculated using the formula shown in equation (II): (II) In formula (II), For image The mean, and These represent the number of rows and columns of the image, respectively. Representing an image midpoint The grayscale value at that location; 4) Calculate the filtered image The variance is calculated using the formula shown in equation (III): (III) In formula (III), For image The variance; 5) Set threshold ,like If the image represents a screen without any abnormalities, then it is determined that the screen does not exhibit any abnormalities; if If so, it is determined that the screen has encountered an abnormal situation; threshold The value ranges from 50 to 100; Extract the screen image to be detected from the display images of multiple screens in the frame image to be tested, including: 11) Based on the image of the frame to be detected for each display screen The fixed position in the middle, will the first k Each screen displays an image. From respectively Extracted from the middle, , for Total number of displays in the system; 12) Using perspective transformation to display images on the screen Perform corrections and acquire the image displayed on the screen. The first after correction k A screen image to be detected ,include: Set the screen to display an image In the frame image to be detected The coordinates of the four vertices are as follows, starting from the top left corner and proceeding clockwise: , , , The perspective transformation matrix H is shown in equation (V): （V） In equation (V), , , These represent the scaling, shearing, and translation degrees of the transformed image along the x-axis, respectively. , , These represent the scaling, shearing, and translation degrees of the transformed image along the y-axis, respectively. This indicates the degree of distortion in the depth direction of the transformed image; This indicates the degree of perspective transformation in the depth direction of the transformed image; Corrected screen image to be tested The coordinates of the four vertices are as follows, clockwise from the top left: , , , ,in, and The calculation formulas are shown in equations (VI) and (VII): The specific parameters of matrix H are determined as shown in equation (VIII): (VIII) Solving the above linear equations, i.e., determining the matrix H... , , , , , , , Specific numerical values; let for The coordinates of a certain pixel are used to calculate a new coordinate system using equation (IX). As shown in equation (IX): (IX) Normalize its coordinates to ensure the image is scaled correctly, as shown in equation (X): （X） In formula (X), That is, after perspective transformation A certain pixel in Coordinates in; right By performing the coordinate transformation above on all pixels, the screen image to be detected can be obtained. , , , for The total number of displays in the system.

2. The real-time detection method for a display screen based on multi-channel video surveillance according to claim 1, characterized in that, The real-time detection method for displays based on multi-channel video surveillance also includes: The interface display specifically refers to displaying the real-time video stream of the meeting venue captured by the camera, the extracted screen image to be detected, and the anomaly detection results on the Qt interface.

3. The real-time detection method for a display screen based on multi-channel video surveillance according to claim 1, characterized in that, The real-time detection method for displays based on multi-channel video surveillance also includes: Recording and storage include: recording screen malfunctions and saving the captured video streams locally, which are then automatically deleted periodically.

4. The real-time detection method for a display screen based on multi-channel video surveillance according to claim 1, characterized in that, Acquiring multiple video streams; including: 6) Based on the location of each display screen, in S A surveillance camera is installed at a fixed location to collect video signals from that location. 7) Connect multiple surveillance cameras to the input end of the video matrix via network cables; 8) Connect the output of the video matrix to the input of the computer using a video capture card to complete the acquisition of multiple video streams; 9) Display multiple video streams; 10) Save the multiple video streams.

5. The real-time detection method for a display screen based on multi-channel video surveillance according to claim 1, characterized in that, Acquire the frame image to be tested from each video stream, including: For the l The video stream is captured by keyframes at fixed time intervals to obtain the first keyframe. l Images of frames to be detected captured from the video stream , Keyframe interval The calculation formula is shown in equation (IV): In equation (IV), The frame rate of the current video stream, expressed in frames per second.

6. A real-time detection method for a display screen based on multi-channel video surveillance according to any one of claims 1-5, characterized in that, Acquire multiple video streams and capture the frame image to be tested from each video stream; this includes: using Qt to develop a multi-threaded mechanism in the visualization system, creating... S The video stream acquisition sub-threads GetVideo1, GetVideo2, ..., GetVideo S Used to achieve S Real-time acquisition, parameter setting, and local saving of video streams; and this S The video stream is sent to the main interface for playback, and key frames of the frame to be detected in each video stream are captured simultaneously. Extracting the screen image to be detected from the display images of multiple screens in the frame image to be tested includes: creating S Each sub-thread extracts the screen image to be detected: GetScreenImg1, GetScreenImg2, ..., GetScreenImg. S It is used to receive the frame image to be detected sent by the video stream acquisition sub-thread, extract the screen image, and correct it through perspective transformation to obtain the screen image to be detected. At the same time, the screen image to be detected is sent to the main interface for real-time display. Perform anomaly detection on each screen image to be inspected to determine if a fault has occurred, including: creating S A quality detection sub-thread DetectVideo1, DetectVideo2, ..., DetectVideo S Used to achieve separate S The screen image to be detected is extracted from the video stream and subjected to quality inspection to determine whether there are any abnormalities on the screen. The detection results are then sent to the main interface for visualization.

7. A real-time detection method for a display screen based on multi-channel video surveillance according to claim 6, characterized in that, An abnormal record storage sub-thread, AbnormalWrite, is used to receive abnormal detection results sent by all quality inspection sub-threads, and simultaneously write the time, location, device, display screen, and current administrator of the abnormality into the SQL database. The VideoAutoDelete sub-thread for deleting local records is used to view the creation dates of Excel files and video files saved in the local folder, compare them with the current date, and automatically delete the file after a certain number of days have passed.

8. A real-time display screen detection system based on multi-channel video surveillance, which employs the real-time display screen detection method based on multi-channel video surveillance as described in claim 1, characterized in that, include: The system includes a video stream acquisition module, a keyframe acquisition module, a screen image extraction module, a quality inspection module, an interface display module, and a recording and storage module. The video stream acquisition module is used to acquire multiple video streams. The keyframe acquisition module is used to acquire the frame image to be tested from each video stream; The screen image extraction module is used to extract the screen image to be detected from the display images of multiple screens in the frame image to be tested; The quality inspection module is used to perform anomaly detection on each screen image to be inspected and to determine whether a fault has occurred. The interface display module is used to display the real-time monitoring images of the meeting room captured by the camera, the screen display images, and the anomaly detection results on the Qt interface; The recording and storage module is used to record screen malfunctions and save the captured video stream locally, which is then automatically deleted periodically.

Images